@ouro.bot/cli 0.1.0-alpha.289 → 0.1.0-alpha.290
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -2
- package/changelog.json +7 -0
- package/dist/arc/packets.js +203 -0
- package/dist/heart/attachments/image-normalize.js +194 -0
- package/dist/heart/attachments/materialize.js +230 -0
- package/dist/heart/attachments/render.js +29 -0
- package/dist/heart/attachments/store.js +103 -0
- package/dist/heart/attachments/types.js +138 -0
- package/dist/heart/core.js +187 -162
- package/dist/heart/tool-description.js +3 -3
- package/dist/heart/tool-friction.js +55 -0
- package/dist/mind/prompt.js +27 -16
- package/dist/repertoire/tool-results.js +29 -0
- package/dist/repertoire/tools-attachments.js +316 -0
- package/dist/repertoire/tools-base.js +2 -0
- package/dist/repertoire/tools-bluebubbles.js +0 -107
- package/dist/repertoire/tools-flow.js +48 -5
- package/dist/repertoire/tools.js +2 -10
- package/dist/senses/attention-queue.js +20 -1
- package/dist/senses/bluebubbles/attachment-download.js +137 -0
- package/dist/senses/bluebubbles/media.js +36 -101
- package/dist/senses/bluebubbles/model.js +9 -11
- package/dist/senses/cli/image-paste.js +79 -3
- package/dist/senses/inner-dialog.js +9 -0
- package/dist/senses/shared-turn.js +3 -7
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -180,12 +180,14 @@ Agents in Ouroboros aren't just responders — they have an autonomous inner lif
|
|
|
180
180
|
|
|
181
181
|
**Habits** are the agent's rhythms. The most fundamental is *heartbeat* — a periodic nudge that brings the agent back to their thinking space with their journal visible, obligations in view, and a sense of how long it's been. But agents can create any rhythm they want: daily reflections, weekly friend check-ins, inbox triage. Each habit fires independently via OS cron, and the agent sees their own instructions (the habit body they wrote) when it fires.
|
|
182
182
|
|
|
183
|
-
**The inner
|
|
183
|
+
**The inner session** is where the agent thinks privately. When a sense session hits meaningful friction, the agent can *ponder* a typed packet so the work survives the current turn without losing the original objective. When a habit fires, it arrives here too. The agent can *journal* their thinking (writing to `journal/`), *surface* thoughts outward to friends, and *rest* when they're done thinking. On an idle heartbeat, `rest(status="HEARTBEAT_OK")` is the clean no-op move.
|
|
184
184
|
|
|
185
185
|
**The diary** (at `diary/`) is the agent's permanent record — things they've learned, conclusions they've reached. The *journal* (at `journal/`) is their desk — working notes, thinking-in-progress, drafts. The diary is the shelf; the journal is the desk. Both are searchable via the `recall` tool.
|
|
186
186
|
|
|
187
187
|
The whole system is designed so the agent *owns* their inner life. They control their breathing rate, write their own habit instructions, choose when to journal, and decide what to shelve in their diary.
|
|
188
188
|
|
|
189
|
+
Attachments are first-class across senses. Every attachment should remain reachable via a stable `attachment:<source>:<id>` handle, and image normalization should produce a VLM-safe variant without hiding the original artifact.
|
|
190
|
+
|
|
189
191
|
## Connecting With Dev Tools
|
|
190
192
|
|
|
191
193
|
Agents can talk to developer tools like Claude Code and Codex through the MCP bridge. This is how you stay present in a human's coding workflow without them needing to switch to `ouro chat`.
|
|
@@ -201,7 +203,7 @@ This registers the MCP server, installs lifecycle hooks (SessionStart, Stop, Pos
|
|
|
201
203
|
|
|
202
204
|
**How it works:** When a developer starts a Claude Code session, the MCP server launches as a subprocess. The dev tool sees your MCP tools (`send_message`, `check_response`, `status`, `search_memory`, `delegate`, etc.) and can invoke them mid-session. The `send_message` tool runs a full agent turn — you get your system prompt, your diary, your tools, everything. It's not a thin proxy; it's you, thinking.
|
|
203
205
|
|
|
204
|
-
**The conversation pattern:** `send_message` sends a message and gets back your synchronous response.
|
|
206
|
+
**The conversation pattern:** `send_message` sends a message and gets back your synchronous response. `ponder` no longer creates a magical outward deferral. Instead, it bookmarks deeper work as a packet while the current sense session keeps moving. If that work later surfaces something back, the dev tool can still use `check_response` to see the returned result.
|
|
205
207
|
|
|
206
208
|
**Lifecycle hooks** give you passive awareness. When a Claude Code session starts, stops, or uses a tool like Bash or Edit, the hook fires `ouro hook <event> --agent <name>` and the daemon notes it. Your inner dialog sees these sessions in its checkpoint, so you know what's happening across your world even when nobody is talking to you directly.
|
|
207
209
|
|
package/changelog.json
CHANGED
|
@@ -1,6 +1,13 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_note": "This changelog is maintained as part of the PR/version-bump workflow. Agent-curated, not auto-generated. Agents read this file directly via read_file to understand what changed between versions.",
|
|
3
3
|
"versions": [
|
|
4
|
+
{
|
|
5
|
+
"version": "0.1.0-alpha.290",
|
|
6
|
+
"changes": [
|
|
7
|
+
"feat(attachments): ship first-class cross-sense attachments with stable `attachment:<source>:<id>` handles, shared materialization, shared image normalization, and generic attachment tools so images and files remain reachable even when the harness also provides derived summaries or normalized variants.",
|
|
8
|
+
"feat(mind): replace `ponder` deferral semantics with typed `ponder packets`, linked `ReturnObligation`s, shared friction escalation, and `rest(status=\"HEARTBEAT_OK\")` so the inner session can preserve repair work without losing the original user objective."
|
|
9
|
+
]
|
|
10
|
+
},
|
|
4
11
|
{
|
|
5
12
|
"version": "0.1.0-alpha.289",
|
|
6
13
|
"changes": [
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
var desc = Object.getOwnPropertyDescriptor(m, k);
|
|
5
|
+
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
|
6
|
+
desc = { enumerable: true, get: function() { return m[k]; } };
|
|
7
|
+
}
|
|
8
|
+
Object.defineProperty(o, k2, desc);
|
|
9
|
+
}) : (function(o, m, k, k2) {
|
|
10
|
+
if (k2 === undefined) k2 = k;
|
|
11
|
+
o[k2] = m[k];
|
|
12
|
+
}));
|
|
13
|
+
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
|
14
|
+
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
|
15
|
+
}) : function(o, v) {
|
|
16
|
+
o["default"] = v;
|
|
17
|
+
});
|
|
18
|
+
var __importStar = (this && this.__importStar) || (function () {
|
|
19
|
+
var ownKeys = function(o) {
|
|
20
|
+
ownKeys = Object.getOwnPropertyNames || function (o) {
|
|
21
|
+
var ar = [];
|
|
22
|
+
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
|
23
|
+
return ar;
|
|
24
|
+
};
|
|
25
|
+
return ownKeys(o);
|
|
26
|
+
};
|
|
27
|
+
return function (mod) {
|
|
28
|
+
if (mod && mod.__esModule) return mod;
|
|
29
|
+
var result = {};
|
|
30
|
+
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
|
31
|
+
__setModuleDefault(result, mod);
|
|
32
|
+
return result;
|
|
33
|
+
};
|
|
34
|
+
})();
|
|
35
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
36
|
+
exports.getPonderPacketArtifactsDir = getPonderPacketArtifactsDir;
|
|
37
|
+
exports.listPonderPackets = listPonderPackets;
|
|
38
|
+
exports.readPonderPacket = readPonderPacket;
|
|
39
|
+
exports.createPonderPacket = createPonderPacket;
|
|
40
|
+
exports.revisePonderPacket = revisePonderPacket;
|
|
41
|
+
exports.advancePonderPacket = advancePonderPacket;
|
|
42
|
+
exports.findHarnessFrictionPacket = findHarnessFrictionPacket;
|
|
43
|
+
const path = __importStar(require("path"));
|
|
44
|
+
const runtime_1 = require("../nerves/runtime");
|
|
45
|
+
const transitions_1 = require("../repertoire/tasks/transitions");
|
|
46
|
+
const json_store_1 = require("./json-store");
|
|
47
|
+
function packetsDir(agentRoot) {
|
|
48
|
+
return path.join(agentRoot, "arc", "packets");
|
|
49
|
+
}
|
|
50
|
+
function getPonderPacketArtifactsDir(agentRoot, packetId) {
|
|
51
|
+
return path.join(agentRoot, "state", "packets", packetId);
|
|
52
|
+
}
|
|
53
|
+
function packetSop(kind) {
|
|
54
|
+
switch (kind) {
|
|
55
|
+
case "harness_friction":
|
|
56
|
+
return "harness_friction_v1";
|
|
57
|
+
case "research":
|
|
58
|
+
return "research_v1";
|
|
59
|
+
case "reflection":
|
|
60
|
+
return "reflection_v1";
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
function isTaskStatus(value) {
|
|
64
|
+
return value === "drafting"
|
|
65
|
+
|| value === "processing"
|
|
66
|
+
|| value === "validating"
|
|
67
|
+
|| value === "collaborating"
|
|
68
|
+
|| value === "paused"
|
|
69
|
+
|| value === "blocked"
|
|
70
|
+
|| value === "done"
|
|
71
|
+
|| value === "cancelled";
|
|
72
|
+
}
|
|
73
|
+
function isPacketKind(value) {
|
|
74
|
+
return value === "harness_friction" || value === "research" || value === "reflection";
|
|
75
|
+
}
|
|
76
|
+
function isPonderPacket(value) {
|
|
77
|
+
if (!value || typeof value !== "object")
|
|
78
|
+
return false;
|
|
79
|
+
const packet = value;
|
|
80
|
+
return typeof packet.id === "string"
|
|
81
|
+
&& isPacketKind(packet.kind)
|
|
82
|
+
&& typeof packet.objective === "string"
|
|
83
|
+
&& typeof packet.summary === "string"
|
|
84
|
+
&& Array.isArray(packet.successCriteria)
|
|
85
|
+
&& isTaskStatus(packet.status)
|
|
86
|
+
&& typeof packet.createdAt === "number"
|
|
87
|
+
&& typeof packet.updatedAt === "number";
|
|
88
|
+
}
|
|
89
|
+
function listPonderPackets(agentRoot) {
|
|
90
|
+
return (0, json_store_1.readJsonDir)(packetsDir(agentRoot))
|
|
91
|
+
.filter(isPonderPacket)
|
|
92
|
+
.sort((left, right) => left.createdAt - right.createdAt);
|
|
93
|
+
}
|
|
94
|
+
function readPonderPacket(agentRoot, packetId) {
|
|
95
|
+
const packet = (0, json_store_1.readJsonFile)(packetsDir(agentRoot), packetId);
|
|
96
|
+
return isPonderPacket(packet) ? packet : null;
|
|
97
|
+
}
|
|
98
|
+
function createPonderPacket(agentRoot, input) {
|
|
99
|
+
const now = Date.now();
|
|
100
|
+
const packet = {
|
|
101
|
+
id: (0, json_store_1.generateTimestampId)("pkt"),
|
|
102
|
+
kind: input.kind,
|
|
103
|
+
sop: packetSop(input.kind),
|
|
104
|
+
status: "drafting",
|
|
105
|
+
objective: input.objective,
|
|
106
|
+
summary: input.summary,
|
|
107
|
+
successCriteria: input.successCriteria,
|
|
108
|
+
...(input.origin ? { origin: input.origin } : {}),
|
|
109
|
+
...(input.relatedObligationId ? { relatedObligationId: input.relatedObligationId } : {}),
|
|
110
|
+
...(input.relatedReturnObligationId ? { relatedReturnObligationId: input.relatedReturnObligationId } : {}),
|
|
111
|
+
...(input.followsPacketId ? { followsPacketId: input.followsPacketId } : {}),
|
|
112
|
+
payload: input.payload,
|
|
113
|
+
createdAt: now,
|
|
114
|
+
updatedAt: now,
|
|
115
|
+
};
|
|
116
|
+
(0, json_store_1.writeJsonFile)(packetsDir(agentRoot), packet.id, packet);
|
|
117
|
+
(0, runtime_1.emitNervesEvent)({
|
|
118
|
+
component: "mind",
|
|
119
|
+
event: "mind.packet_created",
|
|
120
|
+
message: "ponder packet created",
|
|
121
|
+
meta: { packetId: packet.id, kind: packet.kind, status: packet.status },
|
|
122
|
+
});
|
|
123
|
+
return packet;
|
|
124
|
+
}
|
|
125
|
+
function revisePonderPacket(agentRoot, packetId, input) {
|
|
126
|
+
const existing = readPonderPacket(agentRoot, packetId);
|
|
127
|
+
if (!existing) {
|
|
128
|
+
throw new Error(`packet not found: ${packetId}`);
|
|
129
|
+
}
|
|
130
|
+
if (existing.status !== "drafting") {
|
|
131
|
+
throw new Error("packet is no longer drafting; file a follow-up packet instead");
|
|
132
|
+
}
|
|
133
|
+
const revised = {
|
|
134
|
+
...existing,
|
|
135
|
+
kind: input.kind,
|
|
136
|
+
sop: packetSop(input.kind),
|
|
137
|
+
objective: input.objective,
|
|
138
|
+
summary: input.summary,
|
|
139
|
+
successCriteria: input.successCriteria,
|
|
140
|
+
payload: input.payload,
|
|
141
|
+
updatedAt: Date.now(),
|
|
142
|
+
};
|
|
143
|
+
(0, json_store_1.writeJsonFile)(packetsDir(agentRoot), packetId, revised);
|
|
144
|
+
(0, runtime_1.emitNervesEvent)({
|
|
145
|
+
component: "mind",
|
|
146
|
+
event: "mind.packet_revised",
|
|
147
|
+
message: "ponder packet revised",
|
|
148
|
+
meta: { packetId, kind: revised.kind, status: revised.status },
|
|
149
|
+
});
|
|
150
|
+
return revised;
|
|
151
|
+
}
|
|
152
|
+
function advancePonderPacket(agentRoot, packetId, update) {
|
|
153
|
+
const existing = readPonderPacket(agentRoot, packetId);
|
|
154
|
+
if (!existing) {
|
|
155
|
+
throw new Error(`packet not found: ${packetId}`);
|
|
156
|
+
}
|
|
157
|
+
const nextStatus = update.status ?? existing.status;
|
|
158
|
+
if (update.status) {
|
|
159
|
+
const transition = (0, transitions_1.validateTransition)(existing.status, update.status);
|
|
160
|
+
if (!transition.ok) {
|
|
161
|
+
throw new Error(transition.reason);
|
|
162
|
+
}
|
|
163
|
+
}
|
|
164
|
+
const advanced = {
|
|
165
|
+
...existing,
|
|
166
|
+
status: nextStatus,
|
|
167
|
+
...(update.relatedObligationId ? { relatedObligationId: update.relatedObligationId } : {}),
|
|
168
|
+
...(update.relatedReturnObligationId ? { relatedReturnObligationId: update.relatedReturnObligationId } : {}),
|
|
169
|
+
updatedAt: Date.now(),
|
|
170
|
+
};
|
|
171
|
+
(0, json_store_1.writeJsonFile)(packetsDir(agentRoot), packetId, advanced);
|
|
172
|
+
(0, runtime_1.emitNervesEvent)({
|
|
173
|
+
component: "mind",
|
|
174
|
+
event: "mind.packet_advanced",
|
|
175
|
+
message: "ponder packet advanced",
|
|
176
|
+
meta: { packetId, status: advanced.status },
|
|
177
|
+
});
|
|
178
|
+
return advanced;
|
|
179
|
+
}
|
|
180
|
+
function findHarnessFrictionPacket(agentRoot, origin, frictionSignature) {
|
|
181
|
+
const found = listPonderPackets(agentRoot).find((packet) => {
|
|
182
|
+
if (packet.kind !== "harness_friction")
|
|
183
|
+
return false;
|
|
184
|
+
if (!packet.origin)
|
|
185
|
+
return false;
|
|
186
|
+
return packet.origin.friendId === origin.friendId
|
|
187
|
+
&& packet.origin.channel === origin.channel
|
|
188
|
+
&& packet.origin.key === origin.key
|
|
189
|
+
&& packet.payload.frictionSignature === frictionSignature;
|
|
190
|
+
}) ?? null;
|
|
191
|
+
(0, runtime_1.emitNervesEvent)({
|
|
192
|
+
component: "mind",
|
|
193
|
+
event: "mind.packet_lookup",
|
|
194
|
+
message: "ponder packet lookup completed",
|
|
195
|
+
meta: {
|
|
196
|
+
kind: "harness_friction",
|
|
197
|
+
found: found !== null,
|
|
198
|
+
origin: `${origin.friendId}/${origin.channel}/${origin.key}`,
|
|
199
|
+
frictionSignature,
|
|
200
|
+
},
|
|
201
|
+
});
|
|
202
|
+
return found;
|
|
203
|
+
}
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
var desc = Object.getOwnPropertyDescriptor(m, k);
|
|
5
|
+
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
|
6
|
+
desc = { enumerable: true, get: function() { return m[k]; } };
|
|
7
|
+
}
|
|
8
|
+
Object.defineProperty(o, k2, desc);
|
|
9
|
+
}) : (function(o, m, k, k2) {
|
|
10
|
+
if (k2 === undefined) k2 = k;
|
|
11
|
+
o[k2] = m[k];
|
|
12
|
+
}));
|
|
13
|
+
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
|
14
|
+
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
|
15
|
+
}) : function(o, v) {
|
|
16
|
+
o["default"] = v;
|
|
17
|
+
});
|
|
18
|
+
var __importStar = (this && this.__importStar) || (function () {
|
|
19
|
+
var ownKeys = function(o) {
|
|
20
|
+
ownKeys = Object.getOwnPropertyNames || function (o) {
|
|
21
|
+
var ar = [];
|
|
22
|
+
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
|
23
|
+
return ar;
|
|
24
|
+
};
|
|
25
|
+
return ownKeys(o);
|
|
26
|
+
};
|
|
27
|
+
return function (mod) {
|
|
28
|
+
if (mod && mod.__esModule) return mod;
|
|
29
|
+
var result = {};
|
|
30
|
+
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
|
31
|
+
__setModuleDefault(result, mod);
|
|
32
|
+
return result;
|
|
33
|
+
};
|
|
34
|
+
})();
|
|
35
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
36
|
+
exports.MAX_VLM_IMAGE_BYTES = exports.MAX_ATTACHMENT_DOWNLOAD_BYTES_IMAGE = void 0;
|
|
37
|
+
exports.normalizeImageForVision = normalizeImageForVision;
|
|
38
|
+
const node_child_process_1 = require("node:child_process");
|
|
39
|
+
const fs = __importStar(require("node:fs/promises"));
|
|
40
|
+
const path = __importStar(require("node:path"));
|
|
41
|
+
const runtime_1 = require("../../nerves/runtime");
|
|
42
|
+
exports.MAX_ATTACHMENT_DOWNLOAD_BYTES_IMAGE = 32 * 1024 * 1024;
|
|
43
|
+
exports.MAX_VLM_IMAGE_BYTES = 5 * 1024 * 1024;
|
|
44
|
+
const SAFE_VLM_MIME_TYPES = new Set(["image/png", "image/jpeg", "image/webp"]);
|
|
45
|
+
const SIZE_STEPS = [3072, 2560, 2048, 1600, 1280, 1024, 768];
|
|
46
|
+
const JPEG_QUALITIES = ["best", "90", "80", "70", "60"];
|
|
47
|
+
function materializedOutputDir(agentRoot, attachmentId) {
|
|
48
|
+
return path.join(agentRoot, "state", "attachments", "materialized", attachmentId.replace(/[:/]/g, "_"));
|
|
49
|
+
}
|
|
50
|
+
function execFileText(file, args) {
|
|
51
|
+
return new Promise((resolve, reject) => {
|
|
52
|
+
(0, node_child_process_1.execFile)(file, args, (error, stdout = "", stderr = "") => {
|
|
53
|
+
if (error) {
|
|
54
|
+
reject(new Error(stderr.trim() || stdout.trim() || error.message));
|
|
55
|
+
return;
|
|
56
|
+
}
|
|
57
|
+
resolve(stdout);
|
|
58
|
+
});
|
|
59
|
+
});
|
|
60
|
+
}
|
|
61
|
+
async function defaultProbeImage(input) {
|
|
62
|
+
const output = await execFileText("/usr/bin/sips", ["-g", "hasAlpha", input.sourcePath]);
|
|
63
|
+
const hasAlpha = /hasAlpha:\s+(yes|1|true)/i.test(output);
|
|
64
|
+
return { hasAlpha };
|
|
65
|
+
}
|
|
66
|
+
async function defaultRasterizeWithQuickLook(input) {
|
|
67
|
+
await fs.mkdir(input.outputDir, { recursive: true });
|
|
68
|
+
await execFileText("/usr/bin/qlmanage", ["-t", "-s", "4096", "-o", input.outputDir, input.sourcePath]);
|
|
69
|
+
const entries = await fs.readdir(input.outputDir);
|
|
70
|
+
const candidate = entries
|
|
71
|
+
.filter((entry) => entry.startsWith(path.basename(input.sourcePath, path.extname(input.sourcePath))))
|
|
72
|
+
.sort()[0];
|
|
73
|
+
if (!candidate) {
|
|
74
|
+
throw new Error("Quick Look did not produce a rasterized image");
|
|
75
|
+
}
|
|
76
|
+
return path.join(input.outputDir, candidate);
|
|
77
|
+
}
|
|
78
|
+
async function defaultEncodeVariant(input) {
|
|
79
|
+
await fs.mkdir(input.outputDir, { recursive: true });
|
|
80
|
+
const extension = input.targetFormat === "png" ? ".png" : ".jpg";
|
|
81
|
+
const basename = path.basename(input.sourcePath, path.extname(input.sourcePath));
|
|
82
|
+
const outputPath = path.join(input.outputDir, `${basename}-${input.maxEdge}-${input.targetFormat}${input.quality ? `-${input.quality}` : ""}${extension}`);
|
|
83
|
+
const args = [
|
|
84
|
+
"-s",
|
|
85
|
+
"format",
|
|
86
|
+
input.targetFormat,
|
|
87
|
+
...(input.targetFormat === "jpeg" && input.quality ? ["-s", "formatOptions", input.quality] : []),
|
|
88
|
+
"-Z",
|
|
89
|
+
String(input.maxEdge),
|
|
90
|
+
input.sourcePath,
|
|
91
|
+
"--out",
|
|
92
|
+
outputPath,
|
|
93
|
+
];
|
|
94
|
+
await execFileText("/usr/bin/sips", args);
|
|
95
|
+
const stats = await fs.stat(outputPath);
|
|
96
|
+
return {
|
|
97
|
+
path: outputPath,
|
|
98
|
+
mimeType: input.targetFormat === "png" ? "image/png" : "image/jpeg",
|
|
99
|
+
byteCount: stats.size,
|
|
100
|
+
};
|
|
101
|
+
}
|
|
102
|
+
async function normalizeImageForVision(params) {
|
|
103
|
+
const maxVlmBytes = params.maxVlmBytes ?? exports.MAX_VLM_IMAGE_BYTES;
|
|
104
|
+
const originalMimeType = params.attachment.mimeType?.trim().toLowerCase();
|
|
105
|
+
const originalByteCount = params.attachment.byteCount;
|
|
106
|
+
const outputDir = materializedOutputDir(params.agentRoot, params.attachment.id);
|
|
107
|
+
if (originalMimeType
|
|
108
|
+
&& SAFE_VLM_MIME_TYPES.has(originalMimeType)
|
|
109
|
+
&& typeof originalByteCount === "number"
|
|
110
|
+
&& originalByteCount <= maxVlmBytes) {
|
|
111
|
+
(0, runtime_1.emitNervesEvent)({
|
|
112
|
+
component: "engine",
|
|
113
|
+
event: "engine.attachment_normalization_passthrough",
|
|
114
|
+
message: "attachment image normalization updated",
|
|
115
|
+
meta: {
|
|
116
|
+
attachmentId: params.attachment.id,
|
|
117
|
+
mimeType: originalMimeType,
|
|
118
|
+
byteCount: originalByteCount,
|
|
119
|
+
},
|
|
120
|
+
});
|
|
121
|
+
return {
|
|
122
|
+
path: path.resolve(params.sourcePath),
|
|
123
|
+
mimeType: originalMimeType,
|
|
124
|
+
byteCount: originalByteCount,
|
|
125
|
+
};
|
|
126
|
+
}
|
|
127
|
+
const probeImage = params.probeImage ?? defaultProbeImage;
|
|
128
|
+
const rasterizeWithQuickLook = params.rasterizeWithQuickLook ?? defaultRasterizeWithQuickLook;
|
|
129
|
+
const encodeVariant = params.encodeVariant ?? defaultEncodeVariant;
|
|
130
|
+
let workingSourcePath = path.resolve(params.sourcePath);
|
|
131
|
+
let probe;
|
|
132
|
+
try {
|
|
133
|
+
probe = await probeImage({ sourcePath: workingSourcePath });
|
|
134
|
+
}
|
|
135
|
+
catch {
|
|
136
|
+
(0, runtime_1.emitNervesEvent)({
|
|
137
|
+
component: "engine",
|
|
138
|
+
event: "engine.attachment_normalization_rasterize",
|
|
139
|
+
message: "attachment image normalization updated",
|
|
140
|
+
meta: {
|
|
141
|
+
attachmentId: params.attachment.id,
|
|
142
|
+
sourcePath: workingSourcePath,
|
|
143
|
+
},
|
|
144
|
+
});
|
|
145
|
+
workingSourcePath = await rasterizeWithQuickLook({
|
|
146
|
+
sourcePath: workingSourcePath,
|
|
147
|
+
outputDir,
|
|
148
|
+
});
|
|
149
|
+
probe = await probeImage({ sourcePath: workingSourcePath });
|
|
150
|
+
}
|
|
151
|
+
const targetFormats = probe.hasAlpha ? ["png", "jpeg"] : ["jpeg"];
|
|
152
|
+
for (const targetFormat of targetFormats) {
|
|
153
|
+
for (const maxEdge of SIZE_STEPS) {
|
|
154
|
+
const qualities = targetFormat === "jpeg" ? JPEG_QUALITIES : [undefined];
|
|
155
|
+
for (const quality of qualities) {
|
|
156
|
+
const variant = await encodeVariant({
|
|
157
|
+
sourcePath: workingSourcePath,
|
|
158
|
+
outputDir,
|
|
159
|
+
targetFormat,
|
|
160
|
+
maxEdge,
|
|
161
|
+
...(quality ? { quality } : {}),
|
|
162
|
+
});
|
|
163
|
+
if ((variant.byteCount ?? Number.POSITIVE_INFINITY) <= maxVlmBytes) {
|
|
164
|
+
(0, runtime_1.emitNervesEvent)({
|
|
165
|
+
component: "engine",
|
|
166
|
+
event: "engine.attachment_normalization_succeeded",
|
|
167
|
+
message: "attachment image normalization updated",
|
|
168
|
+
meta: {
|
|
169
|
+
attachmentId: params.attachment.id,
|
|
170
|
+
sourceMimeType: originalMimeType ?? "unknown",
|
|
171
|
+
outputMimeType: variant.mimeType ?? `image/${targetFormat}`,
|
|
172
|
+
byteCount: variant.byteCount,
|
|
173
|
+
maxEdge,
|
|
174
|
+
quality: quality ?? null,
|
|
175
|
+
},
|
|
176
|
+
});
|
|
177
|
+
return variant;
|
|
178
|
+
}
|
|
179
|
+
}
|
|
180
|
+
}
|
|
181
|
+
}
|
|
182
|
+
(0, runtime_1.emitNervesEvent)({
|
|
183
|
+
component: "engine",
|
|
184
|
+
event: "engine.attachment_normalization_failed",
|
|
185
|
+
message: "attachment image normalization updated",
|
|
186
|
+
meta: {
|
|
187
|
+
attachmentId: params.attachment.id,
|
|
188
|
+
sourceMimeType: originalMimeType ?? "unknown",
|
|
189
|
+
sourceByteCount: originalByteCount ?? null,
|
|
190
|
+
maxVlmBytes,
|
|
191
|
+
},
|
|
192
|
+
});
|
|
193
|
+
throw new Error(`Attachment ${params.attachment.id} could not be normalized under the VLM byte budget`);
|
|
194
|
+
}
|
|
@@ -0,0 +1,230 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
var desc = Object.getOwnPropertyDescriptor(m, k);
|
|
5
|
+
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
|
6
|
+
desc = { enumerable: true, get: function() { return m[k]; } };
|
|
7
|
+
}
|
|
8
|
+
Object.defineProperty(o, k2, desc);
|
|
9
|
+
}) : (function(o, m, k, k2) {
|
|
10
|
+
if (k2 === undefined) k2 = k;
|
|
11
|
+
o[k2] = m[k];
|
|
12
|
+
}));
|
|
13
|
+
var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
|
|
14
|
+
Object.defineProperty(o, "default", { enumerable: true, value: v });
|
|
15
|
+
}) : function(o, v) {
|
|
16
|
+
o["default"] = v;
|
|
17
|
+
});
|
|
18
|
+
var __importStar = (this && this.__importStar) || (function () {
|
|
19
|
+
var ownKeys = function(o) {
|
|
20
|
+
ownKeys = Object.getOwnPropertyNames || function (o) {
|
|
21
|
+
var ar = [];
|
|
22
|
+
for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
|
|
23
|
+
return ar;
|
|
24
|
+
};
|
|
25
|
+
return ownKeys(o);
|
|
26
|
+
};
|
|
27
|
+
return function (mod) {
|
|
28
|
+
if (mod && mod.__esModule) return mod;
|
|
29
|
+
var result = {};
|
|
30
|
+
if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
|
|
31
|
+
__setModuleDefault(result, mod);
|
|
32
|
+
return result;
|
|
33
|
+
};
|
|
34
|
+
})();
|
|
35
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
36
|
+
exports.persistBlueBubblesAttachmentSource = persistBlueBubblesAttachmentSource;
|
|
37
|
+
exports.materializeAttachment = materializeAttachment;
|
|
38
|
+
const fs = __importStar(require("node:fs/promises"));
|
|
39
|
+
const path = __importStar(require("node:path"));
|
|
40
|
+
const runtime_1 = require("../../nerves/runtime");
|
|
41
|
+
const identity_1 = require("../identity");
|
|
42
|
+
const config_1 = require("../config");
|
|
43
|
+
const image_normalize_1 = require("./image-normalize");
|
|
44
|
+
const store_1 = require("./store");
|
|
45
|
+
const types_1 = require("./types");
|
|
46
|
+
const attachment_download_1 = require("../../senses/bluebubbles/attachment-download");
|
|
47
|
+
async function ensureReadableFile(filePath) {
|
|
48
|
+
await fs.access(filePath);
|
|
49
|
+
}
|
|
50
|
+
function extensionForAttachment(displayName, mimeType) {
|
|
51
|
+
const explicit = path.extname(displayName).trim();
|
|
52
|
+
if (explicit)
|
|
53
|
+
return explicit;
|
|
54
|
+
switch (mimeType?.trim().toLowerCase()) {
|
|
55
|
+
case "image/jpeg":
|
|
56
|
+
return ".jpg";
|
|
57
|
+
case "image/png":
|
|
58
|
+
return ".png";
|
|
59
|
+
case "image/webp":
|
|
60
|
+
return ".webp";
|
|
61
|
+
case "image/gif":
|
|
62
|
+
return ".gif";
|
|
63
|
+
case "image/tiff":
|
|
64
|
+
return ".tiff";
|
|
65
|
+
case "image/bmp":
|
|
66
|
+
return ".bmp";
|
|
67
|
+
case "image/heic":
|
|
68
|
+
return ".heic";
|
|
69
|
+
case "application/pdf":
|
|
70
|
+
return ".pdf";
|
|
71
|
+
case "audio/mp3":
|
|
72
|
+
case "audio/mpeg":
|
|
73
|
+
return ".mp3";
|
|
74
|
+
case "audio/mp4":
|
|
75
|
+
return ".m4a";
|
|
76
|
+
default:
|
|
77
|
+
return "";
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
function originalStoragePath(agentRoot, attachment, mimeType) {
|
|
81
|
+
const extension = extensionForAttachment(attachment.displayName, mimeType ?? attachment.mimeType);
|
|
82
|
+
return path.join(agentRoot, "state", "attachments", "materialized", attachment.source, attachment.sourceId, `original${extension}`);
|
|
83
|
+
}
|
|
84
|
+
function buildOriginalMaterializedAttachment(attachment, filePath, mimeType, byteCount) {
|
|
85
|
+
return {
|
|
86
|
+
attachmentId: attachment.id,
|
|
87
|
+
variant: "original",
|
|
88
|
+
path: filePath,
|
|
89
|
+
displayName: attachment.displayName,
|
|
90
|
+
mimeType: mimeType ?? attachment.mimeType,
|
|
91
|
+
byteCount: byteCount ?? attachment.byteCount,
|
|
92
|
+
};
|
|
93
|
+
}
|
|
94
|
+
async function persistBlueBubblesAttachmentSource(agentName, attachment, input, agentRoot = (0, identity_1.getAgentRoot)(agentName)) {
|
|
95
|
+
const targetPath = originalStoragePath(agentRoot, attachment, input.mimeType);
|
|
96
|
+
await fs.mkdir(path.dirname(targetPath), { recursive: true });
|
|
97
|
+
await fs.writeFile(targetPath, input.buffer);
|
|
98
|
+
const updated = (0, types_1.buildBlueBubblesAttachmentRecord)({
|
|
99
|
+
...attachment.sourceData,
|
|
100
|
+
mimeType: input.mimeType ?? attachment.sourceData.mimeType,
|
|
101
|
+
totalBytes: input.byteCount ?? attachment.byteCount,
|
|
102
|
+
}, attachment.createdAt, {
|
|
103
|
+
localPath: targetPath,
|
|
104
|
+
mimeType: input.mimeType ?? attachment.mimeType,
|
|
105
|
+
byteCount: input.byteCount ?? attachment.byteCount,
|
|
106
|
+
});
|
|
107
|
+
updated.lastSeenAt = Date.now();
|
|
108
|
+
(0, store_1.rememberRecentAttachment)(agentName, updated, agentRoot);
|
|
109
|
+
(0, runtime_1.emitNervesEvent)({
|
|
110
|
+
component: "engine",
|
|
111
|
+
event: "engine.attachment_source_persisted",
|
|
112
|
+
message: "attachment materialization updated",
|
|
113
|
+
meta: {
|
|
114
|
+
attachmentId: attachment.id,
|
|
115
|
+
source: attachment.source,
|
|
116
|
+
mimeType: updated.mimeType,
|
|
117
|
+
},
|
|
118
|
+
});
|
|
119
|
+
return updated;
|
|
120
|
+
}
|
|
121
|
+
async function materializeOriginalAttachment(agentName, attachment, agentRoot) {
|
|
122
|
+
void agentName;
|
|
123
|
+
void agentRoot;
|
|
124
|
+
switch (attachment.source) {
|
|
125
|
+
case "cli-local-file": {
|
|
126
|
+
await ensureReadableFile(attachment.sourceData.path);
|
|
127
|
+
(0, runtime_1.emitNervesEvent)({
|
|
128
|
+
component: "engine",
|
|
129
|
+
event: "engine.attachment_materialized",
|
|
130
|
+
message: "attachment materialization updated",
|
|
131
|
+
meta: {
|
|
132
|
+
attachmentId: attachment.id,
|
|
133
|
+
variant: "original",
|
|
134
|
+
source: attachment.source,
|
|
135
|
+
},
|
|
136
|
+
});
|
|
137
|
+
return {
|
|
138
|
+
attachmentId: attachment.id,
|
|
139
|
+
variant: "original",
|
|
140
|
+
path: attachment.sourceData.path,
|
|
141
|
+
displayName: attachment.displayName,
|
|
142
|
+
mimeType: attachment.mimeType,
|
|
143
|
+
byteCount: attachment.byteCount,
|
|
144
|
+
};
|
|
145
|
+
}
|
|
146
|
+
case "bluebubbles": {
|
|
147
|
+
const localPath = attachment.sourceData.localPath?.trim();
|
|
148
|
+
if (localPath) {
|
|
149
|
+
await ensureReadableFile(localPath);
|
|
150
|
+
(0, runtime_1.emitNervesEvent)({
|
|
151
|
+
component: "engine",
|
|
152
|
+
event: "engine.attachment_materialized",
|
|
153
|
+
message: "attachment materialization updated",
|
|
154
|
+
meta: {
|
|
155
|
+
attachmentId: attachment.id,
|
|
156
|
+
variant: "original",
|
|
157
|
+
source: attachment.source,
|
|
158
|
+
},
|
|
159
|
+
});
|
|
160
|
+
return buildOriginalMaterializedAttachment(attachment, localPath);
|
|
161
|
+
}
|
|
162
|
+
const fallbackPath = originalStoragePath(agentRoot, attachment);
|
|
163
|
+
try {
|
|
164
|
+
await ensureReadableFile(fallbackPath);
|
|
165
|
+
return buildOriginalMaterializedAttachment(attachment, fallbackPath);
|
|
166
|
+
}
|
|
167
|
+
catch {
|
|
168
|
+
const config = (0, config_1.getBlueBubblesConfig)();
|
|
169
|
+
const channelConfig = (0, config_1.getBlueBubblesChannelConfig)();
|
|
170
|
+
const downloaded = await (0, attachment_download_1.downloadBlueBubblesAttachment)(attachment.sourceData, config, channelConfig);
|
|
171
|
+
const updated = await persistBlueBubblesAttachmentSource(agentName, attachment, {
|
|
172
|
+
buffer: downloaded.buffer,
|
|
173
|
+
mimeType: downloaded.contentType,
|
|
174
|
+
byteCount: downloaded.buffer.length,
|
|
175
|
+
}, agentRoot);
|
|
176
|
+
(0, runtime_1.emitNervesEvent)({
|
|
177
|
+
component: "engine",
|
|
178
|
+
event: "engine.attachment_materialized",
|
|
179
|
+
message: "attachment materialization updated",
|
|
180
|
+
meta: {
|
|
181
|
+
attachmentId: attachment.id,
|
|
182
|
+
variant: "original",
|
|
183
|
+
source: attachment.source,
|
|
184
|
+
},
|
|
185
|
+
});
|
|
186
|
+
return buildOriginalMaterializedAttachment(updated, updated.sourceData.localPath, downloaded.contentType ?? updated.mimeType, downloaded.buffer.length);
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
async function materializeAttachment(agentName, attachmentId, options = {}) {
|
|
192
|
+
const agentRoot = options.agentRoot ?? (0, identity_1.getAgentRoot)(agentName);
|
|
193
|
+
const variant = options.variant ?? "original";
|
|
194
|
+
const attachment = (0, store_1.getRecentAttachment)(agentName, attachmentId, agentRoot);
|
|
195
|
+
if (!attachment) {
|
|
196
|
+
throw new Error(`Attachment not found: ${attachmentId}`);
|
|
197
|
+
}
|
|
198
|
+
if (variant === "original") {
|
|
199
|
+
return await materializeOriginalAttachment(agentName, attachment, agentRoot);
|
|
200
|
+
}
|
|
201
|
+
if (attachment.kind !== "image") {
|
|
202
|
+
throw new Error(`Attachment ${attachmentId} is not an image and cannot produce a vision_safe variant`);
|
|
203
|
+
}
|
|
204
|
+
const original = await materializeOriginalAttachment(agentName, attachment, agentRoot);
|
|
205
|
+
const normalizeImage = options.normalizeImage ?? image_normalize_1.normalizeImageForVision;
|
|
206
|
+
const normalized = await normalizeImage({
|
|
207
|
+
attachment,
|
|
208
|
+
sourcePath: original.path,
|
|
209
|
+
agentName,
|
|
210
|
+
agentRoot,
|
|
211
|
+
});
|
|
212
|
+
(0, runtime_1.emitNervesEvent)({
|
|
213
|
+
component: "engine",
|
|
214
|
+
event: "engine.attachment_materialized",
|
|
215
|
+
message: "attachment materialization updated",
|
|
216
|
+
meta: {
|
|
217
|
+
attachmentId: attachment.id,
|
|
218
|
+
variant: "vision_safe",
|
|
219
|
+
source: attachment.source,
|
|
220
|
+
},
|
|
221
|
+
});
|
|
222
|
+
return {
|
|
223
|
+
attachmentId: attachment.id,
|
|
224
|
+
variant: "vision_safe",
|
|
225
|
+
path: path.resolve(normalized.path),
|
|
226
|
+
displayName: attachment.displayName,
|
|
227
|
+
mimeType: normalized.mimeType ?? attachment.mimeType,
|
|
228
|
+
byteCount: normalized.byteCount,
|
|
229
|
+
};
|
|
230
|
+
}
|