oomi-ai 0.7.0 → 0.7.2

package/agent_instructions.md

@@ -271,12 +271,15 @@ To reach the user on their phone when they are not in the conversation, send a p
 oomi notify "Your 7-day plan is ready. Want to look now?" --title "Fitness" --json
 ```
 
+Add `--connection NAME` when you run it outside an Oomi conversation (for example from a scheduled job). Turns Oomi starts already set `OOMI_CONNECTION`. Omit it for the default OpenClaw connection.
+
 Rules:
 
 - Notify only for things worth interrupting the user for: a reminder they asked for, a finished task, or something time-sensitive. Say everything else in chat.
 - Keep the message short and specific. The title defaults to your agent name.
 - Tapping the notification opens the Oomi app to the conversation, where the message also appears.
-- Messages you send the user on the Oomi channel already notify their phone. Do not also run `oomi notify` for the same message.
+- Replies to the user's own Oomi messages, and OpenClaw sends on the Oomi channel, already notify their phone. Do not also run `oomi notify` for the same message.
+- Anything you start on your own, such as a reminder or finished background work, needs `oomi notify`. Hermes agents have no other way to reach the phone.
 - If `notification.status` is `skipped`, tell the user `notification.guidance` instead of retrying.
 - If the request is rate limited, do not retry until `retryAfterSeconds` has passed.
 
@@ -338,3 +341,9 @@ Use `oomi routines pause|resume|end <routine-id> --file action.json` with expect
 For instructions in another chat, the user must first grant this connection routineManagementEnabled through Oomi Agent settings or `routines settings` using an authenticated Oomi sourceMessageId. Then routine/response mutations can substitute externalInstruction:{channel,messageId,instruction} for sourceMessageId. Record the actual message reference and instruction. Oomi labels this runtime-attested, not independently authenticated. Never turn on the permission yourself or invent external approval. Ownership, data permissions, and primary selection remain separate.
 
 The running bridge receives routine.approved, routine.due, routine.changed, routine.response_recorded, and primary_agent.changed. Ordinary events sync quietly without a model call; explicitly enabled routine wakes are bounded and are not retried after an uncertain model dispatch. Refresh canonical state before acting on a delayed event. Due events already have notificationHandled:true; do not send another phone alert or message for every event. Persist useful context for the next conversation. If a follow-up is explicitly linked to an event, include relatedEventId in the notification API so Oomi suppresses an extra push. Keep Oomi credentials private; all /v1/routines, /v1/agent_preferences, /v1/routine_occurrences, and /v1/agent_events requests use the saved device bearer token, never a user ID supplied by the agent.
+
+### Reply first, then share routine cards
+
+Always answer the person in your own conversational voice after a routine or Space tool succeeds. Explain what you recommend or what changed, and invite a useful next step when appropriate. A tool result or review card is not the reply. Do not return only routine names, IDs, JSON, or a stock list of chips. Mention the routines that actually belong to this request; do not automatically recreate or list three routines on unrelated requests. Oomi places newly created routine cards beneath the completed reply in that creation turn. Later approvals update the original card in place. Keep talking normally after a user starts or changes a routine; do not require another UI click when their instruction is already clear.
+
+Oomi renders simple Markdown in chat: short paragraphs, bullet or numbered lists, and **bold**. Keep replies brief and scannable, and put long plans in a Space note instead of the chat. A Space link in your reply appears as a card, so share it once rather than repeating the URL or listing IDs.

package/bin/oomi-ai.js

@@ -286,9 +286,10 @@ Commands:
     Save user-requested routines and settings. Use the current sourceMessageId,
     expectedRevision, and a stable idempotencyKey. No extra approval button is needed.
 
-  notify "<message>" [--title TITLE]
+  notify "<message>" [--title TITLE] [--connection NAME]
     Send the linked user a push notification on their phone. Use it for things worth
-    interrupting them for; say everything else in chat. Limited per hour.
+    interrupting them for; say everything else in chat. Limited per hour. Turns started
+    by an Oomi bridge already use their own connection (OOMI_CONNECTION).
 
 Common flags:
   --agents-file PATH     Override AGENTS.md path
@@ -2641,7 +2642,7 @@ async function startOpenclawBridge(flags) {
   }
 
   const localRuntime = bridgeState.localAgent
-    ? new LocalAgentRuntime(bridgeState.localAgent, path.dirname(resolveBridgeStatePath()), (health) => updateBridgeStatus(health))
+    ? new LocalAgentRuntime({ ...bridgeState.localAgent, connection: bridgeConnectionName }, path.dirname(resolveBridgeStatePath()), (health) => updateBridgeStatus(health))
     : null;
   const gateway = localRuntime ? { gatewayUrl: 'local-agent:', configPath: resolveBridgeStatePath() } : readOpenclawGatewayConfig();
   if (!localRuntime && !gateway.token && !gateway.password) {
@@ -4546,7 +4547,8 @@ async function main() {
     process.exit(0);
   }
 
-  const connection = args.flags.connection || args.flags.name || (command === 'connect' && args.flags.runtime !== 'openclaw' ? args.flags.runtime : '');
+  // Agents started by an Oomi bridge inherit OOMI_CONNECTION, so their commands reach their own account.
+  const connection = args.flags.connection || args.flags.name || (command === 'connect' ? (args.flags.runtime !== 'openclaw' ? args.flags.runtime : '') : String(process.env.OOMI_CONNECTION || '').trim());
   if (connection && !/^[a-z0-9][a-z0-9_-]{0,63}$/.test(connection)) throw new Error('Connection name must use lowercase letters, numbers, underscores, or hyphens.');
   bridgeConnectionName = connection || '';
 
@@ -4556,6 +4558,7 @@ async function main() {
       spaces: { catalog: 'oomi spaces catalog', list: 'oomi spaces list', create: 'oomi spaces create --file space.json', update: 'oomi spaces update ID --file change.json', revisions: 'oomi spaces revisions ID', guide: 'oomi instructions' },
       health: { steps: 'oomi context health --window week --metric steps --json', calendarWeek: 'oomi context health --window range --from YYYY-MM-DD --to YYYY-MM-DD --metric steps --json', largeResults: '--output new-file.json', plugins: 'oomi context plugins', missing: 'unknown, never zero', week: 'rolling seven days; use explicit range for calendar week' },
       routines: 'oomi routines list; oomi routines context', identity: 'oomi identity get', updates: 'oomi updates',
+      notify: { send: 'oomi notify "<message>" --title TITLE --json', when: 'reminders the user asked for, finished work, time-sensitive news; everything else in chat', skipped: 'tell the user notification.guidance', rateLimited: 'wait retryAfterSeconds' },
       costs: { passiveEvents: 'no model turn', wakeAgent: 'routine.agentWakeEnabled defaults false; enable only on explicit user request', maximumRoutineWakeAttemptsPerHour: 6, retries: 'uncertain event turns are not automatically rerun', connectionCheck: 'first connect or explicit oomi verify makes one model call; reconnect reuses verification', identity: 'a changed identity revision is delivered once to keep names consistent' }
     }, null, 2)); return;
   }

package/lib/localAgent.js

@@ -5,6 +5,56 @@ import { spawn } from 'node:child_process';
 import { EventEmitter } from 'node:events';
 import { createHash, randomUUID } from 'node:crypto';
 
+const IMAGE_EXTENSIONS = { 'image/jpeg': '.jpg', 'image/png': '.png', 'image/webp': '.webp', 'image/gif': '.gif' };
+const MAX_IMAGE_BYTES = 10 * 1024 * 1024;
+
+/**
+ * Chat photos arrive inline as base64 (the shape OpenClaw's gateway also accepts), with an
+ * optional `storagePath` in the owner's Oomi storage so the conversation can show it again
+ * later. Hermes looks at one image per query.
+ */
+export function readImageAttachments(attachments) {
+  if (!Array.isArray(attachments) || attachments.length === 0) return [];
+  if (attachments.length > 1) throw new Error('Send one photo at a time.');
+  return attachments.map((attachment) => {
+    const mimeType = String(attachment?.mimeType || '').toLowerCase();
+    if (attachment?.type !== 'image' || !IMAGE_EXTENSIONS[mimeType]) throw new Error('Only JPEG, PNG, WebP, or GIF photos can be sent.');
+    const bytes = Buffer.from(typeof attachment.content === 'string' ? attachment.content : '', 'base64');
+    if (!bytes.length) throw new Error('The photo arrived empty. Try sending it again.');
+    if (bytes.length > MAX_IMAGE_BYTES) throw new Error('That photo is too large. Photos can be up to 10 MB.');
+    const dimension = (value) => (Number.isFinite(value) && value > 0 ? Math.round(value) : undefined);
+    return {
+      mimeType,
+      bytes,
+      storagePath: typeof attachment.storagePath === 'string' && attachment.storagePath.trim() ? attachment.storagePath.trim() : undefined,
+      width: dimension(attachment.width),
+      height: dimension(attachment.height),
+    };
+  });
+}
+
+// Private files for one turn; the caller removes the folder when the turn ends.
+function writeTurnImages(images) {
+  const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'oomi-turn-'));
+  fs.chmodSync(directory, 0o700);
+  const paths = images.map((image, index) => {
+    const file = path.join(directory, `photo-${index + 1}${IMAGE_EXTENSIONS[image.mimeType]}`);
+    fs.writeFileSync(file, image.bytes, { mode: 0o600 });
+    return file;
+  });
+  return { directory, paths };
+}
+
+// What the conversation remembers: the words, plus where the owner's photo is stored (never the bytes).
+function userContent(text, images) {
+  const stored = images.filter((image) => image.storagePath);
+  if (!stored.length) return text;
+  return [
+    ...(text.trim() ? [{ type: 'text', text }] : []),
+    ...stored.map(({ mimeType, storagePath, width, height }) => ({ type: 'image', mimeType, storagePath, ...(width ? { width } : {}), ...(height ? { height } : {}) })),
+  ];
+}
+
 export function writePrivateJson(file, data) {
   fs.mkdirSync(path.dirname(file), { recursive: true, mode: 0o700 });
   const temporary = `${file}.${process.pid}.tmp`;
@@ -55,8 +105,11 @@ export function runLocalAgent(config, request, { signal, timeoutMs = 180000 } =
       if (config.profile) args.push('--profile', config.profile);
       args.push('chat', '--quiet', '--query-file', '-');
       if (request.runtimeSessionId) args.push('--resume', request.runtimeSessionId);
+      if (request.imagePaths?.[0]) args.push('--image', request.imagePaths[0]);
     }
-    const child = spawn(config.command[0], args, { env: config.hermesHome ? { ...process.env, HERMES_HOME: config.hermesHome } : process.env, cwd: config.cwd, shell: false, detached: process.platform !== 'win32', stdio: ['pipe', 'pipe', 'pipe'] });
+    // The agent's own `oomi` commands (notify, routines, spaces) use this connection by default.
+    const env = { ...process.env, ...(config.hermesHome ? { HERMES_HOME: config.hermesHome } : {}), ...(config.connection ? { OOMI_CONNECTION: config.connection } : {}) };
+    const child = spawn(config.command[0], args, { env, cwd: config.cwd, shell: false, detached: process.platform !== 'win32', stdio: ['pipe', 'pipe', 'pipe'] });
     let output = '';
     let diagnostics = '';
     child.stdout.setEncoding('utf8');
@@ -116,8 +169,11 @@ export class LocalAgentRuntime {
   file(sessionKey) { return path.join(this.directory, 'conversations', createHash('sha256').update(sessionKey).digest('hex') + '.json'); }
   history(sessionKey) { return readJson(this.file(sessionKey), { messages: [], receipts: {} }); }
   async turn({ sessionKey, message, idempotencyKey, attachments = [], retryInterrupted = false }) {
-    if (typeof sessionKey !== 'string' || !sessionKey || typeof idempotencyKey !== 'string' || !idempotencyKey || typeof message !== 'string' || !message.trim()) throw new Error('A session, message, and idempotency key are required.');
-    if (attachments.length) throw new Error('This adapter supports text messages. Attachments require a runtime adapter that supports them.');
+    const images = readImageAttachments(attachments);
+    const text = typeof message === 'string' ? message : '';
+    if (typeof sessionKey !== 'string' || !sessionKey || typeof idempotencyKey !== 'string' || !idempotencyKey || (!text.trim() && !images.length)) throw new Error('A session, message, and idempotency key are required.');
+    const prompt = text.trim() ? text : 'Here is a photo.';
+    message = text;
     const key = `${sessionKey}\0${idempotencyKey}`;
     if (this.active.has(key)) {
       const active = this.active.get(key);
@@ -136,15 +192,18 @@ export class LocalAgentRuntime {
       if (controller.signal.aborted) throw new Error('Agent turn cancelled.');
       state.receipts[receiptKey] = { message, status: 'running' };
       writePrivateJson(this.file(sessionKey), state);
+      let scratch;
       try {
-        const result = await runLocalAgent(this.config, { sessionKey, message, idempotencyKey, history: state.messages, runtimeSessionId: state.runtimeSessionId || this.config.resume }, { signal: controller.signal });
+        scratch = images.length ? writeTurnImages(images) : null;
+        const result = await runLocalAgent(this.config, { sessionKey, message: prompt, idempotencyKey, history: state.messages, runtimeSessionId: state.runtimeSessionId || this.config.resume, imagePaths: scratch?.paths || [] }, { signal: controller.signal });
         state.runtimeSessionId = result.sessionId;
-        state.messages.push({ role: 'user', content: message, timestamp: Date.now() }, { role: 'assistant', content: [{ type: 'text', text: result.text }], timestamp: Date.now() });
+        state.messages.push({ role: 'user', content: userContent(text, images), timestamp: Date.now() }, { role: 'assistant', content: [{ type: 'text', text: result.text }], timestamp: Date.now() });
         state.receipts[receiptKey] = { message, result };
         writePrivateJson(this.file(sessionKey), state);
         this.onHealth({ runtimeReady: true, lastReplyAt: new Date().toISOString(), lastErrorMessage: '' });
         return result;
       } catch (err) { this.onHealth({ runtimeReady: false, status: 'degraded', lastErrorMessage: err.message }); throw err; }
+      finally { if (scratch) fs.rmSync(scratch.directory, { recursive: true, force: true }); }
     });
     this.queue = promise.catch(() => {});
     this.active.set(key, { promise, controller, sessionKey, message, runId: idempotencyKey });

package/openclaw.plugin.json

@@ -2,7 +2,7 @@
   "id": "oomi-ai",
   "name": "Oomi Channel Plugin",
   "description": "Oomi connection, Spaces, health context and routines. Run oomi capabilities and oomi instructions; ordinary state updates do not invoke a model.",
-  "version": "0.7.0",
+  "version": "0.7.1",
   "author": "Oomi",
   "license": "MIT",
   "openclawVersion": ">=0.5.0",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "oomi-ai",
-  "version": "0.7.0",
+  "version": "0.7.2",
   "description": "Connect OpenClaw, Hermes, and other local agents to Oomi with verified runtime adapters",
   "bin": {
     "oomi": "bin/oomi-ai.js"

package/skills/oomi/SKILL.md

@@ -205,11 +205,14 @@ To reach the user on their phone when they are not in the conversation:
 oomi notify "Your 7-day plan is ready. Want to look now?" --title "Fitness" --json
 ```
 
+Add `--connection NAME` when you run it outside an Oomi conversation (for example from a scheduled job); turns Oomi starts already set `OOMI_CONNECTION`. Omit it for the default OpenClaw connection.
+
 Rules:
 - notify only for things worth interrupting the user for (a reminder they asked for, a finished task, something time-sensitive); say everything else in chat
 - keep it short and specific; the title defaults to your agent name
 - tapping it opens the Oomi app to the conversation, where the message also appears
-- messages you send on the Oomi channel already notify the phone; do not also run `oomi notify` for the same message
+- replies to the user's own Oomi messages and OpenClaw sends on the Oomi channel already notify the phone; do not also run `oomi notify` for the same message
+- anything you start on your own (a reminder, finished background work) needs `oomi notify`; Hermes agents have no other way to reach the phone
 - if `notification.status` is `skipped`, tell the user `notification.guidance` instead of retrying
 - if rate limited, wait `retryAfterSeconds` before trying again
 
@@ -336,3 +339,9 @@ Use `oomi routines pause|resume|end <routine-id> --file action.json` with expect
 For instructions in another chat, the user must first grant this connection routineManagementEnabled through Oomi Agent settings or `routines settings` using an authenticated Oomi sourceMessageId. Then routine/response mutations can substitute externalInstruction:{channel,messageId,instruction} for sourceMessageId. Record the actual message reference and instruction. Oomi labels this runtime-attested, not independently authenticated. Never turn on the permission yourself or invent external approval. Ownership, data permissions, and primary selection remain separate.
 
 The running bridge receives routine.approved, routine.due, routine.changed, routine.response_recorded, and primary_agent.changed. It acknowledges after the runtime accepts each event and retries failures. Refresh canonical state before acting on a delayed event. Due events already have notificationHandled:true; do not send another phone alert or message for every event. Persist useful context for the next conversation. If a follow-up is explicitly linked to an event, include relatedEventId in the notification API so Oomi suppresses an extra push. Keep Oomi credentials private; all /v1/routines, /v1/agent_preferences, /v1/routine_occurrences, and /v1/agent_events requests use the saved device bearer token, never a user ID supplied by the agent.
+
+### Reply first, then share routine cards
+
+Always answer the person in your own conversational voice after a routine or Space tool succeeds. Explain what you recommend or what changed, and invite a useful next step when appropriate. A tool result or review card is not the reply. Do not return only routine names, IDs, JSON, or a stock list of chips. Mention the routines that actually belong to this request; do not automatically recreate or list three routines on unrelated requests. Oomi places newly created routine cards beneath the completed reply in that creation turn. Later approvals update the original card in place. Keep talking normally after a user starts or changes a routine; do not require another UI click when their instruction is already clear.
+
+Oomi renders simple Markdown in chat: short paragraphs, bullet or numbered lists, and **bold**. Keep replies brief and scannable, and put long plans in a Space note instead of the chat. A Space link in your reply appears as a card, so share it once rather than repeating the URL or listing IDs.

package/skills/oomi/agent_instructions.md

@@ -280,12 +280,15 @@ To reach the user on their phone when they are not in the conversation, send a p
 oomi notify "Your 7-day plan is ready. Want to look now?" --title "Fitness" --json
 ```
 
+Add `--connection NAME` when you run it outside an Oomi conversation (for example from a scheduled job). Turns Oomi starts already set `OOMI_CONNECTION`. Omit it for the default OpenClaw connection.
+
 Rules:
 
 - Notify only for things worth interrupting the user for: a reminder they asked for, a finished task, or something time-sensitive. Say everything else in chat.
 - Keep the message short and specific. The title defaults to your agent name.
 - Tapping the notification opens the Oomi app to the conversation, where the message also appears.
-- Messages you send the user on the Oomi channel already notify their phone. Do not also run `oomi notify` for the same message.
+- Replies to the user's own Oomi messages, and OpenClaw sends on the Oomi channel, already notify their phone. Do not also run `oomi notify` for the same message.
+- Anything you start on your own, such as a reminder or finished background work, needs `oomi notify`. Hermes agents have no other way to reach the phone.
 - If `notification.status` is `skipped`, tell the user `notification.guidance` instead of retrying.
 - If the request is rate limited, do not retry until `retryAfterSeconds` has passed.
 
@@ -341,3 +344,9 @@ Use `oomi routines pause|resume|end <routine-id> --file action.json` with expect
 For instructions in another chat, the user must first grant this connection routineManagementEnabled through Oomi Agent settings or `routines settings` using an authenticated Oomi sourceMessageId. Then routine/response mutations can substitute externalInstruction:{channel,messageId,instruction} for sourceMessageId. Record the actual message reference and instruction. Oomi labels this runtime-attested, not independently authenticated. Never turn on the permission yourself or invent external approval. Ownership, data permissions, and primary selection remain separate.
 
 The running bridge receives routine.approved, routine.due, routine.changed, routine.response_recorded, and primary_agent.changed. Ordinary events sync quietly without a model call; explicitly enabled routine wakes are bounded and are not retried after an uncertain model dispatch. Refresh canonical state before acting on a delayed event. Due events already have notificationHandled:true; do not send another phone alert or message for every event. Persist useful context for the next conversation. If a follow-up is explicitly linked to an event, include relatedEventId in the notification API so Oomi suppresses an extra push. Keep Oomi credentials private; all /v1/routines, /v1/agent_preferences, /v1/routine_occurrences, and /v1/agent_events requests use the saved device bearer token, never a user ID supplied by the agent.
+
+### Reply first, then share routine cards
+
+Always answer the person in your own conversational voice after a routine or Space tool succeeds. Explain what you recommend or what changed, and invite a useful next step when appropriate. A tool result or review card is not the reply. Do not return only routine names, IDs, JSON, or a stock list of chips. Mention the routines that actually belong to this request; do not automatically recreate or list three routines on unrelated requests. Oomi places newly created routine cards beneath the completed reply in that creation turn. Later approvals update the original card in place. Keep talking normally after a user starts or changes a routine; do not require another UI click when their instruction is already clear.
+
+Oomi renders simple Markdown in chat: short paragraphs, bullet or numbered lists, and **bold**. Keep replies brief and scannable, and put long plans in a Space note instead of the chat. A Space link in your reply appears as a card, so share it once rather than repeating the URL or listing IDs.