@patimweb/pi-teams 1.10.0 → 1.11.0

package/docs/tools.md

@@ -41,10 +41,10 @@ the default account. See [configuration.md](configuration.md).
 
 | Tool | Description |
 |------|-------------|
-| `teams_send_chat_message` | Send a chat message as you |
+| `teams_send_chat_message` | Send a chat message as you, optionally with images |
 | `teams_create_chat` | Start a 1:1 or group chat, optionally with the first message |
-| `teams_send_channel_message` | Post in a channel as you |
-| `teams_reply_channel_message` | Reply inside an existing thread |
+| `teams_send_channel_message` | Post in a channel as you, optionally with images |
+| `teams_reply_channel_message` | Reply inside an existing thread, optionally with images |
 | `teams_react` | Add or remove an emoji reaction |
 | `teams_mark_read` | Mark a chat as read (or unread again) — the natural end of a catch-up |
 | `teams_update_message` | Edit a chat message you already sent |
@@ -137,9 +137,11 @@ subset Teams renders in a chat bubble:
 | `[label](https://…)` | link |
 | `# Heading` | a bold line — Teams has no headings in a chat |
 
-Tables and images are deliberately **not** translated: Teams renders neither in
-a chat, so passing them through would look worse than plain text. Put a table in
-a file and link it instead.
+A markdown image and a table are deliberately **not** translated: Teams renders
+neither in a chat bubble, and an image URL would only work if the recipient's
+client could reach it. A picture that is already on disk goes out through the
+`images` parameter instead — see below. For a table, put it in a file and link
+to it.
 
 Pass `html: true` on a send tool to bypass the conversion and supply raw HTML
 instead. Everything else is escaped, so a message body cannot inject markup.
@@ -148,3 +150,30 @@ This is also why pi writes the way it does — short, answer-first, bullets for
 anything enumerable. The medium is a chat bubble, not a document. The rules the
 model follows are in `skills/teams-collaboration/SKILL.md`, under *How to format
 a Teams message*.
+
+## Sending images
+
+`teams_send_chat_message`, `teams_send_channel_message` and
+`teams_reply_channel_message` take `images: ["/pfad/bild.png"]` — local paths of
+pictures to attach. They appear under the text, in the order given, and a
+message may consist of images alone.
+
+Teams keeps an inline picture inside the message itself: the bytes travel in the
+same request as the text, and the body points at them (`hostedContents`).
+Nothing is uploaded to SharePoint first, which is why this works with the
+permissions the package already asks for and costs no admin consent.
+
+- Accepted: png, jpeg, gif, webp, bmp. Anything else is refused before the
+  message is built, because Teams would show it as a broken picture.
+- **4 MB per image**, the limit Graph enforces on hosted content.
+- The path must be readable from the process that sends. For a pi running in a
+  container that is a path *inside* the container.
+
+This is attachments-for-images only. A PDF, a DOCX or a ZIP cannot be sent: a
+real file attachment needs the file in SharePoint or OneDrive and a `contentUrl`
+pointing at it, which means asking for a write scope on the user's drive. The
+package deliberately does not ask for `Files.ReadWrite`, so a non-image is
+refused rather than half-supported — link to the file instead.
+
+What was sent, pictures included, is recorded in the audit entry and shown in
+the confirmation prompt at `safetyLevel: confirm`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@patimweb/pi-teams",
-  "version": "1.10.0",
+  "version": "1.11.0",
   "description": "Microsoft Teams integration for pi coding agent — acts fully as the signed-in user across multiple accounts and tenants: chats, channels, meetings, presence, with configurable guardrails",
   "keywords": [
     "pi-package",

package/skills/teams-collaboration/SKILL.md

@@ -97,10 +97,10 @@ instead of trying anyway.
 
 | Tool | Use when |
 |------|----------|
-| `teams_send_chat_message` | Sending a chat message |
+| `teams_send_chat_message` | Sending a chat message, with or without images |
 | `teams_create_chat` | Starting a new conversation |
-| `teams_send_channel_message` | Posting a new channel message |
-| `teams_reply_channel_message` | Answering inside an existing thread |
+| `teams_send_channel_message` | Posting a new channel message, with or without images |
+| `teams_reply_channel_message` | Answering inside an existing thread, with or without images |
 | `teams_react` | Adding or removing an emoji reaction |
 | `teams_update_message` | Fixing a message the user already sent |
 | `teams_chat_members` | Bringing someone into a group chat, or taking them out |
@@ -240,6 +240,24 @@ deliberate and must not be worked around:
 
 Ordinary SharePoint and OneDrive attachments download normally.
 
+### Sending images
+
+The three send tools take `images: ["/path/bild.png"]`: local pictures to attach
+to the message. They are shown under the text, a message may be images only, and
+nothing is uploaded anywhere — Teams keeps them inside the message.
+
+- png, jpeg, gif, webp, bmp, **at most 4 MB each**. Anything else is refused
+  before the message is built; report that instead of retrying with the same file.
+- The path is read by the process that sends. Inside a container that is a path
+  *inside* the container, not on the user's machine.
+- Only images. A PDF, DOCX or ZIP cannot be attached — give the user a link to
+  the file instead. Do not promise an attachment you cannot send.
+- The confirmation prompt and the audit entry name the images that go out, so
+  what the user approves is what is sent.
+
+For a table, the same applies as for any other non-image: put it in a file and
+link it.
+
 ## Channel addressing
 
 Channels are addressed as `Team/Channel` or via separate `team` and `channel`

package/src/graph/messages.ts

@@ -12,6 +12,7 @@ import type { ChannelSummary, MessageLocation, MessageSummary, PersonRef } from
 import { graphGetOptional, graphList, graphPatch, graphPost } from "./client.ts";
 import { mapMessage } from "./mappers.ts";
 import { markdownToTeamsHtml } from "../utils/richtext.ts";
+import type { HostedImage } from "../utils/hosted-content.ts";
 
 /* eslint-disable @typescript-eslint/no-explicit-any */
 type Raw = Record<string, any>;
@@ -37,6 +38,13 @@ export interface MessageInput {
 	importance?: string;
 	/** People to @-mention; each must appear in the body as @DisplayName */
 	mentions?: PersonRef[];
+	/**
+	 * Inline images, already read from disk and base64-encoded.
+	 *
+	 * They travel in the same POST as the body — Teams keeps the bytes inside the
+	 * message — so there is no second request and no file upload anywhere.
+	 */
+	images?: HostedImage[];
 }
 
 /**
@@ -77,12 +85,29 @@ export function buildMessageBody(input: MessageInput): Record<string, unknown> {
 		});
 	});
 
+	// The pictures follow the text rather than being placed inside it: the caller
+	// writes the message, and a model that also had to pick a position in the HTML
+	// would be choosing something nobody asked it to choose. The id is ours to
+	// pick — it only has to be the same in the body and in hostedContents, which is
+	// the part that quietly goes wrong.
+	const hostedContents: Record<string, unknown>[] = [];
+	(input.images ?? []).forEach((image, index) => {
+		const id = String(index + 1);
+		content += `<br><img src="../hostedContents/${id}/$value" alt="${escapeHtml(image.name)}">`;
+		hostedContents.push({
+			"@microsoft.graph.temporaryId": id,
+			contentBytes: image.contentBytes,
+			contentType: image.contentType,
+		});
+	});
+
 	const body: Record<string, unknown> = {
 		body: { contentType: "html", content },
 	};
 	if (input.subject) body.subject = input.subject;
 	if (input.importance) body.importance = input.importance;
 	if (mentionEntries.length > 0) body.mentions = mentionEntries;
+	if (hostedContents.length > 0) body.hostedContents = hostedContents;
 
 	return body;
 }

package/src/safety/index.ts

@@ -119,17 +119,29 @@ export function formatMutationSummary(
 		return str.length > max ? `${str.slice(0, max - 1)}…` : str;
 	};
 	const disclosed = (value: unknown): string => applyAiFooter(text(value, 400), footer);
+	/**
+	 * The images that travel with the message, by file name.
+	 *
+	 * Someone approving a send has to see that a picture is part of it: a
+	 * confirmation that shows only the text would be asking about half of it.
+	 */
+	const withImages = (value: unknown): string => {
+		const paths = Array.isArray(value) ? value.filter((p): p is string => typeof p === "string") : [];
+		if (paths.length === 0) return "";
+		const names = paths.map((p) => p.split(/[\\/]/).pop() ?? p).join(", ");
+		return `\n\n(with ${paths.length === 1 ? "1 image" : `${paths.length} images`}: ${text(names, 120)})`;
+	};
 
 	switch (toolName) {
 		case "teams_send_chat_message":
-			return `Send a chat message to "${text(params.chat, 60)}":\n\n${disclosed(params.body)}`;
+			return `Send a chat message to "${text(params.chat, 60)}":\n\n${disclosed(params.body)}${withImages(params.images)}`;
 		case "teams_send_channel_message": {
 			const where = params.team ? `${text(params.team, 40)}/${text(params.channel, 40)}` : text(params.channel, 60);
-			return `Post in channel ${where}:\n\n${disclosed(params.body)}`;
+			return `Post in channel ${where}:\n\n${disclosed(params.body)}${withImages(params.images)}`;
 		}
 		case "teams_reply_channel_message": {
 			const where = params.team ? `${text(params.team, 40)}/${text(params.channel, 40)}` : text(params.channel, 60);
-			return `Reply in thread ${text(params.messageId, 40)} (${where}):\n\n${disclosed(params.body)}`;
+			return `Reply in thread ${text(params.messageId, 40)} (${where}):\n\n${disclosed(params.body)}${withImages(params.images)}`;
 		}
 		case "teams_create_chat": {
 			const people = Array.isArray(params.participants) ? params.participants.join(", ") : text(params.participants);

package/src/tools/teams-send-channel-message.ts

@@ -12,6 +12,7 @@ import { resolveUserId } from "../graph/me.ts";
 import { auditWrite } from "../safety/audit.ts";
 import { CHAT_MESSAGE_BODY_DESCRIPTION, CHAT_SHAPE_GUIDELINE } from "../utils/chat-style.ts";
 import { applyAiFooter } from "../utils/disclosure.ts";
+import { imageNote, readHostedImage } from "../utils/hosted-content.ts";
 import { assertAccess } from "../safety/index.ts";
 import { truncate } from "../utils/formatting.ts";
 import { requireChannel } from "./resolve.ts";
@@ -67,6 +68,13 @@ const sharedParams = {
 	),
 	importance: Type.Optional(Type.String({ description: "'normal' (default), 'high' or 'urgent'" })),
 	html: Type.Optional(Type.Boolean({ description: "Send 'body' as raw HTML instead of converting markdown" })),
+	images: Type.Optional(
+		Type.Array(Type.String(), {
+			description:
+				"Local paths of images to attach (png, jpg, gif, webp, bmp; 4 MB each at most). " +
+				"They are shown inline, under the text.",
+		}),
+	),
 };
 
 export const teamsSendChannelMessageTool = {
@@ -74,7 +82,8 @@ export const teamsSendChannelMessageTool = {
 	description:
 		"Post a new message in a Microsoft Teams channel as the signed-in user. It appears exactly as if the " +
 		"user posted it. Give the channel as a 'Team/Channel' path or pass 'team' separately. " +
-		"'subject' starts a titled post; use teams_reply_channel_message to answer inside an existing thread.",
+		"'subject' starts a titled post; use teams_reply_channel_message to answer inside an existing thread. " +
+		"'images' attaches local image files to the post itself; pi cannot attach other kinds of files.",
 	parameters: Type.Object({
 		...sharedParams,
 		subject: Type.Optional(Type.String({ description: "Optional post title" })),
@@ -99,6 +108,7 @@ export const teamsSendChannelMessageTool = {
 			mentions?: string[];
 			importance?: string;
 			html?: boolean;
+			images?: string[];
 		},
 		signal: AbortSignal | undefined,
 		_onUpdate: undefined,
@@ -106,7 +116,15 @@ export const teamsSendChannelMessageTool = {
 	): Promise<ToolResult> {
 		return run(async () => {
 			const conn = connectionFor(ctx, params.account, params.tenant);
-			if (!params.body.trim()) return errorResult("Refusing to post an empty message.");
+
+			// Read before anything is posted: an image that cannot be read has to stop
+			// the post, not arrive as text promising one.
+			const images = [];
+			for (const path of params.images ?? []) images.push(await readHostedImage(path));
+
+			if (!params.body.trim() && images.length === 0) {
+				return errorResult("Refusing to post an empty message.");
+			}
 
 			const { team, channel } = await requireChannel(conn, params.team, params.channel, "write", signal);
 			const { mentions, error } = await resolveMentions(conn, params.mentions, signal);
@@ -128,6 +146,7 @@ export const teamsSendChannelMessageTool = {
 						subject: params.subject,
 						importance: params.importance,
 						mentions,
+						images,
 					},
 					signal,
 				);
@@ -138,7 +157,7 @@ export const teamsSendChannelMessageTool = {
 					tenant: conn.tenant,
 					actor: me?.upn,
 					target: `channel:${target}`,
-					summary: truncate(body, 200),
+					summary: truncate(body, 200) + imageNote(images),
 				});
 
 				return textResult(
@@ -158,7 +177,7 @@ export const teamsSendChannelMessageTool = {
 					tenant: conn.tenant,
 					actor: me?.upn,
 					target: `channel:${target}`,
-					summary: truncate(body, 200),
+					summary: truncate(body, 200) + imageNote(images),
 					error: err instanceof Error ? err.message : String(err),
 				});
 				throw err;
@@ -172,7 +191,8 @@ export const teamsReplyChannelMessageTool = {
 	description:
 		"Reply inside an existing Microsoft Teams channel thread as the signed-in user. " +
 		"Get the messageId from teams_read_channel, teams_read_thread or teams_search_messages. " +
-		"Replying keeps the conversation in one thread instead of starting a new post.",
+		"Replying keeps the conversation in one thread instead of starting a new post. " +
+		"'images' attaches local image files to the reply itself.",
 	parameters: Type.Object({
 		...sharedParams,
 		messageId: Type.String({ description: "ID of the thread's opening message" }),
@@ -192,6 +212,7 @@ export const teamsReplyChannelMessageTool = {
 			mentions?: string[];
 			importance?: string;
 			html?: boolean;
+			images?: string[];
 		},
 		signal: AbortSignal | undefined,
 		_onUpdate: undefined,
@@ -199,7 +220,15 @@ export const teamsReplyChannelMessageTool = {
 	): Promise<ToolResult> {
 		return run(async () => {
 			const conn = connectionFor(ctx, params.account, params.tenant);
-			if (!params.body.trim()) return errorResult("Refusing to post an empty reply.");
+
+			// Read before anything is posted: an image that cannot be read has to stop
+			// the reply, not arrive as text promising one.
+			const images = [];
+			for (const path of params.images ?? []) images.push(await readHostedImage(path));
+
+			if (!params.body.trim() && images.length === 0) {
+				return errorResult("Refusing to post an empty reply.");
+			}
 
 			const { team, channel } = await requireChannel(conn, params.team, params.channel, "write", signal);
 			const { mentions, error } = await resolveMentions(conn, params.mentions, signal);
@@ -221,6 +250,7 @@ export const teamsReplyChannelMessageTool = {
 						html: params.html,
 						importance: params.importance,
 						mentions,
+						images,
 					},
 					signal,
 				);
@@ -231,7 +261,7 @@ export const teamsReplyChannelMessageTool = {
 					tenant: conn.tenant,
 					actor: me?.upn,
 					target: `channel:${target}#${params.messageId}`,
-					summary: truncate(body, 200),
+					summary: truncate(body, 200) + imageNote(images),
 				});
 
 				return textResult(
@@ -251,7 +281,7 @@ export const teamsReplyChannelMessageTool = {
 					tenant: conn.tenant,
 					actor: me?.upn,
 					target: `channel:${target}#${params.messageId}`,
-					summary: truncate(body, 200),
+					summary: truncate(body, 200) + imageNote(images),
 					error: err instanceof Error ? err.message : String(err),
 				});
 				throw err;

package/src/tools/teams-send-chat-message.ts

@@ -23,6 +23,7 @@ import { auditWrite } from "../safety/audit.ts";
 import { assertAccess } from "../safety/index.ts";
 import { CHAT_MESSAGE_BODY_DESCRIPTION, CHAT_SHAPE_GUIDELINE } from "../utils/chat-style.ts";
 import { applyAiFooter } from "../utils/disclosure.ts";
+import { imageNote, readHostedImage } from "../utils/hosted-content.ts";
 import { truncate } from "../utils/formatting.ts";
 import { requireChat } from "./resolve.ts";
 import {
@@ -46,6 +47,7 @@ interface SendChatParams {
 	mentions?: string[];
 	importance?: string;
 	html?: boolean;
+	images?: string[];
 }
 
 export const teamsSendChatMessageTool = {
@@ -54,7 +56,8 @@ export const teamsSendChatMessageTool = {
 		"Send a message in a Microsoft Teams chat as the signed-in user. The message appears exactly as if the " +
 		"user typed it. Accepts a chat ID, group chat topic, or participant name/e-mail. " +
 		"Use 'mentions' with names or e-mail addresses to @-mention people — write '@Display Name' in the body " +
-		"where the mention should appear.",
+		"where the mention should appear. " +
+		"'images' attaches local image files to the message itself; pi cannot attach other kinds of files.",
 	parameters: Type.Object({
 		chat: Type.String({ description: "Chat ID, group chat topic, or a participant's name/e-mail" }),
 		body: Type.String({ description: CHAT_MESSAGE_BODY_DESCRIPTION }),
@@ -71,6 +74,13 @@ export const teamsSendChatMessageTool = {
 		html: Type.Optional(
 			Type.Boolean({ description: "Send 'body' as raw HTML instead of converting markdown" }),
 		),
+		images: Type.Optional(
+			Type.Array(Type.String(), {
+				description:
+					"Local paths of images to attach to the message (png, jpg, gif, webp, bmp; 4 MB each at most). " +
+					"They are shown inline, under the text.",
+			}),
+		),
 	}),
 	promptSnippet: "Send a message in a Teams chat as the user",
 	promptGuidelines: [
@@ -91,7 +101,14 @@ export const teamsSendChatMessageTool = {
 		return run(async () => {
 			const conn = connectionFor(ctx, params.account, params.tenant);
 
-			if (!params.body.trim()) return errorResult("Refusing to send an empty message.");
+			// Read before anything is sent: a picture that cannot be read has to stop
+			// the message, not arrive as text promising one.
+			const images = [];
+			for (const path of params.images ?? []) images.push(await readHostedImage(path));
+
+			if (!params.body.trim() && images.length === 0) {
+				return errorResult("Refusing to send an empty message.");
+			}
 
 			const chat = await requireChat(conn, params.chat, "write", signal);
 
@@ -129,6 +146,7 @@ export const teamsSendChatMessageTool = {
 						html: params.html,
 						importance: params.importance,
 						mentions,
+						images,
 					},
 					signal,
 				);
@@ -139,7 +157,7 @@ export const teamsSendChatMessageTool = {
 					tenant: conn.tenant,
 					actor: me?.upn,
 					target: `chat:${chat.label}`,
-					summary: truncate(body, 200),
+					summary: truncate(body, 200) + imageNote(images),
 				});
 
 				// Replying is reading: clear the unread marker the answer was written for.

package/src/utils/hosted-content.ts

@@ -0,0 +1,105 @@
+/**
+ * Attaching a local image to a message.
+ *
+ * Teams keeps an inline image inside the message itself as a `hostedContent`:
+ * the body points at `../hostedContents/{id}/$value` and the bytes travel in the
+ * same POST. That is what makes this cheap — nothing has to be uploaded to
+ * SharePoint first, which is what a real file attachment needs, and which would
+ * mean asking for a write scope on the user's drive.
+ *
+ * Graph caps a hosted content at 4 MB, and an image of a type Teams does not
+ * render would reach the reader as a broken picture, so both are checked here,
+ * before the message that carries it is built.
+ */
+
+import { readFile, stat } from "node:fs/promises";
+import { basename, extname } from "node:path";
+
+/** Graph rejects a hosted content larger than this. */
+export const MAX_HOSTED_CONTENT_BYTES = 4 * 1024 * 1024;
+
+/**
+ * The image types Teams shows inline, by file extension.
+ *
+ * A list rather than "anything starting with image/": a type Teams does not
+ * render arrives as a broken image, and refusing it up front is friendlier than
+ * sending it and having the reader find out.
+ */
+export const HOSTED_CONTENT_TYPES: Record<string, string> = {
+	".png": "image/png",
+	".jpg": "image/jpeg",
+	".jpeg": "image/jpeg",
+	".gif": "image/gif",
+	".webp": "image/webp",
+	".bmp": "image/bmp",
+};
+
+/** The content type for a path, or undefined when Teams does not render it inline. */
+export function hostedContentType(path: string): string | undefined {
+	return HOSTED_CONTENT_TYPES[extname(path).toLowerCase()];
+}
+
+/**
+ * One image, ready to send.
+ *
+ * `contentBytes` is base64 because that is what Graph asks for: the bytes are
+ * JSON string data, not a multipart upload.
+ */
+export interface HostedImage {
+	/** The file name, as the attachment is described in audit entries */
+	name: string;
+	contentType: string;
+	contentBytes: string;
+}
+
+/** A byte count as a human reads it — the limits here are in megabytes. */
+export function formatBytes(bytes: number): string {
+	return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
+}
+
+/**
+ * Read one local image for sending.
+ *
+ * Throws with the reason instead of sending something partial: a message that
+ * quietly goes out without the picture the user asked for is worse than a tool
+ * call that says why it did not.
+ */
+export async function readHostedImage(path: string): Promise<HostedImage> {
+	const name = basename(path);
+
+	const contentType = hostedContentType(path);
+	if (!contentType) {
+		const supported = [...new Set(Object.values(HOSTED_CONTENT_TYPES))].join(", ");
+		throw new Error(`"${name}" is not an image Teams shows inline. Supported types: ${supported}.`);
+	}
+
+	let info;
+	try {
+		info = await stat(path);
+	} catch {
+		throw new Error(`"${path}" could not be read. Give a path to a file that exists locally.`);
+	}
+	if (info.isDirectory()) throw new Error(`"${path}" is a directory, not an image.`);
+	if (info.size === 0) throw new Error(`"${name}" is empty.`);
+	if (info.size > MAX_HOSTED_CONTENT_BYTES) {
+		throw new Error(
+			`"${name}" is ${formatBytes(info.size)}; Teams accepts at most ` +
+				`${formatBytes(MAX_HOSTED_CONTENT_BYTES)} per image.`,
+		);
+	}
+
+	const bytes = await readFile(path);
+	return { name, contentType, contentBytes: bytes.toString("base64") };
+}
+
+/**
+ * The images named in one phrase, for the audit entry.
+ *
+ * What went out is the thing an audit entry has to answer, and a message with a
+ * picture in it is not described by its text alone.
+ */
+export function imageNote(images: readonly HostedImage[]): string {
+	if (images.length === 0) return "";
+	const label = images.length === 1 ? "image" : "images";
+	return ` [${label}: ${images.map((image) => image.name).join(", ")}]`;
+}