pi-sap-aicore 0.2.2 → 0.3.0

package/CHANGELOG.md

@@ -7,6 +7,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.0] - 2026-07-01
+
+### Added
+
+- Direct foundation route support for Anthropic/Claude models deployed through SAP AI Core
+  `foundation-models` with the `aws-bedrock` executable. The provider now calls SAP
+  AI Core's Bedrock-compatible `/converse` endpoint and replays the non-streaming
+  response into pi stream events.
+- Foundation executable routing by model family: `gpt-*` uses `azure-openai`,
+  `anthropic--*` uses `aws-bedrock`, and `gemini-*` is reserved for a future
+  `gcp-vertexai` adapter.
+- Bedrock Converse message translation for text, images, assistant tool-use history,
+  and tool-result history on the direct foundation route.
+
+### Changed
+
+- Refactored the foundation provider into an executable dispatcher while preserving
+  the existing Azure OpenAI streaming path for GPT models.
+- Documentation now describes Azure OpenAI and AWS Bedrock direct foundation routes.
+
 ## [0.2.2] - 2026-06-16
 
 ### Fixed
@@ -96,7 +116,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `reasoning_effort` for OpenAI).
 - MIT license and npm packaging.
 
-[Unreleased]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.2.2...HEAD
+[Unreleased]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.2.2...v0.3.0
 [0.2.2]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.2.1...v0.2.2
 [0.2.1]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/ttiimmaahh/pi-sap-aicore/compare/v0.1.2...v0.2.0

package/README.md

@@ -3,8 +3,8 @@
 A custom provider extension for the [pi coding agent](https://pi.dev) that routes
 inference through **SAP AI Core** — via the **orchestration** service (every model
 from a single deployment) and/or **direct foundation deployments** (per-model
-Azure OpenAI endpoints with native streaming). Both register at once and share one
-login, so you pick the route per model. See
+foundation endpoints such as Azure OpenAI or AWS Bedrock). Both register at once
+and share one login, so you pick the route per model. See
 [Orchestration vs. Foundation](#orchestration-vs-foundation).
 
 ## Prerequisites
@@ -12,7 +12,8 @@ login, so you pick the route per model. See
 - pi **0.78.0 or newer** installed (`npm install -g @earendil-works/pi-coding-agent`)
 - An SAP BTP account with AI Core entitlement and an **orchestration deployment**
 - *(optional, for the foundation provider)* one or more **foundation-models
-  deployments** — one per OpenAI model you want to route directly
+  deployments** — one per model you want to route directly (`azure-openai` for
+  GPT/OpenAI models, `aws-bedrock` for Anthropic/Claude models)
 - The service key JSON for your AI Core service binding
 
 ## Credentials
@@ -103,7 +104,8 @@ You'll see the orchestration models under `sap-aicore/` (Claude, GPT-5*, Gemini)
 plus any direct **foundation** models under `sap-aicore-foundation/`:
 - `sap-aicore/anthropic--claude-4.7-opus` — Claude Opus 4.7 (orchestration)
 - `sap-aicore/gpt-5.5` — GPT-5.5 via orchestration
-- `sap-aicore-foundation/gpt-5.5` — GPT-5.5 via its direct foundation deployment
+- `sap-aicore-foundation/gpt-5.5` — GPT-5.5 via its direct Azure OpenAI foundation deployment
+- `sap-aicore-foundation/anthropic--claude-4.8-opus` — Claude Opus 4.8 via its direct AWS Bedrock foundation deployment
 
 Run `pi -e ./index.ts` to launch pi with the local extension loaded; this
 overrides any globally-installed version for the session, which is the fastest
@@ -129,24 +131,27 @@ The extension registers **two providers**, both backed by the same service key:
 | | `sap-aicore` (orchestration) | `sap-aicore-foundation` (direct) |
 |---|---|---|
 | SAP deployment | one orchestration deployment fronts **every** model | one foundation deployment **per model** |
-| Models | Claude, GPT-5*, Gemini | OpenAI (`gpt-*`) only |
-| Streaming | subject to orchestration's per-model allow-list — new models can 400 `Streaming is not supported` (we fall back to non-streaming) | **native** — streams straight from the Azure OpenAI endpoint |
-| Reasoning effort | tunable (`reasoning_effort` / `thinking`) | model **default** only (SDK pins Azure API `2024-10-21`, which has no `reasoning_effort`) |
+| Models | Claude, GPT-5*, Gemini | GPT/OpenAI (`azure-openai`) and Anthropic/Claude (`aws-bedrock`); Gemini/Vertex mapping is reserved but not implemented yet |
+| Streaming | subject to orchestration's per-model allow-list — new models can 400 `Streaming is not supported` (we fall back to non-streaming) | Azure OpenAI streams natively; AWS Bedrock currently uses non-streaming `/converse` and replays the response into pi stream events |
+| Reasoning effort | tunable (`reasoning_effort` / `thinking`) | model **default** only for Azure; Bedrock/Anthropic thinking controls are not wired yet |
 | Content filter / grounding / templating | yes | no — raw model access |
-| SDK | `@sap-ai-sdk/orchestration` | `@sap-ai-sdk/foundation-models` (`AzureOpenAiChatClient`) |
+| SDK / endpoint | `@sap-ai-sdk/orchestration` | `AzureOpenAiChatClient` for `azure-openai`; SAP `/inference/deployments/{id}/converse` for `aws-bedrock` |
 
 Both routes appear in the model list simultaneously, so you choose per model. The
-foundation route exists mainly to get **native streaming** for new OpenAI models
-that orchestration hasn't enabled streaming for yet (e.g. `gpt-5.5`).
+foundation route exists mainly to access new models directly when orchestration
+lags behind model deployment or streaming support (for example `gpt-5.5` on
+Azure OpenAI or a newly deployed Claude model on AWS Bedrock).
 
 **Adding a foundation model:** it needs its own foundation-models deployment in
-SAP AI Core — one per (model, version, resource group); the SDK resolves it by
-model name, so no deployment IDs to wire in. Then add its `id` to the per-machine
-extension overlay at `~/.pi/agent/pi-sap-aicore/models.json`:
+SAP AI Core — one per (model, version, resource group). The extension chooses the
+foundation executable from the model id: `gpt-*` → `azure-openai`,
+`anthropic--*` → `aws-bedrock`, and `gemini-*` → `gcp-vertexai` (reserved; adapter
+not implemented yet). Then add its `id` to the per-machine extension overlay at
+`~/.pi/agent/pi-sap-aicore/models.json`:
 
 ```json
 {
-  "foundation": { "enabledModelIds": ["gpt-5.5"] }
+  "foundation": { "enabledModelIds": ["gpt-5.5", "anthropic--claude-4.8-opus"] }
 }
 ```
 
@@ -327,12 +332,13 @@ future, our `pickReasoning` probe is wired and ready in `stream.ts`.
   shape SAP may reject. Wire-up (likely `thinking_config.thinking_budget`)
   is a future TODO in `src/stream.ts:reasoningParams`.
 
-**Foundation route caveat:** on `sap-aicore-foundation/*` the direct Azure
-OpenAI SDK pins API version `2024-10-21`, which has no `reasoning_effort`
-field — so gpt-5\* reason at their **default** effort and pi's thinking-level
-cycle is a no-op there. The models still reason (reasoning tokens are billed
-and show in `output`); the depth just isn't tunable. Use the orchestration
-route (`sap-aicore/*`) when you need to set the effort level.
+**Foundation route caveat:** on `sap-aicore-foundation/*`, GPT/OpenAI models use
+the direct Azure OpenAI SDK pinned to API version `2024-10-21`, which has no
+`reasoning_effort` field — so gpt-5\* reason at their **default** effort and pi's
+thinking-level cycle is a no-op there. Anthropic/Claude models use SAP's AWS
+Bedrock `/converse` endpoint; model-default reasoning works, but explicit Claude
+thinking controls are not wired yet. Use the orchestration route if you need
+explicit effort control.
 
 To override budgets per model, edit `thinkingLevelMap` on the relevant entry in
 `~/.pi/agent/pi-sap-aicore/models.json`.
@@ -355,11 +361,10 @@ Resolved in this order:
 
 The value is passed via SAP's `OrchestrationClient(..., {resourceGroup})`
 constructor arg, which is the only supported channel — `AI-Resource-Group`
-as a request header is explicitly rejected by SAP's typings. The foundation
-provider applies the same resolved group via
-`AzureOpenAiChatClient({ modelName, resourceGroup })`; both a model's foundation
-deployment and the orchestration deployment must live in the resolved group for
-name-based resolution to find them.
+as a request header is explicitly rejected by SAP's orchestration typings. The
+foundation provider applies the same resolved group when resolving and invoking
+direct deployments; both a model's foundation deployment and the orchestration
+deployment must live in the resolved group for name-based resolution to find them.
 
 ## Prompt caching & cost reporting
 
@@ -386,11 +391,12 @@ currently don't.
 OpenAI/Gemini routes ignore the flag — they have their own automatic
 caching with no breakpoint API.
 
-**Foundation route:** because it talks to the Azure OpenAI endpoint directly
-(not through orchestration's usage-stripping), `prompt_tokens_details.cached_tokens`
-*may* come back populated — `mapUsage` reads it, so `cacheRead` could be non-zero
-on `sap-aicore-foundation/*` turns where orchestration always reports 0. Unverified
-against SAP's proxy; treat as best-effort.
+**Foundation route:** because direct foundation endpoints bypass orchestration's
+usage-stripping, provider-specific cache fields *may* come back populated.
+`mapUsage` reads OpenAI `prompt_tokens_details.cached_tokens` and Anthropic-style
+cache-read fields when SAP exposes them, so `cacheRead` could be non-zero on
+`sap-aicore-foundation/*` turns where orchestration always reports 0. Treat as
+best-effort and provider-dependent.
 
 ## Releasing (maintainers)
 
@@ -454,7 +460,12 @@ npmjs.com:
     ├── to-pi-model.ts           # SapModel → pi's ProviderModelConfig mapper
     ├── stream.ts                # orchestration streamSimple adapter + shared helpers (auth, usage, errors)
     ├── translate.ts             # pi Context ↔ orchestration message shape
-    ├── foundation-params.ts     # Azure OpenAI request params (max_completion_tokens, temperature gating)
-    ├── stream-foundation.ts     # foundation streamSimple adapter (AzureOpenAiChatClient, native streaming)
-    └── translate-foundation.ts  # pi Context ↔ Azure OpenAI message shape
+    ├── foundation-executables.ts         # model id → SAP foundation executable mapping
+    ├── foundation-deployment.ts          # shared foundation deployment resolution helpers
+    ├── foundation-params.ts              # Azure OpenAI request params (max_completion_tokens, temperature gating)
+    ├── stream-foundation.ts              # foundation dispatcher
+    ├── stream-foundation-azure-openai.ts # AzureOpenAiChatClient adapter with native streaming
+    ├── stream-foundation-bedrock.ts      # AWS Bedrock /converse adapter for Anthropic foundation deployments
+    ├── translate-foundation.ts           # pi Context ↔ Azure OpenAI message shape
+    └── translate-foundation-bedrock.ts   # pi Context ↔ Bedrock Converse message shape
 ```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-sap-aicore",
-  "version": "0.2.2",
+  "version": "0.3.0",
   "description": "SAP AI Core (orchestration + foundation) provider for the pi coding agent",
   "license": "MIT",
   "author": "Tim Pearson (https://github.com/ttiimmaahh)",

package/src/foundation-deployment.ts

@@ -0,0 +1,24 @@
+import { DeploymentApi } from "@sap-ai-sdk/ai-api";
+import { resolveDeploymentId } from "@sap-ai-sdk/ai-api/internal.js";
+
+import type { FoundationExecutable } from "./foundation-executables.ts";
+
+export async function resolveFoundationDeploymentId(options: {
+	modelId: string;
+	executableId: FoundationExecutable;
+	resourceGroup?: string;
+}): Promise<string> {
+	return resolveDeploymentId({
+		scenarioId: "foundation-models",
+		executableId: options.executableId,
+		model: { name: options.modelId },
+		resourceGroup: options.resourceGroup,
+	});
+}
+
+export async function listRunningFoundationDeployments(resourceGroup?: string) {
+	return DeploymentApi.deploymentQuery(
+		{ scenarioId: "foundation-models", status: "RUNNING", $top: 100 },
+		{ "AI-Resource-Group": resourceGroup ?? "default" },
+	).execute();
+}

package/src/foundation-executables.ts

@@ -0,0 +1,12 @@
+export type FoundationExecutable = "azure-openai" | "aws-bedrock" | "gcp-vertexai";
+
+export function foundationExecutableForModel(modelId: string): FoundationExecutable {
+	if (modelId.startsWith("anthropic--")) return "aws-bedrock";
+	if (modelId.startsWith("gemini-")) return "gcp-vertexai";
+	if (modelId.startsWith("gpt-")) return "azure-openai";
+
+	throw new Error(
+		`No SAP AI Core foundation executable mapping for model '${modelId}'. ` +
+			"Add a mapping in foundation-executables.ts or route this model through orchestration.",
+	);
+}

package/src/stream-foundation-azure-openai.ts

@@ -0,0 +1,363 @@
+import { randomUUID } from "node:crypto";
+
+import {
+	type Api,
+	type AssistantMessage,
+	type AssistantMessageEventStream,
+	calculateCost,
+	type Context,
+	createAssistantMessageEventStream,
+	type Model,
+	type SimpleStreamOptions,
+} from "@earendil-works/pi-ai";
+import type { AzureOpenAiChatCompletionParameters } from "@sap-ai-sdk/foundation-models";
+
+import { buildAzureOpenAiParams } from "./foundation-params.ts";
+import {
+	debugLog,
+	ensureServiceKey,
+	type ExtendedDelta,
+	formatError,
+	latchFinishReason,
+	mapUsage,
+	pickReasoning,
+	resolveResourceGroup,
+	type ToolCallSlot,
+} from "./stream.ts";
+import { mapFinishReason } from "./translate.ts";
+import { piContextToAzureOpenAi } from "./translate-foundation.ts";
+
+// Loaded dynamically (not at module load) so a missing dependency surfaces as
+// an actionable in-stream error instead of an ERR_MODULE_NOT_FOUND crash at pi
+// startup. Mirrors `importOrchestration` in stream.ts.
+async function importFoundation(): Promise<
+	typeof import("@sap-ai-sdk/foundation-models")
+> {
+	try {
+		return await import("@sap-ai-sdk/foundation-models");
+	} catch (err) {
+		const code = (err as NodeJS.ErrnoException)?.code;
+		const msg = (err as Error)?.message ?? "";
+		const isMissing =
+			code === "ERR_MODULE_NOT_FOUND" &&
+			msg.includes("@sap-ai-sdk/foundation-models");
+		if (!isMissing) throw err;
+
+		throw new Error(
+			"The SAP AI Core foundation-models SDK (@sap-ai-sdk/foundation-models) " +
+				"isn't installed, so the foundation provider can't make requests. Fix: " +
+				"run `npm install` in the pi-sap-aicore directory (where pi installed " +
+				"it, e.g. under ~/.pi/agent/), then restart pi.",
+		);
+	}
+}
+
+// Direct (foundation) provider: routes OpenAI models through their own
+// SAP AI Core deployment via @sap-ai-sdk/foundation-models'
+// AzureOpenAiChatClient — bypassing the orchestration service entirely.
+// Unlike streamSapAiCore there is NO streaming-unsupported fallback: the
+// direct Azure OpenAI endpoint streams natively (that's the whole reason this
+// path exists for new models orchestration won't stream). The SDK injects
+// `stream_options: { include_usage: true }` itself, so usage arrives on the
+// final chunk and `response.getTokenUsage()` is populated.
+export function streamSapFoundationAzureOpenAi(
+	model: Model<Api>,
+	context: Context,
+	options?: SimpleStreamOptions,
+): AssistantMessageEventStream {
+	const stream = createAssistantMessageEventStream();
+
+	const output: AssistantMessage = {
+		role: "assistant",
+		content: [],
+		api: model.api,
+		provider: model.provider,
+		model: model.id,
+		usage: {
+			input: 0,
+			output: 0,
+			cacheRead: 0,
+			cacheWrite: 0,
+			totalTokens: 0,
+			cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
+		},
+		stopReason: "stop",
+		timestamp: Date.now(),
+	};
+
+	(async () => {
+		const requestId = randomUUID();
+		try {
+			stream.push({ type: "start", partial: output });
+
+			const serviceKey = ensureServiceKey(options?.apiKey);
+			process.env.AICORE_SERVICE_KEY = serviceKey.raw;
+			const resourceGroup = resolveResourceGroup(serviceKey);
+
+			const { messages, tools } = piContextToAzureOpenAi(context);
+			const params = buildAzureOpenAiParams(model, options);
+
+			const { AzureOpenAiChatClient } = await importFoundation();
+
+			const request: AzureOpenAiChatCompletionParameters = {
+				messages,
+				...(tools.length > 0 ? { tools } : {}),
+				...params,
+			};
+
+			debugLog({
+				requestId,
+				kind: "request",
+				provider: "foundation-azure-openai",
+				model: model.id,
+				resourceGroup,
+				params,
+				messageRoles: messages.map((m) => m.role),
+				messages,
+			});
+
+			// Name-based deployment resolution: the SDK finds THE foundation
+			// deployment serving this model in the resource group. SAP allows
+			// only one deployment per (model, version, resource group), so the
+			// match is unambiguous — no deployment ID needed.
+			const client = new AzureOpenAiChatClient({
+				modelName: model.id,
+				...(resourceGroup ? { resourceGroup } : {}),
+			});
+
+			const response = await client.stream(request, options?.signal);
+
+			let textIndex = -1;
+			let thinkingIndex = -1;
+			let reasoningField: string | undefined;
+			let refusalText = "";
+			const toolSlots = new Map<number, ToolCallSlot>();
+			let finishReason: string | undefined;
+
+			const closeText = () => {
+				if (textIndex < 0) return;
+				const block = output.content[textIndex];
+				if (block?.type === "text") {
+					stream.push({
+						type: "text_end",
+						contentIndex: textIndex,
+						content: block.text,
+						partial: output,
+					});
+				}
+				textIndex = -1;
+			};
+
+			const closeThinking = () => {
+				if (thinkingIndex < 0) return;
+				const block = output.content[thinkingIndex];
+				if (block?.type === "thinking") {
+					stream.push({
+						type: "thinking_end",
+						contentIndex: thinkingIndex,
+						content: block.thinking,
+						partial: output,
+					});
+				}
+				thinkingIndex = -1;
+			};
+
+			for await (const chunk of response.stream) {
+				if (options?.signal?.aborted) break;
+
+				const choice = chunk.findChoiceByIndex(0);
+				const rawDelta = (choice?.delta ?? {}) as ExtendedDelta;
+
+				// Reasoning first — providers emit it before visible text, and
+				// pi's UI expects the thinking block to precede the text block.
+				// (gpt-5* on the direct route are unlikely to pass structured
+				// reasoning through, but we handle it for free if they do.)
+				const reasoning = pickReasoning(rawDelta, reasoningField);
+				if (reasoning) {
+					reasoningField = reasoning.field;
+					if (thinkingIndex < 0) {
+						closeText();
+						output.content.push({ type: "thinking", thinking: "" });
+						thinkingIndex = output.content.length - 1;
+						stream.push({
+							type: "thinking_start",
+							contentIndex: thinkingIndex,
+							partial: output,
+						});
+					}
+					const block = output.content[thinkingIndex];
+					if (block?.type === "thinking") {
+						block.thinking += reasoning.text;
+						stream.push({
+							type: "thinking_delta",
+							contentIndex: thinkingIndex,
+							delta: reasoning.text,
+							partial: output,
+						});
+					}
+				}
+
+				const delta = chunk.getDeltaContent();
+				if (delta) {
+					if (textIndex < 0) {
+						closeThinking();
+						output.content.push({ type: "text", text: "" });
+						textIndex = output.content.length - 1;
+						stream.push({
+							type: "text_start",
+							contentIndex: textIndex,
+							partial: output,
+						});
+					}
+					const block = output.content[textIndex];
+					if (block?.type === "text") {
+						block.text += delta;
+						stream.push({
+							type: "text_delta",
+							contentIndex: textIndex,
+							delta,
+							partial: output,
+						});
+					}
+				}
+
+				if (
+					typeof rawDelta.refusal === "string" &&
+					rawDelta.refusal.length > 0
+				) {
+					refusalText += rawDelta.refusal;
+				}
+
+				const toolDeltas = chunk.getDeltaToolCalls();
+				if (toolDeltas && toolDeltas.length > 0) {
+					closeText();
+					closeThinking();
+
+					for (const td of toolDeltas) {
+						let slot = toolSlots.get(td.index);
+						if (!slot) {
+							output.content.push({
+								type: "toolCall",
+								id: td.id ?? "",
+								name: td.function?.name ?? "",
+								arguments: {},
+							});
+							slot = {
+								contentIndex: output.content.length - 1,
+								partialJson: "",
+							};
+							toolSlots.set(td.index, slot);
+							stream.push({
+								type: "toolcall_start",
+								contentIndex: slot.contentIndex,
+								partial: output,
+							});
+						}
+
+						const block = output.content[slot.contentIndex];
+						if (block?.type === "toolCall") {
+							if (td.id && !block.id) block.id = td.id;
+							if (td.function?.name && !block.name)
+								block.name = td.function.name;
+
+							const fragment = td.function?.arguments ?? "";
+							if (fragment) {
+								slot.partialJson += fragment;
+								try {
+									block.arguments = JSON.parse(slot.partialJson);
+								} catch {
+									// Partial JSON — keep accumulating until valid
+								}
+								stream.push({
+									type: "toolcall_delta",
+									contentIndex: slot.contentIndex,
+									delta: fragment,
+									partial: output,
+								});
+							}
+						}
+					}
+				}
+
+				finishReason = latchFinishReason(
+					finishReason,
+					chunk.getFinishReason() ?? undefined,
+				);
+			}
+
+			closeText();
+			closeThinking();
+
+			for (const slot of toolSlots.values()) {
+				const block = output.content[slot.contentIndex];
+				if (block?.type === "toolCall") {
+					if (slot.partialJson) {
+						try {
+							block.arguments = JSON.parse(slot.partialJson);
+						} catch {
+							// Leave arguments as last successfully-parsed value
+						}
+					}
+					stream.push({
+						type: "toolcall_end",
+						contentIndex: slot.contentIndex,
+						toolCall: {
+							type: "toolCall",
+							id: block.id,
+							name: block.name,
+							arguments: block.arguments,
+						},
+						partial: output,
+					});
+				}
+			}
+
+			const usage = response.getTokenUsage();
+			if (usage) {
+				output.usage = mapUsage(usage);
+				calculateCost(model, output.usage);
+			}
+
+			// A refusal terminates the turn with no real content. Promote it to a
+			// visible error so pi doesn't render an empty assistant turn.
+			if (refusalText) {
+				output.stopReason = "error";
+				output.errorMessage = `Model refused: ${refusalText}`;
+				stream.push({ type: "error", reason: "error", error: output });
+				stream.end();
+				return;
+			}
+
+			output.stopReason = mapFinishReason(
+				toolSlots.size > 0
+					? "tool_calls"
+					: (finishReason ?? response.getFinishReason() ?? undefined),
+			);
+			stream.push({
+				type: "done",
+				reason: output.stopReason as "stop" | "length" | "toolUse",
+				message: output,
+			});
+			stream.end();
+		} catch (error) {
+			output.stopReason = options?.signal?.aborted ? "aborted" : "error";
+			output.errorMessage = formatError(error);
+			debugLog({
+				requestId,
+				kind: "error",
+				provider: "foundation-azure-openai",
+				model: model.id,
+				stopReason: output.stopReason,
+				error: output.errorMessage,
+			});
+			stream.push({
+				type: "error",
+				reason: output.stopReason as "error" | "aborted",
+				error: output,
+			});
+			stream.end();
+		}
+	})();
+
+	return stream;
+}

package/src/stream-foundation-bedrock.ts

@@ -0,0 +1,223 @@
+import { randomUUID } from "node:crypto";
+
+import {
+	type Api,
+	type AssistantMessage,
+	type AssistantMessageEventStream,
+	calculateCost,
+	type Context,
+	createAssistantMessageEventStream,
+	type Model,
+	type SimpleStreamOptions,
+} from "@earendil-works/pi-ai";
+import { executeRequest } from "@sap-ai-sdk/core";
+
+import { resolveFoundationDeploymentId } from "./foundation-deployment.ts";
+import {
+	debugLog,
+	ensureServiceKey,
+	formatError,
+	mapUsage,
+	resolveResourceGroup,
+} from "./stream.ts";
+import { mapFinishReason } from "./translate.ts";
+import { piContextToBedrockConverse } from "./translate-foundation-bedrock.ts";
+
+type BedrockConverseResponse = {
+	output?: {
+		message?: {
+			role?: string;
+			content?: Array<{
+				text?: string;
+				toolUse?: {
+					toolUseId?: string;
+					name?: string;
+					input?: unknown;
+				};
+			}>;
+		};
+	};
+	stopReason?: string;
+	usage?: {
+		inputTokens?: number;
+		outputTokens?: number;
+		totalTokens?: number;
+		cacheReadInputTokens?: number;
+		cacheReadInputTokenCount?: number;
+		cacheWriteInputTokens?: number;
+		cacheWriteInputTokenCount?: number;
+	};
+};
+
+export function streamSapFoundationBedrock(
+	model: Model<Api>,
+	context: Context,
+	options?: SimpleStreamOptions,
+): AssistantMessageEventStream {
+	const stream = createAssistantMessageEventStream();
+
+	const output: AssistantMessage = {
+		role: "assistant",
+		content: [],
+		api: model.api,
+		provider: model.provider,
+		model: model.id,
+		usage: {
+			input: 0,
+			output: 0,
+			cacheRead: 0,
+			cacheWrite: 0,
+			totalTokens: 0,
+			cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
+		},
+		stopReason: "stop",
+		timestamp: Date.now(),
+	};
+
+	(async () => {
+		const requestId = randomUUID();
+		try {
+			stream.push({ type: "start", partial: output });
+
+			const serviceKey = ensureServiceKey(options?.apiKey);
+			process.env.AICORE_SERVICE_KEY = serviceKey.raw;
+			const resourceGroup = resolveResourceGroup(serviceKey);
+			const deploymentId = await resolveFoundationDeploymentId({
+				modelId: model.id,
+				executableId: "aws-bedrock",
+				resourceGroup,
+			});
+
+			const translated = piContextToBedrockConverse(context);
+			const maxTokens = options?.maxTokens ?? model.maxTokens;
+			const request = {
+				...translated,
+				inferenceConfig: {
+					maxTokens,
+					...(options?.temperature !== undefined
+						? { temperature: options.temperature }
+						: {}),
+				},
+			};
+
+			debugLog({
+				requestId,
+				kind: "request",
+				provider: "foundation-aws-bedrock",
+				model: model.id,
+				resourceGroup,
+				deploymentId,
+				params: request.inferenceConfig,
+				messageRoles: request.messages.map((m) => m.role),
+				messages: request.messages,
+			});
+
+			const response = await executeRequest(
+				{
+					url: `/inference/deployments/${deploymentId}/converse`,
+					resourceGroup,
+				},
+				request,
+				{ signal: options?.signal },
+			);
+
+			const data = response.data as BedrockConverseResponse;
+			replayBedrockConverseResponse(stream, output, data);
+
+			if (data.usage) {
+				output.usage = mapUsage({
+					prompt_tokens: data.usage.inputTokens ?? 0,
+					completion_tokens: data.usage.outputTokens ?? 0,
+					cache_read_input_tokens:
+						data.usage.cacheReadInputTokens ??
+						data.usage.cacheReadInputTokenCount ??
+						0,
+					cache_creation_input_tokens:
+						data.usage.cacheWriteInputTokens ??
+						data.usage.cacheWriteInputTokenCount ??
+						0,
+				});
+				calculateCost(model, output.usage);
+			}
+
+			output.stopReason = mapFinishReason(
+				data.stopReason === "max_tokens" ? "length" : data.stopReason,
+			);
+			stream.push({
+				type: "done",
+				reason: output.stopReason as "stop" | "length" | "toolUse",
+				message: output,
+			});
+			stream.end();
+		} catch (error) {
+			output.stopReason = options?.signal?.aborted ? "aborted" : "error";
+			output.errorMessage = formatError(error);
+			debugLog({
+				requestId,
+				kind: "error",
+				provider: "foundation-aws-bedrock",
+				model: model.id,
+				stopReason: output.stopReason,
+				error: output.errorMessage,
+			});
+			stream.push({
+				type: "error",
+				reason: output.stopReason as "error" | "aborted",
+				error: output,
+			});
+			stream.end();
+		}
+	})();
+
+	return stream;
+}
+
+function replayBedrockConverseResponse(
+	stream: AssistantMessageEventStream,
+	output: AssistantMessage,
+	data: BedrockConverseResponse,
+): void {
+	const blocks = data.output?.message?.content ?? [];
+	for (const block of blocks) {
+		if (typeof block.text === "string" && block.text.length > 0) {
+			const contentIndex = output.content.length;
+			output.content.push({ type: "text", text: "" });
+			stream.push({ type: "text_start", contentIndex, partial: output });
+			const outBlock = output.content[contentIndex];
+			if (outBlock?.type === "text") outBlock.text = block.text;
+			stream.push({
+				type: "text_delta",
+				contentIndex,
+				delta: block.text,
+				partial: output,
+			});
+			stream.push({
+				type: "text_end",
+				contentIndex,
+				content: block.text,
+				partial: output,
+			});
+		}
+
+		if (block.toolUse) {
+			const contentIndex = output.content.length;
+			const toolCall = {
+				type: "toolCall" as const,
+				id: block.toolUse.toolUseId ?? randomUUID(),
+				name: block.toolUse.name ?? "",
+				arguments:
+					block.toolUse.input && typeof block.toolUse.input === "object"
+						? (block.toolUse.input as Record<string, unknown>)
+						: {},
+			};
+			output.content.push(toolCall);
+			stream.push({ type: "toolcall_start", contentIndex, partial: output });
+			stream.push({
+				type: "toolcall_end",
+				contentIndex,
+				toolCall,
+				partial: output,
+			});
+		}
+	}
+}

package/src/stream-foundation.ts

@@ -1,363 +1,31 @@
-import { randomUUID } from "node:crypto";
-
-import {
-	type Api,
-	type AssistantMessage,
-	type AssistantMessageEventStream,
-	calculateCost,
-	type Context,
-	createAssistantMessageEventStream,
-	type Model,
-	type SimpleStreamOptions,
+import type {
+	Api,
+	AssistantMessageEventStream,
+	Context,
+	Model,
+	SimpleStreamOptions,
 } from "@earendil-works/pi-ai";
-import type { AzureOpenAiChatCompletionParameters } from "@sap-ai-sdk/foundation-models";
-
-import { buildAzureOpenAiParams } from "./foundation-params.ts";
-import {
-	debugLog,
-	ensureServiceKey,
-	type ExtendedDelta,
-	formatError,
-	latchFinishReason,
-	mapUsage,
-	pickReasoning,
-	resolveResourceGroup,
-	type ToolCallSlot,
-} from "./stream.ts";
-import { mapFinishReason } from "./translate.ts";
-import { piContextToAzureOpenAi } from "./translate-foundation.ts";
-
-// Loaded dynamically (not at module load) so a missing dependency surfaces as
-// an actionable in-stream error instead of an ERR_MODULE_NOT_FOUND crash at pi
-// startup. Mirrors `importOrchestration` in stream.ts.
-async function importFoundation(): Promise<
-	typeof import("@sap-ai-sdk/foundation-models")
-> {
-	try {
-		return await import("@sap-ai-sdk/foundation-models");
-	} catch (err) {
-		const code = (err as NodeJS.ErrnoException)?.code;
-		const msg = (err as Error)?.message ?? "";
-		const isMissing =
-			code === "ERR_MODULE_NOT_FOUND" &&
-			msg.includes("@sap-ai-sdk/foundation-models");
-		if (!isMissing) throw err;
 
-		throw new Error(
-			"The SAP AI Core foundation-models SDK (@sap-ai-sdk/foundation-models) " +
-				"isn't installed, so the foundation provider can't make requests. Fix: " +
-				"run `npm install` in the pi-sap-aicore directory (where pi installed " +
-				"it, e.g. under ~/.pi/agent/), then restart pi.",
-		);
-	}
-}
+import { foundationExecutableForModel } from "./foundation-executables.ts";
+import { streamSapFoundationAzureOpenAi } from "./stream-foundation-azure-openai.ts";
+import { streamSapFoundationBedrock } from "./stream-foundation-bedrock.ts";
 
-// Direct (foundation) provider: routes OpenAI models through their own
-// SAP AI Core deployment via @sap-ai-sdk/foundation-models'
-// AzureOpenAiChatClient — bypassing the orchestration service entirely.
-// Unlike streamSapAiCore there is NO streaming-unsupported fallback: the
-// direct Azure OpenAI endpoint streams natively (that's the whole reason this
-// path exists for new models orchestration won't stream). The SDK injects
-// `stream_options: { include_usage: true }` itself, so usage arrives on the
-// final chunk and `response.getTokenUsage()` is populated.
 export function streamSapFoundation(
 	model: Model<Api>,
 	context: Context,
 	options?: SimpleStreamOptions,
 ): AssistantMessageEventStream {
-	const stream = createAssistantMessageEventStream();
-
-	const output: AssistantMessage = {
-		role: "assistant",
-		content: [],
-		api: model.api,
-		provider: model.provider,
-		model: model.id,
-		usage: {
-			input: 0,
-			output: 0,
-			cacheRead: 0,
-			cacheWrite: 0,
-			totalTokens: 0,
-			cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
-		},
-		stopReason: "stop",
-		timestamp: Date.now(),
-	};
-
-	(async () => {
-		const requestId = randomUUID();
-		try {
-			stream.push({ type: "start", partial: output });
-
-			const serviceKey = ensureServiceKey(options?.apiKey);
-			process.env.AICORE_SERVICE_KEY = serviceKey.raw;
-			const resourceGroup = resolveResourceGroup(serviceKey);
-
-			const { messages, tools } = piContextToAzureOpenAi(context);
-			const params = buildAzureOpenAiParams(model, options);
-
-			const { AzureOpenAiChatClient } = await importFoundation();
-
-			const request: AzureOpenAiChatCompletionParameters = {
-				messages,
-				...(tools.length > 0 ? { tools } : {}),
-				...params,
-			};
-
-			debugLog({
-				requestId,
-				kind: "request",
-				provider: "foundation",
-				model: model.id,
-				resourceGroup,
-				params,
-				messageRoles: messages.map((m) => m.role),
-				messages,
-			});
-
-			// Name-based deployment resolution: the SDK finds THE foundation
-			// deployment serving this model in the resource group. SAP allows
-			// only one deployment per (model, version, resource group), so the
-			// match is unambiguous — no deployment ID needed.
-			const client = new AzureOpenAiChatClient({
-				modelName: model.id,
-				...(resourceGroup ? { resourceGroup } : {}),
-			});
-
-			const response = await client.stream(request, options?.signal);
-
-			let textIndex = -1;
-			let thinkingIndex = -1;
-			let reasoningField: string | undefined;
-			let refusalText = "";
-			const toolSlots = new Map<number, ToolCallSlot>();
-			let finishReason: string | undefined;
-
-			const closeText = () => {
-				if (textIndex < 0) return;
-				const block = output.content[textIndex];
-				if (block?.type === "text") {
-					stream.push({
-						type: "text_end",
-						contentIndex: textIndex,
-						content: block.text,
-						partial: output,
-					});
-				}
-				textIndex = -1;
-			};
-
-			const closeThinking = () => {
-				if (thinkingIndex < 0) return;
-				const block = output.content[thinkingIndex];
-				if (block?.type === "thinking") {
-					stream.push({
-						type: "thinking_end",
-						contentIndex: thinkingIndex,
-						content: block.thinking,
-						partial: output,
-					});
-				}
-				thinkingIndex = -1;
-			};
-
-			for await (const chunk of response.stream) {
-				if (options?.signal?.aborted) break;
-
-				const choice = chunk.findChoiceByIndex(0);
-				const rawDelta = (choice?.delta ?? {}) as ExtendedDelta;
-
-				// Reasoning first — providers emit it before visible text, and
-				// pi's UI expects the thinking block to precede the text block.
-				// (gpt-5* on the direct route are unlikely to pass structured
-				// reasoning through, but we handle it for free if they do.)
-				const reasoning = pickReasoning(rawDelta, reasoningField);
-				if (reasoning) {
-					reasoningField = reasoning.field;
-					if (thinkingIndex < 0) {
-						closeText();
-						output.content.push({ type: "thinking", thinking: "" });
-						thinkingIndex = output.content.length - 1;
-						stream.push({
-							type: "thinking_start",
-							contentIndex: thinkingIndex,
-							partial: output,
-						});
-					}
-					const block = output.content[thinkingIndex];
-					if (block?.type === "thinking") {
-						block.thinking += reasoning.text;
-						stream.push({
-							type: "thinking_delta",
-							contentIndex: thinkingIndex,
-							delta: reasoning.text,
-							partial: output,
-						});
-					}
-				}
-
-				const delta = chunk.getDeltaContent();
-				if (delta) {
-					if (textIndex < 0) {
-						closeThinking();
-						output.content.push({ type: "text", text: "" });
-						textIndex = output.content.length - 1;
-						stream.push({
-							type: "text_start",
-							contentIndex: textIndex,
-							partial: output,
-						});
-					}
-					const block = output.content[textIndex];
-					if (block?.type === "text") {
-						block.text += delta;
-						stream.push({
-							type: "text_delta",
-							contentIndex: textIndex,
-							delta,
-							partial: output,
-						});
-					}
-				}
-
-				if (
-					typeof rawDelta.refusal === "string" &&
-					rawDelta.refusal.length > 0
-				) {
-					refusalText += rawDelta.refusal;
-				}
-
-				const toolDeltas = chunk.getDeltaToolCalls();
-				if (toolDeltas && toolDeltas.length > 0) {
-					closeText();
-					closeThinking();
-
-					for (const td of toolDeltas) {
-						let slot = toolSlots.get(td.index);
-						if (!slot) {
-							output.content.push({
-								type: "toolCall",
-								id: td.id ?? "",
-								name: td.function?.name ?? "",
-								arguments: {},
-							});
-							slot = {
-								contentIndex: output.content.length - 1,
-								partialJson: "",
-							};
-							toolSlots.set(td.index, slot);
-							stream.push({
-								type: "toolcall_start",
-								contentIndex: slot.contentIndex,
-								partial: output,
-							});
-						}
-
-						const block = output.content[slot.contentIndex];
-						if (block?.type === "toolCall") {
-							if (td.id && !block.id) block.id = td.id;
-							if (td.function?.name && !block.name)
-								block.name = td.function.name;
-
-							const fragment = td.function?.arguments ?? "";
-							if (fragment) {
-								slot.partialJson += fragment;
-								try {
-									block.arguments = JSON.parse(slot.partialJson);
-								} catch {
-									// Partial JSON — keep accumulating until valid
-								}
-								stream.push({
-									type: "toolcall_delta",
-									contentIndex: slot.contentIndex,
-									delta: fragment,
-									partial: output,
-								});
-							}
-						}
-					}
-				}
-
-				finishReason = latchFinishReason(
-					finishReason,
-					chunk.getFinishReason() ?? undefined,
-				);
-			}
-
-			closeText();
-			closeThinking();
-
-			for (const slot of toolSlots.values()) {
-				const block = output.content[slot.contentIndex];
-				if (block?.type === "toolCall") {
-					if (slot.partialJson) {
-						try {
-							block.arguments = JSON.parse(slot.partialJson);
-						} catch {
-							// Leave arguments as last successfully-parsed value
-						}
-					}
-					stream.push({
-						type: "toolcall_end",
-						contentIndex: slot.contentIndex,
-						toolCall: {
-							type: "toolCall",
-							id: block.id,
-							name: block.name,
-							arguments: block.arguments,
-						},
-						partial: output,
-					});
-				}
-			}
-
-			const usage = response.getTokenUsage();
-			if (usage) {
-				output.usage = mapUsage(usage);
-				calculateCost(model, output.usage);
-			}
-
-			// A refusal terminates the turn with no real content. Promote it to a
-			// visible error so pi doesn't render an empty assistant turn.
-			if (refusalText) {
-				output.stopReason = "error";
-				output.errorMessage = `Model refused: ${refusalText}`;
-				stream.push({ type: "error", reason: "error", error: output });
-				stream.end();
-				return;
-			}
-
-			output.stopReason = mapFinishReason(
-				toolSlots.size > 0
-					? "tool_calls"
-					: (finishReason ?? response.getFinishReason() ?? undefined),
+	const executable = foundationExecutableForModel(model.id);
+	switch (executable) {
+		case "azure-openai":
+			return streamSapFoundationAzureOpenAi(model, context, options);
+		case "aws-bedrock":
+			return streamSapFoundationBedrock(model, context, options);
+		case "gcp-vertexai":
+			throw new Error(
+				`SAP AI Core foundation executable '${executable}' is mapped for '${model.id}', ` +
+					"but the Vertex AI/Gemini foundation adapter has not been implemented yet. " +
+					"Use sap-aicore orchestration for this model until the gcp-vertexai adapter is added.",
 			);
-			stream.push({
-				type: "done",
-				reason: output.stopReason as "stop" | "length" | "toolUse",
-				message: output,
-			});
-			stream.end();
-		} catch (error) {
-			output.stopReason = options?.signal?.aborted ? "aborted" : "error";
-			output.errorMessage = formatError(error);
-			debugLog({
-				requestId,
-				kind: "error",
-				provider: "foundation",
-				model: model.id,
-				stopReason: output.stopReason,
-				error: output.errorMessage,
-			});
-			stream.push({
-				type: "error",
-				reason: output.stopReason as "error" | "aborted",
-				error: output,
-			});
-			stream.end();
-		}
-	})();
-
-	return stream;
+	}
 }

package/src/translate-foundation-bedrock.ts

@@ -0,0 +1,175 @@
+import type {
+	AssistantMessage,
+	Context,
+	Message,
+	TextContent,
+	ToolResultMessage,
+	UserMessage,
+} from "@earendil-works/pi-ai";
+
+export type BedrockConverseContentBlock =
+	| { text: string }
+	| {
+			image: {
+				format: string;
+				source: { bytes: string };
+			};
+	  }
+	| {
+			toolUse: {
+				toolUseId: string;
+				name: string;
+				input: unknown;
+			};
+	  }
+	| {
+			toolResult: {
+				toolUseId: string;
+				content: Array<{ text: string }>;
+				status?: "success" | "error";
+			};
+	  };
+
+export type BedrockConverseMessage = {
+	role: "user" | "assistant";
+	content: BedrockConverseContentBlock[];
+};
+
+export function piContextToBedrockConverse(context: Context): {
+	system?: Array<{ text: string }>;
+	messages: BedrockConverseMessage[];
+} {
+	const messages: BedrockConverseMessage[] = [];
+
+	for (const msg of context.messages) {
+		const translated = piMessageToBedrockConverse(msg);
+		if (translated) messages.push(translated);
+	}
+
+	return {
+		...(context.systemPrompt ? { system: [{ text: context.systemPrompt }] } : {}),
+		messages: coalesceAdjacentMessages(messages),
+	};
+}
+
+function piMessageToBedrockConverse(
+	msg: Message,
+): BedrockConverseMessage | undefined {
+	switch (msg.role) {
+		case "user":
+			return piUserToBedrockConverse(msg);
+		case "assistant":
+			return piAssistantToBedrockConverse(msg);
+		case "toolResult":
+			return piToolResultToBedrockConverse(msg);
+	}
+}
+
+function piUserToBedrockConverse(msg: UserMessage): BedrockConverseMessage {
+	if (typeof msg.content === "string") {
+		return { role: "user", content: [{ text: msg.content }] };
+	}
+
+	const content = msg.content.map((part): BedrockConverseContentBlock => {
+		if (part.type === "text") return { text: part.text };
+		return {
+			image: {
+				format: imageFormatFromMimeType(part.mimeType),
+				source: { bytes: part.data },
+			},
+		};
+	});
+
+	return { role: "user", content: content.length > 0 ? content : [{ text: " " }] };
+}
+
+function piAssistantToBedrockConverse(
+	msg: AssistantMessage,
+): BedrockConverseMessage {
+	const content: BedrockConverseContentBlock[] = [];
+
+	for (const block of msg.content) {
+		if (block.type === "text" && block.text) {
+			content.push({ text: block.text });
+		} else if (block.type === "toolCall") {
+			content.push({
+				toolUse: {
+					toolUseId: block.id,
+					name: block.name,
+					input: block.arguments,
+				},
+			});
+		}
+	}
+
+	return {
+		role: "assistant",
+		content: content.length > 0 ? content : [{ text: " " }],
+	};
+}
+
+function piToolResultToBedrockConverse(
+	msg: ToolResultMessage,
+): BedrockConverseMessage {
+	const text = toolResultText(msg) || " ";
+	return {
+		role: "user",
+		content: [
+			{
+				toolResult: {
+					toolUseId: msg.toolCallId,
+					content: [{ text }],
+					status: msg.isError ? "error" : "success",
+				},
+			},
+			...toolResultImagesAsUserContent(msg),
+		],
+	};
+}
+
+function toolResultText(msg: ToolResultMessage): string {
+	return msg.content
+		.filter((part): part is TextContent => part.type === "text")
+		.map((part) => part.text)
+		.join("\n");
+}
+
+function toolResultImagesAsUserContent(
+	msg: ToolResultMessage,
+): BedrockConverseContentBlock[] {
+	return msg.content
+		.filter(
+			(part): part is { type: "image"; data: string; mimeType: string } =>
+				part.type === "image",
+		)
+		.map((part) => ({
+			image: {
+				format: imageFormatFromMimeType(part.mimeType),
+				source: { bytes: part.data },
+			},
+		}));
+}
+
+function imageFormatFromMimeType(mimeType: string): string {
+	const format = mimeType.split("/")[1]?.toLowerCase();
+	if (format === "jpg") return "jpeg";
+	if (format === "jpeg" || format === "png" || format === "gif" || format === "webp") {
+		return format;
+	}
+	return "png";
+}
+
+function coalesceAdjacentMessages(
+	messages: BedrockConverseMessage[],
+): BedrockConverseMessage[] {
+	const result: BedrockConverseMessage[] = [];
+	for (const msg of messages) {
+		const previous = result[result.length - 1];
+		if (previous && previous.role === msg.role) {
+			previous.content.push(...msg.content);
+		} else {
+			result.push({ role: msg.role, content: [...msg.content] });
+		}
+	}
+	return result;
+}