AbstractRuntime 0.4.3__tar.gz → 0.4.5__tar.gz

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/.github/workflows/ci.yml

@@ -39,6 +39,8 @@ jobs:
             tests/test_tool_approval_resume_executes.py \
             tests/test_remote_llm_client.py \
             tests/test_prompt_cache_modules.py \
+            tests/test_multimodal_abstractcore_integration.py \
+            tests/test_visualflow_llm_call_multimodal_output.py \
             tests/test_visual_multi_entry_loop_overrides.py \
             tests/test_control_adapter_while.py \
             tests/test_jsonl_ledger_recovery.py \

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/.github/workflows/release.yml

@@ -51,6 +51,8 @@ jobs:
             tests/test_tool_approval_resume_executes.py \
             tests/test_remote_llm_client.py \
             tests/test_prompt_cache_modules.py \
+            tests/test_multimodal_abstractcore_integration.py \
+            tests/test_visualflow_llm_call_multimodal_output.py \
             tests/test_visual_multi_entry_loop_overrides.py \
             tests/test_control_adapter_while.py \
             tests/test_jsonl_ledger_recovery.py \

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/CHANGELOG.md

@@ -7,9 +7,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.4.5] - 2026-05-07
+
+### Changed
+- Minimum `abstractcore` optional dependency increased to `>=2.13.9` so Runtime can use AbstractCore's public output-selector contract instead of mirroring provider-private multimodal selector logic.
+- Runtime's AbstractCore output-spec adapter now delegates selector detection, normalization, generated-media detection, non-chat dispatch detection, and runtime metadata stripping to `abstractcore.core.output_specs`.
+
+### Fixed
+- Explicit `voice_clone` output specs no longer require a Runtime `ArtifactStore` before dispatch because AbstractCore exposes them as generated resources rather than binary media outputs.
+
+## [0.4.4] - 2026-05-07
+
 ### Added
+- **AbstractCore multimodal generation integration**:
+  - `LLM_CALL` now forwards AbstractCore's unified `generate(..., output=...)` selector for image generation, TTS/voice output, and audio transcription
+  - generated binary outputs are normalized into JSON-safe runtime results with ArtifactStore-backed refs instead of inline bytes
+  - local runtimes can use AbstractCore capability plugins such as AbstractVision and AbstractVoice through the same runtime effect shape
+  - remote runtimes support AbstractCore Server image generation, speech, and transcription endpoints, plus OpenAI-compatible chat media content arrays
+- **Multimodal packaging extra**:
+  - new `abstractruntime[multimodal]` extra installs `abstractcore[media,openai,vision,voice,audio]>=2.13.8`
+- **VisualFlow LLM media selectors**:
+  - LLM nodes lowered from VisualFlow can request generated media through `output` / `outputs` from node config or input data
+
 ### Changed
+- Minimum `abstractcore` optional dependency increased to `>=2.13.8` for the unified multimodal response types.
+- `LLM_CALL` accepts top-level `text`, top-level `output`, and top-level `outputs` as a runtime alias for AbstractCore `output`.
+- `LLM_CALL.media` accepts one media item or a list; artifact refs are materialized to provider-ready temporary files before model calls.
+- Remote AbstractCore clients now preserve existing OpenAI-style content arrays when adding media attachments.
+- Remote AbstractCore clients now resolve ArtifactStore-backed media refs for direct client use, matching the runtime effect-handler path.
+- VisualFlow LLM pending-call lowering now carries `output` / `outputs` selectors into runtime LLM effects.
+- VisualFlow LLM result syncing now projects generated media artifacts into node outputs such as `outputs`, `resources`, `artifact_ref`, `artifact_id`, and `meta.output_mode`.
+
 ### Fixed
+- Runtime artifact metadata (`run_id`, tags, artifact ids) is kept out of AbstractCore provider/capability kwargs while still being applied to stored generated media artifacts.
+- Generated binary media now fails closed without an ArtifactStore instead of embedding base64 bytes in durable state.
+- Remote image/TTS/STT calls no longer reuse the chat model unless an output-specific media model is supplied.
+- Remote media inputs now either convert to a provider-ready content item or fail before dispatch; unsupported remote image edits, voice reference inputs, and non-file STT inputs are rejected explicitly.
+- Turn-grounding injection now preserves structured multimodal message content arrays instead of stringifying them.
+- Session-scoped prompt-cache key derivation now uses the effective AbstractCore client provider/model identity when an `LLM_CALL` payload omits explicit provider/model overrides.
+
+### Documentation
+- Documented multimodal `LLM_CALL` payloads, artifact-backed response shape, remote endpoint coverage, cached-session/prompt-cache boundaries, and the `abstractruntime[multimodal]` extra in the AbstractCore integration guide, API reference, architecture guide, FAQ, README, getting-started guide, docs index, and AI-ready `llms*.txt` files.
+- Added a planned workspace/media access policy item covering default workspace-only access, explicit user allow/deny paths, and a conscious full-machine access mode for long-running agency deployments.
+
+### Testing
+- Added focused coverage for multimodal response normalization, artifact-backed generated media, media-only transcription calls, remote image/TTS/STT endpoints, remote media guardrails, remote chat media content arrays, content-array prompt extraction, direct remote artifact-ref media resolution, text-alias routing, provider-request redaction, and runtime metadata/tag boundaries.
+- Added coverage for effective prompt-cache key identity and VisualFlow LLM media selector/result projection.
 
 ## [0.4.3] - 2026-05-06
 
@@ -19,7 +62,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   - local clients can maintain compartmentalized `system | tools | history` prompt-cache modules when providers support `local_control_plane`
   - remote clients proxy `/acore/prompt_cache/*` endpoints for gateway/CLI hosts
 - **Artifact-backed media for AbstractCore LLM calls**:
-  - local/hybrid AbstractCore clients can resolve runtime artifact refs into provider-ready media inputs
+  - local and remote AbstractCore clients can resolve runtime artifact refs into provider-ready media inputs
   - AbstractCore runtime factories now pass the runtime artifact store into LLM clients
 - **Durable tool approval execution**:
   - `ToolApprovalPolicy` and `ApprovalToolExecutor` support safe auto-approval, durable approval waits, and approved re-execution
@@ -289,7 +332,9 @@ AbstractRuntime is the durable execution substrate designed to pair with Abstrac
 
 Initial development version with basic proof-of-concept features.
 
-[Unreleased]: https://github.com/lpalbou/abstractruntime/compare/v0.4.3...HEAD
+[Unreleased]: https://github.com/lpalbou/abstractruntime/compare/v0.4.5...HEAD
+[0.4.5]: https://github.com/lpalbou/abstractruntime/compare/v0.4.4...v0.4.5
+[0.4.4]: https://github.com/lpalbou/abstractruntime/releases/tag/v0.4.4
 [0.4.3]: https://github.com/lpalbou/abstractruntime/releases/tag/v0.4.3
 [0.4.2]: https://github.com/lpalbou/abstractruntime/releases/tag/v0.4.2
 [0.4.1]: https://github.com/lpalbou/abstractruntime/releases/tag/v0.4.1

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: AbstractRuntime
-Version: 0.4.3
+Version: 0.4.5
 Summary: AbstractRuntime: a durable graph runner designed to pair with AbstractCore.
 Project-URL: AbstractCore (website), https://www.abstractcore.ai/
 Project-URL: AbstractRuntime (GitHub), https://github.com/lpalbou/abstractruntime
@@ -22,13 +22,15 @@ Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.10
 Requires-Dist: abstractsemantics>=0.0.2
 Provides-Extra: abstractcore
-Requires-Dist: abstractcore>=2.13.5; extra == 'abstractcore'
+Requires-Dist: abstractcore>=2.13.9; extra == 'abstractcore'
 Provides-Extra: docs
 Requires-Dist: mkdocs-material>=9.0.0; extra == 'docs'
 Requires-Dist: mkdocs>=1.6.0; extra == 'docs'
 Requires-Dist: pymdown-extensions>=10.0; extra == 'docs'
 Provides-Extra: mcp-worker
-Requires-Dist: abstractcore[tools]>=2.13.5; extra == 'mcp-worker'
+Requires-Dist: abstractcore[tools]>=2.13.9; extra == 'mcp-worker'
+Provides-Extra: multimodal
+Requires-Dist: abstractcore[audio,media,openai,vision,voice]>=2.13.9; extra == 'multimodal'
 Provides-Extra: test
 Requires-Dist: pytest>=7.0.0; extra == 'test'
 Description-Content-Type: text/markdown
@@ -39,7 +41,7 @@ Description-Content-Type: text/markdown
 
 It is designed for long-running workflows that must survive restarts and explicitly model blocking (human input, timers, external events, subworkflows) without keeping Python stacks alive.
 
-**Version:** 0.4.3 (see `pyproject.toml`) • **Python:** 3.10+
+**Version:** 0.4.5 • **Python:** 3.10+
 
 **Status:** pre-1.0 (API may evolve). For production use, pin versions and follow `CHANGELOG.md`.
 
@@ -73,7 +75,7 @@ AbstractCore integration (LLM + tools):
 pip install "abstractruntime[abstractcore]"
 ```
 
-The `abstractcore` extra installs AbstractCore 2.13.5 or newer so the hardened server auth model, provider-key header routing, prompt-cache control plane, and current tool catalog are available.
+The `abstractcore` extra installs AbstractCore 2.13.9 or newer so the hardened server auth model, provider-key header routing, prompt-cache control plane, current tool catalog, unified multimodal generation response types, and public output-selector contract are available. Use `abstractruntime[multimodal]` when you need common media, vision, voice, and audio dependencies.
 
 MCP worker entrypoint (default toolsets over stdio):
 
@@ -122,7 +124,7 @@ state = rt.resume(
 assert state.status.value == "completed"
 ```
 
-## What’s included (v0.4.3)
+## What’s included (v0.4.5)
 
 Kernel (dependency-light):
 - workflow graphs: `WorkflowSpec` (`src/abstractruntime/core/spec.py`)
@@ -146,7 +148,7 @@ Drivers + distribution:
 - run history export: `export_run_history_bundle(...)` (`src/abstractruntime/history_bundle.py`)
 
 Optional integrations:
-- AbstractCore (LLM + tools, prompt cache, tool approval waits): `docs/integrations/abstractcore.md`
+- AbstractCore (LLM + tools, cached sessions/prompt cache, media inputs, generated media, tool approval waits): `docs/integrations/abstractcore.md`
 - comms toolset gating (email/WhatsApp/Telegram): `docs/tools-comms.md`
 
 ## Built-in scheduler (zero-config)

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/README.md

@@ -4,7 +4,7 @@
 
 It is designed for long-running workflows that must survive restarts and explicitly model blocking (human input, timers, external events, subworkflows) without keeping Python stacks alive.
 
-**Version:** 0.4.3 (see `pyproject.toml`) • **Python:** 3.10+
+**Version:** 0.4.5 • **Python:** 3.10+
 
 **Status:** pre-1.0 (API may evolve). For production use, pin versions and follow `CHANGELOG.md`.
 
@@ -38,7 +38,7 @@ AbstractCore integration (LLM + tools):
 pip install "abstractruntime[abstractcore]"
 ```
 
-The `abstractcore` extra installs AbstractCore 2.13.5 or newer so the hardened server auth model, provider-key header routing, prompt-cache control plane, and current tool catalog are available.
+The `abstractcore` extra installs AbstractCore 2.13.9 or newer so the hardened server auth model, provider-key header routing, prompt-cache control plane, current tool catalog, unified multimodal generation response types, and public output-selector contract are available. Use `abstractruntime[multimodal]` when you need common media, vision, voice, and audio dependencies.
 
 MCP worker entrypoint (default toolsets over stdio):
 
@@ -87,7 +87,7 @@ state = rt.resume(
 assert state.status.value == "completed"
 ```
 
-## What’s included (v0.4.3)
+## What’s included (v0.4.5)
 
 Kernel (dependency-light):
 - workflow graphs: `WorkflowSpec` (`src/abstractruntime/core/spec.py`)
@@ -111,7 +111,7 @@ Drivers + distribution:
 - run history export: `export_run_history_bundle(...)` (`src/abstractruntime/history_bundle.py`)
 
 Optional integrations:
-- AbstractCore (LLM + tools, prompt cache, tool approval waits): `docs/integrations/abstractcore.md`
+- AbstractCore (LLM + tools, cached sessions/prompt cache, media inputs, generated media, tool approval waits): `docs/integrations/abstractcore.md`
 - comms toolset gating (email/WhatsApp/Telegram): `docs/tools-comms.md`
 
 ## Built-in scheduler (zero-config)

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/README.md

@@ -24,7 +24,7 @@ In this repo, the AbstractCore wiring lives under `src/abstractruntime/integrati
 
 - `faq.md` — common questions (recommended)
 - `manual_testing.md` — manual smoke tests and how to run the test suite
-- `integrations/abstractcore.md` — wiring `LLM_CALL` / `TOOL_CALLS`, prompt-cache control, and tool approval waits via AbstractCore
+- `integrations/abstractcore.md` — wiring `LLM_CALL` / `TOOL_CALLS`, cached sessions/prompt-cache control, media inputs, generated media outputs, and tool approval waits via AbstractCore
 - `tools-comms.md` — enabling the optional comms toolset (email/WhatsApp/Telegram)
 
 ## Features (reference)

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/api.md

@@ -125,6 +125,7 @@ Key types:
 Artifacts are used by:
 - offloading wrappers (`src/abstractruntime/storage/offloading.py`)
 - evidence capture (`docs/evidence.md`, `src/abstractruntime/evidence/recorder.py`)
+- AbstractCore media integration: input artifact refs can be materialized for LLM calls, and generated image/voice/audio outputs are stored as artifact refs
 
 ## Snapshots / bookmarks
 
@@ -175,7 +176,7 @@ This produces a portable record of a run’s state + ledger + artifacts suitable
 
 ### AbstractCore (LLM + tools)
 
-Requires: `pip install "abstractruntime[abstractcore]"` (AbstractCore 2.13.5 or newer).
+Requires: `pip install "abstractruntime[abstractcore]"` (AbstractCore 2.13.9 or newer).
 
 Implementation: `src/abstractruntime/integrations/abstractcore/*`.
 
@@ -185,6 +186,24 @@ Entry points:
 - tool executors: `MappingToolExecutor`, `AbstractCoreToolExecutor`, `PassthroughToolExecutor`, `ApprovalToolExecutor`, `ToolApprovalPolicy` (`src/abstractruntime/integrations/abstractcore/tool_executor.py`)
 - prompt-cache control methods on the configured LLM client: `get_prompt_cache_capabilities`, `get_prompt_cache_stats`, `prompt_cache_set`, `prompt_cache_update`, `prompt_cache_fork`, `prompt_cache_clear`, `prompt_cache_prepare_modules` (`src/abstractruntime/integrations/abstractcore/llm_client.py`)
 
+`LLM_CALL` payloads are JSON-safe effect payloads. Common fields:
+- `prompt`, `messages`, `system_prompt`, and convenience `text`
+- `media`: a media path, artifact ref (`{"$artifact": "..."}` or `{"artifact_id": "..."}`), media dict, or list of those
+- `output`: AbstractCore output selector; top-level `outputs` is accepted as a runtime alias
+- `params`: provider/model routing, generation controls, prompt-cache keys, structured-output schema options, and tracing metadata
+
+Multimodal support:
+- install `abstractruntime[multimodal]` for common AbstractCore media, vision, voice, and audio dependencies
+- local clients call AbstractCore's unified `generate(..., media=..., output=...)`
+- remote and hybrid clients support AbstractCore Server chat media content arrays plus image generation, speech, and transcription endpoints; pass an output-specific `model` for remote media provider routing, otherwise the server endpoint can use its configured capability default
+- remote transcription requires one audio media item that resolves to a local file path or artifact-backed temporary file
+- generated image/voice/audio bytes require a runtime `ArtifactStore`; the result contains `artifact_id` / `artifact_ref` instead of inline bytes
+
+Prompt cache / cached sessions:
+- LLM clients expose cache control methods listed above for host-side preparation and inspection
+- `LLM_CALL.params.prompt_cache_key` selects a cache key for a call; runtime can also derive a session-scoped key from `run.vars["_runtime"]["prompt_cache"]`
+- provider cache/session handles are not durable runtime state and should not be stored in `RunState.vars`
+
 Docs: `integrations/abstractcore.md`.
 
 ### AbstractMemory bridge (KG effects)

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/architecture.md

@@ -1,7 +1,7 @@
 # AbstractRuntime — Architecture
 
-> Updated: 2026-05-06
-> Version: 0.4.3 (see `pyproject.toml`)
+> Updated: 2026-05-07
+> Version: 0.4.5
 > Scope: this describes **what is implemented in this repository**.
 
 AbstractRuntime is a **durable workflow runtime**: it executes workflow graphs as a persisted state machine with explicit waits (user, time, events, jobs, subworkflows). A run can pause for hours/days and resume **without** keeping Python stacks/coroutines alive.
@@ -26,6 +26,20 @@ Key invariants (enforced by code, not convention):
 - **Append-only observability**: every step is recorded as a `StepRecord` in a `LedgerStore` (`src/abstractruntime/core/models.py`, `src/abstractruntime/storage/base.py`).
 - **Side effects are mediated**: nodes request work via `Effect`/`EffectType`; execution happens via effect handlers (`src/abstractruntime/core/runtime.py`).
 
+## AbstractCore capability boundary
+
+AbstractRuntime's job is persistence and orchestration. AbstractCore owns model/provider capability execution: chat, structured output, cached sessions/prompt cache, media input analysis, image generation, voice/audio generation, transcription, and future modalities such as music or video.
+
+The boundary is intentionally narrow:
+- Workflow nodes request model work with `EffectType.LLM_CALL`; the runtime persists the request/result and delegates execution to the configured AbstractCore client.
+- `media` inputs remain JSON-safe in the effect payload. Artifact refs are materialized into temporary provider-ready files for the call, then cleaned up.
+- Generated binary outputs are written to `ArtifactStore` and returned as `artifact_id` / `artifact_ref`, keeping `RunState.vars` and ledger records bounded and JSON-safe.
+- Remote chat media is sent to AbstractCore Server as provider-ready content arrays, but persisted provider-request metadata redacts data URLs so checkpoints and ledgers do not embed media bytes.
+- Provider sessions and prompt-cache objects are not runtime state. Runtime may carry stable cache keys, while AbstractCore clients/servers manage warm caches.
+- Local execution can use richer AbstractCore capability plugins. Remote and hybrid execution map the common media cases to AbstractCore Server endpoints and OpenAI-compatible content arrays, while hybrid keeps tool execution local.
+
+This keeps the runtime usable by `../abstractgateway` and application layers such as `../abstractflow`, `../abstractassistant`, `../abstractobserver`, and `../abstractcode` without embedding provider-specific model logic in the durable kernel.
+
 ## Component map
 
 ```mermaid
@@ -152,7 +166,7 @@ Registered in `Runtime._register_builtin_handlers()` (`src/abstractruntime/core/
 
 ### Host-wired effects
 The kernel defines the protocol; concrete integrations provide handlers:
-- `LLM_CALL`, `TOOL_CALLS`: provided by AbstractCore integration (`src/abstractruntime/integrations/abstractcore/effect_handlers.py`). The integration supports local/remote/hybrid execution, prompt-cache control, provider-key header routing for remote servers, passthrough tools, and approval-gated local tool execution.
+- `LLM_CALL`, `TOOL_CALLS`: provided by AbstractCore integration (`src/abstractruntime/integrations/abstractcore/effect_handlers.py`). The integration supports local/remote/hybrid execution, cached sessions/prompt-cache control, media inputs, generated media outputs, provider-key header routing for remote servers, passthrough tools, and approval-gated local tool execution.
 - `MEMORY_KG_*`: provided by the AbstractMemory bridge (`src/abstractruntime/integrations/abstractmemory/effect_handlers.py`)
 
 ### Reliability: retries + idempotency

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/backlog/README.md

@@ -33,8 +33,8 @@ If you are new to the project, start with `../README.md` and `../architecture.md
 | 014 | `planned/014_remote_tool_worker_executor.md` |
 | 015 | `planned/015_agent_integration_improvements.md` |
 | 017 | `planned/017_limit_warnings_and_observability.md` |
+| 018 | `planned/018_workspace_access_policy_for_media_and_tools.md` |
 
 ## Deprecated
 
 See `deprecated/DEPRECATED_README.md` for context on the deprecated backlog set.
-

abstractruntime-0.4.5/docs/backlog/planned/018_workspace_access_policy_for_media_and_tools.md

@@ -0,0 +1,74 @@
+## 018_workspace_access_policy_for_media_and_tools (planned)
+
+**Status**: Planned
+**Priority**: High
+**Depends on**: 005_abstractcore_integration (completed), 009_artifact_store (completed)
+
+---
+
+## Goal
+
+Define and verify the runtime policy for local filesystem access used by LLMs,
+agents, media inputs, generated outputs, and tool execution.
+
+This item should determine which checks belong in AbstractRuntime, which checks
+belong in host applications, and how the effective policy is represented in
+durable run state and effect payloads.
+
+---
+
+## Context / Problem
+
+AbstractRuntime can pass local media paths to AbstractCore and can execute tools
+that read and write files. That is expected for local agency, but it must be
+controlled consistently.
+
+The default posture should be deny by default:
+- all filesystem paths are blacklisted unless explicitly allowed
+- the active local workspace folder is whitelisted by default
+- callers may consciously add whitelist or blacklist entries
+- a separate risky mode may allow access to the whole machine, limited only by
+  the OS permissions of the user/group running the runtime process
+
+This matters especially for remote AbstractCore execution, where a local media
+path may be read by the runtime and sent to a remote server as request content.
+
+---
+
+## Proposed Investigation
+
+1. Inventory every runtime path where filesystem content can be read or written:
+   LLM media inputs, artifact materialization, generated media storage, tool
+   calls, workspace-scoped tools, and attachment/session helpers.
+2. Identify the current source of truth for workspace roots, allowlists,
+   denylists, and risky full-machine access.
+3. Decide whether raw path authorization should be enforced before effect
+   handlers call providers, inside shared path helpers, or entirely by hosts
+   before they submit workflows.
+4. Define durable policy metadata for a run so replay, resume, and audit can
+   explain why a path was allowed.
+5. Make artifact refs the preferred boundary for externally supplied media;
+   raw local paths should be trusted only after policy validation.
+
+---
+
+## Acceptance Criteria
+
+- [ ] Document the effective default policy: deny all paths except the active workspace.
+- [ ] Document how users consciously add whitelist and blacklist paths.
+- [ ] Document the semantics and warning requirements for risky full-machine access.
+- [ ] Determine the enforcement layer for raw path media passed to AbstractCore.
+- [ ] Determine the enforcement layer for tool read/write access.
+- [ ] Add tests or conformance fixtures for allowed workspace paths, denied external paths, explicit allowlist paths, explicit blacklist overrides, and full-machine mode.
+- [ ] Ensure remote media forwarding cannot bypass the chosen path policy.
+
+---
+
+## Notes
+
+- Policy should be expressed in terms of canonical resolved paths where possible.
+- Blacklist entries should override broad whitelist entries.
+- The active workspace should be explicit in runtime-visible configuration, not
+  inferred from process current working directory alone.
+- Full-machine access is intentionally risky and should require an explicit mode
+  or consent signal.

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/faq.md

@@ -83,6 +83,35 @@ LLM and tool execution are wired via the **AbstractCore integration**:
 
 Docs: `integrations/abstractcore.md`. Code: `src/abstractruntime/integrations/abstractcore/*`.
 
+## Can `LLM_CALL` analyze images, audio, or files?
+
+Yes, when the configured AbstractCore provider/model supports the media. Pass `payload.media` as a path, a media dict, an artifact ref such as `{"$artifact": "..."}`, or a list of those. The runtime keeps the effect payload JSON-safe and materializes artifact refs into temporary provider-ready files for the call.
+
+Install `abstractruntime[multimodal]` for common media/vision/audio dependencies.
+Docs: `integrations/abstractcore.md`. Code: `src/abstractruntime/integrations/abstractcore/effect_handlers.py`, `src/abstractruntime/integrations/abstractcore/llm_client.py`.
+
+## How do I generate images or voice/audio?
+
+Use `LLM_CALL` with AbstractCore's `output` selector:
+
+```python
+{"text": "A red cube on a white table", "output": {"modality": "image", "format": "png"}}
+{"text": "Hello from Runtime", "output": {"modality": "voice", "voice": "alloy", "format": "wav"}}
+```
+
+Generated bytes require a runtime `ArtifactStore`. The durable result contains `artifact_id` / `artifact_ref`, not inline binary data. Remote and hybrid runtimes support common AbstractCore Server endpoints for image generation, speech, transcription, and chat media. Local runtimes can use richer AbstractCore capability plugins for image edits, voice cloning, and reference-guided generation.
+
+## Does AbstractRuntime implement image, voice, music, or video engines?
+
+No. AbstractRuntime provides the durable graph runner, checkpoint/ledger model, waits, and artifact boundary. AbstractCore provides the LLM/media generation and analysis capabilities. Future music/video support should use the same JSON-safe `output` selector plus artifact-backed result shape unless a new workflow-level wait/effect semantic is needed.
+
+## Where should cached session or prompt-cache state live?
+
+Store stable cache keys or cache configuration in runtime-visible JSON, for example `payload.params.prompt_cache_key` or `run.vars["_runtime"]["prompt_cache"]`. Do not store provider session objects, cache handles, clients, or warm-cache state in `RunState.vars`. AbstractCore clients/servers own those objects, and runtime correctness should still hold when a cache is cold.
+
+Hosts can inspect or prepare caches through the configured `_abstractcore_llm_client` control-plane methods (`get_prompt_cache_capabilities`, `prompt_cache_prepare_modules`, and related methods).
+Docs: `integrations/abstractcore.md`. Code: `src/abstractruntime/integrations/abstractcore/llm_client.py`.
+
 ## What are “local / remote / hybrid” execution modes?
 
 They refer to where LLM and tools execute:

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/getting-started.md

@@ -18,7 +18,7 @@ Optional (LLM + tools via AbstractCore):
 pip install "abstractruntime[abstractcore]"
 ```
 
-This installs AbstractCore 2.13.5 or newer, which matches the documented server auth, provider-key header routing, prompt-cache control plane, and current tool behavior.
+This installs AbstractCore 2.13.9 or newer, which matches the documented server auth, provider-key header routing, prompt-cache control plane, current tool behavior, unified multimodal generation response types, and public output-selector contract. Use `abstractruntime[multimodal]` when your workflows need common media, vision, voice, or audio dependencies.
 
 ## Mental model (source of truth)
 
@@ -142,4 +142,4 @@ rt = create_local_runtime(provider="ollama", model="qwen3:4b")
 - `architecture.md` — component map + durability invariants (with diagrams)
 - `manual_testing.md` — smoke tests and how to run `pytest`
 - `../examples/README.md` — runnable scripts
-- `integrations/abstractcore.md` — `LLM_CALL` / `TOOL_CALLS`
+- `integrations/abstractcore.md` — `LLM_CALL` / `TOOL_CALLS`, cached sessions/prompt-cache, media inputs, generated media

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/integrations/abstractcore.md

@@ -16,7 +16,15 @@ Implementation pointers (this repo):
 pip install "abstractruntime[abstractcore]"
 ```
 
-This extra installs AbstractCore 2.13.5 or newer. That is the supported baseline for the current server auth split (`Authorization` for server auth, `X-AbstractCore-Provider-API-Key` for provider overrides), prompt-cache control-plane endpoints, and current tool catalog.
+This extra installs AbstractCore 2.13.9 or newer. That is the supported baseline for the current server auth split (`Authorization` for server auth, `X-AbstractCore-Provider-API-Key` for provider overrides), prompt-cache control-plane endpoints, current tool catalog, the unified multimodal `generate(..., output=...)` response types, and AbstractCore's public output-selector contract.
+
+For AbstractCore's multimodal `generate(..., output=...)` path, use the newer baseline and optional media packages:
+
+```bash
+pip install "abstractruntime[multimodal]"
+```
+
+This installs `abstractcore[media,openai,vision,voice,audio]>=2.13.9`. Local image/voice generation still depends on the configured AbstractCore capability backends (for example AbstractVision and AbstractVoice, or OpenAI/OpenAI-compatible remote engines).
 
 The MCP worker entrypoint uses the `mcp-worker` extra:
 
@@ -82,8 +90,11 @@ print(state.output)
 ```json
 {
   "prompt": "...",
+  "text": "optional text alias, useful for TTS",
   "messages": [{"role": "user", "content": "..."}],
   "system_prompt": "...",
+  "media": ["path/or/artifact-ref"],
+  "output": {"modality": "text|image|voice", "task": "optional"},
   "tools": [{"name": "...", "description": "...", "parameters": {...}}],
   "params": {
     "temperature": 0.0,
@@ -97,6 +108,73 @@ Notes:
 - Remote mode supports per-request dynamic routing by forwarding `params.base_url` to the AbstractCore server request body (`src/abstractruntime/integrations/abstractcore/llm_client.py`).
 - Remote mode sends per-request provider key overrides from `params.api_key` / `params.provider_api_key` as `X-AbstractCore-Provider-API-Key` headers. Server/master auth should be supplied separately through the client's configured headers, usually `Authorization: Bearer <ABSTRACTCORE_SERVER_API_KEY>`.
 - Local mode treats `base_url` as a provider construction concern; the local client intentionally strips `params.base_url`.
+- `media` accepts one item or a list. Durable artifact refs such as `{"$artifact": "...", "filename": "speech.wav"}` are materialized to temporary files for AbstractCore and never stored as raw bytes in `RunState`.
+- `output` may be top-level or inside `params`; top-level `outputs` is accepted as a runtime alias for AbstractCore's `output`.
+- `output.tags`, when present, are merged into the generated artifact metadata. Runtime metadata such as `run_id` and `tags` is used by AbstractRuntime's ArtifactStore boundary and is not forwarded as provider-specific generation kwargs.
+
+## Multimodal generation
+
+AbstractRuntime forwards AbstractCore's unified `generate(..., output=...)` selector and normalizes multimodal responses into JSON-safe, artifact-backed results.
+
+Generate an image:
+
+```python
+Effect(
+    type=EffectType.LLM_CALL,
+    payload={
+        "prompt": "A red ceramic mug on a white table.",
+        "output": {"modality": "image", "format": "png", "width": 1024, "height": 1024},
+    },
+    result_key="image_result",
+)
+```
+
+Generate speech:
+
+```python
+Effect(
+    type=EffectType.LLM_CALL,
+    payload={
+        "text": "Hello from AbstractRuntime.",
+        "output": {"modality": "voice", "voice": "coral", "format": "wav"},
+    },
+    result_key="speech_result",
+)
+```
+
+Transcribe/analyze audio:
+
+```python
+Effect(
+    type=EffectType.LLM_CALL,
+    payload={
+        "media": {"$artifact": "audio_artifact_id", "filename": "speech.wav"},
+        "output": "text",
+    },
+    result_key="transcript",
+)
+```
+
+Generated binary media requires a runtime `ArtifactStore` and is stored there. The persisted result contains artifact references:
+
+```json
+{
+  "outputs": {
+    "image": [
+      {
+        "modality": "image",
+        "task": "image_generation",
+        "artifact_id": "...",
+        "artifact_ref": {"$artifact": "...", "content_type": "image/png"}
+      }
+    ]
+  }
+}
+```
+
+Remote runtimes support chat media by sending OpenAI-compatible data URL content arrays to AbstractCore Server. They also support image generation (`/v1/images/generations`), TTS (`/v1/audio/speech`), and STT (`/v1/audio/transcriptions`) with the same artifact-backed result shape. Remote media endpoint calls do not inherit the chat model by default; pass an output-specific `model` only when you want a remote provider/model instead of the server's configured capability default. Remote STT requires exactly one audio media item that resolves to a local file path or artifact-backed temporary file. For image edits, input-media image generation, voice clone/register, or reference-guided TTS, use local execution so AbstractCore can use its in-process capability dispatcher.
+
+Remote multimodal generation currently supports one `output` selector per `LLM_CALL`. Hybrid runtimes use the same remote LLM/media path as remote mode while executing tools locally. Local runtimes can use AbstractCore's in-process multimodal dispatcher for richer capability plugin behavior.
 
 Remote auth example:
 

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/limits.md

@@ -62,7 +62,7 @@ Returns a structured dict for UI/status display. (`src/abstractruntime/core/runt
 
 Returns a list of `LimitWarning` objects for limits approaching/exceeded. (`src/abstractruntime/core/models.py`, `src/abstractruntime/core/runtime.py`)
 
-As of v0.4.3, warnings are computed for:
+As of v0.4.5, warnings are computed for:
 - `iterations` (`current_iteration` vs `max_iterations`)
 - `tokens` (`estimated_tokens_used` vs `max_tokens`)
 

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/proposal.md

@@ -79,7 +79,7 @@ AbstractRuntime stays dependency-light at the kernel level; concrete integration
 
 ## Status (implemented in this repository)
 
-As of v0.4.3 (`pyproject.toml`):
+As of v0.4.5:
 - durable kernel: `RunState`, `WaitState`, `Runtime.start/tick/resume`
 - built-in waits + events: `WAIT_EVENT`, `WAIT_UNTIL`, `ASK_USER`, `EMIT_EVENT`
 - persistence backends: in-memory, JSON/JSONL, SQLite
@@ -88,7 +88,7 @@ As of v0.4.3 (`pyproject.toml`):
 - snapshots, tamper-evident ledger chain, ledger subscriptions
 - VisualFlow compiler + WorkflowBundles (`src/abstractruntime/visualflow_compiler/*`, `src/abstractruntime/workflow_bundle/*`)
 - VisualFlow multi-entry lowering for fan-in execution routes (`join_exec` / `path_mux`)
-- AbstractCore integration with local/remote/hybrid LLM execution, prompt-cache control, tool approval waits, and provider-key header routing (`integrations/abstractcore.md`)
+- AbstractCore integration with local/remote/hybrid LLM execution, cached sessions/prompt-cache control, media inputs, generated media outputs, tool approval waits, and provider-key header routing (`integrations/abstractcore.md`)
 - evidence capture helpers (`src/abstractruntime/evidence/recorder.py`, `Runtime.list_evidence/load_evidence`)
 - run history bundle export (`src/abstractruntime/history_bundle.py`)
 

{abstractruntime-0.4.3 → abstractruntime-0.4.5}/docs/provenance.md

@@ -8,7 +8,7 @@ Implementation pointers:
 - model fields: `src/abstractruntime/core/models.py` (`StepRecord.prev_hash`, `StepRecord.record_hash`, `StepRecord.signature`)
 - hash-chain decorator + verifier: `src/abstractruntime/storage/ledger_chain.py`
 
-## What is implemented (v0.4.3)
+## What is implemented (v0.4.5)
 
 - `HashChainedLedgerStore(inner_store)` — wraps any `LedgerStore` to compute hashes on append
 - `verify_ledger_chain(records)` — validates the chain and returns a verification report