traceAI-openwebui 0.1.0__tar.gz

traceai_openwebui-0.1.0/.gitignore

@@ -0,0 +1,87 @@
+*.py[cod]
+
+# C extensions
+*.so
+
+# IDEs
+.idea/
+.vscode/
+
+# Packages
+*.egg
+*.egg-info
+.eggs/
+# build
+parts
+bin
+var
+sdist
+dist
+develop-eggs
+.installed.cfg
+lib
+lib64
+
+# Installer logs
+pip-log.txt
+
+# Unit test / coverage reports
+.coverage
+.tox
+nosetests.xml
+
+# Complexity
+output/*.html
+output/*/index.html
+
+# Sphinx
+docs/_build
+
+# Cookiecutter
+output/
+
+.workon
+
+**.wav
+**.joblib
+
+**/__pycache__
+
+#Excel files
+*.xlsx
+#csv files
+*.csv
+#virtual environments
+**/.venv
+**/env
+**/.env
+*.DS_Store
+*.pdf
+**/node_modules
+**/dist
+*.log
+.DS_Store
+**/package-lock.json
+# Standalone npm example (outside the pnpm workspace): its lockfile pins the tested dependency set.
+!typescript/examples/tanstack-ai/package-lock.json
+**/tsconfig.tsbuildinfo
+**/.fi
+**/.pytest_cache
+planning/
+
+# Java build artifacts
+**/target/
+**/.settings/
+**/.project
+**/.classpath
+**/.factorypath
+*.class
+*.jar
+*.war
+
+# C# build artifacts
+**/bin/
+**/obj/
+
+# Python
+**/uv.lock

traceai_openwebui-0.1.0/PKG-INFO

@@ -0,0 +1,94 @@
+Metadata-Version: 2.5
+Name: traceAI-openwebui
+Version: 0.1.0
+Summary: Trace real Open WebUI model calls with a maintained Pipe Function
+Project-URL: Repository, https://github.com/future-agi/traceAI
+Project-URL: Documentation, https://docs.futureagi.com/docs/integrations/no-code/open-webui
+Author-email: Future AGI <no-reply@futureagi.com>
+License-Expression: Apache-2.0
+Requires-Python: <3.13,>=3.11
+Requires-Dist: certifi>=2024.7.4
+Requires-Dist: opentelemetry-exporter-otlp-proto-http<2,>=1.27
+Requires-Dist: opentelemetry-sdk<2,>=1.27
+Requires-Dist: packaging<27,>=24
+Requires-Dist: pydantic<3,>=2.9
+Requires-Dist: starlette<1,>=0.40
+Provides-Extra: dev
+Requires-Dist: build<2,>=1.2; extra == 'dev'
+Requires-Dist: mypy<2,>=1.18; extra == 'dev'
+Requires-Dist: pytest-asyncio<1,>=0.24; extra == 'dev'
+Requires-Dist: pytest<9,>=8; extra == 'dev'
+Requires-Dist: ruff<1,>=0.14; extra == 'dev'
+Requires-Dist: types-requests>=2.32; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# traceAI-openwebui
+
+Trace each model call made through an Open WebUI Pipe Function in Future AGI, with streaming duration, provider-reported usage, and failure or cancellation status.
+
+## Install
+
+Use Python 3.11 or 3.12 and Open WebUI 0.11.4–0.12.x. Configure an OpenAI-compatible **Chat Completions** connection model in Open WebUI first.
+
+Native Ollama and **Responses API** connections are not supported. Open WebUI's conversions can hide failures or incomplete responses, so the Pipe rejects these connections before making a model request or recording a span. Responses API connections are rejected even with streaming enabled: a provider can return a non-streaming response to a streaming request. Choose a connection configured for Chat Completions instead.
+
+Install the package in the same Python environment as Open WebUI, then generate its Function import file:
+
+```bash
+pip install traceAI-openwebui
+traceai-openwebui export-function --output traceai-openwebui.json
+```
+
+For a container deployment, install the package in your image and run the export command in that image. Make the generated JSON file available to your browser. The file contains a small package loader, not instrumentation source to maintain.
+
+## Configure the Function
+
+1. In **Admin Panel → Functions**, import `traceai-openwebui.json`.
+2. Open the **Traced model** Function's valves. Set `target_model` to the exact model ID from your existing Chat Completions connection. Responses API connections, native Ollama models, workspace model presets, other Pipes, and Arena models cannot be targets.
+3. Set `api_key`, `secret_key`, and `project` to your Future AGI credentials and project name. The default endpoint is `https://api.futureagi.com`. For self-hosted Future AGI, set its HTTPS origin; `/tracer/v1/traces` is added automatically.
+4. Enable the Function, and select **Traced model** in a new chat. Access to the target model is still checked using the signed-in Open WebUI user.
+
+Set `ENABLE_VALVE_ENCRYPTION=true` in Open WebUI before storing production credentials so its database encrypts the valves.
+
+Use the wrapper model's settings for tools and knowledge. Target-model workspace presets are not inherited. Enable **Usage** in the wrapper model's capabilities when your provider supports streamed token counts. Missing usage stays missing; the integration does not estimate it.
+
+Send a message such as “Explain how a cache works in two sentences.” In Future AGI, open **Observe**, choose your project, and inspect an `openwebui.model` span. Native tool loops that invoke this Pipe again produce a separate span for each actual model call, not one span for the whole conversation.
+
+## What is captured
+
+- One `LLM` span per delegated model request, from dispatch through response consumption.
+- Provider-reported input and output token counts on that call only. Repeated usage snapshots replace previous values; turn totals are never added again.
+- Fixed error messages for failures, cancellation, or an incomplete stream. Provider exception text and stack traces are not exported.
+- The active OpenTelemetry span as parent, when one exists. The package does not export Open WebUI's native HTTP, database, Redis, or authentication spans.
+
+This is **model-call tracing**, not a trace of the entire chat workflow. Retrieval, tool execution, approval pauses, and calls to unwrapped models are outside the Pipe boundary. The integration does not fabricate spans for them or reconstruct model calls from the final transcript. If you also instrument the target gateway, avoid counting both its LLM spans and this Pipe's spans as separate model usage.
+
+Prompts, responses, user identifiers, raw email, headers, URLs, tool arguments, and retrieval content are excluded by default. To include message content, enable `capture_content`; `max_chars` bounds each captured input/output to 32,000 characters by default. Temporary and local chats always exclude content. Model IDs and your configured project name are exported.
+
+Open WebUI's own telemetry is independent. Enabling it can export additional personal data through its separately configured exporter; this package does not sanitize that exporter.
+
+## Replace the old Filter
+
+Disable and delete the old Future AGI Filter Function before enabling this Pipe. The old filter recorded whole-turn summaries as an LLM child and could not observe failed or cancelled turns reliably. The Pipe is not a drop-in filter: choose **Traced model** for the calls you want to trace. Remove any separate tracing filter or gateway integration that would duplicate usage.
+
+Existing Function copies in Open WebUI's database do not change when this repository changes. Import the generated Function after each package upgrade, then restart Open WebUI to load the new Python package.
+
+## Troubleshooting
+
+**No spans:** Confirm you selected **Traced model**, supplied both Future AGI keys, and enabled the Function. Tracing configuration/export failures do not block the model response. Allow a few seconds for the batch exporter.
+
+**Target model rejected:** Use an OpenAI-compatible Chat Completions connection model ID, not a Responses API connection, native Ollama model, workspace preset, Arena model, or another Pipe. The signed-in user must also have permission to use that target.
+
+**Invalid non-streaming response:** The provider returned a scalar, array, or undecodable body instead of a completion object. Check the connection URL and API type; the Pipe returns an explicit error rather than presenting that body as generated text.
+
+**No token counts:** Enable Usage on the wrapper model and check that the provider returns usage. Cancelled calls may never receive a final usage snapshot.
+
+**Incomplete stream:** The provider ended its stream without confirmed completion, or the consumer closed it early. The span is not marked successful. When the request explicitly supplies a positive integer `n`, every requested choice must finish unless the provider sends `[DONE]`. Without explicit `n`, the Pipe requires `[DONE]`: native model defaults can change the choice count after dispatch, so a single finished choice does not prove the request completed. If your provider omits `[DONE]`, supply the intended `n` in the wrapper request rather than relying on target-model defaults. This does not change the response sent to your chat. An abrupt process kill cannot export pending spans.
+
+**Oversized streaming event:** The Pipe inspects up to 1,048,576 decoded characters per SSE event, including framing and line endings. Larger events still reach your chat unchanged, but the Pipe discards them from telemetry parsing to bound its buffer. The span records `openwebui.observation_lost=true` and an `incomplete` outcome with error status, even if `[DONE]` or finished choices are observed; a separately observed failure or cancellation retains its own outcome. This indicates incomplete observation, not proof that the provider failed. Content and usage from discarded events are unavailable; token counts reflect only observed snapshots. Configure your provider to send smaller streaming events if you need complete telemetry. The `max_chars` content-capture setting does not change this inspection limit.
+
+**Self-hosted collector:** Remote endpoints must use HTTPS; HTTP is accepted only for loopback testing. Redirects, host proxy settings, `.netrc`, and unrelated native OTLP client certificates are not used. Use a directly reachable collector endpoint.
+
+## Embedding lifecycle
+
+If you manage the Pipe lifecycle in Python, `pipe.flush(timeout_millis=10000)` bounds the caller's wait and returns `False` when the flush does not complete within that budget. An in-progress export can continue in the background; retrying does not create concurrent flush workers. Call `pipe.close()` to retire the exporter after active model calls finish.

traceai_openwebui-0.1.0/README.md

@@ -0,0 +1,70 @@
+# traceAI-openwebui
+
+Trace each model call made through an Open WebUI Pipe Function in Future AGI, with streaming duration, provider-reported usage, and failure or cancellation status.
+
+## Install
+
+Use Python 3.11 or 3.12 and Open WebUI 0.11.4–0.12.x. Configure an OpenAI-compatible **Chat Completions** connection model in Open WebUI first.
+
+Native Ollama and **Responses API** connections are not supported. Open WebUI's conversions can hide failures or incomplete responses, so the Pipe rejects these connections before making a model request or recording a span. Responses API connections are rejected even with streaming enabled: a provider can return a non-streaming response to a streaming request. Choose a connection configured for Chat Completions instead.
+
+Install the package in the same Python environment as Open WebUI, then generate its Function import file:
+
+```bash
+pip install traceAI-openwebui
+traceai-openwebui export-function --output traceai-openwebui.json
+```
+
+For a container deployment, install the package in your image and run the export command in that image. Make the generated JSON file available to your browser. The file contains a small package loader, not instrumentation source to maintain.
+
+## Configure the Function
+
+1. In **Admin Panel → Functions**, import `traceai-openwebui.json`.
+2. Open the **Traced model** Function's valves. Set `target_model` to the exact model ID from your existing Chat Completions connection. Responses API connections, native Ollama models, workspace model presets, other Pipes, and Arena models cannot be targets.
+3. Set `api_key`, `secret_key`, and `project` to your Future AGI credentials and project name. The default endpoint is `https://api.futureagi.com`. For self-hosted Future AGI, set its HTTPS origin; `/tracer/v1/traces` is added automatically.
+4. Enable the Function, and select **Traced model** in a new chat. Access to the target model is still checked using the signed-in Open WebUI user.
+
+Set `ENABLE_VALVE_ENCRYPTION=true` in Open WebUI before storing production credentials so its database encrypts the valves.
+
+Use the wrapper model's settings for tools and knowledge. Target-model workspace presets are not inherited. Enable **Usage** in the wrapper model's capabilities when your provider supports streamed token counts. Missing usage stays missing; the integration does not estimate it.
+
+Send a message such as “Explain how a cache works in two sentences.” In Future AGI, open **Observe**, choose your project, and inspect an `openwebui.model` span. Native tool loops that invoke this Pipe again produce a separate span for each actual model call, not one span for the whole conversation.
+
+## What is captured
+
+- One `LLM` span per delegated model request, from dispatch through response consumption.
+- Provider-reported input and output token counts on that call only. Repeated usage snapshots replace previous values; turn totals are never added again.
+- Fixed error messages for failures, cancellation, or an incomplete stream. Provider exception text and stack traces are not exported.
+- The active OpenTelemetry span as parent, when one exists. The package does not export Open WebUI's native HTTP, database, Redis, or authentication spans.
+
+This is **model-call tracing**, not a trace of the entire chat workflow. Retrieval, tool execution, approval pauses, and calls to unwrapped models are outside the Pipe boundary. The integration does not fabricate spans for them or reconstruct model calls from the final transcript. If you also instrument the target gateway, avoid counting both its LLM spans and this Pipe's spans as separate model usage.
+
+Prompts, responses, user identifiers, raw email, headers, URLs, tool arguments, and retrieval content are excluded by default. To include message content, enable `capture_content`; `max_chars` bounds each captured input/output to 32,000 characters by default. Temporary and local chats always exclude content. Model IDs and your configured project name are exported.
+
+Open WebUI's own telemetry is independent. Enabling it can export additional personal data through its separately configured exporter; this package does not sanitize that exporter.
+
+## Replace the old Filter
+
+Disable and delete the old Future AGI Filter Function before enabling this Pipe. The old filter recorded whole-turn summaries as an LLM child and could not observe failed or cancelled turns reliably. The Pipe is not a drop-in filter: choose **Traced model** for the calls you want to trace. Remove any separate tracing filter or gateway integration that would duplicate usage.
+
+Existing Function copies in Open WebUI's database do not change when this repository changes. Import the generated Function after each package upgrade, then restart Open WebUI to load the new Python package.
+
+## Troubleshooting
+
+**No spans:** Confirm you selected **Traced model**, supplied both Future AGI keys, and enabled the Function. Tracing configuration/export failures do not block the model response. Allow a few seconds for the batch exporter.
+
+**Target model rejected:** Use an OpenAI-compatible Chat Completions connection model ID, not a Responses API connection, native Ollama model, workspace preset, Arena model, or another Pipe. The signed-in user must also have permission to use that target.
+
+**Invalid non-streaming response:** The provider returned a scalar, array, or undecodable body instead of a completion object. Check the connection URL and API type; the Pipe returns an explicit error rather than presenting that body as generated text.
+
+**No token counts:** Enable Usage on the wrapper model and check that the provider returns usage. Cancelled calls may never receive a final usage snapshot.
+
+**Incomplete stream:** The provider ended its stream without confirmed completion, or the consumer closed it early. The span is not marked successful. When the request explicitly supplies a positive integer `n`, every requested choice must finish unless the provider sends `[DONE]`. Without explicit `n`, the Pipe requires `[DONE]`: native model defaults can change the choice count after dispatch, so a single finished choice does not prove the request completed. If your provider omits `[DONE]`, supply the intended `n` in the wrapper request rather than relying on target-model defaults. This does not change the response sent to your chat. An abrupt process kill cannot export pending spans.
+
+**Oversized streaming event:** The Pipe inspects up to 1,048,576 decoded characters per SSE event, including framing and line endings. Larger events still reach your chat unchanged, but the Pipe discards them from telemetry parsing to bound its buffer. The span records `openwebui.observation_lost=true` and an `incomplete` outcome with error status, even if `[DONE]` or finished choices are observed; a separately observed failure or cancellation retains its own outcome. This indicates incomplete observation, not proof that the provider failed. Content and usage from discarded events are unavailable; token counts reflect only observed snapshots. Configure your provider to send smaller streaming events if you need complete telemetry. The `max_chars` content-capture setting does not change this inspection limit.
+
+**Self-hosted collector:** Remote endpoints must use HTTPS; HTTP is accepted only for loopback testing. Redirects, host proxy settings, `.netrc`, and unrelated native OTLP client certificates are not used. Use a directly reachable collector endpoint.
+
+## Embedding lifecycle
+
+If you manage the Pipe lifecycle in Python, `pipe.flush(timeout_millis=10000)` bounds the caller's wait and returns `False` when the flush does not complete within that budget. An in-progress export can continue in the background; retrying does not create concurrent flush workers. Call `pipe.close()` to retire the exporter after active model calls finish.

traceai_openwebui-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["hatchling>=1.27,<2"]
+build-backend = "hatchling.build"
+
+[project]
+name = "traceAI-openwebui"
+version = "0.1.0"
+description = "Trace real Open WebUI model calls with a maintained Pipe Function"
+readme = "README.md"
+license = "Apache-2.0"
+authors = [{name = "Future AGI", email = "no-reply@futureagi.com"}]
+requires-python = ">=3.11,<3.13"
+dependencies = [
+  "opentelemetry-sdk>=1.27,<2",
+  "opentelemetry-exporter-otlp-proto-http>=1.27,<2",
+  "pydantic>=2.9,<3",
+  "starlette>=0.40,<1",
+  "packaging>=24,<27",
+  "certifi>=2024.7.4",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8,<9", "pytest-asyncio>=0.24,<1", "build>=1.2,<2", "ruff>=0.14,<1", "mypy>=1.18,<2", "types-requests>=2.32"]
+
+[project.scripts]
+traceai-openwebui = "traceai_openwebui.cli:main"
+
+[project.urls]
+Repository = "https://github.com/future-agi/traceAI"
+Documentation = "https://docs.futureagi.com/docs/integrations/no-code/open-webui"
+
+[tool.hatch.build.targets.wheel]
+packages = ["traceai_openwebui"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"

traceai_openwebui-0.1.0/tests/conftest.py

@@ -0,0 +1,167 @@
+import ast
+import inspect
+import json
+import logging
+import os
+import sys
+import time
+import uuid
+from collections.abc import AsyncGenerator, Generator, Iterator
+from pathlib import Path
+from types import ModuleType, SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock
+from zipfile import ZipFile
+
+import pytest
+import traceai_openwebui
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import SimpleSpanProcessor
+from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
+from pydantic import BaseModel, SecretStr
+from starlette.responses import Response, StreamingResponse
+
+
+@pytest.fixture
+def setup_pipe(monkeypatch):
+    modules = {}
+    for name in (
+        "open_webui",
+        "open_webui.models",
+        "open_webui.models.users",
+        "open_webui.routers",
+        "open_webui.routers.openai",
+        "open_webui.utils",
+        "open_webui.utils.chat",
+    ):
+        modules[name] = ModuleType(name)
+        monkeypatch.setitem(sys.modules, name, modules[name])
+    user = SimpleNamespace(
+        id="synthetic-user",
+        role="user",
+        model_dump=lambda: {"id": "synthetic-user", "email": "secret@example.invalid"},
+    )
+    modules["open_webui.models.users"].Users = SimpleNamespace(
+        get_user_by_id=AsyncMock(return_value=user)
+    )
+    delegate = AsyncMock(
+        return_value={
+            "choices": [{"message": {"content": "private-output"}}],
+            "usage": {"prompt_tokens": 7, "completion_tokens": 3},
+        }
+    )
+    modules["open_webui.utils.chat"].generate_chat_completion = delegate
+    modules["open_webui.routers.openai"].get_all_models = AsyncMock()
+    modules["open_webui.routers.openai"].get_openai_connection = AsyncMock(
+        return_value=("http://127.0.0.1/v1", "synthetic-key", {})
+    )
+    monkeypatch.setattr(traceai_openwebui, "version", lambda name: "0.12.5")
+    exporter = InMemorySpanExporter()
+    provider = TracerProvider()
+    provider.add_span_processor(SimpleSpanProcessor(exporter))
+    pipe = traceai_openwebui.Pipe()
+    pipe.valves.target_model = "synthetic"
+    pipe.valves.api_key = SecretStr("synthetic-api")
+    pipe.valves.secret_key = SecretStr("synthetic-secret")
+    monkeypatch.setattr(
+        pipe._providers, "acquire", lambda key: (provider, lambda: None)
+    )
+    request = SimpleNamespace(
+        app=SimpleNamespace(
+            state=SimpleNamespace(
+                OPENAI_MODELS={"synthetic": {"id": "synthetic", "urlIdx": 0}},
+                MODELS={
+                    "synthetic": {"id": "synthetic", "owned_by": "openai"},
+                    "traceai_openwebui": {
+                        "id": "traceai_openwebui",
+                        "pipe": True,
+                        "owned_by": "openai",
+                    },
+                },
+            )
+        ),
+        state=SimpleNamespace(),
+    )
+    body = {
+        "model": "traceai_openwebui",
+        "messages": [{"role": "user", "content": "private-input"}],
+    }
+    yield SimpleNamespace(
+        pipe=pipe,
+        request=request,
+        body=body,
+        user=user,
+        delegate=delegate,
+        modules=modules,
+        provider=provider,
+        exporter=exporter,
+    )
+    provider.shutdown()
+
+
+@pytest.fixture
+def host(setup_pipe, monkeypatch):
+    s = setup_pipe
+    path = os.environ.get("OPENWEBUI_WHEEL")
+    if not path:
+        pytest.skip(
+            "Set OPENWEBUI_WHEEL to run the real released-host dispatch contract"
+        )
+    with ZipFile(Path(path)) as wheel:
+        metadata_path = next(
+            n for n in wheel.namelist() if n.endswith(".dist-info/METADATA")
+        )
+        version = next(
+            line[9:]
+            for line in wheel.read(metadata_path).decode().splitlines()
+            if line.startswith("Version: ")
+        )
+        monkeypatch.setattr("traceai_openwebui.version", lambda name: version)
+        ns = {
+            "Any": Any,
+            "Request": Any,
+            "Response": Response,
+            "StreamingResponse": StreamingResponse,
+            "inspect": inspect,
+            "time": time,
+            "uuid": uuid,
+            "BaseModel": BaseModel,
+            "JSONCodec": SimpleNamespace(dumps=json.dumps, loads=json.loads),
+            "Generator": Generator,
+            "AsyncGenerator": AsyncGenerator,
+            "Iterator": Iterator,
+            "BYPASS_MODEL_ACCESS_CONTROL": False,
+            "BYPASS_ADMIN_ACCESS_CONTROL": False,
+            "log": logging.getLogger("host_contract"),
+            "UserModel": type(s.user),
+            "Models": SimpleNamespace(get_model_by_id=AsyncMock(return_value=None)),
+            "get_system_oauth_token": AsyncMock(return_value=None),
+            "get_event_emitter": AsyncMock(return_value=None),
+            "get_event_call": AsyncMock(return_value=None),
+            "check_model_access": AsyncMock(),
+            "get_function_module_by_id": AsyncMock(return_value=s.pipe),
+            "generate_openai_chat_completion": s.delegate,
+        }
+        for filename, name in (
+            ("open_webui/functions.py", "generate_function_chat_completion"),
+            ("open_webui/utils/chat.py", "generate_chat_completion"),
+            ("open_webui/utils/misc.py", "openai_chat_message_template"),
+            ("open_webui/utils/misc.py", "openai_chat_completion_message_template"),
+            ("open_webui/utils/misc.py", "openai_chat_chunk_message_template"),
+        ):
+            module = ast.parse(wheel.read(filename).decode(), filename=filename)
+            node = next(
+                n
+                for n in module.body
+                if isinstance(n, (ast.FunctionDef, ast.AsyncFunctionDef))
+                and n.name == name
+            )
+            # Compile the entire unchanged function, never rewritten hook snippets.
+            exec(  # noqa: S102 - execute released host contract only in isolated tests
+                compile(ast.Module(body=[node], type_ignores=[]), filename, "exec"), ns
+            )
+    s.request.cookies = {}
+    s.modules["open_webui.utils.chat"].generate_chat_completion = ns[
+        "generate_chat_completion"
+    ]
+    return s, ns

traceai_openwebui-0.1.0/tests/test_host_contract.py

@@ -0,0 +1,180 @@
+"""Released host dispatch and streaming contract, not a full application test."""
+
+import asyncio
+from unittest.mock import AsyncMock
+
+import pytest
+from starlette.responses import JSONResponse, Response, StreamingResponse
+from test_pipe import frame, iterator
+
+
+async def test_released_host_nonstream_and_access(host):
+    s, ns = host
+    result = await ns["generate_chat_completion"](s.request, s.body, s.user)
+    assert result is s.delegate.return_value
+    assert [call.args[1]["id"] for call in ns["check_model_access"].call_args_list] == [
+        "traceai_openwebui",
+        "synthetic",
+    ]
+    (span,) = s.exporter.get_finished_spans()
+    assert span.attributes["llm.token_count.total"] == 10
+
+
+async def test_released_host_stream_is_not_double_framed(host):
+    s, ns = host
+    chunks = [
+        frame(
+            {"choices": [{"delta": {"content": "synthetic"}, "finish_reason": "stop"}]}
+        ),
+        frame({"usage": {"input_tokens": 6, "output_tokens": 2}}),
+        "data: [DONE]\n\n",
+    ]
+    s.delegate.return_value = StreamingResponse(
+        iterator(chunks), media_type="text/event-stream"
+    )
+    result = await ns["generate_chat_completion"](
+        s.request, {**s.body, "stream": True}, s.user
+    )
+    assert [chunk async for chunk in result.body_iterator] == chunks
+    (span,) = s.exporter.get_finished_spans()
+    assert span.attributes["llm.token_count.total"] == 8
+
+
+async def test_released_host_stream_cancellation(host):
+    s, ns = host
+    waiting = asyncio.Event()
+
+    async def source():
+        waiting.set()
+        await asyncio.Event().wait()
+        yield "unreachable"
+
+    s.delegate.return_value = StreamingResponse(source())
+    result = await ns["generate_chat_completion"](
+        s.request, {**s.body, "stream": True}, s.user
+    )
+
+    async def consume():
+        return [chunk async for chunk in result.body_iterator]
+
+    task = asyncio.create_task(consume())
+    await waiting.wait()
+    task.cancel()
+    with pytest.raises(asyncio.CancelledError):
+        await task
+    (span,) = s.exporter.get_finished_spans()
+    assert span.attributes["openwebui.outcome"] == "cancelled"
+
+
+async def test_released_host_failure_is_traced_even_if_host_catches_it(host):
+    s, ns = host
+    s.delegate.side_effect = RuntimeError("synthetic provider failure")
+    result = await ns["generate_chat_completion"](s.request, s.body, s.user)
+    assert "error" in result
+    (span,) = s.exporter.get_finished_spans()
+    assert span.status.description == "model request failed"
+
+
+async def test_released_host_access_failure_never_calls_provider(host):
+    s, ns = host
+
+    async def check(user, model):
+        if model["id"] == "synthetic":
+            raise PermissionError("synthetic denial")
+
+    ns["check_model_access"].side_effect = check
+    result = await ns["generate_chat_completion"](s.request, s.body, s.user)
+    assert "error" in result
+    s.delegate.assert_not_awaited()
+    (span,) = s.exporter.get_finished_spans()
+    assert span.status.description == "model request failed"
+
+
+@pytest.mark.parametrize("stream", [False, True])
+@pytest.mark.parametrize(
+    "result",
+    [
+        JSONResponse({"error": {"message": "synthetic denial"}}, status_code=429),
+        JSONResponse({"detail": "synthetic denial"}, status_code=403),
+        Response(b"synthetic unavailable", status_code=503),
+        Response(b"not a completion", status_code=200),
+        JSONResponse(["not a completion"], status_code=200),
+        JSONResponse({}, status_code=200),
+        JSONResponse({"choices": [{"delta": {"content": "not a completion"}}]}),
+    ],
+)
+async def test_released_host_http_error_is_not_empty_success(host, stream, result):
+    s, ns = host
+    s.delegate.return_value = result
+    response = await ns["generate_chat_completion"](
+        s.request, {**s.body, "stream": stream}, s.user
+    )
+    if stream:
+        chunks = [x async for x in response.body_iterator]
+        assert len(chunks) == 1 and '"error"' in chunks[0]
+        assert "finish_reason" not in chunks[0]
+    else:
+        assert isinstance(response, dict) and response["error"]
+        assert "choices" not in response
+    (span,) = s.exporter.get_finished_spans()
+    assert span.status.description == "model request failed"
+
+
+async def test_released_host_nonstream_json_response_preserves_completion(host):
+    s, ns = host
+    completion = s.delegate.return_value
+    s.delegate.return_value = JSONResponse(completion)
+    response = await ns["generate_chat_completion"](s.request, s.body, s.user)
+    assert response == completion
+    (span,) = s.exporter.get_finished_spans()
+    assert span.attributes["llm.token_count.total"] == 10
+
+
+@pytest.mark.parametrize("stream", [False, True])
+async def test_released_host_native_ollama_is_rejected_before_dispatch(host, stream):
+    s, ns = host
+    s.request.app.state.MODELS["synthetic"]["owned_by"] = "ollama"
+    native = ns["generate_ollama_chat_completion"] = AsyncMock()
+    response = await ns["generate_chat_completion"](
+        s.request, {**s.body, "stream": stream}, s.user
+    )
+    if stream:
+        chunks = [x async for x in response.body_iterator]
+        assert len(chunks) == 1 and "Native Ollama" in chunks[0]
+        assert '"error"' in chunks[0] and "finish_reason" not in chunks[0]
+    else:
+        assert "Native Ollama" in str(response["error"])
+    native.assert_not_awaited()
+    s.delegate.assert_not_awaited()
+    assert not s.exporter.get_finished_spans()
+
+
+async def test_released_host_shielded_cancellation_cleanup(host):
+    s, ns = host
+    s.delegate.return_value = StreamingResponse(
+        iterator(["data: {}\n\n", "data: [DONE]\n\n"])
+    )
+    response = await ns["generate_chat_completion"](
+        s.request, {**s.body, "stream": True}, s.user
+    )
+    waiting = asyncio.Event()
+
+    async def consume():
+        try:
+            await anext(response.body_iterator)
+            waiting.set()
+            await asyncio.Event().wait()
+        except asyncio.CancelledError:
+            # Released middleware closes the outer dispatcher under shield BEFORE
+            # the consuming task becomes cancelled. Its inner stream is finalized.
+            await asyncio.shield(response.body_iterator.aclose())
+            raise
+
+    task = asyncio.create_task(consume())
+    await waiting.wait()
+    task.cancel()
+    with pytest.raises(asyncio.CancelledError):
+        await task
+    (span,) = s.exporter.get_finished_spans()
+    assert span.attributes["openwebui.outcome"] == "cancelled"
+    assert span.status.description == "model request cancelled"