agentops-accelerator 0.4.3__tar.gz → 0.4.5__tar.gz

{agentops_accelerator-0.4.3 → agentops_accelerator-0.4.5}/AGENTS.md

@@ -335,7 +335,11 @@ Full schema:
 | `response_field` | no | HTTP / invocations only. Dot-path to extract the response text (default: `text`). |
 | `tool_calls_field` | no | HTTP / invocations only. Dot-path to extract tool calls for agent-workflow evaluators. |
 | `headers` | no | HTTP / invocations only. Static extra HTTP headers. |
-| `auth_header_env` | no | HTTP / invocations only. Environment variable that holds a Bearer token. |
+| `auth_header_env` | no | HTTP / invocations only. Environment variable that holds the auth token. |
+| `auth_header_name` | no | HTTP / invocations only. Header name for the auth token (default: `Authorization`). |
+| `auth_value_template` | no | HTTP / invocations only. Template for the auth header value; `{token}` is replaced by the `auth_header_env` value (default: `Bearer {token}`). |
+| `response_mode` | no | HTTP / invocations only. `json` (default, single `json.loads` of the body), `sse` (parse `data:` lines), or `text` (concatenate raw streamed text). |
+| `stream` | no | HTTP / invocations only. Streaming aggregation block for `response_mode: sse|text`: `text_field` (dot-path to token text in JSON `data:` lines), `done_marker` (stop token, e.g. `[DONE]`), `strip_leading_token` (drop the first whitespace-delimited token, e.g. a `conversation_id` prefix). |
 | `assert_path` | no | Optional ASSERT policy/results file or directory referenced by Doctor/evidence. AgentOps does not execute ASSERT. |
 | `acs_path` | no | Optional Agent Control Specification contract file or directory referenced by Doctor/evidence. AgentOps does not apply ACS controls. |
 | `redteam_path` | no | Optional red-team plan/results evidence path. AgentOps records metadata and never exposes payload text. |

{agentops_accelerator-0.4.3 → agentops_accelerator-0.4.5}/CHANGELOG.md

@@ -5,6 +5,53 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
 
 ## [Unreleased]
 
+## [0.4.5] - 2026-06-19
+
+### Added
+- **Governance gates for HTTP agents (ASSERT and Red Team).** `agentops assert
+  run` and `agentops redteam run` now work against a live HTTP orchestrator
+  endpoint, not only model/deployment targets. Red Team wraps the HTTP endpoint
+  as an SDK-compatible target and reuses the AgentOps HTTP mapping
+  (`request_field`, `response_mode`, `stream`, custom headers). ASSERT resolves
+  `assert-ai` inside the active virtual environment, accepts non-secret values
+  from `assert.env`, can request an AAD token from the Azure CLI for local
+  auth-disabled Azure AI resources, injects the GPT-5 `max_completion_tokens`
+  shim only when configured, and materializes a runtime ASSERT config so
+  committed configs no longer need absolute artifact paths.
+- **Generated workflows run the ASSERT and Red Team gates.** `agentops workflow
+  generate` now installs the optional ASSERT/Red Team dependencies, runs those
+  gates when `assert:` or `redteam:` is present in `agentops.yaml`, uploads
+  their artifacts, and emits the corrected Red Team command quoting.
+
+### Fixed
+- **Reasoning-model judges no longer fail the eval gate in CI.** The generated
+  GitHub Actions and Azure DevOps eval and Red Team steps now forward
+  `AZURE_OPENAI_MODEL_NAME`, so AgentOps detects reasoning models (such as
+  `gpt-5-nano`) and uses `max_completion_tokens` instead of `max_tokens`. This
+  removes the judge `400` error that could break the eval gate when the judge
+  deployment is a reasoning model.
+
+## [0.4.4] - 2026-06-18
+
+### Added
+- **Streaming HTTP targets.** The `http_json` target now understands streaming
+  responses so AgentOps can evaluate SSE/streaming agents (such as the
+  gpt-rag-orchestrator `/orchestrator` endpoint) directly, without a manual
+  adapter. A new `response_mode: json|sse|text` field selects the response
+  parser (`json` is the default and preserves the existing single-`json.loads`
+  behavior exactly). For `sse`/`text`, an optional `stream` block configures
+  aggregation: `text_field` (dotted path to the token text when each SSE
+  `data:` line is JSON), `done_marker` (stop token, e.g. `[DONE]`), and
+  `strip_leading_token` (drop the leading whitespace-delimited token, e.g. the
+  orchestrator's `conversation_id` prefix). The auth header is now configurable
+  via `auth_header_name` (default `Authorization`) and `auth_value_template`
+  (default `Bearer {token}`, where `{token}` is replaced by the
+  `auth_header_env` value), so targets gated by a shared secret such as
+  `X-API-KEY` are supported without hardcoding the secret in `agentops.yaml`.
+  Streaming uses the same stdlib (`urllib`) transport and 3-try backoff as the
+  JSON path. When a JSON parse fails on a `text/event-stream` response, the
+  error now suggests setting `response_mode: sse|text`.
+
 ## [0.4.3] - 2026-06-17
 
 ### Added
@@ -35,6 +82,22 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
   `eval.yaml`, so users can see why those evaluators were chosen.
   ([#323](https://github.com/Azure/agentops/issues/323))
 
+## [0.4.2] - 2026-06-17
+
+### Fixed
+- **`agentops eval init` now works with both old and new `azure.ai.agents` azd
+  extensions.** Version 0.1.40 of the extension renamed the eval subcommand from
+  `azd ai agent eval init` to `azd ai agent eval generate`, which made
+  `agentops eval init` hard-fail with `Command "init" is deprecated, use 'azd ai
+  agent eval generate' instead`. AgentOps now invokes `generate` first and
+  transparently falls back to the legacy `init` subcommand when an older
+  extension does not recognise `generate`. The fallback only triggers on
+  subcommand-name/deprecation errors; genuine failures (authentication, project
+  endpoint, timeouts) are still surfaced immediately and unchanged. All
+  previously passed flags (`--project-endpoint`, `--agent`,
+  `--gen-instruction-file`, `--eval-model`, `--dataset`, `--evaluator`) and the
+  recipe discovery/persistence behaviour are preserved.
+
 ## [0.4.1] - 2026-06-15
 
 ### Changed

{agentops_accelerator-0.4.3/src/agentops_accelerator.egg-info → agentops_accelerator-0.4.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentops-accelerator
-Version: 0.4.3
+Version: 0.4.5
 Summary: Release readiness gates and evidence for Microsoft Foundry agents
 License: MIT License
         

{agentops_accelerator-0.4.3/src/agentops/templates → agentops_accelerator-0.4.5/plugins/agentops}/skills/agentops-eval/SKILL.md

@@ -301,4 +301,11 @@ dataset reference.
   ```
 - For HTTP/JSON agents that need auth, set
   `auth_header_env: MY_TOKEN_VAR` and AgentOps adds
-  `Authorization: Bearer $MY_TOKEN_VAR`.
+  `Authorization: Bearer $MY_TOKEN_VAR`. For a shared-secret gate, override the
+  header with `auth_header_name: X-API-KEY` and `auth_value_template: "{token}"`.
+- For streaming HTTP agents (e.g. an SSE `text/event-stream` endpoint), set
+  `response_mode: sse` (each `data:` line) or `response_mode: text` (raw
+  streamed text). Use the optional `stream:` block to tune aggregation:
+  `text_field` (dot-path to the token text when `data:` lines are JSON),
+  `done_marker` (e.g. `[DONE]`), and `strip_leading_token: true` (drop a leading
+  `conversation_id` prefix). `response_mode: json` (default) is unchanged.

{agentops_accelerator-0.4.3 → agentops_accelerator-0.4.5}/plugins/agentops/skills/agentops-governance/SKILL.md

@@ -206,6 +206,115 @@ assert-ai init
 It walks them through behavior description, target callable / model /
 endpoint, dimensions, and judge presets, and writes a validated YAML.
 
+### HTTP orchestrator ASSERT
+
+If `agentops.yaml` uses `protocol: http-json` or the user says the target is an
+HTTP orchestrator, do not use ASSERT native endpoint mode. `assert-ai 0.1.0`
+posts `message/history` and expects `response`; AgentOps HTTP targets may use
+custom fields like `ask` and streamed text. Scaffold a callable adapter instead.
+
+Create `.agentops/assert_http_adapter.py`:
+
+```python
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+from agentops.core.config_loader import load_agentops_config
+from agentops.pipeline.invocations import (
+    _aggregate_stream,
+    _dot_path,
+    _http_request_json,
+    _http_request_stream,
+)
+
+
+def target(message: str, history: list[dict[str, Any]] | None = None) -> str:
+    del history
+    config = load_agentops_config(Path("agentops.yaml"))
+    if not config.agent:
+        raise RuntimeError("agentops.yaml must define a top-level HTTP agent endpoint")
+
+    request_field = config.request_field or "message"
+    headers = dict(config.headers)
+    headers.setdefault("Content-Type", "application/json")
+    body = {request_field: message}
+
+    if config.response_mode in ("sse", "text"):
+        raw_body = _http_request_stream(
+            method="POST",
+            url=config.agent,
+            headers=headers,
+            body=body,
+            timeout=120,
+        )
+        return _aggregate_stream(config.response_mode, raw_body, config.stream).strip()
+
+    payload = _http_request_json(
+        method="POST",
+        url=config.agent,
+        headers=headers,
+        body=body,
+        timeout=120,
+    )
+    response_path = config.response_field or "text"
+    response_text = _dot_path(payload, response_path)
+    if response_text is None and isinstance(payload, dict):
+        for fallback in ("response", "output", "content", "message", "text"):
+            response_text = payload.get(fallback)
+            if response_text:
+                break
+    return (
+        response_text
+        if isinstance(response_text, str)
+        else json.dumps(response_text or "", ensure_ascii=False)
+    )
+```
+
+Create an ASSERT smoke from a known-good eval dataset row, not a random general
+question. For the HTTP tutorial, use:
+
+```yaml
+suite: gpt-rag-http-smoke
+run: local-http-contract-smoke
+
+default_model:
+  name: azure/chat
+
+pipeline:
+  systematize:
+    enabled: false
+  test_set:
+    enabled: false
+  inference:
+    test_set_path: test_set.jsonl
+    target:
+      callable: assert_http_adapter:target
+    max_turns: 1
+  judge:
+    taxonomy_path: taxonomy.json
+    preset:
+      - grounding
+```
+
+Append this `assert:` block to `agentops.yaml`. Discover `AZURE_API_BASE` from
+the Azure AI/OpenAI resource and set `AZURE_API_VERSION` to the version used by
+the deployment. These are not secrets. If local auth is disabled, AgentOps will
+use the signed-in Azure CLI token for the ASSERT subprocess.
+
+```yaml
+assert:
+  config: ./assert/eval_config.yaml
+  fail_on_violations: true
+  env:
+    AZURE_API_BASE: https://<azure-ai-resource>.cognitiveservices.azure.com/
+    AZURE_API_VERSION: 2024-12-01-preview
+    AGENTOPS_ASSERT_AZURE_MAX_COMPLETION_TOKENS: "true"
+    PYTHONPATH: .agentops
+```
+
 **3. Append the `assert:` block to `agentops.yaml`** (preserve every existing
 key — read the file, append the block if missing, write back):
 

{agentops_accelerator-0.4.3 → agentops_accelerator-0.4.5}/src/agentops/cli/app.py

@@ -14,13 +14,16 @@ from html import escape as html_escape
 from pathlib import Path
 from textwrap import wrap
 from collections.abc import Sequence
-from typing import Annotated, Any, Optional
+from typing import Annotated, Any, Optional, TYPE_CHECKING
 
 import typer
 
 from agentops.utils.colors import style
 from agentops.utils.logging import get_logger, setup_logging
 
+if TYPE_CHECKING:
+    from agentops.core.agentops_config import AgentOpsConfig
+
 app = typer.Typer(
     name="agentops",
     help="AgentOps - standardized evaluation workflows for AI projects.",
@@ -1574,11 +1577,13 @@ def cmd_init(
     from agentops.services.setup_wizard import (
         AGENT_TITLE,
         DATASET_TITLE,
+        ENDPOINT_SOURCE_AZD_RESOURCE_DISCOVERY,
         PROJECT_ENDPOINT_TITLE,
         REQUIRED_CONFIGURATION_MESSAGE,
         WizardAnswers,
         apply_answers,
         discover_defaults,
+        is_placeholder_agent,
         run_wizard,
         validate_agent,
         validate_dataset,
@@ -1763,12 +1768,14 @@ def cmd_init(
         force_prompt_fields = {"agent", "dataset"} if config_seeded_this_run else set()
         prompt_values = [
             defaults.project_endpoint,
-            defaults.agent,
+            None if is_placeholder_agent(defaults.agent) else defaults.agent,
             defaults.dataset,
         ]
-        will_prompt = reconfigure or bool(force_prompt_fields) or any(
-            v is None or not str(v).strip()
-            for v in prompt_values
+        will_prompt = (
+            reconfigure
+            or bool(force_prompt_fields)
+            or any(v is None or not str(v).strip() for v in prompt_values)
+            or defaults.project_endpoint_source == ENDPOINT_SOURCE_AZD_RESOURCE_DISCOVERY
         )
         if will_prompt:
             typer.echo(style("Press Enter to accept the value in brackets.", "dim"))
@@ -1817,6 +1824,7 @@ def cmd_init(
             workspace,
             prompt=_prompt,
             echo=_wizard_echo,
+            defaults=defaults,
             on_answer=_on_answer,
             reconfigure=reconfigure,
             force_prompt_fields=force_prompt_fields,
@@ -2121,8 +2129,13 @@ def cmd_eval_init(
     if _maybe_explain_leaf(("eval", "init"), explain):
         return
 
+    from agentops.core.config_loader import load_agentops_config
     from agentops.pipeline.azd_runner import AzdBackendError
-    from agentops.services.azd_eval_init import run_azd_eval_init
+    from agentops.services.azd_eval_init import (
+        ensure_local_evaluator_model_env,
+        recommend_evaluators_for_config,
+        run_azd_eval_init,
+    )
 
     workspace = directory.resolve()
     config_path = _resolve_eval_config_path(config)
@@ -2130,8 +2143,48 @@ def cmd_eval_init(
         config_path = workspace / config_path
 
     try:
+        loaded_config = load_agentops_config(config_path)
+        target = loaded_config.resolved_target()
+        if target.kind not in {"foundry_prompt", "foundry_hosted"}:
+            selection = recommend_evaluators_for_config(
+                config_path=config_path,
+                dataset=dataset,
+            )
+            typer.echo(
+                f"{_cli_label('AgentOps eval init')}: local HTTP/model target detected; "
+                "azd eval assets are not required."
+            )
+            typer.echo(f"{_cli_label('Evaluator recommendation')}: {selection.source}")
+            for signal in selection.signals:
+                typer.echo(f" {style('-', 'dim')} {signal}")
+            if selection.names:
+                typer.echo(f"{_cli_label('Evaluators')}: {', '.join(selection.names)}")
+            model_env = ensure_local_evaluator_model_env(
+                workspace=workspace,
+                selection=selection,
+            )
+            if model_env.configured:
+                action = "configured" if model_env.changed_keys else "using"
+                typer.echo(
+                    f"{_cli_label('Evaluator model')}: {action} "
+                    f"{model_env.deployment} ({model_env.model})"
+                )
+                if model_env.changed_keys and model_env.env_path is not None:
+                    typer.echo(
+                        f" {style('-', 'dim')} saved "
+                        f"{', '.join(model_env.changed_keys)} to "
+                        f"{_cli_path(model_env.env_path)}"
+                    )
+            elif selection.names and model_env.source != "not needed":
+                typer.echo(
+                    f"{_cli_warn('Warning')}: could not auto-discover an evaluator "
+                    "model deployment. Set AZURE_OPENAI_DEPLOYMENT and "
+                    "AZURE_OPENAI_MODEL_NAME before `agentops eval run`."
+                )
+            typer.echo(f"{_cli_label('Next')}: {_cli_command('agentops eval run')}")
+            return
         typer.echo(
-            f"{_cli_label('azd eval init')}: checking/generating eval.yaml "
+            f"{_cli_label('azd eval generate')}: checking/generating eval.yaml "
             "(this can take a few minutes on the first run)"
         )
         result = run_azd_eval_init(
@@ -2148,9 +2201,9 @@ def cmd_eval_init(
         raise typer.Exit(code=1) from exc
 
     if result.command_ran:
-        typer.echo(f"{_cli_label('azd eval init')}: completed")
+        typer.echo(f"{_cli_label('azd eval generate')}: completed")
     else:
-        typer.echo(f"{_cli_label('azd eval init')}: existing recipe reused")
+        typer.echo(f"{_cli_label('azd eval generate')}: existing recipe reused")
     if result.evaluators:
         typer.echo(f"{_cli_label('Evaluator recommendation')}: {result.evaluator_source}")
         for signal in result.evaluator_signals:
@@ -2346,6 +2399,17 @@ def cmd_assert_run(
             ),
         ),
     ] = False,
+    cached: Annotated[
+        bool,
+        typer.Option(
+            "--cached",
+            help=(
+                "Reuse cached inference/judge rows from a previous run with the "
+                "same run id. By default ASSERT re-runs inference against the live "
+                "target each time so the gate always exercises the current agent."
+            ),
+        ),
+    ] = False,
     explain: Annotated[str | None, typer.Argument(hidden=True)] = None,
 ) -> None:
     """Invoke the ASSERT (assert-ai) CLI and normalize its results."""
@@ -2403,6 +2467,7 @@ def cmd_assert_run(
     resolved_suite: str | None = suite
     resolved_run_id: str | None = run_id
     fail_on_violations = True
+    subprocess_env: dict[str, str] | None = None
 
     if cfg.assert_run is not None:
         if eval_config_path is None:
@@ -2414,6 +2479,7 @@ def cmd_assert_run(
         if resolved_run_id is None:
             resolved_run_id = cfg.assert_run.run_id
         fail_on_violations = cfg.assert_run.fail_on_violations
+        subprocess_env = dict(cfg.assert_run.env)
     if no_gate:
         fail_on_violations = False
 
@@ -2428,6 +2494,12 @@ def cmd_assert_run(
         typer.echo(
             f"  suite={resolved_suite or '<auto>'} run_id={resolved_run_id or '<auto>'}"
         )
+    if cached:
+        typer.echo("  cache: reusing prior inference/judge rows when available")
+    else:
+        typer.echo("  cache: forcing fresh inference against the live target")
+
+    assert_extra_args = None if cached else ["--force-stage", "inference"]
 
     try:
         result = run_assert(
@@ -2436,6 +2508,8 @@ def cmd_assert_run(
             results_dir=resolved_results_dir,
             suite=resolved_suite,
             run_id=resolved_run_id,
+            env=subprocess_env,
+            extra_args=assert_extra_args,
         )
     except AssertRunnerError as exc:
         typer.echo(f"{_cli_error('Error')}: {exc}", err=True)
@@ -2471,9 +2545,15 @@ def cmd_assert_run(
             violations = bucket.get("violations", 0)
             total = bucket.get("total", 0)
             skipped = bucket.get("skipped", 0)
-            marker = _cli_ok("OK") if violations == 0 else _cli_error("VIOLATIONS")
             suffix = f" (skipped={skipped})" if skipped else ""
-            typer.echo(f"  {name}: {violations}/{total}{suffix} {marker}")
+            if violations == 0:
+                clean = max(total - skipped, 0)
+                typer.echo(f"  {name}: {clean}/{total} clean{suffix} {_cli_ok('OK')}")
+            else:
+                typer.echo(
+                    f"  {name}: {violations}/{total} violating{suffix} "
+                    f"{_cli_error('VIOLATIONS')}"
+                )
 
     typer.echo("")
     typer.echo(_cli_heading("Inspect details"))
@@ -2666,6 +2746,7 @@ def cmd_redteam_run(
                 err=True,
             )
             raise typer.Exit(code=1)
+    _apply_http_redteam_defaults(resolved_target, cfg)
 
     if output_path is not None and not output_path.is_absolute():
         output_path = (workspace / output_path).resolve()
@@ -2786,6 +2867,21 @@ def _derive_redteam_target_from_agent(agent: str | None) -> dict[str, Any]:
     return {"agent": agent}
 
 
+def _apply_http_redteam_defaults(target: dict[str, Any], cfg: AgentOpsConfig) -> None:
+    if "endpoint" not in target:
+        return
+    if cfg.request_field:
+        target.setdefault("request_field", cfg.request_field)
+    if cfg.response_field:
+        target.setdefault("response_field", cfg.response_field)
+    if cfg.response_mode:
+        target.setdefault("response_mode", cfg.response_mode)
+    if cfg.headers:
+        target.setdefault("headers", cfg.headers)
+    if cfg.stream:
+        target.setdefault("stream", cfg.stream.model_dump(exclude_none=True))
+
+
 def _run_flat_schema_eval(
     *,
     config_path: Path,

{agentops_accelerator-0.4.3 → agentops_accelerator-0.4.5}/src/agentops/core/agentops_config.py

@@ -39,6 +39,14 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator, model_valida
 #: Wire protocol for hosted / HTTP targets.
 Protocol = Literal["responses", "invocations", "http-json"]
 
+#: How AgentOps reads an ``http-json`` response body.
+#:
+#: - ``json`` (default): parse a single JSON document. Preserves the exact
+#:   behavior of every existing ``http_json`` config.
+#: - ``sse``: parse a Server-Sent Events body, concatenating ``data:`` lines.
+#: - ``text``: concatenate a raw streamed text body.
+ResponseMode = Literal["json", "sse", "text"]
+
 #: How thresholds compare against measured metric values.
 Criteria = Literal[">=", ">", "<=", "<", "==", "true", "false"]
 
@@ -454,6 +462,14 @@ class AssertRunConfig(BaseModel):
             "results without gating the pipeline."
         ),
     )
+    env: Dict[str, str] = Field(
+        default_factory=dict,
+        description=(
+            "Optional non-secret environment variables passed only to the "
+            "assert-ai subprocess, for example AZURE_API_BASE or "
+            "AZURE_API_VERSION."
+        ),
+    )
 
     model_config = ConfigDict(extra="forbid")
 
@@ -540,6 +556,41 @@ class RedTeamRunConfig(BaseModel):
     model_config = ConfigDict(extra="forbid")
 
 
+class StreamConfig(BaseModel):
+    """Streaming aggregation options for ``http-json`` targets.
+
+    Only meaningful when ``response_mode`` is ``"sse"`` or ``"text"``. These
+    fields control how a streamed response body is parsed and reassembled into
+    a single answer string.
+    """
+
+    text_field: Optional[str] = Field(
+        None,
+        description=(
+            "For SSE where each 'data:' line is a JSON object, the dotted path "
+            "to the token text (e.g. 'delta.content'). Omit (None) when each "
+            "'data:' line is already raw text."
+        ),
+    )
+    done_marker: Optional[str] = Field(
+        None,
+        description=(
+            "Optional sentinel (e.g. '[DONE]'); aggregation stops when a "
+            "'data:' line equals this value."
+        ),
+    )
+    strip_leading_token: bool = Field(
+        False,
+        description=(
+            "Drop the first whitespace-delimited token from the aggregated "
+            "text. Use for endpoints that prefix the stream with an id (e.g. "
+            "the gpt-rag orchestrator emits '<conversation_id> ' first)."
+        ),
+    )
+
+    model_config = ConfigDict(extra="forbid")
+
+
 # ---------------------------------------------------------------------------
 # Top-level config
 # ---------------------------------------------------------------------------
@@ -600,7 +651,20 @@ class AgentOpsConfig(BaseModel):
 
     ``headers`` / ``auth_header_env``
         Optional HTTP request configuration for ``http-json`` and
-        ``invocations`` targets.
+        ``invocations`` targets. ``auth_header_env`` names the environment
+        variable that holds the secret.
+
+    ``response_mode`` / ``stream``
+        ``http-json`` only. ``response_mode`` selects how the response body is
+        read: ``"json"`` (default, single JSON document), ``"sse"`` (Server-Sent
+        Events), or ``"text"`` (raw streamed text). The ``stream`` block tunes
+        SSE/text aggregation (``text_field``, ``done_marker``,
+        ``strip_leading_token``).
+
+    ``auth_header_name`` / ``auth_value_template``
+        ``http-json`` only. Decouple the auth header from the default
+        ``Authorization: Bearer {token}``. For a shared-secret endpoint set
+        ``auth_header_name: X-API-KEY`` and ``auth_value_template: "{token}"``.
 
     ``evaluators``
         Optional escape hatch: explicit list of evaluator names that overrides
@@ -696,6 +760,37 @@ class AgentOpsConfig(BaseModel):
     tool_calls_field: Optional[str] = None
     headers: Dict[str, str] = Field(default_factory=dict)
     auth_header_env: Optional[str] = None
+    response_mode: ResponseMode = Field(
+        "json",
+        description=(
+            "How to read the HTTP/JSON response body. 'json' (default) parses "
+            "a single JSON document and preserves existing behavior. 'sse' "
+            "parses Server-Sent Events 'data:' lines; 'text' concatenates a "
+            "raw streamed text body. Only valid for http-json targets."
+        ),
+    )
+    stream: Optional[StreamConfig] = Field(
+        None,
+        description=(
+            "Streaming aggregation options, used only when response_mode is "
+            "'sse' or 'text'."
+        ),
+    )
+    auth_header_name: Optional[str] = Field(
+        None,
+        description=(
+            "HTTP header that carries the secret read from auth_header_env. "
+            "Defaults to 'Authorization' when unset."
+        ),
+    )
+    auth_value_template: Optional[str] = Field(
+        None,
+        description=(
+            "Template for the auth header value; '{token}' is replaced by the "
+            "auth_header_env value. Defaults to 'Bearer {token}' when unset. "
+            "Use '{token}' alone for a raw shared-secret header like X-API-KEY."
+        ),
+    )
 
     evaluators: Optional[List[EvaluatorOverride]] = None
     rubrics: List[RubricConfig] = Field(
@@ -850,6 +945,10 @@ class AgentOpsConfig(BaseModel):
             or self.tool_calls_field
             or self.headers
             or self.auth_header_env
+            or self.response_mode != "json"
+            or self.stream is not None
+            or self.auth_header_name
+            or self.auth_value_template
         ):
             # Foundry hosted (responses/invocations) defines its own wire
             # format. HTTP-only request/response shaping is invalid there.
@@ -859,8 +958,9 @@ class AgentOpsConfig(BaseModel):
             else:
                 raise ValueError(
                     "request_field / response_field / tool_calls_field / "
-                    "headers / auth_header_env are only valid for HTTP/JSON "
-                    "or Foundry hosted (invocations) targets"
+                    "headers / auth_header_env / response_mode / stream / "
+                    "auth_header_name / auth_value_template are only valid for "
+                    "HTTP/JSON or Foundry hosted (invocations) targets"
                 )
         return self