agentops-accelerator 0.4.2__tar.gz → 0.4.4__tar.gz

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/.github/copilot-instructions.md

@@ -19,6 +19,7 @@ Read these first for detailed architecture and product context:
 - `agentops explain [COMMAND...] [--no-pager] [--format text|markdown|html] [--out PATH] [--open]`
 - `agentops init [--force] [--dir PATH] [--no-prompt] [--no-appinsights] [--azd-env NAME] [--project-endpoint URL] [--agent REF] [--dataset PATH] [--appinsights-connection-string STR]`
 - `agentops init show [--dir PATH] [--reveal-secrets]`
+- `agentops prompt pull [--config PATH] [--out PATH] [--project-endpoint URL] [--force] [--update-config|--no-update-config]`
 - `agentops eval analyze [--dir PATH] [--format text|markdown|json] [--out PATH]`
 - `agentops eval run [--config PATH] [--baseline PATH] [--output DIR]`
 - `agentops eval promote-traces --source PATH [--out PATH] [--max-rows N] [--label-mode self-similarity|pending] [--apply]`

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/AGENTS.md

@@ -40,6 +40,7 @@ Public CLI contract:
 - `agentops init [--force] [--dir PATH] [--no-prompt] [--no-appinsights] [--azd-env NAME] [--project-endpoint URL] [--agent REF] [--dataset PATH] [--appinsights-connection-string STR]`
 - `agentops init show [--dir PATH] [--reveal-secrets]`
 - `agentops init explain [--no-pager] [--format text|markdown|html] [--out PATH] [--open]`
+- `agentops prompt pull [--config PATH] [--out PATH] [--project-endpoint URL] [--force] [--update-config|--no-update-config]`
 - `agentops eval analyze [--dir PATH] [--format text|markdown|json] [--out PATH]`
 - `agentops eval init [--dir PATH] [--force]`
 - `agentops eval run [--config PATH] [--baseline PATH] [--output DIR]`
@@ -222,9 +223,9 @@ Coverage highlights:
 docs/
 ├── concepts.md                                # Core concepts and evaluation scenarios
 ├── how-it-works.md                            # Architecture and request flow
-├── tutorial-prompt-agent-quickstart.md        # Tutorial for Foundry Prompt Agents
-├── tutorial-hosted-agent-quickstart.md        # Tutorial for Foundry hosted / HTTP agents
-├── tutorial-end-to-end.md                     # End-to-end release-readiness tutorial (Foundry + AgentOps)
+├── tutorial-prompt-agent.md                   # Tutorial for Foundry Prompt Agents
+├── tutorial-hosted-agent.md                   # Tutorial for Foundry hosted / HTTP agents
+├── tutorial-end-to-end.md                     # Full end-to-end release-readiness tutorial (Foundry + AgentOps)
 ├── doctor-explained.md                        # Doctor checks, history, and operations model
 ├── ci-github-actions.md                       # CI/CD setup
 ├── release-process.md                         # Release and versioning
@@ -334,7 +335,11 @@ Full schema:
 | `response_field` | no | HTTP / invocations only. Dot-path to extract the response text (default: `text`). |
 | `tool_calls_field` | no | HTTP / invocations only. Dot-path to extract tool calls for agent-workflow evaluators. |
 | `headers` | no | HTTP / invocations only. Static extra HTTP headers. |
-| `auth_header_env` | no | HTTP / invocations only. Environment variable that holds a Bearer token. |
+| `auth_header_env` | no | HTTP / invocations only. Environment variable that holds the auth token. |
+| `auth_header_name` | no | HTTP / invocations only. Header name for the auth token (default: `Authorization`). |
+| `auth_value_template` | no | HTTP / invocations only. Template for the auth header value; `{token}` is replaced by the `auth_header_env` value (default: `Bearer {token}`). |
+| `response_mode` | no | HTTP / invocations only. `json` (default, single `json.loads` of the body), `sse` (parse `data:` lines), or `text` (concatenate raw streamed text). |
+| `stream` | no | HTTP / invocations only. Streaming aggregation block for `response_mode: sse|text`: `text_field` (dot-path to token text in JSON `data:` lines), `done_marker` (stop token, e.g. `[DONE]`), `strip_leading_token` (drop the first whitespace-delimited token, e.g. a `conversation_id` prefix). |
 | `assert_path` | no | Optional ASSERT policy/results file or directory referenced by Doctor/evidence. AgentOps does not execute ASSERT. |
 | `acs_path` | no | Optional Agent Control Specification contract file or directory referenced by Doctor/evidence. AgentOps does not apply ACS controls. |
 | `redteam_path` | no | Optional red-team plan/results evidence path. AgentOps records metadata and never exposes payload text. |

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/CHANGELOG.md

@@ -5,21 +5,56 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
 
 ## [Unreleased]
 
-## [0.4.2] - 2026-06-17
+## [0.4.4] - 2026-06-18
 
-### Fixed
-- **`agentops eval init` now works with both old and new `azure.ai.agents` azd
-  extensions.** Version 0.1.40 of the extension renamed the eval subcommand from
-  `azd ai agent eval init` to `azd ai agent eval generate`, which made
-  `agentops eval init` hard-fail with `Command "init" is deprecated, use 'azd ai
-  agent eval generate' instead`. AgentOps now invokes `generate` first and
-  transparently falls back to the legacy `init` subcommand when an older
-  extension does not recognise `generate`. The fallback only triggers on
-  subcommand-name/deprecation errors; genuine failures (authentication, project
-  endpoint, timeouts) are still surfaced immediately and unchanged. All
-  previously passed flags (`--project-endpoint`, `--agent`,
-  `--gen-instruction-file`, `--eval-model`, `--dataset`, `--evaluator`) and the
-  recipe discovery/persistence behaviour are preserved.
+### Added
+- **Streaming HTTP targets.** The `http_json` target now understands streaming
+  responses so AgentOps can evaluate SSE/streaming agents (such as the
+  gpt-rag-orchestrator `/orchestrator` endpoint) directly, without a manual
+  adapter. A new `response_mode: json|sse|text` field selects the response
+  parser (`json` is the default and preserves the existing single-`json.loads`
+  behavior exactly). For `sse`/`text`, an optional `stream` block configures
+  aggregation: `text_field` (dotted path to the token text when each SSE
+  `data:` line is JSON), `done_marker` (stop token, e.g. `[DONE]`), and
+  `strip_leading_token` (drop the leading whitespace-delimited token, e.g. the
+  orchestrator's `conversation_id` prefix). The auth header is now configurable
+  via `auth_header_name` (default `Authorization`) and `auth_value_template`
+  (default `Bearer {token}`, where `{token}` is replaced by the
+  `auth_header_env` value), so targets gated by a shared secret such as
+  `X-API-KEY` are supported without hardcoding the secret in `agentops.yaml`.
+  Streaming uses the same stdlib (`urllib`) transport and 3-try backoff as the
+  JSON path. When a JSON parse fails on a `text/event-stream` response, the
+  error now suggests setting `response_mode: sse|text`.
+
+## [0.4.3] - 2026-06-17
+
+### Added
+- **Prompt-agent tutorials no longer require manual portal copy/paste.**
+  `agentops prompt pull` reads the configured Foundry prompt agent
+  (`agent: name:version`), validates that the Foundry definition is actually a
+  prompt agent, and writes the reviewed Sandbox instructions to
+  `.agentops/prompts/<agent-name>.prompt.md` by default. Before writing, the CLI
+  prints the resolved agent, endpoint, endpoint source, and destination file so
+  operators can catch the wrong environment early. Changed prompt files are
+  protected by default and require `--force` to overwrite reviewed local edits.
+  The command updates `prompt_file` in `agentops.yaml` unless
+  `--no-update-config` is passed, and it can resolve the endpoint from
+  `--project-endpoint`, `agentops.yaml`, `AZURE_AI_FOUNDRY_PROJECT_ENDPOINT`, or
+  the active `.azure/<env>/.env`. The prompt-agent tutorial and packaged
+  `agentops-eval` skill now use this command instead of a manual here-string.
+  ([#322](https://github.com/Azure/agentops/issues/322))
+
+### Changed
+- **`agentops eval init` now recommends evaluators from the agent and dataset
+  shape.** The azd bootstrap path now reuses the same AgentOps evaluator
+  catalog as `agentops eval run`: free-form answer datasets get answer-quality
+  checks, RAG-shaped datasets get groundedness / relevance / retrieval checks,
+  and tool-use datasets get tool-call / intent / task-adherence checks while
+  avoiding literal-answer similarity metrics. Explicit `evaluators:` entries in
+  `agentops.yaml` still win. The CLI prints the recommendation source, detected
+  signals, and selected azd built-ins before reporting the generated
+  `eval.yaml`, so users can see why those evaluators were chosen.
+  ([#323](https://github.com/Azure/agentops/issues/323))
 
 ## [0.4.1] - 2026-06-15
 

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentops-accelerator
-Version: 0.4.2
+Version: 0.4.4
 Summary: Release readiness gates and evidence for Microsoft Foundry agents
 License: MIT License
         
@@ -81,11 +81,16 @@ AgentOps Accelerator helps Microsoft Foundry agent teams evaluate quality, prepa
 ```powershell
 python -m pip install agentops-accelerator
 agentops init
-agentops eval analyze
-agentops eval run
-agentops doctor --evidence-pack
 ```
 
+`agentops init` starts a guided setup that creates your `agentops.yaml` and
+`.agentops/` workspace.
+
+Next, follow the tutorial that matches your agent type:
+
+- [Prompt Agent tutorial](https://azure.github.io/agentops/tutorial-prompt-agent/)
+- [Hosted or HTTP Agent tutorial](https://azure.github.io/agentops/tutorial-hosted-agent/)
+
 ## What it helps you do
 
 Use AgentOps Accelerator when you need to:

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/README.md

@@ -28,11 +28,16 @@ AgentOps Accelerator helps Microsoft Foundry agent teams evaluate quality, prepa
 ```powershell
 python -m pip install agentops-accelerator
 agentops init
-agentops eval analyze
-agentops eval run
-agentops doctor --evidence-pack
 ```
 
+`agentops init` starts a guided setup that creates your `agentops.yaml` and
+`.agentops/` workspace.
+
+Next, follow the tutorial that matches your agent type:
+
+- [Prompt Agent tutorial](https://azure.github.io/agentops/tutorial-prompt-agent/)
+- [Hosted or HTTP Agent tutorial](https://azure.github.io/agentops/tutorial-hosted-agent/)
+
 ## What it helps you do
 
 Use AgentOps Accelerator when you need to:

agentops_accelerator-0.4.2/docs/tutorial-prompt-agent-quickstart.md → agentops_accelerator-0.4.4/docs/tutorial-prompt-agent.md

@@ -130,7 +130,7 @@ cd agentops-prompt-quickstart
 python -m venv .venv
 .\.venv\Scripts\Activate.ps1
 python -m pip install -U pip
-python -m pip install "agentops-accelerator[foundry,agent]"
+python -m pip install agentops-accelerator
 agentops --version
 ```
 
@@ -139,7 +139,7 @@ path, install the aligned reference branch so the CLI, generated
 workflows, and tutorial steps stay in sync:
 
 ```powershell
-python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@develop"
+python -m pip install "agentops-accelerator @ git+https://github.com/Azure/agentops.git@develop"
 ```
 
 ## 2. Install the AgentOps Copilot skills
@@ -426,7 +426,7 @@ In the **sandbox** project only:
 > next-version flow.
 
 > **Prompt-as-code captures only the instructions.** Later in the
-> tutorial you will commit `.agentops/prompts/travel-agent.md` to git
+> tutorial you will commit `.agentops/prompts/travel-agent.prompt.md` to git
 > and let CI use it as the prompt source. That file does not capture
 > the model deployment, parameters (temperature, top-p), tools, or
 > other agent settings — those come from `prompt_agent_bootstrap` on
@@ -709,31 +709,31 @@ evaluates and deploys comes from this file in git, not from manual edits
 in the Foundry portal.
 
 ```powershell
-New-Item -ItemType Directory -Force .agentops\prompts | Out-Null
-@'
-You are Travel Agent, a concise travel planning assistant.
+agentops prompt pull
+```
 
-Help users plan short leisure trips. Always include:
-- a short summary;
-- a day-by-day plan when the user asks for an itinerary;
-- practical notes about budget, transit, weather, or booking constraints;
-- a reminder that you cannot make live reservations or purchases.
+AgentOps reads `agent: travel-agent:2`, resolves the current Foundry
+endpoint, validates that the Foundry definition is a prompt agent, and
+writes the instructions to `.agentops/prompts/travel-agent.prompt.md`.
+Before writing, it prints the resolved agent, endpoint, endpoint source,
+and destination file so you can catch the wrong environment before the
+prompt is saved.
 
-Ask one clarifying question only when the destination, duration, or
-traveler preference is missing. Do not invent booking confirmations,
-prices, or availability.
-'@ | Set-Content -Encoding utf8 .agentops\prompts\travel-agent.md
-```
+By default, the command refuses to overwrite a changed prompt file. Use
+`--force` only when you intentionally want to replace reviewed local
+prompt edits with the current Sandbox version. Use `--out <path>` if you
+need a different file name, but keep prompt source under
+`.agentops/prompts/` unless your repository has a stronger convention.
 
-Then tell `agentops.yaml` where to find the file and add
-`prompt_agent_bootstrap` so CI can auto-create the agent in dev (and
-later qa / prod) on the first deploy:
+Then add `prompt_agent_bootstrap` so CI can auto-create the agent in dev
+(and later qa / prod) on the first deploy. `agentops prompt pull` writes
+`prompt_file` for you when it is missing:
 
 ```yaml
 version: 1
 agent: travel-agent:2
 dataset: .agentops/data/travel-smoke.jsonl
-prompt_file: .agentops/prompts/travel-agent.md
+prompt_file: .agentops/prompts/travel-agent.prompt.md
 prompt_agent_bootstrap:
   model: gpt-4o-mini
   description: "Helps plan short trips and explains tradeoffs."

{agentops_accelerator-0.4.2/src/agentops/templates → agentops_accelerator-0.4.4/plugins/agentops}/skills/agentops-eval/SKILL.md

@@ -93,7 +93,7 @@ done
 ```
 
 PowerShell equivalent: replace `$(...)` with the PowerShell variable
-assignments shown in `docs/tutorial-prompt-agent-quickstart.md`.
+assignments shown in `docs/tutorial-prompt-agent.md`.
 
 If the user has not run `az login` yet, do that first. If
 `az cognitiveservices account list` returns an empty RG, the AI Services
@@ -143,6 +143,23 @@ for the `agent:` field of `agentops.yaml`:
 
 If nothing is found, ask the user once for the agent identifier.
 
+For Foundry Prompt Agents authored in the Sandbox portal, do not copy/paste the
+instructions into a file manually. After `agentops.yaml` contains `agent:
+name:version` and the correct project endpoint is available from
+`agentops.yaml`, `AZURE_AI_FOUNDRY_PROJECT_ENDPOINT`, or the active
+`.azure/<env>/.env`, run:
+
+```bash
+agentops prompt pull
+```
+
+This writes `.agentops/prompts/<agent-name>.prompt.md` by default, updates
+`prompt_file` in `agentops.yaml` when needed, prints the resolved endpoint and
+agent version before writing, validates that the Foundry definition is a prompt
+agent, and refuses to overwrite changed prompt files unless `--force` is used.
+Use `--out` only when the repository already has a stronger prompt-file
+convention.
+
 ## Step 3 - Make sure the dataset exists
 
 `agentops.yaml` points to a JSONL file (default
@@ -284,4 +301,11 @@ dataset reference.
   ```
 - For HTTP/JSON agents that need auth, set
   `auth_header_env: MY_TOKEN_VAR` and AgentOps adds
-  `Authorization: Bearer $MY_TOKEN_VAR`.
+  `Authorization: Bearer $MY_TOKEN_VAR`. For a shared-secret gate, override the
+  header with `auth_header_name: X-API-KEY` and `auth_value_template: "{token}"`.
+- For streaming HTTP agents (e.g. an SSE `text/event-stream` endpoint), set
+  `response_mode: sse` (each `data:` line) or `response_mode: text` (raw
+  streamed text). Use the optional `stream:` block to tune aggregation:
+  `text_field` (dot-path to the token text when `data:` lines are JSON),
+  `done_marker` (e.g. `[DONE]`), and `strip_leading_token: true` (drop a leading
+  `conversation_id` prefix). `response_mode: json` (default) is unchanged.

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/src/agentops/cli/app.py

@@ -36,6 +36,7 @@ eval_app = typer.Typer(
 report_app = typer.Typer(help="Reporting commands.")
 workflow_app = typer.Typer(help="CI/CD workflow commands.")
 skills_app = typer.Typer(help="Coding agent skills management.")
+prompt_app = typer.Typer(help="Foundry prompt-agent source control commands.")
 mcp_app = typer.Typer(help="MCP (Model Context Protocol) server commands.")
 agent_app = typer.Typer(
     help=(
@@ -79,6 +80,7 @@ app.add_typer(eval_app, name="eval")
 app.add_typer(report_app, name="report")
 app.add_typer(workflow_app, name="workflow")
 app.add_typer(skills_app, name="skills")
+app.add_typer(prompt_app, name="prompt")
 app.add_typer(mcp_app, name="mcp")
 app.add_typer(agent_app, name="agent")
 app.add_typer(doctor_app, name="doctor")
@@ -485,7 +487,7 @@ EXPLAIN_PAGES: dict[tuple[str, ...], ExplainPage] = {
             "agentops explain eval run --open",
             "agentops explain cockpit --format markdown --out cockpit.md",
         ),
-        children=("init", "eval", "report", "workflow", "skills", "mcp", "agent", "doctor", "cockpit", "assert", "redteam"),
+        children=("init", "eval", "report", "workflow", "skills", "prompt", "mcp", "agent", "doctor", "cockpit", "assert", "redteam"),
     ),
     ("init",): ExplainPage(
         title="Initialize workspace and configure endpoints",
@@ -609,6 +611,7 @@ EXPLAIN_PAGES: dict[tuple[str, ...], ExplainPage] = {
         ),
         how_it_works=(
             "Checks that azd and the `azure.ai.agents` extension are available.",
+            "Reads `agentops.yaml`, classifies the target, samples the dataset shape, and recommends built-in azd evaluators from the same AgentOps catalog used by `agentops eval run`. Explicit `evaluators:` entries in `agentops.yaml` still win.",
             "Runs `azd ai agent eval init` in the workspace unless an eval recipe already exists.",
             "Discovers the generated `eval.yaml` and writes `eval_recipe:` plus `execution: azd` to `agentops.yaml`.",
         ),
@@ -617,6 +620,55 @@ EXPLAIN_PAGES: dict[tuple[str, ...], ExplainPage] = {
         examples=("agentops eval init", "agentops eval init --dataset .agentops/data/golden.jsonl", "agentops eval init --force"),
         see_also=("agentops explain eval run", "agentops explain doctor"),
     ),
+    ("prompt",): ExplainPage(
+        title="Prompt-agent source control commands",
+        command="agentops prompt",
+        synopsis=("agentops prompt pull [--config PATH] [--out PATH] [--force]", "agentops prompt explain"),
+        summary=(
+            "Pulls tested Foundry prompt-agent instructions into a source-controlled prompt file.",
+            "Use this after iterating in the Foundry Sandbox so git becomes the source of truth without manual portal copy/paste.",
+        ),
+        children=("pull",),
+        examples=("agentops prompt pull", "agentops prompt pull --out .agentops/prompts/travel-agent.prompt.md"),
+        see_also=("agentops explain workflow generate", "agentops explain eval run"),
+    ),
+    ("prompt", "pull"): ExplainPage(
+        title="Pull a Foundry prompt into git",
+        command="agentops prompt pull",
+        synopsis=(
+            "agentops prompt pull [--config agentops.yaml] [--project-endpoint URL]",
+            "agentops prompt pull [--out .agentops/prompts/<agent-name>.prompt.md] [--force]",
+            "agentops prompt pull [--no-update-config]",
+        ),
+        summary=(
+            "Fetches the `agent: name:version` Foundry prompt-agent definition from the configured project and writes its instructions to a local `.prompt.md` file.",
+            "The command prints the resolved endpoint, endpoint source, agent version, and destination before writing so users can catch wrong-environment pulls.",
+            "It never overwrites changed local prompt files unless `--force` is provided.",
+        ),
+        how_it_works=(
+            "Reads `agentops.yaml` and requires `agent` to be a Foundry prompt agent in `name:version` form.",
+            "Resolves the project endpoint from `--project-endpoint`, then `agentops.yaml project_endpoint`, then `AZURE_AI_FOUNDRY_PROJECT_ENDPOINT`, then the active `.azure/<env>/.env`.",
+            "Fetches the Foundry agent version, validates that `definition.kind` is `prompt`, and extracts `definition.instructions` through the same normalization helpers used by prompt deploy.",
+            "Writes to `prompt_file` when already configured, otherwise to `.agentops/prompts/<agent-name>.prompt.md`.",
+            "Updates `prompt_file` in `agentops.yaml` by default; pass `--no-update-config` to leave config unchanged.",
+        ),
+        inputs=(
+            "`agentops.yaml` with `agent: <name>:<version>`.",
+            "A Foundry project endpoint in `--project-endpoint`, `project_endpoint`, `AZURE_AI_FOUNDRY_PROJECT_ENDPOINT`, or the active `.azure/<env>/.env`.",
+            "Azure credentials that can read the Foundry agent definition.",
+        ),
+        outputs=(
+            "Source-controlled prompt file, default `.agentops/prompts/<agent-name>.prompt.md`.",
+            "`agentops.yaml prompt_file` update unless `--no-update-config` is passed.",
+        ),
+        examples=(
+            "agentops prompt pull",
+            "agentops prompt pull --project-endpoint https://acct.services.ai.azure.com/api/projects/sandbox",
+            "agentops prompt pull --out .agentops/prompts/travel-agent.prompt.md --force",
+            "agentops prompt pull --no-update-config",
+        ),
+        see_also=("agentops explain workflow generate", "agentops explain eval run"),
+    ),
     ("eval", "run"): ExplainPage(
         title="Run evaluation",
         command="agentops eval run",
@@ -1407,6 +1459,7 @@ eval_app.command("explain")(_make_group_explain(("eval",)))
 report_app.command("explain")(_make_group_explain(("report",)))
 workflow_app.command("explain")(_make_group_explain(("workflow",)))
 skills_app.command("explain")(_make_group_explain(("skills",)))
+prompt_app.command("explain")(_make_group_explain(("prompt",)))
 mcp_app.command("explain")(_make_group_explain(("mcp",)))
 agent_app.command("explain")(_make_group_explain(("agent",)))
 
@@ -2098,6 +2151,11 @@ def cmd_eval_init(
         typer.echo(f"{_cli_label('azd eval init')}: completed")
     else:
         typer.echo(f"{_cli_label('azd eval init')}: existing recipe reused")
+    if result.evaluators:
+        typer.echo(f"{_cli_label('Evaluator recommendation')}: {result.evaluator_source}")
+        for signal in result.evaluator_signals:
+            typer.echo(f" {style('-', 'dim')} {signal}")
+        typer.echo(f"{_cli_label('Evaluators')}: {', '.join(result.evaluators)}")
     typer.echo(f"{_cli_label('eval.yaml')}: {_cli_path(result.recipe_path)}")
     if result.config_updated:
         typer.echo(_cli_updated(result.config_path))
@@ -3415,6 +3473,96 @@ def cmd_skills_install(
         _print_registration_result(reg_result)
 
 
+# ---------------------------------------------------------------------------
+# agentops prompt pull
+# ---------------------------------------------------------------------------
+
+
+@prompt_app.command("pull")
+def cmd_prompt_pull(
+    config: Annotated[
+        Path,
+        typer.Option("--config", "-c", help="Path to agentops.yaml."),
+    ] = Path("agentops.yaml"),
+    out: Annotated[
+        Path | None,
+        typer.Option(
+            "--out",
+            "-o",
+            help="Prompt file to write.",
+        ),
+    ] = None,
+    project_endpoint: Annotated[
+        str | None,
+        typer.Option(
+            "--project-endpoint",
+            help="Foundry project endpoint. Overrides config and environment.",
+        ),
+    ] = None,
+    force: Annotated[
+        bool,
+        typer.Option(
+            "--force",
+            help="Overwrite the prompt file when local content differs.",
+        ),
+    ] = False,
+    update_config: Annotated[
+        bool,
+        typer.Option(
+            "--update-config/--no-update-config",
+            help="Update agentops.yaml prompt_file to the pulled prompt path.",
+        ),
+    ] = True,
+    explain: Annotated[str | None, typer.Argument(hidden=True)] = None,
+) -> None:
+    """Pull Foundry prompt-agent instructions into a prompt file."""
+    if _maybe_explain_leaf(("prompt", "pull"), explain):
+        return
+
+    from agentops.pipeline.prompt_deploy import pull_prompt_agent_instructions
+
+    def _before_write(details: dict[str, str]) -> None:
+        typer.echo(f"{_cli_label('Agent')}: {details['agent']}")
+        typer.echo(f"{_cli_label('Endpoint')}: {details['endpoint']}")
+        typer.echo(f"{_cli_label('Endpoint source')}: {details['endpoint_source']}")
+        typer.echo(f"{_cli_label('Prompt file')}: {_cli_path(details['prompt_file'])}")
+        if details["endpoint_source"] == "AZURE_AI_FOUNDRY_PROJECT_ENDPOINT":
+            typer.echo(
+                f"{_cli_warn('Warning')}: using endpoint from environment variable. "
+                "Pass --project-endpoint or set project_endpoint in agentops.yaml "
+                "to make this deterministic.",
+                err=True,
+            )
+
+    try:
+        result = pull_prompt_agent_instructions(
+            config_path=config,
+            output_path=out,
+            force=force,
+            update_config=update_config,
+            project_endpoint=project_endpoint,
+            before_write=_before_write,
+        )
+    except FileExistsError as exc:
+        typer.echo(f"{_cli_error('Error')}: {exc}", err=True)
+        raise typer.Exit(code=1) from exc
+    except (FileNotFoundError, ValueError, RuntimeError) as exc:
+        typer.echo(f"{_cli_error('Error')}: {exc}", err=True)
+        raise typer.Exit(code=1) from exc
+    except Exception as exc:
+        typer.echo(f"{_cli_error('Error')}: failed to pull prompt: {exc}", err=True)
+        raise typer.Exit(code=1) from exc
+
+    if result.action == "created":
+        typer.echo(_cli_created(result.prompt_file))
+    elif result.action == "overwritten":
+        typer.echo(_cli_overwritten(result.prompt_file))
+    else:
+        typer.echo(_cli_skipped(result.prompt_file, " (unchanged)"))
+    if result.config_updated:
+        typer.echo(_cli_updated(result.config_path))
+
+
 # ---------------------------------------------------------------------------
 # agentops mcp serve
 # ---------------------------------------------------------------------------

{agentops_accelerator-0.4.2 → agentops_accelerator-0.4.4}/src/agentops/core/agentops_config.py

@@ -39,6 +39,14 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator, model_valida
 #: Wire protocol for hosted / HTTP targets.
 Protocol = Literal["responses", "invocations", "http-json"]
 
+#: How AgentOps reads an ``http-json`` response body.
+#:
+#: - ``json`` (default): parse a single JSON document. Preserves the exact
+#:   behavior of every existing ``http_json`` config.
+#: - ``sse``: parse a Server-Sent Events body, concatenating ``data:`` lines.
+#: - ``text``: concatenate a raw streamed text body.
+ResponseMode = Literal["json", "sse", "text"]
+
 #: How thresholds compare against measured metric values.
 Criteria = Literal[">=", ">", "<=", "<", "==", "true", "false"]
 
@@ -540,6 +548,41 @@ class RedTeamRunConfig(BaseModel):
     model_config = ConfigDict(extra="forbid")
 
 
+class StreamConfig(BaseModel):
+    """Streaming aggregation options for ``http-json`` targets.
+
+    Only meaningful when ``response_mode`` is ``"sse"`` or ``"text"``. These
+    fields control how a streamed response body is parsed and reassembled into
+    a single answer string.
+    """
+
+    text_field: Optional[str] = Field(
+        None,
+        description=(
+            "For SSE where each 'data:' line is a JSON object, the dotted path "
+            "to the token text (e.g. 'delta.content'). Omit (None) when each "
+            "'data:' line is already raw text."
+        ),
+    )
+    done_marker: Optional[str] = Field(
+        None,
+        description=(
+            "Optional sentinel (e.g. '[DONE]'); aggregation stops when a "
+            "'data:' line equals this value."
+        ),
+    )
+    strip_leading_token: bool = Field(
+        False,
+        description=(
+            "Drop the first whitespace-delimited token from the aggregated "
+            "text. Use for endpoints that prefix the stream with an id (e.g. "
+            "the gpt-rag orchestrator emits '<conversation_id> ' first)."
+        ),
+    )
+
+    model_config = ConfigDict(extra="forbid")
+
+
 # ---------------------------------------------------------------------------
 # Top-level config
 # ---------------------------------------------------------------------------
@@ -600,7 +643,20 @@ class AgentOpsConfig(BaseModel):
 
     ``headers`` / ``auth_header_env``
         Optional HTTP request configuration for ``http-json`` and
-        ``invocations`` targets.
+        ``invocations`` targets. ``auth_header_env`` names the environment
+        variable that holds the secret.
+
+    ``response_mode`` / ``stream``
+        ``http-json`` only. ``response_mode`` selects how the response body is
+        read: ``"json"`` (default, single JSON document), ``"sse"`` (Server-Sent
+        Events), or ``"text"`` (raw streamed text). The ``stream`` block tunes
+        SSE/text aggregation (``text_field``, ``done_marker``,
+        ``strip_leading_token``).
+
+    ``auth_header_name`` / ``auth_value_template``
+        ``http-json`` only. Decouple the auth header from the default
+        ``Authorization: Bearer {token}``. For a shared-secret endpoint set
+        ``auth_header_name: X-API-KEY`` and ``auth_value_template: "{token}"``.
 
     ``evaluators``
         Optional escape hatch: explicit list of evaluator names that overrides
@@ -696,6 +752,37 @@ class AgentOpsConfig(BaseModel):
     tool_calls_field: Optional[str] = None
     headers: Dict[str, str] = Field(default_factory=dict)
     auth_header_env: Optional[str] = None
+    response_mode: ResponseMode = Field(
+        "json",
+        description=(
+            "How to read the HTTP/JSON response body. 'json' (default) parses "
+            "a single JSON document and preserves existing behavior. 'sse' "
+            "parses Server-Sent Events 'data:' lines; 'text' concatenates a "
+            "raw streamed text body. Only valid for http-json targets."
+        ),
+    )
+    stream: Optional[StreamConfig] = Field(
+        None,
+        description=(
+            "Streaming aggregation options, used only when response_mode is "
+            "'sse' or 'text'."
+        ),
+    )
+    auth_header_name: Optional[str] = Field(
+        None,
+        description=(
+            "HTTP header that carries the secret read from auth_header_env. "
+            "Defaults to 'Authorization' when unset."
+        ),
+    )
+    auth_value_template: Optional[str] = Field(
+        None,
+        description=(
+            "Template for the auth header value; '{token}' is replaced by the "
+            "auth_header_env value. Defaults to 'Bearer {token}' when unset. "
+            "Use '{token}' alone for a raw shared-secret header like X-API-KEY."
+        ),
+    )
 
     evaluators: Optional[List[EvaluatorOverride]] = None
     rubrics: List[RubricConfig] = Field(
@@ -850,6 +937,10 @@ class AgentOpsConfig(BaseModel):
             or self.tool_calls_field
             or self.headers
             or self.auth_header_env
+            or self.response_mode != "json"
+            or self.stream is not None
+            or self.auth_header_name
+            or self.auth_value_template
         ):
             # Foundry hosted (responses/invocations) defines its own wire
             # format. HTTP-only request/response shaping is invalid there.
@@ -859,8 +950,9 @@ class AgentOpsConfig(BaseModel):
             else:
                 raise ValueError(
                     "request_field / response_field / tool_calls_field / "
-                    "headers / auth_header_env are only valid for HTTP/JSON "
-                    "or Foundry hosted (invocations) targets"
+                    "headers / auth_header_env / response_mode / stream / "
+                    "auth_header_name / auth_value_template are only valid for "
+                    "HTTP/JSON or Foundry hosted (invocations) targets"
                 )
         return self