hiddenlayer-sdk 3.7.0__tar.gz → 3.8.0__tar.gz

hiddenlayer_sdk-3.8.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "3.8.0"
+}

{hiddenlayer_sdk-3.7.0 → hiddenlayer_sdk-3.8.0}/CHANGELOG.md

@@ -1,5 +1,43 @@
 # Changelog
 
+## 3.8.0 (2026-06-24)
+
+Full Changelog: [v3.7.1...v3.8.0](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/compare/v3.7.1...v3.8.0)
+
+### Features
+
+* **api:** api update ([4fd0d62](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/4fd0d62737e1273a0558bf66017fa054e39d1f87))
+* **api:** api update ([c53a283](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/c53a2836f987b7dac46aa1c5677a945f45d21c2a))
+* **api:** api update ([67dccee](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/67dcceea5e111f16a30688efb33bc0e0f06ca12c))
+* **api:** api update ([8f00ab1](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/8f00ab1201dcda0488ca47abcf4cd4a2d4718da7))
+* **api:** api update ([e612915](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/e6129151ba357559bb291bfe0dc6a3d3e81f3dff))
+* **red-team:** forward all start_session params and drop hardcoded defaults ([#206](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/issues/206)) ([45ee409](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/45ee40985d82db175e48140c2a82b9713ea8a83c))
+
+
+### Bug Fixes
+
+* **client:** add missing f-string prefix in file type error message ([3405322](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/3405322c1f1b379b50d464842607bdfa8c3c080f))
+
+
+### Chores
+
+* **internal:** codegen related update ([bb8153f](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/bb8153fc39bdb5dfc962908caad4e3eaeb9fd249))
+* regenerate beta endpoint registry [skip ci] ([89009da](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/89009dab82686744fb693a7e47de13c01ac9909e))
+* regenerate beta endpoint registry [skip ci] ([c08dc03](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/c08dc0341664260b09dc40e021e5c4fba0a00cf7))
+
+
+### Documentation
+
+* **examples:** add red team evaluation notebooks ([#205](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/issues/205)) ([5a1229b](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/5a1229b5cf3930e3a0b1665d97fdb4ea134b83da))
+
+## 3.7.1 (2026-05-14)
+
+Full Changelog: [v3.7.0...v3.7.1](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/compare/v3.7.0...v3.7.1)
+
+### Bug Fixes
+
+* **red_team_session:** fix red team sessions failing with "invalid UUID length: 0" when prompt_set_id is not provided ([#198](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/issues/198)) ([5e3c85e](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/commit/5e3c85e57482ffeda2011b8f7c65a67efe0a084d))
+
 ## 3.7.0 (2026-05-07)
 
 Full Changelog: [v3.6.0...v3.7.0](https://github.com/hiddenlayerai/hiddenlayer-sdk-python/compare/v3.6.0...v3.7.0)

{hiddenlayer_sdk-3.7.0 → hiddenlayer_sdk-3.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: hiddenlayer-sdk
-Version: 3.7.0
+Version: 3.8.0
 Summary: The official Python library for the hiddenlayer API
 Project-URL: Homepage, https://github.com/hiddenlayerai/hiddenlayer-sdk-python
 Project-URL: Repository, https://github.com/hiddenlayerai/hiddenlayer-sdk-python

{hiddenlayer_sdk-3.7.0 → hiddenlayer_sdk-3.8.0}/api.md

@@ -79,11 +79,16 @@ Methods:
 Types:
 
 ```python
-from hiddenlayer.types import RuntimeEvaluateRequestResponse, RuntimeEvaluateResponseResponse
+from hiddenlayer.types import (
+    RuntimeEvaluateInteractionResponse,
+    RuntimeEvaluateRequestResponse,
+    RuntimeEvaluateResponseResponse,
+)
 ```
 
 Methods:
 
+- <code title="post /detection/v2/interaction-evaluations">client.runtime.<a href="./src/hiddenlayer/resources/runtime.py">evaluate_interaction</a>(\*\*<a href="src/hiddenlayer/types/runtime_evaluate_interaction_params.py">params</a>) -> <a href="./src/hiddenlayer/types/runtime_evaluate_interaction_response.py">RuntimeEvaluateInteractionResponse</a></code>
 - <code title="post /detection/v2/request-evaluations">client.runtime.<a href="./src/hiddenlayer/resources/runtime.py">evaluate_request</a>(\*\*<a href="src/hiddenlayer/types/runtime_evaluate_request_params.py">params</a>) -> <a href="./src/hiddenlayer/types/runtime_evaluate_request_response.py">RuntimeEvaluateRequestResponse</a></code>
 - <code title="post /detection/v2/response-evaluations">client.runtime.<a href="./src/hiddenlayer/resources/runtime.py">evaluate_response</a>(\*\*<a href="src/hiddenlayer/types/runtime_evaluate_response_params.py">params</a>) -> <a href="./src/hiddenlayer/types/runtime_evaluate_response_response.py">RuntimeEvaluateResponseResponse</a></code>
 
@@ -236,7 +241,7 @@ Accessed via `client.evaluation_sessions.red_team`.
 
 Methods:
 
-- **start_session(name, target_model, target_system_prompt="", max_turns=3, execution_strategy_type="random", attacker_max_generation_attempts=2, n_random_techniques=1, max_parallel_techniques=1, prompt_set_id="", hiddenlayer_project_id=None, poll_interval=2.0, poll_max_wait=None, sessions_per_technique=1) -> RedTeamSession**  
+- **start_session(name, target_model, target_system_prompt="", objective_ids=omit, refusal_judge_model=omit, objective_judge_model=omit, attacker_model=omit, evaluation_report_model=omit, attacker_guidance=omit, severity_mapping=omit, config_id=omit, max_turns=3, execution_strategy_type="random", attacker_max_generation_attempts=2, n_random_techniques=1, max_parallel_techniques=1, prompt_set_id=omit, hiddenlayer_project_id=None, poll_interval=2.0, poll_max_wait=None, sessions_per_technique=1) -> RedTeamSession**  
   Start a new red team session. Returns a `RedTeamSession` (sync) or `AsyncRedTeamSession` (async).
   Uses adaptive multi-objective evaluation: each session runs to `max_turns` (no short-circuit),
   tests all objectives, and maintains cross-session state for adaptive attacks.

hiddenlayer_sdk-3.8.0/examples/red_team/README.md

@@ -0,0 +1,68 @@
+# Red Team Integrations
+
+Example notebooks for driving HiddenLayer red team evaluations against different
+kinds of targets using the [HiddenLayer Python SDK](https://github.com/hiddenlayerai/hiddenlayer-sdk-python).
+
+Every notebook follows the same pattern:
+
+1. Configure the target and the evaluation parameters.
+2. Implement an async `handler(prompt, history, session_id, target_system_prompt)`
+   that calls your target and returns its reply as a string.
+3. Start a session with `client.evaluation_sessions.red_team.start_session(...)`
+   and drive it with `session.run_with_callback_parallel(handler=handler)`.
+4. Retrieve the report with `client.evaluations.red_team.retrieve_evaluation_results(...)`.
+
+The `handler` is the only integration point — swap it to point at any target.
+
+`red_team_openai.ipynb` additionally shows how to **resume** an existing
+workflow (`resume_session` + `retrieve_status`), and `red_team_http_api.ipynb`
+shows scoping a run to a project via `hiddenlayer_project_id`.
+
+## Notebooks
+
+| Notebook | Target |
+|----------|--------|
+| [`red_team_openai.ipynb`](./red_team_openai.ipynb) | An OpenAI Chat Completions model |
+| [`red_team_http_api.ipynb`](./red_team_http_api.ipynb) | A custom LLM app/agent behind an HTTP/REST endpoint |
+| [`red_team_playwright.ipynb`](./red_team_playwright.ipynb) | A web chat UI driven via Playwright browser automation |
+| [`red_team_static_prompts.ipynb`](./red_team_static_prompts.ipynb) | A target tested with a pre-written static prompt set |
+| [`red_team_webhook.ipynb`](./red_team_webhook.ipynb) | A target that replies asynchronously via webhook callback (includes a self-contained mock target) |
+| [`red_team_websocket.ipynb`](./red_team_websocket.ipynb) | A target reached over a persistent WebSocket connection (includes a self-contained mock target) |
+| [`red_team_template.ipynb`](./red_team_template.ipynb) | Generic skeleton — drop in any target |
+
+## Setup
+
+```bash
+pip install hiddenlayer-sdk jupyter
+
+# Per-notebook extras:
+pip install openai        # red_team_openai.ipynb
+pip install aiohttp       # red_team_webhook.ipynb, red_team_websocket.ipynb
+pip install playwright    # red_team_playwright.ipynb
+playwright install chromium
+```
+
+`red_team_webhook.ipynb` targets a service that replies **asynchronously** via a
+webhook callback. It runs a small in-notebook receiver and ships with a
+self-contained mock target so it works out of the box; for a real external
+target, expose the receiver with a tunnel (e.g. ngrok / cloudflared) and set
+`PUBLIC_CALLBACK_URL` accordingly.
+
+Provide credentials via environment variables (read automatically by the SDK):
+
+- `HIDDENLAYER_CLIENT_ID` and `HIDDENLAYER_CLIENT_SECRET` (OAuth2), **or**
+- `HIDDENLAYER_TOKEN` (Bearer token)
+- `OPENAI_API_KEY` for `red_team_openai.ipynb`
+
+By default the client targets US production (`prod-us`). For EU production pass
+`environment="prod-eu"`, and for a self-hosted deployment pass
+`base_url="http://your-host:port"` to `AsyncHiddenLayer()`.
+
+## Run
+
+```bash
+jupyter lab examples/red_team/red_team_openai.ipynb
+```
+
+Run the cells top to bottom. Results are viewable in the
+[HiddenLayer Console](https://console.hiddenlayer.ai/).

hiddenlayer_sdk-3.8.0/examples/red_team/red_team_http_api.ipynb

@@ -0,0 +1,245 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# HiddenLayer Red Team Evaluation - HTTP/REST API Target\n",
+        "\n",
+        "Evaluate a custom target (LLM app or agent) that is exposed behind an HTTP endpoint. Adapt `to_target_request` / `from_target_response` to match your target's request and response schema.\n",
+        "\n",
+        "**Prerequisites:**\n",
+        "- `pip install hiddenlayer-sdk`\n",
+        "- Credentials read from the environment:\n",
+        "  - `HIDDENLAYER_CLIENT_ID` and `HIDDENLAYER_CLIENT_SECRET` (OAuth2), or `HIDDENLAYER_TOKEN` (Bearer token)\n",
+        "\n",
+        "**SDK Reference:** [hiddenlayer-sdk-python](https://github.com/hiddenlayerai/hiddenlayer-sdk-python)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Setup"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "import httpx\n",
+        "\n",
+        "from hiddenlayer import AsyncHiddenLayer\n",
+        "\n",
+        "# AsyncHiddenLayer() reads credentials from the environment.\n",
+        "# For EU prod pass environment=\"prod-eu\"; for self-hosted pass base_url=\"http://your-host:port\".\n",
+        "client = AsyncHiddenLayer()"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Configurables\n",
+        "\n",
+        "1. Configure target\n",
+        "2. Configure evaluation"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "# Target configuration\n",
+        "TARGET_URL = \"https://api.example.com/v1/chat\"\n",
+        "TARGET_MODEL_NAME = \"gpt-4o-mini\"\n",
+        "TARGET_SYSTEM_PROMPT = \"You are a helpful AI assistant.\"\n",
+        "CUSTOM_HEADERS = {\n",
+        "    # \"Authorization\": \"Bearer your-api-key\",\n",
+        "}\n",
+        "\n",
+        "# Evaluation configuration\n",
+        "EVAL_NAME = \"API Red Team Eval\"  # Name of the evaluation\n",
+        "EXECUTION_STRATEGY = \"single\"  # Options: \"single\", \"random\", \"static_prompt_set\"\n",
+        "MAX_TURNS = 5  # Options: 1-5\n",
+        "SESSIONS_PER_TECHNIQUE = 2  # Options: 1-5\n",
+        "PARALLEL_TECHNIQUES = 10  # Options: 1-10\n",
+        "HIDDENLAYER_PROJECT_ID = None  # Options: None, \"<project_id_or_alias>\""
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Verify payload adapters\n",
+        "\n",
+        "- `to_target_request` needs to align with the target's request payload schema\n",
+        "- `from_target_response` needs to extract the reply text from the target's response payload schema"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "def to_target_request(prompt, history, session_id):\n",
+        "    \"\"\"Format the attack payload to match the target's request schema.\"\"\"\n",
+        "    messages = [{\"role\": msg[\"role\"], \"content\": msg[\"content\"]} for msg in history]\n",
+        "    messages.append({\"role\": \"user\", \"content\": prompt})\n",
+        "    return {\"model\": TARGET_MODEL_NAME, \"messages\": messages}\n",
+        "\n",
+        "\n",
+        "def from_target_response(api_response):\n",
+        "    \"\"\"Extract the response text from the target's response schema.\"\"\"\n",
+        "    if \"choices\" in api_response:\n",
+        "        return api_response[\"choices\"][0][\"message\"][\"content\"]\n",
+        "    return api_response.get(\"response\", str(api_response))"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Create Handler\n",
+        "\n",
+        "The handler acts as a proxy between the attacker and the target: it receives an attack prompt from HiddenLayer, calls the target, and returns the target's reply."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "async def handler(prompt, history, session_id, target_system_prompt):\n",
+        "    \"\"\"Handler acts as proxy between attacker and target.\"\"\"\n",
+        "    payload = to_target_request(prompt, history, session_id)\n",
+        "    headers = {\"Content-Type\": \"application/json\", **CUSTOM_HEADERS}\n",
+        "\n",
+        "    async with httpx.AsyncClient(timeout=None) as http_client:\n",
+        "        response = await http_client.post(TARGET_URL, json=payload, headers=headers)\n",
+        "        response.raise_for_status()\n",
+        "        return from_target_response(response.json())"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Run the Evaluation\n",
+        "\n",
+        "Open a red team session and run attack techniques in parallel against the target."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "session = await client.evaluation_sessions.red_team.start_session(\n",
+        "    name=EVAL_NAME,\n",
+        "    target_model=TARGET_MODEL_NAME,\n",
+        "    target_system_prompt=TARGET_SYSTEM_PROMPT,\n",
+        "    execution_strategy_type=EXECUTION_STRATEGY,\n",
+        "    max_turns=MAX_TURNS,\n",
+        "    sessions_per_technique=SESSIONS_PER_TECHNIQUE,\n",
+        "    max_parallel_techniques=PARALLEL_TECHNIQUES,\n",
+        "    hiddenlayer_project_id=HIDDENLAYER_PROJECT_ID,\n",
+        ")\n",
+        "\n",
+        "print(f\"Session started: {session.workflow_id}\")\n",
+        "\n",
+        "await session.run_with_callback_parallel(handler=handler)\n",
+        "\n",
+        "print(\"Evaluation complete. View results: https://console.hiddenlayer.ai/\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Resume a Session\n",
+        "\n",
+        "Reconnect to a previously started workflow by its `workflow_id` — useful after an interruption or to monitor a long-running evaluation from another process. `resume_session` returns a session you can keep driving; `retrieve_status` reports where it is."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "async def resume(workflow_id):\n",
+        "    \"\"\"Reconnect to an existing workflow, finish it if still running, and return results.\"\"\"\n",
+        "    session = await client.evaluation_sessions.red_team.resume_session(workflow_id=workflow_id)\n",
+        "    status = await client.evaluations.red_team.retrieve_status(workflow_id)\n",
+        "    print(f\"Resumed {session.workflow_id} - status: {status.status}\")\n",
+        "\n",
+        "    if status.status == \"RUNNING\":\n",
+        "        await session.run_with_callback_parallel(handler=handler)\n",
+        "\n",
+        "    return await client.evaluations.red_team.retrieve_evaluation_results(workflow_id)\n",
+        "\n",
+        "\n",
+        "# results = await resume(\"<workflow_id>\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Retrieve Results\n",
+        "\n",
+        "Fetch and summarize the evaluation report for the completed session."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "results = await client.evaluations.red_team.retrieve_evaluation_results(\n",
+        "    session.workflow_id\n",
+        ")\n",
+        "\n",
+        "report = results.result.report\n",
+        "summary = report[\"summary\"]\n",
+        "\n",
+        "print(\"=\" * 60)\n",
+        "print(\"RED TEAM EVALUATION SUMMARY\")\n",
+        "print(\"=\" * 60)\n",
+        "print(f\"Total Sessions:    {summary['total_sessions']}\")\n",
+        "print(f\"Success Rate:      {summary['success_rate_pct']:.1f}%\")\n",
+        "print(f\"Successful:        {summary['success_total']} / {summary['attempts_total']}\")\n",
+        "print(f\"Errors:            {summary['error_total']}\")\n",
+        "print()\n",
+        "\n",
+        "print(\"BY OBJECTIVE\")\n",
+        "print(\"-\" * 60)\n",
+        "for obj_id, obj in report[\"by_objective\"].items():\n",
+        "    status = \"PASS\" if obj[\"success\"] == 0 else \"FAIL\"\n",
+        "    pct = obj[\"success\"] / obj[\"attempts\"] * 100 if obj[\"attempts\"] else 0\n",
+        "    print(f\"  {obj_id}: {obj['success']}/{obj['attempts']} succeeded ({pct:.0f}%) [{status}]\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 4
+}

hiddenlayer_sdk-3.8.0/examples/red_team/red_team_openai.ipynb

@@ -0,0 +1,241 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# HiddenLayer Red Team Evaluation - OpenAI Target\n",
+        "\n",
+        "Run adversarial red team attack techniques against an OpenAI chat model.\n",
+        "\n",
+        "The handler is the integration point: HiddenLayer generates attack prompts, your handler forwards each prompt to the target and returns the target's reply.\n",
+        "\n",
+        "**Prerequisites:**\n",
+        "- `pip install hiddenlayer-sdk openai`\n",
+        "- Credentials read from the environment:\n",
+        "  - `HIDDENLAYER_CLIENT_ID` and `HIDDENLAYER_CLIENT_SECRET` (OAuth2), or `HIDDENLAYER_TOKEN` (Bearer token)\n",
+        "  - `OPENAI_API_KEY` (used to call the target model)\n",
+        "\n",
+        "**SDK Reference:** [hiddenlayer-sdk-python](https://github.com/hiddenlayerai/hiddenlayer-sdk-python)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Setup"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "import os\n",
+        "\n",
+        "from openai import AsyncOpenAI\n",
+        "\n",
+        "from hiddenlayer import AsyncHiddenLayer\n",
+        "\n",
+        "# AsyncHiddenLayer() reads credentials from the environment.\n",
+        "# For EU prod pass environment=\"prod-eu\"; for self-hosted pass base_url=\"http://your-host:port\".\n",
+        "client = AsyncHiddenLayer()\n",
+        "\n",
+        "openai_client = AsyncOpenAI(api_key=os.getenv(\"OPENAI_API_KEY\"))"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Configurables\n",
+        "\n",
+        "1. Configure target\n",
+        "2. Configure evaluation"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "# Target configuration\n",
+        "TARGET_MODEL = \"gpt-4o-mini\"\n",
+        "TARGET_SYSTEM_PROMPT = \"You are a helpful AI assistant.\"\n",
+        "\n",
+        "# Evaluation configuration\n",
+        "EVAL_NAME = \"OpenAI Red Team Eval\"  # Name of the evaluation\n",
+        "EXECUTION_STRATEGY = \"single\"  # Options: \"single\", \"random\", \"static_prompt_set\"\n",
+        "MAX_TURNS = 5  # Options: 1-5\n",
+        "SESSIONS_PER_TECHNIQUE = 1  # Options: 1-5\n",
+        "PARALLEL_TECHNIQUES = 5  # Options: 1-10"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Verify payload adapters\n",
+        "\n",
+        "- `to_target_request` builds the OpenAI Chat Completions payload from the attack prompt and history\n",
+        "- `from_target_response` extracts the assistant's reply text from the target response"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "def to_target_request(prompt, history, session_id):\n",
+        "    \"\"\"Build the target request payload from the attack prompt and history.\"\"\"\n",
+        "    messages = [{\"role\": \"system\", \"content\": TARGET_SYSTEM_PROMPT}]\n",
+        "    messages.extend(history)\n",
+        "    messages.append({\"role\": \"user\", \"content\": prompt})\n",
+        "    return {\"model\": TARGET_MODEL, \"messages\": messages}\n",
+        "\n",
+        "\n",
+        "def from_target_response(response):\n",
+        "    \"\"\"Extract the assistant reply text from the target response.\"\"\"\n",
+        "    return response.choices[0].message.content or \" \""
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Create Handler\n",
+        "\n",
+        "The handler acts as a proxy between the attacker and the target: it receives an attack prompt from HiddenLayer, calls the target, and returns the target's reply."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "async def handler(prompt, history, session_id, target_system_prompt):\n",
+        "    \"\"\"Handler acts as proxy between attacker and target.\"\"\"\n",
+        "    payload = to_target_request(prompt, history, session_id)\n",
+        "    response = await openai_client.chat.completions.create(**payload)\n",
+        "    return from_target_response(response)"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Run the Evaluation\n",
+        "\n",
+        "Open a red team session and run attack techniques in parallel against the target."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "session = await client.evaluation_sessions.red_team.start_session(\n",
+        "    name=EVAL_NAME,\n",
+        "    target_model=TARGET_MODEL,\n",
+        "    target_system_prompt=TARGET_SYSTEM_PROMPT,\n",
+        "    execution_strategy_type=EXECUTION_STRATEGY,\n",
+        "    max_turns=MAX_TURNS,\n",
+        "    sessions_per_technique=SESSIONS_PER_TECHNIQUE,\n",
+        "    max_parallel_techniques=PARALLEL_TECHNIQUES,\n",
+        ")\n",
+        "\n",
+        "print(f\"Session started: {session.workflow_id}\")\n",
+        "\n",
+        "await session.run_with_callback_parallel(handler=handler)\n",
+        "\n",
+        "print(\"Evaluation complete. View results: https://console.hiddenlayer.ai/\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Resume a Session\n",
+        "\n",
+        "Reconnect to a previously started workflow by its `workflow_id` — useful after an interruption or to monitor a long-running evaluation from another process. `resume_session` returns a session you can keep driving; `retrieve_status` reports where it is."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "async def resume(workflow_id):\n",
+        "    \"\"\"Reconnect to an existing workflow, finish it if still running, and return results.\"\"\"\n",
+        "    session = await client.evaluation_sessions.red_team.resume_session(workflow_id=workflow_id)\n",
+        "    status = await client.evaluations.red_team.retrieve_status(workflow_id)\n",
+        "    print(f\"Resumed {session.workflow_id} - status: {status.status}\")\n",
+        "\n",
+        "    if status.status == \"RUNNING\":\n",
+        "        await session.run_with_callback_parallel(handler=handler)\n",
+        "\n",
+        "    return await client.evaluations.red_team.retrieve_evaluation_results(workflow_id)\n",
+        "\n",
+        "\n",
+        "# results = await resume(\"<workflow_id>\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Retrieve Results\n",
+        "\n",
+        "Fetch and summarize the evaluation report for the completed session."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {},
+      "source": [
+        "results = await client.evaluations.red_team.retrieve_evaluation_results(\n",
+        "    session.workflow_id\n",
+        ")\n",
+        "\n",
+        "report = results.result.report\n",
+        "summary = report[\"summary\"]\n",
+        "\n",
+        "print(\"=\" * 60)\n",
+        "print(\"RED TEAM EVALUATION SUMMARY\")\n",
+        "print(\"=\" * 60)\n",
+        "print(f\"Total Sessions:    {summary['total_sessions']}\")\n",
+        "print(f\"Success Rate:      {summary['success_rate_pct']:.1f}%\")\n",
+        "print(f\"Successful:        {summary['success_total']} / {summary['attempts_total']}\")\n",
+        "print(f\"Errors:            {summary['error_total']}\")\n",
+        "print()\n",
+        "\n",
+        "print(\"BY OBJECTIVE\")\n",
+        "print(\"-\" * 60)\n",
+        "for obj_id, obj in report[\"by_objective\"].items():\n",
+        "    status = \"PASS\" if obj[\"success\"] == 0 else \"FAIL\"\n",
+        "    pct = obj[\"success\"] / obj[\"attempts\"] * 100 if obj[\"attempts\"] else 0\n",
+        "    print(f\"  {obj_id}: {obj['success']}/{obj['attempts']} succeeded ({pct:.0f}%) [{status}]\")"
+      ],
+      "execution_count": null,
+      "outputs": []
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 4
+}