inspect-ai 0.3.67__tar.gz → 0.3.69__tar.gz

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/.github/dependabot.yml

@@ -14,3 +14,5 @@ updates:
         versions: [">=1.6.0"]
       - dependency-name: "ruff"
         versions: [">=0.9.5"]
+      - dependency-name: "textual"
+        versions: [">=2.0.0"]

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/.github/workflows/build.yml

@@ -51,6 +51,18 @@ jobs:
         run: |
           mypy .
 
+  pre-commit:
+    runs-on: ubuntu-latest
+    env:
+      SKIP: "ruff"
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - uses: pre-commit/action@v3.0.1
+
   test:
     runs-on: ubuntu-latest
     strategy:

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/.pre-commit-config.yaml

@@ -2,7 +2,7 @@
 # See https://pre-commit.com/hooks.html for more hooks
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.9.5
+    rev: v0.9.6
     hooks:
       # Run the linter.
       - id: ruff
@@ -14,8 +14,9 @@ repos:
     hooks:
       - id: check-added-large-files
       - id: check-json
+        # The following are JSONC files.
+        exclude: '(^\.vscode/|^tools/vscode/|jsconfig\.json$|tsconfig\.json$)'
       - id: check-yaml
       - id: debug-statements
       - id: detect-private-key
-      - id: end-of-file-fixer
       - id: requirements-txt-fixer

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/CHANGELOG.md

@@ -1,3 +1,20 @@
+## v0.3.69 (20 February 2025)
+
+- Google provider updated to use the [Google Gen AI SDK](https://googleapis.github.io/python-genai/), which is now the recommended API for Gemini 2.0 models.
+- Task display: Use cooperative cancellation for cancel buttons in task display.
+- Task display: Print task progress every 5 seconds for 'plain' display mode.
+- Task display: Handle click on running samples tab when there is no transcript.
+- Docker: Print stderr from `compose up` when no services startup successfully. 
+- Docker: Print sample id and epoch for each container when using `--no-sandbox-cleanup`
+- Mistral: Create and destroy client within generate.
+- Inspect View: Fix display of score dictionaries containing boolean values
+- Bugfix: Catch standard `TimeoutError` for subprocess timeouts (ensure kill/cleanup of timed out process).
+
+## v0.3.68 (19 February 2025)
+
+- Task display: Improve spacing/layout of final task display.
+- Textual: speicfy broader range of compatible versions (v0.86.2 to v1.0.0)
+
 ## v0.3.67 (18 February 2025)
 
 - Memoize calls to `get_model()` so that model instances with the same parameters are cached and re-used (pass `memoize=False` to disable).

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: inspect_ai
-Version: 0.3.67
+Version: 0.3.69
 Summary: Framework for large language model evaluations
 Author: UK AI Security Institute
 License: MIT License
@@ -45,7 +45,7 @@ Requires-Dist: s3fs>=2023
 Requires-Dist: semver>=3.0.0
 Requires-Dist: shortuuid
 Requires-Dist: tenacity
-Requires-Dist: textual==1.0.0
+Requires-Dist: textual<=1.0.0,>=0.86.2
 Requires-Dist: typing_extensions>=4.9.0
 Requires-Dist: zipp>=3.19.1
 Provides-Extra: dev
@@ -53,7 +53,7 @@ Requires-Dist: anthropic; extra == "dev"
 Requires-Dist: aioboto3; extra == "dev"
 Requires-Dist: azure-ai-inference; extra == "dev"
 Requires-Dist: google-cloud-aiplatform; extra == "dev"
-Requires-Dist: google-generativeai; extra == "dev"
+Requires-Dist: google-genai; extra == "dev"
 Requires-Dist: goodfire; extra == "dev"
 Requires-Dist: griffe; extra == "dev"
 Requires-Dist: groq; extra == "dev"
@@ -71,7 +71,7 @@ Requires-Dist: pytest-asyncio; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: pytest-dotenv; extra == "dev"
 Requires-Dist: pytest-xdist; extra == "dev"
-Requires-Dist: ruff==0.9.5; extra == "dev"
+Requires-Dist: ruff==0.9.6; extra == "dev"
 Requires-Dist: textual-dev>=0.86.2; extra == "dev"
 Requires-Dist: types-Markdown; extra == "dev"
 Requires-Dist: types-PyYAML; extra == "dev"

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/index.qmd

@@ -56,7 +56,7 @@ inspect eval arc.py --model anthropic/claude-3-5-sonnet-latest
 #### Google
 
 ``` bash
-pip install google-generativeai
+pip install google-genai
 export GOOGLE_API_KEY=your-google-api-key
 inspect eval arc.py --model google/gemini-1.5-pro
 ```

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/models.qmd

@@ -40,24 +40,24 @@ You can specify generation options either on the command line or in direct calls
 
 ``` bash
 inspect eval arc.py --model openai/gpt-4 --temperature 0.9
-inspect eval arc.py --model google/gemini-1.0-pro --max-connections 20
+inspect eval arc.py --model google/gemini-1.5-pro --max-connections 20
 ```
 
 Or:
 
 ``` python
 eval("arc.py", model="openai/gpt-4", temperature=0.9)
-eval("arc.py", model="google/gemini-1.0-pro", max_connections=20)
+eval("arc.py", model="google/gemini-1.5-pro", max_connections=20)
 ```
 
 Use `inspect eval --help` to learn about all of the available generation config options.
 
 ## Model Args
 
-If there is an additional aspect of a model you want to tweak that isn't covered by the `GenerateConfig`, you can use model args to pass additional arguments to model clients. For example, here we specify the `transport` option for a Google Gemini model:
+If there is an additional aspect of a model you want to tweak that isn't covered by the `GenerateConfig`, you can use model args to pass additional arguments to model clients. For example, here we specify the `location` option for a Google Gemini model:
 
 ``` bash
-inspect eval arc.py --model google/gemini-1.0-pro -M transport:grpc
+inspect eval arc.py --model google/gemini-1.5-pro -M location=us-east5
 ```
 
 See the documentation for the requisite model provider for information on how model args are passed through to model clients.

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/providers.qmd

@@ -8,7 +8,6 @@ Inspect has support for a wide variety of language model APIs and can be extende
 
 {{< include _model-providers.md >}}
 
-
 ## OpenAI {#openai}
 
 To use the [OpenAI](https://platform.openai.com/) provider, install the `openai` package, set your credentials, and specify a model using the `--model` option:
@@ -88,15 +87,15 @@ Authentication is doing using the standard Google Cloud CLI (i.e. if you have au
 
 ## Google {#google-gemini}
 
-To use the [Google](https://ai.google.dev/) provider, install the `google-generativeai` package, set your credentials, and specify a model using the `--model` option:
+To use the [Google](https://ai.google.dev/) provider, install the `google-genai` package, set your credentials, and specify a model using the `--model` option:
 
 ``` bash
-pip install google-generativeai
+pip install google-genai
 export GOOGLE_API_KEY=your-google-api-key
 inspect eval arc.py --model google/gemini-1.5-pro
 ```
 
-For the `google` provider, custom model args (`-M`) are forwarded to the `genai.configure` function.
+For the `google` provider, custom model args (`-M`) are forwarded to the `genai.Client` function.
 
 The following environment variables are supported by the Google provider
 
@@ -105,12 +104,38 @@ The following environment variables are supported by the Google provider
 | `GOOGLE_API_KEY`  | API key credentials (required).  |
 | `GOOGLE_BASE_URL` | Base URL for requests (optional) |
 
+### Gemini on Vertex AI
+
+To use Google Gemini models on Vertex, you can use the standard `google` model provider with the `vertex` qualifier (e.g. `google/vertex/gemini-2.0-flash`). You should also set two environment variables indicating your project ID and region. Here is a complete example:
+
+``` bash
+export GOOGLE_CLOUD_PROJECT=project-12345
+export GOOGLE_CLOUD_LOCATION=us-east5
+inspect eval ctf.py --model google/vertex/gemini-2.0-flash
+```
+
+You can alternatively pass the project and location as custom model args (`-M`). For example:
+
+``` bash
+inspect eval ctf.py --model google/vertex/gemini-2.0-flash \
+   -M project=project-12345 -M location=us-east5
+```
+
+Authentication is done using the standard Google Cloud CLI. For example:
+
+```bash
+gcloud auth application-default login
+```
+
+If you have authorised the CLI then no additional auth is needed for the model API.
+
 ### Safety Settings {#safety-settings}
 
-Google models make available [safety settings](https://ai.google.dev/gemini-api/docs/safety-settings) that you can adjust to determine what sorts of requests will be handled (or refused) by the model. The four categories of safety settings are as follows:
+Google models make available [safety settings](https://ai.google.dev/gemini-api/docs/safety-settings) that you can adjust to determine what sorts of requests will be handled (or refused) by the model. The five categories of safety settings are as follows:
 
 | Category | Description |
-|-----------------------|------------------------------------------------|
+|------------------------|-----------------------------------------------|
+| `civic_integrity`   |  Election-related queries.          |
 | `sexually_explicit` | Contains references to sexual acts or other lewd content. |
 | `hate_speech` | Content that is rude, disrespectful, or profane. |
 | `harassment` | Negative or harmful comments targeting identity and/or protected attributes. |
@@ -121,7 +146,7 @@ Google models make available [safety settings](https://ai.google.dev/gemini-api/
 For each category, the following block thresholds are available:
 
 | Block Threshold | Description |
-|------------------------|------------------------------------------------|
+|-------------------------|-----------------------------------------------|
 | `none` | Always show regardless of probability of unsafe content |
 | `only_high` | Block when high probability of unsafe content |
 | `medium_and_above` | Block when medium or high probability of unsafe content |
@@ -250,20 +275,24 @@ You can also use this option to disable tool emulation for Llama models with `em
 ## Vertex AI {#vertex-ai}
 
 ::: {.callout-note appearance="simple"}
-Vertex AI is a distinct service from Google AI, see a comparison matrix [here](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai#google-ai). Make sure you are using the appropriate model provider.
+If you are using Gemini or Anthropic models on Vertex AI, we recommend you use the Google and Anthropic providers (respectively) which both support models hosted on Vertex:
+
+- [Anthropic on Vertex AI](#anthropic-on-vertex-ai)
+- [Gemini on Vertex AI](#gemini-on-vertex-ai)
+
+If you are using other models hosted on Vertex (e.g. Mistral, Llama, Gemma, etc.) then you should instead use the `vertex` provider as described below.
 :::
 
 To use the [Vertex AI](https://cloud.google.com/vertex-ai) provider, install the `google-cloud-aiplatform` package, [configure your environment](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal#expandable-1) for Vertex API access, and specify a model using the `--model` option:
 
 ``` bash
-inspect eval eval.py --model vertex/gemini-1.5-flash
+inspect eval eval.py --model vertex/mistral-large-2411
 ```
 
 The core libraries for Vertex AI interact directly with Google Cloud Platform so this provider doesn't use the standard `BASE_URL`/`API_KEY` approach that others do. Consequently you don't need to set these environment variables.
 
 Vertex AI also provides the same `safety_settings` outlined in the [Google](#safety-settings) provider.
 
-If you are using Anthropic on Vertex AI, you can alternatively use the [Anthropic provider](#anthropic-on-vertex-ai) as your means of access.
 
 ## Together AI {#together-ai}
 

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/reasoning.qmd

@@ -32,9 +32,7 @@ Reasoning traces are currently captured in two ways:
 
 2.  From `<think></think>` tags included in the main assistant `content` (this is how Together, Groq, and Ollama currently present reasoning traces).
 
-Gemini 2.0 Flash Thinking currently includes its reasoning inline with response content (there is currently no structured way to extract it, although this seems likely to change in the future).
-
-We have confirmed that `reasoning` traces can be extracted from models using the `together`, `groq`, and `ollama` providers.
+3.  Using the `Final Answer: ` delimiter currently used by Gemini 2.0 Flash Thinking (the API has a separate field for `thinking` but it is not currently used in responses).
 
 We would like to confirm this for other providers (e.g. `bedrock` and `azureai`) but as of yet have not been able to access reasoning models on those services (we'd very much welcome others contributing here, either to confirm that things work or to provide PRs which make the required changes).
 

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/reference/_sidebar.yml

@@ -44,6 +44,8 @@ website:
           href: reference/inspect_ai.solver.qmd#system_message
         - text: user_message
           href: reference/inspect_ai.solver.qmd#user_message
+        - text: assistant_message
+          href: reference/inspect_ai.solver.qmd#assistant_message
         - text: chain_of_thought
           href: reference/inspect_ai.solver.qmd#chain_of_thought
         - text: self_critique

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/docs/scorers.qmd

@@ -117,7 +117,7 @@ The built-in model graded scorers also support using multiple grader models (whe
 ``` python
 model_graded_qa(
     model = [
-        "google/gemini-1.0-pro",
+        "google/gemini-1.5-pro",
         "anthropic/claude-3-opus-20240229" 
         "together/meta-llama/Llama-3-70b-chat-hf",
     ]
@@ -740,7 +740,7 @@ log = eval(popularity, model="openai/gpt-4")[0]
 grader_models = [
     "openai/gpt-4",
     "anthropic/claude-3-opus-20240229",
-    "google/gemini-1.0-pro",
+    "google/gemini-1.5-pro",
     "mistral/mistral-large-latest"
 ]
 
@@ -760,7 +760,7 @@ log = read_eval_log(input_log_path)
 grader_models = [
     "openai/gpt-4",
     "anthropic/claude-3-opus-20240229",
-    "google/gemini-1.0-pro",
+    "google/gemini-1.5-pro",
     "mistral/mistral-large-latest"
 ]
 

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/pyproject.toml

@@ -115,7 +115,7 @@ dev = [
     "aioboto3", 
     "azure-ai-inference",
     "google-cloud-aiplatform",
-    "google-generativeai", 
+    "google-genai",
     "goodfire",
     "griffe",
     "groq", 
@@ -133,7 +133,7 @@ dev = [
     "pytest-cov",
     "pytest-dotenv",
     "pytest-xdist",
-    "ruff==0.9.5", # match version specified in .pre-commit-config.yaml
+    "ruff==0.9.6", # match version specified in .pre-commit-config.yaml
     "textual-dev>=0.86.2",
     "types-Markdown",
     "types-PyYAML",

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/requirements.txt

@@ -23,6 +23,6 @@ s3fs>=2023
 semver>=3.0.0
 shortuuid
 tenacity
-textual==1.0.0
+textual>=0.86.2,<=1.0.0
 typing_extensions>=4.9.0
 zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_display/core/panel.py

@@ -51,10 +51,9 @@ def task_panel(
 
     # main progress and task info
     if body:
-        table.add_row()
         table.add_row(body)
 
-    # spacing if there is more ocontent
+    # spacing if there is more content
     if footer or log_location:
         table.add_row()
 

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_display/plain/display.py

@@ -119,14 +119,14 @@ class PlainTaskDisplay(TaskDisplay):
         self.samples_complete = 0
         self.samples_total = 0
         self.current_metrics: list[TaskDisplayMetric] | None = None
-        self.last_progress = 0  # Track last progress percentage
+        self.last_progress = 0
 
     @contextlib.contextmanager
     def progress(self) -> Iterator[Progress]:
         self.progress_display = PlainProgress(self.task.profile.steps)
         yield self.progress_display
 
-    @throttle(1)
+    @throttle(5)
     def _print_status_throttled(self) -> None:
         self._print_status()
 
@@ -135,13 +135,8 @@ class PlainTaskDisplay(TaskDisplay):
         if not self.progress_display:
             return
 
-        # Calculate current progress percentage
-        current_progress = int(
-            self.progress_display.current / self.progress_display.total * 100
-        )
-
-        # Only print on percentage changes to avoid too much output
-        if current_progress != self.last_progress:
+        # Only print when step count changes to avoid too much output
+        if self.progress_display.current != self.last_progress:
             status_parts: list[str] = []
 
             # if this is parallel print task and model to distinguish (limit both to 12 chars)
@@ -154,8 +149,11 @@ class PlainTaskDisplay(TaskDisplay):
                 )
 
             # Add step progress
+            progress_percent = int(
+                self.progress_display.current / self.progress_display.total * 100
+            )
             status_parts.append(
-                f"Steps: {self.progress_display.current:3d}/{self.progress_display.total} {current_progress:3d}%"
+                f"Steps: {self.progress_display.current:3d}/{self.progress_display.total} {progress_percent:3d}%"
             )
 
             # Add sample progress
@@ -187,7 +185,7 @@ class PlainTaskDisplay(TaskDisplay):
             # Print on new line
             print(" | ".join(status_parts))
 
-            self.last_progress = current_progress
+            self.last_progress = self.progress_display.current
 
     def sample_complete(self, complete: int, total: int) -> None:
         self.samples_complete = complete

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_display/rich/display.py

@@ -4,7 +4,7 @@ from dataclasses import dataclass
 from typing import Any, AsyncIterator, Callable, Coroutine, Iterator
 
 import rich
-from rich.console import Console, RenderableType
+from rich.console import Console, Group, RenderableType
 from rich.live import Live
 from rich.panel import Panel
 from rich.progress import Progress as RProgress
@@ -298,7 +298,7 @@ def task_live_status(tasks: list[TaskStatus], progress: RProgress) -> Renderable
     return task_panel(
         profile=tasks[0].profile,
         show_model=len(tasks) == 1,
-        body=progress,
+        body=Group("", progress),
         subtitle=subtitle,
         footer=task_footer(theme.light),
         log_location=None,

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_display/textual/app.py

@@ -13,7 +13,6 @@ from typing import (
 
 import rich
 from rich.console import Console
-from rich.text import Text
 from textual.app import App, ComposeResult
 from textual.binding import Binding, BindingType
 from textual.css.query import NoMatches
@@ -316,9 +315,9 @@ class TaskScreenApp(App[TR]):
 
         def set_unread(unread: int | None) -> None:
             if unread is not None:
-                console_tab.label = Text(f"Console ({unread}")
+                console_tab.label = f"Console ({unread}"  # type: ignore[assignment]
             else:
-                console_tab.label = Text("Console")
+                console_tab.label = "Console"  # type: ignore[assignment]
 
         self.watch(console_view, "unread", set_unread)
 
@@ -385,7 +384,7 @@ class TaskScreenApp(App[TR]):
         def set_title(self, title: str) -> None:
             tabs = self.app.query_one(TabbedContent)
             tab = tabs.get_tab(self.tab_id)
-            tab.label = Text(title)
+            tab.label = title  # type: ignore[assignment]
 
         def activate(self) -> None:
             # show the tab

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_display/textual/widgets/samples.py

@@ -6,6 +6,7 @@ from rich.table import Table
 from rich.text import Text
 from textual.app import ComposeResult
 from textual.containers import Horizontal, HorizontalGroup, Vertical, VerticalGroup
+from textual.css.query import NoMatches
 from textual.reactive import reactive
 from textual.widget import Widget
 from textual.widgets import (
@@ -61,7 +62,10 @@ class SamplesView(Widget):
         )
 
     async def notify_active(self, active: bool) -> None:
-        await self.query_one(TranscriptView).notify_active(active)
+        try:
+            await self.query_one(TranscriptView).notify_active(active)
+        except NoMatches:
+            pass
 
     def set_samples(self, samples: list[ActiveSample]) -> None:
         # throttle to no more than 1 second per 100 samples
@@ -408,6 +412,16 @@ class SampleToolbar(Horizontal):
     PENDING_STATUS = "pending_status"
     PENDING_CAPTION = "pending_caption"
 
+    TIMEOUT_TOOL_CALL_ENABLED = (
+        "Cancel the tool call and report a timeout to the model."
+    )
+    TIMEOUT_TOOL_CALL_DISABLED = "Cancelling tool call..."
+    CANCEL_SCORE_OUTPUT_ENABLED = (
+        "Cancel the sample and score whatever output has been generated so far."
+    )
+    CANCEL_RAISE_ERROR_ENABLED = "Cancel the sample and raise an error"
+    CANCEL_DISABLED = "Cancelling sample..."
+
     DEFAULT_CSS = f"""
     SampleToolbar {{
         grid-size: 5 1;
@@ -445,18 +459,18 @@ class SampleToolbar(Horizontal):
         yield Button(
             Text("Timeout Tool"),
             id=self.TIMEOUT_TOOL_CALL,
-            tooltip="Cancel the tool call and report a timeout to the model.",
+            tooltip=self.TIMEOUT_TOOL_CALL_ENABLED,
         )
         yield Horizontal()
         yield Button(
             Text("Cancel (Score)"),
             id=self.CANCEL_SCORE_OUTPUT,
-            tooltip="Cancel the sample and score whatever output has been generated so far.",
+            tooltip=self.CANCEL_SCORE_OUTPUT_ENABLED,
         )
         yield Button(
             Text("Cancel (Error)"),
             id=self.CANCEL_RAISE_ERROR,
-            tooltip="Cancel the sample and raise an error (task will exit unless fail_on_error is set)",
+            tooltip=self.CANCEL_RAISE_ERROR_ENABLED,
         )
 
     def on_mount(self) -> None:
@@ -475,14 +489,26 @@ class SampleToolbar(Horizontal):
                 )
                 if isinstance(last_event, ToolEvent):
                     last_event._cancel()
-            elif event.button.id == self.CANCEL_SCORE_OUTPUT:
-                self.sample.interrupt("score")
-            elif event.button.id == self.CANCEL_RAISE_ERROR:
-                self.sample.interrupt("error")
+                    event.button.disabled = True
+                    event.button.tooltip = self.TIMEOUT_TOOL_CALL_DISABLED
+            else:
+                if event.button.id == self.CANCEL_SCORE_OUTPUT:
+                    self.sample.interrupt("score")
+                elif event.button.id == self.CANCEL_RAISE_ERROR:
+                    self.sample.interrupt("error")
+                cancel_score_output = self.query_one("#" + self.CANCEL_SCORE_OUTPUT)
+                cancel_score_output.disabled = True
+                cancel_score_output.tooltip = self.CANCEL_DISABLED
+                cancel_with_error = self.query_one("#" + self.CANCEL_RAISE_ERROR)
+                cancel_with_error.disabled = True
+                cancel_with_error.tooltip = self.CANCEL_DISABLED
 
     async def sync_sample(self, sample: ActiveSample | None) -> None:
         from inspect_ai.log._transcript import ModelEvent
 
+        # is it a new sample?
+        new_sample = sample != self.sample
+
         # track the sample
         self.sample = sample
 
@@ -499,6 +525,13 @@ class SampleToolbar(Horizontal):
             cancel_score_output.display = True
             cancel_with_error.display = not sample.fails_on_error
 
+            # if its a new sample then reset enabled states
+            if new_sample:
+                cancel_score_output.disabled = False
+                cancel_score_output.tooltip = self.CANCEL_SCORE_OUTPUT_ENABLED
+                cancel_with_error.disabled = False
+                cancel_with_error.tooltip = self.CANCEL_RAISE_ERROR_ENABLED
+
             # if we have a pending event then start the clock and show pending status
             last_event = (
                 sample.transcript.events[-1]
@@ -520,6 +553,8 @@ class SampleToolbar(Horizontal):
                 )
 
                 timeout_tool.display = isinstance(last_event, ToolEvent)
+                timeout_tool.disabled = False
+                timeout_tool.tooltip = self.TIMEOUT_TOOL_CALL_ENABLED
 
                 clock.start(last_event.timestamp.timestamp())
             else:

inspect_ai-0.3.69/src/inspect_ai/_util/interrupt.py

@@ -0,0 +1,9 @@
+import asyncio
+
+
+def check_sample_interrupt() -> None:
+    from inspect_ai.log._samples import sample_active
+
+    sample = sample_active()
+    if sample and sample.interrupt_action:
+        raise asyncio.CancelledError()

{inspect_ai-0.3.67 → inspect_ai-0.3.69}/src/inspect_ai/_util/logger.py

@@ -90,6 +90,10 @@ class LogHandler(RichHandler):
         if "Event loop is closed" in record.getMessage():
             return
 
+        # skip google-genai AFC message
+        if "AFC is enabled with max remote calls" in record.getMessage():
+            return
+
         # write to stderr if we are at or above the threshold
         if record.levelno >= self.display_level:
             super().emit(record)