benchflow 0.7.1.dev1969__tar.gz → 0.7.2__tar.gz

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/CHANGELOG.md

@@ -2,6 +2,125 @@
 
 ## [Unreleased]
 
+## 0.7.2 — 2026-08-16
+
+### Added
+- **The upload preview itemizes what redaction masked, by kind.** The
+  redactor now categorizes every replacement by the rule that fired — API
+  keys, bearer tokens, private key blocks, passwords, URL credentials, and
+  credential-bearing field values — and the terminal trajectory report shows
+  a `Masked for you: 2 API keys, 1 bearer token — originals never leave this
+  machine` breakdown under the masked-count row, plus a reassurance that
+  redaction ran locally and the server independently rescans staged
+  artifacts (or `No secrets or personal identifiers detected — nothing
+  needed masking.` when nothing matched). `bench traj upload --dry-run`
+  prints the same breakdown as a plain `Masked for you:` line, and
+  `bench eval view --confirm` gains a display-only `--redaction-summary`
+  flag that renders it in the confirm bar next to the Approve button; the
+  `benchflow-traj-upload` skill stages a dry run first and passes the line
+  through. The total `redaction_replacements` count and the manifest
+  `trajectory_report` contract are unchanged (the server validates the
+  report with a closed schema and exact recompute equality, so per-category
+  counts stay display-only).
+- **The trajectory viewer can collect the eval-prize confirmation in the
+  browser.** `bench eval view PATH --confirm` renders the normal page plus a
+  sticky site-styled bottom bar ("Submit this trajectory to the BenchFlow
+  eval prize?") with **Approve & submit** / **Not this one** buttons. A click
+  POSTs to `/decision`; the server prints a machine-readable
+  `DECISION: approved` or `DECISION: rejected` line to stdout, shuts down,
+  and the CLI exits `0` on approve and `3` on reject (non-1/2 so rejection
+  never collides with error or usage exits). Without the flag, behavior is
+  unchanged (no bar, no endpoint, Ctrl+C to stop). The
+  `benchflow-traj-upload` skill now prefers the button flow and falls back
+  to chat confirmation on CLIs older than 0.7.2.
+- **Trajectory uploads are tagged with the session's repository by
+  default.** Unless `--source-id` is given, `bench traj upload` reads the
+  session's recorded working directory (Claude `cwd` events, Codex
+  `session_meta`), resolves its git `origin` remote, and stores
+  `repo/<owner>/<name>` as the manifest source id, printing
+  `Repo: owner/name (from session cwd /path; use --no-repo to omit)` (the
+  local path is terminal output only, never uploaded). `--no-repo` opts out
+  — the `benchflow-traj-upload` skill now surfaces the detected tag during
+  the confirm step so contributors can decline it for private repos — and
+  undetectable repos fall back silently to the path-derived source id.
+
+### Fixed
+- **The repo tag derives only from the session's own recorded cwd.** The
+  initial repo-tagging implementation (#1015) fell back to the upload
+  invocation directory's git remote when the session cwd yielded nothing; a
+  collector-side audit showed this mis-attributes provenance — a session
+  recorded in a non-repo directory, uploaded from the benchflow checkout,
+  was tagged `repo/benchflow-ai/benchflow` (two community-dataset entries
+  carry the mis-tag). The fallback is removed: no session cwd, a missing
+  directory, or no GitHub remote now mean no repo tag, exactly like
+  `--no-repo`.
+- **The trajectory viewer header no longer shows `?` badges on real Claude
+  Code sessions.** `bench eval view` on a `~/.claude` session JSONL rendered
+  `model: ?`, `session: ?...`, `claude code: ?`, and `total cost: $0.0000`
+  because the header only read a `type: system` event that real session
+  files don't contain. The header now derives its metadata from what the
+  file actually carries (first assistant event's `message.model`, per-event
+  `version` / `sessionId`, the filename stem as a session fallback) and
+  hides any badge whose value is unknown — including the cost badge when no
+  event carries cost data. Presentation only.
+- **Upload progress no longer claims the broker is waking up.** A warm
+  retry printed `Uploading… the first request can take a minute while the
+  service wakes up` even when the service was already up. The line is now
+  `Uploading… this can take up to a minute; retries are safe.`
+- **`bench traj setup --list` no longer wraps session paths mid-token.**
+  Each hit prints index/source/time, then the path on its own line, then
+  the snippet, via plain `print` so Rich does not split a long JSONL path
+  and break copy-paste.
+
+### Changed
+- **Trajectory viewer tool calls are color-coded and backgrounds are light.**
+  Each tool kind now gets a muted GitHub-label-style accent on its name pill
+  and a left border strip on the card: shell/exec → amber, write/edit → blue,
+  read → teal, agent/task/skill → purple, web/search/fetch → cyan, everything
+  else → neutral gray. Tool arguments and tool outputs render on light
+  surfaces (`#f5f5f5` / white with dark ink) instead of near-black blocks;
+  the dark `#141414` terminal treatment is reserved for shell-command output
+  only, and the ink-black result card stays as the deliberate bento-ink
+  accent. Presentation only (CSS classes + a tool-name→accent mapping);
+  content strings and behavior are unchanged. Applies to all three viewer
+  templates, which share one stylesheet since #1019.
+- **The contributor prompt now tells the agent to upgrade BenchFlow first.**
+  `CONTRIBUTOR_PROMPT` (kept in sync in `README.md`, `docs/traj-upload.md`,
+  and the `benchflow-traj-upload` skill evals) is a three-line block that
+  says to run `uv tool install --python 3.12 --upgrade --force benchflow`
+  before reading the skill, so agents that skim the skill or hit a stale copy
+  still install the latest CLI. The prompt also names OpenCode and Cursor
+  sessions and the re:Agent hackathon 72-hour window; README/docs render it
+  as a blockquote (soft-wraps on GitHub) behind an explicit "send this to
+  your coding agent" framing, and `bench traj setup` prints the same framing.
+  The skill's Discover step gains best-effort Cursor
+  (`~/.cursor/projects/*/agent-transcripts/`) and OpenCode
+  (`~/.local/share/opencode/opencode.db`, `opencode db path`) locations plus
+  a prefer-recent-matching-sessions note. Follow-up to the version
+  precondition from #1013/#1014.
+- **Trajectory viewer restyled to match www.benchflow.ai.** All three
+  `bench eval view` pages (stream-json/JSONL, ACP events, multi-turn trial)
+  now share one inline stylesheet with the site's design language: light
+  monochrome palette (`#fafafa` page, white cards, `#0a0a0a` ink, dark
+  `#141414` code blocks), Satoshi/Google Sans Code font stacks with
+  system-safe fallbacks, mono pill badges, and a small BenchFlow wordmark
+  header with the inline SVG logo. Pages remain fully offline (no external
+  font or CDN requests) and content/structure semantics are unchanged;
+  follows the contributor paste-line flow from #1013.
+
+## 0.7.1 — 2026-08-16
+
+### Added
+- **`bench traj setup` / `bench traj upload` print an upgrade hint when
+  outdated.** Both commands start with a lightweight PyPI latest-version
+  check (2 s timeout, completely silent on any network or parse failure) and
+  print a one-line `uv tool install --python 3.12 --upgrade --force
+  benchflow` hint when the installed version is older than the latest
+  release; dev/prereleases of a newer-or-equal base are not outdated.
+  `BENCHFLOW_SKIP_UPDATE_CHECK=1` disables the check. The
+  `benchflow-traj-upload` skill and docs now tell contributors to upgrade to
+  the latest BenchFlow (0.7.1+) before using the trajectory-upload flow.
+
 ### Changed
 - **Trajectory contribution is a copy-paste line, plus optional setup.**
   Contributors paste one line into their agent; the agent reads

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: benchflow
-Version: 0.7.1.dev1969
+Version: 0.7.2
 Summary: Multi-turn agent benchmarking with ACP — run any agent, any model, any provider.
 Project-URL: Homepage, https://github.com/benchflow-ai/benchflow
 Project-URL: Repository, https://github.com/benchflow-ai/benchflow
@@ -89,17 +89,24 @@ BenchFlow is a universal environment framework: it runs AI agents against task e
 
 ## Quick start: 1. Submit a trajectory
 
-Proud of a Claude Code or Codex session? Copy this line into Claude Code,
-Codex, Cursor, or any coding agent. That is the $2,000 eval prize path. No
-BenchFlow account, API key, or Azure login.
+**Send these to your coding agent.** Copy
+the block below and paste it as a message to Claude Code, Codex, OpenCode, or
+Cursor. That is the $2,000 eval prize path. No BenchFlow account, API key, or
+Azure login.
 
-```
-Submit my best local Claude Code or Codex session to the BenchFlow eval prize. Read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, upload only after I review it.
-```
+> Submit my relevant local Claude Code, Codex, OpenCode, or Cursor session from the re:Agent e2e agentic science / ai4bio hackathon (last 72 hours, on this laptop) to the BenchFlow eval prize.
+>
+> 1. First make sure the latest benchflow CLI is installed: uv tool install --python 3.12 --upgrade --force benchflow
+>
+> 2. Then read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, and upload only after I (the human) review it.
 
 The agent finds sessions on this machine, opens the viewer, and uploads after
 you like what you see.
 
+The prompt has the agent install the latest BenchFlow first (`uv tool
+install --python 3.12 --upgrade --force benchflow`). The `bench traj` commands
+also print a one-line upgrade hint when a newer release is available.
+
 Prefer the terminal instead? The guided upload inspects before anything leaves
 your machine — it renders a redacted trajectory report (step counts, masked
 secrets, preview) and asks for confirmation:
@@ -124,7 +131,7 @@ bench traj setup
 ```
 
 `npx skills add` asks which agents to install for. `bench traj setup` copies
-the skill into this project and prints the same paste line. See the
+the skill into this project and prints the same agent prompt. See the
 [upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md).
 
 ## Quick start: 2. Run with a ChatGPT or Claude subscription

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/README.md

@@ -15,17 +15,24 @@ BenchFlow is a universal environment framework: it runs AI agents against task e
 
 ## Quick start: 1. Submit a trajectory
 
-Proud of a Claude Code or Codex session? Copy this line into Claude Code,
-Codex, Cursor, or any coding agent. That is the $2,000 eval prize path. No
-BenchFlow account, API key, or Azure login.
+**Send these to your coding agent.** Copy
+the block below and paste it as a message to Claude Code, Codex, OpenCode, or
+Cursor. That is the $2,000 eval prize path. No BenchFlow account, API key, or
+Azure login.
 
-```
-Submit my best local Claude Code or Codex session to the BenchFlow eval prize. Read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, upload only after I review it.
-```
+> Submit my relevant local Claude Code, Codex, OpenCode, or Cursor session from the re:Agent e2e agentic science / ai4bio hackathon (last 72 hours, on this laptop) to the BenchFlow eval prize.
+>
+> 1. First make sure the latest benchflow CLI is installed: uv tool install --python 3.12 --upgrade --force benchflow
+>
+> 2. Then read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, and upload only after I (the human) review it.
 
 The agent finds sessions on this machine, opens the viewer, and uploads after
 you like what you see.
 
+The prompt has the agent install the latest BenchFlow first (`uv tool
+install --python 3.12 --upgrade --force benchflow`). The `bench traj` commands
+also print a one-line upgrade hint when a newer release is available.
+
 Prefer the terminal instead? The guided upload inspects before anything leaves
 your machine — it renders a redacted trajectory report (step counts, masked
 secrets, preview) and asks for confirmation:
@@ -50,7 +57,7 @@ bench traj setup
 ```
 
 `npx skills add` asks which agents to install for. `bench traj setup` copies
-the skill into this project and prints the same paste line. See the
+the skill into this project and prints the same agent prompt. See the
 [upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md).
 
 ## Quick start: 2. Run with a ChatGPT or Claude subscription

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "benchflow"
-version = "0.7.1.dev1969"
+version = "0.7.2"
 description = "Multi-turn agent benchmarking with ACP — run any agent, any model, any provider."
 readme = "README.md"
 requires-python = ">=3.12"

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/src/benchflow/cli/_traj_upload_ui.py

@@ -20,10 +20,29 @@ from rich.progress import (
 from rich.table import Table
 from rich.text import Text
 
-from benchflow.publish.redact import REDACTED
+from benchflow.publish.redact import REDACTED, format_redaction_breakdown
 from benchflow.publish.traj_capture import StagedFile
 from benchflow.publish.traj_report import PREVIEW_WORD_LIMIT, TrajectoryReport
 
+# Redaction-transparency copy shown with the masked-value count. The wording is
+# deliberately honest: redaction is a local structural pass (no agents, no
+# token spend), and the server independently rescans staged artifacts.
+NO_MASKING_NEEDED_COPY = (
+    "No secrets or personal identifiers detected — nothing needed masking."
+)
+REDACTION_REASSURANCE_COPY = (
+    "Redaction ran locally before anything was staged; "
+    "the server independently rescans and rejects any survivor."
+)
+
+
+def masked_for_you_line(report: TrajectoryReport) -> str:
+    """Itemized ``Masked for you`` copy for a report with masked values."""
+    breakdown = format_redaction_breakdown(dict(report.masked_categories))
+    if not breakdown:  # category data unavailable — fall back to the total
+        breakdown = f"{report.masked_values} secret-like values"
+    return f"{breakdown} — originals never leave this machine"
+
 
 def render_trajectory_report(report: TrajectoryReport, *, console: Console) -> None:
     """Render upload facts and a bounded, already-redacted trajectory preview."""
@@ -43,8 +62,17 @@ def render_trajectory_report(report: TrajectoryReport, *, console: Console) -> N
         "API keys / secrets masked",
         Text(str(report.masked_values), style="bold green"),
     )
+    if report.masked_values:
+        facts.add_row(
+            "Masked for you",
+            Text(masked_for_you_line(report), style="green"),
+        )
+    else:
+        facts.add_row("Masked for you", Text(NO_MASKING_NEEDED_COPY, style="dim"))
     facts.add_row("Safe replacement", Text(REDACTED))
     console.print(Panel(facts, title="Trajectory report", border_style="cyan"))
+    if report.masked_values:
+        console.print(f"[dim]{REDACTION_REASSURANCE_COPY}[/dim]")
 
     if not report.preview:
         return

{benchflow-0.7.1.dev1969 → benchflow-0.7.2}/src/benchflow/cli/main.py

@@ -1228,11 +1228,41 @@ def eval_view(
         ),
     ],
     port: Annotated[int, typer.Option(help="Server port")] = 8888,
+    confirm: Annotated[
+        bool,
+        typer.Option(
+            "--confirm",
+            help=(
+                "Show an Approve & submit / Not this one bar on the page; "
+                "print DECISION: approved|rejected and exit 0 on approve, "
+                "3 on reject."
+            ),
+        ),
+    ] = False,
+    redaction_summary: Annotated[
+        str | None,
+        typer.Option(
+            "--redaction-summary",
+            help=(
+                "Masked-secret breakdown shown in the --confirm bar, e.g. "
+                "'2 API keys, 1 bearer token' (lift it from "
+                "`bench traj upload PATH --dry-run`). Display-only; no effect "
+                "without --confirm."
+            ),
+        ),
+    ] = None,
 ) -> None:
     """View a trial or session trajectory in the browser."""
-    from benchflow.trajectories.viewer import serve
+    from benchflow.trajectories import viewer
 
-    serve(str(rollout_dir), port)
+    decision = viewer.serve(
+        str(rollout_dir), port, confirm=confirm, redaction_summary=redaction_summary
+    )
+    # Exit-code contract for --confirm: 0 approved, 3 rejected. 3 is chosen
+    # so a rejection never collides with the CLI's existing error exits
+    # (1 = errors, 2 = Typer usage errors).
+    if decision == "rejected":
+        raise typer.Exit(3)
 
 
 # ── Command-group wiring ──────────────────────────────────────────────