benchflow 0.7.1.dev1965__tar.gz → 0.7.1.dev1972__tar.gz

{benchflow-0.7.1.dev1965 → benchflow-0.7.1.dev1972}/CHANGELOG.md

@@ -2,6 +2,33 @@
 
 ## [Unreleased]
 
+### Added
+- **`bench traj setup` / `bench traj upload` print an upgrade hint when
+  outdated.** Both commands start with a lightweight PyPI latest-version
+  check (2 s timeout, completely silent on any network or parse failure) and
+  print a one-line `uv tool install --python 3.12 --upgrade --force
+  benchflow` hint when the installed version is older than the latest
+  release; dev/prereleases of a newer-or-equal base are not outdated.
+  `BENCHFLOW_SKIP_UPDATE_CHECK=1` disables the check. The
+  `benchflow-traj-upload` skill and docs now tell contributors to upgrade to
+  the latest BenchFlow (0.7.1+) before using the trajectory-upload flow.
+
+### Changed
+- **Trajectory contribution is a copy-paste line, plus optional setup.**
+  Contributors paste one line into their agent; the agent reads
+  `benchflow-traj-upload`, opens the viewer, then uploads. Optional setup is
+  `npx skills add benchflow-ai/benchflow --skill benchflow-traj-upload` or
+  `bench traj setup`. `bench eval view` accepts a raw session JSONL file and
+  does not write `trajectory.html` next to it. The CLI infers GitHub
+  username and email from `gh` / `git` before prompting, prints `Submitted` /
+  `Already submitted` plus a digest for public uploads (not a private Azure
+  inbox URL), waits up to 90s for broker cold start, and treats Azure
+  `403 UnauthorizedBlobOverwrite` as an idempotent skip. (#1008's interactive
+  report, local secret masking to `<XXX-benchflow-key-values-XXX>`, schema-1.2
+  manifest report binding, and byte progress are included; the PR #1008
+  operator manual now lives at `benchflow-traj-upload-ops` so the public skill
+  name stays contributor-facing.)
+
 ## 0.6.9 — 2026-08-15
 
 ### Changed

{benchflow-0.7.1.dev1965 → benchflow-0.7.1.dev1972}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: benchflow
-Version: 0.7.1.dev1965
+Version: 0.7.1.dev1972
 Summary: Multi-turn agent benchmarking with ACP — run any agent, any model, any provider.
 Project-URL: Homepage, https://github.com/benchflow-ai/benchflow
 Project-URL: Repository, https://github.com/benchflow-ai/benchflow
@@ -87,36 +87,51 @@ Description-Content-Type: text/markdown
 
 BenchFlow is a universal environment framework: it runs AI agents against task environments and scores them through one hardened contract. **A benchmark is just a frozen environment** — point BenchFlow at any of them, drive it with *any* ACP agent, and run single-agent, multi-agent, or multi-round patterns over the same Scene-based lifecycle.
 
-## Qucik start: 1. Upload a trajectory
+## Quick start: 1. Submit a trajectory
 
-For local Claude Code or Codex jsonl formatted trajectory files that you are proud of (contain challenging tasks you dealed with via chatting to agents), you can upload that file to BenchFlow to join the competition for winning $2,000 cash reward. No BenchFlow account, credentials, or API key is required:
+Proud of a Claude Code or Codex session? Copy this line into Claude Code,
+Codex, Cursor, or any coding agent. That is the $2,000 eval prize path. No
+BenchFlow account, API key, or Azure login.
+
+```
+Submit my best local Claude Code or Codex session to the BenchFlow eval prize. Read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, upload only after I review it.
+```
+
+The agent finds sessions on this machine, opens the viewer, and uploads after
+you like what you see.
+
+Be on the latest BenchFlow first — `uv tool install --python 3.12 --upgrade
+--force benchflow`. The `bench traj` commands print a one-line upgrade hint
+when a newer release is available.
+
+Prefer the terminal instead? The guided upload inspects before anything leaves
+your machine — it renders a redacted trajectory report (step counts, masked
+secrets, preview) and asks for confirmation:
 
 ```bash
-# Install or upgrade BenchFlow
 uv tool install --python 3.12 --upgrade benchflow
-
-# Interactive upload (prompts for path, GitHub ID, then email)
 bench traj upload
+```
+
+Detected secret values are replaced locally with
+`<XXX-benchflow-key-values-XXX>` before upload, and the full redacted report is
+retained in the uploaded `manifest.json`. See the
+[upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md) or the
+[trajectory upload guide](./docs/traj-upload.md).
 
-# Or provide every input up front
-bench traj upload /absolute/path/to/trial \
-  --github-id YOUR_GITHUB_ID \
-  --email YOU@example.com
+Optional — set the skill up once, then keep talking to the agent:
+
+```bash
+npx skills add benchflow-ai/benchflow --skill benchflow-traj-upload
+# or, if BenchFlow is already installed:
+bench traj setup
 ```
 
-The path may be a trial directory, a directory of JSONL files, or one JSONL
-file. Path and contributor details are required inputs; any that are omitted are
-prompted for interactively. Detected secret values are replaced locally with
-`<XXX-benchflow-key-values-XXX>` before upload. Contributor details are stored
-in `manifest.json`. After the path is entered, the CLI shows a redacted preview
-and a trajectory report with partitioned human, thinking, and tool-call counts,
-creation time, file size, and the number of masked key values. Interactive users
-confirm the report before a byte-progress display starts the upload. The full
-redacted report is retained in the uploaded `manifest.json`.
-See the concise [upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md)
-or the [trajectory upload guide](./docs/traj-upload.md).
-
-## Qucik start: 2. Run with a ChatGPT or Claude subscription
+`npx skills add` asks which agents to install for. `bench traj setup` copies
+the skill into this project and prints the same paste line. See the
+[upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md).
+
+## Quick start: 2. Run with a ChatGPT or Claude subscription
 
 No OpenAI or Anthropic API key is required. Start Docker, install BenchFlow,
 then run **one** of these options. BenchFlow detects the saved host login and

{benchflow-0.7.1.dev1965 → benchflow-0.7.1.dev1972}/README.md

@@ -13,36 +13,51 @@
 
 BenchFlow is a universal environment framework: it runs AI agents against task environments and scores them through one hardened contract. **A benchmark is just a frozen environment** — point BenchFlow at any of them, drive it with *any* ACP agent, and run single-agent, multi-agent, or multi-round patterns over the same Scene-based lifecycle.
 
-## Qucik start: 1. Upload a trajectory
+## Quick start: 1. Submit a trajectory
 
-For local Claude Code or Codex jsonl formatted trajectory files that you are proud of (contain challenging tasks you dealed with via chatting to agents), you can upload that file to BenchFlow to join the competition for winning $2,000 cash reward. No BenchFlow account, credentials, or API key is required:
+Proud of a Claude Code or Codex session? Copy this line into Claude Code,
+Codex, Cursor, or any coding agent. That is the $2,000 eval prize path. No
+BenchFlow account, API key, or Azure login.
+
+```
+Submit my best local Claude Code or Codex session to the BenchFlow eval prize. Read https://raw.githubusercontent.com/benchflow-ai/benchflow/main/.agents/skills/benchflow-traj-upload/SKILL.md and follow it: find a session, open the viewer, upload only after I review it.
+```
+
+The agent finds sessions on this machine, opens the viewer, and uploads after
+you like what you see.
+
+Be on the latest BenchFlow first — `uv tool install --python 3.12 --upgrade
+--force benchflow`. The `bench traj` commands print a one-line upgrade hint
+when a newer release is available.
+
+Prefer the terminal instead? The guided upload inspects before anything leaves
+your machine — it renders a redacted trajectory report (step counts, masked
+secrets, preview) and asks for confirmation:
 
 ```bash
-# Install or upgrade BenchFlow
 uv tool install --python 3.12 --upgrade benchflow
-
-# Interactive upload (prompts for path, GitHub ID, then email)
 bench traj upload
+```
+
+Detected secret values are replaced locally with
+`<XXX-benchflow-key-values-XXX>` before upload, and the full redacted report is
+retained in the uploaded `manifest.json`. See the
+[upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md) or the
+[trajectory upload guide](./docs/traj-upload.md).
 
-# Or provide every input up front
-bench traj upload /absolute/path/to/trial \
-  --github-id YOUR_GITHUB_ID \
-  --email YOU@example.com
+Optional — set the skill up once, then keep talking to the agent:
+
+```bash
+npx skills add benchflow-ai/benchflow --skill benchflow-traj-upload
+# or, if BenchFlow is already installed:
+bench traj setup
 ```
 
-The path may be a trial directory, a directory of JSONL files, or one JSONL
-file. Path and contributor details are required inputs; any that are omitted are
-prompted for interactively. Detected secret values are replaced locally with
-`<XXX-benchflow-key-values-XXX>` before upload. Contributor details are stored
-in `manifest.json`. After the path is entered, the CLI shows a redacted preview
-and a trajectory report with partitioned human, thinking, and tool-call counts,
-creation time, file size, and the number of masked key values. Interactive users
-confirm the report before a byte-progress display starts the upload. The full
-redacted report is retained in the uploaded `manifest.json`.
-See the concise [upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md)
-or the [trajectory upload guide](./docs/traj-upload.md).
-
-## Qucik start: 2. Run with a ChatGPT or Claude subscription
+`npx skills add` asks which agents to install for. `bench traj setup` copies
+the skill into this project and prints the same paste line. See the
+[upload skill](./.agents/skills/benchflow-traj-upload/SKILL.md).
+
+## Quick start: 2. Run with a ChatGPT or Claude subscription
 
 No OpenAI or Anthropic API key is required. Start Docker, install BenchFlow,
 then run **one** of these options. BenchFlow detects the saved host login and

{benchflow-0.7.1.dev1965 → benchflow-0.7.1.dev1972}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "benchflow"
-version = "0.7.1.dev1965"
+version = "0.7.1.dev1972"
 description = "Multi-turn agent benchmarking with ACP — run any agent, any model, any provider."
 readme = "README.md"
 requires-python = ">=3.12"

{benchflow-0.7.1.dev1965 → benchflow-0.7.1.dev1972}/src/benchflow/cli/main.py

@@ -1223,11 +1223,13 @@ def eval_metrics(
 def eval_view(
     rollout_dir: Annotated[
         Path,
-        typer.Argument(help="Rollout or job directory with trajectories"),
+        typer.Argument(
+            help="Rollout directory, job directory, or trajectory JSONL file"
+        ),
     ],
     port: Annotated[int, typer.Option(help="Server port")] = 8888,
 ) -> None:
-    """View a trial trajectory in the browser."""
+    """View a trial or session trajectory in the browser."""
     from benchflow.trajectories.viewer import serve
 
     serve(str(rollout_dir), port)