agentops-accelerator 0.3.1__tar.gz → 0.3.3__tar.gz

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/.claude-plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "agentops-accelerator",
       "source": "../../plugins/agentops",
       "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
-      "version": "0.3.1",
+      "version": "0.3.3",
       "keywords": [
         "agentops",
         "evaluation",

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/.github/plugin/marketplace.json

@@ -13,7 +13,7 @@
       "name": "agentops-accelerator",
       "source": "../../plugins/agentops",
       "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
-      "version": "0.3.1",
+      "version": "0.3.3",
       "keywords": [
         "agentops",
         "evaluation",

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/CHANGELOG.md

@@ -5,6 +5,173 @@ This format follows [Keep a Changelog](https://keepachangelog.com/) and adheres
 
 ## [Unreleased]
 
+## [0.3.3] - 2026-05-31
+
+### Changed
+- **Runtime dependencies now have upper bounds so a future SDK major release
+  cannot silently break installs.** `pyproject.toml` previously declared every
+  Azure-SDK dependency with only a lower bound (e.g. `azure-ai-projects>=2.0.1`),
+  so `pip install agentops-accelerator` could resolve `azure-ai-projects 3.x`
+  the day after that ships and break the agent-definition serialization (the
+  exact failure mode that produced the `invalid_payload — Required properties
+  ["kind"] are not present` regression below). Each Azure SDK dependency
+  (`azure-ai-projects`, `azure-ai-evaluation`, `azure-identity`, `azure-monitor-*`,
+  `azure-mgmt-*`) is now constrained to its current major. `pandas`, `fastapi`,
+  `uvicorn`, `httpx`, and `markdown` are similarly capped to their next major.
+  `cryptography` is intentionally left unbounded so security patches can flow
+  through without a coordinated AgentOps release. Lift any of these bounds via
+  an explicit PR that exercises the new SDK against `tests/`.
+
+- **`agentops workflow generate` now stamps the installed agentops version
+  into generated CI/CD templates instead of always installing from
+  `git+...@main`.** Every generated `agentops-pr.yml`, `agentops-deploy-*.yml`,
+  `agentops-watchdog.yml` (and their Azure DevOps pipeline equivalents) used to
+  contain `pip install "agentops-accelerator[...] @ git+https://github.com/Azure/agentops.git@main"`,
+  with no version pin and a stale "NOTE: pinned to GitHub main until the next
+  package release" comment. User CI runs were therefore non-reproducible: the
+  same workflow file pulled different agentops snapshots day to day, which is
+  how PO's recorded tutorial took a hard SDK regression mid-record. The
+  generator now writes a literal `==X.Y.Z` pin derived from the agentops version
+  currently installed on the machine running `agentops workflow generate` — so
+  a user who generates workflows against AgentOps `0.3.3` always installs
+  `agentops-accelerator==0.3.3` on every CI run, and `agentops-accelerator`
+  brings exact-major Azure SDKs along (per the upper bounds above). Editable
+  installs (versions carrying a local segment like `+gabcdef` or marked
+  `.devN`) keep the `@main` fallback so contributors testing template changes
+  still get a resolvable install. Existing user workflows are unaffected until
+  the user re-runs `agentops workflow generate --force` against a release of
+  AgentOps that ships this change.
+
+### Fixed
+- **Doctor regression check no longer flags the previous PR run as "current"
+  in CI.** The results-history loader (`agent/sources/results_history.py`)
+  was reading the wrong fields from `results.json` and excluding
+  `.agentops/results/latest/` from the candidate list. Three coordinated
+  schema-alignment fixes restore correctness:
+  1. `_summarize` now reads top-level `aggregate_metrics` first (the field
+     the orchestrator actually writes, per `core/results.py`), then falls
+     back to legacy `metrics`/`run_metrics`. Previously the loader looked
+     only at the legacy fields, so every freshly-written local
+     `RunSummary` had `metrics = {}` and the regression check could never
+     see the current run's metrics.
+  2. `_summarize` now reads `summary.overall_passed` first when deriving
+     the `run_pass` flag, then falls back to the legacy `summary.run_pass`
+     / `metrics.run_pass` shapes.
+  3. `_summarize` now orders runs by `timestamp` → `finished_at` →
+     `started_at` → `created_at` → `summary.timestamp`. The previous list
+     omitted `finished_at`/`started_at`, which are the two fields
+     `results.json` actually contains, so every loaded run defaulted to
+     epoch-zero ordering.
+  4. `_collect_local_runs` now includes `.agentops/results/latest/` when it
+     is the only local results directory. In CI, generated workflows run
+     `agentops eval run --output .agentops/results/latest` and write
+     nowhere else; the old loader unconditionally skipped `latest/` for
+     dev-mode dedup, so in CI `local_runs` was always empty. With cloud
+     listing trailing behind by seconds (eventual consistency), the
+     regression check would then compute `latest = previous_run` and
+     blame the just-completed candidate's coherence/groundedness on the
+     prior PR. Dev-mode dedup is preserved: when a timestamped sibling
+     exists, `latest/` is still skipped.
+- **Prompt-agent deploy: `stage` no longer fails with `Required properties ["kind"] are not present` against `azure-ai-projects` 2.x.**
+  `_copy_definition` previously called `.copy()` on the typed
+  `PromptAgentDefinition` returned by `get_version`. In SDK 1.x that
+  preserved the typed model so the body serialized as a flat
+  `{"kind": "prompt", "model": ..., "instructions": ...}`. In SDK 2.x
+  the same `.copy()` returns a stripped base `Model` whose JSON shape
+  is `{"_data": {"kind": "prompt", ...}}`, and `.get("kind")` returns
+  `None` — so the request body that reached the Foundry Agents service
+  contained `definition: {"_data": {...}}` with no top-level `kind`,
+  and the service rejected it with `invalid_payload`. This regression
+  only fired on the `created` action path (i.e. when the user's prompt
+  differed from the seed); the `reused` and bootstrap paths were
+  unaffected because they don't round-trip the typed model through
+  `.copy()`. `_copy_definition` now normalizes any SDK definition
+  object to a plain `dict` before mutation, and `_create_agent_version`
+  no longer puts a root-level `kind` on the request body (the new API
+  treats `kind` strictly as the discriminator inside `definition`).
+- **Tutorial: prompt-agent step 13 now shows the steady-state `foundry-agent.json` (action: reused) instead of the bootstrap edge case.**
+  The example JSON in step 13 previously showed `action: bootstrapped`
+  with `candidate_agent: "travel-agent:1"` and a "the two numbers are
+  expected to differ until the environment has caught up to the seed"
+  explanation. In practice the merge-triggered deploy is almost never
+  the run that bootstraps — by the time the user reaches step 13, the
+  skill's verification dispatch in step 12 plus the first PR run have
+  already settled dev to `travel-agent:2`, so the merge deploy reports
+  `action: reused` with `candidate_agent: "travel-agent:2"` (matching
+  `source_agent`). The example now shows the steady-state shape (taken
+  from a real recording), uses the runner-resolved absolute paths the
+  user actually sees (`/home/runner/work/<your-repo>/...`), and uses a
+  real 64-char `prompt_sha256` + a real ISO timestamp. The
+  three-outcome list (`reused` / `created` / `bootstrapped`) below the
+  JSON keeps the bootstrap case as the documented edge condition.
+- **Tutorial: prompt-agent step 13 now matches what the workflow skill actually does (dispatches both workflows).**
+  PR #211 mistakenly narrowed the step 13 callout to say the workflow
+  skill only dispatches `agentops-pr.yml` as a verification run, based
+  on incorrect reasoning about `push:` triggers (the skill actually
+  uses `workflow_dispatch`, which works against any branch regardless
+  of the workflow's `push:` block). In practice — verified against a
+  live recording — the skill dispatches **both** `agentops-pr.yml`
+  and `agentops-deploy-dev.yml` end-to-end as part of CI verification,
+  asking the user to approve first per SKILL.md rule #14. The step 13
+  callout now reflects this and explains the expected outcome (both
+  runs may exit `threshold_failed` on first contact with an empty dev
+  project because the bootstrap path produces a fresh `travel-agent:1`
+  that has not been measured against the seed thresholds yet — by
+  design, not a CI wiring failure). The "What you should see in the
+  first PR workflow run" section also updates from the
+  "dev is still empty" assumption (which becomes false after the
+  skill's verification dispatch) to the three possible outcomes
+  (`reused` / `created` / `bootstrapped`) you can actually see at this
+  point. The "After the merge" paragraph now calls out that the
+  merge-triggered deploy is the **second** deploy-dev run for the
+  repo, not the first.
+- **Tutorials: end-to-end audit caught misleading dist URLs, phantom CLI commands, missing JSON fields, and stale Doctor advisory text.**
+  All three tutorials previously installed the development build from a
+  personal fork URL (`git+https://github.com/placerda/agentops.git@develop`);
+  they now point at the canonical
+  `git+https://github.com/Azure/agentops.git@develop`. The prompt-agent
+  tutorial referenced a non-existent `prompt_deploy record` subcommand in
+  two places — the actual command is `prompt_deploy summarize`, matching
+  `src/agentops/pipeline/prompt_deploy.py` and the deploy template's
+  `Mark candidate as deployed` step. The same tutorial's `foundry-agent.json`
+  sample was missing the `eval_config` field that the code writes at
+  `src/agentops/pipeline/prompt_deploy.py:186`. The step 12 skill prompt
+  and the step 13 prose did not tell the reader to rewrite the dev-deploy
+  trigger from `develop` to `main` for this trunk-on-`main` tutorial; the
+  generator's stock default is `develop`, which would silently no-op after
+  the first merge. Step 12 now instructs the skill to do the rewrite (and
+  the bullet list of skill actions calls it out as a required step, with
+  a manual-edit fallback). Step 13's "deploy fires automatically on `main`"
+  sentence now states the dependency on the step 12 rewrite explicitly,
+  and the placeholder phrase "your trunk branch" is now disambiguated as
+  "`main` in this tutorial". The end-to-end tutorial's step 5 and step 9
+  Doctor descriptions still read as if Doctor were advisory-only in PR
+  workflows — that text predates the `--doctor-gate critical` default;
+  both blocks now describe the actual behavior (critical findings block
+  the PR by default; warning/info are evidence-only).
+
+### Changed
+- **Tutorials: skip-if-skill callouts now state the skill's outcome directly and accurately.**
+  The `step 13` callout in `docs/tutorial-prompt-agent-quickstart.md` and the
+  baseline-run paragraph in `docs/tutorial-end-to-end.md` previously opened
+  with "if you used the workflow skill, this is already done…" plus a
+  manual-fallback block. That conditional framing was confusing because the
+  preceding step (`step 12` of the prompt-agent tutorial, `step 5` of the
+  end-to-end tutorial) only documents the workflow-skill path — there is no
+  alternative wired-by-hand path the reader could have taken. Both callouts
+  now state the skill's outcome directly, and the redundant `git add` /
+  `commit` / `push` and `gh workflow run agentops-pr.yml --ref main` blocks
+  have been removed (the skill already triggers the first run). A small
+  `gh run list` / `gh run watch` snippet remains as an opt-in way to wait
+  on the run from the terminal instead of the Actions UI. The previous
+  wording also over-claimed that the skill triggered verification runs of
+  **both** `agentops-pr.yml` and `agentops-deploy-dev.yml`; the skill only
+  dispatches the PR workflow as a sanity check (`workflow_dispatch`), while
+  `agentops-deploy-dev.yml` triggers on the first real merge into the trunk
+  branch. The callout now reflects this accurately and notes that the
+  deploy-dev run happens at the end of the section, not during the skill's
+  setup.
+
 ## [0.3.1] - 2026-05-29
 
 ### Changed

{agentops_accelerator-0.3.1/src/agentops_accelerator.egg-info → agentops_accelerator-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentops-accelerator
-Version: 0.3.1
+Version: 0.3.3
 Summary: Release readiness gates and evidence for Microsoft Foundry agents
 License: MIT License
         
@@ -30,25 +30,25 @@ License-File: LICENSE
 Requires-Dist: typer<1.0,>=0.12
 Requires-Dist: pydantic<3,>=2
 Requires-Dist: ruamel.yaml<1.0,>=0.18
-Requires-Dist: azure-ai-projects>=2.0.1
+Requires-Dist: azure-ai-projects<3.0,>=2.0.1
 Provides-Extra: mcp
 Requires-Dist: mcp<2,>=1.0; extra == "mcp"
 Provides-Extra: foundry
-Requires-Dist: azure-ai-evaluation>=1.0; extra == "foundry"
-Requires-Dist: azure-identity>=1.17; extra == "foundry"
-Requires-Dist: azure-monitor-opentelemetry>=1.6; extra == "foundry"
-Requires-Dist: pandas>=2.0; extra == "foundry"
+Requires-Dist: azure-ai-evaluation<2.0,>=1.0; extra == "foundry"
+Requires-Dist: azure-identity<2.0,>=1.17; extra == "foundry"
+Requires-Dist: azure-monitor-opentelemetry<2.0,>=1.6; extra == "foundry"
+Requires-Dist: pandas<3.0,>=2.0; extra == "foundry"
 Provides-Extra: agent
-Requires-Dist: fastapi>=0.110; extra == "agent"
-Requires-Dist: uvicorn[standard]>=0.30; extra == "agent"
-Requires-Dist: httpx>=0.27; extra == "agent"
+Requires-Dist: fastapi<1.0,>=0.110; extra == "agent"
+Requires-Dist: uvicorn[standard]<1.0,>=0.30; extra == "agent"
+Requires-Dist: httpx<1.0,>=0.27; extra == "agent"
 Requires-Dist: cryptography>=42; extra == "agent"
-Requires-Dist: markdown>=3.6; extra == "agent"
-Requires-Dist: azure-monitor-query>=1.3; extra == "agent"
-Requires-Dist: azure-monitor-opentelemetry>=1.6; extra == "agent"
-Requires-Dist: azure-identity>=1.17; extra == "agent"
-Requires-Dist: azure-mgmt-cognitiveservices>=13.5; extra == "agent"
-Requires-Dist: azure-mgmt-monitor>=6.0; extra == "agent"
+Requires-Dist: markdown<4.0,>=3.6; extra == "agent"
+Requires-Dist: azure-monitor-query<2.0,>=1.3; extra == "agent"
+Requires-Dist: azure-monitor-opentelemetry<2.0,>=1.6; extra == "agent"
+Requires-Dist: azure-identity<2.0,>=1.17; extra == "agent"
+Requires-Dist: azure-mgmt-cognitiveservices<14.0,>=13.5; extra == "agent"
+Requires-Dist: azure-mgmt-monitor<7.0,>=6.0; extra == "agent"
 Dynamic: license-file
 
 <h1 align="center">AgentOps Accelerator</h1>

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/docs/tutorial-end-to-end.md

@@ -129,7 +129,7 @@ install the aligned reference branch so the CLI, generated workflows, and
 tutorial steps stay in sync:
 
 ```powershell
-python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/placerda/agentops.git@develop"
+python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@develop"
 ```
 
 You will provide the target values through the interactive `agentops init`
@@ -451,9 +451,13 @@ The generated workflow prepares a temporary cloud config, runs
 
 It also records release evidence after the gate.
 
-In PR workflows, that Doctor evidence is advisory: the eval step is the merge
-gate, and `Release readiness: blocked` means the evidence found release work to
-review. Production deploy workflows still run Doctor as a critical release gate.
+Doctor runs in the PR workflow with `--severity-fail critical` (the
+`--doctor-gate critical` default). A critical Doctor finding — for example
+`regression.coherence: critical` from a metric drop that still passes
+thresholds — fails the PR check the same way an eval threshold breach
+does. Warning- and info-level findings are advisory and attached to the
+PR as evidence. Production deploy workflows always run Doctor with
+`--severity-fail critical` regardless of this flag.
 
 No tutorial-only Action replacement is needed. The generated workflow keeps the
 evaluation in Foundry while AgentOps owns the CI threshold decision and the
@@ -468,14 +472,10 @@ and rerun the same gate.
 
 ### Prompt Agent regression
 
-Make sure the original prompt version has one green workflow run before you
-change it. **If you used the workflow skill in step 5 above, this is already
-done** — the skill commits your changes, pushes to GitHub, and triggers a first
-verification run of `agentops-pr.yml`. Open the latest workflow run's Foundry
-Evaluations link and keep that page open as the baseline. If you skipped the
-skill and wired CI by hand, commit the generated workflow and `agentops.yaml`,
-run `gh workflow run agentops-pr.yml --ref main`, and use that run as the
-baseline.
+The workflow skill in step 5 above already committed your changes, pushed
+`main` to GitHub, and triggered a first verification run of `agentops-pr.yml`.
+Open the latest workflow run's Foundry Evaluations link and keep that page
+open as the baseline.
 
 1. In Foundry, edit the `travel-agent` instructions to this intentionally bad
    version:
@@ -755,10 +755,12 @@ show whether the repo has the release machinery reviewers expect. Use critical
 findings as release blockers and warning/info findings as the backlog that turns
 the POC into an operated service.
 
-If those same Doctor findings appear inside a PR workflow, treat them as
-evidence attached to the PR rather than as the merge gate. The eval step gates
-the PR; production deploy workflows are where critical Doctor findings block the
-release path.
+If those same Doctor findings appear inside a PR workflow, critical
+findings block the merge by default (the PR template runs Doctor with
+`--severity-fail critical`, the `--doctor-gate critical` default); warning
+and info findings are attached to the PR as evidence rather than as a
+gate. Production deploy workflows always run Doctor with
+`--severity-fail critical` and are the last-mile release gate.
 
 Open both files. The Doctor report is the diagnostic view: it tells you which
 signals are present, which are missing, and whether the finding is blocking or

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/docs/tutorial-hosted-agent-quickstart.md

@@ -160,7 +160,7 @@ install the aligned reference branch so the CLI, generated workflows, and
 tutorial steps stay in sync:
 
 ```powershell
-python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/placerda/agentops.git@develop"
+python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@develop"
 ```
 
 ## 2. Create the Travel Agent endpoint

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/docs/tutorial-prompt-agent-quickstart.md

@@ -137,7 +137,7 @@ path, install the aligned reference branch so the CLI, generated
 workflows, and tutorial steps stay in sync:
 
 ```powershell
-python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/placerda/agentops.git@develop"
+python -m pip install "agentops-accelerator[foundry,agent] @ git+https://github.com/Azure/agentops.git@develop"
 ```
 
 ## 2. Install the AgentOps Copilot skills
@@ -670,7 +670,7 @@ The PR workflow now has two jobs:
 > candidate version numbers.
 
 The dev deploy workflow stages a candidate (same logic), evaluates it,
-records the deployment via `prompt_deploy record`, and uploads
+summarizes the deployment via `prompt_deploy summarize`, and uploads
 `.agentops/deployments/foundry-agent.json` as a workflow artifact.
 
 The `--doctor-gate critical` flag controls the Doctor severity floor in
@@ -725,6 +725,14 @@ the OpenAI User role, the Foundry cloud graders fail with a 401 and every
 metric comes back null), and do not set up `qa`, `production`, scheduled
 Doctor, or hosted deployment workflows yet.
 
+I am using trunk-based development with `main` as both my trunk and dev
+branch. The generator's stock dev-deploy trigger is `push: branches:
+[develop]`. Rewrite the `agentops-deploy-dev.yml` (and the matching
+`agentops-pr.yml` `pull_request: branches:` list, if it references
+`develop`) so they fire on `main` instead. The PR gate must run on PRs
+targeting `main`, and the dev deploy must auto-run on push to `main`
+after a merge.
+
 The dev Foundry project endpoint is in `.azure/dev/.env`; the sandbox
 endpoint is local-only and must not be added to CI.
 
@@ -741,6 +749,14 @@ it skips:
 - Set Actions variables `AZURE_TENANT_ID`, `AZURE_SUBSCRIPTION_ID`,
   `AZURE_CLIENT_ID`, `AZURE_AI_FOUNDRY_PROJECT_ENDPOINT` (the dev
   endpoint), and `APPLICATIONINSIGHTS_CONNECTION_STRING` if available.
+- **Rewrite the dev deploy trigger to `main`.** The generator emits the
+  stock GitFlow defaults (`pull_request: branches: [develop, "release/**",
+  main]` on `agentops-pr.yml`, `push: branches: [develop]` on
+  `agentops-deploy-dev.yml`). For this trunk-on-`main` tutorial the
+  skill should rewrite both so the PR gate fires on PRs into `main` and
+  the deploy fires on push to `main`. If the skill skips this rewrite,
+  open the two YAML files in `.github/workflows/` and edit the
+  `branches:` lists by hand before opening the first PR.
 - Verify the OIDC principal has **two** Azure RBAC roles before the first
   run. Both are required and the eval step fails silently (every metric
   returns `null`) if only one is in place:
@@ -761,66 +777,71 @@ This is the happy path. Before the regression step, you need a clean
 green baseline so the rolling-history Doctor checks (regression, drift)
 have something to compare against.
 
-> **If you used the workflow skill in step 12, the next two code
-> blocks have already been done — skip straight to "Now open a feature
-> branch..." below.** The `agentops-workflow` skill commits your local
-> changes, pushes `main` to the GitHub remote, and triggers a first
-> verification run of both `agentops-pr.yml` and `agentops-deploy-dev.yml`
-> as part of its end-to-end setup. Open the repo's **Actions** tab and
-> confirm both runs are present — `Stage Foundry prompt candidate (PR)`
-> + `AgentOps eval (PR gate)` should be green on the PR workflow, and
-> `agentops-deploy-dev.yml` should have a matching run that also went
-> green. Only run the `git add` / `commit` / `push` and `gh workflow run`
-> commands below if you skipped the skill and wired CI by hand, or if
-> the skill stopped before pushing your latest local changes (run
-> `git status` first — if the working tree is clean and the remote has
-> your commits, the manual commands are no-ops).
-
-```powershell
-git add agentops.yaml .agentops .github\workflows
-git commit -m "Add AgentOps prompt agent gate + dev deploy"
-git push -u origin main
-```
-
-Open the repository in GitHub and confirm both workflows appear under
-**Actions**. Trigger the PR workflow once on `main` so the dev project
-has a known-good candidate run:
+The workflow skill in step 12 already committed your local changes,
+pushed `main` to the GitHub remote, and dispatched first verification
+runs of **both** `agentops-pr.yml` and `agentops-deploy-dev.yml` (via
+`workflow_dispatch`, after asking you to approve) so the CI wiring is
+verified end-to-end. Open the repo's **Actions** tab and confirm both
+runs reached the eval stage:
+
+- `agentops-pr.yml` — `Stage Foundry prompt candidate (PR)` and
+  `AgentOps eval (PR gate)` jobs both ran.
+- `agentops-deploy-dev.yml` — `stage-candidate`, `eval`, and the
+  `Mark candidate as deployed` step all ran (the deploy job uses
+  `prompt_deploy summarize`, not a real Foundry promotion — it writes
+  the deployment record artifact + workflow summary).
+
+It is **expected** for one or both of these first runs to exit
+`threshold_failed` (`exit 2`) when the dev Foundry project starts
+empty: the bootstrap path creates a fresh `travel-agent:1` (and, on
+the next run, `:2`) in dev and evaluates it against the seed
+`agentops.yaml` thresholds, which can miss on first contact. That is
+by design, not a CI wiring failure. What you are really verifying at
+this point is the plumbing — OIDC, Foundry RBAC, the evaluator
+deployment, the staging step, the deploy summary writer — and that
+dev now contains a bootstrapped version of the agent.
+
+`agentops-deploy-dev.yml` will fire **again** automatically when you
+merge the baseline PR at the end of this section, because the skill
+rewrote its trigger from `develop` to `main` in step 12.
+
+If you want to wait on the first PR-workflow verification run from the
+terminal instead of the Actions UI:
 
 ```powershell
-gh workflow run agentops-pr.yml --ref main
-Start-Sleep -Seconds 10
 $runId = gh run list --workflow agentops-pr.yml --branch main --limit 1 --json databaseId --jq '.[0].databaseId'
 gh run view $runId --web
 gh run watch $runId --exit-status
 ```
 
-What you should see in the **first** PR workflow run (dev is still
-empty at this point):
+What you should see in the **first** PR workflow run, after the
+skill's verification dispatches have already touched dev:
 
 1. **Stage Foundry prompt candidate (PR)** job runs first. The
-   `prompt_deploy stage` step tries to look up `travel-agent:2` in the
-   dev project and gets a 404. Because `agentops.yaml` includes a
-   `prompt_agent_bootstrap` block, the step:
-   - reads the `model` (`gpt-4o-mini`) and optional `description` from
-     `prompt_agent_bootstrap`,
-   - reads the instructions from `prompt_file`,
-   - creates a new version of `travel-agent` in the dev project from
-     those defaults via the Foundry SDK. The SDK assigns the version
-     number per-project; in an empty dev project the first version is
-     normally `travel-agent:1` (not `:2`, because dev has no draft
-     history of its own),
-   - reports `action: bootstrapped` and uses the freshly-bootstrapped
-     version as the candidate.
+   `prompt_deploy stage` step looks up `travel-agent:2` in the dev
+   project. Three outcomes are possible depending on what the skill's
+   verification dispatches produced:
+   - `action: reused` — dev already has `travel-agent:2` with the
+     same instructions as the seed (no new version created).
+   - `action: created` — dev has the seed version but with different
+     instructions, so Foundry auto-creates the next number (likely
+     `travel-agent:3`).
+   - `action: bootstrapped` — dev still does not have `travel-agent:2`
+     (only `:1`, because the bootstrap can fire `:1` and `:2`
+     back-to-back over two runs). The step reads
+     `prompt_agent_bootstrap` plus `prompt_file` and creates the next
+     SDK-assigned version, then uses it as the candidate.
 2. **AgentOps eval (PR gate)** job runs second. It evaluates the
-   bootstrapped candidate using cloud eval. Thresholds pass.
-   Doctor runs with `--severity-fail critical`; advisory findings are
-   listed but do not fail the job.
+   candidate using cloud eval. Doctor runs with
+   `--severity-fail critical`; advisory findings are listed but do not
+   fail the job. The first one or two PR runs against a fresh dev
+   project can still fail thresholds while bootstrap catches up. After
+   that, normal reuse / create flow takes over and the baseline PR
+   should go green.
 
-On the **second** PR run, dev has `travel-agent:1` but the seed still
-points at `:2`. The stage step gets another 404 looking up `:2` and
-bootstraps a second version — `travel-agent:2` — into dev. After this,
-the dev project finally has `travel-agent:2` matching the seed, and
-every subsequent PR takes the normal lookup path:
+Successive PR runs walk the same three branches above until dev's
+version count catches up to the seed (`travel-agent:2`). Once it does,
+every PR run hits the normal lookup path:
 
 - If `prompt_file` is byte-identical to the seed's instructions: the
   stage step reports `reused` and uses `travel-agent:2` as the
@@ -849,18 +870,26 @@ git push -u origin chore/agentops-baseline
 gh pr create --base main --head chore/agentops-baseline --title "Baseline AgentOps run" --body "First green PR to establish history."
 ```
 
-Open the PR in GitHub. The PR check runs the same staging + eval flow,
-green again because the prompt is unchanged. Merge.
+Open the PR in GitHub. The PR check runs the same staging + eval flow.
+Whether this baseline PR goes green on the first try depends on how
+many bootstrap rounds the dev project has already absorbed (from the
+skill's verification dispatches plus any failed PRs). Once bootstrap
+catches up to the seed and the prompt is stable, the PR goes green —
+re-run the workflow on the PR if needed. Then merge.
 
 After the merge, the **AgentOps deploy (dev)** workflow runs
-automatically on `main`. It stages the candidate (likely re-using the
-same version as the PR run, or bootstrapping again if dev has not yet
-caught up to the seed), evaluates it, runs `prompt_deploy record` to
-mark it as the dev deployment, and uploads the deployment artifact.
+automatically on `main` (the skill rewrote its trigger from `develop`
+to `main` in step 12 because this tutorial uses trunk-based flow).
+This is the **second** deploy-dev run for this repo — the first was
+the skill's verification dispatch in step 12. It stages the candidate
+(by this point most likely `action: reused` or `created`), evaluates
+it, runs `prompt_deploy summarize` to write the dev deployment summary,
+and uploads the deployment artifact.
 
 Open the deploy run and download the `foundry-agent-dev-deployment`
-artifact. Inside, open `foundry-agent.json`. On the very first deploy
-into an empty dev project (the bootstrap case), the file looks like
+artifact. Inside, open `foundry-agent.json`. In the **steady-state**
+case (the most common — the seed `travel-agent:2` already exists in
+dev and matches the prompt the PR shipped), the file looks like
 this — note the actual field names AgentOps writes:
 
 ```json
@@ -868,34 +897,47 @@ this — note the actual field names AgentOps writes:
   "version": 1,
   "type": "foundry_prompt_agent_deployment",
   "environment": "dev",
-  "action": "bootstrapped",
+  "action": "reused",
   "agent_name": "travel-agent",
   "source_agent": "travel-agent:2",
-  "candidate_agent": "travel-agent:1",
+  "candidate_agent": "travel-agent:2",
   "source_version": "2",
-  "candidate_version": "1",
+  "candidate_version": "2",
   "project_endpoint": "https://<your-resource>.services.ai.azure.com/api/projects/travel-agent-dev",
-  "prompt_file": ".agentops/prompts/travel-agent.md",
-  "prompt_sha256": "9c3a...e0b1",
-  "created_at": "2025-...",
-  "git_sha": "5f1a2c...",
-  "workflow_url": "https://github.com/.../actions/runs/...",
-  "foundry_agent_version_id": "..."
+  "prompt_file": "/home/runner/work/<your-repo>/<your-repo>/.agentops/prompts/travel-agent.md",
+  "prompt_sha256": "9727437db863b00d52bc8ef1f314b70ed22e3e562f5a3a1f9dd68e26f7ea0975",
+  "eval_config": "/home/runner/work/<your-repo>/<your-repo>/.agentops/deployments/agentops.candidate.yaml",
+  "created_at": "2026-05-30T17:57:53.135435+00:00",
+  "git_sha": "3078df74c3b18625553dec8ecd4ed4282f1ca1ca",
+  "workflow_url": "https://github.com/<owner>/<your-repo>/actions/runs/26690922142",
+  "foundry_agent_version_id": "travel-agent:2"
 }
 ```
 
-The `source_agent` field records what `agent:` in `agentops.yaml`
-pointed at (`travel-agent:2`, the sandbox seed). The `candidate_agent`
-field records what CI actually produced and evaluated in this
-environment (`travel-agent:1`, because dev was empty and the SDK
-assigned `:1`). The two numbers are expected to differ until the
-environment has caught up to the seed.
-
-On every subsequent deploy, `action` switches to either `reused` (when
-the prompt is byte-identical to the previous seed) or `created` (when
-Foundry auto-created a new version because the prompt changed), and
-`candidate_agent` reflects the actual version that was evaluated and
-recorded.
+In the steady-state, `source_agent` and `candidate_agent` are
+**identical** (`travel-agent:2`) because the dev project already had
+`travel-agent:2` with the same instructions as the PR's `prompt_file`,
+so `prompt_deploy stage` reported `action: reused` and nothing new
+was created. The `prompt_file` and `eval_config` paths are absolute
+because they are resolved inside the GitHub Actions runner workspace
+(`/home/runner/work/<your-repo>/<your-repo>/...`).
+
+`action` will be one of:
+
+- **`reused`** — dev already had `travel-agent:2` with byte-identical
+  instructions. No new Foundry version was created. (Steady-state and
+  most-common case.)
+- **`created`** — dev had `travel-agent:2` but with **different**
+  instructions, so Foundry auto-created the next number (e.g.
+  `travel-agent:3`). `candidate_agent` would then be `travel-agent:3`.
+- **`bootstrapped`** — dev did not yet have `travel-agent:2` at all,
+  so the stage step fell back to `prompt_agent_bootstrap` defaults
+  plus `prompt_file` and asked the SDK to create the first version.
+  In a fresh, empty dev project the SDK starts at `:1`, so you would
+  see `candidate_agent: "travel-agent:1"` and `candidate_version: "1"`
+  while `source_agent` still reports the seed (`travel-agent:2`). The
+  two numbers stay different until subsequent runs catch dev up to
+  the seed.
 
 That `prompt_sha256` + `git_sha` pair is what the mental-model diagram
 at the start of the tutorial referred to as **cross-environment

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/plugins/agentops/package.json

@@ -2,7 +2,7 @@
   "name": "agentops-accelerator",
   "displayName": "AgentOps Accelerator — Skills for GitHub Copilot",
   "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Accelerator and Microsoft Foundry agents.",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "publisher": "AgentOpsAccelerator",
   "icon": "icon.png",
   "license": "MIT",

{agentops_accelerator-0.3.1 → agentops_accelerator-0.3.3}/plugins/agentops/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "agentops-accelerator",
   "description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Accelerator and Microsoft Foundry agents.",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "author": {
     "name": "AgentOps Accelerator",
     "url": "https://github.com/Azure/agentops"