agentv 5.3.0-next.1 → 5.3.2-next.1

package/dist/skills/agentv-eval-writer/SKILL.md

@@ -20,14 +20,14 @@ Promptfoo parity matrix: https://agentv.dev/docs/reference/promptfoo-parity/
 Treat YAML as the canonical portable model. Prefer authoring `.eval.yaml` / `EVAL.yaml` first, then use TypeScript helpers, Python scripts, or executable graders only when they lower to the same fields or when the evaluation logic must actually run code.
 
 Eval files define what is tested and how it runs: prompts, datasets, assertions,
-task fixtures, top-level `target`, and suite run controls. Use field-local file
+task fixtures, top-level `providers`, and suite run controls. Use field-local file
 refs such as `tests: file://...`, `prompts: file://...`, `default_test:
 file://...`, and `environment: file://...`. String-valued `tests` and string
 entries inside `tests[]` are raw-case refs for direct paths, directories, and
 globs. Run several full eval suites directly with CLI multi-file selection and
 tags. Use scoped `run:` on individual tests only for `threshold`, `repeat`,
-`timeout_seconds`, and legacy `budget_usd`; keep target selection at top-level
-`target` or CLI `--target`, put suite budget caps under
+`timeout_seconds`, and legacy `budget_usd`; keep provider selection at top-level
+`providers` or CLI `--provider`, put suite budget caps under
 `evaluate_options.budget_usd`, authored concurrency under
 `evaluate_options.max_concurrency`, suite repeat policy under
 `evaluate_options.repeat`, coding-agent testbed setup under `environment`,
@@ -39,8 +39,8 @@ Use `@agentv/sdk` for TypeScript helper imports. Do not use `@agentv/eval` for n
 ## Authoring Checklist
 
 - Put grading criteria in `assert`, not in test-level `criteria`. Plain assertion strings become an `llm-rubric` grader.
-- Prefer plain assertion strings for semantic checks when the default rubric grader can judge them. Use `type: llm-rubric` for structured criteria, custom prompts, custom grader targets, or assertion-level transforms, and `type: script` when grading must execute code.
-- Write `expected_output` as a golden/reference answer the target could have produced. Do not write criteria, scoring instructions, or "the agent should..." rubric prose there.
+- Prefer plain assertion strings for semantic checks when the default rubric grader can judge them. Use `type: llm-rubric` for structured criteria, custom prompts, custom grader providers, or assertion-level transforms. Use `type: agent-rubric` when the grader itself must be an agent-capable provider that can inspect the workspace. Use `type: script` when grading must execute code.
+- Put reference answers in `tests[].vars.expected_output` or `default_test.vars.expected_output`, and consume them with an explicit assertion such as `type: llm-rubric` with `value: "Matches the reference answer: {{ expected_output }}"`. Do not write criteria, scoring instructions, or "the agent should..." rubric prose as the reference answer.
 - For historical or repo-state evals, materialize the repo through a pinned `environment` setup recipe. Mentioning a SHA only in prompt prose is not enough because the agent needs an actual checkout to inspect.
 
 ## Evaluation Types
@@ -61,7 +61,12 @@ agentv convert evals.json
 agentv eval evals.json
 ```
 
-The converter maps `prompt` → `input`, `expected_output` → `expected_output`, and Agent Skills `assertions` → AgentV `assert` (`llm-rubric` checks), and resolves `files[]` paths. The generated YAML includes TODO comments for AgentV features to add (environment setup, script graders, rubrics, required gates).
+The converter maps Agent Skills prompt text into AgentV prompt/vars data,
+promotes Agent Skills `expected_output` into explicit `llm-rubric` criteria or
+`vars.expected_output` when it is true reference data, and maps Agent Skills
+`assertions` to AgentV `assert` (`llm-rubric` checks). The generated YAML
+includes TODO comments for AgentV features to add (environment setup, script
+graders, rubrics, required gates).
 
 After converting, enhance the YAML with AgentV-specific capabilities shown below.
 
@@ -103,18 +108,22 @@ prompts:
 
 tests:
   - id: multi-turn-context
-    expected_output: "Your name is Alice."
+    vars:
+      expected_output: "Your name is Alice."
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - Correctly recalls the user's name from earlier in the conversation
 ```
 
-**Guidelines:** preserve exact wording in `expected_output`; aim for 5–15 tests per transcript; pick exchanges that test different capabilities.
+**Guidelines:** preserve exact wording in `vars.expected_output`; aim for 5–15 tests per transcript; pick exchanges that test different capabilities.
 
 ## Quick Start
 
 ```yaml
 description: Example eval
-target: default
+providers:
+  - default
 
 prompts:
   - "{{ prompt }}"
@@ -123,8 +132,10 @@ tests:
   - id: greeting
     vars:
       prompt: "Say hello"
-    expected_output: "Hello! How can I help you?"
+      expected_output: "Hello! How can I help you?"
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - Greeting is friendly and warm
       - Offers to help
 ```
@@ -132,7 +143,7 @@ tests:
 ## Eval File Structure
 
 **Required:** `tests` (array or string raw-case path) or `scenarios`
-**Optional:** `name`, `description`, `experiment`, `version`, `author`, `tags`, `license`, `requires`, `target`, `targets`, `prompts`, `default_test`, `timeout_seconds`, `evaluate_options`, `threshold`, `suite`, `environment`, `env`, `extensions`, `assert`
+**Optional:** `name`, `description`, `version`, `author`, `tags`, `license`, `requires`, `providers`, `prompts`, `default_test`, `timeout_seconds`, `evaluate_options`, `threshold`, `suite`, `environment`, `env`, `extensions`, `assert`
 
 **Test fields:**
 
@@ -140,24 +151,35 @@ tests:
 |-------|----------|-------------|
 | `id` | yes | Unique identifier |
 | `vars` | yes when the prompt needs row data | Prompt-template variables for this row |
-| `expected_output` | no | Gold-standard reference answer (string shorthand or full message array) |
-| `assert` | yes | Graders: deterministic checks, `llm-rubric` checks, script graders, or plain string rubric criteria |
-| `execution` | no | Per-case grader/default overrides such as `skip_defaults`; target selection belongs in top-level `target` or CLI `--target` |
+| `vars.expected_output` | no | Conventional reference-answer var consumed by explicit graders |
+| `assert` | yes | Graders: deterministic checks, `llm-rubric` / `agent-rubric` checks, script graders, or plain string rubric criteria |
+| `execution` | no | Per-case grader/default overrides such as `skip_defaults`; provider selection belongs in top-level `providers` or CLI `--provider` |
 | `environment` | no | Per-case coding-agent testbed config (overrides suite-level) |
 | `metadata` | no | Arbitrary key-value pairs passed to setup/teardown scripts |
 | `conversation_id` | no | Thread grouping |
 
+**Provider declarations:** AgentV accepts Promptfoo-shaped provider entries
+where they map cleanly: strings such as `openai:gpt-4.1-mini`, object form with
+`id`, `label`, `config`, `env`, `prompts`, `transform`, `delay`, and `inputs`,
+and provider maps such as `{ "openai:gpt-4": { label, config } }`. In object
+form, `id` is the backend/spec and `label` is the stable AgentV selection and
+result identity. AgentV-only `runtime`, provider-local `environment`, and
+provider `hooks` must be explicit extensions; direct Promptfoo runs do not
+execute them, and export must lower supported cases or reject unsupported ones
+clearly.
+
 ## Prompt Templates and Vars
 
 Use top-level `prompts` plus `tests[].vars` for the Promptfoo-compatible canonical
 input shape. Shared data defaults belong in `default_test.vars`; per-test
 `vars` override those defaults by key. AgentV renders every prompt with each
 test's merged vars, then expands the run across prompts, targets, tests, and
-repeat attempts.
+repeat samples.
 
 ```yaml
 description: Prompt matrix example
-target: default
+providers:
+  - default
 
 prompts:
   - id: support-chat
@@ -176,15 +198,19 @@ tests:
   - id: password-reset
     vars:
       question: How do I reset my password?
-    expected_output: Password reset guidance
+      expected_output: Password reset guidance
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - Gives correct password reset guidance
   - id: admin-access
     vars:
       audience: admins
       question: How do I revoke a user's access?
-    expected_output: Access revocation guidance
+      expected_output: Access revocation guidance
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - Gives safe access revocation guidance
 ```
 
@@ -204,7 +230,7 @@ then render those vars from the prompt template next to the input.
 **Shorthand forms:**
 - Prompt entries can be strings, message arrays, file references, or prompt objects.
 - Put chat/system/user messages in `prompts`, not in `tests[].input`.
-- `expected_output` (string/object) expands to `[{role: "assistant", content: ...}]`
+- `vars.expected_output` is a conventional reference-answer variable; explicit assertions decide how to grade it
 - Use these canonical field names on disk; keep the wire format `snake_case`
 
 **Message format:** `{role, content}` where role is `system`, `user`, `assistant`, or `tool`
@@ -356,12 +382,14 @@ tests:
         verification guidance was added.
 
         Decide what durable repo change should be made and explain why.
-    expected_output: |
-      The durable repo change is to update .agents/verification.md with the
-      reusable verification workflow lessons. AGENTS.md already routes this
-      class of work to .agents/verification.md, so no extra AGENTS.md edit is
-      needed unless that routing is missing.
+      expected_output: |
+        The durable repo change is to update .agents/verification.md with the
+        reusable verification workflow lessons. AGENTS.md already routes this
+        class of work to .agents/verification.md, so no extra AGENTS.md edit is
+        needed unless that routing is missing.
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - The answer recommends updating .agents/verification.md rather than leaving the learning only in PR comments or private evidence.
       - The answer uses the pinned ./agentv checkout to verify the AGENTS.md routing.
       - The answer preserves the historical commit SHA as context.
@@ -371,7 +399,7 @@ tests:
 
 When `assert` is defined, **only the declared graders run**. For
 semantic checks, add plain rubric strings. If you need a custom LLM prompt or
-grader target, declare `llm-rubric` explicitly:
+grader provider, declare `llm-rubric` explicitly:
 
 ```yaml
 prompts:
@@ -387,9 +415,9 @@ tests:
         value: "fix"
 ```
 
-`expected_output` is passive reference data. It is available to graders through
-`{{expected_output}}` and the script stdin payload, but it does not create an
-implicit LLM grading call by itself.
+`vars.expected_output` is passive reference data. It is available to graders
+through `{{ expected_output }}` and the script stdin payload, but it does not
+create an implicit LLM grading call by itself.
 
 **Common mistake:** putting rubric prose in `expected_output` instead of an
 assertion:
@@ -402,7 +430,7 @@ tests:
   - id: bad-example
     vars:
       prompt: "What is 2+2?"
-    expected_output: The assistant should explain why the answer is 4. # reference answer field, not a grader
+      expected_output: The assistant should explain why the answer is 4. # reference answer var, not a grader
 ```
 
 Write this as:
@@ -415,8 +443,10 @@ tests:
   - id: good-example
     vars:
       prompt: "What is 2+2?"
-    expected_output: "4"
+      expected_output: "4"
     assert:
+      - type: llm-rubric
+        value: "Matches the reference answer: {{ expected_output }}"
       - The answer is 4 and explains the arithmetic briefly
 ```
 
@@ -438,7 +468,7 @@ assert:
         weight: 5.0
 ```
 
-If a required grader scores below its threshold, the overall verdict is forced to `fail`.
+If a required grader scores below its threshold, the overall case status is forced to `fail`.
 
 ## Environment Setup/Teardown
 
@@ -521,11 +551,12 @@ Configure via the `assert` array. Multiple graders produce a weighted average sc
   type: script
   command: [uv, run, validate.py]
   cwd: ./scripts          # optional working directory
-  target: {}              # optional: enable LLM target proxy (max_calls: 50)
+  provider: {}            # optional: enable LLM provider proxy (max_calls: 50)
 ```
-Contract: stdin JSON -> stdout JSON `{score, assertions: [{text, passed, evidence?}], reasoning}`
+Contract: stdin JSON -> stdout JSON `{pass, score, reason, checks?: [{text, pass, score?, reason}]}`
 Raw stdin uses snake_case and includes: `input`, `expected_output`, `output` (final answer string), `messages`, `trace`, `trace_summary`, `token_usage`, `cost_usd`, `duration_ms`, `start_time`, `end_time`, `file_changes`, `workspace_path`, `config`
 SDK handlers receive the same payload in camelCase: `expectedOutput`, `traceSummary`, `tokenUsage`, `costUsd`, `durationMs`, `startTime`, `endTime`, `fileChanges`, `workspacePath`.
+`checks` is an SDK/script convenience shape; public `grading.json` artifacts normalize checks into recursive `component_results`.
 When an environment prepares a workspace directory, `workspace_path` is the absolute path to that directory (also available as `AGENTV_WORKSPACE_PATH` env var). Use this for functional grading (e.g., running `npm test` in the prepared workdir).
 For deterministic workspace checks that fit normal Vitest `expect(...)` tests, prefer a plain verifier file and the built-in adapter:
 ```yaml
@@ -541,7 +572,7 @@ See the Script Graders docs for the full stdin/stdout contract.
 - name: quality
   type: llm-rubric
   prompt: ./prompts/eval.md     # markdown template or command config
-  target: grader_gpt_5_mini     # optional: override the grader target for this grader
+  provider: grader_gpt_5_mini   # optional: override the grader provider for this grader
   model: gpt-5-chat            # optional model override
   config:                       # passed to prompt templates as context.config
     strictness: high
@@ -549,7 +580,7 @@ See the Script Graders docs for the full stdin/stdout contract.
 Variables: `{{criteria}}`, `{{input}}`, `{{expected_output}}`, `{{output}}`, `{{metadata}}`, `{{metadata_json}}`, `{{rubrics}}`, `{{rubrics_json}}`, `{{file_changes}}`, `{{tool_calls}}`
 - Markdown templates: use `{{variable}}` syntax
 - TypeScript templates: use `definePromptTemplate(fn)` from `@agentv/sdk`, receives context object with all variables + `config`
-- Use `target:` to run different `llm-rubric` graders against different named LLM targets in the same eval (useful for grader panels / ensembles)
+- Use `provider:` to run different `llm-rubric` graders against different named LLM providers in the same eval (useful for grader panels / ensembles)
 
 ### assert-set
 ```yaml
@@ -567,7 +598,7 @@ Variables: `{{criteria}}`, `{{input}}`, `{{expected_output}}`, `{{output}}`, `{{
       type: llm-rubric
       weight: 0.7
 ```
-Use `assert-set` for Promptfoo-aligned assertion grouping. Without `threshold`, the set passes only when every nonzero-weight child assertion passes. With `threshold`, the weighted aggregate score determines the set verdict. Parent `config` is inherited by children, and child `config` keys override parent keys. Do not use `type: composite`; AgentV rejects it.
+Use `assert-set` for Promptfoo-aligned assertion grouping. Without `threshold`, the set passes only when every nonzero-weight child assertion passes. With `threshold`, the weighted aggregate score determines the set pass/fail result. Parent `config` is inherited by children, and child `config` keys override parent keys. Do not use `type: composite`; AgentV rejects it.
 
 ### Skill And Trajectory Assertions
 ```yaml
@@ -722,24 +753,24 @@ agentv eval assert <grader-name> --agent-output "..." --agent-input "..."
 agentv import claude --session-id <uuid>
 
 # Re-run only execution errors from a previous run
-agentv eval <file.yaml> --retry-errors .agentv/results/default/<timestamp>/index.jsonl
+agentv eval <file.yaml> --retry-errors .agentv/results/<run_id>/.internal/index.jsonl
 
 # Validate eval file
 agentv validate <file.yaml>
 
 # Compare completed runs
 agentv results compare \
-  .agentv/results/default/<baseline-timestamp>/index.jsonl \
-  .agentv/results/default/<candidate-timestamp>/index.jsonl
+  .agentv/results/<baseline-run-id>/.internal/index.jsonl \
+  .agentv/results/<candidate-run-id>/.internal/index.jsonl
 agentv results combine \
-  .agentv/results/default/<baseline-timestamp> \
-  .agentv/results/default/<candidate-timestamp> \
-  .agentv/results/default/<third-target-timestamp> \
-  --output .agentv/results/default/combined
-agentv results compare .agentv/results/default/combined/index.jsonl
+  .agentv/results/<baseline-run-id> \
+  .agentv/results/<candidate-run-id> \
+  .agentv/results/<third-target-run-id> \
+  --output .agentv/results/combined
+agentv results compare .agentv/results/combined/.internal/index.jsonl
 agentv results compare \
-  .agentv/results/default/<baseline-timestamp>/index.jsonl \
-  .agentv/results/default/<candidate-timestamp>/index.jsonl \
+  .agentv/results/<baseline-run-id>/.internal/index.jsonl \
+  .agentv/results/<candidate-run-id>/.internal/index.jsonl \
   --json
 
 # Author assertions directly in the eval file
@@ -755,32 +786,36 @@ Use `@agentv/sdk` as the public lightweight SDK package for TypeScript/JavaScrip
 
 ### YAML-aligned eval authoring
 ```typescript
-import { defineEval, graders } from '@agentv/sdk';
+// evals/helper-suite.eval.ts
+import { graders, type EvalConfig } from '@agentv/sdk';
 
-export default defineEval({
+const config: EvalConfig = {
   name: 'helper-suite',
   target: 'default',
-  // The SDK helper lowers this to evaluate_options.repeat in generated YAML.
+  // TypeScript config loading lowers this to evaluate_options.repeat.
   repeat: {
     count: 3,
     strategy: 'pass_any',
     earlyExit: false,
   },
   threshold: 0.8,
+  prompts: ['{{ task }}'],
   tests: [
     {
       id: 'json-answer',
-      input: 'Return a JSON answer with a status field.',
+      vars: { task: 'Return a JSON answer with a status field.' },
       assert: [
         graders.json({ name: 'valid-json', required: true }),
         graders.regex(/"status"\s*:/, { name: 'status-key' }),
       ],
     },
   ],
-});
+};
+
+export default config;
 ```
 
-The `graders` catalog returns ordinary `assert` entries such as `type: is-json`, `type: regex`, `type: llm-rubric`, and `type: script`. `defineEval()` lowers camelCase TypeScript fields such as `expectedOutput`, `inputFiles`, and `maxSteps` to canonical snake_case YAML/runtime keys.
+The `graders` catalog returns ordinary `assert` entries such as `type: is-json`, `type: regex`, `type: llm-rubric`, and `type: script`. Explicit `*.eval.ts` and `*.eval.mts` files should default-export an `EvalConfig`; `defineEval(config)` is only an optional thin helper over that same shape. TypeScript config loading lowers camelCase fields such as `expectedOutput`, `inputFiles`, and `maxSteps` to canonical snake_case YAML/runtime keys.
 
 If adapting Braintrust `scores` or DeepEval metrics, write small AgentV helper factories that return `graders.*` configs:
 
@@ -790,7 +825,7 @@ import { graders } from '@agentv/sdk';
 export function ragFaithfulness() {
   return graders.llmRubric(undefined, {
     name: 'rag-faithfulness',
-    target: 'grader-target',
+    provider: 'grader-provider',
     prompt: 'Grade whether the answer is supported by the retrieved context.',
   });
 }
@@ -805,9 +840,11 @@ import { defineAssertion } from '@agentv/sdk';
 
 export default defineAssertion(({ output, trace }) => {
   const finalOutput = output ?? '';
+  const pass = finalOutput.length > 0 && (trace?.eventCount ?? 0) <= 10;
   return {
-    pass: finalOutput.length > 0 && (trace?.eventCount ?? 0) <= 10,
-    reasoning: 'Checks content exists and is efficient',
+    pass,
+    score: pass ? 1 : 0,
+    reason: 'Checks content exists and is efficient',
   };
 });
 ```
@@ -823,17 +860,21 @@ import { defineScriptGrader } from '@agentv/sdk';
 
 export default defineScriptGrader(({ output, trace }) => {
   const finalOutput = output ?? '';
+  const hasOutput = finalOutput.length > 0;
+  const efficient = (trace?.eventCount ?? 0) <= 5;
   return {
-    score: finalOutput.length > 0 && (trace?.eventCount ?? 0) <= 5 ? 1.0 : 0.5,
-    assert: [
-      { text: 'Output is not empty', passed: finalOutput.length > 0 },
-      { text: 'Efficient tool usage', passed: (trace?.eventCount ?? 0) <= 5 },
+    pass: hasOutput && efficient,
+    score: hasOutput && efficient ? 1.0 : 0.5,
+    reason: 'Checks content exists and tool usage is bounded',
+    checks: [
+      { text: 'Output is not empty', pass: hasOutput, reason: hasOutput ? 'Output text is present' : 'Output is empty' },
+      { text: 'Efficient tool usage', pass: efficient, reason: efficient ? 'Trace event count is within limit' : 'Trace event count is too high' },
     ],
   };
 });
 ```
 
-Use `defineScriptGrader()` when the custom component is a command-backed grader with explicit score control, custom assertion-result arrays, workspace commands, or LLM calls through a grader target. `defineScriptGrader()` scripts are referenced in YAML with `type: script` and `command: [bun, run, grader.ts]`. Plain Vitest workspace verifier files can use `command: [agentv, eval, graders/check.test.ts]`.
+Use `defineScriptGrader()` when the custom component is a command-backed grader with explicit score control, check arrays, workspace commands, or LLM calls through a grader provider. `defineScriptGrader()` scripts are referenced in YAML with `type: script` and `command: [bun, run, grader.ts]`. Plain Vitest workspace verifier files can use `command: [agentv, eval, graders/check.test.ts]`.
 
 ### Convention-Based Discovery
 

package/dist/skills/agentv-eval-writer/references/custom-evaluators.md

@@ -56,18 +56,20 @@ Promptfoo normally calls eval custom logic assertions and uses fixed assertion t
 
 ```typescript
 import {
-  createTargetClient,
+  createProviderClient,
   defineScriptGrader,
   defineEval,
+  type EvalConfig,
   definePromptTemplate,
   graders,
 } from '@agentv/sdk';
 ```
 
 - `defineScriptGrader(fn)` - Wraps evaluation function with stdin/stdout handling
-- `defineEval(definition)` - Defines a YAML-aligned `.eval.ts` suite
+- `EvalConfig` - Public TypeScript eval authoring type for default-exported `*.eval.ts` and `*.eval.mts` files
+- `defineEval(definition)` - Optional thin helper over the same `EvalConfig` shape
 - `graders` - Helper catalog that returns ordinary AgentV `assert` entries
-- `createTargetClient()` - Returns LLM proxy client (when `target: {}` configured)
+- `createProviderClient()` - Returns LLM proxy client (when `provider: {}` configured)
   - `.invoke({question, systemPrompt})` - Single LLM call
   - `.invokeBatch(requests)` - Batch LLM calls
 - `definePromptTemplate(fn)` - Wraps prompt generation function
@@ -81,29 +83,32 @@ For Python, the repo-local helper example in `examples/features/sdk-python/` kee
 Use helper factories for reusable Braintrust/DeepEval-inspired checks, but keep the result as AgentV `assert` entries:
 
 ```typescript
-import { defineEval, graders } from '@agentv/sdk';
+import { graders, type EvalConfig } from '@agentv/sdk';
 
 function ragFaithfulness() {
   return graders.llmRubric(undefined, {
     name: 'rag-faithfulness',
-    target: 'grader-target',
+    provider: 'grader-provider',
     prompt: 'Grade whether the answer is supported by the retrieved context.',
   });
 }
 
-export default defineEval({
+const config: EvalConfig = {
   name: 'rag-suite',
+  prompts: ['{{ task }}'],
   tests: [
     {
       id: 'grounded-answer',
-      input: 'Answer using the retrieved context.',
+      vars: { task: 'Answer using the retrieved context.' },
       assert: [
         graders.contains('source', { name: 'mentions-source' }),
         ragFaithfulness(),
       ],
     },
   ],
-});
+};
+
+export default config;
 ```
 
 The helper lowers to ordinary YAML:
@@ -115,7 +120,7 @@ assert:
     value: source
   - name: rag-faithfulness
     type: llm-rubric
-    target: grader-target
+    provider: grader-provider
     prompt: Grade whether the answer is supported by the retrieved context.
 ```
 
@@ -176,6 +181,6 @@ Derived from test fields (users never author these directly):
 | `input` | Full resolved input array (JSON) |
 | `expected_output` | Full resolved expected array (JSON) |
 | `output` | Final answer / scored result string |
-| `messages` | Transcript messages from target execution |
+| `messages` | Transcript messages from provider execution |
 
 Markdown templates use `{{variable}}` syntax. TypeScript templates receive context object.