@capagents/forgeqe 1.1.7 → 1.1.8

package/HOW_TO_USE.md

@@ -67,7 +67,7 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
 | Release | **Requirement coverage and release readiness**: Gives requirement coverage and a GO / GO WITH RISKS / NO-GO verdict for a sprint, area, epic, or stories, from test cases, results, bugs, and flaky tests; can publish it to the wiki. | `release_readiness` | [docs](#requirement-coverage-and-release-readiness) |
 | Operations | **Scheduled monitoring with alerts**: Runs checks on a schedule and alerts Teams or email when one keeps failing, and again when it recovers. | `monitor` | [docs](#9-monitoring) |
 | Operations | **Teams and email notifications**: Tells the team in Microsoft Teams channels or by email when a suite fails or passes, when a release readiness verdict is in, and when the domain evaluation falls below its gate, with the failing cases, their bugs, and links to the report and the CI build. Each channel picks its events; the agent also posts a summary to a channel when asked. | `notify` | [docs](#17-notifications) |
-| Release | **Reports you can share**: Writes up what ForgeQE's agent found, such as release status across projects, open defects, or pipeline health, as a Markdown file and an HTML page in the reports folder, with a summary in the chat and links to open them. | `write_report` | [docs](#reports-from-the-agent) |
+| Release | **Reports you can share**: Writes up what ForgeQE's agent found, such as open defects across projects, release status, or pipeline health, as a page with filters (project, state, priority, assignee, …), charts, and a sortable table, plus Markdown and CSV. Every table is built from evidence: the results of the tool calls the agent made, or a read-only query ForgeQE runs itself, listed on the page's Sources with each raw result kept beside the report. | `write_report` | [docs](#reports-from-the-agent) |
 | Operations | **CI pipelines**: Writes a GitHub Actions or Azure Pipelines workflow that runs ForgeQE, stores the secrets it needs, pushes a branch, and makes the pipeline green. | `push_branch`, `set_ci_secret` | [docs](#8-pipelines) |
 | Systems | **Work with Azure DevOps, Jira, GitHub, and Confluence**: Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, puts run evidence (screenshots, reports) on work items and issues as attachments or comments, and builds Azure DevOps dashboards and wiki pages. | `query_api`, `read_confluence`, `create_ado_dashboard`, `write_ado_wiki_page`, `attach_to_work_item` | [docs](README.md#azure-devops-jira-and-other-systems) |
 | Setup | **Setup, skills, and documentation**: Shows the model profiles and MCP servers, installs or repairs the browser engine, creates a forgeqe.yaml, manages skills, and answers questions about ForgeQE from its documentation and this list. | `forgeqe_setup`, `capabilities` | [docs](#2-scaffold-config) |
@@ -390,15 +390,25 @@ The text is the agent's own result. Failed runs show `✗ Task failed` with the
 
 ### Reports from the agent
 
-Ask for a report and ForgeQE's agent writes one you can share: "create a release report for all projects", "write up the open defects", "summarise this sprint's test results as a report". It gathers the facts with its tools (Azure DevOps, Jira, the runs it made), then saves the report with `write_report` as a Markdown file and an HTML page in the reports folder (`export.out_dir`, `.forgeqe/reports/` by default):
+Ask for a report and ForgeQE's agent writes one you can share: "generate a defect report for all open defects", "create a release report for all projects", "summarise this sprint's test results as a report". Every table in it is built from **evidence**, not typed by the model:
+
+- ForgeQE keeps the result of every tool call in the session (its own tools, and the Azure DevOps and Jira MCP servers). A dataset of the report names the calls it comes from, and ForgeQE takes the rows from their results.
+- Or the dataset is a read-only query that ForgeQE runs itself when it writes the report: a WIQL query reads up to 1000 work items with their project, type, state, priority, severity, assignee, area, iteration, dates, and a link. The agent uses this for "all" of something.
+- The agent writes only the summary paragraph.
+
+The report goes to the reports folder (`export.out_dir`, `.forgeqe/reports/` by default):
 
 ```
-.forgeqe/reports/release-status-all-projects-20261009-001512/
-  report.md     # to edit, paste into a wiki, or attach
-  report.html   # to open or share; light and dark, tables scroll on a phone
+.forgeqe/reports/open-defects-all-projects-20261013-101512/
+  report.html         # one file: summary, filters, charts, sortable table, Sources
+  report.md           # the same, to edit, paste into a wiki, or attach
+  open-defects.csv    # each dataset's rows
+  evidence/E1.json    # the raw result of each tool call the report used
 ```
 
-The chat gets a short summary and a card with both paths as links, and **Open the report** appears as a next step. The report holds only what the agent read in that request. To post it, ask for it ("post the report to the release channel") once `notify.channels` is set up; the reports folder is linked when it is served (`notify.report_url`).
+The page adds filters for the columns that are in the data (Project, State, Type, Priority, Severity, Assigned To, Area, Iteration, …, and other columns with a short list of values). The counts, the charts (click a bar to filter by it), and the table follow the filters, and **Export CSV** saves the rows you filtered. **Sources** lists each tool call, when it ran, and how many rows it gave, with a link to its raw result.
+
+The chat gets a short summary and a card with the paths as links, and **Open the report** appears as a next step. To post it, ask for it ("post the report to the release channel") once `notify.channels` is set up; the reports folder is linked when it is served (`notify.report_url`).
 
 ### The screen
 

package/docs/user-guide.html

@@ -1804,22 +1804,31 @@ forgeqe visual approve --page /home  # make this run's /home the baseline (asks
 </article>
 <article class="spec feat" id="cap-reports">
 <div class="hd"><span class="mono">Release</span><h3>Reports you can share</h3></div>
-<p class="ds">Writes up what ForgeQE's agent found, such as release status across projects, open defects, or pipeline health, as a Markdown file and an HTML page in the reports folder, with a summary in the chat and links to open them.</p>
+<p class="ds">Writes up what ForgeQE's agent found, such as open defects across projects, release status, or pipeline health, as a page with filters (project, state, priority, assignee, …), charts, and a sortable table, plus Markdown and CSV. Every table is built from evidence: the results of the tool calls the agent made, or a read-only query ForgeQE runs itself, listed on the page's Sources with each raw result kept beside the report.</p>
 <div class="facts">
 <div class="fact"><span class="mono">Ask it</span><ul class="asks"><li>Create a release report for all projects</li><li>Write up the open defects as a report I can share</li></ul></div>
 <div class="fact"><span class="mono">What it needs</span><ul class="needs"><li><span class="need req">Required</span><span>A live model profile in llms.json (an API key or Microsoft Entra ID sign-in)</span></li></ul></div>
-<div class="fact"><span class="mono">Limits</span><ul class="plainlist"><li>The report holds what the agent read with its tools in this request; it does not invent figures, and it does not publish the page anywhere unless the reports folder is served (notify.report_url).</li></ul></div>
+<div class="fact"><span class="mono">Limits</span><ul class="plainlist"><li>Rows come only from tool results in the session or a query ForgeQE runs; the agent writes only the summary, which is not checked figure by figure.</li><li>A WIQL query reads up to 1000 work items; the page says when a query matched more.</li><li>The page is not published anywhere unless the reports folder is served (notify.report_url).</li></ul></div>
 <div class="fact"><span class="mono">Tools</span><p class="tools"><code>write_report</code></p></div>
 </div>
 <div class="detail"><span class="mono">In detail</span>
 <section class="doc" id="doc-how_to_use-reports-from-the-agent">
 <h4>Reports from the agent</h4>
-<div class="prose"><p>Ask for a report and ForgeQE's agent writes one you can share: &quot;create a release report for all projects&quot;, &quot;write up the open defects&quot;, &quot;summarise this sprint's test results as a report&quot;. It gathers the facts with its tools (Azure DevOps, Jira, the runs it made), then saves the report with <code>write_report</code> as a Markdown file and an HTML page in the reports folder (<code>export.out_dir</code>, <code>.forgeqe/reports/</code> by default):</p>
-<pre><code>.forgeqe/reports/release-status-all-projects-20261009-001512/
-  report.md     # to edit, paste into a wiki, or attach
-  report.html   # to open or share; light and dark, tables scroll on a phone
-</code></pre>
-<p>The chat gets a short summary and a card with both paths as links, and <strong>Open the report</strong> appears as a next step. The report holds only what the agent read in that request. To post it, ask for it (&quot;post the report to the release channel&quot;) once <code>notify.channels</code> is set up; the reports folder is linked when it is served (<code>notify.report_url</code>).</p>
+<div class="prose"><p>Ask for a report and ForgeQE's agent writes one you can share: &quot;generate a defect report for all open defects&quot;, &quot;create a release report for all projects&quot;, &quot;summarise this sprint's test results as a report&quot;. Every table in it is built from <strong>evidence</strong>, not typed by the model:</p>
+<ul>
+<li>ForgeQE keeps the result of every tool call in the session (its own tools, and the Azure DevOps and Jira MCP servers). A dataset of the report names the calls it comes from, and ForgeQE takes the rows from their results.</li>
+<li>Or the dataset is a read-only query that ForgeQE runs itself when it writes the report: a WIQL query reads up to 1000 work items with their project, type, state, priority, severity, assignee, area, iteration, dates, and a link. The agent uses this for &quot;all&quot; of something.</li>
+<li>The agent writes only the summary paragraph.</li>
+</ul>
+<p>The report goes to the reports folder (<code>export.out_dir</code>, <code>.forgeqe/reports/</code> by default):</p>
+<pre><code>.forgeqe/reports/open-defects-all-projects-20261013-101512/
+  report.html         # one file: summary, filters, charts, sortable table, Sources
+  report.md           # the same, to edit, paste into a wiki, or attach
+  open-defects.csv    # each dataset's rows
+  evidence/E1.json    # the raw result of each tool call the report used
+</code></pre>
+<p>The page adds filters for the columns that are in the data (Project, State, Type, Priority, Severity, Assigned To, Area, Iteration, …, and other columns with a short list of values). The counts, the charts (click a bar to filter by it), and the table follow the filters, and <strong>Export CSV</strong> saves the rows you filtered. <strong>Sources</strong> lists each tool call, when it ran, and how many rows it gave, with a link to its raw result.</p>
+<p>The chat gets a short summary and a card with the paths as links, and <strong>Open the report</strong> appears as a next step. To post it, ask for it (&quot;post the report to the release channel&quot;) once <code>notify.channels</code> is set up; the reports folder is linked when it is served (<code>notify.report_url</code>).</p>
 </div>
 </section>
 </div>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@capagents/forgeqe",
-  "version": "1.1.7",
+  "version": "1.1.8",
   "description": "ForgeQE: a browser agent CLI that runs goals and test cases in local Chrome or Chromium, or BrowserStack and TestMu AI/LambdaTest when configured, and turns passing runs into tests in your repo",
   "license": "MIT",
   "type": "module",

package/prompts/brain.md

@@ -63,7 +63,7 @@ A message can also be a task specification with the parts Role, Objective, Conte
 - logins: manages saved logins. It lists them. It lets the user sign in by hand in a browser (sign_in). It removes a session.
 - page_memory: gives what ForgeQE knows about the pages, locators, and recorded flows of a site. Clear it only when the user asks.
 - notify: posts a message to the Teams or email channels of the team. Use it only when the user asks. Suites, release verdicts, and failed evaluations go to their channels automatically from runs outside the shell.
-- write_report: saves a report that the user asks for as a Markdown file and an HTML page in the reports folder.
+- write_report: saves a report that the user asks for as a page (filters, charts, the sources of its data) and a Markdown file. ForgeQE builds its tables from evidence: the results of tool calls in this session, or a read-only query that it runs itself. You write only the summary.
 - capabilities: tells what ForgeQE can do, and whether each capability works in this setup. It gives what to set up, example requests, limits, and the documentation.
 - forgeqe_setup: gives model profiles, the MCP servers that ForgeQE sets up, and the documentation of ForgeQE (docs). It installs or repairs the browser engine. It creates a forgeqe.yaml (init).
 - push_branch: the only way to push. It refuses the default branch, and it never forces.
@@ -335,7 +335,9 @@ The shell shows replies as markdown in a narrow chat pane. Write them so that th
 - You MUST use bold only for the one word or number that is important in a line, such as a status or a count.
 - You SHOULD use a table only for two or three short columns.
 - You MUST keep your own comments short. You MUST give in full the test cases, tests, and reviews that the user asked for.
-- When the user asks for a report, a write-up, or a summary to share, you MUST collect the facts with your tools and save the report with write_report.
+- When the user asks for a report, a write-up, or a summary to share, you MUST save the report with write_report, with its data as datasets. You MUST NOT type rows, counts, or tables into the report yourself.
+- For "all" of something (all open defects, every project), you SHOULD give the dataset a query (a WIQL query or a REST path), so that ForgeQE reads every item itself. Otherwise, name the tool calls that you made in this session (from: the tool name or the evidence id).
+- The summary MUST hold only facts that are in the datasets. After write_report, give the counts that it returned.
 - You MUST then reply with three to five lines of summary. The shell shows the paths of the report as links, so you MUST NOT repeat them or paste the full report.
 - You MUST NOT end a reply with "If you want, I can…" options. Put the follow-ups that the user will probably want in next, 3 or fewer, each with a short label and the request as the user would type it.
 - You MUST end each turn with the structured answer: reply, outcome (done, failed, or needs_input), and next.

package/src/brain/index.ts

@@ -159,6 +159,10 @@ export async function openBrain(opts: BrainOptions): Promise<Brain> {
         [MCP_NAME]: tools.entry,
       },
       hideTool: (tool) => tool.startsWith(`${MCP_NAME}_`),
+      // ForgeQE's own tools record their results themselves; this adds the other MCP servers' (Azure DevOps, Jira, …).
+      onToolResult: (r) => {
+        if (r.ok && !r.tool.startsWith(`${MCP_NAME}_`) && r.tool.includes("_")) tools.server.evidence.record(r.tool, r.input, r.output)
+      },
       title: "ForgeQE",
     })
   } catch (err) {

package/src/capabilities/registry.ts

@@ -317,11 +317,15 @@ export const CAPABILITIES: Capability[] = [
     area: "Release",
     title: "Reports you can share",
     summary:
-      "Writes up what ForgeQE's agent found, such as release status across projects, open defects, or pipeline health, as a Markdown file and an HTML page in the reports folder, with a summary in the chat and links to open them.",
+      "Writes up what ForgeQE's agent found, such as open defects across projects, release status, or pipeline health, as a page with filters (project, state, priority, assignee, …), charts, and a sortable table, plus Markdown and CSV. Every table is built from evidence: the results of the tool calls the agent made, or a read-only query ForgeQE runs itself, listed on the page's Sources with each raw result kept beside the report.",
     tools: ["write_report"],
     examples: ["Create a release report for all projects", "Write up the open defects as a report I can share"],
     requires: [{ kind: "live_model" }],
-    limits: ["The report holds what the agent read with its tools in this request; it does not invent figures, and it does not publish the page anywhere unless the reports folder is served (notify.report_url)."],
+    limits: [
+      "Rows come only from tool results in the session or a query ForgeQE runs; the agent writes only the summary, which is not checked figure by figure.",
+      "A WIQL query reads up to 1000 work items; the page says when a query matched more.",
+      "The page is not published anywhere unless the reports folder is served (notify.report_url).",
+    ],
     docs: [{ doc: "HOW_TO_USE.md", heading: "Reports from the agent" }],
   },
   {

package/src/cases/api.ts

@@ -16,37 +16,88 @@ export type ApiQuery = {
 }
 
 const WIQL_TOP = 50
+/** What a report may read in one WIQL search. */
+export const REPORT_WIQL_TOP = 1000
 const WIQL_FIELDS = ["System.Id", "System.WorkItemType", "System.Title", "System.State", "System.AssignedTo", "System.ChangedDate", "System.IterationPath"]
+/** Read too when the organization has them; a process without one of them falls back to WIQL_FIELDS. */
+const WIQL_MORE = ["System.TeamProject", "Microsoft.VSTS.Common.Priority", "Microsoft.VSTS.Common.Severity", "System.AreaPath", "System.CreatedDate", "System.Tags"]
+const BATCH = 200
+
+export type WorkItemRow = {
+  id: number
+  project?: string
+  type?: unknown
+  title?: unknown
+  state?: unknown
+  priority?: unknown
+  severity?: unknown
+  assignedTo?: string
+  area?: unknown
+  iteration?: unknown
+  tags?: unknown
+  created?: unknown
+  changed?: unknown
+  url: string
+}
 
 /**
  * A WIQL search: Azure DevOps answers it with ids only (a POST, but WIQL has no statement that changes anything), so
- * the first WIQL_TOP are read back with their main fields in one batch read.
+ * the first `top` are read back with their main fields in batch reads.
  */
-async function adoWiql(cfg: SourcesConfig, wiql: string, project?: string): Promise<unknown> {
+export async function adoWorkItems(cfg: SourcesConfig, wiql: string, project?: string, top = WIQL_TOP): Promise<{ count: number; total: number; items: WorkItemRow[] }> {
   const { ado, auth } = adoSettings(cfg)
   const org = ado.orgUrl.replace(/\/+$/, "")
   const scope = project || ado.project
   const base = scope ? `${org}/${encodeURIComponent(scope)}` : org
   const { data } = await http<{ workItems?: { id: number }[]; workItemRelations?: { source?: { id: number } | null; target?: { id: number } }[] }>(
-    `${base}/_apis/wit/wiql?api-version=7.1&$top=${WIQL_TOP}`,
+    `${base}/_apis/wit/wiql?api-version=7.1&$top=${top}`,
     { auth, body: { query: wiql } },
   )
-  const ids = [...new Set([...(data.workItems || []).map((w) => w.id), ...(data.workItemRelations || []).flatMap((r) => [r.source?.id, r.target?.id])])].filter((id): id is number => typeof id === "number").slice(0, WIQL_TOP)
-  if (!ids.length) return { count: 0, items: [] }
-  const batch = await http<{ value: { id: number; fields: Record<string, unknown> }[] }>(`${org}/_apis/wit/workitemsbatch?api-version=7.1`, { auth, body: { ids, fields: WIQL_FIELDS } })
+  const all = [...new Set([...(data.workItems || []).map((w) => w.id), ...(data.workItemRelations || []).flatMap((r) => [r.source?.id, r.target?.id])])].filter((id): id is number => typeof id === "number")
+  const ids = all.slice(0, top)
+  const read = async (fields: string[]) => {
+    const out: { id: number; fields: Record<string, unknown> }[] = []
+    for (let i = 0; i < ids.length; i += BATCH) {
+      const batch = await http<{ value: { id: number; fields: Record<string, unknown> }[] }>(`${org}/_apis/wit/workitemsbatch?api-version=7.1`, { auth, body: { ids: ids.slice(i, i + BATCH), fields } })
+      out.push(...batch.data.value)
+    }
+    return out
+  }
+  const values = ids.length ? await read([...WIQL_FIELDS, ...WIQL_MORE]).catch(() => read(WIQL_FIELDS)) : []
   const name = (v: unknown) => (v && typeof v === "object" ? String((v as { displayName?: string }).displayName || "") : String(v || ""))
   return {
     count: ids.length,
-    items: batch.data.value.map((w) => ({
-      id: w.id,
-      type: w.fields["System.WorkItemType"],
-      title: w.fields["System.Title"],
-      state: w.fields["System.State"],
-      assignedTo: name(w.fields["System.AssignedTo"]) || undefined,
-      changed: w.fields["System.ChangedDate"],
-      iteration: w.fields["System.IterationPath"],
-    })),
-    note: `Read each one in full with path <project>/_apis/wit/workitems/<id>?$expand=all.${ids.length === WIQL_TOP ? ` Only the first ${WIQL_TOP} are listed; narrow the query for the rest.` : ""}`,
+    total: all.length,
+    items: values.map((w) => {
+      const f = w.fields
+      const itemProject = String(f["System.TeamProject"] || scope || "")
+      return {
+        id: w.id,
+        ...(f["System.TeamProject"] ? { project: String(f["System.TeamProject"]) } : {}),
+        type: f["System.WorkItemType"],
+        title: f["System.Title"],
+        state: f["System.State"],
+        ...(f["Microsoft.VSTS.Common.Priority"] !== undefined ? { priority: f["Microsoft.VSTS.Common.Priority"] } : {}),
+        ...(f["Microsoft.VSTS.Common.Severity"] !== undefined ? { severity: f["Microsoft.VSTS.Common.Severity"] } : {}),
+        assignedTo: name(f["System.AssignedTo"]) || undefined,
+        ...(f["System.AreaPath"] ? { area: f["System.AreaPath"] } : {}),
+        iteration: f["System.IterationPath"],
+        ...(f["System.Tags"] ? { tags: f["System.Tags"] } : {}),
+        ...(f["System.CreatedDate"] ? { created: f["System.CreatedDate"] } : {}),
+        changed: f["System.ChangedDate"],
+        url: `${org}/${itemProject ? `${encodeURIComponent(itemProject)}/` : ""}_workitems/edit/${w.id}`,
+      }
+    }),
+  }
+}
+
+async function adoWiql(cfg: SourcesConfig, wiql: string, project?: string): Promise<unknown> {
+  const { count, items } = await adoWorkItems(cfg, wiql, project, WIQL_TOP)
+  if (!count) return { count: 0, items: [] }
+  return {
+    count,
+    items,
+    note: `Read each one in full with path <project>/_apis/wit/workitems/<id>?$expand=all.${count === WIQL_TOP ? ` Only the first ${WIQL_TOP} are listed; narrow the query for the rest, or pass the query to write_report, which reads up to ${REPORT_WIQL_TOP}.` : ""}`,
   }
 }
 
@@ -85,31 +136,33 @@ function clipJson(data: unknown): unknown {
  * Read-only requests with the credentials from forgeqe.yaml and the environment: GET and WIQL searches for Azure
  * DevOps, GET for Jira (Xray Server/Data Center is under Jira's /rest/raven), GraphQL queries for Xray Cloud.
  */
-export async function queryApi(cfg: SourcesConfig, q: ApiQuery): Promise<unknown> {
+export async function queryApi(cfg: SourcesConfig, q: ApiQuery, opts: { full?: boolean } = {}): Promise<unknown> {
+  // A report reads the whole answer; the agent gets it clipped, so it fits its context.
+  const clip = opts.full ? (d: unknown) => d : clipJson
   switch (q.system) {
     case "azure_devops": {
-      if (q.wiql?.trim()) return clipJson(await adoWiql(cfg, q.wiql, q.project))
+      if (q.wiql?.trim()) return opts.full ? adoWorkItems(cfg, q.wiql, q.project, REPORT_WIQL_TOP) : clipJson(await adoWiql(cfg, q.wiql, q.project))
       if (!q.path) throw new Error("path or wiql is required for azure_devops")
       const { ado, auth } = adoSettings(cfg)
       let url = resolve(q.path, adoBases(ado.orgUrl), "Azure DevOps")
       if (!/[?&]api-version=/.test(url)) url += `${url.includes("?") ? "&" : "?"}api-version=7.1`
-      return clipJson((await http(url, { auth })).data)
+      return clip((await http(url, { auth })).data)
     }
     case "jira": {
       if (!q.path) throw new Error("path is required for jira")
       const j = jiraApi(cfg)
-      return clipJson((await http(resolve(q.path, [j.url], "Jira"), { auth: j.auth })).data)
+      return clip((await http(resolve(q.path, [j.url], "Jira"), { auth: j.auth })).data)
     }
     case "confluence": {
       if (!q.path) throw new Error("path is required for confluence")
       const c = confluenceApi(cfg)
-      return clipJson((await http(resolve(q.path, [c.url], "Confluence"), { auth: c.auth })).data)
+      return clip((await http(resolve(q.path, [c.url], "Confluence"), { auth: c.auth })).data)
     }
     case "xray": {
       if (cfg.jira && !cfg.jira.cloud) throw new Error("Xray Server/Data Center is part of Jira: use system jira with a /rest/raven/1.0/... path.")
       if (!q.query) throw new Error("query (GraphQL) is required for xray")
       if (isMutation(q.query)) throw new Error("query_api only reads; GraphQL mutations are not allowed.")
-      return clipJson(await xrayGraphql(cfg, await xrayCloudToken(cfg), q.query, q.variables || {}))
+      return clip(await xrayGraphql(cfg, await xrayCloudToken(cfg), q.query, q.variables || {}))
     }
     default:
       throw new Error(`Unknown system "${String(q.system)}". Use azure_devops, jira, xray, or confluence.`)

package/src/code/runtime.ts

@@ -539,6 +539,8 @@ export async function openSession<R = AgentReport>(opts: {
   mcp?: Record<string, unknown>
   /** Tool calls not to show as ⚙ lines (the caller shows them itself). */
   hideTool?: (tool: string) => boolean
+  /** Every finished tool call with what it returned, shown or not: the evidence reports are built from. */
+  onToolResult?: (r: { tool: string; input: Record<string, unknown>; output: string; ok: boolean }) => void
   title?: string
   protect?: Protect
   /** Stop the turn when one shell command has failed this many times since the last resetStall. */
@@ -672,6 +674,7 @@ export async function openSession<R = AgentReport>(opts: {
           emit({ type: "code", kind: "start", tool: part.tool, id: part.id, text: toolLine(opts.repo, part.tool, part.state.input || {}, part.state.title) })
         }
         if (part?.type === "tool" && (part.state?.status === "completed" || part.state?.status === "error")) {
+          opts.onToolResult?.({ tool: part.tool, input: part.state.input || {}, output: String(part.state.output ?? ""), ok: part.state.status === "completed" })
           if (part.tool === "StructuredOutput" && part.state.status === "completed" && !turn.structured) {
             turn.structured = part.state.input || null
           }

package/src/mcp/reporttools.ts

@@ -1,11 +1,29 @@
 import { mkdirSync, writeFileSync } from "node:fs"
 import path from "node:path"
 
+import { queryApi, type ApiQuery } from "../cases/api.ts"
+import { reportPage } from "../report/html.ts"
+import {
+  chooseColumns,
+  chooseFilters,
+  dedupe,
+  describeCall,
+  EvidenceLog,
+  evidenceList,
+  pickEvidence,
+  rowsOf,
+  type Cell,
+  type Column,
+  type EvidenceEntry,
+  type Row,
+} from "../report/evidence.ts"
 import type { McpTool } from "./tools.ts"
 
 /**
- * Reports the agent writes from what it found (release status, open defects, pipeline health): a Markdown file to
- * edit or paste, and the same report as a page to open or share. The chat gets the summary and the links.
+ * Reports the agent writes from what it found (open defects, release status, pipeline health). Every table in one is
+ * built from evidence: the output of tool calls in this session, or a read-only query ForgeQE runs itself. The model
+ * writes only the summary. The report is a page (report-ui, with filters, charts, and the sources it came from) and
+ * the same report as Markdown; the raw results are kept beside them.
  */
 
 /** "Release status: all projects" → "release-status-all-projects". */
@@ -21,7 +39,7 @@ export function reportSlug(title: string): string {
 
 const escapeHtml = (s: string) => s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;").replace(/"/g, "&quot;")
 
-/** The report as a page: the Markdown rendered, with raw HTML in it shown as text so a report can never run script. */
+/** A Markdown document as a plain page (forgeqe bi's page without Evidence): the Markdown rendered, with raw HTML in it shown as text so a report can never run script. */
 export function reportHtml(title: string, markdown: string, written: Date): string {
   const safe = markdown.replace(/<(?=[A-Za-z/!?])/g, "&lt;")
   const body = Bun.markdown.html(safe)
@@ -66,36 +84,209 @@ ${body}
 `
 }
 
+/** The summary as HTML, with raw HTML in it shown as text so a report can never run script. */
+export function summaryHtml(markdown: string): string {
+  return Bun.markdown.html(markdown.replace(/<(?=[A-Za-z/!?])/g, "&lt;"))
+}
+
+const MAX_ROWS = 5000
+const MD_ROWS = 200
+
+export type DatasetSpec = { name?: string; from?: string; match?: string; query?: ApiQuery; columns?: string[]; filters?: string[] }
+
+export type Dataset = { name: string; columns: Column[]; filters: string[]; rows: Row[]; sources: string[]; total: number; note?: string }
+export type Source = { id: string; tool: string; call: string; at: string; rows: number | null; file: string }
+
+/** The rows of one dataset, and the evidence they came from. */
+export async function buildDataset(spec: DatasetSpec, log: EvidenceLog, readApi: (q: ApiQuery) => Promise<unknown>): Promise<{ dataset: Dataset; used: EvidenceEntry[] }> {
+  const name = String(spec.name || "Data").trim()
+  let used: EvidenceEntry[]
+  if (spec.query) {
+    const q = spec.query
+    const out = await readApi(q)
+    const entry = log.record("query_api", { ...q }, typeof out === "string" ? out : JSON.stringify(out))
+    if (!entry) throw new Error(`Dataset "${name}": the query returned nothing.`)
+    used = [entry]
+  } else if (spec.from) {
+    used = pickEvidence(log.all(), String(spec.from), spec.match ? String(spec.match) : undefined)
+    if (!used.length) throw new Error(`Dataset "${name}": no tool result matches "${spec.from}"${spec.match ? ` with "${spec.match}"` : ""}. Results in this session:\n${evidenceList(log.all())}`)
+  } else {
+    throw new Error(`Dataset "${name}" needs "from" (a tool result of this session, e.g. E3 or the tool's name) or "query" (a read-only query ForgeQE runs).`)
+  }
+  const parts = used.map((e) => ({ e, rows: rowsOf(e.output) }))
+  const noData = parts.filter((p) => p.rows === null)
+  if (noData.length === parts.length) {
+    throw new Error(`Dataset "${name}": ${noData.map((p) => p.e.id).join(", ")} returned no data (no JSON records). Results in this session:\n${evidenceList(log.all())}`)
+  }
+  const all = dedupe(parts.flatMap((p) => p.rows || []))
+  const rows = all.slice(0, MAX_ROWS)
+  const columns = rows.length ? chooseColumns(rows, spec.columns) : []
+  const totals = parts.map((p) => {
+    const v = (() => {
+      try {
+        return JSON.parse(p.e.output) as { total?: unknown; count?: unknown }
+      } catch {
+        return null
+      }
+    })()
+    return typeof v?.total === "number" ? v.total : 0
+  })
+  const total = Math.max(all.length, ...totals)
+  const notes = [
+    all.length > MAX_ROWS ? `The page holds the first ${MAX_ROWS} of ${all.length} rows.` : "",
+    total > all.length ? `The query matched ${total}; the first ${all.length} were read.` : "",
+  ].filter(Boolean)
+  return {
+    dataset: { name, columns, filters: chooseFilters(rows, columns, spec.filters), rows, sources: used.map((e) => e.id), total, ...(notes.length ? { note: notes.join(" ") } : {}) },
+    used,
+  }
+}
+
+const mdCell = (v: Cell | undefined) => (v === null || v === undefined ? "" : String(v).replace(/\|/g, "\\|").replace(/\s+/g, " "))
+const csvCell = (v: Cell | undefined) => {
+  const s = v === null || v === undefined ? "" : String(v)
+  return /[",\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s
+}
+
+/** Counts of each value of a column, most first. */
+export function countBy(rows: Row[], key: string): [string, number][] {
+  const m = new Map<string, number>()
+  for (const r of rows) {
+    const v = r[key] === null || r[key] === undefined || r[key] === "" ? "(none)" : String(r[key])
+    m.set(v, (m.get(v) || 0) + 1)
+  }
+  return [...m].sort((a, b) => b[1] - a[1])
+}
+
+/** The report as Markdown: the summary, each dataset's counts and table, and the sources. */
+export function reportMarkdown(title: string, summary: string, datasets: Dataset[], sources: Source[], written: Date): string {
+  const out = [`# ${title}`, "", `_Written by ForgeQE on ${written.toISOString().slice(0, 16).replace("T", " ")} UTC, from ${sources.length === 1 ? "1 source" : `${sources.length} sources`}._`, "", summary, ""]
+  for (const d of datasets) {
+    out.push(`## ${d.name}`, "", `**${d.rows.length}** ${d.rows.length === 1 ? "row" : "rows"}${d.note ? ` (${d.note})` : ""} · from ${d.sources.join(", ")}`, "")
+    for (const f of d.filters.slice(0, 3)) out.push(`- By ${f}: ${countBy(d.rows, f).map(([v, n]) => `${v} ${n}`).join(", ")}`)
+    if (d.filters.length) out.push("")
+    if (d.rows.length) {
+      const cols = d.columns.filter((c) => c.key !== "Link")
+      const idCol = cols.find((c) => c.key === "ID" || c.key === "Key")
+      out.push(`| ${cols.map((c) => c.label).join(" | ")} |`, `| ${cols.map(() => "---").join(" | ")} |`)
+      for (const r of d.rows.slice(0, MD_ROWS)) {
+        out.push(`| ${cols.map((c) => (c === idCol && r.Link ? `[${mdCell(r[c.key])}](${r.Link})` : mdCell(r[c.key]))).join(" | ")} |`)
+      }
+      if (d.rows.length > MD_ROWS) out.push("", `The first ${MD_ROWS} rows; the page and ${csvName(d.name)} have all ${d.rows.length}.`)
+      out.push("")
+    }
+  }
+  out.push("## Sources", "", ...sources.map((s) => `- ${s.id} · ${s.call} · ${s.at.slice(0, 16).replace("T", " ")} UTC · ${s.rows === null ? "no data" : `${s.rows} rows`} · [raw](${s.file})`), "")
+  return out.join("\n")
+}
+
+const csvName = (name: string) => `${reportSlug(name)}.csv`
+
 export const REPORT_TOOLS: McpTool[] = [
   {
     name: "write_report",
     description:
-      'Save a report the user asks for ("create a release report", "write up the open defects", "give me a report I can share") as a Markdown file and an HTML page in the reports folder. Gather the facts with your other tools first, then pass the whole report as Markdown: a one-paragraph summary first, then ## sections, tables for per-item status (one row per project, case, or build), and links to builds, work items, and pull requests. Do not invent data you did not read. The user is shown the report paths as links, so reply with three to five lines of summary only: do not repeat the paths or paste the report.',
+      'Save a report the user asks for ("create a defect report for all open defects", "release status for all projects", "a report I can share") as a page with filters, charts, and its sources, and as Markdown. A report is built from evidence: each dataset is the output of tool calls you made in this session (from: the tool\'s name, e.g. wit_get_work_items_batch_by_ids or query_api, or an evidence id like E3; match narrows to calls whose input mentions it), or a read-only query ForgeQE runs itself (query: same fields as query_api; a WIQL query reads up to 1000 work items with project, state, priority, severity, assignee, area, and links, so prefer it for "all" of something). ForgeQE takes the rows from those results, not from you; columns and filters are optional (default: the usual ones in the data, such as Project, State, Priority, Assigned To). Write only the summary yourself: one short paragraph of what the data shows, with no figures the datasets do not hold. The user is shown the report paths as links, so reply with three to five lines of summary only.',
     inputSchema: {
       type: "object",
       properties: {
-        title: { type: "string", description: "The report's title, e.g. Release status: all projects, 9 October 2026" },
-        markdown: { type: "string", description: "The report body in Markdown, without the title (it is added as the heading)" },
+        title: { type: "string", description: "The report's title, e.g. Open defects: all projects, 13 October 2026" },
+        summary: { type: "string", description: "One short Markdown paragraph (or a few bullets) on what the data shows; the tables, counts, and charts are made from the datasets" },
+        datasets: {
+          type: "array",
+          minItems: 1,
+          items: {
+            type: "object",
+            properties: {
+              name: { type: "string", description: "e.g. Open defects" },
+              from: { type: "string", description: "Evidence ids (E3, or E3,E4) or the name of a tool called in this session; every successful call of it is used" },
+              match: { type: "string", description: "With a tool name: only its calls whose input contains this text" },
+              query: {
+                type: "object",
+                description: "A read-only query ForgeQE runs for this dataset (instead of from)",
+                properties: {
+                  system: { type: "string", enum: ["azure_devops", "jira", "xray", "confluence"] },
+                  wiql: { type: "string", description: "azure_devops: e.g. SELECT [System.Id] FROM WorkItems WHERE [System.WorkItemType] = 'Bug' AND [System.State] NOT IN ('Closed','Done','Removed'). Across projects, leave out @project" },
+                  project: { type: "string" },
+                  path: { type: "string", description: "A REST path, as for query_api (e.g. Jira /rest/api/3/search/jql?jql=...&fields=summary,status,priority,assignee,project)" },
+                  query: { type: "string", description: "xray: a GraphQL query" },
+                  variables: { type: "object" },
+                },
+                required: ["system"],
+              },
+              columns: { type: "array", items: { type: "string" }, description: "Columns to show, in order (default: the usual ones in the data)" },
+              filters: { type: "array", items: { type: "string" }, description: "Columns to filter by (default: Project, State, Priority, Assigned To, … when present)" },
+            },
+            required: ["name"],
+          },
+        },
       },
-      required: ["title", "markdown"],
+      required: ["title", "datasets"],
     },
     async run(args, ctx) {
       const title = String(args.title || "Report").trim()
-      const markdown = String(args.markdown || "").trim()
-      if (!markdown) throw new Error("The report is empty: pass its body as markdown.")
+      const summary = String(args.summary ?? args.markdown ?? "").trim()
+      const specs = (Array.isArray(args.datasets) ? args.datasets : []) as DatasetSpec[]
+      const log = ctx.evidence || new EvidenceLog()
+      if (!specs.length) {
+        throw new Error(`A report is built from evidence: pass datasets, each from a tool result of this session or a read-only query. Results in this session:\n${evidenceList(log.all())}`)
+      }
+      const readApi = (q: ApiQuery) => queryApi(ctx.config.sources, q, { full: true })
+      const built: Dataset[] = []
+      const used = new Map<string, EvidenceEntry>()
+      for (const spec of specs) {
+        const { dataset, used: entries } = await buildDataset(spec, log, readApi)
+        built.push(dataset)
+        for (const e of entries) used.set(e.id, e)
+      }
+
       const written = new Date()
       const stamp = written.toISOString().replace(/[-:]/g, "").replace("T", "-").slice(0, 15)
       // Beside the run reports, so the shell links it and outDir publishing (notify.report_url) serves it too.
       const dir = path.resolve(ctx.config.export.outDir, `${reportSlug(title)}-${stamp}`)
-      mkdirSync(dir, { recursive: true })
+      mkdirSync(path.join(dir, "evidence"), { recursive: true })
+      const sources: Source[] = [...used.values()].map((e) => {
+        const file = `evidence/${e.id}.json`
+        writeFileSync(path.join(dir, file), JSON.stringify({ id: e.id, tool: e.tool, input: e.input, at: e.at, output: e.output }, null, 2))
+        const rows = rowsOf(e.output)
+        return { id: e.id, tool: e.tool, call: describeCall(e), at: e.at, rows: rows ? rows.length : null, file }
+      })
+      for (const d of built) {
+        const cols = d.columns
+        writeFileSync(path.join(dir, csvName(d.name)), [cols.map((c) => csvCell(c.label)).join(","), ...d.rows.map((r) => cols.map((c) => csvCell(r[c.key])).join(","))].join("\n") + "\n")
+      }
       const md = path.join(dir, "report.md")
       const html = path.join(dir, "report.html")
-      writeFileSync(md, `# ${title}\n\n${markdown}\n`)
-      writeFileSync(html, reportHtml(title, markdown, written))
+      writeFileSync(md, reportMarkdown(title, summary, built, sources, written))
+      writeFileSync(
+        html,
+        reportPage(title, {
+          kind: "data",
+          title,
+          id: path.basename(dir),
+          startedAt: written.toISOString(),
+          ms: 0,
+          stopped: false,
+          generatedAt: written.toISOString(),
+          cases: [],
+          posted: [],
+          history: [],
+          report: { summary, summaryHtml: summary ? summaryHtml(summary) : "", datasets: built.map((d) => ({ ...d, csv: csvName(d.name) })), sources },
+        }),
+      )
       const show = (p: string) => path.relative(process.cwd(), p) || p
+      const rows = built.reduce((n, d) => n + d.rows.length, 0)
       ctx.emit?.({ type: "report", title, html, markdown: md })
-      ctx.emit?.({ type: "card", card: { status: "ok", title: `Report: ${title}`, body: [`Page      ${show(html)}`, `Markdown  ${show(md)}`] } })
-      return { html: show(html), markdown: show(md) }
+      ctx.emit?.({
+        type: "card",
+        card: { status: "ok", title: `Report: ${title}`, body: [`Page      ${show(html)}`, `Markdown  ${show(md)}`, `Evidence  ${rows} rows from ${sources.map((s) => s.id).join(", ")}`] },
+      })
+      return {
+        html: show(html),
+        markdown: show(md),
+        datasets: built.map((d) => ({ name: d.name, rows: d.rows.length, total: d.total, columns: d.columns.map((c) => c.key), filters: d.filters, counts: Object.fromEntries(d.filters.map((f) => [f, Object.fromEntries(countBy(d.rows, f).slice(0, 8))])), ...(d.note ? { note: d.note } : {}) })),
+        sources: sources.map((s) => `${s.id} ${s.call}`),
+      }
     },
   },
 ]