@capagents/forgeqe 1.1.0 → 1.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/HOW_TO_USE.md CHANGED
@@ -467,23 +467,24 @@ The chat gets a short summary and a card with both paths as links, and **Open th
467
467
 
468
468
  **`/` and `@` in the prompt.** Typing `/` lists the commands that match; typing `@` suggests references: recent runs (*last run*, then each by its folder), files in the repository by name or path (`@Login` finds `pages/Login.ts`), and, after `@` and a number such as `@1761`, the Azure DevOps work item with its title (read-only). Tab completes, ↑/↓ choose, Esc closes the list.
469
469
 
470
- **Refine.** Ctrl-R (or *Refine my request* in the palette) turns what you typed into a task specification for ForgeQE's agent, in ASD-STE100 Simplified Technical English, before it is sent:
470
+ **Refine.** Ctrl-R (or *Refine my request* in the palette) turns what you typed into a task specification for ForgeQE's agent, laid out as a system prompt and written in ASD-STE100 Simplified Technical English, before it is sent. It has these parts, in this order: **Role** (who the agent acts as), **Objective** (one instruction), **Context** (the facts, with names), **Steps** (only when they add something), **Rules** (your limits and the setup's), **Output format** (what comes back and how it is laid out), and **Summary** (what you get, in one sentence, read last):
471
471
 
472
472
  ```
473
- Goal: Check the Azure DevOps QI-Payments pipelines and make a shareable report for the manager.
473
+ Role: A release analyst who reads Azure DevOps deployments.
474
+
475
+ Objective: Get what is deployed now in each environment of each Azure DevOps project.
474
476
 
475
477
  Context:
476
- - Project: Azure DevOps project QI-Payments.
477
- - Pipeline: build definition 32, QI-Payments-CD.
478
- - Latest checked CD build: build 1021, failed, finished 2026-09-21.
478
+ - System: Azure DevOps organization devcgazuredevops.
479
+ - Projects: Autothon, CQube.AI, BlackBox_AI, QI-Payments, QI-MobileBanking, Atlas.Ai, QI-Claims.
479
480
 
480
- Steps:
481
- 1. Review the recent runs of QI-Payments-CI and QI-Payments-CD.
482
- 2. Make a shareable report with a short manager summary.
481
+ Output format: One section for each project, with a line for each environment: the deployed build, its date, and its status.
483
482
 
484
- Result: A shareable report file with a short summary for the manager.
483
+ Summary: Show the deployed build in each environment, and why each deployment failed.
485
484
  ```
486
485
 
486
+ Refine is one model session of its own, separate from the conversation: a few model steps (usually two) and read-only tool calls, typically 15K–25K tokens and 20–30 seconds. The panel title shows what it cost and how much of the request keeps the ASD-STE100 rules (*✦ Refined request · STE 100% · 20.8K tokens · 21s*), and its tokens count in the session total in the footer.
487
+
487
488
  A read-only agent writes it. It starts knowing every capability and whether it works in this setup, and the domain; it checks names and ids in the systems that are set up (work items, test plans, pipelines, sprints, recent runs). The panel shows the request with what it **checked** ✓, what it **assumed** ~, and what is **missing** ? (for example "Which story? Give the work item id"); a capability that is not set up shows as missing too. Enter sends it, `e` puts it in the prompt to edit, Esc keeps yours. Refine never runs anything, writes files, or changes a work item; it never invents an id, URL, project, or file.
488
489
 
489
490
  **Next steps.** When a request finishes, up to three suggestions appear above the prompt: from what it produced (*Open the report*, *Write a test for this run*, *Why did it fail?*, *Rerun the cases that failed*) and from the agent's answer, which suggests the follow-ups you are likely to want (*Write a release report*, *Show the failed CD logs*). Click one or press Alt-1 to Alt-3; suggestions that start work are put in the prompt first, to adjust and send.
@@ -622,20 +622,20 @@ Add a GitHub Actions workflow that runs tests/cases.yaml on every PR, and make i
622
622
  <p><strong>Setup checklist.</strong> Until everything is set up, the start screen's <strong>Setup</strong> line shows how many of the five things most features rest on are set up, and names the ones that are not: <strong>Model</strong> (a live model profile), <strong>Coding agent</strong>, <strong>Work items</strong> (Azure DevOps, Jira, TestRail, Zephyr, qTest, or Xray), <strong>Domain</strong>, and <strong>Notifications</strong>. <strong>▸ how to set up</strong> puts the steps for each missing one in the chat (for <strong>Work items</strong>, type <code>/auth</code>). The checks are the same ones <code>/features</code> uses, so the two always agree.</p>
623
623
  <p><strong>Command palette.</strong> Ctrl-P opens a palette of every action, command, profile, and example request; type a few letters to find one (&quot;rr&quot; finds <em>Refine my request</em>, &quot;auth&quot; finds the sign-in commands), ↑/↓ to choose, Enter to run it. Example requests are put in the prompt for you to edit, never sent straight away.</p>
624
624
  <p><strong><code>/</code> and <code>@</code> in the prompt.</strong> Typing <code>/</code> lists the commands that match; typing <code>@</code> suggests references: recent runs (<em>last run</em>, then each by its folder), files in the repository by name or path (<code>@Login</code> finds <code>pages/Login.ts</code>), and, after <code>@</code> and a number such as <code>@1761</code>, the Azure DevOps work item with its title (read-only). Tab completes, ↑/↓ choose, Esc closes the list.</p>
625
- <p><strong>Refine.</strong> Ctrl-R (or <em>Refine my request</em> in the palette) turns what you typed into a task specification for ForgeQE's agent, in ASD-STE100 Simplified Technical English, before it is sent:</p>
626
- <pre><code>Goal: Check the Azure DevOps QI-Payments pipelines and make a shareable report for the manager.
625
+ <p><strong>Refine.</strong> Ctrl-R (or <em>Refine my request</em> in the palette) turns what you typed into a task specification for ForgeQE's agent, laid out as a system prompt and written in ASD-STE100 Simplified Technical English, before it is sent. It has these parts, in this order: <strong>Role</strong> (who the agent acts as), <strong>Objective</strong> (one instruction), <strong>Context</strong> (the facts, with names), <strong>Steps</strong> (only when they add something), <strong>Rules</strong> (your limits and the setup's), <strong>Output format</strong> (what comes back and how it is laid out), and <strong>Summary</strong> (what you get, in one sentence, read last):</p>
626
+ <pre><code>Role: A release analyst who reads Azure DevOps deployments.
627
+
628
+ Objective: Get what is deployed now in each environment of each Azure DevOps project.
627
629
 
628
630
  Context:
629
- - Project: Azure DevOps project QI-Payments.
630
- - Pipeline: build definition 32, QI-Payments-CD.
631
- - Latest checked CD build: build 1021, failed, finished 2026-09-21.
631
+ - System: Azure DevOps organization devcgazuredevops.
632
+ - Projects: Autothon, CQube.AI, BlackBox_AI, QI-Payments, QI-MobileBanking, Atlas.Ai, QI-Claims.
632
633
 
633
- Steps:
634
- 1. Review the recent runs of QI-Payments-CI and QI-Payments-CD.
635
- 2. Make a shareable report with a short manager summary.
634
+ Output format: One section for each project, with a line for each environment: the deployed build, its date, and its status.
636
635
 
637
- Result: A shareable report file with a short summary for the manager.
636
+ Summary: Show the deployed build in each environment, and why each deployment failed.
638
637
  </code></pre>
638
+ <p>Refine is one model session of its own, separate from the conversation: a few model steps (usually two) and read-only tool calls, typically 15K–25K tokens and 20–30 seconds. The panel title shows what it cost and how much of the request keeps the ASD-STE100 rules (<em>✦ Refined request · STE 100% · 20.8K tokens · 21s</em>), and its tokens count in the session total in the footer.</p>
639
639
  <p>A read-only agent writes it. It starts knowing every capability and whether it works in this setup, and the domain; it checks names and ids in the systems that are set up (work items, test plans, pipelines, sprints, recent runs). The panel shows the request with what it <strong>checked</strong> ✓, what it <strong>assumed</strong> ~, and what is <strong>missing</strong> ? (for example &quot;Which story? Give the work item id&quot;); a capability that is not set up shows as missing too. Enter sends it, <code>e</code> puts it in the prompt to edit, Esc keeps yours. Refine never runs anything, writes files, or changes a work item; it never invents an id, URL, project, or file.</p>
640
640
  <p><strong>Next steps.</strong> When a request finishes, up to three suggestions appear above the prompt: from what it produced (<em>Open the report</em>, <em>Write a test for this run</em>, <em>Why did it fail?</em>, <em>Rerun the cases that failed</em>) and from the agent's answer, which suggests the follow-ups you are likely to want (<em>Write a release report</em>, <em>Show the failed CD logs</em>). Click one or press Alt-1 to Alt-3; suggestions that start work are put in the prompt first, to adjust and send.</p>
641
641
  <div class="tbl"><table>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@capagents/forgeqe",
3
- "version": "1.1.0",
3
+ "version": "1.1.1",
4
4
  "description": "ForgeQE: a browser agent CLI that runs goals and test cases in Chrome and turns passing runs into tests in your repo",
5
5
  "license": "MIT",
6
6
  "type": "module",
package/prompts/brain.md CHANGED
@@ -12,7 +12,7 @@ Find what the user wants from their message. Then do it with your tools. ForgeQE
12
12
  - A request to write or repair tests, or a CI pipeline task.
13
13
  - A question.
14
14
 
15
- A message can also be a task specification with the parts Goal, Context, Steps, Rules, and Result. Refine in the shell writes this type of message. Do the goal. Use the context as facts. Do the steps in sequence. Obey the rules. Give the result that it asks for.
15
+ A message can also be a task specification with the parts Role, Objective, Context, Steps, Rules, Output format, and Summary. Refine in the shell writes this type of message. Take the role. Do the objective. Use the context as facts. Do the steps in sequence. Obey the rules. Give the answer in the output format.
16
16
 
17
17
  ForgeQE tools (the client adds the forgeqe_ prefix):
18
18
  - browser_run: does one goal in a real browser, and verifies it. Use it for all work that must occur on a website. Set purpose to test for a flow of the application under test. Set purpose to task for all other work, such as finding, linking, or checking work items, or reading data from a tool. A task never becomes an automated test.
@@ -80,6 +80,13 @@ Azure DevOps, Jira, Xray, and other systems:
80
80
  - Do not call the APIs of these systems from the shell, with curl, Python, PowerShell, or a script. The shell does not have the credentials, and ForgeQE blocks these commands. Use query_api.
81
81
  - Azure DevOps classic releases are not in the MCP tools. Read them with query_api and the release host, for example https://vsrm.dev.azure.com/<organization>/<project>/_apis/release/releases?$top=10 or _apis/release/deployments.
82
82
  - When the user asks about releases in Azure DevOps, read both: the classic releases, and the runs of the deployment pipelines (the pipelines MCP tools). Say which of them each project uses.
83
+ - For release details, find what is deployed now, not only the last run. Do these steps for each project:
84
+ 1. Read the environments with query_api: <project>/_apis/distributedtask/environments.
85
+ 2. Read the deployments of each environment: <project>/_apis/distributedtask/environments/<id>/environmentdeploymentrecords?top=5&api-version=7.1-preview.1.
86
+ 3. Read the stages of each deployment run: <project>/_apis/build/builds/<id>/timeline. Give each stage with its result.
87
+ 4. When a stage failed, give the cause from the timeline: the failed task, check, or approval, and its message.
88
+ 5. Compare the newest CI build with the build that the environments have. Say when a newer build is not deployed.
89
+ - Keep these cases apart: a project without a deployment pipeline, a pipeline without runs, and a run that failed.
83
90
  - Create or change items only when the user asks. Examples are to file a bug, add a comment, move an issue, or edit a work item. Say what you changed, with its id or key.
84
91
  - The only exception is to save test cases that you wrote from a story (see below).
85
92
  - Azure DevOps dashboards: use create_ado_dashboard, not the browser.
@@ -326,6 +333,7 @@ Rules:
326
333
  - Use bold only for the one word or number that is important in a line, such as a status or a count. Do not make full sentences bold.
327
334
  - Use a table only for two or three short columns.
328
335
  - Keep replies short and specific. Give in full the test cases, tests, and reviews that the user asked for. Keep only your own comments short.
336
+ - A status alone is not an answer. When a result is "failed", empty, or "none found", find the cause with your tools before you reply. Give the cause and the next action that fixes it.
329
337
  - When the user asks for a report, a write-up, or a summary to share, collect the facts with your tools. Examples are "create a release report" and "write up the open defects".
330
338
  - Then save the report with write_report. Reply with three to five lines of summary. The shell shows the paths of the report as links, so do not repeat them.
331
339
  - Do not only paste a long report into the reply.
package/prompts/refine.md CHANGED
@@ -5,37 +5,41 @@ used_by: "the shell's Refine (ctrl+r)"
5
5
  ---
6
6
  You refine a request that the user typed for ForgeQE, a test automation assistant. ForgeQE then sends the refined request to its agent. You do not do the request. You do not change data.
7
7
 
8
- Write the refined request as a short task specification for the agent. Write it in ASD-STE100 Simplified Technical English:
8
+ Write the refined request in ASD-STE100 Simplified Technical English:
9
9
  - Use short sentences: 20 words or fewer in an instruction, 25 words or fewer in a description.
10
10
  - Give one instruction in each sentence. Start an instruction with a verb.
11
11
  - Use the active voice.
12
12
  - Use simple words with one meaning. For example, use "make sure", not "ensure", and "use", not "utilize".
13
13
  - Use the same name for the same thing each time.
14
14
 
15
- Fill in these parts:
16
- - goal: one instruction that says what the agent must do. Start with a verb. Name the exact thing to get or do, not a general word. Example: "Run the test cases of Azure DevOps test plan 412, suite 77." Example: "Get the latest classic release and the latest deployment pipeline run of each project."
15
+ Write the request as a system prompt for the agent. Fill in these parts, in this sequence:
16
+ - role: the role that the agent takes for this task, as one short description. Examples: "A release analyst who reads Azure DevOps deployments." and "A test engineer who runs manual test cases in the browser."
17
+ - objective: one instruction that says what the agent must do. Start with a verb. Name the exact thing to get or do, not a general word. Example: "Run the test cases of Azure DevOps test plan 412, suite 77." Example: "Get what is deployed now in each environment of each project, with the stages and the cause of each failed deployment."
17
18
  - context: the facts that the agent needs. Give the system, the project, the work items, the files, and the runs with their exact names. Give one fact for each line.
18
19
  - Give names, not internal ids such as GUIDs. Give an id only when it is the name that people use, for example a work item number, a test plan or suite number, or a build number.
19
20
  - Put a list of items of the same type on one line, for example "Projects: Autothon, CQube.AI, QI-Payments".
20
21
  - Give only the facts that this task needs. Do not give the working directory or the settings of forgeqe.yaml. The agent already has them.
21
- - steps: the actions, in sequence, when the request has more than one action. Give one action for each step. Each step must add information that the goal does not give. Leave this empty for a request with one action.
22
+ - steps: the actions, in sequence, when the request has more than one action. Give one action for each step. Each step must add information that the objective does not give, for example which data to read or how to compare it. When the steps only say the objective again, leave steps empty.
22
23
  - rules: the limits that the user gave or that the setup sets. Examples are "Do not send results back to Azure DevOps", "Use 2 browsers at the same time", and "Show the browser". Give only rules that come from the user or from "This session".
23
- - result: what the agent must give back. Examples are a report saved as a file with a short summary, a test file and its run, or a list of test cases with links.
24
+ - output: the output format. Say what the agent gives back and how it is laid out. Examples: "One section for each project, with a line for each environment: the deployed build, its date, and its status." and "A report file, with a summary of three lines in the chat."
25
+ - summary: what the user gets, in one sentence of 15 words or fewer. The agent reads it last. Do not copy the objective. Example: "Show the deployed build in each environment, and why each deployment failed."
24
26
 
25
27
  Rules for your work:
26
28
  - Keep the intent and the scope of the user exactly. Do not add work that the user did not ask for. Do not remove work that the user asked for.
29
+ - Keep each word that changes the result, such as "current", "all", "details", or "failed". For example, "current release" means what is deployed now, not the last run.
27
30
  - Use the names that ForgeQE uses, for example "Azure DevOps project QI-Payments", "test plan 412, suite 77", "story 1761", or "SHOP-12".
28
31
  - Use "This session" and "What ForgeQE can do here" to know the capabilities and the setup.
29
32
  - Describe each action in plain words, for example "run the test cases" or "write a Playwright test". Do not name the tools of ForgeQE or the capability ids.
30
33
  - Change references such as "the last run", "that story", or "the payments plan" into the item that they point to. Do this only when exactly one item fits.
31
34
  - Check facts with your read-only tools. For example, check that a work item, a test plan, a suite, or a run exists, what its name is, and how many cases it has.
32
35
  - Put each fact of context in grounded, with the same words, and say if you checked it. Do not add facts to grounded that are not in context.
33
- - A request can mean more than one thing. Then select the meaning that the capabilities of ForgeQE support best, and write the goal for it. Say the meaning that you selected in assumed.
34
- - Put a question in missing only when a wrong selection gives a different result. In the question, give the possible meanings, for example "Classic releases, deployment pipeline runs, or release readiness (GO / NO-GO)?".
36
+ - A request can mean more than one thing. Then select the meaning that the capabilities of ForgeQE support best, and write the objective for it. Say the meaning that you selected in assumed.
37
+ - Do not ask about a meaning that you selected in assumed. The user sees assumed and can change it.
38
+ - Put a question in missing only when you cannot select a meaning and a wrong selection gives a different result. In the question, give the possible meanings, for example "Classic releases, deployment pipeline runs, or release readiness (GO / NO-GO)?".
35
39
  - Do not invent an id, a URL, a project, or a file.
36
40
  - If you cannot find or check an item, put a short question for the user in missing.
37
41
  - If a capability that the request needs is not set up, put this in missing. Say what the user must set up.
38
42
  - Keep the settings that the user gave, for example the number of browsers, a visible browser, or whether results go back. Do not add other settings.
39
43
  - Write in the language of the user.
40
44
 
41
- Finish with the structured result: goal, context, steps, rules, result, grounded, assumed, and missing.
45
+ Finish with the structured result: role, objective, context, steps, rules, output, summary, grounded, assumed, and missing.
package/src/mcp/server.ts CHANGED
@@ -54,8 +54,18 @@ export function createMcpServer(opts: McpServerOptions) {
54
54
  base.emit?.(e)
55
55
  }
56
56
  : undefined
57
- const out = await tool.run(params.arguments || {}, { ...base, emit, signal: controller.signal })
57
+ const said = (ok: boolean) => {
58
+ if (tool.line && emit) emit({ type: "call", text: tool.line(params.arguments || {}), ok })
59
+ }
60
+ let out: unknown
61
+ try {
62
+ out = await tool.run(params.arguments || {}, { ...base, emit, signal: controller.signal })
63
+ } catch (err) {
64
+ if (!controller.signal.aborted) said(false)
65
+ throw err
66
+ }
58
67
  if (controller.signal.aborted) return finish({ content: [{ type: "text", text: "Stopped by the user." }], isError: true })
68
+ said(true)
59
69
  return finish({ content: [{ type: "text", text: typeof out === "string" ? out : JSON.stringify(out, null, 2) }] })
60
70
  } catch (err) {
61
71
  return finish({ content: [{ type: "text", text: controller.signal.aborted ? "Stopped by the user." : (err as Error).message || String(err) }], isError: true })
package/src/mcp/tools.ts CHANGED
@@ -98,6 +98,25 @@ export type ToolEvent =
98
98
  | { type: "card"; card: Card }
99
99
  /** write_report saved a report: its page and its Markdown. */
100
100
  | { type: "report"; title: string; html: string; markdown: string }
101
+ /** A finished call of a tool that has no card of its own, as one line: "Azure DevOps API · QI-Claims/_apis/build/builds (top 5)". */
102
+ | { type: "call"; text: string; ok: boolean }
103
+
104
+ const API_LABEL: Record<string, string> = { azure_devops: "Azure DevOps", jira: "Jira", xray: "Xray", confluence: "Confluence" }
105
+
106
+ /**
107
+ * A query_api call as a line: "Azure DevOps API · QI-Claims/_apis/release/releases (top 1, queryOrder descending)". The
108
+ * host and organization are left out, as are api-version and the start of a WIQL query.
109
+ */
110
+ export function queryLine(args: Record<string, any>): string {
111
+ const name = `${API_LABEL[String(args.system)] || String(args.system || "API")} API`
112
+ if (args.wiql) return `${name} · WIQL (${clip(String(args.wiql).replace(/\s+/g, " ").replace(/^SELECT .*? FROM WorkItems WHERE /i, ""), 90)})`
113
+ if (args.query) return `${name} · GraphQL (${clip(String(args.query).replace(/\s+/g, " "), 90)})`
114
+ const raw = String(args.path || "")
115
+ const url = /^https?:\/\//.test(raw) ? raw.replace(/^https?:\/\/[^/]+\/(?:[^/]+\/(?=[^/]+\/_apis\/))?/, "") : raw.replace(/^\//, "")
116
+ const [where, search = ""] = url.split("?")
117
+ const params = [...new URLSearchParams(search)].filter(([k]) => k !== "api-version").map(([k, v]) => `${k.replace(/^\$/, "")} ${v}`)
118
+ return `${name} · ${decodeURIComponent(where || "")}${params.length ? ` (${clip(params.join(", "), 90)})` : ""}`
119
+ }
101
120
 
102
121
  export type ToolContext = {
103
122
  config: AppConfig
@@ -135,6 +154,8 @@ export type McpTool = {
135
154
  description: string
136
155
  inputSchema: Record<string, unknown>
137
156
  run(args: Record<string, any>, ctx: ToolContext): Promise<unknown>
157
+ /** The call as one line in the conversation, for a tool that shows nothing else: the ForgeQE tools are hidden there. */
158
+ line?(args: Record<string, any>): string
138
159
  }
139
160
 
140
161
  const clip = (s: string | undefined, n: number) => {
@@ -563,6 +584,7 @@ export const TOOLS: McpTool[] = [
563
584
  limit: { type: "integer", minimum: 1, maximum: 25, description: "Most pages to return (default 10)" },
564
585
  },
565
586
  },
587
+ line: (args) => `Confluence · ${args.cql ? `search (${clip(String(args.cql), 90)})` : `pages (${clip((Array.isArray(args.pages) ? args.pages : []).map(String).join(", "), 90)})`}`,
566
588
  async run(args, ctx) {
567
589
  const pages = Array.isArray(args.pages) ? args.pages.map(String) : undefined
568
590
  return readConfluence(ctx.config.sources, { pages, cql: args.cql ? String(args.cql) : undefined, limit: Number(args.limit) || undefined })
@@ -592,6 +614,7 @@ export const TOOLS: McpTool[] = [
592
614
  },
593
615
  required: ["system"],
594
616
  },
617
+ line: queryLine,
595
618
  async run(args, ctx) {
596
619
  return queryApi(ctx.config.sources, { system: args.system, path: args.path, query: args.query, variables: args.variables, wiql: args.wiql, project: args.project })
597
620
  },
package/src/refine.ts CHANGED
@@ -10,6 +10,7 @@ import { CAPABILITIES } from "./capabilities/registry.ts"
10
10
  import { allStatuses } from "./capabilities/status.ts"
11
11
  import { domainNotes } from "./domain.ts"
12
12
  import type { CodeEvent } from "./types.ts"
13
+ import { steScore } from "./ste.ts"
13
14
 
14
15
  /**
15
16
  * The shell's Refine (ctrl+r): what the user typed becomes a precise request for ForgeQE's agent, checked against what
@@ -20,13 +21,18 @@ import type { CodeEvent } from "./types.ts"
20
21
  /** The only tools the refiner may call: each reads, none changes anything or starts a run. */
21
22
  export const REFINE_TOOLS = ["capabilities", "query_api", "find_test_cases", "list_sprints", "run_details"]
22
23
 
23
- /** The refined request as a task specification: what ForgeQE's agent is asked to do, in the parts its own prompt uses. */
24
- export type RefinedSpec = { goal: string; context: string[]; steps: string[]; rules: string[]; result: string }
24
+ /**
25
+ * The refined request as a task specification, laid out as a system prompt: the role the agent takes, the objective,
26
+ * the context, the steps, the rules, the output format, and the objective again in one sentence at the end.
27
+ */
28
+ export type RefinedSpec = { role: string; objective: string; context: string[]; steps: string[]; rules: string[]; output: string; summary: string }
25
29
 
26
30
  export type Refined = {
27
- /** The request that is sent: the spec written out (Goal, Context, Steps, Rules, Result). */
31
+ /** The request that is sent: the spec written out (Role, Objective, Context, Steps, Rules, Output format, Summary). */
28
32
  refined: string
29
33
  spec?: RefinedSpec
34
+ /** The share of the request's sentences that keep the ASD-STE100 rules, 0 to 1. */
35
+ ste?: number
30
36
  grounded: { fact: string; checked: boolean }[]
31
37
  assumed: string[]
32
38
  missing: string[]
@@ -37,11 +43,13 @@ const lines = { type: "array", items: { type: "string" } } as const
37
43
  export const REFINE_SCHEMA = {
38
44
  type: "object",
39
45
  properties: {
40
- goal: { type: "string", description: "One instruction that starts with a verb: what the agent must do" },
46
+ role: { type: "string", description: "The role the agent takes for this task, as one description, e.g. A release analyst who reads Azure DevOps deployments" },
47
+ objective: { type: "string", description: "One instruction that starts with a verb: what the agent must do" },
41
48
  context: { ...lines, description: "Facts the agent needs, one per line: system, project, work items, files, runs, with exact names and ids" },
42
49
  steps: { ...lines, description: "The actions in sequence, one per step; empty for a request with one action" },
43
50
  rules: { ...lines, description: "Limits the user gave or the setup sets, one per line" },
44
- result: { type: "string", description: "What the agent must give back" },
51
+ output: { type: "string", description: "The output format: what the agent gives back and how it is laid out (sections, a table, a file)" },
52
+ summary: { type: "string", description: "The objective again, in one short sentence, as the last line" },
45
53
  grounded: {
46
54
  type: "array",
47
55
  items: { type: "object", properties: { fact: { type: "string" }, checked: { type: "boolean" } }, required: ["fact", "checked"] },
@@ -49,7 +57,7 @@ export const REFINE_SCHEMA = {
49
57
  assumed: { type: "array", items: { type: "string" } },
50
58
  missing: { type: "array", items: { type: "string" } },
51
59
  },
52
- required: ["goal", "context", "steps", "rules", "result", "grounded", "assumed", "missing"],
60
+ required: ["role", "objective", "context", "steps", "rules", "output", "summary", "grounded", "assumed", "missing"],
53
61
  } as const
54
62
 
55
63
  const REFINE_AGENT = {
@@ -64,11 +72,14 @@ const REFINE_AGENT = {
64
72
 
65
73
  /** The spec as the request ForgeQE's agent gets: labelled parts, empty ones left out. */
66
74
  export function composeRefined(spec: RefinedSpec): string {
67
- const parts = [`Goal: ${spec.goal}`]
75
+ const parts: string[] = []
76
+ if (spec.role) parts.push(`Role: ${spec.role}`)
77
+ parts.push(`Objective: ${spec.objective}`)
68
78
  if (spec.context.length) parts.push(["Context:", ...spec.context.map((c) => `- ${c}`)].join("\n"))
69
79
  if (spec.steps.length) parts.push(["Steps:", ...spec.steps.map((st, i) => `${i + 1}. ${st}`)].join("\n"))
70
80
  if (spec.rules.length) parts.push(["Rules:", ...spec.rules.map((r) => `- ${r}`)].join("\n"))
71
- if (spec.result) parts.push(`Result: ${spec.result}`)
81
+ if (spec.output) parts.push(`Output format: ${spec.output}`)
82
+ if (spec.summary) parts.push(`Summary: ${spec.summary}`)
72
83
  return parts.join("\n\n")
73
84
  }
74
85
 
@@ -97,13 +108,18 @@ export function refineRequest(text: string, recent: { user: string; reply: strin
97
108
  }
98
109
 
99
110
  /** Keeps only what the schema allows, so the shell never shows a malformed answer. */
100
- export function cleanRefined(r: (Partial<Refined> & Partial<RefinedSpec>) | null | undefined, original: string): Refined {
111
+ export function cleanRefined(r: (Partial<Refined> & Partial<RefinedSpec> & { goal?: string; result?: string }) | null | undefined, original: string): Refined {
101
112
  const strings = (v: unknown) => (Array.isArray(v) ? v.map(String).map((s) => s.trim()).filter(Boolean) : [])
102
- const goal = String(r?.goal || "").trim()
103
- const spec: RefinedSpec | undefined = goal ? { goal, context: strings(r?.context), steps: strings(r?.steps), rules: strings(r?.rules), result: String(r?.result || "").trim() } : undefined
113
+ const text = (v: unknown) => String(v || "").trim()
114
+ // goal and result: the names an older refine prompt (a project's own copy) gives the objective and the output.
115
+ const objective = text(r?.objective) || text(r?.goal)
116
+ const spec: RefinedSpec | undefined = objective
117
+ ? { role: text(r?.role), objective, context: strings(r?.context), steps: strings(r?.steps), rules: strings(r?.rules), output: text(r?.output) || text(r?.result), summary: text(r?.summary) }
118
+ : undefined
119
+ const refined = spec ? composeRefined(spec) : text(r?.refined) || original
104
120
  return {
105
- refined: spec ? composeRefined(spec) : String(r?.refined || "").trim() || original,
106
- ...(spec ? { spec } : {}),
121
+ refined,
122
+ ...(spec ? { spec, ste: steScore(refined).score } : {}),
107
123
  grounded: Array.isArray(r?.grounded) ? r!.grounded.filter((g) => g && String(g.fact || "").trim()).map((g) => ({ fact: String(g.fact).trim(), checked: g.checked === true })) : [],
108
124
  assumed: strings(r?.assumed),
109
125
  missing: strings(r?.missing),
@@ -150,7 +166,7 @@ export async function refine(opts: {
150
166
  })
151
167
  try {
152
168
  let out = await session.prompt(refineRequest(opts.text, opts.recent))
153
- if (!out.report && !out.error && !opts.signal?.aborted) out = await session.prompt("Give the structured result: goal, context, steps, rules, result, grounded, assumed, and missing.")
169
+ if (!out.report && !out.error && !opts.signal?.aborted) out = await session.prompt("Give the structured result: role, objective, context, steps, rules, output, summary, grounded, assumed, and missing.")
154
170
  if (!out.report) throw new Error(out.error ? `Refine failed: ${out.error}` : "Refine gave no result.")
155
171
  return cleanRefined(out.report, opts.text)
156
172
  } finally {
package/src/tui/App.tsx CHANGED
@@ -519,6 +519,7 @@ export function App({ options }: { options: TuiOptions }) {
519
519
  }
520
520
 
521
521
  /** Refines what is in the prompt; the user then decides what is sent. */
522
+ const refineCount = useRef(0)
522
523
  const startRefine = () => {
523
524
  const text = promptTextRef.current.trim()
524
525
  const cfg = configRef.current
@@ -534,10 +535,19 @@ export function App({ options }: { options: TuiOptions }) {
534
535
  const controller = new AbortController()
535
536
  refineAbort.current = controller
536
537
  setRefine({ phase: "working", original: text })
537
- void (options.refine || refine)({ config: cfg, cwd: sessionSetup(cfg, options.repo).cwd, text, recent: convoRef.current, signal: controller.signal })
538
+ // Refine is a model session of its own: its tokens count in the session total, under a key per refine.
539
+ const key = `refine-${++refineCount.current}`
540
+ const started = Date.now()
541
+ let used = 0
542
+ const onEvent = (e: CodeEvent) => {
543
+ if (e.kind !== "usage") return
544
+ used = e.usage.input + e.usage.output
545
+ useTokens(key, used)
546
+ }
547
+ void (options.refine || refine)({ config: cfg, cwd: sessionSetup(cfg, options.repo).cwd, text, recent: convoRef.current, signal: controller.signal, onEvent })
538
548
  .then((result) => {
539
549
  if (controller.signal.aborted || refiningRef.current?.phase !== "working") return
540
- setRefine({ phase: "done", original: text, result, choice: initialChoice(result.missing) })
550
+ setRefine({ phase: "done", original: text, result, choice: initialChoice(result.missing), ...(used ? { tokens: used } : {}), ms: Date.now() - started })
541
551
  })
542
552
  .catch((err) => {
543
553
  if (controller.signal.aborted) return
@@ -914,6 +924,10 @@ export function App({ options }: { options: TuiOptions }) {
914
924
  case "info":
915
925
  push(e.ok === undefined ? "code" : e.ok ? "ok" : "err", e.text)
916
926
  return
927
+ case "call":
928
+ pushTool(e.text, e.ok)
929
+ if (!aborted()) setActivity(`Agent: ${clip(e.text, 60)}`)
930
+ return
917
931
  case "agent": {
918
932
  const ev = e.event
919
933
  inspect(ev)
@@ -1,11 +1,12 @@
1
1
  import type { Refined } from "../refine.ts"
2
2
  import { Spinner } from "./components.tsx"
3
- import { theme } from "./theme.ts"
3
+ import { formatDuration, formatTokens, theme } from "./theme.ts"
4
4
 
5
5
  /** Refine's panel, in the prompt's place: working, the result with what was checked, or why it could not refine. */
6
6
  export type RefineState =
7
7
  | { phase: "working"; original: string }
8
- | { phase: "done"; original: string; result: Refined; choice: number }
8
+ /** `tokens` and `ms`: what Refine's own model session cost, shown in the title. */
9
+ | { phase: "done"; original: string; result: Refined; choice: number; tokens?: number; ms?: number }
9
10
  | { phase: "error"; original: string; message: string }
10
11
 
11
12
  export const REFINE_CHOICES = [
@@ -28,7 +29,10 @@ export function initialChoice(missing: string[]): number {
28
29
 
29
30
  export function RefinePanel(props: { state: RefineState; width: number; onChoose: (id: RefineChoice) => void; onHover: (i: number) => void }) {
30
31
  const { state, width } = props
31
- const title = state.phase === "working" ? " ✦ Refining " : state.phase === "error" ? " ✦ Could not refine " : " ✦ Refined request "
32
+ const cost = state.phase === "done" && state.tokens ? ` · ${formatTokens(state.tokens)} tokens${state.ms ? ` · ${formatDuration(state.ms)}` : ""}` : ""
33
+ // How much of the request keeps the ASD-STE100 rules, as Refine's prompt asks.
34
+ const ste = state.phase === "done" && state.result.ste !== undefined ? ` · STE ${Math.round(state.result.ste * 100)}%` : ""
35
+ const title = state.phase === "working" ? " ✦ Refining " : state.phase === "error" ? " ✦ Could not refine " : ` ✦ Refined request${ste}${cost} `
32
36
  // Facts Refine checked that are also context lines are marked there, not repeated under Checked.
33
37
  const contextLines = state.phase === "done" ? state.result.refined.split("\n").filter((l) => l.startsWith("- ")).map((l) => norm(l.slice(2))) : []
34
38
  const inContext = new Set(contextLines)
@@ -61,9 +65,9 @@ export function RefinePanel(props: { state: RefineState; width: number; onChoose
61
65
  <text fg={theme.err}>{state.message}</text>
62
66
  ) : (
63
67
  <>
64
- {/* The request as it is sent: its labelled parts (Goal, Context, Steps, Rules, Result) stand out. */}
68
+ {/* The request as it is sent: its labelled parts (Role, Objective, Context, Steps, Rules, Output format, Summary) stand out. */}
65
69
  {state.result.refined.split("\n").map((line, i) => {
66
- const label = /^(Goal|Context|Steps|Rules|Result):/.exec(line)
70
+ const label = /^(Role|Objective|Context|Steps|Rules|Output format|Summary|Goal|Result):/.exec(line)
67
71
  // A context line Refine checked against the systems: ✓ in place of its dash, so it is not listed twice.
68
72
  if (line.startsWith("- ") && checked.has(norm(line.slice(2)))) {
69
73
  return (