@capagents/forgeqe 1.1.0 → 1.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/HOW_TO_USE.md +10 -9
- package/docs/user-guide.html +9 -9
- package/package.json +1 -1
- package/prompts/brain.md +9 -1
- package/prompts/refine.md +12 -8
- package/src/mcp/server.ts +11 -1
- package/src/mcp/tools.ts +23 -0
- package/src/refine.ts +30 -14
- package/src/tui/App.tsx +16 -2
- package/src/tui/refine-panel.tsx +9 -5
package/HOW_TO_USE.md
CHANGED
|
@@ -467,23 +467,24 @@ The chat gets a short summary and a card with both paths as links, and **Open th
|
|
|
467
467
|
|
|
468
468
|
**`/` and `@` in the prompt.** Typing `/` lists the commands that match; typing `@` suggests references: recent runs (*last run*, then each by its folder), files in the repository by name or path (`@Login` finds `pages/Login.ts`), and, after `@` and a number such as `@1761`, the Azure DevOps work item with its title (read-only). Tab completes, ↑/↓ choose, Esc closes the list.
|
|
469
469
|
|
|
470
|
-
**Refine.** Ctrl-R (or *Refine my request* in the palette) turns what you typed into a task specification for ForgeQE's agent, in ASD-STE100 Simplified Technical English, before it is sent:
|
|
470
|
+
**Refine.** Ctrl-R (or *Refine my request* in the palette) turns what you typed into a task specification for ForgeQE's agent, laid out as a system prompt and written in ASD-STE100 Simplified Technical English, before it is sent. It has these parts, in this order: **Role** (who the agent acts as), **Objective** (one instruction), **Context** (the facts, with names), **Steps** (only when they add something), **Rules** (your limits and the setup's), **Output format** (what comes back and how it is laid out), and **Summary** (what you get, in one sentence, read last):
|
|
471
471
|
|
|
472
472
|
```
|
|
473
|
-
|
|
473
|
+
Role: A release analyst who reads Azure DevOps deployments.
|
|
474
|
+
|
|
475
|
+
Objective: Get what is deployed now in each environment of each Azure DevOps project.
|
|
474
476
|
|
|
475
477
|
Context:
|
|
476
|
-
-
|
|
477
|
-
-
|
|
478
|
-
- Latest checked CD build: build 1021, failed, finished 2026-09-21.
|
|
478
|
+
- System: Azure DevOps organization devcgazuredevops.
|
|
479
|
+
- Projects: Autothon, CQube.AI, BlackBox_AI, QI-Payments, QI-MobileBanking, Atlas.Ai, QI-Claims.
|
|
479
480
|
|
|
480
|
-
|
|
481
|
-
1. Review the recent runs of QI-Payments-CI and QI-Payments-CD.
|
|
482
|
-
2. Make a shareable report with a short manager summary.
|
|
481
|
+
Output format: One section for each project, with a line for each environment: the deployed build, its date, and its status.
|
|
483
482
|
|
|
484
|
-
|
|
483
|
+
Summary: Show the deployed build in each environment, and why each deployment failed.
|
|
485
484
|
```
|
|
486
485
|
|
|
486
|
+
Refine is one model session of its own, separate from the conversation: a few model steps (usually two) and read-only tool calls, typically 15K–25K tokens and 20–30 seconds. The panel title shows what it cost and how much of the request keeps the ASD-STE100 rules (*✦ Refined request · STE 100% · 20.8K tokens · 21s*), and its tokens count in the session total in the footer.
|
|
487
|
+
|
|
487
488
|
A read-only agent writes it. It starts knowing every capability and whether it works in this setup, and the domain; it checks names and ids in the systems that are set up (work items, test plans, pipelines, sprints, recent runs). The panel shows the request with what it **checked** ✓, what it **assumed** ~, and what is **missing** ? (for example "Which story? Give the work item id"); a capability that is not set up shows as missing too. Enter sends it, `e` puts it in the prompt to edit, Esc keeps yours. Refine never runs anything, writes files, or changes a work item; it never invents an id, URL, project, or file.
|
|
488
489
|
|
|
489
490
|
**Next steps.** When a request finishes, up to three suggestions appear above the prompt: from what it produced (*Open the report*, *Write a test for this run*, *Why did it fail?*, *Rerun the cases that failed*) and from the agent's answer, which suggests the follow-ups you are likely to want (*Write a release report*, *Show the failed CD logs*). Click one or press Alt-1 to Alt-3; suggestions that start work are put in the prompt first, to adjust and send.
|
package/docs/user-guide.html
CHANGED
|
@@ -622,20 +622,20 @@ Add a GitHub Actions workflow that runs tests/cases.yaml on every PR, and make i
|
|
|
622
622
|
<p><strong>Setup checklist.</strong> Until everything is set up, the start screen's <strong>Setup</strong> line shows how many of the five things most features rest on are set up, and names the ones that are not: <strong>Model</strong> (a live model profile), <strong>Coding agent</strong>, <strong>Work items</strong> (Azure DevOps, Jira, TestRail, Zephyr, qTest, or Xray), <strong>Domain</strong>, and <strong>Notifications</strong>. <strong>▸ how to set up</strong> puts the steps for each missing one in the chat (for <strong>Work items</strong>, type <code>/auth</code>). The checks are the same ones <code>/features</code> uses, so the two always agree.</p>
|
|
623
623
|
<p><strong>Command palette.</strong> Ctrl-P opens a palette of every action, command, profile, and example request; type a few letters to find one ("rr" finds <em>Refine my request</em>, "auth" finds the sign-in commands), ↑/↓ to choose, Enter to run it. Example requests are put in the prompt for you to edit, never sent straight away.</p>
|
|
624
624
|
<p><strong><code>/</code> and <code>@</code> in the prompt.</strong> Typing <code>/</code> lists the commands that match; typing <code>@</code> suggests references: recent runs (<em>last run</em>, then each by its folder), files in the repository by name or path (<code>@Login</code> finds <code>pages/Login.ts</code>), and, after <code>@</code> and a number such as <code>@1761</code>, the Azure DevOps work item with its title (read-only). Tab completes, ↑/↓ choose, Esc closes the list.</p>
|
|
625
|
-
<p><strong>Refine.</strong> Ctrl-R (or <em>Refine my request</em> in the palette) turns what you typed into a task specification for ForgeQE's agent, in ASD-STE100 Simplified Technical English, before it is sent:</p>
|
|
626
|
-
<pre><code>
|
|
625
|
+
<p><strong>Refine.</strong> Ctrl-R (or <em>Refine my request</em> in the palette) turns what you typed into a task specification for ForgeQE's agent, laid out as a system prompt and written in ASD-STE100 Simplified Technical English, before it is sent. It has these parts, in this order: <strong>Role</strong> (who the agent acts as), <strong>Objective</strong> (one instruction), <strong>Context</strong> (the facts, with names), <strong>Steps</strong> (only when they add something), <strong>Rules</strong> (your limits and the setup's), <strong>Output format</strong> (what comes back and how it is laid out), and <strong>Summary</strong> (what you get, in one sentence, read last):</p>
|
|
626
|
+
<pre><code>Role: A release analyst who reads Azure DevOps deployments.
|
|
627
|
+
|
|
628
|
+
Objective: Get what is deployed now in each environment of each Azure DevOps project.
|
|
627
629
|
|
|
628
630
|
Context:
|
|
629
|
-
-
|
|
630
|
-
-
|
|
631
|
-
- Latest checked CD build: build 1021, failed, finished 2026-09-21.
|
|
631
|
+
- System: Azure DevOps organization devcgazuredevops.
|
|
632
|
+
- Projects: Autothon, CQube.AI, BlackBox_AI, QI-Payments, QI-MobileBanking, Atlas.Ai, QI-Claims.
|
|
632
633
|
|
|
633
|
-
|
|
634
|
-
1. Review the recent runs of QI-Payments-CI and QI-Payments-CD.
|
|
635
|
-
2. Make a shareable report with a short manager summary.
|
|
634
|
+
Output format: One section for each project, with a line for each environment: the deployed build, its date, and its status.
|
|
636
635
|
|
|
637
|
-
|
|
636
|
+
Summary: Show the deployed build in each environment, and why each deployment failed.
|
|
638
637
|
</code></pre>
|
|
638
|
+
<p>Refine is one model session of its own, separate from the conversation: a few model steps (usually two) and read-only tool calls, typically 15K–25K tokens and 20–30 seconds. The panel title shows what it cost and how much of the request keeps the ASD-STE100 rules (<em>✦ Refined request · STE 100% · 20.8K tokens · 21s</em>), and its tokens count in the session total in the footer.</p>
|
|
639
639
|
<p>A read-only agent writes it. It starts knowing every capability and whether it works in this setup, and the domain; it checks names and ids in the systems that are set up (work items, test plans, pipelines, sprints, recent runs). The panel shows the request with what it <strong>checked</strong> ✓, what it <strong>assumed</strong> ~, and what is <strong>missing</strong> ? (for example "Which story? Give the work item id"); a capability that is not set up shows as missing too. Enter sends it, <code>e</code> puts it in the prompt to edit, Esc keeps yours. Refine never runs anything, writes files, or changes a work item; it never invents an id, URL, project, or file.</p>
|
|
640
640
|
<p><strong>Next steps.</strong> When a request finishes, up to three suggestions appear above the prompt: from what it produced (<em>Open the report</em>, <em>Write a test for this run</em>, <em>Why did it fail?</em>, <em>Rerun the cases that failed</em>) and from the agent's answer, which suggests the follow-ups you are likely to want (<em>Write a release report</em>, <em>Show the failed CD logs</em>). Click one or press Alt-1 to Alt-3; suggestions that start work are put in the prompt first, to adjust and send.</p>
|
|
641
641
|
<div class="tbl"><table>
|
package/package.json
CHANGED
package/prompts/brain.md
CHANGED
|
@@ -12,7 +12,7 @@ Find what the user wants from their message. Then do it with your tools. ForgeQE
|
|
|
12
12
|
- A request to write or repair tests, or a CI pipeline task.
|
|
13
13
|
- A question.
|
|
14
14
|
|
|
15
|
-
A message can also be a task specification with the parts
|
|
15
|
+
A message can also be a task specification with the parts Role, Objective, Context, Steps, Rules, Output format, and Summary. Refine in the shell writes this type of message. Take the role. Do the objective. Use the context as facts. Do the steps in sequence. Obey the rules. Give the answer in the output format.
|
|
16
16
|
|
|
17
17
|
ForgeQE tools (the client adds the forgeqe_ prefix):
|
|
18
18
|
- browser_run: does one goal in a real browser, and verifies it. Use it for all work that must occur on a website. Set purpose to test for a flow of the application under test. Set purpose to task for all other work, such as finding, linking, or checking work items, or reading data from a tool. A task never becomes an automated test.
|
|
@@ -80,6 +80,13 @@ Azure DevOps, Jira, Xray, and other systems:
|
|
|
80
80
|
- Do not call the APIs of these systems from the shell, with curl, Python, PowerShell, or a script. The shell does not have the credentials, and ForgeQE blocks these commands. Use query_api.
|
|
81
81
|
- Azure DevOps classic releases are not in the MCP tools. Read them with query_api and the release host, for example https://vsrm.dev.azure.com/<organization>/<project>/_apis/release/releases?$top=10 or _apis/release/deployments.
|
|
82
82
|
- When the user asks about releases in Azure DevOps, read both: the classic releases, and the runs of the deployment pipelines (the pipelines MCP tools). Say which of them each project uses.
|
|
83
|
+
- For release details, find what is deployed now, not only the last run. Do these steps for each project:
|
|
84
|
+
1. Read the environments with query_api: <project>/_apis/distributedtask/environments.
|
|
85
|
+
2. Read the deployments of each environment: <project>/_apis/distributedtask/environments/<id>/environmentdeploymentrecords?top=5&api-version=7.1-preview.1.
|
|
86
|
+
3. Read the stages of each deployment run: <project>/_apis/build/builds/<id>/timeline. Give each stage with its result.
|
|
87
|
+
4. When a stage failed, give the cause from the timeline: the failed task, check, or approval, and its message.
|
|
88
|
+
5. Compare the newest CI build with the build that the environments have. Say when a newer build is not deployed.
|
|
89
|
+
- Keep these cases apart: a project without a deployment pipeline, a pipeline without runs, and a run that failed.
|
|
83
90
|
- Create or change items only when the user asks. Examples are to file a bug, add a comment, move an issue, or edit a work item. Say what you changed, with its id or key.
|
|
84
91
|
- The only exception is to save test cases that you wrote from a story (see below).
|
|
85
92
|
- Azure DevOps dashboards: use create_ado_dashboard, not the browser.
|
|
@@ -326,6 +333,7 @@ Rules:
|
|
|
326
333
|
- Use bold only for the one word or number that is important in a line, such as a status or a count. Do not make full sentences bold.
|
|
327
334
|
- Use a table only for two or three short columns.
|
|
328
335
|
- Keep replies short and specific. Give in full the test cases, tests, and reviews that the user asked for. Keep only your own comments short.
|
|
336
|
+
- A status alone is not an answer. When a result is "failed", empty, or "none found", find the cause with your tools before you reply. Give the cause and the next action that fixes it.
|
|
329
337
|
- When the user asks for a report, a write-up, or a summary to share, collect the facts with your tools. Examples are "create a release report" and "write up the open defects".
|
|
330
338
|
- Then save the report with write_report. Reply with three to five lines of summary. The shell shows the paths of the report as links, so do not repeat them.
|
|
331
339
|
- Do not only paste a long report into the reply.
|
package/prompts/refine.md
CHANGED
|
@@ -5,37 +5,41 @@ used_by: "the shell's Refine (ctrl+r)"
|
|
|
5
5
|
---
|
|
6
6
|
You refine a request that the user typed for ForgeQE, a test automation assistant. ForgeQE then sends the refined request to its agent. You do not do the request. You do not change data.
|
|
7
7
|
|
|
8
|
-
Write the refined request
|
|
8
|
+
Write the refined request in ASD-STE100 Simplified Technical English:
|
|
9
9
|
- Use short sentences: 20 words or fewer in an instruction, 25 words or fewer in a description.
|
|
10
10
|
- Give one instruction in each sentence. Start an instruction with a verb.
|
|
11
11
|
- Use the active voice.
|
|
12
12
|
- Use simple words with one meaning. For example, use "make sure", not "ensure", and "use", not "utilize".
|
|
13
13
|
- Use the same name for the same thing each time.
|
|
14
14
|
|
|
15
|
-
Fill in these parts:
|
|
16
|
-
-
|
|
15
|
+
Write the request as a system prompt for the agent. Fill in these parts, in this sequence:
|
|
16
|
+
- role: the role that the agent takes for this task, as one short description. Examples: "A release analyst who reads Azure DevOps deployments." and "A test engineer who runs manual test cases in the browser."
|
|
17
|
+
- objective: one instruction that says what the agent must do. Start with a verb. Name the exact thing to get or do, not a general word. Example: "Run the test cases of Azure DevOps test plan 412, suite 77." Example: "Get what is deployed now in each environment of each project, with the stages and the cause of each failed deployment."
|
|
17
18
|
- context: the facts that the agent needs. Give the system, the project, the work items, the files, and the runs with their exact names. Give one fact for each line.
|
|
18
19
|
- Give names, not internal ids such as GUIDs. Give an id only when it is the name that people use, for example a work item number, a test plan or suite number, or a build number.
|
|
19
20
|
- Put a list of items of the same type on one line, for example "Projects: Autothon, CQube.AI, QI-Payments".
|
|
20
21
|
- Give only the facts that this task needs. Do not give the working directory or the settings of forgeqe.yaml. The agent already has them.
|
|
21
|
-
- steps: the actions, in sequence, when the request has more than one action. Give one action for each step. Each step must add information that the
|
|
22
|
+
- steps: the actions, in sequence, when the request has more than one action. Give one action for each step. Each step must add information that the objective does not give, for example which data to read or how to compare it. When the steps only say the objective again, leave steps empty.
|
|
22
23
|
- rules: the limits that the user gave or that the setup sets. Examples are "Do not send results back to Azure DevOps", "Use 2 browsers at the same time", and "Show the browser". Give only rules that come from the user or from "This session".
|
|
23
|
-
-
|
|
24
|
+
- output: the output format. Say what the agent gives back and how it is laid out. Examples: "One section for each project, with a line for each environment: the deployed build, its date, and its status." and "A report file, with a summary of three lines in the chat."
|
|
25
|
+
- summary: what the user gets, in one sentence of 15 words or fewer. The agent reads it last. Do not copy the objective. Example: "Show the deployed build in each environment, and why each deployment failed."
|
|
24
26
|
|
|
25
27
|
Rules for your work:
|
|
26
28
|
- Keep the intent and the scope of the user exactly. Do not add work that the user did not ask for. Do not remove work that the user asked for.
|
|
29
|
+
- Keep each word that changes the result, such as "current", "all", "details", or "failed". For example, "current release" means what is deployed now, not the last run.
|
|
27
30
|
- Use the names that ForgeQE uses, for example "Azure DevOps project QI-Payments", "test plan 412, suite 77", "story 1761", or "SHOP-12".
|
|
28
31
|
- Use "This session" and "What ForgeQE can do here" to know the capabilities and the setup.
|
|
29
32
|
- Describe each action in plain words, for example "run the test cases" or "write a Playwright test". Do not name the tools of ForgeQE or the capability ids.
|
|
30
33
|
- Change references such as "the last run", "that story", or "the payments plan" into the item that they point to. Do this only when exactly one item fits.
|
|
31
34
|
- Check facts with your read-only tools. For example, check that a work item, a test plan, a suite, or a run exists, what its name is, and how many cases it has.
|
|
32
35
|
- Put each fact of context in grounded, with the same words, and say if you checked it. Do not add facts to grounded that are not in context.
|
|
33
|
-
- A request can mean more than one thing. Then select the meaning that the capabilities of ForgeQE support best, and write the
|
|
34
|
-
-
|
|
36
|
+
- A request can mean more than one thing. Then select the meaning that the capabilities of ForgeQE support best, and write the objective for it. Say the meaning that you selected in assumed.
|
|
37
|
+
- Do not ask about a meaning that you selected in assumed. The user sees assumed and can change it.
|
|
38
|
+
- Put a question in missing only when you cannot select a meaning and a wrong selection gives a different result. In the question, give the possible meanings, for example "Classic releases, deployment pipeline runs, or release readiness (GO / NO-GO)?".
|
|
35
39
|
- Do not invent an id, a URL, a project, or a file.
|
|
36
40
|
- If you cannot find or check an item, put a short question for the user in missing.
|
|
37
41
|
- If a capability that the request needs is not set up, put this in missing. Say what the user must set up.
|
|
38
42
|
- Keep the settings that the user gave, for example the number of browsers, a visible browser, or whether results go back. Do not add other settings.
|
|
39
43
|
- Write in the language of the user.
|
|
40
44
|
|
|
41
|
-
Finish with the structured result:
|
|
45
|
+
Finish with the structured result: role, objective, context, steps, rules, output, summary, grounded, assumed, and missing.
|
package/src/mcp/server.ts
CHANGED
|
@@ -54,8 +54,18 @@ export function createMcpServer(opts: McpServerOptions) {
|
|
|
54
54
|
base.emit?.(e)
|
|
55
55
|
}
|
|
56
56
|
: undefined
|
|
57
|
-
const
|
|
57
|
+
const said = (ok: boolean) => {
|
|
58
|
+
if (tool.line && emit) emit({ type: "call", text: tool.line(params.arguments || {}), ok })
|
|
59
|
+
}
|
|
60
|
+
let out: unknown
|
|
61
|
+
try {
|
|
62
|
+
out = await tool.run(params.arguments || {}, { ...base, emit, signal: controller.signal })
|
|
63
|
+
} catch (err) {
|
|
64
|
+
if (!controller.signal.aborted) said(false)
|
|
65
|
+
throw err
|
|
66
|
+
}
|
|
58
67
|
if (controller.signal.aborted) return finish({ content: [{ type: "text", text: "Stopped by the user." }], isError: true })
|
|
68
|
+
said(true)
|
|
59
69
|
return finish({ content: [{ type: "text", text: typeof out === "string" ? out : JSON.stringify(out, null, 2) }] })
|
|
60
70
|
} catch (err) {
|
|
61
71
|
return finish({ content: [{ type: "text", text: controller.signal.aborted ? "Stopped by the user." : (err as Error).message || String(err) }], isError: true })
|
package/src/mcp/tools.ts
CHANGED
|
@@ -98,6 +98,25 @@ export type ToolEvent =
|
|
|
98
98
|
| { type: "card"; card: Card }
|
|
99
99
|
/** write_report saved a report: its page and its Markdown. */
|
|
100
100
|
| { type: "report"; title: string; html: string; markdown: string }
|
|
101
|
+
/** A finished call of a tool that has no card of its own, as one line: "Azure DevOps API · QI-Claims/_apis/build/builds (top 5)". */
|
|
102
|
+
| { type: "call"; text: string; ok: boolean }
|
|
103
|
+
|
|
104
|
+
const API_LABEL: Record<string, string> = { azure_devops: "Azure DevOps", jira: "Jira", xray: "Xray", confluence: "Confluence" }
|
|
105
|
+
|
|
106
|
+
/**
|
|
107
|
+
* A query_api call as a line: "Azure DevOps API · QI-Claims/_apis/release/releases (top 1, queryOrder descending)". The
|
|
108
|
+
* host and organization are left out, as are api-version and the start of a WIQL query.
|
|
109
|
+
*/
|
|
110
|
+
export function queryLine(args: Record<string, any>): string {
|
|
111
|
+
const name = `${API_LABEL[String(args.system)] || String(args.system || "API")} API`
|
|
112
|
+
if (args.wiql) return `${name} · WIQL (${clip(String(args.wiql).replace(/\s+/g, " ").replace(/^SELECT .*? FROM WorkItems WHERE /i, ""), 90)})`
|
|
113
|
+
if (args.query) return `${name} · GraphQL (${clip(String(args.query).replace(/\s+/g, " "), 90)})`
|
|
114
|
+
const raw = String(args.path || "")
|
|
115
|
+
const url = /^https?:\/\//.test(raw) ? raw.replace(/^https?:\/\/[^/]+\/(?:[^/]+\/(?=[^/]+\/_apis\/))?/, "") : raw.replace(/^\//, "")
|
|
116
|
+
const [where, search = ""] = url.split("?")
|
|
117
|
+
const params = [...new URLSearchParams(search)].filter(([k]) => k !== "api-version").map(([k, v]) => `${k.replace(/^\$/, "")} ${v}`)
|
|
118
|
+
return `${name} · ${decodeURIComponent(where || "")}${params.length ? ` (${clip(params.join(", "), 90)})` : ""}`
|
|
119
|
+
}
|
|
101
120
|
|
|
102
121
|
export type ToolContext = {
|
|
103
122
|
config: AppConfig
|
|
@@ -135,6 +154,8 @@ export type McpTool = {
|
|
|
135
154
|
description: string
|
|
136
155
|
inputSchema: Record<string, unknown>
|
|
137
156
|
run(args: Record<string, any>, ctx: ToolContext): Promise<unknown>
|
|
157
|
+
/** The call as one line in the conversation, for a tool that shows nothing else: the ForgeQE tools are hidden there. */
|
|
158
|
+
line?(args: Record<string, any>): string
|
|
138
159
|
}
|
|
139
160
|
|
|
140
161
|
const clip = (s: string | undefined, n: number) => {
|
|
@@ -563,6 +584,7 @@ export const TOOLS: McpTool[] = [
|
|
|
563
584
|
limit: { type: "integer", minimum: 1, maximum: 25, description: "Most pages to return (default 10)" },
|
|
564
585
|
},
|
|
565
586
|
},
|
|
587
|
+
line: (args) => `Confluence · ${args.cql ? `search (${clip(String(args.cql), 90)})` : `pages (${clip((Array.isArray(args.pages) ? args.pages : []).map(String).join(", "), 90)})`}`,
|
|
566
588
|
async run(args, ctx) {
|
|
567
589
|
const pages = Array.isArray(args.pages) ? args.pages.map(String) : undefined
|
|
568
590
|
return readConfluence(ctx.config.sources, { pages, cql: args.cql ? String(args.cql) : undefined, limit: Number(args.limit) || undefined })
|
|
@@ -592,6 +614,7 @@ export const TOOLS: McpTool[] = [
|
|
|
592
614
|
},
|
|
593
615
|
required: ["system"],
|
|
594
616
|
},
|
|
617
|
+
line: queryLine,
|
|
595
618
|
async run(args, ctx) {
|
|
596
619
|
return queryApi(ctx.config.sources, { system: args.system, path: args.path, query: args.query, variables: args.variables, wiql: args.wiql, project: args.project })
|
|
597
620
|
},
|
package/src/refine.ts
CHANGED
|
@@ -10,6 +10,7 @@ import { CAPABILITIES } from "./capabilities/registry.ts"
|
|
|
10
10
|
import { allStatuses } from "./capabilities/status.ts"
|
|
11
11
|
import { domainNotes } from "./domain.ts"
|
|
12
12
|
import type { CodeEvent } from "./types.ts"
|
|
13
|
+
import { steScore } from "./ste.ts"
|
|
13
14
|
|
|
14
15
|
/**
|
|
15
16
|
* The shell's Refine (ctrl+r): what the user typed becomes a precise request for ForgeQE's agent, checked against what
|
|
@@ -20,13 +21,18 @@ import type { CodeEvent } from "./types.ts"
|
|
|
20
21
|
/** The only tools the refiner may call: each reads, none changes anything or starts a run. */
|
|
21
22
|
export const REFINE_TOOLS = ["capabilities", "query_api", "find_test_cases", "list_sprints", "run_details"]
|
|
22
23
|
|
|
23
|
-
/**
|
|
24
|
-
|
|
24
|
+
/**
|
|
25
|
+
* The refined request as a task specification, laid out as a system prompt: the role the agent takes, the objective,
|
|
26
|
+
* the context, the steps, the rules, the output format, and the objective again in one sentence at the end.
|
|
27
|
+
*/
|
|
28
|
+
export type RefinedSpec = { role: string; objective: string; context: string[]; steps: string[]; rules: string[]; output: string; summary: string }
|
|
25
29
|
|
|
26
30
|
export type Refined = {
|
|
27
|
-
/** The request that is sent: the spec written out (
|
|
31
|
+
/** The request that is sent: the spec written out (Role, Objective, Context, Steps, Rules, Output format, Summary). */
|
|
28
32
|
refined: string
|
|
29
33
|
spec?: RefinedSpec
|
|
34
|
+
/** The share of the request's sentences that keep the ASD-STE100 rules, 0 to 1. */
|
|
35
|
+
ste?: number
|
|
30
36
|
grounded: { fact: string; checked: boolean }[]
|
|
31
37
|
assumed: string[]
|
|
32
38
|
missing: string[]
|
|
@@ -37,11 +43,13 @@ const lines = { type: "array", items: { type: "string" } } as const
|
|
|
37
43
|
export const REFINE_SCHEMA = {
|
|
38
44
|
type: "object",
|
|
39
45
|
properties: {
|
|
40
|
-
|
|
46
|
+
role: { type: "string", description: "The role the agent takes for this task, as one description, e.g. A release analyst who reads Azure DevOps deployments" },
|
|
47
|
+
objective: { type: "string", description: "One instruction that starts with a verb: what the agent must do" },
|
|
41
48
|
context: { ...lines, description: "Facts the agent needs, one per line: system, project, work items, files, runs, with exact names and ids" },
|
|
42
49
|
steps: { ...lines, description: "The actions in sequence, one per step; empty for a request with one action" },
|
|
43
50
|
rules: { ...lines, description: "Limits the user gave or the setup sets, one per line" },
|
|
44
|
-
|
|
51
|
+
output: { type: "string", description: "The output format: what the agent gives back and how it is laid out (sections, a table, a file)" },
|
|
52
|
+
summary: { type: "string", description: "The objective again, in one short sentence, as the last line" },
|
|
45
53
|
grounded: {
|
|
46
54
|
type: "array",
|
|
47
55
|
items: { type: "object", properties: { fact: { type: "string" }, checked: { type: "boolean" } }, required: ["fact", "checked"] },
|
|
@@ -49,7 +57,7 @@ export const REFINE_SCHEMA = {
|
|
|
49
57
|
assumed: { type: "array", items: { type: "string" } },
|
|
50
58
|
missing: { type: "array", items: { type: "string" } },
|
|
51
59
|
},
|
|
52
|
-
required: ["
|
|
60
|
+
required: ["role", "objective", "context", "steps", "rules", "output", "summary", "grounded", "assumed", "missing"],
|
|
53
61
|
} as const
|
|
54
62
|
|
|
55
63
|
const REFINE_AGENT = {
|
|
@@ -64,11 +72,14 @@ const REFINE_AGENT = {
|
|
|
64
72
|
|
|
65
73
|
/** The spec as the request ForgeQE's agent gets: labelled parts, empty ones left out. */
|
|
66
74
|
export function composeRefined(spec: RefinedSpec): string {
|
|
67
|
-
const parts = [
|
|
75
|
+
const parts: string[] = []
|
|
76
|
+
if (spec.role) parts.push(`Role: ${spec.role}`)
|
|
77
|
+
parts.push(`Objective: ${spec.objective}`)
|
|
68
78
|
if (spec.context.length) parts.push(["Context:", ...spec.context.map((c) => `- ${c}`)].join("\n"))
|
|
69
79
|
if (spec.steps.length) parts.push(["Steps:", ...spec.steps.map((st, i) => `${i + 1}. ${st}`)].join("\n"))
|
|
70
80
|
if (spec.rules.length) parts.push(["Rules:", ...spec.rules.map((r) => `- ${r}`)].join("\n"))
|
|
71
|
-
if (spec.
|
|
81
|
+
if (spec.output) parts.push(`Output format: ${spec.output}`)
|
|
82
|
+
if (spec.summary) parts.push(`Summary: ${spec.summary}`)
|
|
72
83
|
return parts.join("\n\n")
|
|
73
84
|
}
|
|
74
85
|
|
|
@@ -97,13 +108,18 @@ export function refineRequest(text: string, recent: { user: string; reply: strin
|
|
|
97
108
|
}
|
|
98
109
|
|
|
99
110
|
/** Keeps only what the schema allows, so the shell never shows a malformed answer. */
|
|
100
|
-
export function cleanRefined(r: (Partial<Refined> & Partial<RefinedSpec>) | null | undefined, original: string): Refined {
|
|
111
|
+
export function cleanRefined(r: (Partial<Refined> & Partial<RefinedSpec> & { goal?: string; result?: string }) | null | undefined, original: string): Refined {
|
|
101
112
|
const strings = (v: unknown) => (Array.isArray(v) ? v.map(String).map((s) => s.trim()).filter(Boolean) : [])
|
|
102
|
-
const
|
|
103
|
-
|
|
113
|
+
const text = (v: unknown) => String(v || "").trim()
|
|
114
|
+
// goal and result: the names an older refine prompt (a project's own copy) gives the objective and the output.
|
|
115
|
+
const objective = text(r?.objective) || text(r?.goal)
|
|
116
|
+
const spec: RefinedSpec | undefined = objective
|
|
117
|
+
? { role: text(r?.role), objective, context: strings(r?.context), steps: strings(r?.steps), rules: strings(r?.rules), output: text(r?.output) || text(r?.result), summary: text(r?.summary) }
|
|
118
|
+
: undefined
|
|
119
|
+
const refined = spec ? composeRefined(spec) : text(r?.refined) || original
|
|
104
120
|
return {
|
|
105
|
-
refined
|
|
106
|
-
...(spec ? { spec } : {}),
|
|
121
|
+
refined,
|
|
122
|
+
...(spec ? { spec, ste: steScore(refined).score } : {}),
|
|
107
123
|
grounded: Array.isArray(r?.grounded) ? r!.grounded.filter((g) => g && String(g.fact || "").trim()).map((g) => ({ fact: String(g.fact).trim(), checked: g.checked === true })) : [],
|
|
108
124
|
assumed: strings(r?.assumed),
|
|
109
125
|
missing: strings(r?.missing),
|
|
@@ -150,7 +166,7 @@ export async function refine(opts: {
|
|
|
150
166
|
})
|
|
151
167
|
try {
|
|
152
168
|
let out = await session.prompt(refineRequest(opts.text, opts.recent))
|
|
153
|
-
if (!out.report && !out.error && !opts.signal?.aborted) out = await session.prompt("Give the structured result:
|
|
169
|
+
if (!out.report && !out.error && !opts.signal?.aborted) out = await session.prompt("Give the structured result: role, objective, context, steps, rules, output, summary, grounded, assumed, and missing.")
|
|
154
170
|
if (!out.report) throw new Error(out.error ? `Refine failed: ${out.error}` : "Refine gave no result.")
|
|
155
171
|
return cleanRefined(out.report, opts.text)
|
|
156
172
|
} finally {
|
package/src/tui/App.tsx
CHANGED
|
@@ -519,6 +519,7 @@ export function App({ options }: { options: TuiOptions }) {
|
|
|
519
519
|
}
|
|
520
520
|
|
|
521
521
|
/** Refines what is in the prompt; the user then decides what is sent. */
|
|
522
|
+
const refineCount = useRef(0)
|
|
522
523
|
const startRefine = () => {
|
|
523
524
|
const text = promptTextRef.current.trim()
|
|
524
525
|
const cfg = configRef.current
|
|
@@ -534,10 +535,19 @@ export function App({ options }: { options: TuiOptions }) {
|
|
|
534
535
|
const controller = new AbortController()
|
|
535
536
|
refineAbort.current = controller
|
|
536
537
|
setRefine({ phase: "working", original: text })
|
|
537
|
-
|
|
538
|
+
// Refine is a model session of its own: its tokens count in the session total, under a key per refine.
|
|
539
|
+
const key = `refine-${++refineCount.current}`
|
|
540
|
+
const started = Date.now()
|
|
541
|
+
let used = 0
|
|
542
|
+
const onEvent = (e: CodeEvent) => {
|
|
543
|
+
if (e.kind !== "usage") return
|
|
544
|
+
used = e.usage.input + e.usage.output
|
|
545
|
+
useTokens(key, used)
|
|
546
|
+
}
|
|
547
|
+
void (options.refine || refine)({ config: cfg, cwd: sessionSetup(cfg, options.repo).cwd, text, recent: convoRef.current, signal: controller.signal, onEvent })
|
|
538
548
|
.then((result) => {
|
|
539
549
|
if (controller.signal.aborted || refiningRef.current?.phase !== "working") return
|
|
540
|
-
setRefine({ phase: "done", original: text, result, choice: initialChoice(result.missing) })
|
|
550
|
+
setRefine({ phase: "done", original: text, result, choice: initialChoice(result.missing), ...(used ? { tokens: used } : {}), ms: Date.now() - started })
|
|
541
551
|
})
|
|
542
552
|
.catch((err) => {
|
|
543
553
|
if (controller.signal.aborted) return
|
|
@@ -914,6 +924,10 @@ export function App({ options }: { options: TuiOptions }) {
|
|
|
914
924
|
case "info":
|
|
915
925
|
push(e.ok === undefined ? "code" : e.ok ? "ok" : "err", e.text)
|
|
916
926
|
return
|
|
927
|
+
case "call":
|
|
928
|
+
pushTool(e.text, e.ok)
|
|
929
|
+
if (!aborted()) setActivity(`Agent: ${clip(e.text, 60)}`)
|
|
930
|
+
return
|
|
917
931
|
case "agent": {
|
|
918
932
|
const ev = e.event
|
|
919
933
|
inspect(ev)
|
package/src/tui/refine-panel.tsx
CHANGED
|
@@ -1,11 +1,12 @@
|
|
|
1
1
|
import type { Refined } from "../refine.ts"
|
|
2
2
|
import { Spinner } from "./components.tsx"
|
|
3
|
-
import { theme } from "./theme.ts"
|
|
3
|
+
import { formatDuration, formatTokens, theme } from "./theme.ts"
|
|
4
4
|
|
|
5
5
|
/** Refine's panel, in the prompt's place: working, the result with what was checked, or why it could not refine. */
|
|
6
6
|
export type RefineState =
|
|
7
7
|
| { phase: "working"; original: string }
|
|
8
|
-
|
|
8
|
+
/** `tokens` and `ms`: what Refine's own model session cost, shown in the title. */
|
|
9
|
+
| { phase: "done"; original: string; result: Refined; choice: number; tokens?: number; ms?: number }
|
|
9
10
|
| { phase: "error"; original: string; message: string }
|
|
10
11
|
|
|
11
12
|
export const REFINE_CHOICES = [
|
|
@@ -28,7 +29,10 @@ export function initialChoice(missing: string[]): number {
|
|
|
28
29
|
|
|
29
30
|
export function RefinePanel(props: { state: RefineState; width: number; onChoose: (id: RefineChoice) => void; onHover: (i: number) => void }) {
|
|
30
31
|
const { state, width } = props
|
|
31
|
-
const
|
|
32
|
+
const cost = state.phase === "done" && state.tokens ? ` · ${formatTokens(state.tokens)} tokens${state.ms ? ` · ${formatDuration(state.ms)}` : ""}` : ""
|
|
33
|
+
// How much of the request keeps the ASD-STE100 rules, as Refine's prompt asks.
|
|
34
|
+
const ste = state.phase === "done" && state.result.ste !== undefined ? ` · STE ${Math.round(state.result.ste * 100)}%` : ""
|
|
35
|
+
const title = state.phase === "working" ? " ✦ Refining " : state.phase === "error" ? " ✦ Could not refine " : ` ✦ Refined request${ste}${cost} `
|
|
32
36
|
// Facts Refine checked that are also context lines are marked there, not repeated under Checked.
|
|
33
37
|
const contextLines = state.phase === "done" ? state.result.refined.split("\n").filter((l) => l.startsWith("- ")).map((l) => norm(l.slice(2))) : []
|
|
34
38
|
const inContext = new Set(contextLines)
|
|
@@ -61,9 +65,9 @@ export function RefinePanel(props: { state: RefineState; width: number; onChoose
|
|
|
61
65
|
<text fg={theme.err}>{state.message}</text>
|
|
62
66
|
) : (
|
|
63
67
|
<>
|
|
64
|
-
{/* The request as it is sent: its labelled parts (
|
|
68
|
+
{/* The request as it is sent: its labelled parts (Role, Objective, Context, Steps, Rules, Output format, Summary) stand out. */}
|
|
65
69
|
{state.result.refined.split("\n").map((line, i) => {
|
|
66
|
-
const label = /^(
|
|
70
|
+
const label = /^(Role|Objective|Context|Steps|Rules|Output format|Summary|Goal|Result):/.exec(line)
|
|
67
71
|
// A context line Refine checked against the systems: ✓ in place of its dash, so it is not listed twice.
|
|
68
72
|
if (line.startsWith("- ") && checked.has(norm(line.slice(2)))) {
|
|
69
73
|
return (
|