@capagents/forgeqe 2.2.2 → 2.2.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/HOW_TO_USE.md CHANGED
@@ -98,7 +98,7 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
98
98
  | Setup | **Test environments and their apps**: Knows the test environments in env.yaml (QA, UAT, …) and each one's apps: web apps and APIs by URL, Windows desktop apps, and mobile apps, with what each is for. Every agent works from the active one; switches environment on request, picks the apps a scenario needs, and carries values such as an order number from one app to the next. | `test_apps`, `use_environment` | [docs](#environments-and-apps) |
99
99
  | Desktop | **Automate Windows desktop apps**: Carries out a plain-language goal in a Windows thick-client app with UFO² and saves it as a workflow with checks of the outcome, or records the steps a tester does by hand, with no model; replays saved workflows with no model, healing a step whose control moved, and fixes a workflow from chat (a timeout, a wait, a dialog to dismiss) after showing the change. Apps and the secrets they may type come from env.yaml or forgeqe.yaml desktop:. | `desktop_explore`, `desktop_record`, `desktop_replay`, `desktop_edit_workflow` | [docs](#desktop-apps-windows) |
100
100
  | Mobile | **Automate Android and iOS apps**: Carries out a plain-language goal in an Android or iOS app through Appium and saves it as a mobile workflow with checks of the outcome; replays workflows with no model on Appium (healing a moved control) or Maestro; runs the team's own Maestro flows after checking them against an allowlist; uploads an app build to TestMu AI and saves its lt:// id as the app's build. Apps come from env.yaml (kind: mobile) or forgeqe.yaml mobile:. | `mobile_explore`, `mobile_replay`, `mobile_maestro`, `mobile_upload` | [docs](#mobile-apps-android-ios) |
101
- | Operations | **Services and mocks in Docker or Podman**: Starts the services a test run needs (a database, a queue, the API under test) in Docker or Podman, as forgeqe.yaml declares them, and waits until each is ready; stops them, and shows their status and logs. Starts mocks of an API on WireMock, from the calls a browser run recorded or from an OpenAPI description, and can point an API app of the environment at the mock for the session. | `services`, `mock_service` | [docs](#containers-and-mock-services) |
101
+ | Operations | **Services, mocks, workspace and browser in Docker or Podman**: Starts the services a test run needs (a database, a queue, the API under test) in Docker or Podman, as forgeqe.yaml declares them, and waits until each is ready; stops them, and shows their status and logs. Starts mocks of an API on WireMock, from the calls a browser run recorded or from an OpenAPI description, and can point an API app of the environment at the mock for the session. With containers.workspace, the coding agents' shell commands (build, restore, run) and every test ForgeQE runs go to a container of the project's image; with browser.provider docker, the browser agent, the replay and the coding agent's live browser use Chromium in a container. | `services`, `mock_service` | [docs](#containers-and-mock-services) |
102
102
  | Data | **Ask a database in plain language**: Connects to the project's databases (PostgreSQL, MySQL and MariaDB, SQL Server and Azure SQL, Oracle, Snowflake, Redshift, CockroachDB, SQLite, DuckDB, and MongoDB) from a connection string or host and user, signing in with a password, Microsoft Entra ID or AWS IAM. Reads the schema through DB-GPT's connectors and turns questions into SQL that is checked before it runs: reads in a read-only session that is rolled back, writes only on a database marked writable, after you confirm them. | `databases`, `database_schema`, `run_sql` | [docs](#databases) |
103
103
  | Data | **Test data on demand**: Makes realistic test data from a spec that fits the database's schema (types, sizes, NOT NULL, unique keys, CHECK lists, enums), parents before children, the same rows for the same seed. Or takes a masked subset of a real database: the rows you pick and every row they need, with names, emails, phones, addresses and other personal columns replaced the same way every time. Shows a preview, writes CSV, JSON or SQL files for automation, or inserts the rows into a database marked writable after you confirm, as a batch that can be removed again. Specs live in the project, so tests and pipelines make the same data with forgeqe data gen. | `generate_test_data`, `test_data_batches` | [docs](#test-data) |
104
104
  | Data | **Data flows between systems**: Checks the data that passes between your systems' databases (the digital channel, CRM, billing, a warehouse), by batch ETL or in real time: each hop's rows arrived, complete, with every mapped column right after its rule, on time for its SLA, without duplicates. Follows one record end to end, says what a change to a table or column reaches downstream, and writes the flows from your mapping documents wherever they are, or suggests them from the schemas. | `data_flows`, `check_flows`, `trace_record`, `read_mapping_file` | [docs](#data-flows) |
@@ -120,7 +120,7 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
120
120
  | Operations | **CI pipelines**: Writes a GitHub Actions or Azure Pipelines workflow that runs ForgeQE, stores the secrets it needs, pushes a branch, and makes the pipeline green. | `push_branch`, `set_ci_secret` | [docs](#8-pipelines) |
121
121
  | Systems | **Work with Azure DevOps, Jira, GitHub, and Confluence**: Reads anything in the configured systems (work items, issues, queries, boards, pages), makes changes the user asks for, puts run evidence (screenshots, reports) on work items and issues as attachments or comments, and builds Azure DevOps dashboards and wiki pages. | `read_confluence`, `create_ado_dashboard`, `write_ado_wiki_page`, `attach_to_work_item` | [docs](README.md#azure-devops-jira-and-other-systems) |
122
122
  | Systems | **Read any data in Azure DevOps, Jira, Xray, and Confluence**: Reads through a system's REST API what its MCP tools do not cover: pull requests and repositories, test plans and suites, classic releases, and anything else in Azure DevOps, Jira, Xray, or Confluence. Given on every turn once a system is set up, so the agent never turns to the browser or the shell to read it. | `query_api` | [docs](README.md#azure-devops-jira-and-other-systems) |
123
- | Setup | **What ForgeQE wrote**: Says how much of the code ForgeQE wrote is still in the repository, and gives the trailer and footer that mark a commit message or pull request description covering code it wrote. Works with any repository, whatever system the team uses. | `generated_attribution` | [docs](#2-scaffold-config) |
123
+ | Setup | **What ForgeQE wrote**: Says how much of the code ForgeQE wrote is still in the repository, and gives the trailer and footer that mark a commit message or pull request description covering code it wrote. Works with any repository, whatever system the team uses. | `generated_attribution` | [docs](#generated-code) |
124
124
  | Setup | **Every forgeqe command from the shell**: Runs the forgeqe commands no other tool covers when the user asks in words: the context budget, guardrails, prompts (show, diff, check, reset), the domain eval, skills on or off, session logs, saved API sets and packs, pack pipelines, the BI site, a test notification, desktop and mobile doctor and validate, and updating ForgeQE. Reading commands run at once; the others are shown and asked first. | `forgeqe_command` | [docs](#15-commands-cheat-sheet) |
125
125
  | Setup | **Setup, skills, and documentation**: Shows the model profiles and MCP servers, installs or repairs the browser engine, creates a forgeqe.yaml, manages skills, and answers any question about ForgeQE from its documentation, this list, and its own code (the prompts, skills, and source it runs). | `forgeqe_setup`, `capabilities`, `playbook`, `use_capabilities` | [docs](#2-scaffold-config) |
126
126
 
@@ -167,11 +167,10 @@ forgeqe models # check the profiles
167
167
  forgeqe start # the shell
168
168
  ```
169
169
 
170
- With GitHub Copilot licences instead of a model key, sign in rather than exporting one, and set `llm.active: copilot` in `forgeqe.yaml` (or `"default": "copilot"` in `llms.json`):
170
+ With GitHub Copilot licences instead of a model key, sign in rather than exporting one, and set `llm.active: copilot` in `forgeqe.yaml` (or `"default": "copilot"` in `llms.json`). The browser engine then runs on your Copilot seat; ForgeQE's agent and Scribe need another profile for now:
171
171
 
172
172
  ```bash
173
- forgeqe auth login copilot # ForgeQE's agent and Scribe on your Copilot seat
174
- forgeqe auth login github # the browser engine, through GitHub Models
173
+ forgeqe auth login copilot # a fine-grained token with Copilot Requests, or the GitHub CLI's sign-in
175
174
  forgeqe models # the copilot profile is active
176
175
  ```
177
176
 
@@ -276,60 +275,63 @@ Give that identity the **Cognitive Services OpenAI User** role on the resource.
276
275
 
277
276
  ### GitHub Copilot seats as the model
278
277
 
279
- With GitHub Copilot licences (Pro, Pro+, Business or Enterprise), a `github_copilot` profile runs ForgeQE's agent and Scribe on the Copilot seat. They work through the agent runtime's own GitHub Copilot sign-in, [which GitHub supports](https://github.blog/changelog/2026-01-16-github-copilot-now-supports-opencode/). Copilot gives other tools no model API of their own, so the browser engine and the desktop agent use [GitHub Models](https://docs.github.com/en/github-models) instead, with your GitHub sign-in:
278
+ With GitHub Copilot licences (Pro, Pro+, Business or Enterprise), a `github_copilot` profile runs the browser engine and the desktop and mobile agents on your Copilot seat, through [GitHub's Copilot SDK](https://docs.github.com/en/copilot/how-tos/copilot-sdk). Copilot gives other tools no model API of its own, so ForgeQE starts a small gateway on `127.0.0.1` that runs the SDK and speaks the OpenAI chat API to the engine. It signs in with a GitHub token, so no other app is authorized on your GitHub account.
280
279
 
281
280
  ```json
282
281
  "copilot": {
283
282
  "provider": "github_copilot",
284
- "model": "claude-sonnet-4.6",
285
- "engine_model": "openai/gpt-4.1",
286
- "github_org": "acme"
283
+ "model": "gpt-4.1"
287
284
  }
288
285
  ```
289
286
 
290
287
  | Field | What it is |
291
288
  |---|---|
292
- | `model` | The Copilot model for ForgeQE's agent and Scribe, as Copilot names it (`claude-sonnet-4.6`, `gpt-5.4`). The sign-in lists the ones your seat has. |
293
- | `engine_model` | The GitHub Models model for the browser engine and the desktop agent, as `publisher/model`. It must take images. Default `openai/gpt-4.1`. |
294
- | `github_org` | Optional. Bills the engine's GitHub Models calls to this organization and applies its policies, rather than your personal free allowance. |
295
- | `api_key_env` | Optional. The variable holding the GitHub Models token; default `GITHUB_TOKEN`, else the GitHub CLI's sign-in. |
289
+ | `model` | The Copilot model, as the Copilot SDK names it (`gpt-4.1`, `claude-sonnet-4.6`). It must take screenshots. The sign-in lists your seat's models and which of them take images. |
290
+ | `engine_model` | Optional. Another Copilot model for the browser engine and the desktop agent. |
291
+ | `api_key_env` | Optional. The variable holding the GitHub token; without it ForgeQE reads `COPILOT_GITHUB_TOKEN`, then the token `forgeqe auth login copilot` saved, then the GitHub CLI's sign-in. |
296
292
 
297
293
  Sign in once per machine:
298
294
 
299
295
  ```bash
300
- forgeqe auth login copilot # Copilot, for ForgeQE's agent and Scribe (GitHub.com or GitHub Enterprise, e.g. acme.ghe.com)
301
- forgeqe auth login github # the GitHub CLI, whose sign-in the browser engine uses for GitHub Models
296
+ forgeqe auth login copilot
302
297
  ```
303
298
 
304
- `forgeqe auth login copilot` shows a one-time code and opens GitHub's device page; authorize with the account that has the seat. The sign-in goes to your system's credential store like ForgeQE's other secrets. The agent runtime gets it in its environment for each session, never in a file, and the agent's shell commands never see it. A Copilot sign-in made with `opencode auth login` also works. In CI, set `GITHUB_TOKEN` to a token with the `models:read` permission for the engine.
299
+ It offers two ways:
305
300
 
306
- `forgeqe auth` shows the sign-in on its `Copilot` line (`saved sign-in as dana`, or `agent runtime's own sign-in` for one made with `opencode auth login`); `forgeqe auth logout copilot` forgets it.
301
+ - **A fine-grained token** (recommended). ForgeQE opens GitHub's token page: set **Resource owner** to your personal account and add the **Copilot Requests** permission, then paste the token (`github_pat_...`). The token goes to your system's credential store like ForgeQE's other secrets. Classic tokens (`ghp_`) do not work with Copilot.
302
+ - **The GitHub CLI's sign-in**, when `gh` is signed in.
303
+
304
+ Either way ForgeQE checks the token by asking Copilot for your seat's models and lists them; the first time, the Copilot SDK downloads Copilot's runtime from GitHub (checked against GitHub's checksums). `forgeqe auth` shows the sign-in on its `Copilot` line; `forgeqe auth logout copilot` forgets a saved token.
305
+
306
+ The token reaches only the gateway's process. The engine gets the gateway's address and a one-time key that works nowhere else; the gateway refuses requests without that key and any request a web page sends. Model-run shell commands never see the token or the key.
307
307
 
308
308
  Which part uses what:
309
309
 
310
310
  | Part | Model | Signs in with |
311
311
  |---|---|---|
312
- | ForgeQE's agent (the shell, `forgeqe run`) | `model`, on the Copilot seat | `forgeqe auth login copilot` |
313
- | Scribe (writing, fixing and reviewing tests) | `model`, on the Copilot seat | `forgeqe auth login copilot` |
314
- | The browser engine (every browser run: `run --direct`, suites, monitoring) | `engine_model`, on GitHub Models | `GITHUB_TOKEN`, or `forgeqe auth login github` |
315
- | The desktop agent | `engine_model`, on GitHub Models | `GITHUB_TOKEN`, or `forgeqe auth login github` |
312
+ | The browser engine (every browser run: `run --direct`, suites, monitoring) | `engine_model` or `model`, on the Copilot seat | `forgeqe auth login copilot` |
313
+ | The desktop and mobile agents | `engine_model` or `model`, on the Copilot seat | `forgeqe auth login copilot` |
314
+ | ForgeQE's agent and Scribe | Not on Copilot yet: give them another profile with `--profile`, or `code.model` | |
316
315
 
317
- `code.model` still overrides `model` for ForgeQE's agent and Scribe; `code.model: github-copilot/<model>` uses the Copilot sign-in too, with any other profile active.
316
+ Copilot runs one SDK session per model request, with no tools of its own and your request's system prompt in place of Copilot's. Each request is billed to the seat (GitHub AI credits or premium requests, by your plan), and GitHub may rate-limit frequent automated requests. If a model is refused, check that your organization's Copilot policies allow it. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
318
317
 
319
- What each part costs: the agent's and Scribe's requests count against the seat's premium requests. The engine's GitHub Models calls are billed by GitHub Models: the free allowance is rate-limited, so for suites set `github_org` to an organization with paid GitHub Models usage turned on. If a Copilot model is refused, check that your organization's Copilot policies allow that model. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
318
+ **In pipelines.** `forgeqe suite ... --no-code` uses only the browser engine. Set `COPILOT_GITHUB_TOKEN` to a fine-grained token with the Copilot Requests permission, stored as a pipeline secret. GitHub also documents the workflow's own `GITHUB_TOKEN` with `permissions: copilot-requests: write` for organization repositories whose Copilot policy allows it ([server tokens](https://docs.github.com/en/copilot/how-tos/copilot-sdk/auth/server-to-server-tokens)). A Copilot seat belongs to one person: prefer an organization-billed token over signing a shared pipeline in to someone's seat.
320
319
 
321
- **In pipelines.** `forgeqe suite ... --no-code`, which pipelines run, uses only the browser engine, so it needs GitHub Models and no Copilot sign-in. In GitHub Actions, give the job `permissions: models: read` and pass the workflow's own token as `GITHUB_TOKEN`; on Azure Pipelines or elsewhere, store a fine-grained token with the `models:read` permission as the `GITHUB_TOKEN` secret. A Copilot seat belongs to one person, so keep ForgeQE's agent and Scribe on developers' machines rather than signing a shared pipeline in to someone's seat.
320
+ GitHub Models, which ForgeQE used for the engine before, was [retired by GitHub](https://github.blog/changelog/2026-07-30-github-models-is-now-retired/) on July 30, 2026. A profile that still has `engine_model: "openai/..."` or `github_org` needs updating: drop `github_org`, and use a Copilot model id.
322
321
 
323
322
  **When something goes wrong.**
324
323
 
325
324
  | ForgeQE says | What to do |
326
325
  |---|---|
327
- | `The github_copilot profile for … needs a GitHub Copilot sign-in` | Run `forgeqe auth login copilot` (or `/auth copilot` in the shell). |
328
- | `The browser engine reaches GitHub Models with a GitHub sign-in, and there is none` | Run `forgeqe auth login github`, or set `GITHUB_TOKEN` to a token with `models:read`. |
329
- | `… must be a Copilot model id`, `engine_model must be a GitHub Models id`, `github_org must be a GitHub organization name` | Fix that field in `llms.json`: the sign-in lists the seat's Copilot models, and [the GitHub Models catalog](https://github.com/marketplace?type=models) lists `publisher/model` ids. |
330
- | The agent's model is refused, or not found | The seat's plan or your organization's Copilot policies do not allow that model: pick one the sign-in listed, or ask a Copilot admin to enable it. |
331
- | The browser engine gets `429` / rate limit errors | The GitHub Models free allowance is used up: set `github_org` to an organization with paid GitHub Models usage, or wait for the limit to reset. |
332
- | The code page does not open, or the sign-in says `not a GitHub Enterprise domain` | Give only the host of your GitHub Enterprise (`acme.ghe.com`). ForgeQE opens only GitHub's own `https` device page. |
326
+ | `GitHub Copilot needs a GitHub token, and there is none` | Run `forgeqe auth login copilot` (or `/auth copilot` in the shell), or set `COPILOT_GITHUB_TOKEN`. |
327
+ | `GitHub Copilot does not take classic tokens (ghp_)` | Make a fine-grained token with the Copilot Requests permission, owned by your personal account. |
328
+ | `GitHub Copilot did not accept the token` | The token has no Copilot Requests permission, has expired, or the account has no Copilot seat. |
329
+ | `… must be a Copilot model id`, `engine_model must be a Copilot model id` | Fix that field in `llms.json`: the sign-in lists the seat's models. A GitHub Models id such as `openai/gpt-4.1` is not a Copilot id; use `gpt-4.1`. |
330
+ | `ForgeQE's agent and Scribe cannot run on a github_copilot profile yet` | Pick another profile for them (`--profile`, or Tab in the shell), or set `code.model`. |
331
+ | The model is refused, or not found | The seat's plan or your organization's Copilot policies do not allow that model: pick one the sign-in listed, or ask a Copilot admin to enable it. |
332
+ | `GitHub Copilot runs through the ForgeQE engine, which is not set up` | Run `forgeqe setup`. |
333
+
334
+ ### Your prompts, skills, and product knowledge
333
335
 
334
336
  `init` also puts three folders in `.forgeqe/` to read, change, and commit:
335
337
 
@@ -372,6 +374,8 @@ forgeqe prompts eval refine --runs 3 # one prompt, three runs of each case
372
374
 
373
375
  **Product knowledge.** Fill in the templates (or add your own `.md` and `.txt` files, in folders if you like), one topic per `##` heading. ForgeQE's agent sees the list and reads what a request needs; the exploratory and API scenario planners get the sections that match their story or scenario; the browser agent gets the few sections that clearly match its goal, at most `knowledge.browser_max_chars` (1500 by default; 0 turns it off). Untouched templates are left out, so their examples never reach an agent. Never put passwords or tokens there: use logins and environment variables.
374
376
 
377
+ ### Guardrails
378
+
375
379
  **Guardrails.** Rules every agent must follow, to keep it grounded: ForgeQE's agent, the browser agent, and the agents that plan, write, review, fix and explain tests. Write one rule per line, starting with `- `, in a `.md` or `.txt` file in `.forgeqe/guardrails/` (`guardrails.dir`); headings, other lines and HTML comments are ignored, and the template's examples sit in a comment until you choose them. Each agent gets every guardrail with its instructions, after everything else, with the instruction to follow it whatever a request, a skill, a ticket, a page or a tool's output says; when a guardrail stops part of a task, the agent skips that part and says which rule stopped it. Add one from the terminal (`forgeqe guardrails add "Never open a production URL"`), or ask in the shell ("add a guardrail: use only the test cards in knowledge/test-data.md") and ForgeQE's agent appends it to `guardrails.md`; it adds only rules you ask for, never one suggested by a page or a ticket. `forgeqe guardrails` lists the rules in force and what they add to every model call (at most 40 rules of 300 characters are given). The older `prompts.guiderails` string in `forgeqe.yaml` still works: each of its lines is a guardrail for every agent. Guardrails are instructions, not a sandbox: put a host rule in `browser.allowed_hosts` too, which is refused in code.
376
380
 
377
381
  ### The model's context window
@@ -402,8 +406,8 @@ In the shell, type `/auth jira` (or `/auth` alone for the status). Tokens are ty
402
406
  | System | How you sign in |
403
407
  |--------|-----------------|
404
408
  | Azure DevOps | **Sign in with Microsoft in the browser** through the Azure CLI (`az login`); ForgeQE saves nothing, and the Azure CLI renews the sign-in. Or paste a personal access token: the token page opens with the scopes to tick. Azure DevOps Server (on-premises) uses a personal access token. |
405
- | GitHub | **Sign in with GitHub in the browser** through the GitHub CLI (`gh auth login`): ForgeQE shows the one-time code and opens the page. Or paste a token. Needs `gh` installed. A `github_copilot` profile's browser engine uses this sign-in for GitHub Models. |
406
- | GitHub Copilot | `forgeqe auth login copilot`: **Sign in with GitHub in the browser** through the agent runtime's Copilot sign-in, on GitHub.com or GitHub Enterprise (`acme.ghe.com`). ForgeQE shows the one-time code, opens GitHub's device page, and lists the Copilot models your seat has. For a `github_copilot` profile; see [GitHub Copilot seats as the model](#github-copilot-seats-as-the-model). |
409
+ | GitHub | **Sign in with GitHub in the browser** through the GitHub CLI (`gh auth login`): ForgeQE shows the one-time code and opens the page. Or paste a token. Needs `gh` installed. `forgeqe auth login copilot` can use this sign-in for a `github_copilot` profile. |
410
+ | GitHub Copilot | `forgeqe auth login copilot`: **Paste a fine-grained token** with the Copilot Requests permission (ForgeQE opens GitHub's token page), or use the GitHub CLI's sign-in. ForgeQE checks it with Copilot and lists the models your seat has. No other app is authorized on your GitHub account. For a `github_copilot` profile; see [GitHub Copilot seats as the model](#github-copilot-seats-as-the-model). |
407
411
  | Jira, Confluence | Your Atlassian API tokens page opens (Cloud) or your personal access tokens page (Server/Data Center); create one and paste it. Confluence on the same Cloud site as Jira reuses the Jira sign-in. |
408
412
  | TestRail, Zephyr Scale, qTest, Xray Cloud | The page with your API key opens, or ForgeQE says where to find it; paste it. |
409
413
  | Anything else | `forgeqe auth login OPENAI_API_KEY` (any variable name) saves that one secret: a model key, a Teams webhook URL, an SMTP or saved-login password. |
@@ -412,6 +416,8 @@ Atlassian's OAuth sign-in needs a client secret, which a command-line tool canno
412
416
 
413
417
  Each token is checked against the system before it is saved, and the system is switched on: a saved Jira sign-in sets the site URL too, so `sources.jira` is not needed in `forgeqe.yaml`. Saved values go in under the variable names the config already reads (`JIRA_API_TOKEN`, `ADO_PAT`, ...). A variable exported in the terminal is used before a saved sign-in, and a saved sign-in before a `.env` file, so pipelines keep using their secrets. The shell commands the agents run never see the saved secrets.
414
418
 
419
+ ### Model prices and usage
420
+
415
421
  **Prices.** ForgeQE prices every model call at the model's **list price**, from two public price lists: [models.dev](https://models.dev) (the list the agent runtime uses) and LiteLLM's model price list, which also has Azure's Data Zone prices. It keeps the few fields it needs in `~/.forgeqe/prices.json`, waits a few seconds for it the first time, refreshes it once a day in the background, and uses the last copy offline; only the public lists are downloaded, nothing about your runs is sent (`FORGEQE_PRICES=off` stops the downloads). Costs at list price say so: `$1.73 (list price of gpt-5.4 (Azure Global) from models.dev, 2026-10-06)`. For an Azure Data Zone deployment, which costs about 10% more, set `"deployment_type": "data_zone"` on the profile.
416
422
 
417
423
  **Model parameters.** Models differ in what a request may carry: reasoning models (o1, o3, o4, gpt-5) take `max_completion_tokens` instead of `max_tokens` and only the default temperature, and some OpenAI-compatible servers refuse `max_completion_tokens`, `reasoning_effort` or `response_format`. ForgeQE works this out by itself. A profile is a reasoning model when its `model` or `deployment` name says so (an Azure deployment called `chat-prod` with `"model": "gpt-5.4"` counts), or when it sets `"reasoning": true`; `"reasoning": false` says it is not. When the API still refuses a parameter, the browser, desktop and mobile agents send the call again without it and leave it out of later calls to that model. Nothing needs setting for a new model.
@@ -424,6 +430,32 @@ A profile's own `"price": { "input": 2.5, "output": 15, "cached_input": 0.25 }`
424
430
  This request used 1.1M tokens (1.1M in, 30.0K of it cached · 34.7K out) · $3.07 — ForgeQE's agent 42.0K · $0.12, browser agent 52.7K · $0.15, Scribe 1.0M · $2.80
425
431
  ```
426
432
 
433
+ ### Spending budgets
434
+
435
+ A budget warns you before model spend gets out of hand. Set it under `budget:` in `forgeqe.yaml`, in the currency of your prices (see [Model prices and usage](#model-prices-and-usage)):
436
+
437
+ ```yaml
438
+ budget:
439
+ run: 2 # one browser run
440
+ day: 25
441
+ month: 400
442
+ # currency: USD # default: the currency of the model's price
443
+ ```
444
+
445
+ Set any of the three. A budget **only warns; it never stops a run**. Each budget warns once at 80%, once at 90%, and once at 100%. When spend jumps past more than one of these at once, you get one warning, for the highest:
446
+
447
+ ```
448
+ Budget: this month's model spend is at $245.10, 82% of the $300.00 month budget (budget.month in forgeqe.yaml). Nothing is stopped.
449
+ ```
450
+
451
+ - **Where you see it.** In the shell, the warning is a line in the conversation. Every other `forgeqe` command writes it to the terminal's error output.
452
+ - **What counts.** `run` is the spend of one browser run. `day` and `month` add up every ForgeQE process on this machine (days and months are in UTC). Runs that the agent starts through the MCP server count too, because every process writes its model calls to the usage ledger in `~/.forgeqe/usage/` (`FORGEQE_USAGE_DIR` moves it). The ledger holds counts, times, and names, never prompt or response text. With `FORGEQE_USAGE=off`, nothing is kept on disk, and a budget counts only the calls of the process that warns.
453
+ - **What does not count.** A call with no price, such as a `github_copilot` profile without a `price`, and a call priced in a different currency than the budget.
454
+ - **Spend from before.** If you start ForgeQE when the day or month is already past 80%, it does not repeat that warning. It warns only for a threshold that spend passes from then on.
455
+ - **In reports.** The **Trends and forecast** page of a report's Tokenomics shows spend on this machine and the next days at the recent pace, against the budget.
456
+
457
+ ### How Scribe writes a test
458
+
427
459
  **Reuse before rebuild, extend before duplicate.** Before Scribe writes anything, `write_test` measures what the repository's tests already cover of the run, from evidence rather than guesswork: each step and check the browser agent made on an element is matched, through page memory, to the page-object member that locates it, and the code graph says which tests reach that member (through the page methods they call, or a property they assert, such as `Expect(practicePage.AjaxStatus)`). Steps on elements no page object locates yet are listed as not measured, since a test may do them its own way. Then:
428
460
 
429
461
  | What it finds | What happens |
@@ -443,8 +475,12 @@ A card says which it did and why (*Extending PracticeTests.Flow · covers 12 of
443
475
 
444
476
  **Keeping Scribe's cost down.** ForgeQE reads the repository's test setup itself before Scribe starts (stack, settings files and their browser values, the test id attribute, base classes, page objects and their methods, one test command) and hands it over, so Scribe does not discover it with dozens of file reads. Each attempt after the first starts a fresh session from a summary of the job (handoff, files changed, what each attempt did, the latest failure), so later attempts cost about what the first did. `code.max_attempts` (default 3) and `code.max_tokens` (default 1,500,000; `0` turns it off) cap a job, and the token budget is checked during an attempt too, so one long attempt is stopped when the total passes it. The reviewer sorts its issues into correctness (the test does not test what the goal says, or not reliably) and coding standards (test data or credentials in a page object, naming, hard waits, leftovers). Coding standards never fail a test that works: it is accepted, and the coding agent gets one more round (not counted in `max_attempts`) to fix them; if that round breaks the test, ForgeQE puts back the version that passed and lists the standards issues in the notes (`code.tidy: false` skips the round). A test the reviewer rejects twice for correctness stops the job with its issues, since that usually means the test case and the page disagree. Under each step, the handoff shows that step's part of the page (its section and neighbours, from the replay's page snapshot) instead of whole snapshots. When a test fails, ForgeQE shows the failing element on the test's page at the failure and on the replay's page, so most failures are fixed without opening a browser. Scribe's live browser is switched on only when the same failure repeats, and it does not send a page snapshot after every click or keystroke. `code.max_steps` (default 50) caps the tool calls in one of Scribe's turns. `code.review_profile` runs the reviewer on a cheaper llms.json profile.
445
477
 
478
+ ### Generated code
479
+
446
480
  **Generated code.** What ForgeQE writes is marked, so the team can see it and measure it. A file ForgeQE created starts with one line in that language's comment, such as `// Generated with ForgeQE · fq:k3x7m2qa4bdi · run 20261019-162847`; the `fq:` id is a hash of the file's contents without that line, so it shows whether the file was changed since (`code.header: false` leaves the line out; a JSON file never gets one). Files it only edited get no header. Every file it created or edited is recorded per run in `.forgeqe/generated.jsonl` (commit it with the code): the lines it added, as short hashes, never their text. `forgeqe generated` (or `/generated` in the shell, `--json` for a pipeline) says how many lines ForgeQE wrote, how many are still in the code, its share of the files it touched, and which created files are as it wrote them. Commit messages ForgeQE's agent writes end with a `Generated-by: ForgeQE (fq:…)` trailer, and its pull request descriptions with a line naming the runs' `fq:` ids and how much they wrote. The ids are plain hashes anyone can recompute: they show what ForgeQE wrote, not who owns the code.
447
481
 
482
+ ### Choosing the model and the browser
483
+
448
484
  `forgeqe.yaml` selects the active profile:
449
485
 
450
486
  ```yaml
@@ -462,6 +498,8 @@ browser:
462
498
  allowed_hosts: []
463
499
  judge: false # true: a second model call checks each finished run and can fail it
464
500
  overlay: true
501
+ window: maximized # visible browser: maximized | viewport (a window of viewport's size)
502
+ video: true # video.mp4 of each run, shown in its report
465
503
  # start_url: https://staging.example.com
466
504
  ```
467
505
 
@@ -471,8 +509,12 @@ Each step waits for the tab the agent is on to finish loading (up to 8 seconds,
471
509
 
472
510
  With a visible browser (`headless: false` or `--headed`), ForgeQE opens on its own start page, and every page it drives gets a glowing edge and a bar along the bottom showing, in plain words, what the step is for and what the agent is doing ("Step 19 · Check whether the results can be sorted by Fastest · Looking through the page's buttons"). Your mouse and keyboard are locked while the agent thinks and unlocked while it clicks and types, so a stray click can't change the test. None of this appears in the screenshots the model sees or in the report. `overlay: false` turns it off. `forgeqe login <name>` shows the same bar without the lock, since you are the one signing in.
473
511
 
512
+ A visible browser opens maximized. `window: viewport` opens it at `viewport`'s width and height instead; a headless page is always that size (1280×800 by default). Each run is recorded to `video.mp4` in its run folder, and the report plays it on the run's page; `video: false` turns recording off. The video needs packages that `forgeqe setup` adds to the engine (an engine set up before this version gets them on its next run).
513
+
474
514
  `start_url` is optional. Leave it out and the agent picks the site from the goal; ForgeQE never guesses a URL from the goal text. `allowed_hosts` empty allows every host. Set `["example.com"]` to keep the browser on that domain and its subdomains. `prompts.step_instruction` is appended to the engine's system prompt; the lines of `prompts.guiderails` are guardrails for every agent (see Guardrails above).
475
515
 
516
+ ### Model routing
517
+
476
518
  `llm.routing` is `off` by default for existing projects (`forgeqe init` writes `auto`); `shadow` logs each pick in the session log but changes nothing. `--profile` and `code.model` always win, and `llm.routes` limits or pins the profiles an area may use. ForgeQE checks and rates the models once and keeps the result in `.forgeqe/model-profile.json` (you can edit the ratings). Details: [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md).
477
519
 
478
520
  List profiles, and the model each area gets:
@@ -544,7 +586,7 @@ The slash commands control the session and show a few things directly (`/help` l
544
586
 
545
587
  `start_url` in `forgeqe.yaml` or `--url` pins a start page for browser runs, such as a staging app. Passed runs become tests in the current folder; `forgeqe start --repo ../app` puts them in that repo instead, and `--no-code` turns tests off.
546
588
 
547
- ForgeQE's agent runs on the agent runtime with the active profile's model (or `code.model`), or, with `llm.routing: auto`, the model the router picks for the `agent` area when the session starts (`--profile` and `code.model` still win); a `github_copilot` profile runs it on your Copilot seat. ForgeQE downloads the agent runtime into `~/.forgeqe/tools` when it is not installed. When the agent runtime cannot be downloaded, ForgeQE says the agent is off and runs each message as one browser goal.
589
+ ForgeQE's agent runs on the agent runtime with the active profile's model (or `code.model`), or, with `llm.routing: auto`, the model the router picks for the `agent` area when the session starts (`--profile` and `code.model` still win); it cannot run on a `github_copilot` profile yet. ForgeQE downloads the agent runtime into `~/.forgeqe/tools` when it is not installed. When the agent runtime cannot be downloaded, ForgeQE says the agent is off and runs each message as one browser goal.
548
590
 
549
591
  While a run is going, the bar under the prompt shows a spinner, the tokens used, and the elapsed time, and the transcript ends with what the agent is doing now (`Thinking about step 3…`, `Acting in the browser (step 3)…`). Each step starts with `+ Thought: 4.2s`, the time the model took to decide. When the run ends, a summary card closes the transcript:
550
592
 
@@ -890,7 +932,7 @@ After ForgeQE's reply, the request ends with the files it produced:
890
932
 
891
933
  When no test was written, the Test row says why: the run did not pass, the session has `--no-code`, or the run was not handed to the coding agent.
892
934
 
893
- Scribe uses the active `llms.json` profile, or, with `llm.routing: auto`, the profile the router picks for the `coding` area (`--profile` and `code.model` still win; see [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md)). The key stays in your environment: the agent runtime's config refers to it as `{env:WP_AZURE_API_KEY}` and never contains the value. An `auth: entra` profile reaches the model through ForgeQE's local token gateway (see [Signing in to the model without a key](#signing-in-to-the-model-without-a-key-microsoft-entra-id)). An Azure profile goes through the agent runtime's `azure` provider on `<endpoint>/openai/v1` (the Responses API, with the deployment as the model). `openai_compatible` and `ollama` profiles use their `base_url`. A `github_copilot` profile goes through the agent runtime's `github-copilot` provider with the profile's `model`, signed in with `forgeqe auth login copilot`. `code.model: provider/model` (or `--code-model`) uses one of the agent runtime's built-in providers instead, with that provider's usual key variable (for example `ANTHROPIC_API_KEY` for `anthropic/...`).
935
+ Scribe uses the active `llms.json` profile, or, with `llm.routing: auto`, the profile the router picks for the `coding` area (`--profile` and `code.model` still win; see [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md)). The key stays in your environment: the agent runtime's config refers to it as `{env:WP_AZURE_API_KEY}` and never contains the value. An `auth: entra` profile reaches the model through ForgeQE's local token gateway (see [Signing in to the model without a key](#signing-in-to-the-model-without-a-key-microsoft-entra-id)). An Azure profile goes through the agent runtime's `azure` provider on `<endpoint>/openai/v1` (the Responses API, with the deployment as the model). `openai_compatible` and `ollama` profiles use their `base_url`. A `github_copilot` profile cannot run Scribe yet, and `code.model: github-copilot/...` is refused: the agent runtime's own Copilot provider signs in through a third-party app on your GitHub account. `code.model: provider/model` (or `--code-model`) uses one of the agent runtime's built-in providers instead, with that provider's usual key variable (for example `ANTHROPIC_API_KEY` for `anthropic/...`).
894
936
 
895
937
  Scribe may edit files and run commands anywhere in the repo without asking. `git push` in any form is denied (pushing goes through ForgeQE's guarded `push_branch` tool, section 8), and it cannot touch files outside the repo except the run and memory directories, which it may read.
896
938
 
@@ -1829,6 +1871,29 @@ same rules (forgeqe.yaml's own declaration of a service wins):
1829
1871
  `x-forgeqe: { ready: … }` sets the ready check. Without it, a service is ready when its first port takes connections.
1830
1872
  The project's containers share their own network, so services reach each other by name (`db:5432`).
1831
1873
 
1874
+ **Docker as the workspace and the browser.** Two settings move the rest of the work into containers:
1875
+
1876
+ ```yaml
1877
+ browser:
1878
+ provider: docker # the browser agent, the replay and Scribe's live browser use Chromium in a container
1879
+ containers:
1880
+ workspace: # Scribe's shell commands and every test ForgeQE runs use this image
1881
+ image: mcr.microsoft.com/playwright/dotnet:v1.63.0-noble
1882
+ env: { NEXUS_ENVIRONMENT: CI }
1883
+ browser: # optional
1884
+ headless: false # visible on a virtual display inside the container (some sites refuse headless)
1885
+ ```
1886
+
1887
+ - **Workspace:** Scribe builds, restores, runs and fixes tests in a Linux container of the project's image, with the
1888
+ repository at the same path. Each test run gets its own container from that image, and `HOME` is a volume of the
1889
+ project's, so package caches outlive the session. The agents and the model key stay on this machine, and ForgeQE's
1890
+ own agent keeps this machine's shell. `.env`, keys and other credential files in the repository are covered inside
1891
+ the containers.
1892
+ - **Browser:** each browser run gets a fresh container, reached through a gateway that wants a random token. A
1893
+ declared service is reached by its name; an app on this machine is `host.docker.internal`, not `localhost`.
1894
+ `--browser-provider docker` picks it for one run.
1895
+ - `FORGEQE_HOST_RUN=1` runs Scribe's commands and the tests on this machine for a session.
1896
+
1832
1897
  **Tests in a container.** With `containers.tests`, `run_tests` runs the repository's tests in that image instead of
1833
1898
  on this machine, after starting the services it lists:
1834
1899
 
@@ -2400,7 +2465,7 @@ What it does:
2400
2465
  5. Stores the keys the pipeline needs (the model's key from `llms.json`, and `ADO_PAT`, `JIRA_EMAIL` and `JIRA_API_TOKEN`, or `XRAY_CLIENT_ID` and `XRAY_CLIENT_SECRET`) with ForgeQE's `set_ci_secret` tool. The agent only names the variable; ForgeQE passes the value from its own environment to `gh secret set` (on stdin) or `az pipelines variable` (through the environment), and never shows it. A variable that is not set in ForgeQE's environment is reported, not guessed.
2401
2466
  6. Triggers the run with `gh workflow run` or `az pipelines run`, watches it, reads the logs of failed steps, fixes the pipeline (or the code), pushes again, and repeats until the run passes or `code.max_attempts` runs have failed. It reports the run link and the result.
2402
2467
 
2403
- It needs `gh` logged in for GitHub, or `az` with the `azure-devops` extension logged in for Azure DevOps. Without them it still writes, validates, and commits the pipeline, then tells you the commands to run. Set `llm.active` to a live profile in the committed `forgeqe.yaml`. With a `github_copilot` profile the pipeline needs only a GitHub Models token: in GitHub Actions, `permissions: models: read` and the workflow's `GITHUB_TOKEN` (see [In pipelines](#github-copilot-seats-as-the-model)).
2468
+ It needs `gh` logged in for GitHub, or `az` with the `azure-devops` extension logged in for Azure DevOps. Without them it still writes, validates, and commits the pipeline, then tells you the commands to run. Set `llm.active` to a live profile in the committed `forgeqe.yaml`. With a `github_copilot` profile the pipeline needs a GitHub token for Copilot in `COPILOT_GITHUB_TOKEN` (see [In pipelines](#github-copilot-seats-as-the-model)).
2404
2469
 
2405
2470
  ## 9. Monitoring
2406
2471
 
@@ -2702,8 +2767,8 @@ the list again. For a client that ignores it, `forgeqe mcp --all-tools` lists ev
2702
2767
  | `WP_LLMS_JSON` | Path to `llms.json` |
2703
2768
  | `WP_LLM_PROFILE` | Active profile, unless `--profile` is set |
2704
2769
  | `WP_AZURE_API_KEY` / `OPENAI_API_KEY` / `WP_API_KEY` | Keys referenced by profiles |
2705
- | `GITHUB_TOKEN` | A `github_copilot` profile's GitHub Models token for the browser engine (needs `models:read`), unless the profile's `api_key_env` names another; else the GitHub CLI's sign-in |
2706
- | `FORGEQE_COPILOT_AUTH` | The Copilot sign-in `forgeqe auth login copilot` saves; set by ForgeQE from the credential store, not by hand |
2770
+ | `COPILOT_GITHUB_TOKEN` | A `github_copilot` profile's GitHub token (fine-grained, with the Copilot Requests permission), unless the profile's `api_key_env` names another; else the saved token, then the GitHub CLI's sign-in |
2771
+ | `FORGEQE_COPILOT_TOKEN` | The token `forgeqe auth login copilot` saves; set by ForgeQE from the credential store, not by hand |
2707
2772
  | `FORGEQE_CONFIG_DIR` | Global config folder (default `~/.config/forgeqe`): `forgeqe.yaml`, `llms.json` and `.env` used when a project has none |
2708
2773
  | `FORGEQE_PYTHON` | A Python that already has the engine's libraries, used instead of the managed venv (skipped, with a note, when it cannot run the engine) |
2709
2774
  | `FORGEQE_DEBUG` | `1` logs at debug level, like `--debug` |
package/README.md CHANGED
@@ -51,7 +51,7 @@ ForgeQE sends no telemetry and turns it off in everything it starts:
51
51
  - The agent runtime: auto-update, session sharing, OpenTelemetry spans, and the model catalogue download are off; it uses its bundled model list.
52
52
  - The repo's test command: `DO_NOT_TRACK=1`, plus the opt-outs for Next.js, Nuxt, Astro, Gatsby, Storybook, Turborepo, Angular, .NET, and Cypress.
53
53
 
54
- `OTEL_EXPORTER_OTLP_*` variables are removed from those processes, and your environment cannot turn any of this back on. The only traffic is to your model endpoint and the sites the agent visits. The engine also downloads its ad-block and cookie-banner extensions from the Chrome Web Store once.
54
+ `OTEL_EXPORTER_OTLP_*` variables are removed from those processes, and your environment cannot turn any of this back on. The traffic is to your model endpoint and the sites the agent visits. Besides that, ForgeQE downloads only public files: the model price lists from models.dev and LiteLLM, once a day (`FORGEQE_PRICES=off` stops them); the agent runtime from its GitHub releases, when it is not installed; and the engine's ad-block and cookie-banner extensions from the Chrome Web Store, once. Nothing about your runs is sent.
55
55
 
56
56
  ## Configure
57
57
 
@@ -88,7 +88,7 @@ The active profile is `llm.active` in `forgeqe.yaml`, else `default` in `llms.js
88
88
 
89
89
  **Model router.** With more than one model in `llms.json`, `llm.routing: auto` gives each area of work (ForgeQE's agent, browser runs, coding, review, analysis, planning) the cheapest configured model that is good enough for it. ForgeQE checks each model once and rates it, and keeps the result in `.forgeqe/model-profile.json`. `forgeqe models --routes` shows the model each area gets, and why. `--profile` and `code.model` still win. See [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md).
90
90
 
91
- **GitHub Copilot.** With Copilot licences, a `github_copilot` profile runs ForgeQE's agent and Scribe on the Copilot seat through the agent runtime's GitHub Copilot sign-in (`forgeqe auth login copilot`, GitHub.com or GitHub Enterprise). The browser engine uses GitHub Models with your GitHub sign-in (`forgeqe auth login github`, or `GITHUB_TOKEN`), optionally billed to an organization with `github_org`. See HOW_TO_USE.md, *GitHub Copilot seats as the model*.
91
+ **GitHub Copilot.** With Copilot licences, a `github_copilot` profile runs the browser engine and the desktop and mobile agents on the Copilot seat through GitHub's Copilot SDK, behind a gateway on `127.0.0.1`. It signs in with a GitHub token (`forgeqe auth login copilot`: a fine-grained token with the Copilot Requests permission, or the GitHub CLI's sign-in), so no third-party app is authorized on the account. ForgeQE's agent and Scribe do not run on Copilot yet. See HOW_TO_USE.md, *GitHub Copilot seats as the model*.
92
92
 
93
93
  ## CLI
94
94
 
@@ -60,7 +60,8 @@ ForgeQE downloads these onto the machine it runs on, from their own sources, whe
60
60
  | Bun | the JavaScript runtime, when the machine has none (bin/forgeqe.js) | MIT | https://bun.sh |
61
61
  | OpenCode | the agent runtime for ForgeQE's agent and Scribe (src/code/runtime.ts) | MIT | https://opencode.ai |
62
62
  | uv and Python | the browser engine's interpreter (src/engine/setup.ts) | MIT or Apache-2.0; PSF | https://github.com/astral-sh/uv |
63
- | browser-use, Playwright, PyYAML, azure-identity | the browser engine's libraries and Chromium (src/engine/setup.ts) | MIT; Apache-2.0; MIT; MIT | https://pypi.org |
63
+ | browser-use, Playwright, PyYAML, azure-identity, github-copilot-sdk | the browser engine's libraries and Chromium (src/engine/setup.ts) | MIT; Apache-2.0; MIT; MIT; MIT | https://pypi.org |
64
+ | GitHub Copilot CLI runtime | downloaded by GitHub's Copilot SDK for github_copilot profiles (src/copilot.ts) | GitHub Copilot CLI license (GitHub) | https://github.com/github/copilot-cli |
64
65
 
65
66
  ## Licence texts
66
67