@capagents/forgeqe 2.2.9 → 2.3.0

package/HOW_TO_USE.md

@@ -167,11 +167,12 @@ forgeqe models                        # check the profiles
 forgeqe start                         # the shell
 ```
 
-With GitHub Copilot licences instead of a model key, sign in rather than exporting one, and set `llm.active: copilot` in `forgeqe.yaml` (or `"default": "copilot"` in `llms.json`). The browser engine then runs on your Copilot seat; ForgeQE's agent and Scribe need another profile for now:
+With GitHub Copilot licences instead of a model key, sign in rather than exporting one. The sign-in lists your seat's models, recommends one (it takes screenshots and, where the plan has one, uses no premium requests), and writes your choice to `llms.json` as the `copilot` profile, made the default. Everything then runs on your Copilot seat: ForgeQE's agent, Scribe, and the browser, desktop and mobile engines. No other model is needed:
 
 ```bash
-forgeqe auth login copilot            # opens GitHub in your browser to sign in
-forgeqe models                        # the copilot profile is active
+forgeqe setup                         # the engine and GitHub's Copilot SDK
+forgeqe auth login copilot            # opens GitHub in your browser to sign in, then pick the model
+forgeqe doctor                        # checks the engine, the SDK, the agent runtime, the sign-in and each model
 ```
 
 In the shell, type a request and press Enter:
@@ -333,7 +334,7 @@ It offers two ways, the ones GitHub's Copilot SDK recommends:
 - **Sign in with GitHub in the browser** (recommended on your own computer). ForgeQE runs GitHub's own Copilot sign-in (`copilot login` from GitHub's Copilot CLI, downloaded once into `~/.forgeqe/tools` and checked against GitHub's checksums). Your browser opens GitHub; approve **GitHub Copilot CLI** there and you are signed in. GitHub's CLI keeps the sign-in in your system's credential store (Windows Credential Manager, the macOS Keychain); ForgeQE never sees a token. On a remote machine it shows a code to enter at github.com/login/device instead.
 - **A fine-grained token**, for machines without a browser. ForgeQE opens GitHub's token page: set **Resource owner** to your personal account and add the **Copilot Requests** permission, then paste the token (`github_pat_...`). Each character shows as `*`, and ForgeQE confirms what it got (`github_pat_11AB…Wz9Q (93 characters)`). The token goes to your system's credential store like ForgeQE's other secrets. Classic tokens (`ghp_`) do not work with Copilot.
 
-Either way ForgeQE checks the sign-in by asking Copilot for your seat's models and lists them; the first time, the Copilot SDK downloads Copilot's runtime from GitHub (checked against GitHub's checksums). `forgeqe auth` shows the sign-in on its `Copilot` line; `forgeqe auth logout copilot` makes ForgeQE stop using it (GitHub keeps a browser sign-in until you revoke **GitHub Copilot CLI** at github.com/settings/applications).
+Either way ForgeQE checks the sign-in by asking Copilot for your seat's models, then asks which one to use (recommended first; models your seat has but has not turned on are named, with the settings page) and writes it to `llms.json`; the first time, the Copilot SDK downloads Copilot's runtime from GitHub (checked against GitHub's checksums). `forgeqe auth` shows the sign-in on its `Copilot` line; `forgeqe auth logout copilot` makes ForgeQE stop using it (GitHub keeps a browser sign-in until you revoke **GitHub Copilot CLI** at github.com/settings/applications).
 
 If your organization uses SAML single sign-on, GitHub asks you to authorize the organization during the browser sign-in. A Copilot Business or Enterprise seat needs your organization to allow **Copilot CLI** in its Copilot policies.
 
@@ -345,9 +346,11 @@ Which part uses what:
 |---|---|---|
 | The browser engine (every browser run: `run --direct`, suites, monitoring) | `engine_model` or `model`, on the Copilot seat | `forgeqe auth login copilot` |
 | The desktop and mobile agents | `engine_model` or `model`, on the Copilot seat | `forgeqe auth login copilot` |
-| ForgeQE's agent and Scribe | Not on Copilot yet: give them another profile with `--profile`, or `code.model` | |
+| ForgeQE's agent and Scribe | `model`, on the Copilot seat | `forgeqe auth login copilot` |
 
-Copilot runs one SDK session per model request, with no tools of its own and your request's system prompt in place of Copilot's. Each request is billed to the seat (GitHub AI credits or premium requests, by your plan), and GitHub may rate-limit frequent automated requests. If a model is refused, check that your organization's Copilot policies allow it. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
+Copilot runs one SDK session per model request, with none of Copilot's own tools and your request's system prompt in place of Copilot's. The agent's tools are offered to the model as the session's only tools; when it calls one, the gateway ends the turn and hands the call back to ForgeQE's agent runtime, which runs it (with the same permissions as on any other model) and sends the result with the next request. The gateway itself runs nothing.
+
+**With Copilot and a key-based model both in `llms.json`**, Copilot comes first: a sign-in makes it the default, and without a `"default"` it is the active profile. Before a model is used, ForgeQE checks that it works (its sign-in or key, and one live call, kept for a day once it passes). If Copilot does not work (not signed in, the Copilot SDK missing, the model not on your seat), the other model is used and ForgeQE says why and how to fix Copilot; if a key-based model you made the default does not work, Copilot is tried first. A model that fails its check is never used, and when none works the error lists each one with what to do. `--profile` turns this off: the model you name is used as it is. Each request is billed to the seat (GitHub AI credits or premium requests, by your plan), and GitHub may rate-limit frequent automated requests. If a model is refused, check that your organization's Copilot policies allow it. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
 
 **In pipelines.** `forgeqe suite ... --no-code` uses only the browser engine. Set `COPILOT_GITHUB_TOKEN` to a fine-grained token with the Copilot Requests permission, stored as a pipeline secret. GitHub also documents the workflow's own `GITHUB_TOKEN` with `permissions: copilot-requests: write` for organization repositories whose Copilot policy allows it ([server tokens](https://docs.github.com/en/copilot/how-tos/copilot-sdk/auth/server-to-server-tokens)). A Copilot seat belongs to one person: prefer an organization-billed token over signing a shared pipeline in to someone's seat.
 
@@ -362,9 +365,11 @@ GitHub Models, which ForgeQE used for the engine before, was [retired by GitHub]
 | `GitHub Copilot does not take classic tokens (ghp_)` | Sign in in the browser instead, or make a fine-grained token with the Copilot Requests permission, owned by your personal account. |
 | `GitHub Copilot did not accept the token` | The token has no Copilot Requests permission, has expired, or the account has no Copilot seat. |
 | `… must be a Copilot model id`, `engine_model must be a Copilot model id` | Fix that field in `llms.json`: the sign-in lists the seat's models. A GitHub Models id such as `openai/gpt-4.1` is not a Copilot id; use `gpt-4.1`. |
-| `ForgeQE's agent and Scribe cannot run on a github_copilot profile yet` | Pick another profile for them (`--profile`, or Tab in the shell), or set `code.model`. |
+| `No model in llms.json can … right now` | Each model is listed with why it cannot be used and the fix. `forgeqe doctor` checks everything at once. |
+| `GitHub's Copilot SDK is not installed in the ForgeQE engine` | Run `forgeqe setup`. It installs the SDK from pypi.org; if that is blocked, `forgeqe doctor` says what to ask IT for. |
+| `Model for agent: main, because copilot failed its live check …` | ForgeQE used your other model because Copilot did not answer; the message ends with the fix. |
 | The model is refused, or not found | The seat's plan or your organization's Copilot policies do not allow that model: pick one the sign-in listed, or ask a Copilot admin to enable it. |
-| `GitHub Copilot runs through the ForgeQE engine, which is not set up` | Run `forgeqe setup`. |
+| `GitHub Copilot runs through the ForgeQE engine, which is not set up` (or `could not be set up (…)`) | Run `forgeqe setup`; the reason in brackets says what failed. |
 | `CERTIFICATE_VERIFY_FAILED` while downloading Copilot's runtime | Your network inspects HTTPS. Update ForgeQE and run `forgeqe setup`, then sign in again: it uses the certificates your computer trusts (see [Company networks](#company-networks)). If it still fails, run `forgeqe doctor` and send its output to IT. |
 
 ### Your prompts, skills, and product knowledge
@@ -511,6 +516,8 @@ A card says which it did and why (*Extending PracticeTests.Flow · covers 12 of
 
 **Keeping Scribe's cost down.** ForgeQE reads the repository's test setup itself before Scribe starts (stack, settings files and their browser values, the test id attribute, base classes, page objects and their methods, one test command) and hands it over, so Scribe does not discover it with dozens of file reads. Each attempt after the first starts a fresh session from a summary of the job (handoff, files changed, what each attempt did, the latest failure), so later attempts cost about what the first did. `code.max_attempts` (default 3) and `code.max_tokens` (default 1,500,000; `0` turns it off) cap a job, and the token budget is checked during an attempt too, so one long attempt is stopped when the total passes it. The reviewer sorts its issues into correctness (the test does not test what the goal says, or not reliably) and coding standards (test data or credentials in a page object, naming, hard waits, leftovers). Coding standards never fail a test that works: it is accepted, and the coding agent gets one more round (not counted in `max_attempts`) to fix them; if that round breaks the test, ForgeQE puts back the version that passed and lists the standards issues in the notes (`code.tidy: false` skips the round). A test the reviewer rejects twice for correctness stops the job with its issues, since that usually means the test case and the page disagree. Under each step, the handoff shows that step's part of the page (its section and neighbours, from the replay's page snapshot) instead of whole snapshots. When a test fails, ForgeQE shows the failing element on the test's page at the failure and on the replay's page, so most failures are fixed without opening a browser. Scribe's live browser is switched on only when the same failure repeats, and it does not send a page snapshot after every click or keystroke. `code.max_steps` (default 50) caps the tool calls in one of Scribe's turns. `code.review_profile` runs the reviewer on a cheaper llms.json profile.
 
+**Scribe's rulebook: the same mistake once.** When a job passes after a failed test run or a reviewer's change, Scribe's report gives up to three one-line rules that would have avoided the mistake ("Wait for the cart badge with toHaveText before clicking Checkout; it updates after a request"). ForgeQE keeps them in `.forgeqe/scribe-rules.json` in the repository, and every attempt of later jobs, and every healing session, starts with them. A lesson from a job that did not pass, or that passed at once, is not kept: only a fix that worked becomes a rule. The book stays small, since it is sent with each model call: at most 15 rules of one line each; a lesson learned again is merged into its rule (shown as "learned 3 times") instead of added; when it is full, the rule confirmed least and longest ago goes, and a rule not learned again for 180 days expires. The rules are the coding agent's own words, so a rule with a link, something like a token, a destructive command, or wording that tells an agent to drop its instructions is refused, and the agents are told they are hints that never override their instructions or the team's guardrails. Commit the file to share the rules with the team and see each change in review; edit or delete a rule there (the agents cannot change `.forgeqe`). ForgeQE says each new rule as it learns it (`Learned for next time: …`). `code.rulebook: false` turns it off.
+
 ### Generated code
 
 **Generated code.** What ForgeQE writes is marked, so the team can see it and measure it. A file ForgeQE created starts with one line in that language's comment, such as `// Generated with ForgeQE · fq:k3x7m2qa4bdi · run 20261019-162847`; the `fq:` id is a hash of the file's contents without that line, so it shows whether the file was changed since (`code.header: false` leaves the line out; a JSON file never gets one). Files it only edited get no header. Every file it created or edited is recorded per run in `.forgeqe/generated.jsonl` (commit it with the code): the lines it added, as short hashes, never their text. `forgeqe generated` (or `/generated` in the shell, `--json` for a pipeline) says how many lines ForgeQE wrote, how many are still in the code, its share of the files it touched, and which created files are as it wrote them. Commit messages ForgeQE's agent writes end with a `Generated-by: ForgeQE (fq:…)` trailer, and its pull request descriptions with a line naming the runs' `fq:` ids and how much they wrote. The ids are plain hashes anyone can recompute: they show what ForgeQE wrote, not who owns the code.
@@ -622,7 +629,7 @@ The slash commands control the session and show a few things directly (`/help` l
 
 `start_url` in `forgeqe.yaml` or `--url` pins a start page for browser runs, such as a staging app. Passed runs become tests in the current folder; `forgeqe start --repo ../app` puts them in that repo instead, and `--no-code` turns tests off.
 
-ForgeQE's agent runs on the agent runtime with the active profile's model (or `code.model`), or, with `llm.routing: auto`, the model the router picks for the `agent` area when the session starts (`--profile` and `code.model` still win); it cannot run on a `github_copilot` profile yet. ForgeQE downloads the agent runtime into `~/.forgeqe/tools` when it is not installed. When the agent runtime cannot be downloaded, ForgeQE says the agent is off and runs each message as one browser goal.
+ForgeQE's agent runs on the agent runtime with the active profile's model (or `code.model`), or, with `llm.routing: auto`, the model the router picks for the `agent` area when the session starts (`--profile` and `code.model` still win). A `github_copilot` profile runs it on your Copilot seat. ForgeQE downloads the agent runtime into `~/.forgeqe/tools` when it is not installed. When the agent runtime cannot be downloaded, ForgeQE says the agent is off and runs each message as one browser goal.
 
 While a run is going, the bar under the prompt shows a spinner, the tokens used, and the elapsed time, and the transcript ends with what the agent is doing now (`Thinking about step 3…`, `Acting in the browser (step 3)…`). Each step starts with `+ Thought: 4.2s`, the time the model took to decide. When the run ends, a summary card closes the transcript:
 
@@ -839,6 +846,7 @@ Confidence starts from the locator kind (a test id or role and name ranks above
 How memory saves time and tokens:
 
 - **Replay.** Running a goal that passed before replays its recorded steps directly in the browser, with no model calls. A different goal replays nothing: the agent gets the steps of earlier runs on the site, and the controls that worked there, as hints, and decides itself which of them (a banner, a sign-in, a search form) apply to its task, so a new task does not re-explore the setup. Each step finds its element by the highest-confidence locator that still matches exactly one element, then by the element's fingerprint, so a renamed button or a changed id still resolves. The transcript marks these steps `↻` and shows the locator used (`click "Search"  [role 88%]`).
+- **The site's rulebook.** Page memory keeps what worked; the rulebook keeps what did not. When an action fails during a run and the agent then finds one that works, it may end its answer with a line or two starting `Lesson:` ("On the booking page, type the date in the Check-in field; clicking a day in the calendar does nothing"). ForgeQE keeps a lesson only when that run passed, in `rules/<host>.json` in page memory, and every later run that starts on that site gets the rules, so it does not spend steps on the same dead end. The lesson lines are taken out of the run's summary. The book works like Scribe's rulebook (under Keeping Scribe's cost down): at most 15 one-line rules, merged when learned again, the least confirmed and oldest dropped first, expired after 180 days unconfirmed, and a rule with a link, a token, a destructive command, or wording that drops instructions is refused. ForgeQE says each new rule (`Learned for this site: …`). `browser.rulebook: false` turns it off; with page memory off there is none.
 - **Hand-off.** With `replay: verify` (the default) the agent then looks at the page once and confirms the result. `replay: trust` finishes without the model when every step of this same task replayed. When a step cannot be found, the agent continues from that point instead of starting over. `replay: off` records memory but never replays.
 - **Failed runs.** A run that fails keeps the steps that worked before it went wrong (`resume.json` in its run folder, and in the task's flow). Running the same task again replays them and the agent picks up at the point of failure, told what went wrong last time. When ForgeQE's agent retries a failed goal with different wording, it passes the failed run (`resume_from`), so the retry starts from the same point instead of step 1.
 - **Cut-off runs.** After every step, a run saves the steps that worked so far and the page memory it has learned, so nothing is lost if ForgeQE is closed, the machine restarts, or the run is stopped. Running the same task again replays those steps and the agent carries on from there; the folder of the cut-off run also has `resume.json`, so `resume_from` works with it too. When the model can't be reached (the connection dropped), the run ends as failed and keeps its steps the same way. A run that finishes removes the saved point.
@@ -968,7 +976,7 @@ After ForgeQE's reply, the request ends with the files it produced:
 
 When no test was written, the Test row says why: the run did not pass, the session has `--no-code`, or the run was not handed to the coding agent.
 
-Scribe uses the active `llms.json` profile, or, with `llm.routing: auto`, the profile the router picks for the `coding` area (`--profile` and `code.model` still win; see [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md)). The key stays in your environment: the agent runtime's config refers to it as `{env:WP_AZURE_API_KEY}` and never contains the value. An `auth: entra` profile reaches the model through ForgeQE's local token gateway (see [Signing in to the model without a key](#signing-in-to-the-model-without-a-key-microsoft-entra-id)). An Azure profile goes through the agent runtime's `azure` provider on `<endpoint>/openai/v1` (the Responses API, with the deployment as the model). `openai_compatible` and `ollama` profiles use their `base_url`. A `github_copilot` profile cannot run Scribe yet, and `code.model: github-copilot/...` is refused: the agent runtime's own Copilot provider signs in through a third-party app on your GitHub account. `code.model: provider/model` (or `--code-model`) uses one of the agent runtime's built-in providers instead, with that provider's usual key variable (for example `ANTHROPIC_API_KEY` for `anthropic/...`).
+Scribe uses the active `llms.json` profile, or, with `llm.routing: auto`, the profile the router picks for the `coding` area (`--profile` and `code.model` still win; see [docs/MODEL_ROUTING.md](docs/MODEL_ROUTING.md)). The key stays in your environment: the agent runtime's config refers to it as `{env:WP_AZURE_API_KEY}` and never contains the value. An `auth: entra` profile reaches the model through ForgeQE's local token gateway (see [Signing in to the model without a key](#signing-in-to-the-model-without-a-key-microsoft-entra-id)). An Azure profile goes through the agent runtime's `azure` provider on `<endpoint>/openai/v1` (the Responses API, with the deployment as the model). `openai_compatible` and `ollama` profiles use their `base_url`. A `github_copilot` profile runs Scribe through ForgeQE's Copilot gateway (see [GitHub Copilot seats as the model](#github-copilot-seats-as-the-model)). `code.model: github-copilot/...` is refused: the agent runtime's own Copilot provider signs in through a third-party app on your GitHub account. `code.model: provider/model` (or `--code-model`) uses one of the agent runtime's built-in providers instead, with that provider's usual key variable (for example `ANTHROPIC_API_KEY` for `anthropic/...`).
 
 Scribe may edit files and run commands anywhere in the repo without asking. `git push` in any form is denied (pushing goes through ForgeQE's guarded `push_branch` tool, section 8), and it cannot touch files outside the repo except the run and memory directories, which it may read.