@capagents/forgeqe 2.3.13 → 2.3.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -387,7 +387,7 @@ Before the model is called, `replay_known` asks `AppMemory.plan` for the goal's
387
387
 
388
388
  - **API scenario plans** (`src/api/plancache.ts`): `api_scenarios` from cases or text keeps each plan under `<memory dir>/api/plans/<hash>.json`, keyed by a hash of the cases' titles, descriptions, preconditions, steps and expected results (or the text) and the set's operations. The same input reuses the plan with no model call, replacing its earlier scenarios in the set; `fresh: true` plans again.
389
389
  - **Registry plans of every tool** (`src/registry`): `indexRuns` also indexes mobile and desktop explores (tool `mobile_explore` / `desktop_explore`, keyed by goal and app) and API runs (`api/<set>/runs/*`: `run_api_contracts` / `run_api_scenarios`, known as "the API contracts" of a set). `plans.args` holds the tool's arguments (added in place; schema version stays 1, so older versions still open the file). `runTier0` (`src/registry/run.ts`) calls the plan's own tool and writes the reply from its result; the tools still ask before they operate a device or change data. The shell and `forgeqe run` both route through it.
390
- - **Measurement:** `forgeqe memory stats` (`src/memorystats.ts`) counts, per area, the runs that reused history, those that needed no model, steps replayed and model steps, the API plans kept, and the requests the registry answered without the agent. The **memory gate** workflow (`.github/workflows/forgeqe-memory-gate.yml`) runs the tests that fail when known work reaches the model.
390
+ - **Measurement:** `forgeqe memory stats` (`src/memorystats.ts`) counts, per area, the runs that reused history, those that needed no model, steps replayed and model steps, the API plans kept, and the requests the registry answered without the agent. The **memory gate** (the `gate` job of `.github/workflows/ci.yml`) runs the tests that fail when known work reaches the model.
391
391
 
392
392
  ## Containers (`src/containers`)
393
393
 
@@ -544,5 +544,5 @@ That is the active profile. With `llm.routing: auto`, each area of work can run
544
544
  - `tests/heal.test.ts`: the healing loop with a scripted session that calls ForgeQE's tools over the real MCP server (only the four heal tools are listed; the browser run lands in the heal folder): nothing to heal, healed and confirmed, an app bug left failing, a healed test that still fails or has no command, finding the command, `code.test_command`, a missing report, a fatal error, and stop. `tests/monitor.test.ts`: `decide` sequences (fail_after, flaky, reminders, recovery, stopped, a round that could not run), duration and `monitor:` parsing, rounds through the real `runSuite` with state carried over, report links, attachments, `alerts.json`, pruning, the loop and `--once` exit codes, the lock against a live pid, the Teams card against a fake webhook, email through nodemailer to a local fake SMTP server (login, recipients, subject, screenshot), and secrets scrubbed from errors. `tests/cli.test.ts` runs `forgeqe monitor --test-alerts` and `--once` with the mock profile against a fake Teams webhook.
545
545
  - `tests/commands.test.ts`: the command tools called as the agent calls them: `write_test` with a scripted session (agent events, the card, the mock profile refused), `heal_tests` clean and healed, `monitor` round and status, `start` refused in a `once` session, start/status/stop in a `shell` session (a round card, a second start and a one-off round refused, the lock released), `test_alerts` against a fake webhook, `logins` list/clear and `sign_in` without a person, `page_memory` flows and clear, and `forgeqe_setup` profiles, MCP servers, docs by topic, and init. `tests/repotests.test.ts`: `parseResults` on JUnit and TRX; `find_tests` with a scripted Scribe (the read-only agent, the listing fed back, a corrected list command listed again, nothing found, mock profile and bad dates refused); `run_tests` declined (nothing runs), confirmed (per-test card from JUnit, the heal suggestion), and without `ask` (`needs_confirmation`, then `confirmed: true`, exit code only); sprints from fake Azure DevOps and Jira APIs (paging, the pick through `ask`, cancel, `needs_choice`, a name matched). `tests/engine/test_step_notes.py`: the agent's note built from its own fields and stored on the flow steps a step recorded, `evaluate` steps with their code. `tests/code.test.ts` (handoff): notes before each group of steps, once per step, and the JavaScript shown. `tests/update.test.ts`: bun and pip installs found in PATH order; waiting while the registry 404s, then the cache-skipping commands per install; `--check` running nothing; a release never served failing with a retry hint. `tests/sessionLog.test.ts`: the session header and events with remembered passwords, secret environment variables, and inline API keys replaced; tool calls and results; the bundle's contents without screenshots or keys; `/logs`. `tests/engine/test_page_settle.py`: the judge off by default and on by setting, `page_settled`, `settle_page` waiting for a new tab and giving up on one that stays blank, and note actions as read-only. `tests/resume.test.ts`: resume steps in the run result, `loadResume` by folder or run name, and `browser_run` returning the retry and passing `resume_from`. `tests/engine/test_resume.py`: `resume_point`, `resume_steps`, the loop start and nudge text, `_install_nudges` draining after the library's hook, resume steps in the flow file, the resume hand-off, checkpoints (kept apart from flows, cleared only by their own run, kept by a stopped run), and the hand-off after a cut-off run. `tests/code.test.ts` (resuming): a stopped job keeps `scribe-session.json` and a finished one removes it, a rerun reopens the session with the saved replay and a carry-on prompt, and a gone session starts fresh; `tests/code.e2e.test.ts` reopens a session by id after the runtime restarted and finds its earlier messages. `tests/engine.e2e.test.ts` runs a page where clicking "One way" resets the date: a model that takes the nudge finishes; one that ignores it is stopped with its resume steps saved, and a retry with `resume_from` replays the cookie click and needs two model calls. `tests/sessionLogins.test.ts`: remembered logins (domains from a URL, hosts, and wildcards; the engine payload with values; no process env), redaction next to punctuation and in carried-over turns, a remembered login replacing a configured one in the notes, ForgeQE's test runs getting the env names with the output redacted, and the `logins` tool's remember/list/clear without the password in any result. `tests/engine/test_session_logins.py`: remembered values as placeholders on every sign-in domain, env values still working, `step_outcomes`, and `making_progress`. `tests/tui.test.tsx` renders a monitor round's card in the transcript and checks the **Your turn** dialog holds a tool until **Done, continue**.
546
546
  - `tests/skills.test.ts`: `skills:` parsing; the prebuilt skills loading cleanly; precedence and `overrides` across user, project, the repo's own folder, and `dirs`; the repo's `.claude` skills only with `external`; the default and opt-in `agents`; `disable` and unknown names; every validation problem (including the unquoted `: ` hint); the per-agent `skill` permission and `skills.paths` in the runtime config; links readable through and removed without touching the skill; `browserSkills` given and choose, unknown names, and the remembered choice; the engine request through a stub engine and `skills` back on the result; the skills catalogue in the environment notes; `newSkill`, `setSkillDisabled` keeping comments, and `ignoredNote`; the overview and `/skills`; and `forgeqe skills new/show/disable/enable/check` end to end. `tests/code.e2e.test.ts` checks, against the real agent runtime, that Scribe's skill listing has only the registry's skills for it, an allowed skill's text arrives, and a disabled skill, the repo's `.opencode` and `.claude` skills, the runtime's built-in skill, and `~/.claude/skills` stay out unless `external` adds them through the registry. `tests/engine/test_skills.py`: `choose_skills` order, the cap of 3, errors and an empty catalogue without a model call; `skills_block` within `skills_max_chars`; the request fields.
547
- - `tests/desktop.test.ts`, `tests/mobile.test.ts`, `tests/scope.test.ts`, `tests/capabilities.test.ts`: the desktop and mobile commands and tools, workflow edits, uploads, and the tool scope. `<engine-python> -m pytest tests/desktop`: the desktop and mobile engines against fake drivers, a fake Appium server and a stand-in Maestro (any OS). The `*_live` tests need Windows or an emulator, and run in `.github/workflows/forgeqe-desktop-windows.yml` and `forgeqe-mobile-android.yml`.
547
+ - `tests/desktop.test.ts`, `tests/mobile.test.ts`, `tests/scope.test.ts`, `tests/capabilities.test.ts`: the desktop and mobile commands and tools, workflow edits, uploads, and the tool scope. `<engine-python> -m pytest tests/desktop`: the desktop and mobile engines against fake drivers, a fake Appium server and a stand-in Maestro (any OS). The `*_live` tests need Windows or an emulator, and run in the `desktop` and `android` jobs of `.github/workflows/ci.yml`.
548
548
  - `<engine-python> -m pytest tests/engine`: the engine directly, with a scripted model, including `test_memory.py` (locators, confidence, identity, flows, secrets) and replay, trust mode, and self-healing against real Chrome. `test_brand.py` checks the overlay in real Chrome: the start page, trusted CDP clicks and keys dropped while locked and let through while acting, the unlocked state and step text on a page loaded mid-step, and a clean screenshot and DOM for the model. `WP_E2E_HEADED=1` runs the bun e2e test in a visible browser, with the overlay on.
@@ -8,7 +8,7 @@ real Docker (and is skipped without it):
8
8
  - A tests container that reaches the database by name and writes its results into the repository as this machine's
9
9
  user.
10
10
 
11
- `.github/workflows/forgeqe-containers.yml` runs both on GitHub's Docker. There, a skipped live test fails the job.
11
+ The `containers` job of `.github/workflows/ci.yml` runs both on GitHub's Docker. There, a skipped live test fails the job.
12
12
 
13
13
  ForgeQE uses Docker or Podman, when one is installed, for five jobs:
14
14
 
package/docs/DESKTOP.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # ForgeQE desktop: thick-client automation
2
2
 
3
- Status: runs on Windows. The CI job `.github/workflows/forgeqe-desktop-windows.yml` (GitHub's `windows-2022`) installs the
3
+ Status: runs on Windows. The `desktop` job of `.github/workflows/ci.yml` (GitHub's `windows-2022`) installs the
4
4
  engine and the pinned UFO², replays Notepad live with the pywinauto driver and through `forgeqe desktop replay`, heals a
5
5
  moved control from the live window, and runs a whole explore: UFO² drives Notepad with a scripted model behind an
6
6
  OpenAI-compatible server, the action is recorded with the control's AutomationId, and the recording replays. Not yet
@@ -272,7 +272,7 @@ nightly workflow (`ci/desktop-runner/desktop-nightly.example.yml`), and `--junit
272
272
 
273
273
  UFO² is pinned to one commit. Moving it is checked before it is pinned:
274
274
 
275
- 1. `.github/workflows/forgeqe-desktop-ufo-update.yml` runs weekly, or by hand with a commit. It sets up that UFO on
275
+ 1. The `ufo-update` job of `.github/workflows/ci.yml` runs weekly, or by hand (Run workflow, job `ufo-update`) with a commit. It sets up that UFO on
276
276
  Windows (`FORGEQE_UFO_COMMIT`, which takes a full 40-character commit id only), runs the adapter tests against its
277
277
  real code and a scripted explore of Notepad, and says in the job summary whether it passed.
278
278
  2. When it passed: `bun scripts/ufo-bump.ts <commit>` writes the new pin into `src/desktop/setup.ts`,
package/docs/MOBILE.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # ForgeQE mobile: Android and iOS apps
2
2
 
3
3
  Status: the engine and its tests run on any OS, against a fake Appium server and a stand-in for Maestro. The CI job
4
- `.github/workflows/forgeqe-mobile-android.yml` runs it on an Android emulator: Appium explores and replays the
4
+ The `android` job of `.github/workflows/ci.yml` runs it on an Android emulator: Appium explores and replays the
5
5
  emulator's Settings app, and Maestro replays the same workflow. iOS runs the same code with the XCUITest driver, but
6
6
  no CI job tests it yet (it needs a macOS runner with a simulator).
7
7
 
@@ -193,7 +193,7 @@ mobile:
193
193
  the value in its own vault). Replay it on TestMu with `--driver appium`.
194
194
  - The access key is passed to the CLI as an argument. It is removed from the CLI's output and from every file in the
195
195
  run folder. Neither the user name nor the key is passed to the flows' environment.
196
- - `.github/workflows/forgeqe-mobile-testmu.yml` runs all three on TestMu (Appium, Maestro with a workflow, a team flow)
196
+ - The `testmu` job of `.github/workflows/ci.yml` (Run workflow, job `testmu`) runs all three on TestMu (Appium, Maestro with a workflow, a team flow)
197
197
  with TestMu's Wikipedia sample app, once the repository has the secrets `LT_USERNAME` and `LT_ACCESS_KEY`.
198
198
 
199
199
  ## Security
@@ -719,19 +719,22 @@ Add a GitHub Actions workflow that runs tests/cases.yaml on every PR, and make i
719
719
  <tr><td><code>/stop</code></td><td>Stop what is running (or press Esc)</td></tr>
720
720
  <tr><td><code>/rerun [last|run]</code></td><td>Run a passed run again as recorded, without ForgeQE's agent (the last one, a run folder, or its goal)</td></tr>
721
721
  <tr><td><code>/headed on|off</code></td><td>Show the browser window (live profiles)</td></tr>
722
+ <tr><td><code>/theme light|dark|auto</code></td><td>Colours for a light or a dark terminal. ForgeQE asks the terminal which it is when it starts and follows it when it changes (auto); a terminal that does not say is drawn dark. <code>FORGEQE_THEME=light</code> (or <code>dark</code>) sets it at start; <code>/theme</code> alone says which is in use</td></tr>
722
723
  <tr><td><code>/details [on|off]</code></td><td>Show everything the agents do while they work: the browser agent's reasoning, page memory, engine logs, every file the coding agent reads (off by default: step results, files written, test results, problems). <code>--verbose</code> or <code>FORGEQE_VERBOSE=1</code> starts with it on, for <code>start</code>, <code>run</code>, <code>suite</code>, <code>code</code>, and <code>heal</code> too</td></tr>
723
724
  <tr><td><code>/model</code></td><td>Pick the model from the profiles in <code>llms.json</code> (or click the model ▾ in the status line; Tab switches to the next one)</td></tr>
724
725
  <tr><td><code>/profile &lt;name&gt;</code></td><td>Switch LLM profile (or press Tab); the agent restarts with it</td></tr>
725
726
  <tr><td><code>/env [name]</code></td><td>The test environments in <code>env.yaml</code> and their apps, or switch to one</td></tr>
726
727
  <tr><td><code>/logs [shots]</code></td><td>Save this session's log and runs in one archive to share (<code>shots</code>: with screenshots)</td></tr>
727
728
  <tr><td><code>/features</code></td><td>What ForgeQE can do, and what each capability still needs here</td></tr>
728
- <tr><td><code>/runs</code>, <code>/reports</code></td><td>The runs and reports in the reports folder, newest first</td></tr>
729
- <tr><td><code>/jobs</code></td><td>The scheduled jobs: when each runs next, and how its last run ended</td></tr>
730
- <tr><td><code>/knowledge</code></td><td>The product knowledge the agents read (<code>.forgeqe/knowledge</code>)</td></tr>
731
- <tr><td><code>/memory</code></td><td>Page memory: the sites and pages ForgeQE remembers, and their flows</td></tr>
732
- <tr><td><code>/guardrails</code></td><td>The rules every agent follows here</td></tr>
733
- <tr><td><code>/doctor</code></td><td>Check what ForgeQE needs: programs, sign-ins, and the services it reaches</td></tr>
734
- <tr><td><code>/settings</code></td><td><code>forgeqe.yaml</code>: where it is and what it sets, opened to edit</td></tr>
729
+ <tr><td><code>/runs</code></td><td>The Runs page: every run with its result, steps and what page memory replayed; the week as a chart; click a run (or ↑↓) to see how it ended and every step it took: by page memory or the model, the page, what it did, how long it took, and the step that failed with its error. enter opens its report, f its folder, r runs it again</td></tr>
730
+ <tr><td><code>/reports</code></td><td>The Reports page: every report ForgeQE wrote here (suites, pipelines, <code>/report</code>), newest first, each with <strong>open ↗</strong> to view it in the browser and a short summary of the chosen one. / searches by title, kind and summary (esc clears). A single run's report is on the Runs page. f opens its folder, b builds a new report</td></tr>
731
+ <tr><td><code>/jobs</code></td><td>The Jobs screen: every scheduled job with its next run, last result and recent runs; the week's runs as a chart; the chosen job's runs and output. ↑↓ choose, r runs now, p pauses or resumes, s stops, l opens the log, esc goes back</td></tr>
732
+ <tr><td><code>/knowledge</code></td><td>The Knowledge page: what ForgeQE has kept here as it works (run history, page memory, defect history, the code it wrote and how much is still there, product knowledge, visual baselines, watched pipelines), each with its entries</td></tr>
733
+ <tr><td><code>/skills</code></td><td>The Skills page: every skill with the agents that use it and where it comes from; enter or <strong>open ↗</strong> opens its SKILL.md (<code>/skills &lt;name&gt;</code> shows one in the chat)</td></tr>
734
+ <tr><td><code>/memory</code></td><td>The Page memory page: each site with its pages, elements and flows; the chosen site's pages and flows</td></tr>
735
+ <tr><td><code>/guardrails</code></td><td>The Guardrails page: every rule and the file it is in. o opens the file to edit</td></tr>
736
+ <tr><td><code>/doctor</code></td><td>The Doctor page: what ForgeQE needs on this machine, with the fix for what is missing. r also checks the model, the trackers and every service ForgeQE reaches</td></tr>
737
+ <tr><td><code>/settings</code></td><td>The Settings page: the model, config file, trackers, environments, folders, domain and notifications, each opening where it is changed</td></tr>
735
738
  <tr><td><code>/home</code></td><td>Back to the start screen; the session stays</td></tr>
736
739
  <tr><td><code>/docs</code></td><td>The user guide</td></tr>
737
740
  <tr><td><code>/generated</code></td><td>How much of this repository's code ForgeQE wrote, and how much is still there</td></tr>
@@ -833,7 +836,7 @@ Add a GitHub Actions workflow that runs tests/cases.yaml on every PR, and make i
833
836
  </table></div>
834
837
  <p><strong>Links.</strong> Report folders, report files, and web addresses in the chat are links: Cmd-click (Ctrl-click on Windows and Linux) opens them in terminals that support links, such as VS Code, iTerm2, Windows Terminal, and recent GNOME Terminal. A path that no longer exists stays plain text.</p>
835
838
  <p><strong>Setup checklist.</strong> Until everything is set up, the start screen shows how many of the five things most features rest on are set up: <strong>Model</strong> (a live model profile), <strong>Coding agent</strong>, <strong>Work items</strong> (Azure DevOps, Jira, TestRail, Zephyr, qTest, or Xray), <strong>Domain</strong>, and <strong>Notifications</strong>. In a terminal 110 columns or wider it is the sidebar's <strong>Get set up</strong>: each finished item with what it found, and the next one as a card with <strong>▸ Set up</strong> (its steps in the chat; for <strong>Work items</strong>, it runs <code>/auth</code>) and <strong>skip</strong>. A skipped item is kept in <code>.forgeqe/setup.json</code> and not asked about again. Narrower, it is one <strong>Setup</strong> line with <strong>▸ how to set up</strong> and <strong>skip</strong>. The checks are the same ones <code>/features</code> uses, so the two always agree.</p>
836
- <p><strong>Sidebar and rail.</strong> The sidebar has three groups. <strong>Work</strong>: <strong>Home</strong>, <strong>This session</strong> (once you have asked something), <strong>Runs</strong>, <strong>Reports</strong>, <strong>Jobs</strong>, and <strong>Defects</strong> (with a tracker set up). <strong>Knowledge</strong>: <strong>Page memory</strong>, <strong>Test cases</strong> (with a tracker), <strong>Guardrails</strong>, <strong>Product</strong> (the knowledge files) and <strong>Skills</strong>. <strong>System</strong>: <strong>Models</strong>, <strong>Sign in</strong>, <strong>Environments</strong>, <strong>Doctor</strong>, <strong>Settings</strong>, <strong>Help</strong> and <strong>Docs</strong>. Each shows how many there are and opens its list (<code>/runs</code>, <code>/reports</code>, <code>/jobs</code>, <code>/memory</code>, <code>/guardrails</code>, <code>/knowledge</code>, <code>/skills</code>, <code>/doctor</code>, <code>/settings</code>, <code>/help</code>, <code>/docs</code>); <strong>Defects</strong> and <strong>Test cases</strong> put a request in the composer, since the agent reads them from the tracker. Narrower than 110 columns, and beside the chat at every width, the sidebar folds to a rail of its icons, so the start screen is always one click away: <strong>Home</strong> (or <code>/home</code>) shows it with the session still open, and <strong>▸ Back to the session</strong> or any request returns to it.</p>
839
+ <p><strong>Sidebar and rail.</strong> The sidebar has three short groups. <strong>Work</strong>: <strong>Home</strong>, <strong>This session</strong> (once you have asked something), <strong>Runs</strong>, <strong>Reports</strong> and <strong>Jobs</strong>. <strong>Knowledge</strong>: <strong>Knowledge</strong> (history and what ForgeQE has kept), <strong>Page memory</strong>, <strong>Test cases</strong>, <strong>Guardrails</strong> and <strong>Skills</strong>. <strong>System</strong>: <strong>Settings</strong> and <strong>Doctor</strong>. Each opens a page in the chat's place, laid out for what it shows, with its keys at the foot: tables to move through with ↑↓, and <strong>open ↗</strong> links that open a report or a skill in the browser. <strong>Test cases</strong> lives in your tracker, so its page offers the requests that fetch it and puts the one you choose in the composer to read before sending, beside the suite files in the repository. esc goes back. Models, sign-ins and environments are a command away (<code>/model</code>, <code>/auth</code>, <code>/env</code>; <code>/help</code> lists them all), and <strong>? help</strong> and <strong>docs ↗</strong> are at the top right of the start screen. Beside the chat the sidebar is a column with its labels from 194 columns, and a rail of its icons below that, so the start screen is always one click away: <strong>Home</strong> (or <code>/home</code>) shows it with the session still open, and <strong>▸ Back to the session</strong> or any request returns to it.</p>
837
840
  <p><strong>Needs you and Today.</strong> Once there are runs, the start screen shows what needs you (runs that failed this week and have not passed since, and jobs whose last run failed, each with <strong>open</strong> and <strong>rerun</strong>) and what ran today. The greeting says how many runs there were today and how many failed. The model is in the composer beside <strong>⏎ Send</strong>: click it to change it. The status line shows the branch, the tracker, and when the next scheduled job runs.</p>
838
841
  <p><strong>Command palette.</strong> Ctrl-P opens a palette of every action, command, profile, and example request; type a few letters to find one (&quot;rr&quot; finds <em>Refine my request</em>, &quot;auth&quot; finds the sign-in commands), ↑/↓ to choose, Enter to run it. Example requests are put in the prompt for you to edit, never sent straight away.</p>
839
842
  <p><strong><code>/</code> and <code>@</code> in the prompt.</strong> Typing <code>/</code> lists the commands that match; typing <code>@</code> suggests references: recent runs (<em>last run</em>, then each by its folder), files in the repository by name or path (<code>@Login</code> finds <code>pages/Login.ts</code>), and, after <code>@</code> and a number such as <code>@1761</code>, the Azure DevOps work item with its title (read-only). Tab completes, ↑/↓ choose, Esc closes the list.</p>
@@ -895,6 +898,69 @@ forgeqe run &quot;Run the cases in smoke.yaml and don't post results&quot; --pla
895
898
  <li><code>failure.png</code>: last screenshot, only when the run failed</li>
896
899
  </ul>
897
900
  </div>
901
+ <section class="doc sub" id="doc-how_to_use-a-live-agent-for-programs-forgeqe-chat---jsonl"><h4>A live agent for programs (<code>forgeqe chat --jsonl</code>)</h4><div class="prose"><p>A program that needs what the shell has (one live agent per conversation, follow-ups, credentials remembered for the session, <strong>Your turn</strong> and questions, stop), such as a web UI or an IDE, starts <code>forgeqe chat --jsonl</code> and talks to it in JSON lines. It is the shell's agent and the shell's events, not a screen to scrape. ForgeQE serves no HTTP here: the program that starts the process owns transport, users, and sign-in.</p>
902
+ <pre><code class="language-bash">forgeqe chat --jsonl [--repo DIR | --no-code] [--profile NAME] [--url URL] [--headed]
903
+ [--browser-provider local|docker|browserstack|testmu] [--devices a,b] [--max-steps N]
904
+ [--replay MODE] [--no-memory] [--carry-over FILE] [--config forgeqe.yaml] [--verbose] [--debug]
905
+ </code></pre>
906
+ <p>The flags mean what they mean for <code>forgeqe run</code> and <code>forgeqe start</code>. <code>--jsonl</code> is required (the only format so far). <code>--carry-over FILE</code> is a JSON list <code>[{&quot;user&quot;: &quot;...&quot;, &quot;reply&quot;: &quot;...&quot;}, ...]</code>: the turns of an earlier conversation whose process has gone (a server closing idle chats). The first message then reaches the agent with those turns before it, as when you switch profiles in the shell. To change profiles, start a new process with <code>--profile</code> and <code>--carry-over</code>. <code>--verbose</code> also writes readable event lines to stderr.</p>
907
+ <p>The process opens ForgeQE's agent on the first message and keeps it for every later one; it ends it on <code>close</code>, the end of stdin, or a signal. It works like the shell: background monitors are allowed and stop when it exits, and passed runs become tests unless <code>--no-code</code>. A request that was run before and passed is run again as recorded, without the agent, as in the shell (Tier 0, <code>registry.mode</code>); its turn ends with the result as the reply. When the agent cannot run (no model, no agent runtime), the process writes <code>fatal</code> and exits 1 before it reads anything: it never falls back to a browser goal.</p>
908
+ <p><strong>stdout</strong> carries only JSON lines: one object per line, UTF-8, ending in <code>\n</code>. Logs, warnings, and anything a library prints go to stderr. Lines can be long (a card, a suite result); do not assume a maximum length. Every line has <code>&quot;v&quot;: 1</code>, <code>&quot;seq&quot;</code> (1, 2, 3, … in this process), and <code>&quot;t&quot;</code> (ISO time). Every string is passed through the session's secret redaction, so a password given in a message never comes back out, not even inside a tool result; profile keys and the values of ForgeQE's own secret variables never appear.</p>
909
+ <p><strong>Input</strong> (stdin), one JSON object per line:</p>
910
+ <div class="tbl"><table>
911
+ <thead>
912
+ <tr><th><code>type</code></th><th>Fields</th><th>Meaning</th></tr>
913
+ </thead>
914
+ <tbody>
915
+ <tr><td><code>message</code></td><td><code>id</code> (your turn id), <code>text</code></td><td>A new user message. Refused with <code>error</code> <code>busy</code> while a turn runs</td></tr>
916
+ <tr><td><code>answer</code></td><td><code>id</code> (from an <code>ask</code>), <code>choice</code> (an option id, or <code>null</code> to cancel)</td><td>Answers the pending question</td></tr>
917
+ <tr><td><code>continue</code></td><td><code>id</code> (from a <code>wait</code>), <code>ok</code> (<code>true</code> = Done, continue; <code>false</code> = Cancel)</td><td>Ends the pending wait</td></tr>
918
+ <tr><td><code>stop</code></td><td></td><td>Stops the running turn (Esc or <code>/stop</code> in the shell). Nothing when idle</td></tr>
919
+ <tr><td><code>close</code></td><td></td><td>Stops any turn, closes the agent, stops monitors and the containers it started, writes <code>closed</code>, exits 0</td></tr>
920
+ </tbody>
921
+ </table></div>
922
+ <p>A line that is not JSON, has an unknown <code>type</code>, or misses a field gets <code>error</code> <code>bad_input</code> with the reason and the first 200 characters of the line; the process goes on. The end of stdin is the same as <code>close</code>.</p>
923
+ <p><strong>Output</strong> (stdout):</p>
924
+ <div class="tbl"><table>
925
+ <thead>
926
+ <tr><th><code>type</code></th><th>Fields</th><th>When</th></tr>
927
+ </thead>
928
+ <tbody>
929
+ <tr><td><code>ready</code></td><td><code>protocol</code> (1), <code>version</code>, <code>cwd</code>, <code>writeTests</code>, <code>profile</code> (<code>name</code> and <code>provider</code>, never a key)</td><td>Once, after the config is loaded and checked, before stdin is read</td></tr>
930
+ <tr><td><code>turn_start</code></td><td><code>id</code></td><td>A message was accepted</td></tr>
931
+ <tr><td><code>agent</code></td><td><code>id</code>, <code>event</code>: a <code>CodeEvent</code> (<code>stream</code>, <code>say</code>, <code>start</code>, <code>tool</code>, <code>plan</code>, <code>usage</code>, <code>context</code>, <code>test</code>, <code>info</code>, …)</td><td>While the turn runs</td></tr>
932
+ <tr><td><code>tool</code></td><td><code>id</code>, <code>event</code>: a <code>ToolEvent</code> (<code>run-start</code>, <code>run-event</code>, <code>run-done</code>, <code>suite-*</code>, <code>card</code>, <code>info</code>, <code>agent</code>, <code>code-*</code>, …)</td><td>While the turn runs; <code>id</code> is <code>null</code> for a background monitor between turns</td></tr>
933
+ <tr><td><code>ask</code></td><td><code>id</code> (<code>ask-N</code>), <code>turn</code>, <code>question</code> (<code>title</code>, <code>detail</code>, <code>lines</code>, <code>options[{id, label}]</code>)</td><td>A tool asks you to choose</td></tr>
934
+ <tr><td><code>wait</code></td><td><code>id</code> (<code>wait-N</code>), <code>turn</code>, <code>prompt</code></td><td>A tool waits for you to do something by hand</td></tr>
935
+ <tr><td><code>turn_done</code></td><td><code>id</code>, <code>outcome</code> (<code>done</code>, <code>failed</code>, <code>needs_input</code>, <code>stopped</code>), <code>reply</code> (the visible reply), <code>next</code> (follow-ups), <code>error</code> (when there is one), <code>usage</code> (the agent's usage so far, <code>null</code> before it opened)</td><td>The turn ended, whatever the way</td></tr>
936
+ <tr><td><code>error</code></td><td><code>code</code> (<code>bad_input</code>, <code>busy</code>, <code>unknown_id</code>, <code>internal</code>), <code>message</code>, <code>turn</code> (when one runs)</td><td>A problem that does not end the process</td></tr>
937
+ <tr><td><code>fatal</code></td><td><code>message</code></td><td>The process cannot go on; exit code 1 follows</td></tr>
938
+ <tr><td><code>closed</code></td><td></td><td>The last line before a clean exit</td></tr>
939
+ </tbody>
940
+ </table></div>
941
+ <p><code>CodeEvent</code> and <code>ToolEvent</code> are passed through as they are: their types in <code>src/types.ts</code> and <code>src/mcp/tools.ts</code> are the contract. Rules:</p>
942
+ <ul>
943
+ <li>Every turn that got <code>turn_start</code> gets exactly one <code>turn_done</code>, also after an error (<code>outcome: &quot;failed&quot;</code> with the <code>error</code>).</li>
944
+ <li>One <code>ask</code> and one <code>wait</code> can be pending at a time. <code>answer</code> or <code>continue</code> with an id that is not pending gets <code>error</code> <code>unknown_id</code>, and nothing else happens.</li>
945
+ <li><code>stop</code> and <code>close</code> answer a pending <code>ask</code> with <code>null</code> and a pending <code>wait</code> with <code>false</code>, then stop the turn, which ends with <code>turn_done</code> <code>stopped</code>.</li>
946
+ <li>If the agent fails to start on a message, that turn ends <code>failed</code> with the error, and the next message tries again.</li>
947
+ <li>Exit codes: 0 after <code>closed</code>, 1 after <code>fatal</code>, 2 after SIGINT or SIGTERM (the turn is stopped and the agent closed first).</li>
948
+ </ul>
949
+ <p>A complete exchange (<code>→</code> stdout, <code>←</code> stdin; <code>t</code> left out):</p>
950
+ <pre><code class="language-text">→ {&quot;v&quot;:1,&quot;seq&quot;:1,&quot;type&quot;:&quot;ready&quot;,&quot;protocol&quot;:1,&quot;version&quot;:&quot;2.3.11&quot;,&quot;cwd&quot;:&quot;/srv/studio/users/ana/workspace&quot;,&quot;writeTests&quot;:true,&quot;profile&quot;:{&quot;name&quot;:&quot;copilot&quot;,&quot;provider&quot;:&quot;github_copilot&quot;}}
951
+ ← {&quot;type&quot;:&quot;message&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;text&quot;:&quot;Run SHOP-12 on staging&quot;}
952
+ → {&quot;v&quot;:1,&quot;seq&quot;:2,&quot;type&quot;:&quot;turn_start&quot;,&quot;id&quot;:&quot;m1&quot;}
953
+ → {&quot;v&quot;:1,&quot;seq&quot;:3,&quot;type&quot;:&quot;agent&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;event&quot;:{&quot;type&quot;:&quot;code&quot;,&quot;kind&quot;:&quot;stream&quot;,&quot;id&quot;:&quot;p1&quot;,&quot;text&quot;:&quot;I'll run SHOP-12&quot;}}
954
+ → {&quot;v&quot;:1,&quot;seq&quot;:4,&quot;type&quot;:&quot;agent&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;event&quot;:{&quot;type&quot;:&quot;code&quot;,&quot;kind&quot;:&quot;start&quot;,&quot;text&quot;:&quot;forgeqe_run_test_cases&quot;,&quot;tool&quot;:&quot;forgeqe_run_test_cases&quot;,&quot;id&quot;:&quot;c1&quot;}}
955
+ → {&quot;v&quot;:1,&quot;seq&quot;:5,&quot;type&quot;:&quot;tool&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;event&quot;:{&quot;type&quot;:&quot;run-start&quot;,&quot;goal&quot;:&quot;SHOP-12: Checkout with a saved card&quot;}}
956
+ → {&quot;v&quot;:1,&quot;seq&quot;:6,&quot;type&quot;:&quot;ask&quot;,&quot;id&quot;:&quot;ask-1&quot;,&quot;turn&quot;:&quot;m1&quot;,&quot;question&quot;:{&quot;title&quot;:&quot;Which sprint?&quot;,&quot;options&quot;:[{&quot;id&quot;:&quot;s41&quot;,&quot;label&quot;:&quot;Sprint 41&quot;},{&quot;id&quot;:&quot;s42&quot;,&quot;label&quot;:&quot;Sprint 42&quot;}]}}
957
+ ← {&quot;type&quot;:&quot;answer&quot;,&quot;id&quot;:&quot;ask-1&quot;,&quot;choice&quot;:&quot;s42&quot;}
958
+ → {&quot;v&quot;:1,&quot;seq&quot;:7,&quot;type&quot;:&quot;tool&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;event&quot;:{&quot;type&quot;:&quot;card&quot;,&quot;card&quot;:{&quot;status&quot;:&quot;passed&quot;,&quot;title&quot;:&quot;SHOP-12&quot;,&quot;stats&quot;:[],&quot;body&quot;:&quot;Checkout with a saved card passed in 41 s.&quot;}}}
959
+ → {&quot;v&quot;:1,&quot;seq&quot;:8,&quot;type&quot;:&quot;turn_done&quot;,&quot;id&quot;:&quot;m1&quot;,&quot;outcome&quot;:&quot;done&quot;,&quot;reply&quot;:&quot;SHOP-12 passed on staging.&quot;,&quot;next&quot;:[{&quot;label&quot;:&quot;Write a test for it&quot;,&quot;request&quot;:&quot;Write a test for SHOP-12&quot;}],&quot;usage&quot;:{&quot;input&quot;:5200,&quot;output&quot;:310}}
960
+ ← {&quot;type&quot;:&quot;close&quot;}
961
+ → {&quot;v&quot;:1,&quot;seq&quot;:9,&quot;type&quot;:&quot;closed&quot;}
962
+ </code></pre>
963
+ </div></section>
898
964
  </section>
899
965
  </section>
900
966
  <section class="chapter" id="area-browser" aria-labelledby="area-browser-h">
@@ -1605,6 +1671,7 @@ Signing in stays <code>/auth</code> and <code>forgeqe login</code>; <code>forgeq
1605
1671
  <tr><td><code>forgeqe start [--repo DIR] [--headed]</code></td><td>The shell</td></tr>
1606
1672
  <tr><td><code>forgeqe run &quot;&lt;request&gt;&quot; [--plain]</code></td><td>One request without the shell</td></tr>
1607
1673
  <tr><td><code>forgeqe run --direct &quot;&lt;goal&gt;&quot; [--skill NAME]</code></td><td>One browser goal, no agent</td></tr>
1674
+ <tr><td><code>forgeqe chat --jsonl [--carry-over FILE]</code></td><td>One live agent for a program, in JSON lines (see <a href="../HOW_TO_USE.md#a-live-agent-for-programs-forgeqe-chat---jsonl">A live agent for programs</a>)</td></tr>
1608
1675
  <tr><td><code>forgeqe suite &lt;source...&gt; [--parallel N] [--skill NAME]</code></td><td>Run test cases the same way every time (pipelines)</td></tr>
1609
1676
  <tr><td><code>forgeqe cases &lt;source...&gt;</code></td><td>List the cases a source gives, without running them</td></tr>
1610
1677
  <tr><td><code>forgeqe bi [build] [--from FILE|DIR]... [--static|--evidence]</code></td><td>The team report of every run here and the shared bundles (see <a href="#doc-how_to_use-team-report">Team report</a>)</td></tr>
@@ -1678,6 +1745,7 @@ Signing in stays <code>/auth</code> and <code>forgeqe login</code>; <code>forgeq
1678
1745
  <tr><td><code>/stop</code> or Esc</td><td>Stop what is running</td></tr>
1679
1746
  <tr><td><code>/rerun [last|run]</code></td><td>Run a passed run again as recorded, without ForgeQE's agent</td></tr>
1680
1747
  <tr><td><code>/headed on|off</code></td><td>Show the browser window</td></tr>
1748
+ <tr><td><code>/theme light|dark|auto</code></td><td>Colours for a light or a dark terminal. ForgeQE asks the terminal which it is when it starts and follows it when it changes (auto); a terminal that does not say is drawn dark. <code>FORGEQE_THEME=light</code> (or <code>dark</code>) sets it at start; <code>/theme</code> alone says which is in use</td></tr>
1681
1749
  <tr><td><code>/details [on|off]</code></td><td>Show everything the agents do while they work: the browser agent's reasoning, page memory, engine logs, every file the coding agent reads (off by default: step results, files written, test results, problems). <code>--verbose</code> or <code>FORGEQE_VERBOSE=1</code> starts with it on, for <code>start</code>, <code>run</code>, <code>suite</code>, <code>code</code>, and <code>heal</code> too</td></tr>
1682
1750
  <tr><td><code>/model</code>, or click the model ▾ in the status line</td><td>Pick the model from the profiles in <code>llms.json</code></td></tr>
1683
1751
  <tr><td><code>/profile &lt;name&gt;</code> or Tab</td><td>Switch the model profile</td></tr>
@@ -2012,7 +2080,7 @@ first; use a test machine and a test account. Details: <a href="#doc-docs-deskto
2012
2080
  </section>
2013
2081
  <section class="doc" id="doc-docs-desktop-forgeqe-desktop-thick-client-automation">
2014
2082
  <h4>ForgeQE desktop: thick-client automation</h4>
2015
- <div class="prose"><p>Status: runs on Windows. The CI job <code>.github/workflows/forgeqe-desktop-windows.yml</code> (GitHub's <code>windows-2022</code>) installs the
2083
+ <div class="prose"><p>Status: runs on Windows. The <code>desktop</code> job of <code>.github/workflows/ci.yml</code> (GitHub's <code>windows-2022</code>) installs the
2016
2084
  engine and the pinned UFO², replays Notepad live with the pywinauto driver and through <code>forgeqe desktop replay</code>, heals a
2017
2085
  moved control from the live window, and runs a whole explore: UFO² drives Notepad with a scripted model behind an
2018
2086
  OpenAI-compatible server, the action is recorded with the control's AutomationId, and the recording replays. Not yet
@@ -2274,7 +2342,7 @@ nightly workflow (<code>ci/desktop-runner/desktop-nightly.example.yml</code>), a
2274
2342
  <h4>Updating UFO²</h4>
2275
2343
  <div class="prose"><p>UFO² is pinned to one commit. Moving it is checked before it is pinned:</p>
2276
2344
  <ol>
2277
- <li><code>.github/workflows/forgeqe-desktop-ufo-update.yml</code> runs weekly, or by hand with a commit. It sets up that UFO on
2345
+ <li>The <code>ufo-update</code> job of <code>.github/workflows/ci.yml</code> runs weekly, or by hand (Run workflow, job <code>ufo-update</code>) with a commit. It sets up that UFO on
2278
2346
  Windows (<code>FORGEQE_UFO_COMMIT</code>, which takes a full 40-character commit id only), runs the adapter tests against its
2279
2347
  real code and a scripted explore of Notepad, and says in the job summary whether it passed.</li>
2280
2348
  <li>When it passed: <code>bun scripts/ufo-bump.ts &lt;commit&gt;</code> writes the new pin into <code>src/desktop/setup.ts</code>,
@@ -2544,7 +2612,7 @@ download the <code>hyperexecute</code> CLI into the folder, then run <code>forge
2544
2612
  <section class="doc" id="doc-docs-mobile-forgeqe-mobile-android-and-ios-apps">
2545
2613
  <h4>ForgeQE mobile: Android and iOS apps</h4>
2546
2614
  <div class="prose"><p>Status: the engine and its tests run on any OS, against a fake Appium server and a stand-in for Maestro. The CI job
2547
- <code>.github/workflows/forgeqe-mobile-android.yml</code> runs it on an Android emulator: Appium explores and replays the
2615
+ The <code>android</code> job of <code>.github/workflows/ci.yml</code> runs it on an Android emulator: Appium explores and replays the
2548
2616
  emulator's Settings app, and Maestro replays the same workflow. iOS runs the same code with the XCUITest driver, but
2549
2617
  no CI job tests it yet (it needs a macOS runner with a simulator).</p>
2550
2618
  <p>ForgeQE drives mobile apps two ways, and one workflow format runs on both:</p>
@@ -2734,7 +2802,7 @@ result in the report.</li>
2734
2802
  the value in its own vault). Replay it on TestMu with <code>--driver appium</code>.</li>
2735
2803
  <li>The access key is passed to the CLI as an argument. It is removed from the CLI's output and from every file in the
2736
2804
  run folder. Neither the user name nor the key is passed to the flows' environment.</li>
2737
- <li><code>.github/workflows/forgeqe-mobile-testmu.yml</code> runs all three on TestMu (Appium, Maestro with a workflow, a team flow)
2805
+ <li>The <code>testmu</code> job of <code>.github/workflows/ci.yml</code> (Run workflow, job <code>testmu</code>) runs all three on TestMu (Appium, Maestro with a workflow, a team flow)
2738
2806
  with TestMu's Wikipedia sample app, once the repository has the secrets <code>LT_USERNAME</code> and <code>LT_ACCESS_KEY</code>.</li>
2739
2807
  </ul>
2740
2808
  </div>
@@ -2883,7 +2951,7 @@ real Docker (and is skipped without it):</p>
2883
2951
  <li>A tests container that reaches the database by name and writes its results into the repository as this machine's
2884
2952
  user.</li>
2885
2953
  </ul>
2886
- <p><code>.github/workflows/forgeqe-containers.yml</code> runs both on GitHub's Docker. There, a skipped live test fails the job.</p>
2954
+ <p>The <code>containers</code> job of <code>.github/workflows/ci.yml</code> runs both on GitHub's Docker. There, a skipped live test fails the job.</p>
2887
2955
  <p>ForgeQE uses Docker or Podman, when one is installed, for five jobs:</p>
2888
2956
  <ul>
2889
2957
  <li><strong>Services a test run needs:</strong> a database, a queue, a cache, or the API under test. The project declares them in
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@capagents/forgeqe",
3
- "version": "2.3.13",
3
+ "version": "2.3.15",
4
4
  "description": "ForgeQE: a browser agent CLI that runs goals and test cases in local Chrome or Chromium, or BrowserStack and TestMu AI/LambdaTest when configured, and turns passing runs into tests in your repo",
5
5
  "license": "SEE LICENSE IN LICENSE",
6
6
  "repository": {
@@ -165,7 +165,7 @@ def write_project(project: Path, files: Sequence[Tuple[Path, str]], tests: Seque
165
165
  "framework": {"name": "raw", "args": args},
166
166
  "env": {"MAESTRO": True, "MAESTRO_LOGS_DIR": "MaestroLogs", "MAESTRO_CLI_NO_ANALYTICS": "1"},
167
167
  "pre": ["chmod +x ./setup.sh ./run.sh", "./setup.sh"],
168
- "testDiscovery": {"command": "cat ./discover.txt", "mode": "static", "type": "raw"},
168
+ "testDiscovery": {"value": names, "mode": "static", "type": "raw"},
169
169
  "testRunnerCommand": "./run.sh $test",
170
170
  "frameworkStatusOnly": True,
171
171
  "report": True,