@capagents/forgeqe 2.3.4 → 2.3.6

package/HOW_TO_USE.md

@@ -115,6 +115,7 @@ Everything ForgeQE can do, by area. ForgeQE's agent answers "can you…?" questi
 | Quality | **Defect intelligence: trends and prediction**: Groups failures into defects, keeps their history across suites (new, known, gone, back again), and gives defect trends and where a change is likely to cause defects, with the cases to run first. | `defect_trends`, `predict_defects` | [docs](#bugs-for-failures) |
 | Release | **Requirement coverage and release readiness**: Gives requirement coverage and a GO / GO WITH RISKS / NO-GO verdict for a sprint, area, epic, or stories, from test cases, results, bugs, and flaky tests; can publish it to the wiki. | `release_readiness` | [docs](#requirement-coverage-and-release-readiness) |
 | Operations | **Scheduled monitoring with alerts**: Runs checks on a schedule and alerts Teams or email when one keeps failing, and again when it recovers. | `monitor` | [docs](#9-monitoring) |
+| Operations | **Background jobs on a schedule**: Runs ForgeQE commands (suites, monitor rounds, regression packs, healing, pull-request tests, browser goals) on a schedule or once in the jobs service, apart from the conversation and after it closes, and watches an existing GitHub Actions or Azure DevOps pipeline (read-only). A failed pipeline run or pack run is investigated: the healing agent works on each failure on a local branch of its own (a fix for review, or an app bug), never pushing. Says how each job goes when asked, shows runs as they end, and sends failures, recoveries and findings to Teams or email. | `jobs` | [docs](#the-jobs-service-scheduled-and-long-jobs-apart-from-the-shell) |
 | Operations | **Teams and email notifications**: Tells the team in Microsoft Teams channels or by email when a suite fails or passes, when a release readiness verdict is in, and when the domain evaluation falls below its gate, with the failing cases, their bugs, and links to the report and the CI build. Each channel picks its events; the agent also posts a summary to a channel when asked. | `notify` | [docs](#17-notifications) |
 | Release | **Reports you can share**: Writes up what the Orchestrator found, such as open defects across projects, release status, or pipeline health, as a page with filters (project, state, priority, assignee, …), charts, and a sortable table, plus Markdown and CSV. Every table is built from evidence: the results of the tool calls the agent made, or a read-only query ForgeQE runs itself, listed on the page's Sources with each raw result kept beside the report. | `write_report` | [docs](#reports-from-the-agent) |
 | Operations | **CI pipelines**: Writes a GitHub Actions or Azure Pipelines workflow that runs ForgeQE, stores the secrets it needs, pushes a branch, and makes the pipeline green. | `push_branch`, `set_ci_secret` | [docs](#8-pipelines) |
@@ -350,7 +351,7 @@ Which part uses what:
 
 Copilot runs one SDK session per model request, with none of Copilot's own tools and your request's system prompt in place of Copilot's. The agent's tools are offered to the model as the session's only tools; when it calls one, the gateway ends the turn and hands the call back to ForgeQE's agent runtime, which runs it (with the same permissions as on any other model) and sends the result with the next request. The gateway itself runs nothing.
 
-**With Copilot and a key-based model both in `llms.json`**, Copilot comes first: a sign-in makes it the default, and without a `"default"` it is the active profile. Before a model is used, ForgeQE checks that it works (its sign-in or key, and one live call, kept for a day once it passes). If Copilot does not work (not signed in, the Copilot SDK missing, the model not on your seat), the other model is used and ForgeQE says why and how to fix Copilot; if a key-based model you made the default does not work, Copilot is tried first. A model that fails its check is never used, and when none works the error lists each one with what to do. `--profile` turns this off: the model you name is used as it is. Each request is billed to the seat (GitHub AI credits or premium requests, by your plan), and GitHub may rate-limit frequent automated requests. If a model is refused, check that your organization's Copilot policies allow it. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
+**With Copilot and a key-based model both in `llms.json`**, Copilot comes first: a sign-in makes it the default, and without a `"default"` it is the active profile. Before a model is used, ForgeQE checks that it works (its sign-in or key, and one live call, kept for a day once it passes). If Copilot does not work (not signed in, the Copilot SDK missing, the model not on your seat), the other model is used and ForgeQE says why and how to fix Copilot; if a key-based model you made the default does not work, Copilot is tried first. A model that fails its check is never used, and when none works the error lists each one with what to do. This happens with `llm.routing` off too: routing picks a model for each kind of work, the fallback only replaces one that does not work. `--profile` turns it off for one run, and `llm.fallback: false` in `forgeqe.yaml` for good: the model you chose is used as it is, and its own error is shown. Each request is billed to the seat (GitHub AI credits or premium requests, by your plan), and GitHub may rate-limit frequent automated requests. If a model is refused, check that your organization's Copilot policies allow it. ForgeQE shows no cost for a `github_copilot` profile unless you give it a `price`.
 
 **In pipelines.** `forgeqe suite ... --no-code` uses only the browser engine. Set `COPILOT_GITHUB_TOKEN` to a fine-grained token with the Copilot Requests permission, stored as a pipeline secret. GitHub also documents the workflow's own `GITHUB_TOKEN` with `permissions: copilot-requests: write` for organization repositories whose Copilot policy allows it ([server tokens](https://docs.github.com/en/copilot/how-tos/copilot-sdk/auth/server-to-server-tokens)). A Copilot seat belongs to one person: prefer an organization-billed token over signing a shared pipeline in to someone's seat.
 
@@ -2583,6 +2584,51 @@ Monitoring checks/smoke.yaml every 15m · replay verify · alerts: Teams, email
 
 You can also ask ForgeQE's agent. In the shell, "check the smoke cases once" runs one round and shows it as a card, and "monitor the smoke cases every 10 minutes" starts a monitor in the background: you keep working, each round appears as a card, "is anything failing?" shows the state, and "stop monitoring" ends it. A background monitor lives as long as the shell (or `forgeqe mcp`) and stops when it closes; `forgeqe run` refuses to start one, because it would stop when the command exits. While one is running, a one-off round on the same `out_dir` is refused rather than run twice. For monitoring that keeps going after you close the shell, "monitor the smoke cases every 10 minutes and alert the QA channel in Teams, always on" makes the agent set up `monitor:` in `forgeqe.yaml`, send a test alert, and tell you how to keep `forgeqe monitor` running (or add a scheduled pipeline).
 
+### The jobs service: scheduled and long jobs apart from the shell
+
+`forgeqe jobs` runs ForgeQE commands on a schedule, or once, in a service of their own, so they keep going while you work on something else in the shell, or with the shell closed. Each run is its own `forgeqe` process with its own output and session log; you ask how a job is going at any time, and its failures and recoveries go to Teams.
+
+```bash
+forgeqe jobs add nightly-pack --cron "0 2 * * 1-5" -- pack run release-2.3   # weekdays at 02:00, this machine's time
+forgeqe jobs add smoke --every 30m -- suite checks/smoke.yaml
+forgeqe jobs add heal-login --now -- heal --focus login                     # once, now
+forgeqe jobs                       # every job: running or how its last run ended, and when it runs next
+forgeqe jobs status nightly-pack   # its last runs, and the last lines of the latest
+forgeqe jobs logs nightly-pack --tail 100
+forgeqe jobs run nightly-pack      # run it now as well
+forgeqe jobs stop nightly-pack     # stop the run that is going
+forgeqe jobs disable|enable|remove nightly-pack
+forgeqe jobs shutdown              # stop the service; scheduled jobs wait until it starts again
+```
+
+- **What a job runs.** One ForgeQE command: `suite`, `monitor`, `run`, `heal`, `pr-tests`, `pack run`, `pack triage` or `pipeline watch`, with its arguments after `--`. Nobody is there to answer a question while a job runs, so a `run` job is a browser run of the goal (`--direct`), and a `pack run` job carries `--yes`: adding the job is your approval, and the command you agree to shows it. It runs with the project's own `forgeqe.yaml`, model profile and approvals: a job cannot set `--config`, `--profile` or `--yes`, and nothing else is started (no shell, no other program). A `monitor` job runs one round each time it is due (`--once`), on the job's schedule; it sends its own `monitor.alerts`, so its job is quiet unless you add it with notices on.
+- **When.** `--every 15m` (at least 1m), `--cron "0 2 * * 1-5"` (minute hour day month weekday, local time), or neither: it runs when asked, and `add` starts it at once. `--now` also runs a scheduled job straight away. A run missed while the service was down starts once when it is back. When several are due, the one that has waited longest goes first.
+- **How many.** `jobs.parallel` in `forgeqe.yaml` (default 2) runs at once over all jobs, and never two runs of one job. A run past `--timeout` (default 6h, at most 24h) is stopped and marked timed out.
+- **The service.** `forgeqe jobs add` and `forgeqe jobs run` start it when it is not running, as a process of its own that outlives the command and the shell; `forgeqe jobs serve` runs it in the foreground (for a container or a system service). It is one per project, on this machine, until you stop it or the machine restarts. It listens on 127.0.0.1 only, with a random token kept in your own ForgeQE folder (`~/.forgeqe/jobs/`, readable by your user only, never in the project, which others may read), and refuses requests from web pages. A second service for the same project is refused. `.forgeqe/jobs/` and `.forgeqe/pipelines/` get their own `.gitignore`, so runs and logs are never committed.
+- **Files.** `.forgeqe/jobs/<job>/job.json` and, for each of its last 50 runs, `runs/<run>/run.json` and `output.log`; `.forgeqe/jobs/events.jsonl` lists what happened; `.forgeqe/jobs/service.log` is the service's own output. A run that was going when the service stopped is marked interrupted.
+- **Watching a CI pipeline.** `forgeqe jobs add ci --every 10m -- pipeline watch --github org/app --workflow ci.yml --branch main` (or `--ado-project Shop --pipeline 12`) looks at the pipeline's latest completed run each time, read-only: it starts, re-runs and changes nothing. A new failed run is triaged like a pack's (the JUnit or TRX files in its artifacts on GitHub, the Test Runs API on Azure DevOps; without results, the job and step that failed), in `.forgeqe/reports/pipelines/`. The job fails while the latest run is failed, so the team hears once when the pipeline breaks and once when it is green again; a cancelled run changes nothing. `forgeqe pipeline watch …` on its own does one look. GitHub needs the `gh` CLI signed in; Azure DevOps the same sign-in as the rest of ForgeQE.
+- **Investigating failures.** When a `pipeline watch` or `pack run` job fails, the failures that healing can decide (an element not found, an expected value that does not hold; not the environment, flaky tests or broken test code, and nothing while the environment is down) are investigated, at most 3 a run, `jobs.parallel` at once. Each goes to the healing agent in a git worktree of its own, on a new local branch `forgeqe/investigate/<test>-<time>-<n>`, from the commit the failing run tested when this clone has it (fetched when it can be, else `HEAD`, which the finding says). The agent re-runs the flow in the browser: when the flow works it heals the test, and the fix is committed on that branch for you to review; when the browser fails at the same step it is an app bug, and no branch is kept. Your checkout, its branch and its files are never touched; nothing is pushed or merged; the worktree is removed at the end. `node_modules`, `.venv` and `venv` from your checkout are linked into the worktree so the tests can run, and never committed. The failure is told at once (`job_failed`), the findings when they are in (`job_investigated`); `forgeqe jobs status` lists them, `.forgeqe/jobs/<job>/runs/<run>/investigations.json` and `investigate.log` have the detail, and `forgeqe investigate <run folder>` runs an investigation by hand. `--no-investigate` turns it off for a job.
+- **From the shell.** Ask ForgeQE's agent: "run the release-2.3 pack every weekday at 2am in the background" makes it propose the job, and ForgeQE asks you to agree to its exact command and schedule before adding it (**Add the job** / **Don't add it**). You keep working; when a job's run ends, a line says so in the conversation ("Job nightly-pack failed: …"), and "how are my jobs?" or "show the end of the nightly-pack log" get answers from the service. The agent adds a job only when you ask for one, and never one that runs anything but the commands above.
+- **Notices.** With `notify:` channels set up (section 17), a job that fails sends `job_failed` once (not every run while it keeps failing), `job_recovered` when it passes again, and a job that runs only when asked sends `job_done` when it ends. `--no-notify` keeps a job quiet.
+
+```yaml
+jobs:
+  parallel: 2
+```
+
+**On company machines.** Nothing here needs administrator rights:
+
+| What | Needs |
+|---|---|
+| The jobs service | Runs as you, in your project and your ForgeQE folder. It listens on 127.0.0.1 only, so no firewall rule is needed. Endpoint security may see a background `bun` process: it is ForgeQE's own (in its install folder), which IT can allow. Jobs pause while the machine sleeps or is off and a missed run starts once when it is back; for always-on jobs, run `forgeqe jobs serve` on a VM or build agent under its service manager (systemd, a Windows Task Scheduler task "at log on", which needs no administrator, or a container). |
+| `pipeline watch` on GitHub | The `gh` CLI signed in (`gh auth login`) with read access to the repository's Actions. A fine-grained token needs Actions: read and Contents: read; with SAML single sign-on, authorize the token for the organization. GitHub Enterprise Server: `--github host/owner/repo` and `gh auth login --hostname host`. |
+| `pipeline watch` on Azure DevOps | The Azure DevOps sign-in ForgeQE already uses (`forgeqe auth login azure_devops`, or a token in `ADO_PAT`) with Build: read and Test Management: read. Azure DevOps Server works through its collection URL. |
+| Investigating failures | git, read access to fetch the failing commit, the tests' dependencies installed in your checkout, and a model: a key-based profile, or GitHub Copilot where the organization's policy allows Copilot CLI and the SDK. Branches are local; nothing is pushed. |
+| Teams notices | A Workflows webhook ("post to a channel when a webhook request is received"), or an incoming webhook where the tenant still allows them; where neither is allowed, email. |
+| Proxy and inspected HTTPS | The same as the rest of ForgeQE (section "Company networks"); `gh` uses `HTTPS_PROXY`. `forgeqe doctor` checks the machine and the network. |
+
+On Windows, the dependency folders are linked into worktrees as junctions (no administrator or developer mode needed), a stopped run ends at once rather than being asked to finish, and the token file is protected by your user profile's permissions.
+
 ## 10. MCP
 
 ### Servers ForgeQE's agent connects to
@@ -3031,8 +3077,12 @@ notify:
 | `suite_passed` | Every case passed | The count, how long it took, where it ran |
 | `release` | A release readiness verdict | GO, GO WITH RISKS, or NO-GO, the counts, every blocker and risk, and the wiki page when published |
 | `eval_failed` | `forgeqe eval domain --min` falls below its gate | Coverage for each variant and invented ids |
+| `job_failed` | A job of the jobs service fails or runs out of time, after passing or for the first time (not again while it keeps failing) | The job, its command and schedule, how long it ran, its last lines of output, and where its output is |
+| `job_recovered` | A job passes again after failing | The same, for the run that passed |
+| `job_done` | A job that runs only when asked ends | The same, whether it passed or failed |
+| `job_investigated` | The investigation of a job's failed run is done | Each failure: a test out of date with its fix on a branch, an app bug, or not decided, with the agent's reasons |
 
-A channel without `on` gets `suite_failed`, `release`, and `eval_failed`. Email adds the failure screenshots. A stopped suite is not announced. Monitoring keeps its own `monitor.alerts`, with the same cards.
+A channel without `on` gets `suite_failed`, `release`, `eval_failed`, `job_failed`, `job_recovered`, and `job_investigated`. Email adds the failure screenshots. A stopped suite is not announced. Monitoring keeps its own `monitor.alerts`, with the same cards.
 
 Events are sent from runs nobody is watching: `forgeqe suite` in a pipeline or on a schedule, `forgeqe run`, and the MCP server. In the shell, where people try things out, nothing is sent on its own; ask instead ("post the results to the QA channel", "tell the release channel the verdict"), and ForgeQE's agent posts a card with `notify`.