@engineeros/connector 0.27.1 → 1.6.1

package/README.md

@@ -1,9 +1,60 @@
 # EngineerOS Connector
 
-The connector uses an authenticated EngineerOS WebSocket for pairing, workspace identity, run lifecycle, and evidence. Coding work can run through any agent in the official ACP registry or a custom ACP v1-compatible command over stdio. Direct Codex CLI remains available as the compatibility path.
+The connector uses an authenticated EngineerOS WebSocket for pairing, workspace identity, run lifecycle, and evidence. Coding work can run through any agent in the official ACP registry, a custom ACP v1-compatible command over stdio, or the Forge command line. Direct Codex CLI remains available as the compatibility path.
+
+## Independent Goal runner
+
+Connector 1.2.0 hands Goal jobs to a separate local runner. Closing the connector terminal or losing its WebSocket does not stop an executing Goal. Progress and final results use HTTP; undelivered results remain on disk for retry. Windows supervisors and workers use a windowless launcher; Goal runs and agent commands do not open new terminals.
+
+Install from a permanent global installation or repository checkout, then run:
+
+```text
+engineeros-connector runner install --workspace D:\Puli\work\workspaces\caresched
+engineeros-connector runner status --workspace D:\Puli\work\workspaces\caresched
+```
+
+The first Goal dispatch also installs/starts the runner when using a permanent executable path. Temporary `npx` cache paths cannot host a persistent service: install the connector globally first. Update the backend for the HTTP claim endpoint before dispatching Goals.
+
+Windows uses Task Scheduler under your current user, restarts after failure, and resumes saved jobs at sign-in following reboot. Linux uses a systemd user service; enable user lingering separately if recovery must start before login. Existing running jobs stay with their current process; resume an interrupted run to hand it to the new service.
+
+Files under your project's `.engineeros` directory:
+
+- `runner/inbox/`: durable job dispatch requests.
+- `runner/service.json`: supervisor PID and heartbeat.
+- `runner/service.log`: supervisor startup and cleanup diagnostics.
+- `runs/<run-id>/job.json`: execution/delivery status, saved assignment, pending result or failure.
+- `runs/<run-id>/state.json`: coding session and implementation/verification checkpoints.
+- `runs/<run-id>/activity.log`: timestamped activity and periodic inactivity status.
+- `runs/<run-id>/worker.log`: worker console diagnostics.
+
+The runner stops and records a resumable failure after ten minutes without agent events, except when explicitly waiting for user input. Restore missing prerequisites or agent usage before using **Resume run**. Acceptance and integration remain controlled by EngineerOS.
+
+There is no wall-clock execution cutoff. Runs stop after at most two repair-and-review cycles following the initial review. The inactivity safeguard above still stops a silent agent; waiting for your answer does not consume a repair cycle. Successful results awaiting upload are not executed again.
+
+### Delete a Goal and its sandboxes
+
+Choose **Delete Goal** in the Goal detail header and confirm. This deletes all revisions, runs, receipts, proof results, attachments and run questions for that Goal. Connected runners first stop every run in the revision family, then remove owned isolated Git worktrees (including uncommitted sandbox files), checkpoints, queued dispatches and logs. The source repository, its local changes, Git branches and unrelated worktrees are preserved. External registered worktrees require their matching saved checkpoints to establish ownership. Manually managed external environments remain the external runner's responsibility.
+
+Deletion remains visible as **Deleting** until every assigned connector acknowledges cleanup. The supervisor polls authenticated cleanup requests every 15 seconds, including while the connector terminal is closed. Offline machines and lost acknowledgements retry safely. Cleanup errors are displayed on the Goal with **Retry cleanup**; a live worker lock or ambiguous sandbox ownership prevents removal. Active backend verification must finish before deletion starts. Small markers in `runner/deleted-runs/` prevent delayed dispatches from restarting deleted work.
+
+After updating an existing installation, reinstall the runner task with `runner install` and restart the existing supervisor to load the new code. The Windows scheduled action must use `wscript.exe` and `runner/launch.vbs`. Logs remain in the project's `.engineeros` directory; no terminal is needed for a run.
+
+If the Windows runner task was disabled, `engineeros-connector runner start --workspace PATH` explicitly enables it and starts processing queued jobs. Incoming dispatches remain queued while the task is disabled and print this recovery command; they do not enable it automatically. Do not resend the Goal just to start its queued job.
+
 
 ## Connect an official ACP agent
 
+Connector 0.33.0 supports **Deliver capability**. EngineerOS immediately queues the capability as a Goal request;
+it does not generate or require approval of a separate implementation slice first. The coding agent reads the
+frozen local design, writes its implementation plan inside the run worktree, then implements, tests and repairs
+the work. Independent testing and review agents verify the delivered commit. Required user choices use the
+existing run questions and resume the same work. The final handover includes the committed plan, source changes,
+test evidence and review results. Integration still requires the user's review.
+
+Planning and verification read the assigned design from its local Git commit, so later design synchronization
+does not silently change a running delivery. The implementation plan is stored at
+`docs/delivery/<goal-id>/implementation-plan.md` and remains part of the delivered repository history.
+
 EngineerOS reads the curated [ACP agent registry](https://agentclientprotocol.com/registry), caches it for 24 hours, and uses the registry's pinned distribution for the current platform. The catalog includes Codex, Claude, Gemini, GitHub Copilot, Goose, OpenCode, Qwen Code, Cursor, and other compatible agents as they are published. List the current catalog, prepare the selected agent, then pair the workspace:
 
 ```sh
@@ -26,9 +77,26 @@ npx --yes @engineeros/connector@latest pair PAIRING-CODE --url http://localhost:
 
 Connect a local Codex CLI workspace to EngineerOS through an outbound WebSocket.
 
+## Pair Forge
+
+[Forge](https://forgecode.dev) is driven through its own command line rather than ACP. Install it, sign in to a provider, then pair the workspace:
+
+```sh
+forge provider login
+npx --yes @engineeros/connector@latest pair PAIRING-CODE --url https://your-engineeros.example --workspace . --onboard --agent-protocol forge
+```
+
+Forge runs its own agents, so `--agent` and `--agent-command` are rejected for this protocol. The connector selects the Forge agent that carries the access an assignment is entitled to: `sage` for read-only research, planning, assessment, and verification work, and `forge` for the writable implementation phase of a registered Goal. Pairing stops with the corrective command when Forge is missing, unauthenticated, or no longer provides both agents.
+
+Each EngineerOS session maps onto a Forge conversation, so project prompts and multi-stage assessments resume with their history intact. Token usage is reported per turn, measured as the growth of the conversation while the turn ran.
+
+Every model Forge offers through the providers you are signed in to is published to EngineerOS, named with its provider because the same model is offered by more than one, along with every reasoning effort Forge accepts. Forge takes the provider, model and reasoning effort of a run from the environment of its process, so the choice made in EngineerOS applies to that run alone: it does not change the model set with `forge config set model`, and it does not disturb a run beside it. The model and effort configured in Forge are marked as the default. Pairing stops when Forge offers no models, which means no provider is signed in: run `forge provider login`, then restart the connector.
+
+The connector registers the EngineerOS project artifact tools with Forge for the current user, which is the only scope Forge trusts without a person approving a file in the workspace, so nothing is written into the repository being worked on. Forge shares that one registration across every workspace and discovers the tools once, so the run each tool acts on is carried in the environment of the Forge process rather than in the registration. Concurrent runs stay separate, and a tool called outside an EngineerOS run is refused with the reason.
+
 ## Agent roles and skills
 
-The selected ACP or Codex agent is the execution engine. EngineerOS chooses a provider-neutral role for each activity and the connector injects exactly one bundled skill:
+The selected ACP, Forge, or Codex agent is the execution engine. EngineerOS chooses a provider-neutral role for each activity and the connector injects exactly one bundled skill:
 
 - `research` uses `project-research` to answer questions from observed project evidence.
 - `assessment` uses the internal `assessment-maintenance` skill to inspect source without changing it and edit only the assigned assessment Markdown document.
@@ -54,11 +122,22 @@ From **Project steering -> Workspace**, run the workspace assessment to use the
 
 Every backend `422` content-validation response is returned to the stage agent with the exact failed item. Distinct validation failures continue through correction and resubmission until the complete stage is accepted. Three repetitions of the same unresolved validation error stop the automatic loop while preserving the latest local report for retry. Authentication, cancellation, source drift, and transport failures remain operational errors rather than agent-correction prompts.
 
+If Codex reports that its refresh token was already used, stop every connector process using Codex before reauthenticating. Run `codex logout`, then `codex login` in a normal terminal, and restart one connector with `start --workspace ...`. The existing EngineerOS workspace pairing remains valid. The connector stops the failed ACP runtime and reports these recovery steps without exposing its internal stack trace.
+
 Assessment runs start by reporting whether an accepted assessment exists and how stale it is. Every assigned stage still runs against the current repository revision; durable reports do not contain a trustworthy dependency manifest, so the connector never presents an old section as newly inspected. Connector-owned files under `.engineeros` are ignored before drafts or context packs are created and therefore do not pollute repository change scans.
 
 After onboarding, every project prompt is routed to this connection. Copilot, shaping, planning, architecture, and experience generation use the connected agent subscription and workspace context. Interactive prompts run independently from assessments and Goal scheduling. Prompt runs are read-only; only an explicitly registered Goal Run receives workspace-write access. If the connector is offline, EngineerOS asks the user to reconnect instead of silently switching models.
 
-Greenfield prompts also receive a bounded project-evidence manifest. EngineerOS normalizes uploaded text,
+Capability implementation planning uses committed local context in connector `0.32.0`. The synchronized
+`INDEX.md` links to the design, product intent, preparation context, decision revisions, implementation plans,
+current Facts and normalized Sources. A planning assignment carries the task, new guidance, record references
+and the expected snapshot revision; it does not resend those artifact bodies or an assignment-specific evidence
+copy. Before starting the agent, the connector checks the revision, saves the snapshot in local Git and supplies
+a `git show` pointer to that commit. All linked records are read from the same commit, so background sync does
+not change a running plan's input. The backend rechecks the context before saving a generated plan; stale input
+or invalid output leaves the accepted plan intact. No remote push is performed.
+
+Other greenfield generation prompts receive a bounded project-evidence manifest. EngineerOS normalizes uploaded text,
 PDF, DOCX, CSV, workbook, presentation, and image Sources, and the connector materializes normalized Markdown
 and available original attachments under an assignment-specific directory in `.engineeros/context/inputs/`.
 These files are connector-owned, ignored project
@@ -86,6 +165,20 @@ An empty or document-only folder establishes a greenfield baseline. A code-beari
 
 ## Reconnect
 
+Connector 1.0.0 requires the updated EngineerOS backend. Upgrade both together, then restart the existing
+connector with `start`; its saved pairing remains valid. The WebSocket carries dispatch and control messages.
+Goal progress, assessment worker status, prompt output, failure reports and results use authenticated HTTP.
+There is no application JSON ping/pong. WebSocket protocol keepalive detects connection loss; quiet Goal work
+sends an HTTP heartbeat after 15 seconds without a progress update. Idle connectors need no application heartbeat.
+
+HTTP progress keeps one request in flight and retains the latest unsent state. Prompt deltas are ordered and
+bounded; a failed progress stream does not discard the final completed response. Requests have timeouts and
+bounded retries. Goal execution attempts and sequence numbers protect against delayed updates.
+
+Run the backend as **one worker and one replica**: dispatch sockets and pending prompt waiters currently live
+in process memory. The production image now uses one worker. Multiple backend workers require a shared dispatch
+broker before enabling them. An incompatible backend or connector fails with an upgrade instruction.
+
 ```sh
 npx --yes @engineeros/connector@latest start --workspace .
 ```
@@ -93,13 +186,27 @@ npx --yes @engineeros/connector@latest start --workspace .
 Credentials are stored per workspace under `~/.engineeros/connectors` with owner-only permissions where supported.
 If a new pairing command is accidentally run from the same folder against the same EngineerOS server, the connector reuses this saved identity instead of creating another baseline assessment.
 
+## Local Git history
+
+Pairing or starting connector 0.31.0 initializes a local Git repository when the project has none and creates its initial commit from source files and durable design/assessment artifacts. An empty project also gets a usable baseline. Existing repositories and their pending source changes are preserved. Git must be installed; no remote is created and nothing is pushed.
+
+As design context syncs, the connector commits new, changed and deleted durable artifact files locally. These explicit artifact paths are included even though connector runtime state is ignored. Source generated by a capability is committed by the existing implementation and verification workflow on its capability branch. Environment secrets, credentials, dependency/build directories, context indexes and temporary handover files remain excluded. Identical snapshots produce no extra commits, and artifact sync waits while an implementation run is active.
+
 ## Run Goals
 
+From capability delivery, choose **Implement capability** and an online coding connector. EngineerOS prepares any missing implementation plan from the saved design, validates its prerequisites, and starts the run. **Preview plan** is optional; the detailed plan remains available without becoming a separate mandatory document review.
+
+The connector coordinates implementation, independent testing and independent code review. Native implementation subagents may work on independent tasks when the selected coding harness supports them. Testing and review run as separate read-only processes after the implementation is committed. At most two repair-and-review cycles follow the initial review; completed allowances survive resume. Repository finalization has its own single recovery attempt. Unresolved findings are then handed back with failed evidence instead of starting another repair. The handover includes the implementation report, exact revision, changed files and final check reports; persistent failures remain failed.
+
+Reuse existing shared foundations. A necessary out-of-scope prerequisite requires a recorded approval through `engineeros_request_input` with `decision_area: scope_expansion`, a stable question key, the precise addition and its reason and impact. Each run permits at most two expansion requests, including rejected or dismissed requests. Select **Approve scope expansion** in EngineerOS to authorize that specific addition; other answers do not authorize work. These approvals reach the implementer, independent reviewers and final handover. They do not reset the repair allowance or rewrite the original frozen Goal. Further expansion requires a separate Goal.
+
+When a consequential choice is missing, the implementation agent calls `engineeros_request_input` and finishes its turn. Answer the question in the active run using a suggestion or your own words. The connector keeps heartbeats active, preserves the workspace and resumes with the saved answer. Dismissal and cancellation never count as answers. Cancellation stops all workers in the run. This feature requires the updated backend and connector; it does not deploy the product or automatically approve its handover.
+
 Keep the connector online to receive Goals assigned from EngineerOS. Each run explicitly selects one workspace
-mode. `local_branch` creates or reuses `engineeros/goal-<goal-id>` in the connected repository, runs with its exact
+mode. `local_branch` creates or reuses the capability branch in the connected repository, runs with its exact
 local dependencies and properties, and returns to the original branch after a clean committed run. It is the
 default for a dedicated agent machine and fails without stashing when another branch has uncommitted work.
-`isolated_worktree` creates one durable worktree at `<repository-parent>/.engineeros/goals/<goal-id>` and reuses it
+`isolated_worktree` creates one durable worktree at `<project>/.engineeros/goals/<goal-id>` and reuses it
 across retries. The worktree shares Git configuration and hooks and copies individual ignored local configuration
 files; ignored dependency and build directories are not duplicated. The coding-agent process inherits the
 connector environment in both modes. Cancellation stops the agent. The connector returns changed paths, a bounded
@@ -110,6 +217,19 @@ branch remains available for explicit review and merge.
 
 During the writable implementation phase, Codex receives an authenticated EngineerOS MCP server automatically. It can list, read, create, update, reclassify, soft-delete, and materialize project artifacts into canonical Product records through the same repository boundary used by Copilot. The backend accepts those calls only while the assigned Goal is running. Read-only project prompts and the independent verification phase do not receive mutation tools.
 
+### Resume a failed or interrupted Goal run
+
+Connector 0.34.0 keeps new isolated Goal worktrees inside the project at `.engineeros/goals/<goal-id>`.
+Execution checkpoints live at `.engineeros/runs/<run-id>`. Start the connector with its existing pairing,
+open the Goal's run panel and select **Resume run**. A disconnected running run becomes resumable after
+90 seconds without a heartbeat. Offline resume requests wait for the assigned connector to reconnect.
+
+Resume preserves the run ID, frozen assignment, branch, files and completed implementation. It restores the
+agent session when supported and reruns interrupted verification. A completed result is retained for another
+upload attempt; changing the code causes it to be verified again. Existing worktrees are reused through Git's
+registry, so an upgrade does not move your current work. A missing workspace or corrupt checkpoint must be
+restored before resuming. Submitted results and final acceptance receipts retain their existing review flow.
+
 ## Local context engine
 
 The connector keeps a local index for each workspace under `~/.engineeros/context/<connector id>/workspaces/`: file classifications, symbols, document headings, and import relationships for every safe file. Separating indexes by resolved workspace keeps the source checkout and concurrent Goal worktrees from replacing one another. An index is built at onboarding, refreshed incrementally from file digests when EngineerOS triggers a workspace refresh, and re-checked before every assignment.
@@ -122,6 +242,20 @@ Every direct Codex assignment — project prompt, assessment, Goal implementatio
 
 After Goal implementation commits its isolated worktree change, the connector rebuilds context from that exact post-commit workspace before starting the independent verifier. Verification therefore receives the current task-ranked working set plus a bounded `Change Impact` section naming changed files, direct importers, and candidate tests rather than relying on the pre-change index. `engineeros_similar` helps the Agent extend existing functions, classes, and types instead of duplicating them. Stored symbol outlines omit literal constant values, MCP search responses are capped at one hundred files, and the index, source map, prompt context, and session memory remain on the local computer.
 
+## Local design artifacts
+
+Connector 0.28.0 synchronizes persisted design records into
+`.engineeros/design/projects/<project-id>/`, with `INDEX.md` linking the named
+Markdown artifacts and their current review status. Sync runs on connection,
+every ten seconds, and before project prompts and Goal execution. Goal worktrees
+receive their own copy. Assignments point the agent to the index explicitly.
+
+The database remains authoritative. Confirm edits through EngineerOS Copilot;
+local changes are overwritten by the next successful sync. Renames and deletions
+remove outdated Markdown files from this managed directory. An offline connector
+refreshes when it reconnects. The backend must provide the matching design-context
+endpoint; sync failures are reported and block new agent tasks from using stale context.
+
 ## Requirements
 
 Node.js 22 or newer, Git, and an authenticated agent. `npx` distributions use npm, `uvx` distributions require uv, and binary archives require `tar` (`unzip` for ZIP files on Linux). Registry agents report their own authentication prerequisites when they start.
@@ -146,3 +280,11 @@ npm install -g @openai/codex@latest
 codex --version
 npx @engineeros/connector start --workspace .
 ```
+
+## Supervisor recovery
+
+The workspace runner monitors worker health separately from progress reporting. Continuous reporting failures pause coding after one minute; missing worker health is detected after ninety seconds. Recovery preserves the sandbox and checkpoint. It allows one restart per failure category and at most two per run, with a cooldown and storage check before restarting.
+
+Unknown runtime failures may use one read-only LLM diagnosis per run through the configured agent, with a two-minute deadline. Known filesystem and network faults do not call an LLM. Diagnosis recommends retrying the checkpoint or a manual fix; it does not execute generated commands or modify the connector. Repeated failures stop automatic recovery. Inspect `runner status` and `.engineeros/runner/recovery/<run-id>.json`, fix the reported cause, then explicitly resume the existing run. Recovery budgets are retained.
+
+Verification results are cached per commit and role. Completed checks are reused. An interrupted check requires inspection before a new revision; it is not automatically billed again. Existing workers must load the new connector code before these protections apply.