jg-agent-hq 1.1.0 → 3.0.0

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 3.0.0
+
+- Check protocol 0.2 plans with jg-plans 0.3.0 at each stage that grants authority: plan approval requires plan.md to conform, and finalization, sequence approval, authorization and execution require plan.md and milestones.md to conform together. Reading, status and revision workflows are unaffected, and plans without the 0.2 marker keep their current behavior.
+
+- Version HQ storage independently of host user_version. Add explicit offline Node/CLI upgrades with a verified pre-upgrade backup, transactional rollback and uncertainty handling, retaining published 2.0.0 snapshots and version-specific restore fences.
+
+- Expose native Codex credit balances and available earned resets without consuming them. Preserve valid single-bucket quota responses when the multi-bucket view is absent.
+- Identify Claude five-hour and seven-day usage meters with explicit model/surface scopes, preserving independent windows, reset times and account-bound freshness. Keep older provider observations compatible.
+
+## 2.0.0
+
+- Add provider-neutral session capabilities, native lifecycle/configuration, file inputs, structured results, permissions, account observations and extension controls. Detect usable native contracts instead of restricting CLI version labels; isolate unavailable providers.
+- Add task-scoped agent identities, streams, subscriptions, requests, verified skill grants and bounded report storage/search. Agents can delegate, yield for named children and continue from their exact results across vendors or within one vendor. Replace the fixed-provider handoff API with this flow.
+- Support optional verified jg-skills delegation without changing native explicit-only loading, later approval checkpoints or standalone workflows. Keep execution grants, native tool permissions, outcomes and operator approval separate.
+- Expose trusted shared SQLite metadata and coordinated host maintenance. Capture WAL data, host tables, reports and registered plan companions; validate offline recovery with the matching private key and paused automation. Hosts own off-machine backup operations.
+- Ship the current schema and API directly. Start with a fresh development state root when changing the schema; no data conversion is provided. Add public host examples and packed-consumer coverage for storage and recovery.
+
 ## 1.1.0
 
 - Rebuild the homepage around a native-session visual, local setup and three complete source-backed walkthroughs. Show cross-provider reviews and retained revision evidence with explicit host decisions.

package/README.md

@@ -1,6 +1,8 @@
 # jg-agent-hq
 
-Workspace primitives for native coding agents. The public API provides owner-aware filesystem locks, Claude and Codex thread control, live input and history, actual account usage, and revision-bound plan authority.
+A TypeScript library and local controller for applications that run coding agents. Create native Claude Code and Codex sessions, send work, handle permissions and follow results through one API. Agents collaborate through indexed streams, directed requests and exact reports, with explicit authority for delegated work.
+
+Use it to build an agent dashboard, editor integration or automated development workflow. HQ owns local session control, agent identity, collaboration state and plan coordination. Your application owns its interface, authentication and off-machine backups. Native CLIs supply the coding harnesses, tools, skills and sign-in; additional providers can implement the same host contract.
 
 Requires Node.js 24.15.0 or later on Linux or macOS. The lock protocol uses POSIX ownership, private directories and process-liveness probes. Windows is outside the supported host contract.
 
@@ -35,21 +37,37 @@ Intentional failures use `Exception` from `jg-exception`, with the named string
 
 ## Local controller
 
-Install Codex and sign in through its native CLI, then start `jg-agent-hq serve --state-root .runtime --workspace-root /path/to/workspace`. Run `jg-agent-hq status --state-root .runtime` in a client. `threads`, `thread`, `turns`, `items` and `usage` read native information; `stop` waits for confirmed runtime shutdown.
+Install the providers you want and sign in through their native CLIs, then start `jg-agent-hq serve --state-root .runtime --workspace-root /path/to/workspace`. Run `jg-agent-hq status --state-root .runtime` in a client. `threads`, `thread`, `turns`, `items` and `usage` read native information; `stop` waits for confirmed runtime shutdown.
 
 The Node API exposes `createAgentHq({ stateRoot, allowedRoots })` and `requestAgentHq(stateRoot, command)`. Configuration comes from these trusted operator inputs. Coding-instruction files are not runtime configuration.
 
-The controller owns its SQLite worker and native connections independently of client connections. Private state contains the lifetime owner, Unix socket and SQLite discovery metadata. Conversation bodies remain in the native providers' storage. Browsing never resumes an external session or starts inference. Quota windows come from native account observations, with explicit stale and unavailable states.
+The controller owns its SQLite worker and native connections independently of client connections. Its provider registry starts each native integration only when selected. Storage and managed-session reads work without an installed coding CLI; an unavailable provider does not disable another provider. Private state contains the lifetime owner, Unix socket and SQLite discovery metadata. Conversation histories remain in native storage; HQ retains complete bounded final results on their action receipts. Browsing never resumes an external session or starts inference. Quota windows come from native account observations, with explicit stale and unavailable states. Usage reads include Codex credit balances and available earned-reset counts when reported, plus Claude five-hour and seven-day meters with their model/surface scopes and reset times.
+
+Use `sessions` for discovery and `session-evidence` for newest-first historical evidence. These reads fit within 65,536 serialized bytes, retain exact native identities and content hashes, and continue large bodies in UTF-8 chunks. Select an item, turn or item kinds with filters. Matching names remain separate sessions. Provenance distinguishes native evidence from exact host submissions; historical text and quoted feedback never grant approval. See [bounded session evidence](api/README.md#bounded-session-evidence).
+
+Trusted Node hosts can open the database described by `hq.storage` and keep application tables under their own prefix. HQ reserves `hq_`, validates its current schema stamp on startup, and leaves application schema ownership and `user_version` with the host. Use `hq.withStorageMaintenance(callback)` for host schema changes; it drains admitted work and rejects active native execution before the callback. Fresh state starts at HQ storage version 2. Published 2.0.0 state requires an explicit offline upgrade; startup never migrates. Stop HQ and independent database/plan writers, then run `jg-agent-hq upgrade-storage --state-root /path/to/state --backup-directory /private/backups/before-upgrade --workspace-root /path/to/workspace`. The backup destination must be absent under an existing private canonical parent. Keep the verified backup. Trusted Node hosts use `upgradeHqStorage(options)` for the same operation. See the [shared storage contract](api/README.md#shared-storage).
+
+`createHqSnapshot(hq, destination)` captures quiescent local state through SQLite's backup API, including host tables, stored reports, action.key and registered plan companions. Busy checkpoints return a conflict without interrupting work. `inspectHqSnapshot(directory)` verifies the complete snapshot; `restoreHqSnapshot({ snapshotDirectory, destination })` validates and stages it into a new private recovery workspace. Version 1 and version 2 snapshots remain supported. Restored version 1 storage requires the explicit offline upgrade before startup. Restored automation is paused, skill grants are revoked, native ownership needs reconciliation and allowed roots must be selected explicitly. Hosts own remote backups; a failed disk can lose changes since the last successful off-machine backup. See [snapshot and restore](api/README.md#snapshot-and-restore) and [recovery](docs/recovery.md).
+
+Provider IDs are bounded lowercase identifiers. Codex and Claude are built in; trusted Node hosts can register additional `IAgentProviderFactory` implementations through `providerFactories`. Capabilities report current operation support, provider/session scope and unavailable reasons. Native version labels are diagnostic metadata and never determine admission. Unknown essential control data fails explicitly. `jg-agent-hq/contracts` contains browser-safe schemas and types; importing it does not import the host runtime.
+
+Use `--provider claude` for Claude history and usage; the default is `codex`. Claude connects lazily through its native structured CLI, with normal settings enabled and no user prompt. Usage requests explicitly disable the optional transcript-behavior scan. Its local history reader validates required identity, workspace and chain fields, tolerates additive fields and mixed version labels, and keeps byte locations rather than conversation bodies in SQLite. Unknown ordered content is presented as unsupported; untrustworthy identity or ordering makes the affected read unavailable. File history cannot establish current execution state. See the [reader limits and freshness contract](api/README.md#claude-history).
+
+Explicit actions create, name, adopt, resume and close threads; send idle messages; steer an exact running turn; queue the next message; interrupt; and answer native questions, permissions or MCP elicitation. Actions have stable UUIDs and durable receipts. Repeating the same request returns its receipt; changing that UUID's payload conflicts. A lost acknowledgement stays uncertain until the exact native response arrives or the operator makes an explicit recovery choice. HQ never resends uncertain input.
 
-The current read contracts were exercised against Codex 0.163.0 and Claude Code versions 2.1.304 and 2.1.306 on Linux. Unsupported native responses fail explicitly. `jg-agent-hq/contracts` contains browser-safe schemas and types; importing it does not import the host runtime.
+Actions also fork native history into a distinct session, compact idle sessions, and use native archive/unarchive and history revert where supported. Forks retain exact native lineage and receive no plan execution authority from their source. Unarchive leaves a session inactive until explicitly resumed. Revert changes conversation history; it does not undo files. Claude exposes compaction and fork, with archive and conversation revert reported as unsupported.
 
-Use `--provider claude` for Claude history and usage; the default is `codex`. Claude connects lazily through its native structured CLI, with normal settings enabled and no user prompt. Usage requests explicitly disable the optional transcript-behavior scan. Its local history reader accepts only qualified transcript versions 2.1.296, 2.1.299, 2.1.304 and 2.1.306, including transcripts spanning these versions, keeps byte locations rather than conversation bodies in SQLite, and reports unsupported history as unavailable. File history cannot establish current execution state. See the [reader limits and freshness contract](api/README.md#claude-history).
+Use `account --provider <id>` for safe native account metadata, and `mcp-servers --thread-id <id>` for configured server status. `mcp-tools` and `mcp-resources` select `--server-name <name>` and support bounded continuation. Native reload and authentication controls use the existing JSON `request` interface and durable action receipts. Login URLs are live presentation and are never saved in SQLite. See [native accounts and extensions](api/README.md#native-accounts-and-extensions).
 
-Explicit actions create, name, adopt, resume and close threads; send idle messages; steer an exact running turn; queue the next message; interrupt; and answer native questions or permissions. Actions have stable UUIDs and durable receipts. Repeating the same request returns its receipt; changing that UUID's payload conflicts. A lost acknowledgement stays uncertain until the exact native response arrives or the operator makes an explicit recovery choice. HQ never resends uncertain input.
+Use `models --provider <id>` for native model and effort choices, and `configuration --provider <id> --workspace-root <path> [--thread-id <id>]` for known effective settings and paged native agent/profile metadata. Creation and explicit resume accept typed startup settings; messages can select a model and effort for the next turn. Startup instructions, native Claude agents and permission settings require an idle session restart. Capability schemas describe supported selections and change modes. Unknown settings remain explicit, and native configuration files stay with each CLI. See the [configuration contract](api/README.md#native-configuration).
 
 Native execution settings, instructions, skills and tool permissions remain with each CLI. Claude uses an owned structured process per controlled session and reports immediate steering as unavailable. Plan approval grants no native tool permission. Close other CLI controllers before confirming adoption, resume or recovery. HQ coordinates sessions sharing one state directory; it cannot lock independently running native CLIs. Integrating browser hosts can use durable host-action admission and operator-scoped reconciliation through the programmatic API; see the [browser host contract](api/README.md#browser-host-integration). The standalone CLI advertises only commands it implements.
 
-Durable state includes `action.key`, which must stay private and accompany its SQLite database. Queued undelivered text is saved; ordinary message bodies and question answers are not.
+Approval reads return indexed, cursor-paginated pending, answered and expired metadata after reconnect or restart. Sensitive presentation is available only while the exact native request is live. Native permissions require explicit responses; unknown request kinds or scopes are denied. See [native replies and reconnect reads](api/README.md#thread-control-and-live-input).
+
+Durable state includes `action.key`, which must stay private and accompany its SQLite database. Queued undelivered inputs and complete bounded final results are saved; ordinary input bodies and question answers are not.
+
+Text and skill inputs can attach bounded workspace files by path and media type. HQ verifies their content hashes again before queued delivery. Output schemas use native Codex turn parameters or Claude's structured-session flag; complete validated results and known token usage remain available on selected receipts after reconnect. Live previews stay bounded and explicitly mark truncation. See [file inputs and complete results](api/README.md#file-inputs-and-complete-results).
 
 Use `suggest-thread-name --topic "Build usage" --state-root <directory>` for an editable label and `managed-thread-name --name build-usage --state-root <directory>` to resolve it. Creation adds a short ordinal suffix when a name is occupied. Actions and relationships retain immutable IDs.
 
@@ -57,6 +75,10 @@ Use `skills --thread-id <id> --state-root <directory>` to browse the controlled
 
 Use `jg-agent-hq request --state-root <directory>` with a closed HQ command on stdin for action and control reads. The [API reference](api/README.md) describes action envelopes, receipts and reconnect reads.
 
+Trusted hosts can create collaboration tasks, add managed agents from any provider, and coordinate them through durable streams. Streams contain small indexed messages and optional directed or claimable requests. Subscription, reading, acknowledgement, assignment and completion are separate records. Metadata operations retain replay receipts and recover notification fan-out after restart. Directed requests can run through the existing native action ledger when the host enables automatic dispatch or explicitly selects a request. An assigned agent can publish direct child requests and yield with a bounded wait set. Exact child outcomes or requester clarifications continue the parent under its current authority; child completion never completes or approves the parent. Information, reads and acknowledgements do not start turns. HQ stores exact bounded outcome reports independently of native histories. Hosts issue exact task-scoped skill grants; role labels and message text grant no authority. See [task collaboration](api/README.md#task-collaboration).
+
+Controlled agents use `hq_identity` to recover their stable managed ID, task memberships, permissions and pending inbox counts. Codex uses experimental native dynamic tools; Claude uses a private session-specific stdio MCP bridge. Both use the same closed operations for streams, verified invocation context, requests, report chunks, session evidence and authorized managed spawning. Children can use either provider and inherit equal or narrower task permissions. The host sets a finite cumulative spawn budget. Native subagents share their owning HQ session identity. See [agent tools](api/README.md#agent-tools).
+
 ## Plan coordination
 
 `plan-status --directory <path>` and `plan-milestones --directory <path>` inspect a selected ordinary planning directory. No project registry is required. Mutations use the existing `request` JSON command with `kind: 'plan'`; see the [plan command reference](api/README.md#plan-coordination).
@@ -67,36 +89,46 @@ Coordinators can retain revision-bound compatibility reviews in that same file.
 
 Begin a revision before native planning. HQ stops bound implementation threads and their owned background work, captures partial-work evidence, and keeps execution paused while the CLI revises the plan and regenerates milestones. The finalizer carries exact matches automatically and asks for one confirmation covering every unmatched completed or partial milestone. Equivalent rewording preserves the original completion evidence. Changed requirements remain pending. Present and approve the canonical sequence, then explicitly authorize the new execution scope and mode. Neither approval nor restart resumes implementation. Whole-plan checkpoints and completion evidence also work without a milestone file. Removing a sequence requires explicit confirmation covering its retained milestone evidence before whole-plan execution can begin. See the [revision recovery contract](api/README.md#revision-and-progress-recovery).
 
-## Cross-provider feedback
+## Agent review and feedback
 
-Create a handoff with the JSON request interface: `kind: 'handoff'` and a `create` command containing a UUID, current `runtimeEpoch`, managed `sourceThreadId` and `purpose: 'challenge' | 'review'`. Creation opens a short-named reviewer session with the opposite provider, without sending a prompt. Preview the exact native skill invocation, then explicitly send its digest. Both CLIs keep their own skills, tool permissions and conversation histories.
+Choose any managed reviewer and author, including the same provider, and send task-scoped stream requests. Optional native skills use exact host-issued grants: jg-review-session can read the selected evidence, and jg-feedback can assess an exact report reference. HQ selects the installed native skill and verifies the invocation before dispatch. Missing or unavailable skills remain blocked; no copied workflow is substituted.
 
-Collect one completed reviewer message by its exact native turn/item reference. Preview and explicitly send it through `jg-feedback` to the original author. Feedback remains untrusted advice; delivery and native turn completion grant no plan approval, execution authority or acceptance of recommendations. Busy authors receive queued feedback through the existing action ledger. An uncertain send is never automatically retried.
-
-`handoff-show --id <uuid>` reads one handoff; `handoffs --workspace-root <path>` pages the inbox. The [handoff reference](api/README.md#cross-provider-handoffs) describes creation, preview, send and native findings collection through the same CLI's JSON request interface.
+Reports return through HQ storage and delivery without manual copying or another reviewer turn. Feedback is untrusted advice and grants no plan approval, execution authority or acceptance of recommendations. Parent work can yield for child results and continue through its existing authority. Busy agents queue; uncertainty never causes automatic resending. See [native skill delegation](api/README.md#native-skill-delegation) and [task collaboration](api/README.md#task-collaboration).
 
 ## Source examples
 
 Follow the [three complete walkthroughs](https://jg-agent-hq.jesusgraterol.dev/examples/), with copyable source, visual results and explicit host decisions.
 
-`examples/control-session.ts` creates a native session and reads its actual delivery, output, permission and usage observations. `examples/review-handoff.ts` presents exact cross-provider review/feedback previews and delivers selected findings. `examples/revise-plan.ts` runs the planning skills through the revision lease, retains verified work through explicit correspondence, and obtains fresh approvals and execution scope.
+`examples/control-session.ts` creates a native session and reads its actual delivery, output, permission and usage observations. `examples/review-handoff.ts` confirms stream review/assessment requests and returns exact report references, with optional skills and host-selected agents. `examples/revise-plan.ts` runs the planning skills through the revision lease, retains verified work through explicit correspondence, and obtains fresh approvals and execution scope.
 
 These complete TypeScript examples use the public client and contracts. Start the local controller, sign in to the native CLIs, and install the required skills and supported project-local jg-plans CLI first. The host supplies actual user decisions and keeps native permission handling separate. Retain request identities before submission; after an interrupted client call, inspect those receipts instead of rerunning a whole example. Examples are development sources, not package exports or automatic website actions.
 
+`examples/shared-storage.ts` uses a quiescent trusted Node runtime to save a singleton application preference under the app\_ prefix, capture and inspect shared SQLite storage, and verify the value after offline restore. It closes its own SQL connections and preserves the host's user_version. Select absent output directories and pause every other writer first. The application write commits before capture; a later backup failure does not roll it back. The recovered state still requires explicit allowed roots and fresh authority before execution.
+
+The [qualification matrix](api/README.md#qualification) records native journeys, host coverage and remaining interface limits.
+
 ## Project blueprint
 
 - `src/index.ts`: the explicit public package boundary.
+- `examples/`: public source helpers for sessions, stream review, plan revision and shared host storage.
 - `src/locks/`: lock ownership, acquisition, status and deliberate recovery.
 - `src/runtime/`: trusted configuration and lifetime ownership.
 - `src/ipc/` and `src/cli/`: private local commands and the installed executable.
 - `src/providers/codex/`: native history and account protocol.
-- `src/providers/claude/`: native structured control and usage, plus the version-checked, worker-owned local history reader.
-- `src/sessions/`: authorized, cursor-bound history reads and short label suggestions.
+- `src/providers/claude/`: native structured control and usage, plus the identity-validated, worker-owned local history reader.
+- `src/providers/registry/`: lazy factories, normalized provider evidence, bounded lifecycle and source isolation.
+- `src/providers/inputs/` and `src/providers/results/`: checked file delivery and complete native result validation.
+- `src/providers/configuration/` and `src/providers/lineage/`: bounded native choices and confirmed immutable session relationships.
+- `src/sessions/`: authorized discovery, bounded evidence and provenance, cursor-bound history reads and short label suggestions.
 - `src/plans/`: execution digests, milestone finalization, revision checkpoints, retained verification and explicit plan authority.
 - `src/skills/`: bounded native catalogs, invocation previews and explicit input preparation.
-- `src/handoffs/`: explicit cross-provider challenges, selected native findings and author assessment links.
+- `src/accounts/` and `src/extensions/`: safe account observations and authorized, bounded native MCP inventories.
+- `src/collaboration/`: host-owned tasks, streams, scoped skill-grant admission and event-driven directed request dispatch.
+- `src/collaboration/identity/` and `src/agent-tools/`: native-bound identity, private bridge credentials, shared agent tools and the stdio MCP entry.
 - `src/actions/`: durable action dispatch, native input routing and bounded live presentation.
-- `src/store/`: SQLite discovery metadata and the action/control ledger in one worker.
+- `src/store/`: SQLite discovery metadata, collaboration streams and the action/control ledger in one worker.
+- `src/storage/`: trusted host metadata, exclusive maintenance, schema lineage and offline upgrades.
+- `src/snapshots/`: quiescent capture, streamed inventory validation and offline recovery with execution paused.
 - `src/contracts/`: browser-safe command and response contracts.
 - `scripts/package/`: build-output cleanup and packed-consumer checks.
 - `scripts/release/`: registry version/integrity checks and trusted publication of the tested archive.
@@ -106,6 +138,8 @@ These complete TypeScript examples use the public client and contracts. Start th
 - `errors/README.md`: stable application errors.
 - `website/`: private public-site workspace with its own README and test/check commands; canonical documentation bodies remain in this root README and `api/README.md`.
 - `docs/website.md`: durable website architecture, verification and generated-asset sources.
+- `docs/architecture.md`: provider and database ownership, schema initialization and host maintenance.
+- `docs/recovery.md`: local snapshot coverage, offline recovery and the host's remote durability responsibilities.
 
 `/docs` is reserved for concise, quickly scannable documentation of essential durable concepts and processes. API references remain outside `/docs`.