nexus-ai-pro 2.8.0 → 2.10.0

package/API_STABILITY.md

@@ -41,6 +41,83 @@ Both are covered by this policy:
 Deep imports into `dist`, `dist-cjs`, or `src` are not supported. Import only from `nexus-ai-pro` or
 one of its explicit subpaths.
 
+## 2.10 stage (2.10.0)
+
+Everything 2.10 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- on `nexus-ai-pro/protocols/a2a`: `A2aHandler`, which `a2aHandler()` now returns, with `owner`,
+  `recover()`, and `close()`; `A2aRecoveryReport`; `A2aTaskLease` and `A2aTaskRun`, and
+  `A2aTaskRecord.lease`, `run`, and `version`; `A2aTaskStore.claim()`, optional;
+  `A2aServerOptions.lease` and `recoverEveryMs`. The handler is still the request function it was;
+- on `nexus-ai-pro/protocols/acp`: `AcpTurnRecord` and `AcpSessionRecord.turn`;
+- on `nexus-ai-pro/deep-agent`: `DeepSubagent.remote`, `DeepRemoteHelper`, and `DeepRemoteRunResult`;
+- `CompiledGraph.continueWith()`, and `ProtocolGraph.continueWith()` and `state()`, both optional;
+- on `nexus-ai-pro/lifecycle`: `ProviderHealthOutcome`, `OperationLifecycle.reportHealth()`,
+  `OperationLifecycleLike.reportHealth()`, optional, and `LifecycleRuntime.providerHealth`;
+- on `nexus-ai-pro/realtime`: `RealtimeSessionConfig.tenantId`, `userId`, `principal`,
+  `traceContext`, `conversationStore`, and `resume`; `RealtimeConversationOptions.replayItems`;
+  `RealtimeConversationStore`, `MemoryRealtimeConversationStore`, `storeRealtimeConversations()`,
+  `replayRealtimeConversation()`, and `REALTIME_REPLAY_PREFIX`; `GeminiLiveProvider` and
+  `GeminiLiveProviderOptions`; `RealtimeSession.sendImage()` and `sendVideoFrame()`,
+  `RealtimeSessionConfig.video`, `RealtimeImageInput`, `RealtimeImageOptions`, and
+  `RealtimeVideoOptions`; `RealtimeTransport.sendVideo()`, optional;
+- `RealtimeConversationOptions.retainRawAudio`, `RealtimeConversationReducerOptions.inPlace`, and
+  `RealtimePeerConnectionLike.addTransceiver()`, optional;
+- `nexus-ai-pro/realtime/gemini-live`, with `GeminiLiveTransport` and `GeminiLiveTransportOptions`;
+- `nexus-ai-pro/realtime/openai-sip`, with `answerOpenAISipCalls()` and its types, also on
+  `nexus-ai-pro/realtime`;
+- on `nexus-ai-pro/realtime/openai-server` and `nexus-ai-pro/realtime`: `acceptOpenAIRealtimeCall()`,
+  `rejectOpenAIRealtimeCall()`, `referOpenAIRealtimeCall()`, `hangupOpenAIRealtimeCall()`,
+  `verifyOpenAIWebhook()`, `OpenAIRealtimeCallOptions`, `OpenAIWebhookEvent`, `OpenAIWebhookHeaders`,
+  and `OpenAIWebhookVerifyOptions`; `OpenAIWebSocketTransportOptions.callId`.
+
+The realtime family stays experimental, and all of these with it.
+
+Changed, each to keep a promise the surface already made:
+- A realtime session's raw events keep an audio event's size, `delta_bytes`, in place of its audio,
+  unless `retainRawAudio`: a session held every frame it played, about 13 MB a 3-minute call. The
+  `openai-events` export follows.
+- A realtime session announces `conversation.updated` only to a listener, as before, but no longer
+  copies its record for each event when none is listening.
+- A realtime session given a lifecycle runs each turn as a `realtime.turn` operation and charges it
+  as it finishes; its `realtime.session` operation reports the total and charges nothing more. Its
+  usage and cost are the sum of its turns', where each turn's replaced the last before.
+
+## 2.9 stage (2.9.0)
+
+Everything 2.9 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- `OperationFamily` gains `video`;
+- `nexus-ai-pro/video`, `/video/google`, `/video/runway`, and `/video/mock` are new and experimental:
+  video generation graduates by the checklist once Veo and Runway pass its conformance suite live;
+- `RunQuery.after` with `RunCursor`, and `runCursor()` and `isAfterCursor()` on
+  `nexus-ai-pro/tracing`. A store of your own that does not honour `after` still works with the
+  online-evaluation worker, which falls back to offsets;
+- `OnlineSchedulePass.paging`;
+- on `nexus-ai-pro/lifecycle`: `BudgetReservation.id`; `BudgetReservationStatus`,
+  `BudgetReservationRecord`, `BudgetReservationOutcome`, `BudgetRecoveryReport`, and
+  `BudgetRecoverOptions`; `BudgetLedger.renew()`, `settle()`, and `recover()`, all optional;
+  `BudgetLedgerOptions.leaseMs` and `uncertain`; `BudgetLedgerWithReport.reservation()`; and the five
+  optional reservation methods of `BudgetUsageStore` with `BudgetReservationChange`. The statuses,
+  and which outcome moves a reservation from which status, are part of the promise;
+- `hold()`, `settle()`, `renew()`, `lapsed()`, and `reservation()` on `MemoryTenantUsage`,
+  `RedisTenantUsage`, and `PostgresTenantUsage`, and the tenancy migration version 2 with its
+  reservations table;
+- on `nexus-ai-pro/media/rag`, experimental with the media family: `MediaIndexOptions.manifest` and
+  `name`, `MediaIndexEntry`, `MediaIndexReader`, `MediaIndexReport.unchanged`, and
+  `MediaIndex.delete()`, `reindex()`, `describe()`, and `list()`.
+
+Changed, each to keep a promise the surface already made:
+- `TraceStore.query()` orders runs that share a start time by id, from last to first, as code units,
+  so a page means the same on every store. Before, that order was not defined.
+- `MemoryBudgetUsage`, the ledger's default store, records reservations, so `reserve()` returns one
+  with an `id`. An online-evaluation worker reserves every run of its evaluators, a run after a crash
+  included, where before it reserved only a first.
+- A deep agent with a checkpointer writes the helpers it creates to `<thread>/helpers-created`, so
+  `createHelpers.max` holds for a thread across processes, as it says.
+
 ## 2.8 stage (2.8.0)
 
 Everything 2.8 adds is stable from the start and additive, unless this section says otherwise.

package/CHANGELOG.md

@@ -4,6 +4,248 @@ Notable changes to this project are recorded here. The format follows [Keep a Ch
 
 ## [Unreleased]
 
+## [2.10.0] - 2026-10-16
+
+Durable conversations, and a second realtime provider.
+
+### Added
+
+- **A2A tasks that outlive their replica.** A working task's record now holds an `A2aTaskLease` and
+  an `A2aTaskRun`. The lease names the replica running the task, which renews it every third of
+  `lease.ms` (30 seconds); the run says what the task was asked to do. `a2aHandler()` returns an
+  `A2aHandler`: the request handler, plus `recover()` and `close()`. `recover()` takes over working
+  tasks whose lease lapsed, and carries each run on in one of three ways:
+  - from the graph's last checkpoint;
+  - by answering its questions again, when a resume never reached the graph;
+  - from the start, when the run left no checkpoint.
+
+  `recoverEveryMs` runs it on a timer. `A2aTaskStore.claim()` takes a lease in one step, so two
+  replicas recovering at once take a task once. `MemoryA2aTaskStore` has it, and `storeA2aTasks()`
+  builds it on `putIfVersion()`. A replica that finds its lease taken stops its run, and never
+  records its result over the other's.
+- **ACP turns that outlive the agent.** While a prompt runs, its session record holds an
+  `AcpTurnRecord`. When the editor reopens the session with `session/load`, the agent carries the
+  turn on, streaming its updates: from the graph's last checkpoint, by answering again, or from the
+  start. A turn stopped while the editor was deciding an approval asks for it again.
+- **Realtime on the client's lifecycle.** A session given `ai.lifecycle` runs as `realtime.session`,
+  and each model turn as `realtime.turn`, carrying the session's `tenantId`, `userId` (the
+  `principal`'s by default), and `traceContext`. A turn is admitted as it starts, rate-limited,
+  authorized, and holding the average turn cost against the budget, and charged its cost when it
+  finishes. A turn the lifecycle refuses is cancelled before the model runs it, and the session ends
+  with `lifecycle_refused`. Connections are reported to the client's provider health and circuit
+  breaker through the new `reportHealth()`, on `OperationLifecycleLike` and `OperationLifecycle`,
+  with `ProviderHealthOutcome` and `LifecycleRuntime.providerHealth`.
+- **Realtime conversations that outlive their process.** `conversationStore`, a
+  `RealtimeConversationStore`, keeps the record after each committed turn:
+  `MemoryRealtimeConversationStore`, or `storeRealtimeConversations()` on any `Store`. `resume`
+  carries a conversation on, from its record or its id: same id, items, metrics, and metadata. On
+  connecting, the provider is given what was said and what the tools returned before anything new,
+  through `replayRealtimeConversation()`. Echoes carry `REALTIME_REPLAY_PREFIX` and are not recorded
+  twice. `conversation.replayItems` caps the replay.
+- **Gemini Live, behind the same realtime session.** `GeminiLiveTransport`, on the new
+  `nexus-ai-pro/realtime/gemini-live`, speaks Gemini's Live API, and `GeminiLiveProvider`, on
+  `nexus-ai-pro/realtime`, creates sessions on it for `createRealtimeAgent()`. The session's
+  configuration becomes Gemini's setup; audio, messages, and tool results become its realtime input,
+  turns, and tool responses; and its audio, transcripts, tool calls, interruptions, and usage become
+  the events the session already reads. It takes an API key on a server or an ephemeral token
+  anywhere, and reconnects a dropped socket with Gemini's latest resumption handle.
+- **Helpers on another server, whose runs outlive their worker.** A `DeepSubagent` with `remote`, a
+  `DeepRemoteHelper` such as `createRemoteGraph()` returns, runs on another agent server, as a run
+  leased to the worker that runs it: when that worker stops, another takes it over. The agent starts
+  the run under a key made from the call that started it, so delegating again or collecting it, in
+  this process or another, finds the same run; a background task is marked on its thread first, so a
+  run that never reached the server is started by the process that collects it. Its answer comes from
+  a `DeepRemoteRunResult`.
+- **Phone calls answered by a realtime agent, over SIP.** `answerOpenAISipCalls()`, on the new
+  `nexus-ai-pro/realtime/openai-sip`, handles OpenAI's `realtime.call.incoming` webhook: it verifies
+  each delivery, claims it so a retry answers its call once, and answers the call with the session
+  `answer()` chooses from the caller, which attaches to the call and runs it with its tools,
+  record, and lifecycle; or declines it with a SIP status. `OpenAISipCall` transfers and hangs up.
+  The call control is on `nexus-ai-pro/realtime/openai-server` too: `acceptOpenAIRealtimeCall()`,
+  `rejectOpenAIRealtimeCall()`, `referOpenAIRealtimeCall()`, `hangupOpenAIRealtimeCall()`, and
+  `verifyOpenAIWebhook()`, on Web Crypto. `OpenAIWebSocketTransportOptions.callId` attaches a
+  transport to a call.
+- **A published method and budget for realtime turn latency and session load.** `npm run
+  bench:realtime`, in the release gate, holds what a session adds to a message (under 5 ms at p99 for
+  one call, under 20 ms with 250 calls at real time), that a turn late in a long call costs what an
+  early one does, that 250 calls at real time keep up, the heap each session holds, and that ended
+  sessions leave nothing. Budgets are in `realtime-budget.json`.
+- **Realtime over real WebRTC in a browser.** `npm run test:browser:realtime`, in `npm test`, drives a
+  session in headless Chromium against a peer connection in the page that plays the provider: real
+  SDP, ICE, DTLS, and data channel, the microphone up, the model's voice down, a tool call, and a
+  session with no microphone.
+- `retainRawAudio` on a realtime conversation's options, `inPlace` on
+  `RealtimeConversationReducerOptions`, and `addTransceiver()`, optional, on
+  `RealtimePeerConnectionLike`.
+- **Video and images in a realtime session.** `sendVideoFrame()` streams camera or screen frames:
+  as realtime video on Gemini Live, as images in the conversation elsewhere, spaced by
+  `RealtimeVideoOptions.minIntervalMs`. `sendImage()` adds an image, with text, to the conversation.
+  `RealtimeTransport.sendVideo()`, optional, is what a transport with video input implements.
+- `continueWith()` on a compiled graph: `continue()`, returning the final result.
+  `ProtocolGraph.continueWith()` and `state()`, both optional, are what A2A and ACP carry a run on
+  with.
+
+### Fixed
+
+- The WebRTC transport trimmed the SDP answer's final line break, and Chromium refuses a description
+  whose last line has none, so a session in Chrome or Edge never connected.
+- A WebRTC session that sent no microphone offered no audio to receive, so it never heard the model.
+- A realtime session copied its whole conversation record twice for every event, so each event cost
+  more as the call went on: a turn took 39 ms by the 500th, and 250 calls took half a core. It now
+  applies events in place and copies the record only for a `conversation.updated` listener.
+- An A2A `SendMessage` with `returnImmediately`, whose store failed while the task ran, raised an
+  unhandled rejection, which ends a Node process by default. A store that cannot record a task's
+  failure now leaves the task working, with its lease, for the replica that takes it over.
+- A realtime session's usage and estimated cost are the sum of its responses'. Before, each
+  response's usage replaced the last, so a session recorded, and settled, only its last turn.
+
+### Proof
+
+- `tests/conversation-crash.test.ts`: an A2A task, an ACP turn, and a realtime conversation are each
+  killed at every write a clean run makes, and taken over elsewhere. Each ends as the clean run did,
+  or its caller was told it never started; nothing is left half done, and nothing is recorded twice.
+  The suite found three defects before they shipped, each fixed: an unhandled rejection from a
+  failing store; a task submitted to a replica that stopped before starting it, which no replica
+  would ever start; and a reopened session's save that dropped the turn it was about to carry on.
+- `tests/deep-agent-remote-helpers.test.ts`: against two replicas of a real agent server, the agent's
+  process and the worker running its remote helper both die; the second replica takes the run over,
+  and the agent, resumed in a new process, collects that same run: one run, its answer, and the
+  agent's note not written again. Without the key, the resumed agent starts a second run, and the
+  proof fails.
+- `tests/realtime-sip.test.ts`: a signed incoming-call delivery is accepted with the agent's session,
+  chosen from the caller's number; the session attaches by the call's id, books a table through its
+  tool when the caller asks, and hangs up. Forged, altered, and stale deliveries answer nothing; a
+  retry after a failed accept answers the call, and a duplicate, on one replica or two, answers it
+  once. Without the signature check, the claim, or the release, a test fails.
+- `tests/realtime-gemini.test.ts`: one agent definition (instructions, a tool, conversation settings)
+  runs on OpenAI's realtime API and on Gemini Live, each over its own scripted wire protocol. The two
+  records match item for item: what the person asked, the tool call and its result, and what the model
+  answered, with the same usage, and the tool runs once on each. Without the tool response, or with
+  the request for a response not dropped after it, the proof fails.
+- `tests/realtime-lifecycle.test.ts`: through a real client with a five-cent budget, two
+  two-cent turns run, and the third is cancelled as it starts and ends the session; the budget holds
+  four cents, and every operation names the tenant and user. Connection failures open the client's
+  circuit. Without the admission at the turn's start, the proof fails.
+- `tests/realtime-resume.test.ts`: a session's process is killed mid-conversation on the memory and
+  SQLite stores. Another resumes it by id on a new transport, and gives the provider what was said
+  before anything new. The conversation goes on under its own id with nothing recorded twice.
+  Without the replay, or without skipping the echoes, the proof fails.
+- `tests/a2a-takeover.test.ts`: a task whose replica dies halfway through a tool is taken over, on
+  the memory, SQLite, Postgres, and Redis stores. The model call the dead replica finished is not
+  made again, and the email it never sent is sent once. Two replicas recovering at the same moment
+  take the task over once, and a lease renewed while a slow task works is never taken. Without the
+  lease check, or carried on from the start instead of the checkpoint, the proof fails.
+- `tests/acp-turns.test.ts`: an agent stopped halfway through a tool is started again and its
+  session reopened; the turn goes on without repeating the finished model call, and sends the email
+  once. A turn stopped during an approval asks again. Without the recovery on load, the proof fails.
+
+## [2.9.0] - 2026-10-15
+
+Video, and everything shipped, proven.
+
+### Added
+
+- **Video generation.** `nexus-ai-pro/video`: `VideoManager` makes videos from a prompt, a first or
+  last frame, or reference images, through a provider's asynchronous job. `generate()` is a durable
+  operation that records the job the moment it exists and each video as it is stored, so
+  `recover()` in another worker collects a job whose process died, never submitting it twice, and
+  streams each video into an asset store. Each generation runs in the lifecycle's new `video`
+  family, and is priced per second by `priceVideo()` from `VIDEO_PRICES`, the Veo 3.1 list prices,
+  or the account's own. `start()`, `resume()`, `status()`, `download()`, and `cancel()` work from a
+  JSON `VideoJobRef` in any process. A request is checked against the model's capabilities before
+  any job, and a failed job is submitted again only when the provider says that may help.
+  - `GoogleVideoProvider` (`/video/google`): Veo through its long-running predictions, on the
+    Gemini API, the video read from Google's URI with the key, or on Vertex AI, read from Cloud
+    Storage or inline.
+  - `RunwayVideoProvider` (`/video/runway`): Runway's task API, for its own models and the hosted
+    ones it serves, Veo, Gemini Omni Flash, and Seedance among them; each video read from its signed
+    URL without the secret.
+  - `MockVideoProvider` (`/video/mock`), and `runVideoConformance()`, which all three pass.
+
+  New guide `docs/video.md`. The family is experimental until Veo and Runway pass the conformance
+  suite live. OpenAI's Videos API was removed on 2026-09-24, so there is no OpenAI adapter.
+- **The experimental families, live.** `npm run test:families:live`, and a weekly *Family
+  conformance* job beside the vector stores: media understanding's conformance suite against an
+  OpenAI or Gemini vision model and Mistral's OCR, on a PDF and a PNG the script builds; a two-input
+  batch through the Gemini embedding batch tier, waited on for 20 minutes and cancelled past that;
+  and, when the repository variable `NEXUS_LIVE_VIDEO` is set, the video conformance suite on Veo 3.1
+  Lite and Runway. Each check runs where its key is set, and what it measured is kept. These are the
+  runs media, the Gemini batch tier, and video graduate on.
+- **Paging by cursor.** `RunQuery.after` takes a `RunCursor`, the start time and id of a page's last
+  run, and returns only the runs after it; `runCursor()` takes the cursor from a run, and
+  `isAfterCursor()` is the test on its own, for a trace store of your own. Every bundled store
+  honours it, and `applyQuery()` does for a store that filters in memory. A new Postgres trace store
+  migration, version 3, indexes runs in query order.
+- **Budget reservations that outlive their process.** Over a store that records reservations, a
+  `budgetLedger()` holds each one as a `BudgetReservationRecord` under a lease (`leaseMs`, 10
+  minutes), and moves it with its total in one step: `reserved`, then `settled`, `released`,
+  `expired`, or `uncertain`. `recover(resolve)` settles reservations whose lease lapsed by what
+  became of each operation, a `BudgetReservationOutcome`; `settle(id, outcome)` does it for one from
+  any process; `renew()` extends a lease; and `uncertain: 'charge' | 'release'` says what an unknown
+  outcome costs. `MemoryBudgetUsage`, `MemoryTenantUsage`, `RedisTenantUsage`, and
+  `PostgresTenantUsage` all record them: Redis keeps a reservation in its total's Cluster slot, so one
+  script moves both, and Postgres does it in one statement, in the reservations table of a new
+  tenancy migration, version 2. A store of your own records them through five optional methods on
+  `BudgetUsageStore`.
+- **A media index that remembers what it holds.** `MediaIndexOptions.manifest`, any `Store`, with a
+  `name`, keeps one `MediaIndexEntry` per medium: a hash of its content, source, metadata, and
+  indexing options, where it is kept, its pages, its chunk ids, the embedding model, and the
+  `MediaIndexReader` that read and described it. `add()` then skips a medium at the same version and
+  replaces exactly the chunks and asset of one that changed, and `MediaIndex` gains `delete()`,
+  `reindex()`, `describe()`, and `list()`. `MediaIndexReport.unchanged` counts what was skipped. A
+  change is recorded as pending first, so a crash leaves nothing the next call cannot finish.
+
+### Changed
+
+- Runs that started in the same millisecond come by id, from last to first, compared as code units,
+  on every trace store. The order of such runs was not defined before, and the memory and Postgres
+  stores disagreed on it.
+
+### Fixed
+
+- An online-evaluation worker reads a backlog of runs that started in the same millisecond through,
+  however many pages it spans. Before, it resumed each page at the last run's start time, so a page
+  that ended among such runs was read again on every pass, and `drain()` never finished. On a trace
+  store of your own that does not honour `RunQuery.after`, the worker pages by offset instead, and
+  its `OnlineSchedulePass` says `paging: 'offset'`.
+- A budget reservation held by an online-evaluation worker that dies no longer stays charged for the
+  period. The worker records the reservation just before it calls its evaluators, and whoever takes
+  over settles it: charged at what the scores cost if they were written, given back if the
+  evaluators never ran, and by the ledger's `uncertain` policy if they ran and wrote nothing.
+- An evaluation run again after a crash is reserved and charged. Its evaluators run again to find
+  the scores still missing, and before, only a first run drew on the budget.
+- A deep agent resumed after a crash counts the helpers it created before. With a checkpointer, each
+  created helper is recorded, with its instructions, on `<thread>/helpers-created` before the call
+  that created it returns, so the resumed agent cannot pass `createHelpers.max` and delegates to them
+  again by name. Before, the count was kept by the process, and a resumed agent could create `max`
+  more and no longer found the ones it had made.
+
+### Proof
+
+- `tests/video.test.ts`: the mock, Veo on the Gemini API and Vertex AI, and Runway pass the video
+  conformance suite against stand-ins for their APIs. A request a model cannot make is refused
+  before any job. A generation runs through the lifecycle, streams three megabytes into a
+  filesystem store, and is priced per second. A generation whose process dies after submitting, or
+  after storing its video, is collected by another worker with one submission and one stored video.
+  If the recorded job is ignored, the job is submitted twice.
+- `tests/online-evaluation-paging.test.ts`: the memory, JSONL, and Postgres trace stores order 5,000
+  runs, 4,000 of them in one millisecond, identically, and page through them by cursor without a
+  repeat or a gap. Ten workers reading 1,000 a page schedule each of them exactly once on every
+  store, and on a store that ignores the cursor. The previous worker never reads past the burst.
+- `tests/budget-reservations.test.ts`: on the memory, tenant memory, Redis, and Postgres stores a
+  reservation moves with its total, is settled from another process by its operation's outcome, and
+  of two processes settling it, one changes the total. An online-evaluation worker killed while
+  holding a reservation, before its evaluator, during it, or after its scores, leaves the budget at
+  what the judge billed once another worker takes over. With recovery switched off, the same test
+  finds the dead worker's estimate still charged.
+- `tests/media-rag.test.ts`: with a manifest, an unchanged medium is neither read nor written, a
+  changed one keeps exactly its chunks and one asset, and a deleted one leaves none, with a crash at
+  each write of a first index, a change, and a delete. Without the pending record, the crash proof
+  fails.
+- `tests/deep-agent-helpers.test.ts`: an agent that created its last allowed helper, killed and
+  continued in another process, is refused a second and delegates to the first again by name. Without
+  the record read back, the second is created.
+
 ## [2.8.0] - 2026-10-14
 
 Retrieval at production scale, and media that is understood.
@@ -2452,7 +2694,9 @@ shared policy vocabulary is in place for a future release that revisits this.
 - URL fetching rejects unsafe private-network targets and limits response reads.
 - CLI and audit findings no longer disclose detected secret values.
 
-[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.8.0...HEAD
+[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.10.0...HEAD
+[2.10.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.9.0...v2.10.0
+[2.9.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.8.0...v2.9.0
 [2.8.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.7.0...v2.8.0
 [2.7.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.6.0...v2.7.0
 [2.6.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.5.0...v2.6.0

package/README.md

@@ -156,6 +156,7 @@ from the doc comments. The guides live in the repository, so these links go to G
 | [Retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) | Hybrid search, reranking, MMR, parent documents, and Redis, Pinecone, Weaviate, and Chroma stores |
 | [Embeddings and retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md) | Embeddings as a routed operation family, and RAG helpers |
 | [Media understanding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md) | Reading documents and images page by page, describing images, and answering questions about them, through any vision model or OCR |
+| [Video generation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/video.md) | Videos from a prompt or a first frame, on Google Veo and Runway, as durable jobs streamed into an asset store and priced per second |
 | [Images (experimental)](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md) | Generation and masked edits over three backends, input safety, moderation, asset stores |
 | [Voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md) | Transcription, speech, voice turns, and voice sessions |
 | [Realtime voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md) | Browser and server realtime sessions with barge-in, tools, and exports |
@@ -217,11 +218,13 @@ interface, or enum, has no doc comment, and `npm run docs:guides` fails when a g
 every export of the entry points it covers, or its generated reference is stale. Both hold at 100%;
 `--list` names what is missing, and `npm run docs:update` regenerates the references.
 
-Real provider conformance, and the vector store contract against real servers, are opt-in:
+Real provider conformance, the vector store contract against real servers, and the experimental
+families against their live services are opt-in:
 
 ```bash
 npm run test:conformance:real
 npm run test:vectors:live
+npm run test:families:live
 ```
 
 ## Roadmap
@@ -285,12 +288,25 @@ Shipped so far:
   Chroma stores out of experimental after their checklist ran against the real servers; a shared
   embedding cache; images, audio, and documents as embedding input; a half-price embedding batch tier
   that ingestion survives a crash through; streaming asset stores; registry data you supply and
-  refresh; and reading, describing, and searching media, with answers given the media themselves.
+  refresh; and reading, describing, and searching media, with answers given the media themselves;
+- 2.9 video, and everything shipped, proven: videos from a prompt or a first frame on Google Veo and
+  Runway, as durable jobs another worker collects after a crash, streamed into an asset store and
+  priced per second; online evaluation that reads every run however many share a millisecond, and
+  budget reservations a dead worker leaves settled by what its call did; a deep agent's helper limit
+  across processes; a media index that remembers what it holds; and the experimental families run
+  against their live services every week;
+- 2.10 durable conversations, and a second realtime provider: A2A tasks, ACP turns, and realtime
+  sessions taken over by another worker when theirs dies, each proven killed at every write; realtime
+  on the client's lifecycle; Gemini Live behind the same session, with video and images; phone calls
+  answered over SIP; helpers on another server whose runs outlive their worker; and realtime over
+  real WebRTC in a browser, with published budgets for turn latency and session load.
 
 Upgrading from 1.x: `npx nexus migrate src --write` moves your imports, and
 [MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers the rest.
-Next, Pinecone, images, the Gemini embedding batch tier, and media understanding graduate once the
-live workflow passes against each. The
+Next, 2.11 separates where an agent's files live from where its code runs, with workspaces, and
+completes telephony: a second provider, messaging, recording, conference, and transfer. Pinecone,
+images, the Gemini embedding batch tier, media understanding, and video graduate once the live
+workflow passes against each. The
 [roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists what comes after, in the order
 applications ask for it.
 

package/dist/batch/poll.d.ts

@@ -1,6 +1,6 @@
 import { type BatchJobState } from '../types/batch.js';
-/** How a batch is waited on. */
-export interface BatchPollOptions {
+/** How a batch, or any job with a status, is waited on. */
+export interface BatchPollOptions<State = BatchJobState> {
     /** The first wait between polls, in milliseconds. */
     intervalMs: number;
     /** The longest wait, once backoff has grown it. */
@@ -8,11 +8,15 @@ export interface BatchPollOptions {
     /** Stops waiting. The provider-side batch keeps running. */
     signal: AbortSignal;
     /** Called with each state that is not yet final. */
-    onState?: (state: BatchJobState) => void;
+    onState?: (state: State) => void;
+    /** Whether a state is final. Defaults to a batch's terminal statuses. */
+    settled?: (state: State) => boolean;
 }
 /**
  * Polls a batch until it reaches a status it never leaves. A batch measured in hours does not deserve
  * a fixed short interval: the wait grows by half each time up to the maximum, which keeps the poll
  * count sane without delaying a fast batch, which the first few short polls still catch.
  */
-export declare function pollUntilSettled(poll: () => Promise<BatchJobState>, options: BatchPollOptions): Promise<BatchJobState>;
+export declare function pollUntilSettled<State extends {
+    status: string;
+} = BatchJobState>(poll: () => Promise<State>, options: BatchPollOptions<State>): Promise<State>;

package/dist/batch/poll.js

@@ -1,8 +1,9 @@
 import { isTerminalBatchStatus } from '../types/batch.js';
 export async function pollUntilSettled(poll, options) {
+    const settled = options.settled ?? ((state) => isTerminalBatchStatus(state.status));
     let interval = options.intervalMs;
     let state = await poll();
-    while (!isTerminalBatchStatus(state.status)) {
+    while (!settled(state)) {
         if (options.signal.aborted)
             throw options.signal.reason ?? new Error('aborted');
         options.onState?.(state);

package/dist/core/lifecycle.d.ts

@@ -1,5 +1,5 @@
 import type { RateLimitConfig } from '../types/config.js';
-import type { LifecycleConfig, LifecycleStage, OperationDescriptor, OperationLifecycleLike, OperationResultInfo, OperationStartOptions, OperationTicket, ProviderCallContext } from '../types/lifecycle.js';
+import type { ProviderHealthOutcome, LifecycleConfig, LifecycleStage, OperationDescriptor, OperationLifecycleLike, OperationResultInfo, OperationStartOptions, OperationTicket, ProviderCallContext } from '../types/lifecycle.js';
 import type { AuditLogger } from '../ops/audit-logger.js';
 import type { MetricsCollector } from '../ops/metrics.js';
 import { type RateLimiter } from '../ops/rate-limiter.js';
@@ -45,6 +45,11 @@ export interface LifecycleRuntime {
     config?: LifecycleConfig;
     /** Default milliseconds an operation may run before its signal aborts. */
     timeoutMs?: number;
+    /** Where `reportHealth()` goes: the client's provider health and circuit breaker. */
+    providerHealth?: {
+        recordSuccess(provider: string, latencyMs: number): void;
+        recordFailure(provider: string, error: unknown): void;
+    };
 }
 /** The family's part of one operation: what each stage does for it. Every stage but `execute` is optional. */
 export interface OperationPlan<T> {
@@ -99,6 +104,11 @@ export declare class OperationLifecycle implements OperationLifecycleLike {
     }, plan: OperationPlan<T>, options?: OperationStartOptions & {
         audit?: Record<string, unknown>;
     }): Promise<T>;
+    /**
+     * Reports how a provider answered a call a family made itself, such as a realtime connection, to the
+     * client's provider health and circuit breaker.
+     */
+    reportHealth(provider: string, outcome: ProviderHealthOutcome): void;
     /** True when nothing is configured, so a caller may skip work that only feeds the lifecycle. */
     get inert(): boolean;
     private checkRateLimit;

package/dist/core/lifecycle.js

@@ -91,6 +91,15 @@ export class OperationLifecycle {
             throw error;
         }
     }
+    reportHealth(provider, outcome) {
+        const health = this.runtime.providerHealth;
+        if (!health || !provider)
+            return;
+        if (outcome.ok)
+            health.recordSuccess(provider, outcome.latencyMs);
+        else
+            health.recordFailure(provider, outcome.error);
+    }
     get inert() {
         const config = this.runtime.config;
         return (!this.runtime.metrics?.enabled &&

package/dist/core/nexus.js

@@ -75,6 +75,10 @@ export class NexusAI {
             rateLimiter: this.rateLimiter,
             rateLimit: this.config.rateLimit,
             config: this.config.lifecycle,
+            providerHealth: {
+                recordSuccess: (provider, latencyMs) => this.attemptHooks.onAttemptSuccess(provider, latencyMs),
+                recordFailure: (provider, error) => this.attemptHooks.onAttemptFailure(provider, error),
+            },
         });
         const familyRuntime = {
             metrics: this.metrics,

package/dist/deep-agent/helpers.d.ts

@@ -6,6 +6,14 @@
  * its checkpoints to the deep agent's checkpointer. A crash leaves the helper's finished steps behind
  * it: collecting the task in a new process carries the helper on from its last checkpoint, and a call
  * the agent repeats after the crash finds the same thread instead of starting the helper again.
+ *
+ * The helpers an agent creates are recorded on a thread of their own beside its own, before the call
+ * that creates one returns, so an agent resumed in another process counts them against its `max` and
+ * delegates to them again by name.
+ *
+ * A remote helper runs on another agent server, as a run that server's workers lease: when the worker
+ * running it stops, another takes it over. The agent starts it under a key made from the call that
+ * started it, so delegating again or collecting it, in this process or another, finds that same run.
  */
 import { type CreateAgentOptions } from '../agent/create-agent.js';
 import type { PermissionPolicyLike } from '../agent/permissions.js';
@@ -25,6 +33,33 @@ export interface DeepHelperLimits {
     /** How many helpers the agent may create in one thread. Defaults to 5. */
     max?: number;
 }
+/** How a run on another agent server ended, as `createRemoteGraph()` reports it. */
+export interface DeepRemoteRunResult {
+    /** The run's id on that server. */
+    runId: string;
+    /** Where it ended: `succeeded`, `failed`, `cancelled`, `expired`, or `awaiting_input`. */
+    status: string;
+    /** What it produced. */
+    output?: unknown;
+    /** Why it failed. */
+    error?: {
+        message?: string;
+    };
+}
+/**
+ * A helper on another agent server: what `createRemoteGraph()` returns, or anything that runs an input
+ * there and waits for it.
+ */
+export interface DeepRemoteHelper {
+    /**
+     * Runs an input there and waits for it to end. A second call with the same `idempotencyKey` waits
+     * for the run the first started, as the agent server does, rather than starting another.
+     */
+    invoke(input: unknown, options: {
+        idempotencyKey: string;
+        signal?: AbortSignal;
+    }): Promise<DeepRemoteRunResult>;
+}
 /** What the deep agent hands the helper tools. */
 export interface HelperSetup {
     /** The helpers the application declared. */
@@ -35,6 +70,7 @@ export interface HelperSetup {
         tools?: readonly string[];
         model?: string;
         maxIterations?: number;
+        remote?: DeepRemoteHelper;
     }>;
     /** Limits on created helpers, when the agent may create them. */
     create?: DeepHelperLimits;