nexus-ai-pro 2.7.0 → 2.9.0

package/API_STABILITY.md

@@ -41,6 +41,121 @@ Both are covered by this policy:
 Deep imports into `dist`, `dist-cjs`, or `src` are not supported. Import only from `nexus-ai-pro` or
 one of its explicit subpaths.
 
+## 2.9 stage (2.9.0)
+
+Everything 2.9 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- `OperationFamily` gains `video`;
+- `nexus-ai-pro/video`, `/video/google`, `/video/runway`, and `/video/mock` are new and experimental:
+  video generation graduates by the checklist once Veo and Runway pass its conformance suite live;
+- `RunQuery.after` with `RunCursor`, and `runCursor()` and `isAfterCursor()` on
+  `nexus-ai-pro/tracing`. A store of your own that does not honour `after` still works with the
+  online-evaluation worker, which falls back to offsets;
+- `OnlineSchedulePass.paging`;
+- on `nexus-ai-pro/lifecycle`: `BudgetReservation.id`; `BudgetReservationStatus`,
+  `BudgetReservationRecord`, `BudgetReservationOutcome`, `BudgetRecoveryReport`, and
+  `BudgetRecoverOptions`; `BudgetLedger.renew()`, `settle()`, and `recover()`, all optional;
+  `BudgetLedgerOptions.leaseMs` and `uncertain`; `BudgetLedgerWithReport.reservation()`; and the five
+  optional reservation methods of `BudgetUsageStore` with `BudgetReservationChange`. The statuses,
+  and which outcome moves a reservation from which status, are part of the promise;
+- `hold()`, `settle()`, `renew()`, `lapsed()`, and `reservation()` on `MemoryTenantUsage`,
+  `RedisTenantUsage`, and `PostgresTenantUsage`, and the tenancy migration version 2 with its
+  reservations table;
+- on `nexus-ai-pro/media/rag`, experimental with the media family: `MediaIndexOptions.manifest` and
+  `name`, `MediaIndexEntry`, `MediaIndexReader`, `MediaIndexReport.unchanged`, and
+  `MediaIndex.delete()`, `reindex()`, `describe()`, and `list()`.
+
+Changed, each to keep a promise the surface already made:
+- `TraceStore.query()` orders runs that share a start time by id, from last to first, as code units,
+  so a page means the same on every store. Before, that order was not defined.
+- `MemoryBudgetUsage`, the ledger's default store, records reservations, so `reserve()` returns one
+  with an `id`. An online-evaluation worker reserves every run of its evaluators, a run after a crash
+  included, where before it reserved only a first.
+- A deep agent with a checkpointer writes the helpers it creates to `<thread>/helpers-created`, so
+  `createHelpers.max` holds for a thread across processes, as it says.
+
+## 2.8 stage (2.8.0)
+
+Everything 2.8 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- `ChromaVectorStoreOptions.filterFields`;
+- `EmbeddingCacheStore`, `EmbeddingCacheEntry`, and `EmbeddingCacheConfig` with its `store`;
+  `nexus-ai-pro/embeddings/cache`: `embeddingCacheKey()` and `EmbeddingCacheContext`, `packVector()`
+  and `unpackVector()`, `MemoryEmbeddingCache` and its options, `RedisEmbeddingCache` with its client
+  and options, and `tieredEmbeddingCache()`. A key's derivation and a vector's packed layout are part
+  of the promise, so a cache one release wrote is read by the next;
+- `EmbeddingModality`, `EmbeddingPart`, and `EmbeddingParts` as `EmbeddingInput`;
+  `EmbeddingProviderRequest.contents` with `EmbeddingProviderPart`, `EmbeddingProviderTextPart`, and
+  `EmbeddingProviderMediaPart`; `inputModalities` on `EmbeddingProviderCapabilities` and
+  `EmbeddingModelCapabilities`, and `costPerImage`; `MockEmbeddingProviderOptions.describeMedia`. An
+  adapter that declares no `inputModalities` is sent text only, as before;
+- `nexus-ai-pro/embeddings/batch`: `EmbeddingBatchManager` and its config and runtime, its request,
+  item, result, and output types, the `EmbeddingBatchProvider` contract with its info, capabilities,
+  request, and item, `batchEmbedder()` with `BatchEmbedder` and its options; the OpenAI and mock
+  adapters on `/openai` and `/mock`. `GoogleEmbeddingBatchProvider` on `/google` is experimental
+  until a live run is recorded;
+- on `nexus-ai-pro/images/assets`: `AssetStore.putStream()` and `getStream()`, both optional;
+  `putAssetStream()` and `readAssetStream()`, `DEFAULT_BUFFERED_ASSET_BYTES`, `AssetByteSource`,
+  `StreamAssetInput`, `AssetByteRange`, `AssetStreamOptions`, `StreamAssetDescriptor`, and
+  `PutAssetStreamOptions`; `S3LikeClient.putObjectStream()` and `getObjectStream()`, both optional.
+  The images family stays experimental, and these with it;
+- `nexus-ai-pro/models/data`: `RegistryData`, `REGISTRY_DATA_FORMAT`, `checkRegistryData()`,
+  `parseRegistryData()` with `RegistryDataError`, `bundledRegistryData()`, `withRegistryData()`,
+  `diffRegistryData()` with `RegistryDiff` and `RegistryFieldChange`, and `registryDataAgeDays()`;
+  `ModelRegistryConfig.verifiedAt` and `DoctorOptions.models`; the commands `nexus models export`,
+  `check`, and `refresh`. The `nexus-registry/1` format is read by every 2.x release;
+- `nexus-ai-pro/media`, `/media/mistral`, `/media/mock`, and `/media/rag` are new and experimental: media
+  understanding graduates by the checklist, once a vision model and the OCR service pass its
+  conformance suite live. `OperationFamily` gains `media`, and `AssetProvenance.operation` gains
+  `upload`, both stable;
+- `IngestionPipelineOptions.batchEmbed` and `batchesInFlight`, and `IngestionReport.batches` and
+  `batchesRecovered`. `IngestionPipelineOptions.embed` is optional when `batchEmbed` is given;
+- `nexus-ai-pro/postgres/embedding-cache`: `PostgresEmbeddingCache`, its options, and
+  `embeddingCacheMigrations()`; `nexus-ai-pro/sqlite/embedding-cache`: `SqliteEmbeddingCache`, its
+  options, and `sqliteEmbeddingCacheMigrations()`.
+
+Changed, each to keep a promise the surface already made:
+- `OpenAIBatchProvider` refuses `endpoint: '/v1/embeddings'` with a `BatchCapabilityError`; it sent
+  chat bodies there, which never embedded anything.
+- The Qdrant, Redis, Pinecone, Weaviate, and Chroma stores refuse a `topK` that is not a positive
+  integer with a `RangeError`. An absent `topK`, or 0, still means 5.
+- A replacement in Chroma removes the metadata fields the new version lacks; before, Chroma merged
+  them. A Weaviate filter is typed by the collection's schema, so a value of another type than its
+  property matches nothing instead of failing the search.
+
+### Graduation
+
+The Qdrant, Redis, Weaviate, and Chroma vector stores leave experimental status, by the same
+checklist as in 2.4. The live evidence comes from the scheduled live workflow, which runs every
+check against each server; the stand-in evidence runs in CI on every change.
+
+**The remote vector stores** (`QdrantVectorStore`, `RedisVectorStore`, `WeaviateVectorStore`,
+`ChromaVectorStore`):
+
+| Requirement | Evidence |
+| --- | --- |
+| API stability | Covered by the 2.x compatibility rules, with no planned breaking change |
+| Conformance | The vector store contract and typed filters, against each real server, Redis over RESP3 and RESP2: `tests/vector-stores.live.ts`; against stand-ins that fail as the servers do: `tests/vector-stores.test.ts` |
+| Crash recovery | n/a: a store keeps no run state, and a reload upserts the same ids |
+| Concurrency | Ten writers replacing the same twenty ids at once leave each id as one writer's version, found only by that writer's filters, live and against stand-ins that interleave as a network does |
+| Load | 2,000 chunks written, then 200 searches within a p95 budget of 200 ms on each server: `tests/vector-stores.live.ts` |
+| Tenant isolation | Each store under `tenantVectorStore()`, live and in `tests/tenant-isolation.test.ts` |
+| Security | [Review](./SECURITY.md#the-qdrant-redis-weaviate-and-chroma-vector-stores): four findings fixed, two documented limitations |
+| Upgrade | What the published 2.7.0 wrote is read, filtered, replaced, and deleted by this release, live and in `tests/upgrade-remote-stores.test.ts` |
+| Soak | 1,000 write, search, and delete cycles on each server, with the heap within 8 MB |
+
+Chroma graduates in both modes. Without `filterFields`, two writers replacing one id at once can
+leave a stale filter field, which its guide states; with `filterFields`, a replacement stays whole
+under any race.
+
+**Still experimental**, each with what remains:
+- **The Pinecone store.** It passes every check against a stand-in, and the live workflow runs the
+  same checks against Pinecone once the repository's Pinecone secret is set.
+- **Images.** Recorded live conformance must pass on all three backends. The live workflow records it
+  for each backend whose credential is set.
+
 ## 2.7 stage (2.7.0)
 
 Everything 2.7 adds is stable from the start and additive.
@@ -996,7 +1111,8 @@ accounts are experimental in production readiness. Their contracts follow the 1.
 ## Retrieval breadth stage (1.22.0)
 
 The loaders, the SQLite vector store, the retrievers, and the MCP registry graduated in 2.4; see the
-2.4 stage for their evidence. The Redis, Pinecone, Weaviate, and Chroma stores remain experimental.
+2.4 stage for their evidence. The Redis, Weaviate, and Chroma stores graduated in 2.8; see the 2.8
+stage. The Pinecone store remains experimental.
 
 The loaders (`nexus-ai-pro/loaders/*`), the Redis, Pinecone, Weaviate, Chroma, and SQLite vector
 stores, the retrievers (`nexus-ai-pro/rag/retrievers`), and the MCP registry
@@ -1028,8 +1144,8 @@ are experimental in production readiness. Their contracts follow the 1.x rules.
 
 ## Retrieval stores stage (1.20.0)
 
-`PostgresVectorStore` graduated in 2.4; see the 2.4 stage for its evidence. `QdrantVectorStore`
-remains experimental.
+`PostgresVectorStore` graduated in 2.4, and `QdrantVectorStore` in 2.8; see those stages for their
+evidence.
 
 `PostgresVectorStore` (`nexus-ai-pro/postgres/vectors`) and `QdrantVectorStore` (`nexus-ai-pro/rag/qdrant`)
 are experimental in production readiness. The `VectorStore` contract they share with

package/CHANGELOG.md

@@ -4,6 +4,244 @@ Notable changes to this project are recorded here. The format follows [Keep a Ch
 
 ## [Unreleased]
 
+## [2.9.0] - 2026-10-15
+
+Video, and everything shipped, proven.
+
+### Added
+
+- **Video generation.** `nexus-ai-pro/video`: `VideoManager` makes videos from a prompt, a first or
+  last frame, or reference images, through a provider's asynchronous job. `generate()` is a durable
+  operation that records the job the moment it exists and each video as it is stored, so
+  `recover()` in another worker collects a job whose process died, never submitting it twice, and
+  streams each video into an asset store. Each generation runs in the lifecycle's new `video`
+  family, and is priced per second by `priceVideo()` from `VIDEO_PRICES`, the Veo 3.1 list prices,
+  or the account's own. `start()`, `resume()`, `status()`, `download()`, and `cancel()` work from a
+  JSON `VideoJobRef` in any process. A request is checked against the model's capabilities before
+  any job, and a failed job is submitted again only when the provider says that may help.
+  - `GoogleVideoProvider` (`/video/google`): Veo through its long-running predictions, on the
+    Gemini API, the video read from Google's URI with the key, or on Vertex AI, read from Cloud
+    Storage or inline.
+  - `RunwayVideoProvider` (`/video/runway`): Runway's task API, for its own models and the hosted
+    ones it serves, Veo, Gemini Omni Flash, and Seedance among them; each video read from its signed
+    URL without the secret.
+  - `MockVideoProvider` (`/video/mock`), and `runVideoConformance()`, which all three pass.
+
+  New guide `docs/video.md`. The family is experimental until Veo and Runway pass the conformance
+  suite live. OpenAI's Videos API was removed on 2026-09-24, so there is no OpenAI adapter.
+- **The experimental families, live.** `npm run test:families:live`, and a weekly *Family
+  conformance* job beside the vector stores: media understanding's conformance suite against an
+  OpenAI or Gemini vision model and Mistral's OCR, on a PDF and a PNG the script builds; a two-input
+  batch through the Gemini embedding batch tier, waited on for 20 minutes and cancelled past that;
+  and, when the repository variable `NEXUS_LIVE_VIDEO` is set, the video conformance suite on Veo 3.1
+  Lite and Runway. Each check runs where its key is set, and what it measured is kept. These are the
+  runs media, the Gemini batch tier, and video graduate on.
+- **Paging by cursor.** `RunQuery.after` takes a `RunCursor`, the start time and id of a page's last
+  run, and returns only the runs after it; `runCursor()` takes the cursor from a run, and
+  `isAfterCursor()` is the test on its own, for a trace store of your own. Every bundled store
+  honours it, and `applyQuery()` does for a store that filters in memory. A new Postgres trace store
+  migration, version 3, indexes runs in query order.
+- **Budget reservations that outlive their process.** Over a store that records reservations, a
+  `budgetLedger()` holds each one as a `BudgetReservationRecord` under a lease (`leaseMs`, 10
+  minutes), and moves it with its total in one step: `reserved`, then `settled`, `released`,
+  `expired`, or `uncertain`. `recover(resolve)` settles reservations whose lease lapsed by what
+  became of each operation, a `BudgetReservationOutcome`; `settle(id, outcome)` does it for one from
+  any process; `renew()` extends a lease; and `uncertain: 'charge' | 'release'` says what an unknown
+  outcome costs. `MemoryBudgetUsage`, `MemoryTenantUsage`, `RedisTenantUsage`, and
+  `PostgresTenantUsage` all record them: Redis keeps a reservation in its total's Cluster slot, so one
+  script moves both, and Postgres does it in one statement, in the reservations table of a new
+  tenancy migration, version 2. A store of your own records them through five optional methods on
+  `BudgetUsageStore`.
+- **A media index that remembers what it holds.** `MediaIndexOptions.manifest`, any `Store`, with a
+  `name`, keeps one `MediaIndexEntry` per medium: a hash of its content, source, metadata, and
+  indexing options, where it is kept, its pages, its chunk ids, the embedding model, and the
+  `MediaIndexReader` that read and described it. `add()` then skips a medium at the same version and
+  replaces exactly the chunks and asset of one that changed, and `MediaIndex` gains `delete()`,
+  `reindex()`, `describe()`, and `list()`. `MediaIndexReport.unchanged` counts what was skipped. A
+  change is recorded as pending first, so a crash leaves nothing the next call cannot finish.
+
+### Changed
+
+- Runs that started in the same millisecond come by id, from last to first, compared as code units,
+  on every trace store. The order of such runs was not defined before, and the memory and Postgres
+  stores disagreed on it.
+
+### Fixed
+
+- An online-evaluation worker reads a backlog of runs that started in the same millisecond through,
+  however many pages it spans. Before, it resumed each page at the last run's start time, so a page
+  that ended among such runs was read again on every pass, and `drain()` never finished. On a trace
+  store of your own that does not honour `RunQuery.after`, the worker pages by offset instead, and
+  its `OnlineSchedulePass` says `paging: 'offset'`.
+- A budget reservation held by an online-evaluation worker that dies no longer stays charged for the
+  period. The worker records the reservation just before it calls its evaluators, and whoever takes
+  over settles it: charged at what the scores cost if they were written, given back if the
+  evaluators never ran, and by the ledger's `uncertain` policy if they ran and wrote nothing.
+- An evaluation run again after a crash is reserved and charged. Its evaluators run again to find
+  the scores still missing, and before, only a first run drew on the budget.
+- A deep agent resumed after a crash counts the helpers it created before. With a checkpointer, each
+  created helper is recorded, with its instructions, on `<thread>/helpers-created` before the call
+  that created it returns, so the resumed agent cannot pass `createHelpers.max` and delegates to them
+  again by name. Before, the count was kept by the process, and a resumed agent could create `max`
+  more and no longer found the ones it had made.
+
+### Proof
+
+- `tests/video.test.ts`: the mock, Veo on the Gemini API and Vertex AI, and Runway pass the video
+  conformance suite against stand-ins for their APIs. A request a model cannot make is refused
+  before any job. A generation runs through the lifecycle, streams three megabytes into a
+  filesystem store, and is priced per second. A generation whose process dies after submitting, or
+  after storing its video, is collected by another worker with one submission and one stored video.
+  If the recorded job is ignored, the job is submitted twice.
+- `tests/online-evaluation-paging.test.ts`: the memory, JSONL, and Postgres trace stores order 5,000
+  runs, 4,000 of them in one millisecond, identically, and page through them by cursor without a
+  repeat or a gap. Ten workers reading 1,000 a page schedule each of them exactly once on every
+  store, and on a store that ignores the cursor. The previous worker never reads past the burst.
+- `tests/budget-reservations.test.ts`: on the memory, tenant memory, Redis, and Postgres stores a
+  reservation moves with its total, is settled from another process by its operation's outcome, and
+  of two processes settling it, one changes the total. An online-evaluation worker killed while
+  holding a reservation, before its evaluator, during it, or after its scores, leaves the budget at
+  what the judge billed once another worker takes over. With recovery switched off, the same test
+  finds the dead worker's estimate still charged.
+- `tests/media-rag.test.ts`: with a manifest, an unchanged medium is neither read nor written, a
+  changed one keeps exactly its chunks and one asset, and a deleted one leaves none, with a crash at
+  each write of a first index, a change, and a delete. Without the pending record, the crash proof
+  fails.
+- `tests/deep-agent-helpers.test.ts`: an agent that created its last allowed helper, killed and
+  continued in another process, is refused a second and delegates to the first again by name. Without
+  the record read back, the second is created.
+
+## [2.8.0] - 2026-10-14
+
+Retrieval at production scale, and media that is understood.
+
+### Added
+
+- **Graduation.** The Qdrant, Redis, Weaviate, and Chroma vector stores leave experimental status.
+  Each passes its contract and the graduation checklist against the real server in the scheduled
+  live workflow: ten writers racing on the same ids, what 2.7.0 wrote read by this release, tenant
+  isolation, a load budget, and a soak. The evidence is in `API_STABILITY.md`, and the threat review
+  in `SECURITY.md`. The Pinecone store passes the same checks against a stand-in, and graduates once
+  the live workflow runs them against Pinecone.
+- **A shared embedding cache.** `embeddings.cache.store` takes an `EmbeddingCacheStore`, which
+  reads every input of a request in one `getMany()` and writes what was embedded in one `setMany()`.
+  Keys are SHA-256 hashes from `embeddingCacheKey()`, so no text is a key, and vectors are packed
+  float32. On `nexus-ai-pro/embeddings/cache`: `MemoryEmbeddingCache`, `RedisEmbeddingCache` (one
+  `MGET` and one script a batch, or a command a key on Redis Cluster), `tieredEmbeddingCache()`, and
+  `packVector()` with `unpackVector()`. `PostgresEmbeddingCache` is on
+  `nexus-ai-pro/postgres/embedding-cache`, and `SqliteEmbeddingCache` on
+  `nexus-ai-pro/sqlite/embedding-cache`, each with its versioned migrations and `purgeExpired()`.
+- **Images, audio, and documents as embedding input.** An `EmbeddingInput` may hold `EmbeddingParts`:
+  the text, image, audio, and document parts a message takes, embedded as one vector in the same
+  space as text. A model takes the `EmbeddingModality` kinds its registry entry or adapter declares in
+  `inputModalities`, and anything else is refused before any call; a fallback that cannot take the
+  media is skipped. An adapter receives each input's parts in `EmbeddingProviderRequest.contents`,
+  media already resolved to base64 or a URL. Google sends media inline or by a Files API URI, and
+  writes `gemini-embedding-2`'s task as the text prefix that model reads instead of a task type; Cohere
+  sends text and data-URI images as `inputs`. `gemini-embedding-2` joins the registry, with
+  `costPerImage`, and `embed-v4.0` takes images. `MockEmbeddingProviderOptions.describeMedia` puts a
+  medium and its caption in one space for tests.
+- **A batch tier for embeddings**, on the new `nexus-ai-pro/embeddings/batch`: `EmbeddingBatchManager`
+  submits, polls, collects, and prices at the provider's batch rate, usually half, with `submit()`,
+  `start()`, `resume()`, `status()`, and `cancel()`. Adapters: `OpenAIEmbeddingBatchProvider` on
+  `/openai`, `GoogleEmbeddingBatchProvider` (`asyncBatchEmbedContent`, media included) on `/google`,
+  and `MockEmbeddingBatchProvider` on `/mock`. `batchEmbedder()` gives an ingestion pipeline a batch
+  tier.
+- **Ingestion through a batch tier.** `IngestionPipelineOptions.batchEmbed` takes a batch tier in
+  place of `embed`. Chunks are gathered into windows, each one provider batch, with `batchesInFlight`
+  (4) outstanding, and written in order. A durable run records each batch the moment it is
+  submitted, and a retry collects it rather than submitting it again. `IngestionReport` counts
+  `batches` and `batchesRecovered`.
+- **Streaming asset stores.** `AssetStore` gains the optional `putStream()` and `getStream()`, and
+  `putAssetStream()` with `readAssetStream()` stream through any store, falling back to `put()` and
+  `get()`. The filesystem and S3 stores write a stream a chunk at a time, checking its size limit and
+  checksum as it flows and leaving nothing behind when either fails, and read it back whole or by an
+  `AssetByteRange`. `S3LikeClient` takes an optional `putObjectStream()` and `getObjectStream()`.
+- **Registry data on its own**, on the new `nexus-ai-pro/models/data`: the model and embedding
+  registries as a `RegistryData` file (`nexus-registry/1`) the application supplies.
+  `checkRegistryData()` and `parseRegistryData()` name every problem with its path,
+  `withRegistryData()` layers a file under the application's own entries and over the bundled ones,
+  `bundledRegistryData()` exports the bundled data, and `diffRegistryData()` compares two files.
+  `models.verifiedAt` measures freshness from the file, in `checkRegistryFreshness()` and `nexus
+  doctor`, which takes the configuration as `models`. `nexus models export`, `check`, and `refresh`
+  work with the file; `refresh` compares it with each provider's own model list.
+- **Media understanding**, on the new `nexus-ai-pro/media`: `MediaManager` reads a document's or an
+  image's text page by page, describes an image, and answers a question about media, routing each
+  task to the provider named, its default, or the first that can, and running each call as one
+  `media` operation of the client's lifecycle. `modelMediaProvider()` does all three through any
+  vision model and the client's own `complete()`; `MistralOcrProvider`, on `/mistral`, reads PDFs,
+  slides, and images into Markdown pages with their sizes and picture boxes; `MockMediaProvider`, on
+  `/mock`, stands in for tests. `runMediaConformance()` checks any provider. `OperationFamily` gains
+  `media`.
+- **Media search and RAG**, on the new `nexus-ai-pro/media/rag`: `createMediaIndex()` keeps media in
+  any vector store as their pages' text, captions, and, on a model that takes them, their own
+  vectors, and keeps each medium in an asset store, or by its URL, so a hit can hand it back.
+  `search()` returns each medium once at its best chunk, `resolve()` returns the medium, and
+  `answerFromMedia()` hands the model the retrieved media themselves. `AssetProvenance.operation`
+  takes `upload`, for media an application brings in.
+- `ChromaVectorStoreOptions.filterFields`: the fields a search can filter on. Every write then sets
+  each declared field, or removes it, so a replacement needs no read and stays whole under any race;
+  a filter on another field is refused.
+
+### Changed
+
+- The Qdrant, Redis, Pinecone, Weaviate, and Chroma stores refuse a `topK` that is not a positive
+  integer, before any request. An absent `topK`, or 0, still means 5.
+- `RedisVectorStore` replaces a chunk with one atomic script instead of a delete and a write.
+- An embedding `cache.adapter` is asked for every input at once, where it was asked for one input
+  after another: a 2,000-text request made 2,000 sequential reads. Its keys are unchanged, so
+  processes of this release and the last share entries during a rollout.
+
+### Fixed
+
+- `RedisVectorStore` failed on every command with an ioredis client, which also has a
+  `sendCommand()` that takes its own command object; it now uses `call()` whenever a client has it.
+  It also reads a search reply in RESP3, which ioredis 6 speaks by default, as well as RESP2.
+- `WeaviateVectorStore` failed a search whose filter value had another type than its property, such
+  as `'3'` on a number. It now chooses the operand from the collection's schema, read once, so such a
+  filter matches nothing, as in every other store, and an `int` property gets `valueInt`.
+- A `topK` from a request could change a Weaviate or Redis query, since both placed it in the query
+  text.
+- A replacement could keep a filter field the new version lacks, so a filter, such as a tenant's,
+  still found the chunk by its old value. In Chroma, which merges metadata, it happened on every
+  replacement, and kept a replaced `source` too; in Redis, when two writers raced.
+- `BatchManager` submitted a batch again when a worker died after submitting it and before collecting
+  it, so the provider ran, and billed, both. A submission now records its batch the moment it exists,
+  and a later attempt collects that one.
+- `OpenAIBatchProvider` read only the output file, so every request that failed was missing from the
+  results; it now reads the error file too. It also refuses the `/v1/embeddings` endpoint its options
+  offered, since it sent chat bodies there: embeddings are batched by `OpenAIEmbeddingBatchProvider`.
+- After two writers replaced one object, Weaviate's filter index could return it for a value it no
+  longer had. Every filtered hit is now checked against the object's own fields, and the store asks
+  for more when a stale entry took a place.
+
+### Proof
+
+- `npm run test:vectors:live` runs the contract and the graduation checklist against each server,
+  Redis over RESP3 and RESP2, and passes 35 of 35 checks locally against Qdrant, Weaviate 1.40.1,
+  Chroma 1.0, and Redis Stack with ioredis 6.0.0. The scheduled workflow keeps its measurements.
+- The stand-ins now fail as the servers do: an ioredis-shaped client, RESP3 replies, Weaviate's typed
+  filter errors and stale filter entries, Chroma's merged metadata, and a seeded network delay that
+  interleaves racing writers. Each failure above reproduces against them on the previous code.
+- `tests/upgrade-remote-stores.test.ts` reads, with this release, what the published 2.7.0 wrote into
+  each store.
+- `tests/media-rag.test.ts`: with stand-in models, answers given the retrieved charts are right on
+  every question, and answers from the same hits' text on none.
+- `tests/media.test.ts`: the vision-model, Mistral OCR, and mock providers pass the media conformance
+  suite, and the wire shapes each sends are asserted.
+- `tests/registry-data.test.ts`: a price raised tenfold in a supplied file makes the next call cost
+  ten times as much, for completions and embeddings alike.
+- `npm run bench:streams`, a step of the release gate: a gigabyte goes through the filesystem and S3
+  stores and back while memory grows by less than 128 MB.
+- `tests/embedding-batch.test.ts`: an ingestion of 100,000 chunks through a batch tier dies while it
+  collects its third window, and across both attempts each of its ten windows is submitted once;
+  without the recorded batches, two are submitted twice.
+- `tests/embeddings-media.test.ts`: a text query finds the image it describes through one vector
+  space, and media never reaches a model that takes text only.
+- `tests/embedding-cache.test.ts`: a second process embeds a 2,000-text batch with no provider call
+  and one cache read, and every store keeps one contract, which the live suite also runs against
+  Redis over RESP3 and RESP2.
+
 ## [2.7.0] - 2026-10-12
 
 Agents that do more, and evaluation that runs on its own.
@@ -2321,7 +2559,9 @@ shared policy vocabulary is in place for a future release that revisits this.
 - URL fetching rejects unsafe private-network targets and limits response reads.
 - CLI and audit findings no longer disclose detected secret values.
 
-[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.7.0...HEAD
+[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.9.0...HEAD
+[2.9.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.8.0...v2.9.0
+[2.8.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.7.0...v2.8.0
 [2.7.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.6.0...v2.7.0
 [2.6.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.5.0...v2.6.0
 [2.5.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.4.0...v2.5.0

package/README.md

@@ -155,6 +155,8 @@ from the doc comments. The guides live in the repository, so these links go to G
 | [Loaders](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md) | Files, Markdown, HTML, CSV, JSON, PDF, web pages, sitemaps, and Git repositories into retrieval |
 | [Retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) | Hybrid search, reranking, MMR, parent documents, and Redis, Pinecone, Weaviate, and Chroma stores |
 | [Embeddings and retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md) | Embeddings as a routed operation family, and RAG helpers |
+| [Media understanding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md) | Reading documents and images page by page, describing images, and answering questions about them, through any vision model or OCR |
+| [Video generation](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/video.md) | Videos from a prompt or a first frame, on Google Veo and Runway, as durable jobs streamed into an asset store and priced per second |
 | [Images (experimental)](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md) | Generation and masked edits over three backends, input safety, moderation, asset stores |
 | [Voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md) | Transcription, speech, voice turns, and voice sessions |
 | [Realtime voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md) | Browser and server realtime sessions with barge-in, tools, and exports |
@@ -216,11 +218,13 @@ interface, or enum, has no doc comment, and `npm run docs:guides` fails when a g
 every export of the entry points it covers, or its generated reference is stale. Both hold at 100%;
 `--list` names what is missing, and `npm run docs:update` regenerates the references.
 
-Real provider conformance, and the vector store contract against real servers, are opt-in:
+Real provider conformance, the vector store contract against real servers, and the experimental
+families against their live services are opt-in:
 
 ```bash
 npm run test:conformance:real
 npm run test:vectors:live
+npm run test:families:live
 ```
 
 ## Roadmap
@@ -279,11 +283,25 @@ Shipped so far:
 - 2.7 agents that do more, and evaluation that runs on its own: workspace search and a multimodal
   workspace, a container sandbox, an interpreter, helpers that run in the background and survive a
   crash, and tool search on every provider; datasets generated from documents and from the
-  guardrails' attacks; and online evaluation on workers, within a budget.
+  guardrails' attacks; and online evaluation on workers, within a budget;
+- 2.8 retrieval at production scale, and media that is understood: the Qdrant, Redis, Weaviate, and
+  Chroma stores out of experimental after their checklist ran against the real servers; a shared
+  embedding cache; images, audio, and documents as embedding input; a half-price embedding batch tier
+  that ingestion survives a crash through; streaming asset stores; registry data you supply and
+  refresh; and reading, describing, and searching media, with answers given the media themselves;
+- 2.9 video, and everything shipped, proven: videos from a prompt or a first frame on Google Veo and
+  Runway, as durable jobs another worker collects after a crash, streamed into an asset store and
+  priced per second; online evaluation that reads every run however many share a millisecond, and
+  budget reservations a dead worker leaves settled by what its call did; a deep agent's helper limit
+  across processes; a media index that remembers what it holds; and the experimental families run
+  against their live services every week.
 
 Upgrading from 1.x: `npx nexus migrate src --write` moves your imports, and
 [MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers the rest.
-Next, the remote vector stores and images graduate once the live workflow passes against each. The
+Next, 2.10 makes every conversation that runs on a worker durable, A2A tasks, ACP turns, and
+realtime sessions, and adds Gemini Live as a second realtime provider; Pinecone, images, the Gemini
+embedding batch tier, media understanding, and video graduate once the live workflow passes against
+each. The
 [roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists what comes after, in the order
 applications ask for it.
 

package/SECURITY.md

@@ -52,7 +52,7 @@ what it could do with that input. Each review is published here with its finding
 before the surface graduates. A finding is closed in one of two ways: by a fix with a test that
 names the attack, or by a documented limitation that the surface's guide states.
 
-These reviews were done for 2.4, and for 2.5 where a review says so.
+These reviews were done for 2.4, and for 2.5 and 2.8 where a review says so.
 
 ### SQLite adapters
 
@@ -77,6 +77,46 @@ Trust boundary: metadata filters, which applications often build from a request.
 
 Findings: none.
 
+### The Qdrant, Redis, Weaviate, and Chroma vector stores
+
+Reviewed for 2.8, when they graduated. Pinecone's adapter was reviewed with them.
+
+Trust boundary: a search's `topK` and metadata filter, which applications often build from a
+request; and the server, which the store trusts as it trusts its own database. Credentials travel
+only in headers, never in a URL or an error message.
+- Qdrant, Pinecone, and Chroma receive a filter as JSON. Qdrant's keys are placed under `metadata.`,
+  so a filter cannot reach a chunk's content or vector.
+- Redis filters only on the fields the index declares. A filter value is tagged with its type, and
+  every character that is not a letter or a digit is escaped.
+- Weaviate's query is GraphQL. The collection and every filtered property must be a valid name, and
+  each value is a quoted literal.
+
+Findings:
+- **Fixed in 2.8: a `topK` from a request could change a Weaviate or Redis query.** Both stores
+  placed it in the query text, so `5) { … }` became part of the query. Every remote store now refuses
+  a `topK` that is not a positive integer before anything is sent. Proof: that value, a fraction, a
+  negative number, and `NaN` are refused by all five stores.
+- **Fixed in 2.8: a replacement could keep a filter field the new version lacks.** A filter could
+  then still find the chunk by its old value, and that includes the `tenantId` that
+  `tenantVectorStore()` filters on.
+  - Chroma merges metadata on upsert, so it happened on every replacement.
+  - Redis deleted and rewrote a chunk in two commands, so it happened when two writers raced.
+
+  Chroma now removes each field the new version lacks, and Redis replaces a chunk in one atomic
+  script. Proof: ten writers race on the same twenty ids, live and against stand-ins that interleave
+  as a network does, and each id ends as exactly one writer's version.
+- **Fixed in 2.8: Weaviate's filter index can keep an entry for a value an object no longer has**,
+  after two writers replace it at once (seen on Weaviate 1.40.1). The store now checks every filtered
+  hit against the object's own fields, and asks for more when a stale entry took a place. A result
+  always matches its filter.
+- **Documented limitation: Chroma without `filterFields` reads before it replaces**, so two writers
+  replacing one id at once can leave a stale field. With `filterFields`, every write sets each
+  declared field, and a replacement stays whole under any race.
+- **Documented limitation: the Redis store has no timeout of its own.** It sends through the client
+  you pass, so the client's command timeout applies.
+- `tenantVectorStore()` checks each result's id and tenant stamp as well as filtering, so a stale
+  filter entry in any server cannot hand one tenant another's chunk.
+
 ### Loaders
 
 Trust boundary: everything a loader reads is untrusted. A repository, a website, a sitemap, or an

package/dist/batch/manager.js

@@ -1,10 +1,10 @@
-import { isTerminalBatchStatus } from '../types/batch.js';
 import { OperationRunner } from '../operations/runner.js';
 import { estimateCost } from '../optimizer/cost.js';
 import { DEFAULT_CURRENCY } from '../optimizer/cost.js';
 import { generateRequestId } from '../utils/ids.js';
 import { withLifecycle } from '../lifecycle/run.js';
 import { BatchCapabilityError, BatchProviderNotFoundError, BatchValidationError } from './errors.js';
+import { pollUntilSettled } from './poll.js';
 const DEFAULT_POLL_INTERVAL_MS = 30_000;
 const DEFAULT_MAX_POLL_INTERVAL_MS = 300_000;
 const DEFAULT_TIMEOUT_MS = 26 * 60 * 60 * 1000;
@@ -58,7 +58,11 @@ export class BatchManager {
                 idempotencyKey: request.idempotencyKey,
                 attempt: operation.attempt,
             };
-            const ref = await provider.submit(request, context);
+            let ref = operation.previousHeartbeat?.batchRef;
+            if (!ref) {
+                ref = await provider.submit(request, context);
+                await operation.heartbeat({ batchRef: ref });
+            }
             operation.report({ completed: 0, total: request.items.length, message: `submitted as ${ref.id}` });
             return this.collect(provider, ref, request, operation, context);
         }, (result) => ({ cost: result.cost.amount })), {
@@ -103,22 +107,16 @@ export class BatchManager {
         });
     }
     async collect(provider, ref, request, operation, context) {
-        const baseInterval = request.pollIntervalMs ?? this.config.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
-        const maxInterval = this.config.maxPollIntervalMs ?? DEFAULT_MAX_POLL_INTERVAL_MS;
-        let interval = baseInterval;
-        let state = await provider.poll(ref, context);
-        while (!isTerminalBatchStatus(state.status)) {
-            if (context.signal.aborted)
-                throw context.signal.reason ?? new Error('aborted');
-            operation?.report({
-                completed: state.counts?.completed,
-                total: state.counts?.total,
-                message: state.status,
-            });
-            await delay(interval, context.signal);
-            interval = Math.min(Math.round(interval * 1.5), maxInterval);
-            state = await provider.poll(ref, context);
-        }
+        const state = await pollUntilSettled(() => provider.poll(ref, context), {
+            intervalMs: request.pollIntervalMs ?? this.config.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS,
+            maxIntervalMs: this.config.maxPollIntervalMs ?? DEFAULT_MAX_POLL_INTERVAL_MS,
+            signal: context.signal,
+            onState: (current) => operation?.report({
+                completed: current.counts?.completed,
+                total: current.counts?.total,
+                message: current.status,
+            }),
+        });
         const items = state.status === 'completed' ? await provider.results(ref, context) : [];
         const counts = state.counts ?? countItems(items, request.items?.length);
         const usage = sumUsage(items);
@@ -232,18 +230,3 @@ function sumUsage(items) {
     }
     return { inputTokens, outputTokens, totalTokens: inputTokens + outputTokens };
 }
-function delay(ms, signal) {
-    if (ms <= 0)
-        return Promise.resolve();
-    return new Promise((resolve) => {
-        const timer = setTimeout(() => {
-            signal?.removeEventListener('abort', onAbort);
-            resolve();
-        }, ms);
-        function onAbort() {
-            clearTimeout(timer);
-            resolve();
-        }
-        signal?.addEventListener('abort', onAbort, { once: true });
-    });
-}