nexus-ai-pro 2.6.0 → 2.8.0

package/API_STABILITY.md

@@ -41,6 +41,139 @@ Both are covered by this policy:
 Deep imports into `dist`, `dist-cjs`, or `src` are not supported. Import only from `nexus-ai-pro` or
 one of its explicit subpaths.
 
+## 2.8 stage (2.8.0)
+
+Everything 2.8 adds is stable from the start and additive, unless this section says otherwise.
+
+New and stable:
+- `ChromaVectorStoreOptions.filterFields`;
+- `EmbeddingCacheStore`, `EmbeddingCacheEntry`, and `EmbeddingCacheConfig` with its `store`;
+  `nexus-ai-pro/embeddings/cache`: `embeddingCacheKey()` and `EmbeddingCacheContext`, `packVector()`
+  and `unpackVector()`, `MemoryEmbeddingCache` and its options, `RedisEmbeddingCache` with its client
+  and options, and `tieredEmbeddingCache()`. A key's derivation and a vector's packed layout are part
+  of the promise, so a cache one release wrote is read by the next;
+- `EmbeddingModality`, `EmbeddingPart`, and `EmbeddingParts` as `EmbeddingInput`;
+  `EmbeddingProviderRequest.contents` with `EmbeddingProviderPart`, `EmbeddingProviderTextPart`, and
+  `EmbeddingProviderMediaPart`; `inputModalities` on `EmbeddingProviderCapabilities` and
+  `EmbeddingModelCapabilities`, and `costPerImage`; `MockEmbeddingProviderOptions.describeMedia`. An
+  adapter that declares no `inputModalities` is sent text only, as before;
+- `nexus-ai-pro/embeddings/batch`: `EmbeddingBatchManager` and its config and runtime, its request,
+  item, result, and output types, the `EmbeddingBatchProvider` contract with its info, capabilities,
+  request, and item, `batchEmbedder()` with `BatchEmbedder` and its options; the OpenAI and mock
+  adapters on `/openai` and `/mock`. `GoogleEmbeddingBatchProvider` on `/google` is experimental
+  until a live run is recorded;
+- on `nexus-ai-pro/images/assets`: `AssetStore.putStream()` and `getStream()`, both optional;
+  `putAssetStream()` and `readAssetStream()`, `DEFAULT_BUFFERED_ASSET_BYTES`, `AssetByteSource`,
+  `StreamAssetInput`, `AssetByteRange`, `AssetStreamOptions`, `StreamAssetDescriptor`, and
+  `PutAssetStreamOptions`; `S3LikeClient.putObjectStream()` and `getObjectStream()`, both optional.
+  The images family stays experimental, and these with it;
+- `nexus-ai-pro/models/data`: `RegistryData`, `REGISTRY_DATA_FORMAT`, `checkRegistryData()`,
+  `parseRegistryData()` with `RegistryDataError`, `bundledRegistryData()`, `withRegistryData()`,
+  `diffRegistryData()` with `RegistryDiff` and `RegistryFieldChange`, and `registryDataAgeDays()`;
+  `ModelRegistryConfig.verifiedAt` and `DoctorOptions.models`; the commands `nexus models export`,
+  `check`, and `refresh`. The `nexus-registry/1` format is read by every 2.x release;
+- `nexus-ai-pro/media`, `/media/mistral`, `/media/mock`, and `/media/rag` are new and experimental: media
+  understanding graduates by the checklist, once a vision model and the OCR service pass its
+  conformance suite live. `OperationFamily` gains `media`, and `AssetProvenance.operation` gains
+  `upload`, both stable;
+- `IngestionPipelineOptions.batchEmbed` and `batchesInFlight`, and `IngestionReport.batches` and
+  `batchesRecovered`. `IngestionPipelineOptions.embed` is optional when `batchEmbed` is given;
+- `nexus-ai-pro/postgres/embedding-cache`: `PostgresEmbeddingCache`, its options, and
+  `embeddingCacheMigrations()`; `nexus-ai-pro/sqlite/embedding-cache`: `SqliteEmbeddingCache`, its
+  options, and `sqliteEmbeddingCacheMigrations()`.
+
+Changed, each to keep a promise the surface already made:
+- `OpenAIBatchProvider` refuses `endpoint: '/v1/embeddings'` with a `BatchCapabilityError`; it sent
+  chat bodies there, which never embedded anything.
+- The Qdrant, Redis, Pinecone, Weaviate, and Chroma stores refuse a `topK` that is not a positive
+  integer with a `RangeError`. An absent `topK`, or 0, still means 5.
+- A replacement in Chroma removes the metadata fields the new version lacks; before, Chroma merged
+  them. A Weaviate filter is typed by the collection's schema, so a value of another type than its
+  property matches nothing instead of failing the search.
+
+### Graduation
+
+The Qdrant, Redis, Weaviate, and Chroma vector stores leave experimental status, by the same
+checklist as in 2.4. The live evidence comes from the scheduled live workflow, which runs every
+check against each server; the stand-in evidence runs in CI on every change.
+
+**The remote vector stores** (`QdrantVectorStore`, `RedisVectorStore`, `WeaviateVectorStore`,
+`ChromaVectorStore`):
+
+| Requirement | Evidence |
+| --- | --- |
+| API stability | Covered by the 2.x compatibility rules, with no planned breaking change |
+| Conformance | The vector store contract and typed filters, against each real server, Redis over RESP3 and RESP2: `tests/vector-stores.live.ts`; against stand-ins that fail as the servers do: `tests/vector-stores.test.ts` |
+| Crash recovery | n/a: a store keeps no run state, and a reload upserts the same ids |
+| Concurrency | Ten writers replacing the same twenty ids at once leave each id as one writer's version, found only by that writer's filters, live and against stand-ins that interleave as a network does |
+| Load | 2,000 chunks written, then 200 searches within a p95 budget of 200 ms on each server: `tests/vector-stores.live.ts` |
+| Tenant isolation | Each store under `tenantVectorStore()`, live and in `tests/tenant-isolation.test.ts` |
+| Security | [Review](./SECURITY.md#the-qdrant-redis-weaviate-and-chroma-vector-stores): four findings fixed, two documented limitations |
+| Upgrade | What the published 2.7.0 wrote is read, filtered, replaced, and deleted by this release, live and in `tests/upgrade-remote-stores.test.ts` |
+| Soak | 1,000 write, search, and delete cycles on each server, with the heap within 8 MB |
+
+Chroma graduates in both modes. Without `filterFields`, two writers replacing one id at once can
+leave a stale filter field, which its guide states; with `filterFields`, a replacement stays whole
+under any race.
+
+**Still experimental**, each with what remains:
+- **The Pinecone store.** It passes every check against a stand-in, and the live workflow runs the
+  same checks against Pinecone once the repository's Pinecone secret is set.
+- **Images.** Recorded live conformance must pass on all three backends. The live workflow records it
+  for each backend whose credential is set.
+
+## 2.7 stage (2.7.0)
+
+Everything 2.7 adds is stable from the start and additive.
+
+New and stable:
+- on `nexus-ai-pro/agent/sandbox`: `Sandbox.glob()`, `grep()`, and `readBytes()`, all optional;
+  `SandboxGlobOptions`, `SandboxGrepOptions`, and `SandboxMatch`; `SandboxToolsOptions.maxMediaBytes`;
+  and the tool names `glob_files`, `grep_files`, and `read_media`;
+- `DocumentContent` and the `document` member of `ContentPart`, exported from
+  `nexus-ai-pro/runtime`; the `base64` source of `AudioContent`;
+- `nexus-ai-pro/agent/sandbox/container`: `containerSandbox()` and `ContainerSandboxOptions`, with
+  their defaults: no network, every capability dropped, no new privileges, 512 MB of memory and swap,
+  one CPU, and 256 processes;
+- `nexus-ai-pro/agent/interpreter`: `Interpreter`, `InterpreterFunction`, `InterpreterRunOptions`,
+  `InterpreterResult`, `InterpreterCall`, `workerInterpreter()` and `WorkerInterpreterOptions`,
+  `codeTool()` and `CodeToolOptions` with the tool name `run_code`, and `runInterpreterConformance()`
+  with `InterpreterConformanceReport`; `DeepAgentOptions.interpreter`;
+- `DeepAgentOptions.createHelpers` and `maxBackgroundHelpers`, `DeepHelperLimits`, the `background`
+  and `instructions` arguments of `delegate`, the tool name `collect_helper`, the helper thread id
+  `<thread>/helper/<name>@<call id>`, and the helper namespace `<name>@<call id>`;
+- `ToolContext.forward`;
+- `ToolDefinition.deferred` and `tool({ deferred })`; `ToolReferenceContent` and the `tool_reference`
+  member of `ContentPart`, exported from `nexus-ai-pro/runtime`; on `nexus-ai-pro/agent`,
+  `toolSearchTool()`, `ToolSearchOptions`, `searchTools()`, `ToolSearchHit`, and `TOOL_SEARCH_NAME`
+  with the tool name `search_tools`.
+
+- `nexus-ai-pro/evaluate/generate`: `syntheticDataset()` with `SyntheticDatasetOptions`,
+  `SyntheticDocument`, `SyntheticPassage`, `SyntheticDraft`, `SyntheticCitation`, `SyntheticModelClient`,
+  and `SyntheticDatasetResult`; `adversarialDataset()` with `AdversarialDatasetOptions`,
+  `AdversarialInputs`, `AdversarialExpected`, `AttackCarrier`, and `AttackDisguise`; `resistsAttack()`;
+  `promoteIssue()` with `PromoteIssueOptions`;
+- on `nexus-ai-pro/security`: `attackSeeds()`, `AttackSeed`, and `AttackCategory`;
+- `nexus-ai-pro/evaluate/online`: `onlineEvaluationWorker()`, `OnlineEvaluationWorker`,
+  `OnlineEvaluationWorkerOptions`, `OnlineEvaluationWorkerReport`, and `OnlineSchedulePass`; the
+  operation kind `online-evaluation:<source>` and the score mark `value.evaluation`;
+- `EvaluateOptions.budget` and `EvaluationBudgetError` on `nexus-ai-pro/evaluate`.
+
+An agent with no deferred tool offers exactly the tools it did before. Adversarial example ids and
+canaries are stable across runs, so experiments over the same options compare.
+
+A graph run's first checkpoint carries the run's `metadata`, as every later checkpoint already did.
+
+`sandboxTools()` offers the new tools only for a sandbox that implements the methods behind them, so
+a sandbox of the application's own offers the same tools as before. A deep agent over
+`processSandbox()` gets the three new tools and a line of its system prompt naming them.
+
+On OpenAI, a tool message whose media is not only images is followed by a user message that starts
+`Files the tools returned:`; one with only images keeps `Images the tools returned:`.
+
+`ContentPart` and `AudioContent['source']` gain members, as the compatibility rules allow in a minor
+release. Code that switches over either exhaustively should handle the new ones.
+
 ## 2.6 stage (2.6.0)
 
 Everything 2.6 adds is stable from the start and additive.
@@ -944,7 +1077,8 @@ accounts are experimental in production readiness. Their contracts follow the 1.
 ## Retrieval breadth stage (1.22.0)
 
 The loaders, the SQLite vector store, the retrievers, and the MCP registry graduated in 2.4; see the
-2.4 stage for their evidence. The Redis, Pinecone, Weaviate, and Chroma stores remain experimental.
+2.4 stage for their evidence. The Redis, Weaviate, and Chroma stores graduated in 2.8; see the 2.8
+stage. The Pinecone store remains experimental.
 
 The loaders (`nexus-ai-pro/loaders/*`), the Redis, Pinecone, Weaviate, Chroma, and SQLite vector
 stores, the retrievers (`nexus-ai-pro/rag/retrievers`), and the MCP registry
@@ -976,8 +1110,8 @@ are experimental in production readiness. Their contracts follow the 1.x rules.
 
 ## Retrieval stores stage (1.20.0)
 
-`PostgresVectorStore` graduated in 2.4; see the 2.4 stage for its evidence. `QdrantVectorStore`
-remains experimental.
+`PostgresVectorStore` graduated in 2.4, and `QdrantVectorStore` in 2.8; see those stages for their
+evidence.
 
 `PostgresVectorStore` (`nexus-ai-pro/postgres/vectors`) and `QdrantVectorStore` (`nexus-ai-pro/rag/qdrant`)
 are experimental in production readiness. The `VectorStore` contract they share with

package/CHANGELOG.md

@@ -4,6 +4,254 @@ Notable changes to this project are recorded here. The format follows [Keep a Ch
 
 ## [Unreleased]
 
+## [2.8.0] - 2026-10-14
+
+Retrieval at production scale, and media that is understood.
+
+### Added
+
+- **Graduation.** The Qdrant, Redis, Weaviate, and Chroma vector stores leave experimental status.
+  Each passes its contract and the graduation checklist against the real server in the scheduled
+  live workflow: ten writers racing on the same ids, what 2.7.0 wrote read by this release, tenant
+  isolation, a load budget, and a soak. The evidence is in `API_STABILITY.md`, and the threat review
+  in `SECURITY.md`. The Pinecone store passes the same checks against a stand-in, and graduates once
+  the live workflow runs them against Pinecone.
+- **A shared embedding cache.** `embeddings.cache.store` takes an `EmbeddingCacheStore`, which
+  reads every input of a request in one `getMany()` and writes what was embedded in one `setMany()`.
+  Keys are SHA-256 hashes from `embeddingCacheKey()`, so no text is a key, and vectors are packed
+  float32. On `nexus-ai-pro/embeddings/cache`: `MemoryEmbeddingCache`, `RedisEmbeddingCache` (one
+  `MGET` and one script a batch, or a command a key on Redis Cluster), `tieredEmbeddingCache()`, and
+  `packVector()` with `unpackVector()`. `PostgresEmbeddingCache` is on
+  `nexus-ai-pro/postgres/embedding-cache`, and `SqliteEmbeddingCache` on
+  `nexus-ai-pro/sqlite/embedding-cache`, each with its versioned migrations and `purgeExpired()`.
+- **Images, audio, and documents as embedding input.** An `EmbeddingInput` may hold `EmbeddingParts`:
+  the text, image, audio, and document parts a message takes, embedded as one vector in the same
+  space as text. A model takes the `EmbeddingModality` kinds its registry entry or adapter declares in
+  `inputModalities`, and anything else is refused before any call; a fallback that cannot take the
+  media is skipped. An adapter receives each input's parts in `EmbeddingProviderRequest.contents`,
+  media already resolved to base64 or a URL. Google sends media inline or by a Files API URI, and
+  writes `gemini-embedding-2`'s task as the text prefix that model reads instead of a task type; Cohere
+  sends text and data-URI images as `inputs`. `gemini-embedding-2` joins the registry, with
+  `costPerImage`, and `embed-v4.0` takes images. `MockEmbeddingProviderOptions.describeMedia` puts a
+  medium and its caption in one space for tests.
+- **A batch tier for embeddings**, on the new `nexus-ai-pro/embeddings/batch`: `EmbeddingBatchManager`
+  submits, polls, collects, and prices at the provider's batch rate, usually half, with `submit()`,
+  `start()`, `resume()`, `status()`, and `cancel()`. Adapters: `OpenAIEmbeddingBatchProvider` on
+  `/openai`, `GoogleEmbeddingBatchProvider` (`asyncBatchEmbedContent`, media included) on `/google`,
+  and `MockEmbeddingBatchProvider` on `/mock`. `batchEmbedder()` gives an ingestion pipeline a batch
+  tier.
+- **Ingestion through a batch tier.** `IngestionPipelineOptions.batchEmbed` takes a batch tier in
+  place of `embed`. Chunks are gathered into windows, each one provider batch, with `batchesInFlight`
+  (4) outstanding, and written in order. A durable run records each batch the moment it is
+  submitted, and a retry collects it rather than submitting it again. `IngestionReport` counts
+  `batches` and `batchesRecovered`.
+- **Streaming asset stores.** `AssetStore` gains the optional `putStream()` and `getStream()`, and
+  `putAssetStream()` with `readAssetStream()` stream through any store, falling back to `put()` and
+  `get()`. The filesystem and S3 stores write a stream a chunk at a time, checking its size limit and
+  checksum as it flows and leaving nothing behind when either fails, and read it back whole or by an
+  `AssetByteRange`. `S3LikeClient` takes an optional `putObjectStream()` and `getObjectStream()`.
+- **Registry data on its own**, on the new `nexus-ai-pro/models/data`: the model and embedding
+  registries as a `RegistryData` file (`nexus-registry/1`) the application supplies.
+  `checkRegistryData()` and `parseRegistryData()` name every problem with its path,
+  `withRegistryData()` layers a file under the application's own entries and over the bundled ones,
+  `bundledRegistryData()` exports the bundled data, and `diffRegistryData()` compares two files.
+  `models.verifiedAt` measures freshness from the file, in `checkRegistryFreshness()` and `nexus
+  doctor`, which takes the configuration as `models`. `nexus models export`, `check`, and `refresh`
+  work with the file; `refresh` compares it with each provider's own model list.
+- **Media understanding**, on the new `nexus-ai-pro/media`: `MediaManager` reads a document's or an
+  image's text page by page, describes an image, and answers a question about media, routing each
+  task to the provider named, its default, or the first that can, and running each call as one
+  `media` operation of the client's lifecycle. `modelMediaProvider()` does all three through any
+  vision model and the client's own `complete()`; `MistralOcrProvider`, on `/mistral`, reads PDFs,
+  slides, and images into Markdown pages with their sizes and picture boxes; `MockMediaProvider`, on
+  `/mock`, stands in for tests. `runMediaConformance()` checks any provider. `OperationFamily` gains
+  `media`.
+- **Media search and RAG**, on the new `nexus-ai-pro/media/rag`: `createMediaIndex()` keeps media in
+  any vector store as their pages' text, captions, and, on a model that takes them, their own
+  vectors, and keeps each medium in an asset store, or by its URL, so a hit can hand it back.
+  `search()` returns each medium once at its best chunk, `resolve()` returns the medium, and
+  `answerFromMedia()` hands the model the retrieved media themselves. `AssetProvenance.operation`
+  takes `upload`, for media an application brings in.
+- `ChromaVectorStoreOptions.filterFields`: the fields a search can filter on. Every write then sets
+  each declared field, or removes it, so a replacement needs no read and stays whole under any race;
+  a filter on another field is refused.
+
+### Changed
+
+- The Qdrant, Redis, Pinecone, Weaviate, and Chroma stores refuse a `topK` that is not a positive
+  integer, before any request. An absent `topK`, or 0, still means 5.
+- `RedisVectorStore` replaces a chunk with one atomic script instead of a delete and a write.
+- An embedding `cache.adapter` is asked for every input at once, where it was asked for one input
+  after another: a 2,000-text request made 2,000 sequential reads. Its keys are unchanged, so
+  processes of this release and the last share entries during a rollout.
+
+### Fixed
+
+- `RedisVectorStore` failed on every command with an ioredis client, which also has a
+  `sendCommand()` that takes its own command object; it now uses `call()` whenever a client has it.
+  It also reads a search reply in RESP3, which ioredis 6 speaks by default, as well as RESP2.
+- `WeaviateVectorStore` failed a search whose filter value had another type than its property, such
+  as `'3'` on a number. It now chooses the operand from the collection's schema, read once, so such a
+  filter matches nothing, as in every other store, and an `int` property gets `valueInt`.
+- A `topK` from a request could change a Weaviate or Redis query, since both placed it in the query
+  text.
+- A replacement could keep a filter field the new version lacks, so a filter, such as a tenant's,
+  still found the chunk by its old value. In Chroma, which merges metadata, it happened on every
+  replacement, and kept a replaced `source` too; in Redis, when two writers raced.
+- `BatchManager` submitted a batch again when a worker died after submitting it and before collecting
+  it, so the provider ran, and billed, both. A submission now records its batch the moment it exists,
+  and a later attempt collects that one.
+- `OpenAIBatchProvider` read only the output file, so every request that failed was missing from the
+  results; it now reads the error file too. It also refuses the `/v1/embeddings` endpoint its options
+  offered, since it sent chat bodies there: embeddings are batched by `OpenAIEmbeddingBatchProvider`.
+- After two writers replaced one object, Weaviate's filter index could return it for a value it no
+  longer had. Every filtered hit is now checked against the object's own fields, and the store asks
+  for more when a stale entry took a place.
+
+### Proof
+
+- `npm run test:vectors:live` runs the contract and the graduation checklist against each server,
+  Redis over RESP3 and RESP2, and passes 35 of 35 checks locally against Qdrant, Weaviate 1.40.1,
+  Chroma 1.0, and Redis Stack with ioredis 6.0.0. The scheduled workflow keeps its measurements.
+- The stand-ins now fail as the servers do: an ioredis-shaped client, RESP3 replies, Weaviate's typed
+  filter errors and stale filter entries, Chroma's merged metadata, and a seeded network delay that
+  interleaves racing writers. Each failure above reproduces against them on the previous code.
+- `tests/upgrade-remote-stores.test.ts` reads, with this release, what the published 2.7.0 wrote into
+  each store.
+- `tests/media-rag.test.ts`: with stand-in models, answers given the retrieved charts are right on
+  every question, and answers from the same hits' text on none.
+- `tests/media.test.ts`: the vision-model, Mistral OCR, and mock providers pass the media conformance
+  suite, and the wire shapes each sends are asserted.
+- `tests/registry-data.test.ts`: a price raised tenfold in a supplied file makes the next call cost
+  ten times as much, for completions and embeddings alike.
+- `npm run bench:streams`, a step of the release gate: a gigabyte goes through the filesystem and S3
+  stores and back while memory grows by less than 128 MB.
+- `tests/embedding-batch.test.ts`: an ingestion of 100,000 chunks through a batch tier dies while it
+  collects its third window, and across both attempts each of its ten windows is submitted once;
+  without the recorded batches, two are submitted twice.
+- `tests/embeddings-media.test.ts`: a text query finds the image it describes through one vector
+  space, and media never reaches a model that takes text only.
+- `tests/embedding-cache.test.ts`: a second process embeds a 2,000-text batch with no provider call
+  and one cache read, and every store keeps one contract, which the live suite also runs against
+  Redis over RESP3 and RESP2.
+
+## [2.7.0] - 2026-10-12
+
+Agents that do more, and evaluation that runs on its own.
+
+### Added
+
+- **Workspace search.** A `Sandbox` may implement `glob()`, `grep()`, and `readBytes()`, with
+  `SandboxGlobOptions`, `SandboxGrepOptions`, and `SandboxMatch`. `sandboxTools()` then offers
+  `glob_files` and `grep_files`, bounded at 500 paths and 200 lines, each declaring the directory it
+  reads. `processSandbox()` implements all three: its search follows no links, skips `.git` and
+  `node_modules`, walks at most 20,000 entries, and skips files over 1 MB and binary files.
+- **A multimodal workspace.** `read_media` reads an image, audio, or a PDF in the sandbox as content
+  the model sees, as base64 so it survives a checkpoint, up to `SandboxToolsOptions.maxMediaBytes`
+  (10 MB by default). A deep agent gets all three tools when its sandbox supports them.
+- **Documents and audio reach the model.** The new `DocumentContent` part carries a document, such
+  as a PDF, and `AudioContent` takes base64 text. Anthropic receives a document as a document block,
+  inside a tool result too; OpenAI chat as a file, OpenAI Responses as an input file, and Google as
+  inline data. Audio reaches Google, and OpenAI chat's audio models as `input_audio`. A tool's media
+  follows its results on OpenAI and Google, as its images already did. Any other provider, or a
+  source it cannot read, receives one line naming the part.
+- **A container sandbox**, on the new `nexus-ai-pro/agent/sandbox/container`: `containerSandbox()`
+  drives the `docker` or `podman` binary, with no dependency, one container per sandbox. By default
+  the container has no network, drops every capability, cannot gain privileges, and is held to
+  512 MB, one CPU, and 256 processes; nothing of the host is mounted. A command past its timeout is
+  killed inside the container with everything it started. Files, glob, grep, and media work inside
+  it. `ContainerSandboxOptions` sets the image, binary, working directory, network, and limits.
+- **An interpreter contract**, on the new `nexus-ai-pro/agent/interpreter`. An `Interpreter` runs a
+  snippet that calls the agent's tools as `await tools.<name>(args)`, so many calls take one step.
+  `codeTool()` offers it to a model as `run_code`: it declares `code`, and the policy decides each
+  call the code makes as it decides the model's own, refusing one it would ask a person about.
+  `workerInterpreter()` is the JavaScript reference: a fresh worker per run, a 64 MB heap, a
+  10-second timeout, and caps on calls and logs. It is not a security boundary.
+  `runInterpreterConformance()` checks any interpreter. `DeepAgentOptions.interpreter` adds
+  `run_code` over the deep agent's own tools, and its default policy then grants code.
+- **Helpers beside the deep agent.**
+  - `delegate` takes `background`: it returns a task id at once, and the new `collect_helper` tool
+    returns the answer later, waiting up to `wait_seconds`. `maxBackgroundHelpers` (4) bounds how
+    many work at once.
+  - A background helper keeps checkpoints in the deep agent's checkpointer, on a thread named after
+    the call that started it. After a crash, collecting the task, or repeating the call, carries the
+    same helper on from its last checkpoint.
+  - `createHelpers` lets the agent create a helper from instructions it writes, within
+    `DeepHelperLimits`: the tools, model, model calls, and number per thread.
+  - Streamed with `subgraphs: true`, a helper's events reach the agent's stream under its namespace,
+    such as `tools/researcher@call-7`. `ToolContext.forward` lets any tool do the same for a run it
+    starts.
+- **Tool search.** `ToolDefinition.deferred` (and `tool({ deferred })`) loads a tool only once the
+  model finds it. An agent with a deferred tool offers `search_tools`, which ranks the deferred tools
+  by BM25 over their names, descriptions, and arguments, and answers with `ToolReferenceContent`
+  parts. On Claude 4.5 and later every tool is sent, a deferred one with `defer_loading`, and a
+  found tool loads through a `tool_reference` block, so the tool list never changes; every other
+  model, OpenAI's and Google's included, is sent a deferred tool once it has been found.
+  `toolSearchTool()`, `searchTools()`, and `TOOL_SEARCH_NAME` are on `nexus-ai-pro/agent`.
+- **Datasets that grow**, on the new `nexus-ai-pro/evaluate/generate`:
+  - `syntheticDataset()` writes examples from documents, from your function or a model. Each keeps
+    a citation of the quote that supports it, checked against its passage; unsupported drafts and
+    near-duplicate questions are dropped and counted.
+  - `adversarialDataset()` grows attacks from the guardrails' own patterns, in the user's message, a
+    document, or a tool's result, plain or disguised, each after a canary in the system prompt.
+    `resistsAttack()` scores whether the canary stayed in.
+  - `promoteIssue()` turns an insights issue's runs into examples that keep their run and the issue.
+  - `attackSeeds()` on `nexus-ai-pro/security` returns each injection pattern's example attack.
+- **Online evaluation on workers**, on the new `nexus-ai-pro/evaluate/online`.
+  `onlineEvaluationWorker()` samples runs by a stable hash, queues one operation per run and source
+  on a shared operation store, stops queuing at `maxQueued`, and scores what it claims within a
+  budget ledger. Each score carries a mark naming its operation, so a retry after a crash writes
+  only what is missing, and each run is scored exactly once per source.
+- **A spend cap for experiments.** `evaluate()` takes `budget`. Each example run is reserved and
+  reconciled, and once the budget runs out no further example starts; `evaluate()` rejects with the
+  new `EvaluationBudgetError`, which carries the results it paid for.
+
+### Fixed
+
+- An operation whose store failed under it, such as a database going away while it recorded its
+  result, left its handle pending forever, with an unhandled rejection; the handle now fails with
+  the store's error, and the record's lease lapses for recovery.
+
+### Changed
+
+- A graph run's first checkpoint now carries the run's `metadata`, as every later checkpoint did.
+- A deep agent with helpers offers `collect_helper` beside `delegate`, and its prompt says how to use
+  both; its default policy allows `collect_helper`.
+
+### Proof
+
+- Glob and grep on the reference sandbox: alternatives, `**/` matching no directory, results relative
+  to the root, a limit, a path outside the root refused, a link out of the root not followed, and
+  binary, oversized, vendored, and past-the-line-limit matches skipped.
+- An image, a PDF, and audio from a tool, after a JSON round trip, on the wire of Anthropic, OpenAI
+  chat (with and without an audio model), OpenAI Responses, and Google.
+- A deep agent that greps its workspace, finds a PDF, and reads it as a document.
+- The container sandbox passes the sandbox conformance suite against a real Docker in CI, which now
+  requires one. A command cannot find a host file anywhere in the container, cannot reach the host or
+  the internet, while the same probe from a container with a network can; a command over a 64 MB
+  limit is stopped; and a timeout leaves nothing it started running.
+- The worker interpreter passes the interpreter conformance suite. An agent totals three orders and
+  describes one from code in one model round trip, while the refund in the same code, which the
+  policy asks about, never runs; a deep agent searches and reads its workspace from code, and a
+  command its policy denies is denied from code too.
+- A background helper answers after the agent has moved on, with its events in the agent's stream. A
+  crash while the agent waits for it, and a crash before the agent saved the call that started it,
+  each end with one lookup, one note, and one start of the helper.
+- An agent with 500 deferred tools finishes all 40 tasks on Anthropic, OpenAI, and Google, with a
+  model that calls a tool only once that provider's request has loaded it. Anthropic is sent all 501
+  tools on every call with 500 deferred; OpenAI and Google never more than 11. The search ranks each
+  of the 500 tools first for the step that describes it.
+- A synthetic dataset written from all 39 guides cites, for every example, words that stand in its
+  guide where the citation says; an invented quote and a repeated question are dropped every time.
+  An adversarial dataset of 75 examples gets its canary out of an unguarded agent every time and
+  never out of a guarded one, which still answers the benign examples. Each attack the guardrails
+  block at the input is critical or high, and each medium one is redacted on the way out.
+- Ten online-evaluation workers score 10,000 runs, each score exactly once, while one dies right
+  after writing a run's scores and before recording them; that run is taken back and found already
+  scored, and the queue stays near its limit. Workers on a budget stop at it, leave the rest queued,
+  and carry on the next period without scoring any run twice. Two sources sharing one store never
+  cross, and sampling picks the same runs on every worker. An experiment stops at its budget.
+
 ## [2.6.0] - 2026-10-11
 
 Protocols complete, and progress on every stream.
@@ -2204,7 +2452,9 @@ shared policy vocabulary is in place for a future release that revisits this.
 - URL fetching rejects unsafe private-network targets and limits response reads.
 - CLI and audit findings no longer disclose detected secret values.
 
-[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.6.0...HEAD
+[Unreleased]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.8.0...HEAD
+[2.8.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.7.0...v2.8.0
+[2.7.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.6.0...v2.7.0
 [2.6.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.5.0...v2.6.0
 [2.5.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.4.0...v2.5.0
 [2.4.0]: https://github.com/mkhitar-abrahamyan/nexus-ai/compare/v2.3.0...v2.4.0

package/README.md

@@ -155,6 +155,7 @@ from the doc comments. The guides live in the repository, so these links go to G
 | [Loaders](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/loaders.md) | Files, Markdown, HTML, CSV, JSON, PDF, web pages, sitemaps, and Git repositories into retrieval |
 | [Retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/retrieval.md) | Hybrid search, reranking, MMR, parent documents, and Redis, Pinecone, Weaviate, and Chroma stores |
 | [Embeddings and retrieval](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/embeddings.md) | Embeddings as a routed operation family, and RAG helpers |
+| [Media understanding](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/media.md) | Reading documents and images page by page, describing images, and answering questions about them, through any vision model or OCR |
 | [Images (experimental)](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/images.md) | Generation and masked edits over three backends, input safety, moderation, asset stores |
 | [Voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/voice.md) | Transcription, speech, voice turns, and voice sessions |
 | [Realtime voice](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/docs/realtime.md) | Browser and server realtime sessions with barge-in, tools, and exports |
@@ -275,14 +276,23 @@ Shipped so far:
   the protocols, traces, and the studio; AG-UI state deltas and frontend tools; A2A tasks and ACP
   sessions that survive a restart, with push notifications, `session/load`, and the editor's files,
   terminal, and MCP servers; and every protocol proven in a browser, against the previous release,
-  and against its reference client.
+  and against its reference client;
+- 2.7 agents that do more, and evaluation that runs on its own: workspace search and a multimodal
+  workspace, a container sandbox, an interpreter, helpers that run in the background and survive a
+  crash, and tool search on every provider; datasets generated from documents and from the
+  guardrails' attacks; and online evaluation on workers, within a budget;
+- 2.8 retrieval at production scale, and media that is understood: the Qdrant, Redis, Weaviate, and
+  Chroma stores out of experimental after their checklist ran against the real servers; a shared
+  embedding cache; images, audio, and documents as embedding input; a half-price embedding batch tier
+  that ingestion survives a crash through; streaming asset stores; registry data you supply and
+  refresh; and reading, describing, and searching media, with answers given the media themselves.
 
 Upgrading from 1.x: `npx nexus migrate src --write` moves your imports, and
 [MIGRATING.md](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/MIGRATING.md) covers the rest.
-Next, 2.7 lets agents do more: provider tool search, an interpreter contract, workspace search,
-helpers that run beside the agent, and a container sandbox. Evaluation grows its own datasets and
-runs on workers, within a budget. The [roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists the
-releases after it.
+Next, Pinecone, images, the Gemini embedding batch tier, and media understanding graduate once the
+live workflow passes against each. The
+[roadmap](https://github.com/mkhitar-abrahamyan/nexus-ai/blob/main/ROADMAP.md) lists what comes after, in the order
+applications ask for it.
 
 ## Before Production
 

package/SECURITY.md

@@ -52,7 +52,7 @@ what it could do with that input. Each review is published here with its finding
 before the surface graduates. A finding is closed in one of two ways: by a fix with a test that
 names the attack, or by a documented limitation that the surface's guide states.
 
-These reviews were done for 2.4, and for 2.5 where a review says so.
+These reviews were done for 2.4, and for 2.5 and 2.8 where a review says so.
 
 ### SQLite adapters
 
@@ -77,6 +77,46 @@ Trust boundary: metadata filters, which applications often build from a request.
 
 Findings: none.
 
+### The Qdrant, Redis, Weaviate, and Chroma vector stores
+
+Reviewed for 2.8, when they graduated. Pinecone's adapter was reviewed with them.
+
+Trust boundary: a search's `topK` and metadata filter, which applications often build from a
+request; and the server, which the store trusts as it trusts its own database. Credentials travel
+only in headers, never in a URL or an error message.
+- Qdrant, Pinecone, and Chroma receive a filter as JSON. Qdrant's keys are placed under `metadata.`,
+  so a filter cannot reach a chunk's content or vector.
+- Redis filters only on the fields the index declares. A filter value is tagged with its type, and
+  every character that is not a letter or a digit is escaped.
+- Weaviate's query is GraphQL. The collection and every filtered property must be a valid name, and
+  each value is a quoted literal.
+
+Findings:
+- **Fixed in 2.8: a `topK` from a request could change a Weaviate or Redis query.** Both stores
+  placed it in the query text, so `5) { … }` became part of the query. Every remote store now refuses
+  a `topK` that is not a positive integer before anything is sent. Proof: that value, a fraction, a
+  negative number, and `NaN` are refused by all five stores.
+- **Fixed in 2.8: a replacement could keep a filter field the new version lacks.** A filter could
+  then still find the chunk by its old value, and that includes the `tenantId` that
+  `tenantVectorStore()` filters on.
+  - Chroma merges metadata on upsert, so it happened on every replacement.
+  - Redis deleted and rewrote a chunk in two commands, so it happened when two writers raced.
+
+  Chroma now removes each field the new version lacks, and Redis replaces a chunk in one atomic
+  script. Proof: ten writers race on the same twenty ids, live and against stand-ins that interleave
+  as a network does, and each id ends as exactly one writer's version.
+- **Fixed in 2.8: Weaviate's filter index can keep an entry for a value an object no longer has**,
+  after two writers replace it at once (seen on Weaviate 1.40.1). The store now checks every filtered
+  hit against the object's own fields, and asks for more when a stale entry took a place. A result
+  always matches its filter.
+- **Documented limitation: Chroma without `filterFields` reads before it replaces**, so two writers
+  replacing one id at once can leave a stale field. With `filterFields`, every write sets each
+  declared field, and a replacement stays whole under any race.
+- **Documented limitation: the Redis store has no timeout of its own.** It sends through the client
+  you pass, so the client's command timeout applies.
+- `tenantVectorStore()` checks each result's id and tenant stamp as well as filtering, so a stale
+  filter entry in any server cannot hand one tenant another's chunk.
+
 ### Loaders
 
 Trust boundary: everything a loader reads is untrusted. A repository, a website, a sitemap, or an

package/dist/agent/create-agent.js

@@ -1,8 +1,9 @@
 import { appendList, counter, lastValue } from '../graph/channels.js';
-import { createGraph } from '../graph/graph.js';
+import { createGraph, forwardedEvents } from '../graph/graph.js';
 import { Command, END, Send } from '../types/graph.js';
 import { capabilitiesOf } from './capabilities.js';
 import { ToolExecutor, toolMessageContent } from './tool.js';
+import { TOOL_SEARCH_NAME, toolSearchTool } from './tool-search.js';
 const agentChannels = () => ({
     messages: appendList(),
     iterations: counter(),
@@ -18,7 +19,13 @@ export function createAgent(options) {
         throw new TypeError('createAgent needs a client with a complete() method');
     }
     const maxIterations = Math.max(1, options.maxIterations ?? 8);
-    const executor = new ToolExecutor(options.tools ?? []);
+    const searchable = (tools) => {
+        const deferred = tools.list().filter((definition) => definition.deferred);
+        if (deferred.length === 0 || tools.has(TOOL_SEARCH_NAME))
+            return tools;
+        return new ToolExecutor([...tools.list(), toolSearchTool(deferred)]);
+    };
+    const executor = searchable(new ToolExecutor(options.tools ?? []));
     const middleware = options.middleware ?? [];
     const approvals = options.interruptOn ?? {};
     const toolsFor = async (context) => {
@@ -30,7 +37,7 @@ export function createAgent(options) {
             ...(context.principal ? { principal: context.principal } : {}),
             signal: context.signal,
         });
-        return supplied.length ? new ToolExecutor([...supplied, ...executor.list()]) : executor;
+        return supplied.length ? searchable(new ToolExecutor([...supplied, ...executor.list()])) : executor;
     };
     const graph = createGraph({ channels: agentChannels() })
         .addNode('model', async (context) => {
@@ -187,6 +194,7 @@ export function createAgent(options) {
                 }
             }
         }
+        const forward = forwardedEvents(context);
         const run = async (current) => {
             if (options.permissions && current.args !== args) {
                 const recheck = await options.permissions.decide({
@@ -205,6 +213,11 @@ export function createAgent(options) {
                 signal: context.signal,
                 ...(context.store ? { store: context.store } : {}),
                 progress: (progress) => context.tool({ phase: 'progress', id: call.id, name: description.name, progress }),
+                ...(forward
+                    ? {
+                        forward: (event, namespace) => forward({ ...event, namespace: [...namespace, ...(event.namespace ?? [])] }),
+                    }
+                    : {}),
             });
         };
         context.tool({ phase: 'start', id: call.id, name: description.name, args });

package/dist/agent/glob.d.ts

@@ -0,0 +1,24 @@
+/**
+ * File globs for workspace search, shared by the sandboxes: `*` matches within a path segment, `**`
+ * across segments (`**\/` also matches none), `?` one character, and `{a,b}` either alternative.
+ * Paths use `/`, are relative to the search root, and match case-sensitively.
+ */
+export declare function globMatcher(pattern: string): (path: string) => boolean;
+/** Directories workspace search never enters. */
+export declare const SKIPPED_DIRECTORIES: ReadonlySet<string>;
+/** The most directory entries, or files, one search walks. */
+export declare const WALK_LIMIT = 20000;
+/** Files larger than this are not searched. */
+export declare const GREP_FILE_BYTES = 1048576;
+/** One search reads no more than this, in all. */
+export declare const GREP_TOTAL_BYTES: number;
+/**
+ * The lines of a file that match, numbered from 1 and cut to 300 characters, at most `limit`; none
+ * for a binary file, one with a NUL byte in its first 8,000 bytes.
+ */
+export declare function matchingLines(bytes: Uint8Array, expression: RegExp, limit: number): Array<{
+    line: number;
+    text: string;
+}>;
+/** MIME types for the media a workspace tool reads as content the model sees, by file extension. */
+export declare const MEDIA_TYPES: Readonly<Record<string, string>>;