@procedures/mongo 0.9.0 → 0.9.2

package/README.md

@@ -253,675 +253,35 @@ Check if a default connection exists.
 
 ### Registered Procedures
 
-All procedures are automatically registered when the package is imported. They can be called via `client.call()` or used directly.
-
-#### Database Procedures
-
-##### `mongo.database.ping`
-
-Test database connectivity and measure latency.
-
-```typescript
-// Via RPC
-const result = await client.exec(
-  ["mongo", "database", "ping"],
-  { timeout: 5000 }
-);
-
-// Direct import
-import { pingProcedure } from "@procedures/mongo";
-```
-
-**Input:**
-```typescript
-interface PingInput {
-  timeout?: number;  // Optional timeout in milliseconds
-}
-```
-
-**Output:**
-```typescript
-interface PingOutput {
-  latencyMs: number;  // Response time in milliseconds
-  ok: boolean;        // Server status
-}
-```
-
-##### `mongo.database.info`
-
-Get database information and statistics.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "database", "info"],
-  { includeStats: true }
-);
-```
-
-**Input:**
-```typescript
-interface InfoInput {
-  includeStats?: boolean;
-}
-```
-
-**Output:**
-```typescript
-interface DatabaseInfo {
-  name: string;         // Database name
-  collections: number;  // Number of collections
-  views: number;        // Number of views
-  sizeOnDisk: number;   // Size on disk in bytes
-  empty: boolean;       // Whether database is empty
-}
-```
-
-#### Collection Procedures
-
-##### `mongo.collections.list`
-
-List all collections in the database.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "collections", "list"],
-  { filter: "user" }
-);
-```
-
-**Input:**
-```typescript
-interface ListCollectionsInput {
-  filter?: string;  // Filter by name pattern
-}
-```
-
-**Output:**
-```typescript
-interface ListCollectionsOutput {
-  collections: string[];  // Collection names
-}
-```
-
-##### `mongo.collections.create`
-
-Create a new collection.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "collections", "create"],
-  {
-    name: "logs",
-    options: {
-      capped: true,
-      size: 10485760,  // 10MB
-      max: 1000
-    }
-  }
-);
-```
-
-**Input:**
-```typescript
-interface CreateCollectionInput {
-  name: string;
-  options?: CollectionOptions;
-}
-
-interface CollectionOptions {
-  capped?: boolean;
-  size?: number;      // Max size in bytes (for capped)
-  max?: number;       // Max documents (for capped)
-  validator?: Document;  // JSON Schema validator
-}
-```
-
-**Output:**
-```typescript
-interface CreateCollectionOutput {
-  created: boolean;
-  name: string;
-}
-```
-
-##### `mongo.collections.drop`
-
-Drop a collection.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "collections", "drop"],
-  { name: "old_collection" }
-);
-```
-
-**Input:**
-```typescript
-interface DropCollectionInput {
-  name: string;
-}
-```
-
-**Output:**
-```typescript
-interface DropCollectionOutput {
-  dropped: boolean;
-}
-```
-
-##### `mongo.collections.stats`
-
-Get collection statistics.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "collections", "stats"],
-  {},
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface StatsInput {
-  // Collection from metadata
-}
-```
-
-**Output:**
-```typescript
-interface CollectionStats {
-  count: number;           // Number of documents
-  size: number;            // Size of documents in bytes
-  avgObjSize: number;      // Average document size
-  storageSize: number;     // Total storage size
-  nindexes: number;        // Number of indexes
-  totalIndexSize: number;  // Total index size
-}
-```
-
-#### Document Procedures
-
-##### `mongo.documents.find`
-
-Find documents with pagination, filtering, sorting, and projection.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "documents", "find"],
-  {
-    query: { status: "active", age: { $gte: 18 } },
-    projection: { name: 1, email: 1 },
-    sort: { createdAt: -1 },
-    page: 1,
-    limit: 20
-  },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface FindInput {
-  query?: DocumentQuery;           // MongoDB filter
-  projection?: Record<string, 0 | 1>;  // Field projection
-  sort?: SortSpec;                 // Sort specification
-  page?: number;                   // Page number (1-indexed)
-  limit?: number;                  // Items per page (max 100)
-}
-
-type DocumentQuery = Filter<Document>;
-type SortSpec = Record<string, 1 | -1>;
-```
-
-**Output:**
-```typescript
-interface FindOutput {
-  documents: MongoDocument[];
-  pagination: PaginationOutput;
-}
-
-interface PaginationOutput {
-  page: number;
-  limit: number;
-  total: number;
-  totalPages: number;
-  hasNext: boolean;
-  hasPrev: boolean;
-}
-```
-
-##### `mongo.documents.get`
-
-Get a single document by ID.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "documents", "get"],
-  { id: "507f1f77bcf86cd799439011" },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface GetInput {
-  id: string;  // Document ID (converted to ObjectId)
-}
-```
-
-**Output:**
-```typescript
-interface GetOutput {
-  document: MongoDocument | null;
-}
-```
-
-##### `mongo.documents.insert`
-
-Insert one or more documents.
-
-```typescript
-// Insert single document
-const result = await client.exec(
-  ["mongo", "documents", "insert"],
-  {
-    documents: {
-      name: "John Doe",
-      email: "john@example.com",
-      createdAt: new Date()
-    }
-  },
-  { metadata: { collection: "users" } }
-);
-
-// Insert multiple documents
-const result = await client.exec(
-  ["mongo", "documents", "insert"],
-  {
-    documents: [
-      { name: "Alice" },
-      { name: "Bob" }
-    ]
-  },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface InsertInput {
-  documents: MongoDocument | MongoDocument[];
-}
-```
-
-**Output:**
-```typescript
-interface InsertOutput {
-  insertedCount: number;
-  insertedIds: string[];
-}
-```
-
-##### `mongo.documents.update`
-
-Update one document by `id`, or the documents that match a `filter`.
-
-A write needs exactly one of `id` or `filter`. An unknown field is an error: `query` is the field of `find` and `count`, not of `update`. An empty filter matches every document, so it needs `confirm: true`.
-
-```typescript
-// Update one document by ID
-const result = await client.exec(
-  ["mongo", "documents", "update"],
-  {
-    collection: "users",
-    id: "...",
-    update: { $set: { status: "inactive" } },
-    upsert: false
-  }
-);
-
-// Update multiple documents
-const result = await client.exec(
-  ["mongo", "documents", "update"],
-  {
-    collection: "users",
-    filter: { status: "pending" },
-    update: { $set: { status: "active" } },
-    multi: true
-  }
-);
-```
-
-**Input:**
-```typescript
-interface UpdateInput {
-  collection?: string;     // Or metadata.collection
-  database?: string;       // Or metadata.database
-  id?: string;             // Exactly one of id and filter
-  idType?: "auto" | "objectId" | "string";
-  filter?: DocumentQuery;
-  update: DocumentUpdate;
-  multi?: boolean;   // Update all matching documents
-  upsert?: boolean;  // Insert if not found
-  confirm?: boolean; // Required for an empty filter
-}
-
-type DocumentUpdate = UpdateFilter<Document>;
-```
-
-An upsert by `id` in the "auto" mode updates the document with that ID in either form. When no document matches, the new document gets the ObjectId form of the ID.
-
-**Output:**
-```typescript
-interface UpdateOutput {
-  acknowledged: boolean;
-  matchedCount: number;
-  modifiedCount: number;
-  upsertedId: string | null;
-  upsertedCount: number;
-}
-```
-
-##### `mongo.documents.delete`
-
-Delete one document by `id`, or the documents that match a `filter`. The rules of `update` apply: exactly one target, no unknown fields, and `confirm: true` for an empty filter.
-
-```typescript
-// Delete one document by ID
-const result = await client.exec(
-  ["mongo", "documents", "delete"],
-  { collection: "users", id: "..." }
-);
-
-// Delete multiple documents
-const result = await client.exec(
-  ["mongo", "documents", "delete"],
-  {
-    collection: "users",
-    filter: { status: "inactive" },
-    multi: true
-  }
-);
-```
-
-**Input:**
-```typescript
-interface DeleteInput {
-  collection?: string;     // Or metadata.collection
-  database?: string;       // Or metadata.database
-  id?: string;             // Exactly one of id and filter
-  idType?: "auto" | "objectId" | "string";
-  filter?: DocumentQuery;
-  multi?: boolean;   // Delete all matching documents
-  confirm?: boolean; // Required for an empty filter
-}
-```
-
-**Output:**
-```typescript
-interface DeleteOutput {
-  acknowledged: boolean;
-  deletedCount: number;
-}
-```
-
-##### `mongo.documents.count`
-
-Count documents matching a query.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "documents", "count"],
-  { query: { status: "active" } },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface CountInput {
-  query?: DocumentQuery;
-}
-```
-
-**Output:**
-```typescript
-interface CountOutput {
-  count: number;
-}
-```
-
-##### `mongo.documents.aggregate`
-
-Execute an aggregation pipeline.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "documents", "aggregate"],
-  {
-    pipeline: [
-      { $match: { status: "active" } },
-      { $group: { _id: "$country", count: { $sum: 1 } } },
-      { $sort: { count: -1 } }
-    ],
-    options: {
-      allowDiskUse: true,
-      maxTimeMS: 5000
-    }
-  },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface AggregateInput {
-  pipeline: AggregationStage[];
-  options?: AggregationOptions;
-}
-
-type AggregationStage = Document;
-
-interface AggregationOptions {
-  allowDiskUse?: boolean;
-  maxTimeMS?: number;
-}
-```
-
-**Output:**
-```typescript
-interface AggregateOutput {
-  results: MongoDocument[];
-}
-```
-
-##### `mongo.documents.distinct`
-
-The procedure gives the distinct values of one field in the documents that match `query`. The default `query` matches all documents. An array field gives each of its elements. Use dots for an embedded field, for example `address.city`.
-
-```typescript
-const { values } = await client.exec(
-  ["mongo", "documents", "distinct"],
-  { collection: "users", field: "country", query: { status: "active" } }
-);
-```
-
-##### `mongo.documents.bulk`
-
-The procedure sends several writes in one request. The operations have the form of the `bulkWrite` of the driver: `insertOne`, `updateOne`, `updateMany`, `deleteOne` and `deleteMany`. By default, the operations run in order, and the first failure stops the rest. With `ordered: false`, the server tries each operation, and the error tells which ones failed.
-
-The rules of `update` and `delete` apply to each operation. An empty filter of an update or a delete needs `confirm: true` on the call. An unknown operation or field is an error.
-
-```typescript
-const result = await client.exec(["mongo", "documents", "bulk"], {
-  collection: "users",
-  operations: [
-    { insertOne: { document: { name: "Di" } } },
-    { updateOne: { filter: { name: "Ann" }, update: { $set: { status: "active" } } } },
-    { updateMany: { filter: { status: "new" }, update: { $set: { status: "active" } } } },
-    { deleteOne: { filter: { name: "Bob" } } },
-    { deleteMany: { filter: { status: "deleted" } } },
-  ],
-});
-// result: { ok, insertedCount, matchedCount, modifiedCount, deletedCount, upsertedCount, insertedIds, upsertedIds }
-```
-
-The ids in `insertedIds` and `upsertedIds` are text, and their keys are the indexes of the operations.
-
-##### `mongo.documents.explain`
-
-The procedure gives the plan of a find or of an aggregation pipeline. With `pipeline`, it explains an aggregation. Without `pipeline`, it explains a find with `query`, `projection`, `sort` and `limit`. The verbosity `queryPlanner` (the default) gives the plan only. With the verbosities `executionStats` and `allPlansExecution`, the server also does the plan and gives its statistics. As `documents.aggregate` does, the procedure refuses `$out` and `$merge`.
-
-```typescript
-const { operation, plan } = await client.exec(["mongo", "documents", "explain"], {
-  collection: "users",
-  query: { email: "ann@example.com" },
-  verbosity: "executionStats",
-});
-console.log(plan.queryPlanner.winningPlan);
-```
-
-#### Index Procedures
-
-##### `mongo.indexes.list`
-
-List all indexes on a collection.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "indexes", "list"],
-  {},
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface ListIndexesInput {
-  // Collection from metadata
-}
-```
-
-**Output:**
-```typescript
-interface ListIndexesOutput {
-  indexes: IndexInfo[];
-}
-
-interface IndexInfo {
-  name: string;
-  key: IndexSpec;
-  unique?: boolean;
-  sparse?: boolean;
-}
-
-type IndexSpec = Record<string, 1 | -1 | "text" | "2dsphere">;
-```
-
-##### `mongo.indexes.create`
-
-Create an index on a collection.
-
-```typescript
-// Simple index
-const result = await client.exec(
-  ["mongo", "indexes", "create"],
-  {
-    keys: { email: 1 },
-    options: { unique: true }
-  },
-  { metadata: { collection: "users" } }
-);
-
-// Compound index
-const result = await client.exec(
-  ["mongo", "indexes", "create"],
-  {
-    keys: { status: 1, createdAt: -1 },
-    options: { name: "status_created_idx" }
-  },
-  { metadata: { collection: "users" } }
-);
-
-// Text index
-const result = await client.exec(
-  ["mongo", "indexes", "create"],
-  {
-    keys: { title: "text", description: "text" }
-  },
-  { metadata: { collection: "articles" } }
-);
-```
-
-**Input:**
-```typescript
-interface CreateIndexInput {
-  keys: IndexSpec;
-  options?: IndexOptions;
-}
-
-interface IndexOptions {
-  name?: string;
-  unique?: boolean;
-  sparse?: boolean;
-  background?: boolean;
-  expireAfterSeconds?: number;  // TTL index
-}
-```
-
-**Output:**
-```typescript
-interface CreateIndexOutput {
-  name: string;  // Index name
-}
-```
-
-##### `mongo.indexes.drop`
-
-Drop an index from a collection.
-
-```typescript
-const result = await client.exec(
-  ["mongo", "indexes", "drop"],
-  { name: "email_1" },
-  { metadata: { collection: "users" } }
-);
-```
-
-**Input:**
-```typescript
-interface DropIndexInput {
-  name: string;  // Index name to drop
-}
-```
-
-**Output:**
-```typescript
-interface DropIndexOutput {
-  dropped: boolean;
-}
-```
-
-#### Change Streams
-
-##### `mongo.watch`
-
-The procedure gives the changes of a collection as a stream. Without `collection`, it gives the changes of each collection of the database. The stream has no end (`metadata.stream: { unbounded: true }`). It stops when the reader stops or the call aborts, and the procedure then closes the change stream.
-
-A change stream needs a replica set or a sharded cluster. A standalone server refuses it, and the reader gets that error.
-
-```typescript
-for await (const change of client.execStream(["mongo", "watch"], {
-  collection: "orders",
-  pipeline: [{ $match: { operationType: { $in: ["insert", "update"] } } }],
-  fullDocument: "updateLookup",
-})) {
-  console.log(change.operationType, change.documentKey, change.resumeToken);
-}
-```
-
-Each item has the `operationType`, the namespace `ns`, the `documentKey` and the `resumeToken`. An insert gives the `fullDocument`, and an update gives the `updateDescription`. With `fullDocument: "updateLookup"`, an update also gives the current document. To continue after a stop, give the last `resumeToken` as `resumeAfter`. The `pipeline` accepts only the stages of a change stream: `$match`, `$project`, `$addFields`, `$set`, `$unset`, `$replaceRoot`, `$replaceWith` and `$redact`.
+`import "@procedures/mongo/register"` registers the procedures in the global registry. The package root registers nothing: a host gives `mongoModule` to `createRegistry`. Each procedure of a collection takes `collection`, and each procedure takes an optional `database`. The catalog of the site shows the input and the output schema of each procedure.
+
+| Procedure | What it does | Main input |
+|---|---|---|
+| `mongo.database.ping` | Tests the connection and gives the latency. | `timeout` |
+| `mongo.database.info` | Gives the name, the counts and the size of the database. | `includeStats` |
+| `mongo.collections.list` | Lists the collections of the database. | `filter` |
+| `mongo.collections.create` | Makes a collection, capped or with a validator. | `name`, `options` |
+| `mongo.collections.drop` | Removes a collection. | `name` |
+| `mongo.collections.stats` | Gives the count, the sizes and the indexes of a collection. | |
+| `mongo.documents.find` | Finds documents, with a projection, a sort and pages. | `query`, `projection`, `sort`, `page`, `limit` |
+| `mongo.documents.get` | Gives one document by its id. | `id` |
+| `mongo.documents.insert` | Inserts one document or more. | `documents` |
+| `mongo.documents.update` | Updates the document of `id`, or the documents that match `filter`. | `id` or `filter`, `update`, `multi`, `upsert`, `confirm` |
+| `mongo.documents.delete` | Deletes the document of `id`, or the documents that match `filter`. | `id` or `filter`, `multi`, `confirm` |
+| `mongo.documents.count` | Counts the documents that match a query. | `query` |
+| `mongo.documents.aggregate` | Runs an aggregation pipeline. It refuses `$out` and `$merge`. | `pipeline`, `allowDiskUse`, `maxTimeMS` |
+| `mongo.documents.distinct` | Gives the distinct values of a field. An array field gives each of its elements. | `field` (dots for an embedded field), `query` |
+| `mongo.documents.bulk` | Sends several writes in one request, in the form of the `bulkWrite` of the driver. | `operations`, `ordered`, `confirm` |
+| `mongo.documents.explain` | Gives the plan of a find, or of a pipeline. It refuses `$out` and `$merge`. | `query` or `pipeline`, `verbosity` |
+| `mongo.indexes.list` | Lists the indexes of a collection. | |
+| `mongo.indexes.create` | Makes an index. | `keys`, `options` |
+| `mongo.indexes.drop` | Removes an index. | `name` |
+| `mongo.watch` | Gives the changes of a collection as a stream, or of each collection without `collection`. | `collection`, `pipeline`, `fullDocument`, `resumeAfter` |
+
+- `update` and `delete` need exactly one target (`id` or `filter`), and refuse unknown fields. An empty filter needs `confirm: true`.
+- The operations of `bulk` are `insertOne`, `updateOne`, `updateMany`, `deleteOne` and `deleteMany`. By default the server does them in order, and the first failure stops the rest. With `ordered: false`, the server tries each operation, and the error tells which ones failed.
+- The verbosity `queryPlanner` of `explain` (the default) gives the plan only. With `executionStats` and `allPlansExecution`, the server also does the plan and gives its statistics.
+- The stream of `watch` has no end (`metadata.stream: { unbounded: true }`). It stops when the reader stops or the call aborts, and the procedure then closes the change stream.
 
 ### Type Definitions
 
@@ -1283,6 +643,105 @@ await client.exec(
 );
 ```
 
+## Document storage: `MongoDocumentStorage`
+
+`MongoDocumentStorage` is a `CollectionStorage` of `@procedures/collections` over MongoDB (roadmap CLI-24). It has flat documents, a change log, versions, queries in the database and partitions. `MongoStorage` keeps its `{ _id, data }` documents.
+
+```ts
+import { MongoDocumentStorage } from "@procedures/mongo";
+
+const notes = new MongoDocumentStorage<Note>({ collection: "notes" });
+await notes.set("n1", { title: "Plan", done: false });
+await notes.query({ done: false }, { sort: { title: 1 }, limit: 20 });
+
+// One storage for each tenant, on one collection (roadmap CLI-17)
+server.usePartitions("notes", { storageOf: (partition) => notes.forPartition(partition), resolve });
+```
+
+| Option | Use |
+| --- | --- |
+| `collection` | The MongoDB collection |
+| `db` | The database. The default is the database of the default connection. |
+| `partition` | The partition of the storage. `forPartition(partition)` gives the storage of a partition, the same one each time. |
+| `maxTombstones` | The most deletes that the change log keeps in each partition (default 10000) |
+| `queryPolicy` | The query policy of `query()` without a policy of the call |
+| `counters` | The collection of the sequence counters (default `__storage_counters`) |
+| `writeLease` | The time after which a pending number of a write that stopped no longer holds the readers (default `seconds(60)`). A plain number is a `TypeError`. |
+| `client` | The client of the database, for the transactions. The default is the client of the default connection when `db` is absent. |
+| `transactions` | `auto` (the default): a transaction for each write on a replica set or a sharded cluster. `never`: the watermark only. |
+
+- **The documents.** A plain object with safe keys is stored flat: its fields beside the fields of the storage (`_key`, `_part`, `_seq`, `_ord`, `_tomb`). Another item is stored as JSON text in `_json`. Both ways give the item back as it was. An item whose `_id` is its id keeps it.
+- **The change log.** Each write gets the next number of the sequence of the collection, and a delete leaves a tombstone. `getChangesSince()` thus gives the exact changes and deletes. A version of another epoch, or a version before the oldest dropped tombstone, gives a full result.
+- **The versions.** `versionOf(id)` gives the number of the last write of an item.
+- **The queries.** `query()` sends the equality of a string, a number or a boolean, and `$in` of them, to MongoDB as a filter. `queryEntries()` of the core then gives the exact result, so a query gives the result of the reference.
+- **Several writers.** On a replica set, a write takes its numbers and writes in one transaction. On a single server, a watermark keeps the log exact. Each write keeps its numbers pending until it ends, and a version stops below the lowest pending number. A write that ends after its lease moves its documents to new numbers.
+- **The case that stays on a single server.** A process that stops between a late write and the move of its documents leaves that change below the watermark. A reader that passed those numbers does not get the change. On a replica set, this case does not occur.
+- **The partitions.** An id cannot have the character U+001F, and each write filter has the partition. A write cannot change a document of another partition.
+- **The tests.** The storage passes the `CollectionStorage` contract on MongoDB 7 in CI. Set `MONGO_TEST_URL` to start them locally.
+
+### The guard of a pipeline: `scopePipeline`
+
+`scopePipeline(pipeline, scope)` gives a pipeline that reads only the documents of a filter, on each collection that it reads. `MongoDocumentStorage.aggregate()` uses it with the filter of its partition. An app can use it for its own aggregations over shared collections:
+
+```ts
+import { boundPipeline, scopePipeline } from "@procedures/mongo";
+
+const scoped = scopePipeline(pipeline, { match: { tenant: "t1" }, collections: ["orders", "customers"] });
+const { pipeline: bounded, limited } = boundPipeline(scoped, 5000);
+```
+
+- The filter goes before the first stage, into the stages that must be first (`$geoNear`, `$search`, `$vectorSearch`), into a `$match` with `$text`, and into each sub-pipeline (`$lookup`, `$graphLookup`, `$unionWith`, `$facet`).
+- A `$lookup` with `localField` and `foreignField` becomes the form with a pipeline, so the filter applies to the other collection too.
+- The pipeline reads and writes only the collections of the scope (`from`, `coll`, `into`). Give only the collections that the caller can read: a pipeline needs the same right as a direct read.
+- `$out` is refused. `$merge` is refused unless the scope has a `stamp`. Then each written document gets the stamp, and the match key of `$merge` gets its fields.
+- Only the stages of `ALLOWED_PIPELINE_STAGES` pass, so a stage of the server is refused, also a stage that a later MongoDB adds. `$documents` is refused. The JavaScript operators of the server (`$function`, `$accumulator`, `$where`) are refused at each depth.
+- A stage must have exactly one key, and a `$unset` of a field of the scope is refused.
+- The function does not change the pipeline of the caller.
+- `MongoDocumentStorage.aggregate(pipeline, { maxTime, maxResults })` is read-only, reads only its own collection, and keeps the tombstones out. It gives `{ docs, truncated }`. The defaults are `seconds(10)` and 5000 documents. `maxTime` is a duration of `@procedures/client`.
+
+### Materialized views: `ViewMaterializer`
+
+A view is a document of a views collection: a pipeline over source collections, whose result goes into a target collection, for one partition. `ViewMaterializer` keeps each target equal to the result of its pipeline, with change streams. Change streams need a replica set.
+
+```ts
+import { ViewMaterializer, viewFilter } from "@procedures/mongo";
+
+await db.collection("_aggregations").insertOne({
+  _id: "totals",
+  _part: "tenant-1",
+  collections: ["orders"],
+  pipeline: [{ $group: { _id: "$status", count: { $sum: 1 } } }],
+  targetCollection: "orderTotals",
+});
+const views = new ViewMaterializer({ db: () => db });
+await views.start();
+
+// A reader of the target reads the current generation of the view
+const view = await db.collection("_aggregations").findOne({ _id: "totals" });
+const totals = await db.collection("orderTotals").find(viewFilter(view)).toArray();
+```
+
+- A change of a source starts a pass of its partition after the burst (`debounce`). A pass computes only the views whose sources changed, in the order of their dependencies. A view whose result changed makes its target dirty, so the views that read it are in the same pass.
+- A run reads the result in groups of 1000. Each stored row keeps the hash of its content (`_viewHash`), so a row that did not change writes nothing.
+- The rows of a view are a generation (`_viewGen`), and the status of the view names the current one. A reader of a target filters with `viewFilter(view)`, so it reads one whole generation. A plain query can see two generations.
+- A run writes in one transaction: the changed rows of the current generation, and the removal of the rows that the result does not have. A reader does not see a part of a run.
+- A view with more rows than `maxTransactionRows` (default 10000), or whose transaction fails, writes a whole new generation in groups. Then its status names the new generation. The previous one stays for `generationGrace` (default 60 s, on the time of the server), so a reader of it can finish. Then it goes.
+- A pass in groups holds a lease on its view (`passLease`, default 60 s, on the time of the server), and renews it after each group. Another process leaves the view and the rows of the lease alone. The change of generation needs the lease and the generation of the start of the pass. A pass that loses either one gives way, and the view runs again later.
+- A view in a cycle of views stays idle. Its status is `cycle` until the cycle ends.
+- The status of each view (`ok`, `errored` with `lastError`, `cycle`) goes into the field `_materializerStatus` of its document.
+- Each pipeline goes through `scopePipeline` by default. It reads only the documents of the partition, in its sources, and only the current generation of the target of another view. It cannot have `$merge` or `$out`. The option `scope` gives another guard, with the same filter.
+- A stream that ends opens again after `rewatch`, and a pass catches up. The times of the options are durations (`ms(100)`, `seconds(60)`) of `@procedures/client`.
+- A delete has no document in its change event. It reaches only the views of its partition when its pre-image tells the partition (a collection with `changeStreamPreAndPostImages`). Without a pre-image, a delete, a replace and an update of the partition field reach the views of each partition.
+- In a source of `storageSources`, the id of a `MongoDocumentStorage` document tells the storage partition of such a change. The function of that source gives the partition of the views (`identityPartition` when the two are the same). Else the delete reaches the views of each partition.
+- With pre-images, an update that moves a document to another partition also reaches the views of its old partition.
+- `sourceFields` gives the fields of each source, as paths with dots. An update starts a pass when it changes one of them (at the field, under it or above it), or the partition field. The positions in arrays do not count: `lines.0.price` is `lines.price`. The partition field is a top-level field.
+- The stop has the parts of a transport of a service: `stopIntake()`, `drain(deadline, signal)`, `inFlight()` and `close()`. `stop()` does the four. A host composes them with no adapter.
+- The option `clock` gives the clock of the timers, for example a fake clock in a test. The timers do not keep a process alive.
+- The target is a plain collection that only the materializer writes. Two views of one partition do not share a target. The stored `_id` of a row carries its partition and its generation: by default `{ partition, generation, id }` (`partitionTargetId`), where `id` is the `_id` of the row. Thus two partitions with the same keys share a target.
+- The option `targetId` gives another form of the stored `_id`. Two partitions, or two generations, must not give the same `_id`. The write of a row checks the partition, so the view of the second partition fails, and the row of the first one stays. A form without the generation works only in one transaction.
+- A pass keeps in memory the key of each row of the result (at most `maxRows`) and one group of rows. It keeps nothing between passes.
+- An app with targets from an older materializer (the plain key as `_id`, no generation) needs no migration step. The first pass of each view writes its rows in a generation, and removes the rows of its partition with no generation. A row without the partition field stays.
+
 ## Integration with Ecosystem
 
 ### Bundle Packages
@@ -1321,42 +780,12 @@ const call = createTypedCaller<AllProcedures>(client.call);
 
 ## Package Structure
 
-```
-client-mongo/
-├── src/
-│   ├── connection.ts           # Connection management
-│   ├── types.ts                # Core type definitions
-│   ├── procedures.types.ts     # Typed procedures interface
-│   ├── register.ts             # Auto-registration
-│   ├── index.ts                # Main entry point
-│   ├── procedures/
-│   │   ├── schema.ts           # Minimal schema helpers
-│   │   ├── index.ts            # Procedure barrel export
-│   │   ├── database.ping.ts
-│   │   ├── database.info.ts
-│   │   ├── collections.list.ts
-│   │   ├── collections.create.ts
-│   │   ├── collections.drop.ts
-│   │   ├── collections.stats.ts
-│   │   ├── documents.find.ts
-│   │   ├── documents.get.ts
-│   │   ├── documents.insert.ts
-│   │   ├── documents.update.ts
-│   │   ├── documents.delete.ts
-│   │   ├── documents.count.ts
-│   │   ├── documents.aggregate.ts
-│   │   ├── documents.distinct.ts
-│   │   ├── documents.bulk.ts
-│   │   ├── documents.explain.ts
-│   │   ├── indexes.list.ts
-│   │   ├── indexes.create.ts
-│   │   ├── indexes.drop.ts
-│   │   └── watch.ts            # mongo.watch
-├── dist/                       # Compiled output
-├── package.json
-├── tsconfig.json
-└── README.md
-```
+- `src/index.ts`: the entry. It registers nothing.
+- `src/module.ts`: `mongoModule`, the procedures as data. `src/register.ts` registers them in the global registry.
+- `src/connection.ts`: the connection and its management.
+- `src/scope.ts`: the database and the collection of a call.
+- `src/procedures/`: one file for each procedure, with the schemas of the inputs.
+- `src/storage/`: `MongoStorage`, `MongoDocumentStorage`, `scopePipeline` and `ViewMaterializer`.
 
 ## Development