@klum-db/lobby 0.2.0-pre.30 → 0.2.0-pre.32

package/README.md

@@ -1,43 +1,65 @@
 # @klum-db/lobby
 
-> **The Lobby** orchestrates a *group* of sovereign [noy-db](https://github.com/vLannaAi/noy-db) vaults — federation, interchange, custody, and scoped sync — without ever dissolving their independence.
->
-> **noy-db is the vault (inward). klum-db is the Lobby (outward).**
-> A container runs perfectly alone; the engine orchestrates many. Docker is to a container what the Lobby is to a vault.
+> **Small enough to own, coordinated enough to scale.**
+> `@klum-db/lobby` is the **control plane** for a fleet of sovereign [noy-db](https://github.com/vLannaAi/noy-db) vaults. Each vault is small, **100%-encrypted**, and complete on its own — its own embedded schema, its own query engine, usable **offline**. The Lobby orchestrates many into one coordinated whole, **online or offline**: **coordination without custody** — it drives the fleet but never owns the data. *One-way — klum drives noy; noy never knows klum exists.*
 
 `@klum-db/lobby` · status: **preview** · depends on `@noy-db/hub`
 
 ---
 
-## The idea in 20 seconds
+## What it is, in 20 seconds
 
-Banking, accounting, health, insurance — the data that matters is **individual, single-entity, and owned by the subject**. noy-db makes each of those a small, sovereign, in-memory vault that is a *complete system on its own*. Working "small, in memory" isn't a limitation — it's the strong core. The limitation only bites when one actor must work across **many** datasets at once.
+A single vault is complete — its own keys, schema, and truth — but completeness is also a ceiling: alone, a vault can only answer for *itself*. The usual fix is to pour everything into one central store, trading away the sovereignty that made each vault worth trusting. **The Lobby takes the other path — it coordinates the vaults where they stand.**
 
-The **Lobby** is the framework for exactly that outward dimension: *the efficiency of a small independent dataset at the core, joined with an actor operating across many at the same time* — a counterbalance to tech giants holding user data hostage. Vaults stay independent and relocatable; the Lobby coordinates them without absorbing them. That's why it's a **group** (Thai *klum* กลุ่ม), never a cluster — federation here is non-aggregative.
-
-## Reads in a sentence
-
-> A firm is a **Custodian** in the **Lobby**: it holds operating grants to client **Vaults** that all reference one shared **Pool**. The client holds the **Deed**. To onboard, the firm **Relocates** a client's vault as a **Bundle**, **Migrates** it to the current schema, and **Merges** the Pool slice by field **Authority** using **Provenance**. The client can **Withdraw** anytime; an abandoned vault can be **Liberated**.
+```mermaid
+flowchart TD
+    L["<b>CONTROL PLANE</b> · @klum-db/lobby<br/>federate · interchange · custody · surface"]:::cp
+    L ==> V1["🔒 vault"]:::dp
+    L ==> V2["🔒 vault"]:::dp
+    L ==> V3["🔒 vault"]:::dp
+    L ==> V4["🔒 vault"]:::dp
+    classDef cp fill:#0f766e,stroke:#0f766e,color:#ffffff
+    classDef dp fill:#ecfdf5,stroke:#10b981,color:#065f46
+```
 
-Every bold word is a real, shipped capability.
+*DATA PLANE · `@noy-db/hub` — each vault sovereign & 100% encrypted, complete on its own. **klum drives noy one-way; noy never depends on klum.***
 
-## Architecture — one-way dependency, two complementary axes
+<details><summary>Text version (npm / non-Mermaid viewers)</summary>
 
 ```
-            outward / orchestration                 inward / the vault
-   ┌─────────────────────────────────┐     ┌──────────────────────────────────┐
-   │          @klum-db/lobby         │     │            @noy-db/hub            │
-   │                                 │     │                                   │
-   │  Lobby  ⊃  many Vaults          │ ──▶ │  Vault ⊃ Collection ⊃ Record ⊃ Field │
-   │   • Federation (fleets)         │     │   • keyring · per-record CEK      │
-   │   • Interchange (move data)     │     │   • computed/derivation · money   │
-   │   • Custody (Deed/Custodian)    │     │   • i18n · history · snapshots    │
-   │   • Surface (scoped sync)       │     │   • a vault + a store = complete  │
-   └─────────────────────────────────┘     └──────────────────────────────────┘
-          binds to the stable  ── @noy-db/hub/kernel ──  surface only
+  ┌────────────────────────────────────────────────────┐
+  │  CONTROL PLANE · @klum-db/lobby                      │
+  │  federate · interchange · custody · surface         │
+  └────────────────────────┬───────────────────────────┘
+                           │ drives — one-way (klum → noy)
+        ┌──────────┬───────┴───────┬──────────┐
+        ▼          ▼               ▼          ▼
+     ┌──────┐   ┌──────┐        ┌──────┐   ┌──────┐
+     │vault │   │vault │        │vault │   │vault │
+     └──────┘   └──────┘        └──────┘   └──────┘
+  DATA PLANE · @noy-db/hub — each vault sovereign & non-fungible,
+  complete on its own.  noy never depends on klum.
 ```
+</details>
+
+- **Coordination without custody.** klum drives the fleet — federate, move data, custody, sync — but **never owns the data**: keys, crypto, and records stay sovereign in each vault, and klum binds to one stable contract (`@noy-db/hub/kernel`), never hub internals. *(One-way, and enforced — a build-time guard means no `@noy-db` package can ever import `@klum-db`.)*
+- **Bring the work to the data, not the data to a lake.** Cross-vault queries resolve *across* the group — each vault answers for its own slice under its own keys, nothing pools. No central honeypot to breach.
+- **Small core, coordinated reach.** Each vault stays small, portable, and individually revocable; orchestration recovers the cross-cutting reach you'd otherwise need a monolith for.
+- **A group, not a cluster.** Vaults are sovereign and non-fungible — one subject = one vault, the subject holds the deed. *Joined, not merged; allied, not absorbed.* (Thai *klum* กลุ่ม = a group.)
+
+## Why it exists
+
+Banking, accounting, health, insurance — the data that matters is **individual, single-subject, and owned by the person it's about**. noy-db makes each of those a small, sovereign, **100%-encrypted** vault — a complete system on its own. *You own your data, in spite of the cloud.* The limit only bites when one actor must work across **many** subjects at once.
+
+That's the Lobby's dimension: the efficiency and privacy of a small sovereign dataset at the core, joined with an actor operating across many at once — **governance by default, not by checklist**. Access is **scoped, purpose-limited, and revocable** (the subject can withdraw anytime); the orchestrator coordinates without absorbing. A counterweight to the central data lake — without the lock-in, the lock-out, or the single honeypot.
 
-The dependency runs **one way**: `@klum-db/lobby` → `@noy-db/hub`. No `@noy-db` package ever imports `@klum-db` (enforced by a build-time architecture guard). The Lobby binds to a stable internal surface, **`@noy-db/hub/kernel`**, not hub internals. A vault is a complete, shippable system *without* the Lobby; the Lobby is what you reach for when one actor must work across many vaults at once.
+**Lineage & market context:** klum-db sits in the local-first / data-sovereignty tradition — [Local-first software](https://www.inkandswitch.com/essay/local-first/), [GDPR data portability](https://gdpr-info.eu/art-20-gdpr/), and the full picture in [**docs/positioning.md**](docs/positioning.md).
+
+## Reads in a sentence
+
+> A firm is a **Custodian** in the **Lobby**: it holds operating grants to client **Vaults** that all reference one shared **Pool**. The client holds the **Deed**. To onboard, the firm **Relocates** a client's vault as a **Bundle**, **Migrates** it to the current schema, and **Merges** the Pool slice by field **Authority** using **Provenance**. The client can **Withdraw** anytime; an abandoned vault can be **Liberated**.
+
+Every bold word is a real, shipped capability.
 
 ## Install
 
@@ -131,17 +153,22 @@ const { bundleBytes, transferKey } = await lobby.exportSurface('payroll-vault',
 await lobby.applySurface('tax-vault', surface, bundleBytes, transferKey)
 ```
 
+### On-ramp · Dock → `graduate()`
+
+A foreign, non-noy-db unit (a legacy DB, a raw export) can't be federated directly — the sovereign tier needs a vault's keyring and per-record keys. **Dock** carries it read-only at a lower tier; **`graduate()`** imports it into a fresh sovereign vault, unlocking the full tier (custody, provenance, field-Authority merge). It's the one move that *adds a node to the data plane* rather than coordinating existing ones.
+
 ---
 
 ## Relationship with noy-db
 
-- **Depends on `@noy-db/hub`**, binds to the stable **`@noy-db/hub/kernel`** subpath — never reaches into hub internals.
+The one-way law and the kernel seam are covered up top — here are the specifics that matter when you build:
+
 - **Custody is a vault-level concern** and lives *in* hub (keyring/CEK/consent primitives); the Lobby **re-exports** it (`createDeedOwner`, `liberateVault`, `CustodyApi`) so consumers have one import surface.
 - **Federation** lives in the Lobby, not in hub — open fleets with `lobby.openVaultGroup` (`@noy-db/hub` no longer ships the `openVaultGroup` / `openStateManagementVault` / `withVaultTemplate` fleet methods).
-- The dependency is enforced one-way at build time; an `@noy-db` package importing `@klum-db` fails the architecture check.
+- **Enforced, not conventional:** an `@noy-db` package importing `@klum-db` fails noy-db's build-time architecture check.
 
 ## Status
 
 Preview. `@klum-db/lobby` is its own repository and the sole publisher of `@klum-db/*` to npm. It depends on the **published** `@noy-db/*` packages through the stable `@noy-db/hub/kernel` boundary and versions **independently** (`0.2.0-pre.N`, decoupled from noy-db). Pilot-1 (FR-1…FR-9), the dock tier, and `Lobby.graduate()` are complete.
 
-See [`PROVENANCE.md`](./PROVENANCE.md) for origin and build history.
+See [`docs/architecture.md`](docs/architecture.md) for the detailed noy-db ↔ klum-db boundary, [`docs/roadmap.md`](docs/roadmap.md) for what's next, and [`PROVENANCE.md`](./PROVENANCE.md) for origin and build history.

package/dist/bin/klum.d.cts

@@ -1,4 +1,4 @@
-import { V as VaultGroup } from '../vault-group-BXjO5kHB.cjs';
+import { V as VaultGroup } from '../vault-group-BchRCDm5.cjs';
 import '@noy-db/hub/kernel';
 import '@noy-db/hub';
 import '@noy-db/hub/bundle';

package/dist/bin/klum.d.ts

@@ -1,4 +1,4 @@
-import { V as VaultGroup } from '../vault-group-BXjO5kHB.js';
+import { V as VaultGroup } from '../vault-group-BchRCDm5.js';
 import '@noy-db/hub/kernel';
 import '@noy-db/hub';
 import '@noy-db/hub/bundle';

package/dist/index.cjs

@@ -1268,6 +1268,43 @@ var init_insight_auto_push = __esm({
   }
 });
 
+// src/federation/partial-reduce.ts
+function canPartialReduce(spec) {
+  return Object.values(spec).every((r) => typeof r.merge === "function");
+}
+function reduceToPartial(records, spec) {
+  const out = {};
+  for (const [key, reducer] of Object.entries(spec)) {
+    const r = reducer;
+    let state = r.init();
+    for (const rec of records) state = r.step(state, rec);
+    out[key] = state;
+  }
+  return out;
+}
+function mergePartials(spec, partials) {
+  const out = {};
+  for (const [key, reducer] of Object.entries(spec)) {
+    const r = reducer;
+    let acc = r.init();
+    for (const p of partials) acc = r.merge(acc, p[key]);
+    out[key] = acc;
+  }
+  return out;
+}
+function finalizePartial(spec, merged) {
+  const out = {};
+  for (const [key, reducer] of Object.entries(spec)) {
+    out[key] = reducer.finalize(merged[key]);
+  }
+  return out;
+}
+var init_partial_reduce = __esm({
+  "src/federation/partial-reduce.ts"() {
+    "use strict";
+  }
+});
+
 // src/federation/aggregate-across.ts
 var import_kernel8, import_kernel9, CrossVaultAggregation, CrossVaultGroupedAggregation;
 var init_aggregate_across = __esm({
@@ -1276,6 +1313,7 @@ var init_aggregate_across = __esm({
     import_kernel8 = require("@noy-db/hub/kernel");
     import_kernel9 = require("@noy-db/hub/kernel");
     init_cross_vault_live();
+    init_partial_reduce();
     CrossVaultAggregation = class {
       constructor(src, spec, bind) {
         this.src = src;
@@ -1286,6 +1324,10 @@ var init_aggregate_across = __esm({
       spec;
       bind;
       async run(options = {}) {
+        if (this.src.fanoutReduce && canPartialReduce(this.spec)) {
+          const { partials, skippedVaults: skippedVaults2 } = await this.src.fanoutReduce(this.spec, options);
+          return { result: finalizePartial(this.spec, mergePartials(this.spec, partials)), skippedVaults: skippedVaults2 };
+        }
         const { records, skippedVaults } = await this.src.fanoutRecords(options);
         return { result: (0, import_kernel8.reduceRecords)(records, this.spec), skippedVaults };
       }
@@ -1374,6 +1416,73 @@ var init_aggregate_across = __esm({
   }
 });
 
+// src/federation/retrieve-across.ts
+async function retrieveAcross(group, collectionName, query, opts = {}) {
+  const { eligible, skipped } = await group.resolveEligible({
+    ...opts.minVersion !== void 0 ? { minVersion: opts.minVersion } : {},
+    ...opts.failFast !== void 0 ? { failFast: opts.failFast } : {}
+  });
+  const across = await group.db.queryAcross(
+    eligible.map((r) => r.vaultId),
+    async (vault) => {
+      group.template.configure(vault);
+      const coll = vault.collection(collectionName);
+      let within;
+      if (opts.where?.length) {
+        let q = coll.query();
+        for (const [f, op, v] of opts.where) q = q.where(f, op, v);
+        within = q;
+      }
+      return coll.retrieve(query, {
+        ...opts.mode ? { mode: opts.mode } : {},
+        ...opts.limit !== void 0 ? { limit: opts.limit } : {},
+        ...opts.minScore !== void 0 ? { minScore: opts.minScore } : {},
+        ...opts.fields ? { fields: opts.fields } : {},
+        ...opts.match ? { match: opts.match } : {},
+        ...opts.prefix ? { prefix: opts.prefix } : {},
+        ...opts.snippetWindow !== void 0 ? { snippetWindow: opts.snippetWindow } : {},
+        ...opts.includeRecord ? { includeRecord: true } : {},
+        ...within ? { within } : {}
+      });
+    },
+    { concurrency: opts.concurrency ?? 1, create: false }
+  );
+  const skippedVaults = [...skipped];
+  const perVault = [];
+  for (const r of across) {
+    if (r.error) {
+      if (opts.failFast) throw r.error;
+      skippedVaults.push({ vaultId: r.vault, reason: classifyShardSkip(r.error), error: r.error });
+    } else {
+      perVault.push({ vault: r.vault, hits: r.result });
+    }
+  }
+  const lists = perVault.map((pv) => {
+    if (pv.vault.includes(SEP)) {
+      throw new Error(`retrieveAcross: vault id "${pv.vault}" contains the reserved NUL separator`);
+    }
+    return pv.hits.map((h) => ({ ...h, id: pv.vault + SEP + h.id }));
+  });
+  const fused = (0, import_kernel10.fuseRetrieval)(lists, {
+    ...opts.limit !== void 0 ? { limit: opts.limit } : {},
+    ...opts.rrfK !== void 0 ? { k: opts.rrfK } : {}
+  });
+  const hits = fused.map((h) => {
+    const i = h.id.indexOf(SEP);
+    return { ...h, vault: h.id.slice(0, i), id: h.id.slice(i + 1) };
+  });
+  return { hits, skippedVaults };
+}
+var import_kernel10, SEP;
+var init_retrieve_across = __esm({
+  "src/federation/retrieve-across.ts"() {
+    "use strict";
+    import_kernel10 = require("@noy-db/hub/kernel");
+    init_classify_skip();
+    SEP = "\0";
+  }
+});
+
 // src/federation/vault-group.ts
 var vault_group_exports = {};
 __export(vault_group_exports, {
@@ -1384,46 +1493,49 @@ __export(vault_group_exports, {
 });
 function assertSafePartitionKey(partitionKey) {
   if (partitionKey.length === 0) {
-    throw new import_kernel10.ValidationError("partitionKey must be a non-empty string");
+    throw new import_kernel11.ValidationError("partitionKey must be a non-empty string");
   }
   if (partitionKey === import_hub4.STATE_VAULT_NAME) {
-    throw new import_kernel10.ReservedVaultNameError(partitionKey);
+    throw new import_kernel11.ReservedVaultNameError(partitionKey);
   }
   if (!SAFE_PARTITION_KEY.test(partitionKey)) {
-    throw new import_kernel10.ValidationError(
+    throw new import_kernel11.ValidationError(
       `partitionKey "${partitionKey}" contains characters outside [A-Za-z0-9._-]. Map your records to a store-safe key in sharding.keyOf.`
     );
   }
   if (partitionKey.includes(SHARD_SEPARATOR)) {
-    throw new import_kernel10.ValidationError(
+    throw new import_kernel11.ValidationError(
       `partitionKey "${partitionKey}" must not contain "--" \u2014 it is reserved as the shard vault-id separator and would risk shard-id collisions.`
     );
   }
 }
-var import_kernel10, SHARD_SEPARATOR, SAFE_PARTITION_KEY, VaultGroup, ShardedCollection, ShardedQuery, ShardedGroupedQuery;
+var import_kernel11, SHARD_SEPARATOR, SAFE_PARTITION_KEY, VaultGroup, ShardedCollection, ShardedQuery, ShardedGroupedQuery;
 var init_vault_group = __esm({
   "src/federation/vault-group.ts"() {
     "use strict";
     init_state_vault();
-    import_kernel10 = require("@noy-db/hub/kernel");
+    import_kernel11 = require("@noy-db/hub/kernel");
     init_constants();
     init_classify_skip();
     init_cross_shard_join();
     init_cross_vault_live();
     init_insight_auto_push();
     init_aggregate_across();
+    init_retrieve_across();
+    init_partial_reduce();
     SHARD_SEPARATOR = "--";
     SAFE_PARTITION_KEY = /^[A-Za-z0-9._-]+$/;
     VaultGroup = class {
-      constructor(db, name, registry, sharding, template, cutoverOnOpen = false) {
+      constructor(db, name, registry, sharding, template, cutoverOnOpen = false, meta) {
         this.db = db;
         this.name = name;
         this.registry = registry;
         this.sharding = sharding;
         this.template = template;
         this.cutoverOnOpen = cutoverOnOpen;
+        this.meta = meta;
         if (name.includes(SHARD_SEPARATOR)) {
-          throw new import_kernel10.ValidationError(
+          throw new import_kernel11.ValidationError(
             `VaultGroup name "${name}" must not contain "--" (reserved shard vault-id separator).`
           );
         }
@@ -1434,6 +1546,7 @@ var init_vault_group = __esm({
       sharding;
       template;
       cutoverOnOpen;
+      meta;
       /** @internal — set when the group is managed (no explicit registry). */
       stateVault;
       /** @internal */
@@ -1467,6 +1580,30 @@ var init_vault_group = __esm({
         const rows = this.registry.query().toArray();
         return rows.filter((r) => r.group === this.name);
       }
+      /**
+       * Federation-level descriptive metadata (#27): this group's `meta` plus each
+       * member vault's `vaultMeta` (read via `getMeta()`). Best-effort per shard — a
+       * shard that fails to open yields `meta: undefined` for that entry, never
+       * throwing. Member meta is transient per-open (noy-db sets it at `openVault`,
+       * does not persist it); this surfaces whatever each vault was opened with.
+       */
+      async federationMeta() {
+        const rows = await this.allRows();
+        const vaults = await Promise.all(
+          rows.map(async (row) => {
+            let meta;
+            try {
+              const vault = await this.db.openVault(row.vaultId);
+              this.template.configure(vault);
+              meta = vault.getMeta();
+            } catch {
+              meta = void 0;
+            }
+            return { vaultId: row.vaultId, partitionKey: row.partitionKey, meta };
+          })
+        );
+        return { meta: this.meta, vaults };
+      }
       /**
        * Open an existing shard and apply the template. When `cutoverOnOpen` is set
        * (#271) and the shard's registry version is behind the template, its cutover
@@ -1503,11 +1640,11 @@ var init_vault_group = __esm({
         const vaultId = this.shardVaultId(partitionKey);
         const row = await this.registry.get(this.registryId(partitionKey));
         const provisioned = await this.db._shardVaultProvisioned(vaultId);
-        if (row && !provisioned) throw new import_kernel10.ShardProvisioningError(vaultId, partitionKey);
+        if (row && !provisioned) throw new import_kernel11.ShardProvisioningError(vaultId, partitionKey);
         if (row && provisioned) return this.openShard(partitionKey);
         if (region !== void 0) {
           const backendRegion = this.db._resolveBackend(vaultId).capabilities?.region;
-          if (backendRegion !== region) throw new import_kernel10.DataResidencyError(vaultId, region, backendRegion);
+          if (backendRegion !== region) throw new import_kernel11.DataResidencyError(vaultId, region, backendRegion);
         }
         const vault = await this.db.openVault(vaultId);
         this.template.configure(vault);
@@ -1541,31 +1678,42 @@ var init_vault_group = __esm({
       async shard(partitionKey) {
         const vaultId = this.shardVaultId(partitionKey);
         const row = await this.registry.get(this.registryId(partitionKey));
-        if (!row) throw new import_kernel10.UnknownShardError(partitionKey, this.name);
+        if (!row) throw new import_kernel11.UnknownShardError(partitionKey, this.name);
         const provisioned = await this.db._shardVaultProvisioned(vaultId);
-        if (!provisioned) throw new import_kernel10.ShardProvisioningError(vaultId, partitionKey);
+        if (!provisioned) throw new import_kernel11.ShardProvisioningError(vaultId, partitionKey);
         return this.openShard(partitionKey);
       }
       /** A sharded view over one logical collection across all shards. */
       collection(collectionName) {
         return new ShardedCollection(this, collectionName);
       }
-      /** @internal — eligible (openable-candidate) rows + drift/divergence skips. */
+      /** @internal — eligible (openable-candidate) rows + drift/divergence/unreachable skips. */
       async resolveEligible(options = {}) {
         const rows = await this.allRows();
+        const candidates = options.only ? rows.filter((r) => options.only.includes(r.partitionKey)) : rows;
         const skipped = [];
         const versionOk = [];
-        for (const row of rows) {
+        for (const row of candidates) {
           if (options.minVersion !== void 0 && row.schemaVersion < options.minVersion) {
             skipped.push({ vaultId: row.vaultId, reason: "schema-drift" });
           } else versionOk.push(row);
         }
-        const provisioned = await Promise.all(versionOk.map((r) => this.db._shardVaultProvisioned(r.vaultId)));
+        const probes = await Promise.all(
+          versionOk.map(async (row) => {
+            try {
+              return { row, provisioned: await this.db._shardVaultProvisioned(row.vaultId) };
+            } catch (err) {
+              if (options.failFast) throw err;
+              return { row, error: err };
+            }
+          })
+        );
         const eligible = [];
-        versionOk.forEach((row, i) => {
-          if (provisioned[i]) eligible.push(row);
-          else skipped.push({ vaultId: row.vaultId, reason: "error", error: new import_kernel10.ShardProvisioningError(row.vaultId, row.partitionKey) });
-        });
+        for (const p of probes) {
+          if ("error" in p) skipped.push({ vaultId: p.row.vaultId, reason: "error", error: p.error });
+          else if (p.provisioned) eligible.push(p.row);
+          else skipped.push({ vaultId: p.row.vaultId, reason: "error", error: new import_kernel11.ShardProvisioningError(p.row.vaultId, p.row.partitionKey) });
+        }
         return { eligible, skipped };
       }
       /** @internal — registered push-model cross-vault derivations (#271 Insight Vault). */
@@ -1601,7 +1749,7 @@ var init_vault_group = __esm({
       withCrossVaultDerivation(spec) {
         const target = spec.target.vault;
         if (target === this.name || target.startsWith(`${this.name}${SHARD_SEPARATOR}`)) {
-          throw new import_kernel10.ValidationError(
+          throw new import_kernel11.ValidationError(
             `withCrossVaultDerivation: target.vault "${target}" is the "${this.name}" group itself or one of its shards \u2014 an Insight summary must target a SEPARATE analytics vault, never write back into client-shard data (it would breach the per-shard DEK boundary). Use a distinct vault name.`
           );
         }
@@ -1625,12 +1773,19 @@ var init_vault_group = __esm({
        * Insight Vault keyed by partition key. Shards behind `minVersion`,
        * unprovisioned, or whose read errors are reported in `skippedVaults` and
        * are not written (a stale summary is never left behind for a failed shard).
+       * A shard whose backend is unreachable (provisioning probe or read throws) is
+       * **skipped** (`reason: 'error'`) and the pass continues for the others — its
+       * prior summary is left intact. Pass `failFast: true` for the legacy
+       * all-or-nothing throw. Reconcile a lagging shard later with
+       * `refreshInsights({ only: [pk] })` or `refreshDerivation(pk)`.
        */
       async refreshInsights(options = {}) {
         if (this.crossVaultDerivations.length === 0) return { written: 0, skippedVaults: [] };
-        const { eligible, skipped } = await this.resolveEligible(
-          options.minVersion !== void 0 ? { minVersion: options.minVersion } : {}
-        );
+        const { eligible, skipped } = await this.resolveEligible({
+          ...options.minVersion !== void 0 ? { minVersion: options.minVersion } : {},
+          ...options.only !== void 0 ? { only: options.only } : {},
+          ...options.failFast !== void 0 ? { failFast: options.failFast } : {}
+        });
         let written = 0;
         for (const spec of this.crossVaultDerivations) {
           const results = await this.db.queryAcross(
@@ -1647,6 +1802,7 @@ var init_vault_group = __esm({
             const row = eligible[i];
             const res = results[i];
             if (!res || res.result === void 0) {
+              if (options.failFast && res?.error) throw res.error;
               skipped.push({ vaultId: row.vaultId, reason: "error", ...res?.error ? { error: res.error } : {} });
               continue;
             }
@@ -1662,6 +1818,14 @@ var init_vault_group = __esm({
         }
         return { written, skippedVaults: skipped };
       }
+      /**
+       * Reconcile one shard's Insight summaries after its backend was unreachable.
+       * Equivalent to `refreshInsights({ only: [partitionKey] })` — runs every
+       * registered derivation (autoPush or not) for just this shard.
+       */
+      async refreshDerivation(partitionKey) {
+        return this.refreshInsights({ only: [partitionKey] });
+      }
       /** @internal — re-derive + push every autoPush derivation's summary for one shard. */
       async _recomputeShardInsights(partitionKey) {
         const row = await this.registry.get(this.registryId(partitionKey));
@@ -1706,7 +1870,7 @@ var init_vault_group = __esm({
       async cutoverShard(partitionKey) {
         const vaultId = this.shardVaultId(partitionKey);
         const row = await this.registry.get(this.registryId(partitionKey));
-        if (!row) throw new import_kernel10.UnknownShardError(partitionKey, this.name);
+        if (!row) throw new import_kernel11.UnknownShardError(partitionKey, this.name);
         const target = this.template.version;
         const sv = await this.ensureStateVault();
         const base = { vaultId, group: this.name, currentVersion: row.schemaVersion, targetVersion: target };
@@ -1789,7 +1953,7 @@ var init_vault_group = __esm({
         let vault;
         if (!row) {
           if (this.group.sharding.autoCreate === false) {
-            throw new import_kernel10.UnknownShardError(key, this.group.name);
+            throw new import_kernel11.UnknownShardError(key, this.group.name);
           }
           vault = await this.group.createShard(key, this.group.sharding.regionOf?.(record));
         } else {
@@ -1801,6 +1965,17 @@ var init_vault_group = __esm({
       query() {
         return new ShardedQuery(this.group, this.collectionName, []);
       }
+      /**
+       * Cross-vault federated retrieval (#26): scatter-gather across all eligible
+       * shards — each shard runs its own trusted-tier `retrieve()`, then results are
+       * RRF-fused by rank only (no cross-vault statistics cross the DEK boundary).
+       * Every hit carries its originating `vault`. An unreachable, schema-drifted, or
+       * un-indexed shard is skipped (`skippedVaults`); pass `failFast` to re-throw
+       * the first per-shard error instead.
+       */
+      async retrieve(query, opts = {}) {
+        return retrieveAcross(this.group, this.collectionName, query, opts);
+      }
     };
     ShardedQuery = class _ShardedQuery {
       constructor(group, collectionName, clauses, coPartitionedLegs = [], broadcastLegs = []) {
@@ -1861,7 +2036,7 @@ var init_vault_group = __esm({
           this.group.template.configure(probe);
           for (const leg of this.coPartitionedLegs) {
             if (!probe.resolveRef(this.collectionName, leg.field)) {
-              throw new import_kernel10.CrossShardJoinError(
+              throw new import_kernel11.CrossShardJoinError(
                 `crossShardJoin("${leg.field}"): no ref() declared for "${leg.field}" on collection "${this.collectionName}" in template "${this.group.sharding.vaultTemplate}". Add refs: { ${leg.field}: ref('<target>') } to the template's collection options.`
               );
             }
@@ -1897,6 +2072,34 @@ var init_vault_group = __esm({
         }
         return { records: results, skippedVaults: skipped };
       }
+      /**
+       * @internal — distributed partial-reduce (#8). Fan out across eligible shards,
+       * but fold each shard's where-filtered records to a partial reducer STATE
+       * inside the per-shard callback (never returning the rows). Only the scalar
+       * `.aggregate()` path calls this, and that path rejects join legs upstream.
+       */
+      async fanoutReduce(spec, options = {}) {
+        const { eligible, skipped } = await this.group.resolveEligible(options);
+        const across = await this.group.db.queryAcross(
+          eligible.map((r) => r.vaultId),
+          async (vault) => {
+            this.group.template.configure(vault);
+            const coll = vault.collection(this.collectionName);
+            await coll.list();
+            let q = coll.query();
+            for (const c of this.clauses) q = q.where(c.field, c.op, c.value);
+            const rows = q.toArray();
+            return reduceToPartial(rows, spec);
+          },
+          { concurrency: options.concurrency ?? 1, create: false }
+        );
+        const partials = [];
+        for (const r of across) {
+          if (r.error) skipped.push({ vaultId: r.vault, reason: classifyShardSkip(r.error), error: r.error });
+          else partials.push(r.result);
+        }
+        return { partials, skippedVaults: skipped };
+      }
       /** Fan out across eligible shards, merge, then apply any broadcast dimension legs. */
       async toArray(options = {}) {
         const { records, skippedVaults } = await this.fanoutRecords(options);
@@ -1918,7 +2121,7 @@ var init_vault_group = __esm({
       /** @internal — joined queries don't support reactive/aggregate surfaces in v1. */
       assertNoJoinLegs(surface) {
         if (this.coPartitionedLegs.length || this.broadcastLegs.length) {
-          throw new import_kernel10.CrossShardJoinError(
+          throw new import_kernel11.CrossShardJoinError(
             `${surface}() is not supported on a ShardedQuery with crossShardJoin/broadcastJoin legs in v1. Use toArray() for joined cross-shard queries.`
           );
         }
@@ -1984,10 +2187,10 @@ var init_vault_group = __esm({
 // src/index.ts
 var index_exports = {};
 __export(index_exports, {
-  CrossShardJoinError: () => import_kernel12.CrossShardJoinError,
+  CrossShardJoinError: () => import_kernel13.CrossShardJoinError,
   CrossVaultDanglingRefError: () => CrossVaultDanglingRefError,
   CustodyApi: () => import_hub6.CustodyApi,
-  DataResidencyError: () => import_kernel12.DataResidencyError,
+  DataResidencyError: () => import_kernel13.DataResidencyError,
   DockedUnit: () => DockedUnit,
   FieldAuthorityPolicyMissingError: () => FieldAuthorityPolicyMissingError,
   FieldLevelDeferredError: () => FieldLevelDeferredError,
@@ -1998,14 +2201,14 @@ __export(index_exports, {
   NOYDB_MULTI_BUNDLE_MAGIC: () => NOYDB_MULTI_BUNDLE_MAGIC,
   NOYDB_MULTI_BUNDLE_PREFIX_BYTES: () => NOYDB_MULTI_BUNDLE_PREFIX_BYTES,
   NOYDB_MULTI_BUNDLE_VERSION: () => NOYDB_MULTI_BUNDLE_VERSION,
-  ReservedVaultNameError: () => import_kernel12.ReservedVaultNameError,
-  ShardProvisioningError: () => import_kernel12.ShardProvisioningError,
+  ReservedVaultNameError: () => import_kernel13.ReservedVaultNameError,
+  ShardProvisioningError: () => import_kernel13.ShardProvisioningError,
   SurfaceCadenceScheduler: () => SurfaceCadenceScheduler,
   SurfaceNotFoundError: () => SurfaceNotFoundError,
   SurfaceStateError: () => SurfaceStateError,
   UnitGraduationError: () => UnitGraduationError,
-  UnknownShardError: () => import_kernel12.UnknownShardError,
-  VaultTemplateNotFoundError: () => import_kernel12.VaultTemplateNotFoundError,
+  UnknownShardError: () => import_kernel13.UnknownShardError,
+  VaultTemplateNotFoundError: () => import_kernel13.VaultTemplateNotFoundError,
   agreeSurface: () => agreeSurface,
   applySurface: () => applySurface,
   createDeedOwner: () => import_hub6.createDeedOwner,
@@ -2035,7 +2238,7 @@ __export(index_exports, {
   writeMultiVaultBundle: () => writeMultiVaultBundle
 });
 module.exports = __toCommonJS(index_exports);
-var import_kernel11 = require("@noy-db/hub/kernel");
+var import_kernel12 = require("@noy-db/hub/kernel");
 var import_hub5 = require("@noy-db/hub");
 
 // src/dock/docked-unit.ts
@@ -2118,7 +2321,7 @@ async function meterGroup(group, opts = {}) {
 }
 
 // src/index.ts
-var import_kernel12 = require("@noy-db/hub/kernel");
+var import_kernel13 = require("@noy-db/hub/kernel");
 init_multi_bundle();
 init_extract_cross_vault();
 init_merge_compartment();
@@ -2206,15 +2409,15 @@ var Lobby = class {
   }
   async openVaultGroup(name, opts) {
     const db = this.noydb;
-    if (db.isClosed) throw new import_kernel11.ValidationError("Instance is closed");
-    if (name === import_hub5.STATE_VAULT_NAME) throw new import_kernel11.ReservedVaultNameError(name);
+    if (db.isClosed) throw new import_kernel12.ValidationError("Instance is closed");
+    if (name === import_hub5.STATE_VAULT_NAME) throw new import_kernel12.ReservedVaultNameError(name);
     const template = this.vaultTemplates.get(opts.sharding.vaultTemplate);
-    if (!template) throw new import_kernel11.VaultTemplateNotFoundError(opts.sharding.vaultTemplate);
+    if (!template) throw new import_kernel12.VaultTemplateNotFoundError(opts.sharding.vaultTemplate);
     const { VaultGroup: VaultGroup2 } = await Promise.resolve().then(() => (init_vault_group(), vault_group_exports));
     const { StateManagementVault: StateManagementVault2 } = await Promise.resolve().then(() => (init_state_vault(), state_vault_exports));
     const stateVault = opts.registry ? void 0 : await StateManagementVault2.open(db);
     const registry = opts.registry ?? stateVault.registry;
-    const group = new VaultGroup2(db, name, registry, opts.sharding, template, opts.cutoverOnOpen ?? false);
+    const group = new VaultGroup2(db, name, registry, opts.sharding, template, opts.cutoverOnOpen ?? false, opts.meta);
     if (stateVault) {
       group._attachStateVault(stateVault);
       await stateVault.recordManifest(opts.sharding.vaultTemplate, template);
@@ -2228,7 +2431,7 @@ var Lobby = class {
   }
   async openStateManagementVault() {
     const db = this.noydb;
-    if (db.isClosed) throw new import_kernel11.ValidationError("Instance is closed");
+    if (db.isClosed) throw new import_kernel12.ValidationError("Instance is closed");
     const { StateManagementVault: StateManagementVault2 } = await Promise.resolve().then(() => (init_state_vault(), state_vault_exports));
     return StateManagementVault2.open(db);
   }