@tanstack/db 0.6.0 → 0.6.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tanstack/db",
-  "version": "0.6.0",
+  "version": "0.6.2",
   "description": "A reactive client store for building super fast apps on sync",
   "author": "Kyle Mathews",
   "license": "MIT",
@@ -12,7 +12,8 @@
   "homepage": "https://tanstack.com/db",
   "keywords": [
     "optimistic",
-    "typescript"
+    "typescript",
+    "tanstack-intent"
   ],
   "type": "module",
   "main": "dist/cjs/index.cjs",

package/skills/db-core/SKILL.md

@@ -10,7 +10,7 @@ description: >
   createPacedMutations. Entry point for all TanStack DB skills.
 type: core
 library: db
-library_version: '0.5.30'
+library_version: '0.6.0'
 ---
 
 # TanStack DB — Core Concepts
@@ -33,6 +33,7 @@ hooks. In framework projects, import from the framework package directly.
 | Query data with where, join, groupBy, select     | db-core/live-queries/SKILL.md                        |
 | Insert, update, delete with optimistic UI        | db-core/mutations-optimistic/SKILL.md                |
 | Build a custom sync adapter                      | db-core/custom-adapter/SKILL.md                      |
+| Persist collections to SQLite (offline cache)    | db-core/persistence/SKILL.md                         |
 | Preload collections in route loaders             | meta-framework/SKILL.md                              |
 | Add offline transaction queueing                 | offline/SKILL.md (in @tanstack/offline-transactions) |
 
@@ -54,8 +55,9 @@ For framework-specific hooks:
 - Using React hooks? → react-db
 - Preloading in route loaders (Start, Next, Remix)? → meta-framework
 - Building an adapter for a new backend? → db-core/custom-adapter
+- Persisting collections to SQLite? → db-core/persistence
 - Need offline transaction persistence? → offline
 
 ## Version
 
-Targets @tanstack/db v0.5.30.
+Targets @tanstack/db v0.6.0.

package/skills/db-core/collection-setup/SKILL.md

@@ -6,13 +6,14 @@ description: >
   (ElectricSQL real-time sync), powerSyncCollectionOptions (PowerSync SQLite),
   rxdbCollectionOptions (RxDB), trailbaseCollectionOptions (TrailBase),
   localOnlyCollectionOptions, localStorageCollectionOptions. CollectionConfig
-  options: getKey, schema, sync, gcTime, autoIndex, syncMode (eager/on-demand/
-  progressive). StandardSchema validation with Zod/Valibot/ArkType. Collection
-  lifecycle (idle/loading/ready/error). Adapter-specific sync patterns including
-  Electric txid tracking and Query direct writes.
+  options: getKey, schema, sync, gcTime, autoIndex (default off), defaultIndexType,
+  syncMode (eager/on-demand, plus progressive for Electric). StandardSchema validation
+  with Zod/Valibot/ArkType. Collection lifecycle (idle/loading/ready/error).
+  Adapter-specific sync patterns including Electric txid tracking, Query direct
+  writes, and PowerSync query-driven sync with onLoad/onLoadSubset hooks.
 type: sub-skill
 library: db
-library_version: '0.5.30'
+library_version: '0.6.0'
 sources:
   - 'TanStack/db:docs/overview.md'
   - 'TanStack/db:docs/guides/schemas.md'
@@ -98,11 +99,29 @@ queryCollectionOptions({
 })
 ```
 
-| Mode          | Best for                                       | Data size |
-| ------------- | ---------------------------------------------- | --------- |
-| `eager`       | Mostly-static datasets                         | <10k rows |
-| `on-demand`   | Search, catalogs, large tables                 | >50k rows |
-| `progressive` | Collaborative apps needing instant first paint | Any       |
+| Mode          | Best for                                                       | Data size |
+| ------------- | -------------------------------------------------------------- | --------- |
+| `eager`       | Mostly-static datasets                                         | <10k rows |
+| `on-demand`   | Search, catalogs, large tables                                 | >50k rows |
+| `progressive` | Collaborative apps needing instant first paint (Electric only) | Any       |
+
+## Indexing
+
+Indexing is opt-in. The `autoIndex` option defaults to `"off"`. To enable automatic indexing, set `autoIndex: "eager"` and provide a `defaultIndexType`:
+
+```ts
+import { BasicIndex } from '@tanstack/db'
+
+createCollection(
+  queryCollectionOptions({
+    autoIndex: 'eager',
+    defaultIndexType: BasicIndex,
+    // ...
+  }),
+)
+```
+
+Without `defaultIndexType`, setting `autoIndex: "eager"` throws a `CollectionConfigurationError`. You can also create indexes manually with `collection.createIndex()` and remove them with `collection.removeIndex()`.
 
 ## Core Patterns
 
@@ -255,7 +274,7 @@ app.post('/api/todos', async (req, res) => {
 })
 ```
 
-`pg_current_xact_id()` must be queried inside the same SQL transaction as the mutation. Otherwise the txid doesn't match and `awaitTxId` stalls forever.
+`pg_current_xact_id()` must be queried inside the same SQL transaction as the mutation. Otherwise the txid doesn't match and `awaitTxId` times out (default 5 seconds).
 
 Source: docs/collections/electric-collection.md
 

package/skills/db-core/collection-setup/references/powersync-adapter.md

@@ -196,6 +196,10 @@ await tx.commit()
 await tx.isPersisted.promise
 ```
 
+## On-Demand Sync Mode
+
+PowerSync supports `on-demand` sync mode (query-driven sync), where only rows matching active live query predicates are loaded from SQLite into the collection. This can be combined with Sync Streams via `onLoad` (eager) or `onLoadSubset` (on-demand) hooks to also control which data the PowerSync Service syncs to the device. Use `extractSimpleComparisons` or `parseWhereExpression` to derive Sync Stream parameters dynamically from live query predicates.
+
 ## Complete Example
 
 ```typescript

package/skills/db-core/collection-setup/references/query-adapter.md

@@ -176,6 +176,38 @@ const productsCollection = createCollection(
 )
 ```
 
+## Common Mistakes
+
+### HIGH Function-based queryKey without shared prefix
+
+Wrong:
+
+```ts
+queryCollectionOptions({
+  queryKey: (opts) => {
+    if (opts.where) {
+      return ['products-filtered', JSON.stringify(opts.where)]
+    }
+    return ['products-all']
+  },
+})
+```
+
+Correct:
+
+```ts
+queryCollectionOptions({
+  queryKey: (opts) => {
+    if (opts.where) {
+      return ['products', JSON.stringify(opts.where)]
+    }
+    return ['products']
+  },
+})
+```
+
+When using a function-based `queryKey`, all derived keys must share the base key (`queryKey({})`) as a prefix. TanStack Query uses prefix matching for cache operations; if derived keys don't share the base prefix, cache updates silently miss entries, leading to stale data.
+
 ## Key Behaviors
 
 - `queryFn` result is treated as **complete state** -- missing items are deleted

package/skills/db-core/custom-adapter/SKILL.md

@@ -2,15 +2,17 @@
 name: db-core/custom-adapter
 description: >
   Building custom collection adapters for new backends. SyncConfig interface:
-  sync function receiving begin, write, commit, markReady, truncate primitives.
-  ChangeMessage format (insert, update, delete). loadSubset for on-demand sync.
-  LoadSubsetOptions (where, orderBy, limit, cursor). Expression parsing:
-  parseWhereExpression, parseOrderByExpression, extractSimpleComparisons,
-  parseLoadSubsetOptions. Collection options creator pattern. rowUpdateMode
-  (partial vs full). Subscription lifecycle and cleanup functions.
+  sync function receiving begin, write, commit, markReady, truncate, metadata
+  primitives. ChangeMessage format (insert, update, delete). loadSubset for
+  on-demand sync. LoadSubsetOptions (where, orderBy, limit, cursor). Expression
+  parsing: parseWhereExpression, parseOrderByExpression,
+  extractSimpleComparisons, parseLoadSubsetOptions. Collection options creator
+  pattern. rowUpdateMode (partial vs full). Subscription lifecycle and cleanup
+  functions. Persisted sync metadata API (metadata.row and metadata.collection)
+  for storing per-row and per-collection adapter state.
 type: sub-skill
 library: db
-library_version: '0.5.30'
+library_version: '0.6.0'
 sources:
   - 'TanStack/db:docs/guides/collection-options-creator.md'
   - 'TanStack/db:packages/db/src/collection/sync.ts'
@@ -38,7 +40,7 @@ function myBackendCollectionOptions<T>(config: {
   return {
     getKey: config.getKey,
     sync: {
-      sync: ({ begin, write, commit, markReady, collection }) => {
+      sync: ({ begin, write, commit, markReady, metadata, collection }) => {
         let isInitialSyncComplete = false
         const bufferedEvents: Array<any> = []
 
@@ -157,6 +159,53 @@ Mutation handlers must not resolve until server changes have synced back to the
 4. **Version/timestamp**: wait until sync stream catches up to mutation time
 5. **Provider method**: `await backend.waitForPendingWrites()`
 
+### Persisted sync metadata
+
+The `metadata` API on the sync config allows adapters to store per-row and per-collection metadata that persists across sync transactions. This is useful for tracking resume tokens, cursors, LSNs, or other adapter-specific state.
+
+The `metadata` object is available as a property on the sync config argument alongside `begin`, `write`, `commit`, etc. It is always provided, but without persistence the metadata is in-memory only and does not survive reloads. With persistence, metadata is durable across sessions.
+
+```ts
+sync: ({ begin, write, commit, markReady, metadata }) => {
+  // Row metadata: store per-row state (e.g. server version, ETag)
+  metadata.row.get(key) // => unknown | undefined
+  metadata.row.set(key, { version: 3, etag: 'abc' })
+  metadata.row.delete(key)
+
+  // Collection metadata: store per-collection state (e.g. resume cursor)
+  metadata.collection.get('cursor') // => unknown | undefined
+  metadata.collection.set('cursor', 'token_abc123')
+  metadata.collection.delete('cursor')
+  metadata.collection.list() // => [{ key: 'cursor', value: 'token_abc123' }]
+  metadata.collection.list('resume') // filter by prefix
+}
+```
+
+Row metadata writes are tied to the current transaction. When a row is deleted via `write({ type: 'delete', ... })`, its row metadata is automatically deleted. When a row is inserted, its metadata is set from `message.metadata` if provided, or deleted otherwise.
+
+Collection metadata writes staged before `truncate()` are preserved and commit atomically with the truncate transaction.
+
+**Typical usage — resume token:**
+
+```ts
+sync: ({ begin, write, commit, markReady, metadata }) => {
+  const lastCursor = metadata.collection.get('cursor') as string | undefined
+
+  const stream = subscribeFromCursor(lastCursor)
+  stream.on('data', (batch) => {
+    begin()
+    for (const item of batch.items) {
+      write({ type: item.type, key: item.id, value: item.data })
+    }
+    metadata.collection.set('cursor', batch.cursor)
+    commit()
+  })
+
+  stream.on('ready', () => markReady())
+  return () => stream.close()
+}
+```
+
 ### Expression parsing for predicate push-down
 
 ```ts
@@ -282,4 +331,4 @@ Source: packages/db/src/collection/sync.ts:110
 
 Getting-started simplicity (localOnly, eager mode) conflicts with production correctness (on-demand sync, race condition prevention, proper markReady handling). Agents optimizing for quick setup tend to skip buffering, markReady, and cleanup functions.
 
-See also: db-core/collection-setup/SKILL.md -- for built-in adapter patterns to model after.
+See also: db-core/collection-setup/SKILL.md — for built-in adapter patterns to model after.

package/skills/db-core/live-queries/SKILL.md

@@ -7,10 +7,14 @@ description: >
   isUndefined, and, or, not. Aggregates: count, sum, avg, min, max. String
   functions: upper, lower, length, concat, coalesce. Math: add. $selected
   namespace. createLiveQueryCollection. Derived collections. Predicate push-down.
-  Incremental view maintenance via differential dataflow (d2ts).
+  Incremental view maintenance via differential dataflow (d2ts). Virtual
+  properties ($synced, $origin, $key, $collectionId). Includes subqueries
+  for hierarchical data. toArray and concat(toArray(...)) scalar includes.
+  queryOnce for one-shot queries. createEffect for reactive side effects
+  (onEnter, onUpdate, onExit, onBatch).
 type: sub-skill
 library: db
-library_version: '0.5.30'
+library_version: '0.6.0'
 sources:
   - 'TanStack/db:docs/guides/live-queries.md'
   - 'TanStack/db:packages/db/src/query/builder/index.ts'
@@ -191,6 +195,162 @@ const activeUserPosts = createLiveQueryCollection((q) =>
 
 Create derived collections once at module scope and reuse them. Do not recreate on every render or navigation.
 
+## Virtual Properties
+
+Live query results include computed, read-only virtual properties on every row:
+
+- `$synced`: `true` when the row is confirmed by sync; `false` when it is still optimistic.
+- `$origin`: `"local"` if the last confirmed change came from this client, otherwise `"remote"`.
+- `$key`: the row key for the result.
+- `$collectionId`: the source collection ID.
+
+These props are added automatically and can be used in `where`, `select`, and `orderBy` clauses. Do not persist them back to storage.
+
+## Includes (Subqueries in Select)
+
+Embed a correlated subquery inside `select()` to produce hierarchical (nested) data. The subquery must contain a `where` with an `eq()` that correlates a parent field with a child field. Three materialization modes are available.
+
+### Collection includes (default)
+
+Return a child `Collection` on each parent row:
+
+```ts
+import { eq, createLiveQueryCollection } from '@tanstack/db'
+
+const projectsWithIssues = createLiveQueryCollection((q) =>
+  q.from({ p: projectsCollection }).select(({ p }) => ({
+    id: p.id,
+    name: p.name,
+    issues: q
+      .from({ i: issuesCollection })
+      .where(({ i }) => eq(i.projectId, p.id))
+      .select(({ i }) => ({
+        id: i.id,
+        title: i.title,
+      })),
+  })),
+)
+
+// Each row's `issues` is a live Collection
+for (const project of projectsWithIssues) {
+  console.log(project.name, project.issues.toArray)
+}
+```
+
+### Array includes with toArray()
+
+Wrap the subquery in `toArray()` to get a plain array of scalar values instead of a Collection:
+
+```ts
+import { eq, toArray, createLiveQueryCollection } from '@tanstack/db'
+
+const messagesWithParts = createLiveQueryCollection((q) =>
+  q.from({ m: messagesCollection }).select(({ m }) => ({
+    id: m.id,
+    contentParts: toArray(
+      q
+        .from({ c: chunksCollection })
+        .where(({ c }) => eq(c.messageId, m.id))
+        .orderBy(({ c }) => c.timestamp)
+        .select(({ c }) => c.text),
+    ),
+  })),
+)
+// row.contentParts is string[]
+```
+
+### Concatenated scalar with concat(toArray())
+
+Wrap `toArray()` in `concat()` to join the scalar results into a single string:
+
+```ts
+import { eq, toArray, concat, createLiveQueryCollection } from '@tanstack/db'
+
+const messagesWithContent = createLiveQueryCollection((q) =>
+  q.from({ m: messagesCollection }).select(({ m }) => ({
+    id: m.id,
+    content: concat(
+      toArray(
+        q
+          .from({ c: chunksCollection })
+          .where(({ c }) => eq(c.messageId, m.id))
+          .orderBy(({ c }) => c.timestamp)
+          .select(({ c }) => c.text),
+      ),
+    ),
+  })),
+)
+// row.content is a single concatenated string
+```
+
+### Includes rules
+
+- The subquery **must** have a `where` clause with an `eq()` correlating a parent alias with a child alias. The library extracts this automatically as the join condition.
+- `toArray()` and `concat(toArray())` require the subquery to use a **scalar** `select` (e.g., `select(({ c }) => c.text)`), not an object select.
+- Collection includes (bare subquery) require an **object** `select`.
+- Includes subqueries are compiled into the same incremental pipeline as the parent query -- they are not separate live queries.
+
+## One-Shot Queries with queryOnce
+
+For non-reactive, one-time snapshots use `queryOnce`. It creates a live query collection, preloads it, extracts the results, and cleans up automatically.
+
+```ts
+import { eq, queryOnce } from '@tanstack/db'
+
+const activeUsers = await queryOnce((q) =>
+  q
+    .from({ user: usersCollection })
+    .where(({ user }) => eq(user.active, true))
+    .select(({ user }) => ({ id: user.id, name: user.name })),
+)
+
+// With findOne — resolves to T | undefined
+const user = await queryOnce((q) =>
+  q
+    .from({ user: usersCollection })
+    .where(({ user }) => eq(user.id, userId))
+    .findOne(),
+)
+```
+
+Use `queryOnce` for scripts, loaders, data export, tests, or AI/LLM context building. For UI bindings and reactive updates, use live queries instead.
+
+## Reactive Effects (createEffect)
+
+Reactive effects respond to query result _changes_ without materializing the full result set. Effects fire callbacks when rows enter, exit, or update within a query result — like a database trigger on an arbitrary live query.
+
+```ts
+import { createEffect, eq } from '@tanstack/db'
+
+const effect = createEffect({
+  query: (q) =>
+    q
+      .from({ msg: messagesCollection })
+      .where(({ msg }) => eq(msg.role, 'user')),
+  skipInitial: true,
+  onEnter: async (event, ctx) => {
+    await processNewMessage(event.value, { signal: ctx.signal })
+  },
+  onExit: (event) => {
+    console.log('Message left result set:', event.key)
+  },
+  onError: (error, event) => {
+    console.error(`Failed to process ${event.key}:`, error)
+  },
+})
+
+// Dispose when no longer needed
+await effect.dispose()
+```
+
+| Use case                        | Approach                                              |
+| ------------------------------- | ----------------------------------------------------- |
+| Display query results in UI     | Live query collection + `useLiveQuery`                |
+| React to changes (side effects) | `createEffect` with `onEnter` / `onUpdate` / `onExit` |
+| Inspect full batch of changes   | `createEffect` with `onBatch`                         |
+
+Key options: `id` (optional), `query`, `skipInitial` (skip existing rows on init), `onEnter`, `onUpdate`, `onExit`, `onBatch`, `onError`, `onSourceError`. The `ctx.signal` aborts when the effect is disposed.
+
 ## Common Mistakes
 
 ### CRITICAL: Using === instead of eq()

package/skills/db-core/mutations-optimistic/SKILL.md

@@ -9,7 +9,7 @@ description: >
   onInsert/onUpdate/onDelete handlers. PendingMutation type. Transaction.isPersisted.
 type: sub-skill
 library: db
-library_version: '0.5.30'
+library_version: '0.6.0'
 sources:
   - 'TanStack/db:docs/guides/mutations.md'
   - 'TanStack/db:packages/db/src/transactions.ts'