@livestore/livestore 0.4.0-dev.18 → 0.4.0-dev.19

package/docs/patterns/encryption/index.md

@@ -0,0 +1,6 @@
+# Encryption
+
+LiveStore doesn't yet support encryption but might in the future.
+See [this issue](https://github.com/livestorejs/livestore/issues/70) for more details.
+
+For now you can implement encryption yourself e.g. by encrypting the events using a custom Effect Schema definition which applies a encryption transformation to the events.

package/docs/patterns/external-data/index.md

@@ -0,0 +1,5 @@
+# External Data
+
+LiveStore doesn't provide any built-in functionality to deal with external data. However, LiveStore was designed with this use case in mind (e.g. Overtone integrates with lots of external data like Spotify, ...). One way to deal with external data is to also model it as an event log and materialize it into LiveStore state as well.
+
+(If you're interested in learning more about the solution we're using for Overtone, get in touch.)

package/docs/patterns/file-management/index.md

@@ -0,0 +1,11 @@
+# File Management
+
+LiveStore doesn't have built-in support for file management but it's easy to use LiveStore alongside existing file storage solutions (e.g. S3).
+
+The basic idea is to store the file metadata (e.g. url, name, size, type) in LiveStore and the file content separately.
+
+## Example
+
+```ts
+// TODO (contribution welcome)
+```

package/docs/patterns/file-structure/index.md

@@ -0,0 +1,14 @@
+# File Structure
+
+While there are no strict requirements/conventions for how to structure your project (files, folders, etc), a common pattern is to have a `src/livestore` folder which contains all the LiveStore related code.
+
+```
+src/
+	livestore/
+		index.ts # re-exports everything
+		schema.ts # schema definitions
+		queries.ts # query definitions
+		events.ts # event definitions
+		...
+	...
+```

package/docs/patterns/list-ordering/index.md

@@ -0,0 +1,369 @@
+# List Ordering
+
+Fractional indexing enables conflict-free list ordering in distributed systems by assigning string-based position values that maintain lexicographic order. This makes it ideal for implementing drag-and-drop reordering in LiveStore applications where multiple clients may reorder items concurrently.
+
+To understand how the algorithm works, see these visual and interactive explanations:
+- [CRDT Fractional Indexing](https://madebyevan.com/algos/crdt-fractional-indexing/) by Evan Wallace - Visual explanation of the algorithm
+- [Implementing Fractional Indexing](https://observablehq.com/@dgreensp/implementing-fractional-indexing) by David Greenspan - Interactive notebook walkthrough
+
+## Why Fractional Indexing?
+
+Traditional numeric ordering (1, 2, 3...) requires renumbering multiple items when inserting or reordering, which creates conflicts in distributed systems. Fractional indexing solves this by:
+
+- Generating position values that can always be inserted between any two existing positions
+- Using lexicographic string ordering that works naturally with SQL `ORDER BY`
+- Eliminating the need for coordination between clients when reordering
+- Avoiding cascading updates when inserting items
+
+For example, inserting a new item between positions "a0" and "b0" generates "aV", which sorts correctly without modifying existing items.
+
+## Installation
+
+Install the `fractional-indexing` package from [Rocicorp](https://github.com/rocicorp/fractional-indexing) (the same team behind Replicache):
+
+```bash
+pnpm install fractional-indexing
+```
+
+## Schema Design
+
+Define a text column to store the fractional index and add a database index for efficient ordering:
+
+## `patterns/list-ordering/schema.ts`
+
+```ts filename="patterns/list-ordering/schema.ts"
+
+export const task = State.SQLite.table({
+  name: 'task',
+  columns: {
+    id: State.SQLite.integer({ primaryKey: true }),
+    title: State.SQLite.text({ default: '' }),
+    completed: State.SQLite.integer({ default: 0 }),
+    /** Fractional index for ordering tasks in the list */
+    order: State.SQLite.text({ nullable: false, default: '' }),
+  },
+  indexes: [
+    /** Index for efficient ordering queries */
+    { name: 'task_order', columns: ['order'] },
+  ],
+  deriveEvents: true,
+})
+
+export type Task = typeof task.Type
+
+export const tables = { task }
+```
+
+The `order` column stores string values like "a0", "a1", "aV" that maintain lexicographic ordering. Adding a database index on this column ensures efficient queries when retrieving ordered lists.
+
+## Creating Ordered Items
+
+Use `generateKeyBetween(a, b)` to generate position values when creating new items:
+
+## `patterns/list-ordering/create-item.ts`
+
+```ts filename="patterns/list-ordering/create-item.ts"
+
+declare const store: Store
+
+/** Create a new task at the end of the list */
+export const createTaskAtEnd = (title: string) => {
+  // Get the highest order value
+  const highestOrder = store.query(tables.task.select('order').orderBy('order', 'desc').limit(1))[0] ?? null
+
+  // Generate new order after the highest
+  const order = generateKeyBetween(highestOrder, null)
+
+  // Commit the event
+  store.commit(events.createTask({ title, order }))
+
+  return order
+}
+
+/** Create a new task at the beginning of the list */
+export const createTaskAtStart = (title: string) => {
+  // Get the lowest order value
+  const lowestOrder = store.query(tables.task.select('order').orderBy('order', 'asc').limit(1))[0] ?? null
+
+  // Generate new order before the lowest
+  const order = generateKeyBetween(null, lowestOrder)
+
+  store.commit(events.createTask({ title, order }))
+
+  return order
+}
+
+/** Create the very first task in an empty list */
+export const createFirstTask = (title: string) => {
+  // When the list is empty, use a simple default value
+  const order = 'a1'
+
+  store.commit(events.createTask({ title, order }))
+
+  return order
+}
+```
+
+### `patterns/list-ordering/events.ts`
+
+```ts filename="patterns/list-ordering/events.ts"
+
+export const events = {
+  createTask: Events.synced({
+    name: 'v1.CreateTask',
+    schema: Schema.Struct({
+      title: Schema.String,
+      order: Schema.String,
+    }),
+  }),
+  updateTaskOrder: Events.synced({
+    name: 'v1.UpdateTaskOrder',
+    schema: Schema.Struct({
+      id: Schema.Number,
+      order: Schema.String,
+    }),
+  }),
+}
+```
+
+### `patterns/list-ordering/schema.ts`
+
+```ts filename="patterns/list-ordering/schema.ts"
+
+export const task = State.SQLite.table({
+  name: 'task',
+  columns: {
+    id: State.SQLite.integer({ primaryKey: true }),
+    title: State.SQLite.text({ default: '' }),
+    completed: State.SQLite.integer({ default: 0 }),
+    /** Fractional index for ordering tasks in the list */
+    order: State.SQLite.text({ nullable: false, default: '' }),
+  },
+  indexes: [
+    /** Index for efficient ordering queries */
+    { name: 'task_order', columns: ['order'] },
+  ],
+  deriveEvents: true,
+})
+
+export type Task = typeof task.Type
+
+export const tables = { task }
+```
+
+Key patterns:
+- `generateKeyBetween(highest, null)` - Append to end of list
+- `generateKeyBetween(null, lowest)` - Prepend to start of list
+- Use `"a1"` as a simple default for the first item in an empty list
+
+## Reordering Items
+
+Handle drag-and-drop by generating a new position between the target boundaries:
+
+## `patterns/list-ordering/reorder.ts`
+
+```ts filename="patterns/list-ordering/reorder.ts"
+
+declare const store: Store
+
+/**
+ * Reorder a task by moving it between two other tasks
+ *
+ * @param taskId - The task to reorder
+ * @param beforeOrder - The order value of the task that will be before this one (null if moving to end)
+ * @param afterOrder - The order value of the task that will be after this one (null if moving to start)
+ */
+export const reorderTask = (taskId: number, beforeOrder: string | null, afterOrder: string | null) => {
+  // Generate a new fractional index between the two positions
+  const newOrder = generateKeyBetween(beforeOrder, afterOrder)
+
+  // Commit the update event
+  store.commit(events.updateTaskOrder({ id: taskId, order: newOrder }))
+
+  return newOrder
+}
+
+/**
+ * Handle drag-and-drop reordering
+ *
+ * This is a more complete example showing how to handle drag-and-drop
+ * with proper boundary checks.
+ */
+export const handleDragDrop = (draggedTaskId: number, targetTaskId: number, dropPosition: 'before' | 'after') => {
+  const before = dropPosition === 'before'
+
+  // Get the target task's order
+  const targetOrder = store.query(tables.task.select('order').where({ id: targetTaskId }).first({ behaviour: 'error' }))
+
+  // Find the nearest task in the drop direction
+  const nearestOrder =
+    store.query(
+      tables.task
+        .select('order')
+        .where({
+          order: { op: before ? '>' : '<', value: targetOrder },
+        })
+        .orderBy('order', before ? 'asc' : 'desc')
+        .limit(1),
+    )[0] ?? null
+
+  // Generate new order between target and nearest
+  const newOrder = generateKeyBetween(before ? targetOrder : nearestOrder, before ? nearestOrder : targetOrder)
+
+  // Commit the update
+  store.commit(events.updateTaskOrder({ id: draggedTaskId, order: newOrder }))
+
+  return newOrder
+}
+```
+
+### `patterns/list-ordering/events.ts`
+
+```ts filename="patterns/list-ordering/events.ts"
+
+export const events = {
+  createTask: Events.synced({
+    name: 'v1.CreateTask',
+    schema: Schema.Struct({
+      title: Schema.String,
+      order: Schema.String,
+    }),
+  }),
+  updateTaskOrder: Events.synced({
+    name: 'v1.UpdateTaskOrder',
+    schema: Schema.Struct({
+      id: Schema.Number,
+      order: Schema.String,
+    }),
+  }),
+}
+```
+
+### `patterns/list-ordering/schema.ts`
+
+```ts filename="patterns/list-ordering/schema.ts"
+
+export const task = State.SQLite.table({
+  name: 'task',
+  columns: {
+    id: State.SQLite.integer({ primaryKey: true }),
+    title: State.SQLite.text({ default: '' }),
+    completed: State.SQLite.integer({ default: 0 }),
+    /** Fractional index for ordering tasks in the list */
+    order: State.SQLite.text({ nullable: false, default: '' }),
+  },
+  indexes: [
+    /** Index for efficient ordering queries */
+    { name: 'task_order', columns: ['order'] },
+  ],
+  deriveEvents: true,
+})
+
+export type Task = typeof task.Type
+
+export const tables = { task }
+```
+
+The `generateKeyBetween` function automatically creates a string value that sorts lexicographically between the two boundary positions. Pass `null` to represent the start or end of the list.
+
+## Querying Ordered Lists
+
+Query items using standard SQL ordering on the fractional index column:
+
+## `patterns/list-ordering/query.ts`
+
+```ts filename="patterns/list-ordering/query.ts"
+
+declare const store: Store
+
+/**
+ * Query tasks in their proper order
+ *
+ * The fractional index values maintain lexicographic ordering,
+ * so we can simply order by the 'order' column.
+ */
+export const getOrderedTasks = () => {
+  return store.query(tables.task.select().orderBy('order', 'asc'))
+}
+
+/**
+ * Get the highest order value (for appending new items)
+ */
+export const getHighestOrder = (): string | null => {
+  const order = store.query(tables.task.select('order').orderBy('order', 'desc').limit(1))[0]
+
+  return order ?? null
+}
+
+/**
+ * Get the lowest order value (for prepending new items)
+ */
+export const getLowestOrder = (): string | null => {
+  const order = store.query(tables.task.select('order').orderBy('order', 'asc').limit(1))[0]
+
+  return order ?? null
+}
+```
+
+### `patterns/list-ordering/schema.ts`
+
+```ts filename="patterns/list-ordering/schema.ts"
+
+export const task = State.SQLite.table({
+  name: 'task',
+  columns: {
+    id: State.SQLite.integer({ primaryKey: true }),
+    title: State.SQLite.text({ default: '' }),
+    completed: State.SQLite.integer({ default: 0 }),
+    /** Fractional index for ordering tasks in the list */
+    order: State.SQLite.text({ nullable: false, default: '' }),
+  },
+  indexes: [
+    /** Index for efficient ordering queries */
+    { name: 'task_order', columns: ['order'] },
+  ],
+  deriveEvents: true,
+})
+
+export type Task = typeof task.Type
+
+export const tables = { task }
+```
+
+Since fractional index values maintain lexicographic ordering, you can use simple `ORDER BY` clauses without special handling.
+
+## Best Practices
+
+- **Use text/string columns** - Fractional index values are strings, not numbers
+- **Add database indexes** - Index the order column for efficient queries
+- **Validate lexicographic ordering** - Ensure your sorting logic uses standard string comparison, not locale-aware comparison (avoid `String.prototype.localeCompare()`)
+- **Handle empty lists** - Use a simple default like `"a1"` for the first item
+- **Consider bulk operations** - For creating multiple items at once, use `generateNKeysBetween(a, b, n)` from the same package
+
+## Real-World Example
+
+The [web-linearlite example](https://github.com/livestorejs/livestore/tree/dev/examples/web-linearlite) demonstrates fractional indexing for Kanban board ordering. It shows:
+- Schema definition with `kanbanorder` column
+- Creating issues with proper ordering
+- Drag-and-drop reordering across columns
+- Querying issues in order by status
+
+Check `examples/web-linearlite/src/livestore/schema/issue.ts` for the schema and `examples/web-linearlite/src/components/column.tsx` for the drag-and-drop implementation.
+
+## How It Works
+
+Fractional indexing generates position strings that always allow insertion between any two positions:
+
+```
+Initial:  a0    a1    a2
+Insert between a0 and a1: a0V (sorts: a0 < a0V < a1)
+Insert between a0V and a1: a0n (sorts: a0 < a0V < a0n < a1)
+```
+
+The algorithm ensures:
+- Position strings stay reasonably short
+- No coordination needed between clients
+- Conflicts resolve naturally through lexicographic ordering
+- Works seamlessly with LiveStore's event sourcing model
+
+For more details on the algorithm, see the [fractional-indexing documentation](https://github.com/rocicorp/fractional-indexing).

package/docs/patterns/offline/index.md

@@ -0,0 +1,32 @@
+# Offline Support
+
+- LiveStore supports offline data management out of the box. In order to make your app work fully offline, you might need to also consider the following:
+  - Design your app in a way to treat the network as an optional feature (e.g. when relying on other APIs / external data)
+  - Use service workers to cache assets locally (e.g. images, videos, etc.)
+
+## Tracking connectivity
+
+Use `store.networkStatus` to react to connectivity transitions. The subscribable emits every time the sync backend connection flips or the devtools latch simulates an offline state.
+
+## `patterns/offline/tracking-connectivity.ts`
+
+```ts filename="patterns/offline/tracking-connectivity.ts"
+
+declare const store: Store
+
+// ---cut---
+
+const status = await store.networkStatus.pipe(Effect.runPromise)
+if (status.isConnected === false) {
+  console.warn('Sync backend offline since', new Date(status.timestampMs))
+}
+
+await store.networkStatus.changes.pipe(
+  Stream.tap((next) => Effect.sync(() => console.log('network status updated', next))),
+  Stream.runDrain,
+  Effect.scoped,
+  Effect.runPromise,
+)
+```
+
+When devtools close the sync latch to simulate an offline client, `status.devtools.latchClosed` is `true`, allowing you to differentiate between real and simulated outages. Remember to dispose of long-lived subscriptions using the Effect scope you already manage for your runtime.

package/docs/patterns/orm/index.md

@@ -0,0 +1,18 @@
+# ORM
+
+- LiveStore has a built-in query builder which should be sufficient for most simple use cases.
+- You can always fall back to using raw SQL queries if you need more complex queries.
+- As long as the ORM allows supports synchronously generating SQL statements (and binding parameters), you should be able to use it with LiveStore.
+- Supported ORMs:
+  - [Knex](https://knexjs.org/)
+	- [Kysely](https://kysely.dev/)
+  - [Drizzle](https://orm.drizzle.team/)
+  - [Objection.js](https://vincit.github.io/objection.js/)
+- Unsupported ORMs:
+  - [Prisma](https://www.prisma.io/) (because it's async)
+
+## Example
+
+```ts
+// TODO (contribution welcome)
+```

package/docs/patterns/presence/index.md

@@ -0,0 +1,11 @@
+# Presence
+
+LiveStore doesn't yet have any built-in presence functionality (e.g. to track online/offline users).
+
+Common presence use cases are:
+- Track which users are online / in a room
+- Track which users are typing (e.g. in a chat)
+- Text cursor (similar to Google Docs)
+- Cursor movements (similar to Figma)
+
+For now it's recommend to implement presence functionality in your application or use a third party service (e.g. Liveblocks).

package/docs/patterns/rich-text-editing/index.md

@@ -0,0 +1,66 @@
+# Rich Text Editing
+
+LiveStore doesn't yet have built-in rich text primitives. It's currently recommended to pair LiveStore with a CRDT library (e.g. [Yjs](https://yjs.dev/), [Automerge](https://automerge.org/), or [Loro](https://loro.dev/)). LiveStore stores a reference (URL/ID) to the CRDT document in its event log. The CRDT library owns the text content and syncs it over its own transport. Both systems sync in parallel. LiveStore handles metadata while the CRDT library handles document bytes.
+
+## Example: linking a note to an Automerge document
+
+## `patterns/rich-text-editing/crdt-link.tsx`
+
+```tsx filename="patterns/rich-text-editing/crdt-link.tsx"
+
+declare const store: Store
+
+type NoteDoc = { body: string }
+
+// ---cut---
+
+export const note = State.SQLite.table({
+  name: 'note',
+  columns: {
+    id: State.SQLite.text({ primaryKey: true }),
+    title: State.SQLite.text({ default: '' }),
+    crdtDocUrl: State.SQLite.text({ nullable: false, default: '' }),
+  },
+})
+
+export type Note = typeof note.Type
+export const tables = { note }
+
+export const events = {
+  createNote: Events.synced({
+    name: 'v1.CreateNote',
+    schema: Schema.Struct({
+      noteId: Schema.String,
+      title: Schema.String,
+      crdtDocUrl: Schema.String,
+    }),
+  }),
+}
+
+export const getNoteCrdtRef = (noteId: string): string | undefined => {
+  return store.query(tables.note.select('crdtDocUrl').where({ id: noteId }))[0]
+}
+
+function Editor({ noteCrdtUrl }: { noteCrdtUrl: string }) {
+  const docUrl = noteCrdtUrl as AutomergeUrl
+  const [doc, changeDoc] = useDocument<NoteDoc>(docUrl, { suspense: true })
+
+  return (
+    <textarea
+      value={doc?.body ?? ''}
+      onChange={(event) =>
+        changeDoc((draft: NoteDoc) => {
+          updateText(draft, ['body'], event.target.value)
+        })
+      }
+    />
+  )
+}
+
+export function RichTextNoteEditor({ noteId }: { noteId: string }) {
+  const noteCrdtUrl = getNoteCrdtRef(noteId)
+  if (!noteCrdtUrl) return null
+
+  return <Editor noteCrdtUrl={noteCrdtUrl} />
+}
+```

package/docs/patterns/side-effects/index.md

@@ -0,0 +1,11 @@
+# Side effect
+
+TODO: Document how to safely run side-effects as response to LiveStore events.
+
+Notes for writing those docs:
+- Scenarios:
+  - Run side-effect in each client session
+  - Run side-effect only once per client (i.e. use a lock between client sessions)
+  - Run side-effect only once globally (will require some kind of global transaction)
+- How to deal with rollbacks/rebases
+- Allow for filtering events based on whether they have been confirmed by the sync backend or include unconfirmed events

package/docs/patterns/state-machines/index.md

@@ -0,0 +1,11 @@
+# State Machines
+
+LiveStore can be used to implement state machines or together with existing state machine libraries (e.g. [XState](https://stately.ai/docs/xstate)).
+
+The basic idea is to listen query results and emit events when the query results change. The state machine side effects can then further commit new mutations to LiveStore.
+
+## Example
+
+```ts
+// TODO (contribution welcome)
+```