@livestore/livestore 0.4.0-dev.17 → 0.4.0-dev.19

package/docs/reference/state/sqlite-schema/index.md

@@ -0,0 +1,306 @@
+# SQLite State Schema
+
+LiveStore provides a schema definition language for defining your database tables and mutation definitions using explicit column configurations. LiveStore automatically migrates your database schema when you change your schema definitions.
+
+> **Alternative Approach**: You can also define tables using [Effect Schema with annotations](/reference/state/sqlite-schema-effect) for type-safe schema definitions.
+
+### Example
+
+<Tabs syncKey="package-manager">
+  <TabItem label="schema.ts">
+
+## `reference/state/sqlite-schema/schema.ts`
+
+```ts filename="reference/state/sqlite-schema/schema.ts"
+
+// You can model your state as SQLite tables (https://docs.livestore.dev/reference/state/sqlite-schema)
+export const tables = {
+  todos: State.SQLite.table({
+    name: 'todos',
+    columns: {
+      id: State.SQLite.text({ primaryKey: true }),
+      text: State.SQLite.text({ default: '' }),
+      completed: State.SQLite.boolean({ default: false }),
+      deletedAt: State.SQLite.integer({ nullable: true, schema: Schema.DateFromNumber }),
+    },
+  }),
+  // Client documents can be used for local-only state (e.g. form inputs)
+  uiState: State.SQLite.clientDocument({
+    name: 'uiState',
+    schema: Schema.Struct({ newTodoText: Schema.String, filter: Schema.Literal('all', 'active', 'completed') }),
+    default: { id: SessionIdSymbol, value: { newTodoText: '', filter: 'all' } },
+  }),
+}
+
+// Events describe data changes (https://docs.livestore.dev/reference/events)
+export const events = {
+  todoCreated: Events.synced({
+    name: 'v1.TodoCreated',
+    schema: Schema.Struct({ id: Schema.String, text: Schema.String }),
+  }),
+  todoCompleted: Events.synced({
+    name: 'v1.TodoCompleted',
+    schema: Schema.Struct({ id: Schema.String }),
+  }),
+  todoUncompleted: Events.synced({
+    name: 'v1.TodoUncompleted',
+    schema: Schema.Struct({ id: Schema.String }),
+  }),
+  todoDeleted: Events.synced({
+    name: 'v1.TodoDeleted',
+    schema: Schema.Struct({ id: Schema.String, deletedAt: Schema.Date }),
+  }),
+  todoClearedCompleted: Events.synced({
+    name: 'v1.TodoClearedCompleted',
+    schema: Schema.Struct({ deletedAt: Schema.Date }),
+  }),
+  uiStateSet: tables.uiState.set,
+}
+
+// Materializers are used to map events to state (https://docs.livestore.dev/reference/state/materializers)
+const materializers = State.SQLite.materializers(events, {
+  'v1.TodoCreated': ({ id, text }) => tables.todos.insert({ id, text, completed: false }),
+  'v1.TodoCompleted': ({ id }) => tables.todos.update({ completed: true }).where({ id }),
+  'v1.TodoUncompleted': ({ id }) => tables.todos.update({ completed: false }).where({ id }),
+  'v1.TodoDeleted': ({ id, deletedAt }) => tables.todos.update({ deletedAt }).where({ id }),
+  'v1.TodoClearedCompleted': ({ deletedAt }) => tables.todos.update({ deletedAt }).where({ completed: true }),
+})
+
+const state = State.SQLite.makeState({ tables, materializers })
+
+export const schema = makeSchema({ events, state })
+```
+
+</TabItem>
+</Tabs>
+
+## Defining Tables
+
+Define SQLite tables using explicit column definitions:
+
+## `reference/state/sqlite-schema/columns/table-basic.ts`
+
+```ts filename="reference/state/sqlite-schema/columns/table-basic.ts"
+
+export const userTable = State.SQLite.table({
+  name: 'users',
+  columns: {
+    id: State.SQLite.text({ primaryKey: true }),
+    email: State.SQLite.text(),
+    name: State.SQLite.text(),
+    age: State.SQLite.integer({ default: 0 }),
+    isActive: State.SQLite.boolean({ default: true }),
+    metadata: State.SQLite.json({ nullable: true }),
+  },
+  indexes: [{ name: 'idx_users_email', columns: ['email'], isUnique: true }],
+})
+```
+
+Use the optional `indexes` array to declare secondary indexes or enforce uniqueness (set `isUnique: true`).
+
+## Column Types
+
+You can use these column types when defining tables:
+
+### Core SQLite column types
+
+- `State.SQLite.text`: A text field, returns `string`.
+- `State.SQLite.integer`: An integer field, returns `number`.
+- `State.SQLite.real`: A real field (floating point number), returns `number`.
+- `State.SQLite.blob`: A blob field (binary data), returns `Uint8Array`.
+
+### Higher level column types
+
+- `State.SQLite.boolean`: An integer field that stores `0` for `false` and `1` for `true` and returns a `boolean`.
+- `State.SQLite.json`: A text field that stores a stringified JSON object and returns a decoded JSON value.
+- `State.SQLite.datetime`: A text field that stores dates as ISO 8601 strings and returns a `Date`.
+- `State.SQLite.datetimeInteger`: A integer field that stores dates as the number of milliseconds since the epoch and returns a `Date`.
+
+### Custom column schemas
+
+You can also provide a custom schema for a column which is used to automatically encode and decode the column value.
+
+### Example: JSON-encoded struct
+
+## `reference/state/sqlite-schema/columns/json-struct.ts`
+
+```ts filename="reference/state/sqlite-schema/columns/json-struct.ts"
+
+export const UserMetadata = Schema.Struct({
+  petName: Schema.String,
+  favoriteColor: Schema.Literal('red', 'blue', 'green'),
+})
+
+export const userTable = State.SQLite.table({
+  name: 'user',
+  columns: {
+    id: State.SQLite.text({ primaryKey: true }),
+    name: State.SQLite.text(),
+    metadata: State.SQLite.json({ schema: UserMetadata }),
+  },
+})
+```
+
+### Schema migrations
+
+Migration strategies:
+
+- `auto`: Automatically migrate the database to the newest schema and rematerializes the state from the eventlog.
+- `manual`: Manually migrate the database to the newest schema.
+
+## Client documents
+
+- Meant for convenience
+- Client-only
+- Goal: Similar ease of use as `React.useState`
+- When schema changes in a non-backwards compatible way, previous events are dropped and the state is reset
+  - Don't use client documents for sensitive data which must not be lost
+- Implies
+  - Table with `id` and `value` columns
+  - `${MyTable}Set` event + materializer (which are auto-registered)
+
+### Basic usage
+
+## `reference/state/sqlite-schema/columns/client-document-basic.tsx`
+
+```tsx filename="reference/state/sqlite-schema/columns/client-document-basic.tsx"
+
+export const uiState = State.SQLite.clientDocument({
+  name: 'UiState',
+  schema: Schema.Struct({
+    newTodoText: Schema.String,
+    filter: Schema.Literal('all', 'active', 'completed'),
+  }),
+  default: { id: SessionIdSymbol, value: { newTodoText: '', filter: 'all' } },
+})
+
+export const readUiState = (store: Store): { newTodoText: string; filter: 'all' | 'active' | 'completed' } =>
+  store.query(uiState.get())
+
+export const setNewTodoText = (store: Store, newTodoText: string): void => {
+  store.commit(uiState.set({ newTodoText }))
+}
+
+export const UiStateFilter: React.FC = () => {
+  const [state, setState] = useClientDocument(uiState)
+
+  const showActive = React.useCallback(() => {
+    setState({ filter: 'active' })
+  }, [setState])
+
+  const showAll = React.useCallback(() => {
+    setState({ filter: 'all' })
+  }, [setState])
+
+  return (
+    <div>
+      <button type="button" onClick={showAll}>
+        All
+      </button>
+      <button type="button" onClick={showActive}>
+        Active ({state.filter === 'active' ? 'selected' : 'select'})
+      </button>
+    </div>
+  )
+}
+```
+
+### KV-style client document
+
+Sometimes you want a simple key-value store for arbitrary values without partial merging. You can model this by using `Schema.Any` as the value schema. With `Schema.Any`, updates fully replace the stored value (no partial merge semantics).
+
+## `reference/state/sqlite-schema/columns/client-document-kv.tsx`
+
+```tsx filename="reference/state/sqlite-schema/columns/client-document-kv.tsx"
+
+export const kv = State.SQLite.clientDocument({
+  name: 'Kv',
+  schema: Schema.Any,
+  default: { value: null },
+})
+
+export const readKvValue = (store: Store, id: string): unknown => store.query(kv.get(id))
+
+export const setKvValue = (store: Store, id: string, value: unknown): void => {
+  store.commit(kv.set(value, id))
+}
+
+export const KvViewer: React.FC<{ id: string }> = ({ id }) => {
+  const [value, setValue] = useClientDocument(kv, id)
+
+  return (
+    <button type="button" onClick={() => setValue('hello')}>
+      Current value: {JSON.stringify(value)}
+    </button>
+  )
+}
+```
+
+## Column Types
+
+You can use these column types:
+
+#### Core SQLite column types
+
+- `State.SQLite.text`: A text field, returns `string`.
+- `State.SQLite.integer`: An integer field, returns `number`.
+- `State.SQLite.real`: A real field (floating point number), returns `number`.
+- `State.SQLite.blob`: A blob field (binary data), returns `Uint8Array`.
+
+#### Higher level column types
+
+- `State.SQLite.boolean`: An integer field that stores `0` for `false` and `1` for `true` and returns a `boolean`.
+- `State.SQLite.json`: A text field that stores a stringified JSON object and returns a decoded JSON value.
+- `State.SQLite.datetime`: A text field that stores dates as ISO 8601 strings and returns a `Date`.
+- `State.SQLite.datetimeInteger`: A integer field that stores dates as the number of milliseconds since the epoch and returns a `Date`.
+
+#### Custom column schemas
+
+You can also provide a custom schema for a column which is used to automatically encode and decode the column value.
+
+#### Example: JSON-encoded struct
+
+## `reference/state/sqlite-schema/columns/json-struct.ts`
+
+```ts filename="reference/state/sqlite-schema/columns/json-struct.ts"
+
+export const UserMetadata = Schema.Struct({
+  petName: Schema.String,
+  favoriteColor: Schema.Literal('red', 'blue', 'green'),
+})
+
+export const userTable = State.SQLite.table({
+  name: 'user',
+  columns: {
+    id: State.SQLite.text({ primaryKey: true }),
+    name: State.SQLite.text(),
+    metadata: State.SQLite.json({ schema: UserMetadata }),
+  },
+})
+```
+
+## Best Practices
+
+### Column Configuration
+
+- Use appropriate SQLite column types for your data (text, integer, real, blob)
+- Set `primaryKey: true` for primary key columns
+- Use `nullable: true` for columns that can contain NULL values
+- Provide meaningful `default` values where appropriate
+- Add unique constraints via table `indexes` using `isUnique: true`
+
+### Schema Design
+
+- Choose column types that match your data requirements
+- Use custom schemas with `State.SQLite.json()` for complex data structures
+- Group related table definitions in the same module
+- Use descriptive table and column names
+
+### General Practices
+
+- It's usually recommend to **not distinguish** between app state vs app data but rather keep all state in LiveStore.
+	- This means you'll rarely use `React.useState` when using LiveStore
+- In some cases for "fast changing values" it can make sense to keep a version of a state value outside of LiveStore with a reactive setter for React and a debounced setter for LiveStore to avoid excessive LiveStore mutations. Cases where this can make sense can include:
+  - Text input / rich text editing
+  - Scroll position tracking, resize events, move/drag events
+  - ...

package/docs/reference/state/sqlite-schema-effect/index.md

@@ -0,0 +1,300 @@
+# SQLite State Schema (Effect Schema)
+
+LiveStore supports defining SQLite tables using Effect Schema with annotations for database constraints. This approach provides strong type safety, composability, and automatic type mapping from TypeScript to SQLite.
+
+> **Note**: This approach will become the default once Effect Schema v4 is released. See [livestore#382](https://github.com/livestorejs/livestore/issues/382) for details.
+>
+> For the traditional column-based approach, see [SQLite State Schema](/reference/state/sqlite-schema).
+
+## Basic Usage
+
+Define tables using Effect Schema with database constraint annotations:
+
+## `reference/state/sqlite-schema/effect/basic.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/basic.ts"
+
+const UserSchema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  email: Schema.String.pipe(State.SQLite.withUnique),
+  name: Schema.String,
+  age: Schema.Int.pipe(State.SQLite.withDefault(0)),
+  isActive: Schema.Boolean.pipe(State.SQLite.withDefault(true)),
+  metadata: Schema.optional(
+    Schema.Record({
+      key: Schema.String,
+      value: Schema.Unknown,
+    }),
+  ),
+}).annotations({ title: 'users' })
+
+export const userTable = State.SQLite.table({ schema: UserSchema })
+```
+
+## Schema Annotations
+
+You can annotate schema fields with database constraints:
+
+### Primary Keys
+
+## `reference/state/sqlite-schema/effect/primary-key.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/primary-key.ts"
+
+const _schema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  // Other fields...
+})
+```
+
+**Important**: Primary key columns cannot be nullable. This will throw an error:
+
+## `reference/state/sqlite-schema/effect/primary-key-nullable.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/primary-key-nullable.ts"
+
+// ❌ This will throw an error at runtime because primary keys cannot be nullable
+const _badSchema = Schema.Struct({
+  id: Schema.NullOr(Schema.String).pipe(State.SQLite.withPrimaryKey),
+})
+```
+
+### Auto-Increment
+
+## `reference/state/sqlite-schema/effect/auto-increment.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/auto-increment.ts"
+
+const _schema = Schema.Struct({
+  id: Schema.Int.pipe(State.SQLite.withPrimaryKey, State.SQLite.withAutoIncrement),
+  // Other fields...
+})
+```
+
+### Default Values
+
+## `reference/state/sqlite-schema/effect/default-values.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/default-values.ts"
+
+const _schema = Schema.Struct({
+  status: Schema.String.pipe(State.SQLite.withDefault('active')),
+  createdAt: Schema.String.pipe(State.SQLite.withDefault('CURRENT_TIMESTAMP')),
+  count: Schema.Int.pipe(State.SQLite.withDefault(0)),
+})
+```
+
+### Unique Constraints
+
+## `reference/state/sqlite-schema/effect/unique-constraints.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/unique-constraints.ts"
+
+const _schema = Schema.Struct({
+  email: Schema.String.pipe(State.SQLite.withUnique),
+  username: Schema.String.pipe(State.SQLite.withUnique),
+})
+```
+
+Unique annotations automatically create unique indexes.
+
+### Custom Column Types
+
+Override the automatically inferred SQLite column type:
+
+## `reference/state/sqlite-schema/effect/custom-column-types.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/custom-column-types.ts"
+
+const _schema = Schema.Struct({
+  // Store a number as text instead of real
+  version: Schema.Number.pipe(State.SQLite.withColumnType('text')),
+  // Store binary data as blob
+  data: Schema.Uint8Array.pipe(State.SQLite.withColumnType('blob')),
+})
+```
+
+### Combining Annotations
+
+Annotations can be chained together:
+
+## `reference/state/sqlite-schema/effect/combining-annotations.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/combining-annotations.ts"
+
+const _schema = Schema.Struct({
+  id: Schema.Int.pipe(State.SQLite.withPrimaryKey, State.SQLite.withAutoIncrement),
+  email: Schema.String.pipe(State.SQLite.withUnique, State.SQLite.withColumnType('text')),
+})
+```
+
+## Table Naming
+
+You can specify table names in several ways:
+
+### Using Schema Annotations
+
+## `reference/state/sqlite-schema/effect/table-name-annotations.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/table-name-annotations.ts"
+
+// Using title annotation
+const UserSchema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  name: Schema.String,
+}).annotations({ title: 'users' })
+
+export const userTable = State.SQLite.table({ schema: UserSchema })
+
+// Using identifier annotation
+const PostSchema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  title: Schema.String,
+}).annotations({ identifier: 'posts' })
+
+export const postTable = State.SQLite.table({ schema: PostSchema })
+```
+
+### Explicit Name
+
+## `reference/state/sqlite-schema/effect/table-name-explicit.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/table-name-explicit.ts"
+
+const UserSchema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  name: Schema.String,
+})
+
+export const userTable = State.SQLite.table({
+  name: 'users',
+  schema: UserSchema,
+})
+```
+
+**Note**: Title annotation takes precedence over identifier annotation.
+
+## Type Mapping
+
+Effect Schema types are automatically mapped to SQLite column types:
+
+| Schema Type | SQLite Type | TypeScript Type |
+|-------------|-------------|-----------------|
+| `Schema.String` | `text` | `string` |
+| `Schema.Number` | `real` | `number` |
+| `Schema.Int` | `integer` | `number` |
+| `Schema.Boolean` | `integer` | `boolean` |
+| `Schema.Date` | `text` | `Date` |
+| `Schema.BigInt` | `text` | `bigint` |
+| Complex types (Struct, Array, etc.) | `text` (JSON encoded) | Decoded type |
+| `Schema.optional(T)` | Nullable column | `T \| undefined` |
+| `Schema.NullOr(T)` | Nullable column | `T \| null` |
+
+## Advanced Examples
+
+### Complex Schema with Multiple Constraints
+
+## `reference/state/sqlite-schema/effect/advanced-product.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/advanced-product.ts"
+
+const ProductSchema = Schema.Struct({
+  id: Schema.Int.pipe(State.SQLite.withPrimaryKey, State.SQLite.withAutoIncrement),
+  sku: Schema.String.pipe(State.SQLite.withUnique),
+  name: Schema.String,
+  price: Schema.Number.pipe(State.SQLite.withDefault(0)),
+  category: Schema.Literal('electronics', 'clothing', 'books'),
+  metadata: Schema.optional(
+    Schema.Struct({
+      weight: Schema.Number,
+      dimensions: Schema.Struct({
+        width: Schema.Number,
+        height: Schema.Number,
+        depth: Schema.Number,
+      }),
+    }),
+  ),
+  isActive: Schema.Boolean.pipe(State.SQLite.withDefault(true)),
+  createdAt: Schema.Date.pipe(State.SQLite.withDefault('CURRENT_TIMESTAMP')),
+}).annotations({ title: 'products' })
+
+export const productTable = State.SQLite.table({ schema: ProductSchema })
+```
+
+### Working with Schema.Class
+
+## `reference/state/sqlite-schema/effect/schema-class.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/schema-class.ts"
+
+class User extends Schema.Class<User>('User')({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  email: Schema.String.pipe(State.SQLite.withUnique),
+  name: Schema.String,
+  age: Schema.Int,
+}) {}
+
+export const userTable = State.SQLite.table({
+  name: 'users',
+  schema: User,
+})
+```
+
+### Custom Indexes
+
+## `reference/state/sqlite-schema/effect/custom-indexes.ts`
+
+```ts filename="reference/state/sqlite-schema/effect/custom-indexes.ts"
+
+const PostSchema = Schema.Struct({
+  id: Schema.String.pipe(State.SQLite.withPrimaryKey),
+  title: Schema.String,
+  authorId: Schema.String,
+  createdAt: Schema.Date,
+}).annotations({ title: 'posts' })
+
+export const postTable = State.SQLite.table({
+  schema: PostSchema,
+  indexes: [
+    { name: 'idx_posts_author', columns: ['authorId'] },
+    { name: 'idx_posts_created', columns: ['createdAt'] },
+  ],
+})
+```
+
+## Best Practices
+
+### Schema Design
+
+- Always use `withPrimaryKey` for primary key columns - never combine it with nullable types
+- Use `Schema.optional()` for truly optional fields that can be undefined
+- Use `Schema.NullOr()` for fields that can explicitly be set to null
+- Leverage schema annotations like `title` or `identifier` to avoid repeating table names
+- Group related schemas in the same module for better organization
+
+### Type Safety
+
+- Let TypeScript infer table types rather than explicitly typing them
+- Use Effect Schema's refinements and transformations for data validation
+- Prefer Effect Schema's built-in types (`Schema.Int`, `Schema.Date`) over generic types where appropriate
+
+### Performance
+
+- Be mindful of complex types stored as JSON - they can impact query performance
+- Use appropriate indexes for frequently queried columns
+- Consider using `withColumnType` to optimize storage for specific use cases
+
+## When to Use This Approach
+
+**Use Effect Schema-based tables when:**
+- You already have Effect Schema definitions to reuse
+- You prefer Effect Schema's composability and transformations
+- Your schemas are shared across different parts of your application
+- You want automatic type mapping and strong type safety
+- You plan to migrate to Effect Schema v4 when it becomes available
+
+**Consider column-based tables when:**
+- You need precise control over SQLite column types
+- You're migrating from existing SQLite schemas
+- You prefer explicit column configuration
+- You're not already using Effect Schema extensively in your project