tablinum 0.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (75) hide show
  1. package/.changeset/README.md +8 -0
  2. package/.changeset/config.json +11 -0
  3. package/.context/attachments/pasted_text_2026-03-07_14-02-40.txt +571 -0
  4. package/.context/attachments/pasted_text_2026-03-07_15-48-27.txt +498 -0
  5. package/.context/notes.md +0 -0
  6. package/.context/plans/add-changesets-to-douala-v4.md +48 -0
  7. package/.context/plans/dexie-js-style-query-language-for-localstr.md +115 -0
  8. package/.context/plans/dexie-js-style-query-language-with-per-collection-.md +336 -0
  9. package/.context/plans/implementation-plan-localstr-v0-2.md +263 -0
  10. package/.context/plans/project-init-effect-v4-bun-oxlint-oxfmt-vitest.md +71 -0
  11. package/.context/plans/revise-localstr-prd-v0-2.md +132 -0
  12. package/.context/plans/svelte-5-runes-bindings-for-localstr.md +233 -0
  13. package/.context/todos.md +0 -0
  14. package/.github/workflows/release.yml +36 -0
  15. package/.oxlintrc.json +8 -0
  16. package/README.md +1 -0
  17. package/bun.lock +705 -0
  18. package/examples/svelte/bun.lock +261 -0
  19. package/examples/svelte/package.json +21 -0
  20. package/examples/svelte/src/app.html +11 -0
  21. package/examples/svelte/src/lib/db.ts +44 -0
  22. package/examples/svelte/src/routes/+page.svelte +322 -0
  23. package/examples/svelte/svelte.config.js +16 -0
  24. package/examples/svelte/tsconfig.json +6 -0
  25. package/examples/svelte/vite.config.ts +6 -0
  26. package/examples/vanilla/app.ts +219 -0
  27. package/examples/vanilla/index.html +144 -0
  28. package/examples/vanilla/serve.ts +42 -0
  29. package/package.json +46 -0
  30. package/prds/localstr-v0.2.md +221 -0
  31. package/prek.toml +10 -0
  32. package/scripts/validate.ts +392 -0
  33. package/src/crud/collection-handle.ts +189 -0
  34. package/src/crud/query-builder.ts +414 -0
  35. package/src/crud/watch.ts +78 -0
  36. package/src/db/create-localstr.ts +217 -0
  37. package/src/db/database-handle.ts +16 -0
  38. package/src/db/identity.ts +49 -0
  39. package/src/errors.ts +37 -0
  40. package/src/index.ts +32 -0
  41. package/src/main.ts +10 -0
  42. package/src/schema/collection.ts +53 -0
  43. package/src/schema/field.ts +25 -0
  44. package/src/schema/types.ts +19 -0
  45. package/src/schema/validate.ts +111 -0
  46. package/src/storage/events-store.ts +24 -0
  47. package/src/storage/giftwraps-store.ts +23 -0
  48. package/src/storage/idb.ts +244 -0
  49. package/src/storage/lww.ts +17 -0
  50. package/src/storage/records-store.ts +76 -0
  51. package/src/svelte/collection.svelte.ts +87 -0
  52. package/src/svelte/database.svelte.ts +83 -0
  53. package/src/svelte/index.svelte.ts +52 -0
  54. package/src/svelte/live-query.svelte.ts +29 -0
  55. package/src/svelte/query.svelte.ts +101 -0
  56. package/src/sync/gift-wrap.ts +33 -0
  57. package/src/sync/negentropy.ts +83 -0
  58. package/src/sync/publish-queue.ts +61 -0
  59. package/src/sync/relay.ts +239 -0
  60. package/src/sync/sync-service.ts +183 -0
  61. package/src/sync/sync-status.ts +17 -0
  62. package/src/utils/uuid.ts +22 -0
  63. package/src/vendor/negentropy.js +616 -0
  64. package/tests/db/create-localstr.test.ts +174 -0
  65. package/tests/db/identity.test.ts +33 -0
  66. package/tests/main.test.ts +9 -0
  67. package/tests/schema/collection.test.ts +27 -0
  68. package/tests/schema/field.test.ts +41 -0
  69. package/tests/schema/validate.test.ts +85 -0
  70. package/tests/setup.ts +1 -0
  71. package/tests/storage/idb.test.ts +144 -0
  72. package/tests/storage/lww.test.ts +33 -0
  73. package/tests/sync/gift-wrap.test.ts +56 -0
  74. package/tsconfig.json +18 -0
  75. package/vitest.config.ts +8 -0
@@ -0,0 +1,498 @@
1
+ # localstr — Product Requirements Document
2
+
3
+ | | |
4
+ |---|---|
5
+ | **Version** | 0.1 — Draft |
6
+ | **Author** | Kevin Åberg Kultalahti |
7
+ | **Package** | `localstr` |
8
+ | **Status** | Pre-implementation |
9
+
10
+ ---
11
+
12
+ ## 1. Purpose
13
+
14
+ localstr is a browser-first local-first sync library. It gives developers a simple, typed API for storing and querying structured data that syncs automatically across devices and users via Nostr relays. The Nostr protocol is an implementation detail. Developers who use localstr never interact with keypairs, relay connections, event kinds, or NIPs. They define a schema, write data, query it, and optionally share it. Everything else happens underneath.
15
+
16
+ The design goal is a library that a developer can integrate into an application in under thirty minutes without reading documentation about distributed systems, cryptography, or peer-to-peer networking.
17
+
18
+ ---
19
+
20
+ ## 2. Problem Statement
21
+
22
+ Local-first applications — where the local device is the primary source of truth and sync happens in the background — are hard to build today. The options available to developers each carry significant costs.
23
+
24
+ CRDTs solve the merge problem correctly but require deep understanding of distributed systems to implement, and the leading libraries expose that complexity directly to the developer. PouchDB and CouchDB require developers to operate their own infrastructure. Firestore and similar cloud databases give you sync for free but lock your data to a vendor and require a server round-trip for every read. Building on raw WebSockets means reinventing sync, conflict resolution, and offline queuing from scratch.
25
+
26
+ The result is that most applications either skip local-first entirely and accept the UX tradeoffs of cloud-only data, or they invest months building sync infrastructure that should be a solved problem.
27
+
28
+ localstr solves this by using the Nostr protocol as a sync layer. Nostr's signed event model maps naturally onto local-first architecture: each write produces a signed, encrypted, self-contained event that can be stored locally, published to multiple relays, and replicated to other devices without conflict. The local IndexedDB store is a materialized view of the event log. The relays are the backup and transport layer. Nothing is stored on a server that the developer controls.
29
+
30
+ ---
31
+
32
+ ## 3. Goals
33
+
34
+ ### 3.1 In Scope
35
+
36
+ - A typed schema definition system where the developer defines collections and field types once, and TypeScript types are derived automatically.
37
+ - A Dexie-inspired query builder API covering equality, range, sort, limit, and count operations.
38
+ - Reactive subscriptions that push updates to callbacks whenever matching data changes.
39
+ - Transparent encryption of all data using NIP-44 before it leaves the device.
40
+ - Identity derived entirely from a password using Argon2id key derivation. The developer and user never handle a private key.
41
+ - Multi-device sync via Nostr relays. Writes are published to relays and fetched on demand.
42
+ - A collaboration model where a collection can be shared via an invite URL, allowing another user to join and receive the same encrypted data.
43
+ - A simple key-value API alongside the collection API for unstructured or singleton data.
44
+ - Browser-only target for v1. No Node.js or Bun support in the initial release.
45
+
46
+ ### 3.2 Out of Scope (v1)
47
+
48
+ - Server-side or Node.js storage adapters (planned for v2 using LMDB).
49
+ - JOIN operations across collections.
50
+ - Full-text search.
51
+ - Transactions spanning multiple writes.
52
+ - Key rotation or member revocation from shared collections.
53
+ - Conflict resolution beyond last-write-wins on replaceable events.
54
+ - Any Nostr social features: profiles, follows, feeds, reactions.
55
+
56
+ ---
57
+
58
+ ## 4. Architecture
59
+
60
+ ### 4.1 Data Flow
61
+
62
+ Every write follows the same path: the developer calls an API method, the library validates the data against the schema, encrypts the content using NIP-44, constructs a Nostr event, writes it to local IndexedDB, and publishes it to configured relays asynchronously. Reads query IndexedDB directly. The relays are never in the read path for local queries.
63
+
64
+ On startup, or when sync is triggered manually, the library fetches events from relays that are newer than the latest locally stored event. These are decrypted, validated, and written into IndexedDB, updating the materialized KV and collection stores. This means the local store is always usable even when offline.
65
+
66
+ ### 4.2 Storage Model
67
+
68
+ IndexedDB holds two logical stores, accessed via the `idb` library:
69
+
70
+ **events** — the canonical append-only log of all Nostr events. Each row stores the raw event fields: `id`, `kind`, `pubkey`, `created_at`, `content` (encrypted), `tags` (JSON array), and `sig`. This store is the source of truth. If it is intact, all other state can be reconstructed from it.
71
+
72
+ **kv** — a materialized view derived from the events store. Each row holds a `key`, the decrypted JSON `value`, `updated_at`, and a reference to the source `event_id`. This store exists for query performance. It is always rebuildable.
73
+
74
+ Indexes on the events store cover: `kind`, `pubkey`, `created_at`, and the `d` tag (the primary identifier for replaceable events). Indexes on the kv store cover the collection prefix on the key and any scalar fields declared in the schema.
75
+
76
+ ### 4.3 Event Mapping
77
+
78
+ Schema collections map to Nostr kind 30078 replaceable events. The `d` tag is set to a namespaced key combining the collection name and the record identifier. The content field holds the NIP-44 encrypted JSON payload of the record. When a record is updated, a new event with the same `d` tag replaces the previous one both locally and on relays.
79
+
80
+ Append-only log entries (for collections marked as append-only in the schema) map to regular non-replaceable events, preserving full history.
81
+
82
+ The key-value API uses the same kind 30078 mapping with a flat key as the `d` tag.
83
+
84
+ ### 4.4 Encryption
85
+
86
+ All content is encrypted with NIP-44 before leaving the local device. The encryption key is the Nostr private key derived from the user's password via Argon2id. For personal data, the content is encrypted to the user's own public key. For shared collections, content is encrypted to a shared symmetric key that is distributed to members via NIP-59 gift wrap events.
87
+
88
+ NIP-59 gift wrap adds three layers of indirection: the plaintext payload (rumor), a sealed inner event signed by the real keypair, and an outer gift wrap event signed by a single-use ephemeral keypair. This means relay operators cannot determine who sent data to whom, and cannot correlate events to identities.
89
+
90
+ ### 4.5 Identity
91
+
92
+ Identity is derived entirely from the user's password. The library passes the password through Argon2id with an application-specific salt to produce 64 bytes. The first 32 bytes become the encryption key. The second 32 bytes become the Nostr private key, from which the public key is derived deterministically.
93
+
94
+ The developer never stores, serializes, or transmits a private key. Recovery is "back up your password". This is the entire key management story in v1.
95
+
96
+ An escape hatch `signer` option in `createLocalstr` allows advanced users to supply their own NIP-07 compatible signer if they want to bring an existing Nostr identity. This is optional and not part of the primary UX.
97
+
98
+ ### 4.6 Relay Strategy
99
+
100
+ The library maintains a pool of relay connections using nostr-tools `SimplePool`. Writes are published to all configured relays. Reads fetch from all relays and deduplicate by event id. The relay pool is managed internally. The developer configures relay URLs at initialization and does not interact with the pool directly.
101
+
102
+ Sync is manual by default. The developer calls `db.sync()` to trigger a fetch of new events from relays. An `autoSync` option can be set at initialization to enable periodic background sync. The sync status is observable via `db.syncStatus()` and `db.onSync(callback)`.
103
+
104
+ ---
105
+
106
+ ## 5. Schema Definition
107
+
108
+ ### 5.1 Overview
109
+
110
+ The schema is the single source of truth for the library. It drives TypeScript type inference, IndexedDB index creation, runtime validation, and query planning. It is defined once at initialization using builder functions exported from the library.
111
+
112
+ ### 5.2 Collection Builder
113
+
114
+ A collection is a named group of records sharing the same field structure. It is declared with the `collection()` function, which takes an object mapping field names to field descriptors.
115
+
116
+ ```typescript
117
+ import { collection, field } from 'localstr'
118
+
119
+ const exercises = collection({
120
+ name: field.string(),
121
+ sets: field.number(),
122
+ reps: field.number(),
123
+ weight: field.number().optional(),
124
+ loggedAt: field.number(),
125
+ })
126
+ ```
127
+
128
+ ### 5.3 Field Types
129
+
130
+ | Field Declaration | Description |
131
+ |---|---|
132
+ | `field.string()` | A required UTF-8 string. Gets an IndexedDB index. |
133
+ | `field.number()` | A required number (integer or float). Gets an IndexedDB index. |
134
+ | `field.boolean()` | A required boolean. Gets an IndexedDB index. |
135
+ | `field.string().array()` | An array of strings. Gets a multiEntry IndexedDB index. |
136
+ | `field.number().array()` | An array of numbers. Gets a multiEntry IndexedDB index. |
137
+ | `field.json()` | An opaque JSON value. Stored as-is, not indexed, not queryable. |
138
+ | `field.*.optional()` | Any field type marked optional. Indexed but sparse. TypeScript type becomes `T \| undefined`. |
139
+
140
+ ### 5.4 Type Inference
141
+
142
+ The TypeScript type for a record in a collection is fully inferred from the schema. The developer does not write a separate interface. Required fields are non-optional in the inferred type. Optional fields carry `| undefined`. Array fields produce `T[]`. The `json()` field produces `unknown`.
143
+
144
+ The inferred types flow through the entire API surface: `add()`, `update()`, `where()`, and `watch()` all receive and return the correct types without any additional annotation from the developer.
145
+
146
+ ### 5.5 Full Schema Example
147
+
148
+ ```typescript
149
+ const schema = {
150
+ exercises: collection({
151
+ name: field.string(),
152
+ sets: field.number(),
153
+ reps: field.number(),
154
+ weight: field.number().optional(),
155
+ loggedAt: field.number(),
156
+ }),
157
+ bodyweight: collection({
158
+ kg: field.number(),
159
+ recordedAt: field.number(),
160
+ }),
161
+ }
162
+
163
+ const db = await createLocalstr({ password, relays, schema })
164
+ ```
165
+
166
+ ---
167
+
168
+ ## 6. API Reference
169
+
170
+ ### 6.1 Initialization
171
+
172
+ ```typescript
173
+ const db = await createLocalstr({
174
+ password: string,
175
+ relays: string[],
176
+ schema: Schema,
177
+ autoSync?: boolean, // default false
178
+ syncInterval?: number, // milliseconds, default off
179
+ signer?: NostrSigner,
180
+ })
181
+ ```
182
+
183
+ `createLocalstr` is async. It derives the keypair from the password, opens the IndexedDB database, creates or migrates stores based on the schema, and establishes relay connections. It resolves with the db handle. It rejects if the password is empty, if no relays are provided, or if IndexedDB is unavailable.
184
+
185
+ ### 6.2 Collection API
186
+
187
+ #### add()
188
+
189
+ ```typescript
190
+ const id = await db.collection('exercises').add({
191
+ name: 'Squat',
192
+ sets: 5,
193
+ reps: 5,
194
+ weight: 100,
195
+ loggedAt: Date.now(),
196
+ })
197
+ // returns: string (the generated record id)
198
+ ```
199
+
200
+ Validates the input against the schema at runtime. Generates a unique id. Writes to IndexedDB immediately. Publishes to relays asynchronously. Returns the record id.
201
+
202
+ #### update()
203
+
204
+ ```typescript
205
+ await db.collection('exercises').update(id, {
206
+ weight: 105,
207
+ })
208
+ ```
209
+
210
+ Accepts a partial record. Merges with the existing record. Validates the merged result. Writes a new replaceable event to IndexedDB and relays. Throws if the id does not exist.
211
+
212
+ #### delete()
213
+
214
+ ```typescript
215
+ await db.collection('exercises').delete(id)
216
+ ```
217
+
218
+ Marks the record as deleted locally. Publishes a deletion event to relays. The record is excluded from all subsequent queries.
219
+
220
+ #### Query Builder
221
+
222
+ Queries are constructed by chaining methods on the collection handle. The query is not executed until `get()`, `first()`, `count()`, or `watch()` is called.
223
+
224
+ ```typescript
225
+ const results = await db.collection('exercises')
226
+ .where('loggedAt').above(startOfDay)
227
+ .sortBy('loggedAt', 'asc')
228
+ .limit(20)
229
+ .get()
230
+ ```
231
+
232
+ | Method | Description |
233
+ |---|---|
234
+ | `.where(field)` | Begins a filter clause. Must be followed by a comparator. |
235
+ | `.equals(value)` | Exact match on the current field. |
236
+ | `.above(value)` | Greater than. Numbers and timestamps only. |
237
+ | `.below(value)` | Less than. Numbers and timestamps only. |
238
+ | `.between(lower, upper)` | Inclusive range. Numbers and timestamps only. |
239
+ | `.anyOf(values[])` | Matches any value in the provided array. |
240
+ | `.sortBy(field, 'asc'\|'desc')` | Orders results. Defaults to ascending. |
241
+ | `.limit(n)` | Maximum number of results to return. |
242
+ | `.offset(n)` | Skip the first n results. Used for pagination. |
243
+ | `.get()` | Executes the query. Returns `Promise<T[]>`. |
244
+ | `.first()` | Executes the query. Returns `Promise<T \| undefined>`. |
245
+ | `.count()` | Executes the query. Returns `Promise<number>`. Does not fetch record content. |
246
+ | `.each(cb)` | Executes the query and calls cb for each result. Returns `Promise<void>`. |
247
+ | `.watch(cb)` | Registers a live subscription. Calls cb immediately with current results, then again on any change. Returns an unsubscribe function. |
248
+
249
+ #### Query Planning
250
+
251
+ When a query is executed, the library splits it into two plans: a relay filter plan and a local filter plan. Fields that map directly to Nostr filter primitives (tag filters, `since`, `until`, `limit`) are included in the relay REQ filter. All other predicates are applied locally after fetching from IndexedDB. The developer sees neither plan. The split is transparent.
252
+
253
+ Fields indexed in IndexedDB use index-based cursors for local filtering rather than full table scans. This keeps local queries fast regardless of event log size.
254
+
255
+ ### 6.3 Key-Value API
256
+
257
+ | Method | Description |
258
+ |---|---|
259
+ | `db.set(key, value)` | Stores any JSON-serializable value at key. Returns `Promise<void>`. |
260
+ | `db.get(key)` | Returns `Promise<unknown \| undefined>`. Returns undefined if key does not exist. |
261
+ | `db.delete(key)` | Removes the key. Returns `Promise<void>`. |
262
+ | `db.has(key)` | Returns `Promise<boolean>`. |
263
+ | `db.subscribe(key, cb)` | Calls cb with the current value, then on every change. Returns unsubscribe function. |
264
+
265
+ ### 6.4 Sync API
266
+
267
+ | Method | Description |
268
+ |---|---|
269
+ | `db.sync()` | Fetches new events from all relays for all collections. Returns `Promise<void>` that resolves when the fetch is complete. |
270
+ | `db.sync(collectionName)` | Fetches new events for a specific collection only. |
271
+ | `db.syncStatus()` | Returns `'idle' \| 'syncing' \| 'error'`. |
272
+ | `db.onSync(cb)` | Registers a callback invoked on every sync cycle with status and event count. Returns unsubscribe function. |
273
+
274
+ ### 6.5 Collaboration API
275
+
276
+ #### share()
277
+
278
+ ```typescript
279
+ const url = await db.share('exercises')
280
+ // returns: string — an invite URL with encrypted key material in the fragment
281
+
282
+ const url = await db.share('exercises', { password: 'optional-extra-password' })
283
+ // password-protected invite — recipient must supply password to join
284
+ ```
285
+
286
+ Generates an invite URL for the specified collection. The URL fragment contains an ephemeral keypair and the collection's shared encryption key, wrapped in a gift wrap event. The fragment is never sent to a server. The `password` option runs the fragment key material through Argon2id before encoding, requiring the recipient to know the password to decrypt.
287
+
288
+ #### join()
289
+
290
+ ```typescript
291
+ await db.join(inviteUrl)
292
+ await db.join(inviteUrl, { password: 'the-password' })
293
+ ```
294
+
295
+ Parses the invite URL, decrypts the key material from the fragment, derives the shared collection keypair, and begins syncing the collection from relays. After `join()` resolves, the collection is available via the standard query API.
296
+
297
+ #### members()
298
+
299
+ ```typescript
300
+ const members = await db.members('exercises')
301
+ // returns: Array<{ pubkey: string, joinedAt: number }>
302
+ ```
303
+
304
+ ### 6.6 Lifecycle API
305
+
306
+ | Method | Description |
307
+ |---|---|
308
+ | `db.close()` | Closes relay connections and IndexedDB handle. Returns `Promise<void>`. |
309
+ | `db.destroy()` | Closes connections and deletes all local data. Returns `Promise<void>`. |
310
+ | `db.export()` | Returns a JSON-serializable snapshot of all local events. Useful for backup. |
311
+ | `db.import(data)` | Replays a previously exported snapshot into local storage. Merges with existing data. |
312
+
313
+ ---
314
+
315
+ ## 7. Internal Services
316
+
317
+ The library is structured as a set of internal services wired together at initialization. Each service has a single responsibility. They communicate through typed interfaces rather than direct imports. This structure makes the codebase testable in isolation and makes it straightforward to swap implementations in future versions.
318
+
319
+ ### 7.1 IdentityService
320
+
321
+ Responsible for all keypair operations. Takes the password and derives a 64-byte seed using Argon2id with a fixed application-level salt concatenated with an optional user-supplied salt. The first 32 bytes become the private key. The public key is derived from the private key using secp256k1. Exposes `getPrivateKey()`, `getPublicKey()`, and `sign(event)` methods. Holds the keypair in memory only. Never serializes private key material to any storage.
322
+
323
+ ### 7.2 CryptoService
324
+
325
+ Responsible for encrypting and decrypting event content. Uses NIP-44 for all encryption. Exposes `encrypt(plaintext, recipientPubkey)` and `decrypt(ciphertext, senderPubkey)` methods. Also exposes `giftWrap(event, recipientPubkey)` and `unwrapGift(wrappedEvent)` for the collaboration invite flow. Depends on IdentityService for the local keypair.
326
+
327
+ ### 7.3 LocalStore
328
+
329
+ Responsible for all IndexedDB operations. Wraps `idb`. Exposes typed methods for writing events, querying the events store by filter, reading and writing the kv store, managing indexes, and handling schema migrations. Does not know about Nostr event semantics. Treats events as typed records. Exposes a change notification mechanism that QueryService uses to power reactive subscriptions.
330
+
331
+ ### 7.4 RelayService
332
+
333
+ Responsible for all relay communication. Wraps nostr-tools `SimplePool`. Exposes `publish(event)`, `fetch(filters)`, and `subscribe(filters, onEvent)` methods. Manages connection state and reconnection. Reports sync status changes. Does not know about encryption or schema. Receives and sends raw Nostr events.
334
+
335
+ ### 7.5 EventBuilder
336
+
337
+ Responsible for constructing valid Nostr events from application-level write operations. Takes a collection name, record id, and plaintext payload. Determines the correct event kind (30078 for replaceable, standard kind for append-only). Sets the `d` tag. Calls CryptoService to encrypt the content. Calls IdentityService to sign the event. Returns a complete signed event ready for storage and publication.
338
+
339
+ ### 7.6 QueryPlanner
340
+
341
+ Responsible for splitting a query builder expression into a relay filter plan and a local IndexedDB filter plan. Inspects the schema to determine which fields are indexed. Produces a `NostrFilter` object for RelayService and an IndexedDB cursor strategy for LocalStore. Ensures that the two plans are semantically equivalent: the intersection of what the relay returns and what the local filter matches is the correct result set.
342
+
343
+ ### 7.7 Validator
344
+
345
+ Responsible for runtime validation of write payloads against the schema. Checks that required fields are present and have the correct types. Checks that optional fields, if present, have the correct types. Returns typed errors rather than throwing. Called by the collection `add()` and `update()` implementations before any write is committed.
346
+
347
+ ### 7.8 SchemaManager
348
+
349
+ Responsible for translating the schema definition into IndexedDB store configurations and managing migrations. On initialization, compares the current schema version against the stored version in IndexedDB. If the schema has changed, creates new indexes, drops removed indexes, and increments the store version. The schema version is a hash of the schema definition, not a manually incremented integer.
350
+
351
+ ---
352
+
353
+ ## 8. Dependencies
354
+
355
+ The library has a minimal, carefully chosen dependency set. All dependencies must be tree-shakeable. The total bundle size target for a minimal integration (single collection, no collaboration) is under 50KB gzipped.
356
+
357
+ | Package | Approx. Size | Purpose |
358
+ |---|---|---|
359
+ | `nostr-tools` | ~30KB tree-shaken | Event construction, signing, relay pool, NIP-44 encryption, NIP-59 gift wrap |
360
+ | `idb` | ~1KB | Promise-based IndexedDB wrapper. Handles the async complexity and transaction management of raw IndexedDB. |
361
+ | `@noble/hashes` | ~10KB | Argon2id implementation for key derivation. Also provides SHA-256 used in event id generation. |
362
+
363
+ Effect is used internally for service composition, typed error handling, and dependency injection. It does not appear in the public API and is not a peer dependency. Consumers of the library do not need to know about or install Effect.
364
+
365
+ ---
366
+
367
+ ## 9. Error Handling
368
+
369
+ All errors are typed and surfaced through the return types of async methods, not through untyped throws. Each service defines its own error variants. The public API methods surface a union of the errors that can occur in their call path.
370
+
371
+ The developer can handle errors granularly or use a catch-all. Relay errors do not cause local operations to fail. A write that publishes successfully to IndexedDB but fails to reach any relay is considered a local success and a sync warning, not a write failure.
372
+
373
+ | Error | Meaning |
374
+ |---|---|
375
+ | `ValidationError` | A write payload failed schema validation. Contains the field name and expected type. |
376
+ | `StorageError` | IndexedDB operation failed. Includes `QuotaExceededError` wrapping. |
377
+ | `CryptoError` | Encryption or decryption failed. Likely a key mismatch on shared collections. |
378
+ | `RelayError` | A relay connection failed or rejected an event. Non-fatal for writes. |
379
+ | `SyncError` | A sync cycle failed to reach any relay. Local data is unaffected. |
380
+ | `InviteError` | An invite URL could not be parsed or decrypted. Likely wrong password or malformed URL. |
381
+ | `NotFoundError` | An `update()` or `delete()` was called with an id that does not exist locally. |
382
+
383
+ ---
384
+
385
+ ## 10. Schema Migration
386
+
387
+ When a developer adds, removes, or changes fields in the schema, the library detects the change on the next initialization. The SchemaManager computes a hash of the schema definition and compares it to the hash stored in IndexedDB metadata.
388
+
389
+ If they differ, the following migration rules apply automatically:
390
+
391
+ - **New required field added:** existing records will have the field set to `undefined` at runtime. TypeScript will not allow this because the field is required, so developers are expected to backfill via a one-time update loop. The library does not block startup waiting for backfill.
392
+ - **New optional field added:** existing records return `undefined` for the new field. No action required.
393
+ - **Field removed:** the field is dropped from new writes. Existing events in the events store retain the old field in their encrypted payload but it is not surfaced in query results.
394
+ - **Field type changed:** treated as a remove and add. Old values are not migrated. The developer is responsible for data coercion if needed.
395
+
396
+ The events store is the source of truth and is never modified by a migration. Migrations only affect the kv materialized view and the IndexedDB indexes. If migration fails, the library falls back to scanning from the events store to rebuild the kv store.
397
+
398
+ ---
399
+
400
+ ## 11. Offline Behaviour
401
+
402
+ All read and write operations work offline. Reads query IndexedDB directly and never require a relay connection. Writes commit to IndexedDB immediately and queue relay publication for when connectivity is restored.
403
+
404
+ The library does not implement its own connectivity detection. It relies on the browser's `navigator.onLine` and the `online`/`offline` events. When the browser comes back online, any queued relay publications are flushed.
405
+
406
+ There is no explicit conflict resolution mechanism in v1. The last-write-wins semantics of Nostr replaceable events (kind 30078) handle concurrent edits: the event with the highest `created_at` timestamp wins. For append-only collections, all events are preserved.
407
+
408
+ ---
409
+
410
+ ## 12. Security Model
411
+
412
+ All data is encrypted at rest in IndexedDB and in transit to relays. Relay operators see only encrypted ciphertext, event timestamps, and public keys. They cannot read the content of any event.
413
+
414
+ Public keys derived from user passwords are pseudonymous. Two different applications using the same password with different application-level salts produce different keypairs, preventing cross-application identity correlation.
415
+
416
+ The invite URL fragment is never sent to a server. URL fragments are not included in HTTP requests and are not logged by servers or CDNs. Password-protected invites require the password to be communicated out-of-band, ensuring that even if the URL is intercepted, the encrypted key material cannot be decrypted.
417
+
418
+ The library does not implement server-side verification, authentication, or authorization. Access control is purely key-based: you can read a shared collection if and only if you have the shared encryption key.
419
+
420
+ ---
421
+
422
+ ## 13. Testing Strategy
423
+
424
+ ### 13.1 Unit Tests
425
+
426
+ Each internal service is tested in isolation with mocked dependencies. The Validator, QueryPlanner, SchemaManager, and EventBuilder have no I/O dependencies and are pure functions suitable for direct unit testing. The CryptoService and IdentityService tests verify correctness of key derivation and encryption round-trips.
427
+
428
+ ### 13.2 Integration Tests
429
+
430
+ The LocalStore service is tested against a real IndexedDB instance using a browser test runner (Vitest with happy-dom or a real browser via Playwright). Tests cover schema migration, index correctness, and concurrent write behaviour. The RelayService is tested against a local relay process spun up for tests.
431
+
432
+ ### 13.3 End-to-End Tests
433
+
434
+ A small set of Playwright tests cover the full developer-facing API: initialization, write, query, sync, and the share/join collaboration flow across two simulated browser contexts with a local relay.
435
+
436
+ ---
437
+
438
+ ## 14. Package Structure
439
+
440
+ ```
441
+ localstr/
442
+ src/
443
+ index.ts — Public API exports
444
+ schema/
445
+ collection.ts — collection() builder
446
+ field.ts — field.* builders and type inference
447
+ types.ts — Inferred type utilities
448
+ services/
449
+ identity.ts — IdentityService
450
+ crypto.ts — CryptoService
451
+ local-store.ts — LocalStore (idb wrapper)
452
+ relay.ts — RelayService (nostr-tools wrapper)
453
+ event-builder.ts — EventBuilder
454
+ query-planner.ts — QueryPlanner
455
+ validator.ts — Validator
456
+ schema-manager.ts — SchemaManager
457
+ api/
458
+ collection.ts — Collection query builder implementation
459
+ kv.ts — Key-value API implementation
460
+ sync.ts — Sync API implementation
461
+ share.ts — Collaboration API implementation
462
+ errors.ts — All typed error definitions
463
+ localstr.ts — createLocalstr() implementation
464
+ test/
465
+ unit/
466
+ integration/
467
+ e2e/
468
+ package.json
469
+ tsconfig.json
470
+ vite.config.ts
471
+ ```
472
+
473
+ ---
474
+
475
+ ## 15. Explicit Non-Goals
476
+
477
+ This section records deliberate decisions not to do certain things, to prevent them from being relitigated during implementation.
478
+
479
+ - localstr does not expose Nostr concepts to the developer. No event kinds, no NIPs, no relay filters, no keypair management surfaces in the public API.
480
+ - localstr does not use SQLite WASM. The query surface is constrained by what Nostr relay filters can express, so the additional power of SQL is not useful. The cold start cost and WASM complexity is not justified.
481
+ - localstr does not use OPFS directly. The performance benefit of raw OPFS requires building a custom storage engine. `idb` on IndexedDB is sufficient for the event log sizes typical in local-first applications.
482
+ - localstr does not use Dexie. The query API is Dexie-inspired but built from scratch on `idb`, keeping the dependency surface minimal and avoiding the mismatch between Dexie's schema model and localstr's schema model.
483
+ - localstr does not use NDK. NDK is a social Nostr client library with wallet integration, web-of-trust, and NIP-90 support. It is not appropriate as a foundation for a general-purpose sync library.
484
+ - localstr does not implement CRDT merge semantics. Last-write-wins is the conflict resolution strategy. For the target use cases (personal data, collaborative documents with low write contention), this is sufficient.
485
+
486
+ ---
487
+
488
+ ## 16. Open Questions
489
+
490
+ | Question | Notes |
491
+ |---|---|
492
+ | Final package name | Settled: `localstr` |
493
+ | npm scope | Publish as standalone `localstr` package. |
494
+ | `autoSync` default | Currently defaults to `false` (manual sync). Should it default to `true` for better out-of-box experience? |
495
+ | Append-only collection syntax | How does the developer declare a collection as append-only in the schema? Flag on `collection()`? Separate `appendCollection()` builder? |
496
+ | Record id generation | Library-generated UUIDs, or developer-supplied ids? Or both? |
497
+ | Relay failure threshold | How many relay failures before `syncStatus` returns `'error'` vs `'degraded'`? |
498
+ | Private browsing | IndexedDB is available in private browsing but may have reduced quota. Should the library warn or degrade gracefully? |
File without changes
@@ -0,0 +1,48 @@
1
+ # Add Changesets to douala-v4
2
+
3
+ ## Context
4
+ The library (`douala-v4`, version `0.0.1`) needs release management tooling. Changesets will handle versioning, changelog generation, and npm publish workflows. The project uses Bun as its package manager and is a single package (not a monorepo).
5
+
6
+ ## Steps
7
+
8
+ ### 1. Install `@changesets/cli`
9
+ ```bash
10
+ bun add -D @changesets/cli
11
+ ```
12
+
13
+ ### 2. Initialize changesets
14
+ ```bash
15
+ bunx changeset init
16
+ ```
17
+ This creates the `.changeset/` directory with a `config.json` and a `README.md`.
18
+
19
+ ### 3. Update `.changeset/config.json`
20
+ Set appropriate defaults:
21
+ - `"access": "public"` (assuming this will be a public npm package)
22
+ - `"baseBranch": "main"`
23
+ - `"commit": false` (don't auto-commit version bumps)
24
+
25
+ ### 4. Add scripts to `package.json`
26
+ Add convenience scripts:
27
+ ```json
28
+ "changeset": "changeset",
29
+ "version": "changeset version",
30
+ "release": "changeset publish"
31
+ ```
32
+
33
+ ### 5. Add GitHub Actions release workflow
34
+ Create `.github/workflows/release.yml` that:
35
+ - Triggers on push to `main`
36
+ - Runs `changeset version` to bump versions and update CHANGELOG
37
+ - Opens a "Version Packages" PR via `changesets/action`
38
+ - When that PR merges, publishes to npm via `changeset publish`
39
+ - Uses `NPM_TOKEN` secret for publishing
40
+
41
+ ## Files to modify
42
+ - `package.json` — add devDependency + scripts
43
+ - `.changeset/config.json` — created by init, set `access: "public"`, `baseBranch: "main"`
44
+ - `.github/workflows/release.yml` — new file for CI release automation
45
+
46
+ ## Verification
47
+ - Run `bunx changeset` to confirm it prompts for a new changeset
48
+ - Run `bunx changeset status` to confirm it reports no changesets
@@ -0,0 +1,115 @@
1
+ # Dexie.js-style Query Language for localstr
2
+
3
+ ## Context
4
+
5
+ localstr currently only supports `where(field).equals(value)` with no sorting, range queries, or pagination. We want a rich, chainable query API inspired by Dexie.js so users can filter, sort, and paginate data expressively.
6
+
7
+ **Key decision: in-memory execution.** All collections share one IDB object store with nested `data.*` fields, so native IDB indices would span all collections and require complex cursor logic for marginal gain on typical local-first dataset sizes. We'll build the query API as in-memory operations on the `getAllRecords()` result set. The `indices` schema option is stored for future optimization but not enforced at the type level yet.
8
+
9
+ ## Target API
10
+
11
+ ```typescript
12
+ // Schema with optional index hints
13
+ const todos =
14
+ yield *
15
+ collection(
16
+ "todos",
17
+ {
18
+ title: field.string(),
19
+ done: field.boolean(),
20
+ priority: field.number(),
21
+ },
22
+ { indices: ["done", "priority"] },
23
+ );
24
+
25
+ // Range queries
26
+ yield * (yield * col.where("priority")).above(3).toArray();
27
+ yield * (yield * col.where("priority")).between(1, 5).toArray();
28
+ yield * (yield * col.where("title")).startsWith("Buy").toArray();
29
+ yield * (yield * col.where("done")).anyOf([true, false]).toArray();
30
+
31
+ // Sorting
32
+ yield * col.orderBy("priority").toArray();
33
+ yield * col.orderBy("priority").reverse().toArray();
34
+
35
+ // Pagination
36
+ yield * col.orderBy("priority").offset(10).limit(5).toArray();
37
+
38
+ // Chained filter + sort
39
+ yield * (yield * col.where("done")).equals(false).sortBy("priority").toArray();
40
+ ```
41
+
42
+ ## Files to Modify
43
+
44
+ ### 1. `src/schema/collection.ts` — Add indices option
45
+
46
+ - Add optional third parameter `options?: { indices?: ReadonlyArray<string & keyof F> }`
47
+ - Store `indices` on `CollectionDef` (default `[]`)
48
+ - Validate each index key exists in `fields` and is not `json`/`array` type
49
+ - Fully backwards compatible — existing calls without options still work
50
+
51
+ ### 2. `src/schema/types.ts` — Add IndexedFields type
52
+
53
+ - Add `IndexedFields<C>` utility type extracting index field names from a `CollectionDef`
54
+ - Not used for enforcement yet, but available for future type narrowing
55
+
56
+ ### 3. `src/crud/query-builder.ts` — Rewrite with rich query builder
57
+
58
+ Replace the current simple WhereClause/QueryExecutor with three interfaces:
59
+
60
+ **`WhereClause<T>`** — returned by `col.where(field)`:
61
+
62
+ - `equals(value)` → `QueryBuilder<T>`
63
+ - `above(value)` / `aboveOrEqual(value)` → `QueryBuilder<T>`
64
+ - `below(value)` / `belowOrEqual(value)` → `QueryBuilder<T>`
65
+ - `between(lower, upper)` → `QueryBuilder<T>`
66
+ - `startsWith(prefix)` → `QueryBuilder<T>`
67
+ - `anyOf(values)` / `noneOf(values)` → `QueryBuilder<T>`
68
+
69
+ **`QueryBuilder<T>`** — chainable intermediate:
70
+
71
+ - `and(fn)` → `QueryBuilder<T>` (JS predicate filter)
72
+ - `sortBy(field)` → `QueryBuilder<T>`
73
+ - `reverse()` → `QueryBuilder<T>`
74
+ - `offset(n)` / `limit(n)` → `QueryBuilder<T>`
75
+ - Terminal: `toArray()`, `first()`, `count()`, `watch()`
76
+ - Alias: `get()` → `toArray()` (backwards compat)
77
+
78
+ **`OrderByBuilder<T>`** — returned by `col.orderBy(field)`:
79
+
80
+ - `reverse()` → `OrderByBuilder<T>`
81
+ - `offset(n)` / `limit(n)` → `OrderByBuilder<T>`
82
+ - Terminal: `toArray()`, `first()`, `count()`, `watch()`
83
+
84
+ Internal `executeQuery()` function applies the pipeline: filter deleted → apply predicates → sort → offset → limit → map.
85
+
86
+ For `watch()`, build a stream that re-executes the full query pipeline on each PubSub change event (reuses `WatchContext` pattern from `watch.ts` but doesn't require modifying it).
87
+
88
+ ### 4. `src/crud/collection-handle.ts` — Add orderBy method
89
+
90
+ - Add `orderBy(field)` to `CollectionHandle` interface returning `OrderByBuilder<InferRecord<C>>`
91
+ - `orderBy` is synchronous (returns builder directly, not wrapped in Effect) — validates field at build time
92
+ - Update `where()` to use the new `createWhereClause` that returns the richer `WhereClause<T>`
93
+
94
+ ### 5. `src/index.ts` — Update exports
95
+
96
+ - Export `QueryBuilder`, `OrderByBuilder` types
97
+ - Keep `QueryExecutor` as deprecated type alias for backwards compat
98
+
99
+ ### 6. `demo/app.ts` — Update demo
100
+
101
+ - Add examples using `orderBy`, `above`, `between`, `sortBy`, `limit`
102
+
103
+ ## Implementation Order
104
+
105
+ 1. `src/schema/collection.ts` + `src/schema/types.ts` (schema changes)
106
+ 2. `src/crud/query-builder.ts` (core query builder rewrite)
107
+ 3. `src/crud/collection-handle.ts` (wire up orderBy + updated where)
108
+ 4. `src/index.ts` (exports)
109
+ 5. `demo/app.ts` (usage examples)
110
+
111
+ ## Verification
112
+
113
+ - `bun run check` (typecheck)
114
+ - Update demo to exercise new query methods and run it
115
+ - Verify backwards compat: existing `where("field").equals(val).get()` still works