@spooky-sync/core 0.0.1-canary.7 → 0.0.1-canary.71

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/AGENTS.md +56 -0
  2. package/dist/index.d.ts +299 -369
  3. package/dist/index.js +2278 -399
  4. package/dist/otel/index.d.ts +21 -0
  5. package/dist/otel/index.js +86 -0
  6. package/dist/types.d.ts +460 -0
  7. package/package.json +37 -7
  8. package/skills/sp00ky-core/SKILL.md +258 -0
  9. package/skills/sp00ky-core/references/auth.md +98 -0
  10. package/skills/sp00ky-core/references/config.md +76 -0
  11. package/src/build-globals.d.ts +12 -0
  12. package/src/events/events.test.ts +2 -1
  13. package/src/events/index.ts +3 -0
  14. package/src/index.ts +3 -2
  15. package/src/modules/auth/events/index.ts +2 -1
  16. package/src/modules/auth/index.ts +17 -20
  17. package/src/modules/cache/index.ts +41 -29
  18. package/src/modules/cache/types.ts +2 -2
  19. package/src/modules/crdt/crdt-field.ts +281 -0
  20. package/src/modules/crdt/crdt-hydration.test.ts +206 -0
  21. package/src/modules/crdt/index.ts +352 -0
  22. package/src/modules/data/data.status.test.ts +108 -0
  23. package/src/modules/data/index.ts +662 -108
  24. package/src/modules/data/window-query.test.ts +52 -0
  25. package/src/modules/data/window-query.ts +130 -0
  26. package/src/modules/devtools/index.ts +89 -21
  27. package/src/modules/devtools/versions.test.ts +74 -0
  28. package/src/modules/devtools/versions.ts +81 -0
  29. package/src/modules/ref-tables.test.ts +56 -0
  30. package/src/modules/ref-tables.ts +57 -0
  31. package/src/modules/sync/engine.ts +97 -37
  32. package/src/modules/sync/events/index.ts +3 -2
  33. package/src/modules/sync/queue/queue-down.ts +5 -4
  34. package/src/modules/sync/queue/queue-up.ts +14 -13
  35. package/src/modules/sync/scheduler.ts +2 -2
  36. package/src/modules/sync/sync.ts +553 -58
  37. package/src/modules/sync/utils.test.ts +239 -2
  38. package/src/modules/sync/utils.ts +166 -17
  39. package/src/otel/index.ts +127 -0
  40. package/src/services/database/database.ts +11 -11
  41. package/src/services/database/events/index.ts +2 -1
  42. package/src/services/database/local-migrator.ts +21 -21
  43. package/src/services/database/local.test.ts +33 -0
  44. package/src/services/database/local.ts +192 -32
  45. package/src/services/database/remote.ts +13 -13
  46. package/src/services/logger/index.ts +6 -101
  47. package/src/services/persistence/localstorage.ts +2 -2
  48. package/src/services/persistence/resilient.ts +41 -0
  49. package/src/services/persistence/surrealdb.ts +9 -9
  50. package/src/services/stream-processor/index.ts +205 -36
  51. package/src/services/stream-processor/permissions.test.ts +47 -0
  52. package/src/services/stream-processor/permissions.ts +53 -0
  53. package/src/services/stream-processor/stream-processor.batch.test.ts +136 -0
  54. package/src/services/stream-processor/stream-processor.test.ts +1 -1
  55. package/src/services/stream-processor/wasm-types.ts +18 -2
  56. package/src/sp00ky.auth-order.test.ts +65 -0
  57. package/src/sp00ky.ts +582 -0
  58. package/src/types.ts +132 -13
  59. package/src/utils/index.ts +35 -13
  60. package/src/utils/parser.ts +3 -2
  61. package/src/utils/surql.ts +24 -15
  62. package/src/utils/withRetry.test.ts +1 -1
  63. package/tsdown.config.ts +55 -1
  64. package/src/spooky.ts +0 -392
package/AGENTS.md ADDED
@@ -0,0 +1,56 @@
1
+ # `@spooky-sync/core` — agent guide
2
+
3
+ ## What this package is
4
+
5
+ The sync engine. A `Sp00kyClient<S>` owns a local database (memory or IndexedDB), a remote SurrealDB connection, a mutation queue, and a CRDT manager. It's framework-agnostic — UI bindings live in `@spooky-sync/client-solid` (or future `client-react`, etc.). Most app code touches this package indirectly through a UI binding's `useDb()` hook.
6
+
7
+ ## Mental model
8
+
9
+ ```
10
+ .surql schema
11
+ │ spky generate
12
+
13
+ schema.gen.ts (typed schema + SURQL_SCHEMA constant)
14
+ │ passed to SyncedDbConfig
15
+
16
+ Sp00kyClient<S> ── local store (memory | IndexedDB)
17
+ │ ↑↓ DBSP reactive query layer
18
+ │ ↑↓ mutation queue
19
+ └── SSP ── remote SurrealDB
20
+ ```
21
+
22
+ Local mutations are applied optimistically, ingested into a DBSP layer that drives reactive query updates, then drained to the remote via SSP. Live updates flow back the same path.
23
+
24
+ ## Key exports (`src/index.ts`)
25
+
26
+ - `Sp00kyClient<S>` — main class. Methods: `init()`, `create(id, payload)`, `update(table, id, payload, options?)`, `delete(table, idOrSelector)`, `query(table, opts?)`, `run(backend, route, payload)`, `bucket(name)`, `useRemote(fn)`, `authenticate(token)`, `signOut()`. Plus `pendingMutationCount` and `subscribeToPendingMutations(cb)`.
27
+ - `BucketHandle` — file storage handle (`put`, `get`, `delete`, `exists`).
28
+ - `AuthService` — token management, sign-in/sign-out events.
29
+ - `CrdtManager`, `CrdtField`, `cursorColorFromName`, `CURSOR_COLORS` — Loro-CRDT integration.
30
+ - Types: `Sp00kyConfig`, `SyncedDbConfig` (re-exported by client-solid as the consumer-facing shape), `QueryTimeToLive`, `PersistenceClient`, `StoreType`, `UpdateOptions`, `RunOptions`.
31
+ - Subpath: `@spooky-sync/core/otel` — `createOtelTransmit(endpoint)` for piping pino logs to OpenTelemetry.
32
+
33
+ ## Common gotchas
34
+
35
+ - **Use UI hooks, not the client directly.** In a Solid app, call `useDb()` from `@spooky-sync/client-solid`. Touch `Sp00kyClient` only inside `provider.client` for advanced flows.
36
+ - **Mutations are optimistic.** `db.create` / `db.update` / `db.delete` return immediately; the queue drains in the background. Inspect progress via `pendingMutationCount`.
37
+ - **Record IDs are full strings.** `db.create('thread:abc123', {...})` — *not* `db.create('thread', { id: 'abc123' })`. The first arg is `'<table>:<id>'`. Generate IDs with `Uuid` from `surrealdb` (re-exported by `client-solid`).
38
+ - **`db.update` takes `(table, id, payload, options?)`**. The `debounced` option (`{ debounced: true }` or `{ debounced: { delay, key } }`) coalesces rapid updates — use it for CRDT text/title fields.
39
+ - **`db.useRemote(fn)` bypasses the cache.** Use only for queries that can't be expressed via the query builder (e.g. graph traversals across `RELATE` edges). Results are not synced into the local cache.
40
+ - **`@parent` columns are auto-populated.** Defined in the `.surql` schema as `record<user>` with the `-- @parent` annotation; never write them yourself — they're set from the auth context server-side.
41
+
42
+ ## Module map (under `src/modules/`)
43
+
44
+ - `data/` — local store + DBSP reactive query layer.
45
+ - `sync/` — SSP client, mutation queue, live-update ingestion.
46
+ - `cache/` — query result cache with TTL.
47
+ - `crdt/` — Loro CRDT manager for collaborative text fields.
48
+ - `auth/` — token storage, sign-in flow.
49
+ - `devtools/` — devtools bridge (talks to the browser extension and `@spooky-sync/devtools-mcp`).
50
+
51
+ ## Pointers
52
+
53
+ - UI bindings: `node_modules/@spooky-sync/client-solid/AGENTS.md`
54
+ - Query builder DSL: `node_modules/@spooky-sync/query-builder/AGENTS.md`
55
+ - Schema codegen / migrations: `node_modules/@spooky-sync/cli/AGENTS.md`
56
+ - Live introspection: `node_modules/@spooky-sync/devtools-mcp/AGENTS.md`