@evolu/common 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2023 Evolu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,205 @@
+# Evolu
+
+[Local-first](https://www.inkandswitch.com/local-first/) platform designed for privacy, ease of use, and no vendor lock-in to sync&back precious data.
+
+- [SQLite](https://sqlite.org/) in all browsers, Electron, and React Native
+- E2E encrypted sync and backup with [CRDT](https://crdt.tech/) (merging changes without conflicts)
+- Free Evolu server for testing (paid production-ready soon, or you can run your own)
+- Typed database schema with branded types (`NonEmptyString1000`, `PositiveInt`, etc.)
+- Reactive queries with React Suspense support
+- Real-time experience via revalidation on focus and network recovery
+- Schema evolving via `filterMap` ad-hoc migration
+- No signup/login, no email collection, only Bitcoin-like mnemonic (12 words)
+- Fetching nested objects and arrays in a single SQL query
+- JSON support with automatic parsing and stringifying
+
+## Local-first apps
+
+Local-first apps allow users to own their data. Evolu stores data in the user's device(s), so Evolu apps can work offline and without a specific server. How is it different from keeping files on disk? Files are not the right abstraction for apps and are complicated to synchronize among devices. That's why client-server architecture rules the world. But as with everything, it has trade-offs.
+
+### The trade-offs of the client-server architecture
+
+Client-server architecture provides us with easy backup and synchronization, but all that depends on the ability of a server to fulfill its promises. Internet is offline, companies go bankrupt, users are banned, and errors occur. All those things happen all the time, and then what? Right, that's why the world needs local-first apps. But until now, writing local-first apps has been challenging because of the lack of libraries and design patterns. That's why I created Evolu.
+
+## Overview
+
+### Define Data
+
+To start using Evolu, define tables for your database and export React Hooks.
+
+```ts
+import * as Schema from "@effect/schema/Schema";
+import * as Evolu from "@evolu/react";
+
+const TodoId = Evolu.id("Todo");
+type TodoId = Schema.Schema.To<typeof TodoId>;
+
+const TodoTable = Schema.struct({
+  id: TodoId,
+  title: Evolu.NonEmptyString1000,
+  isCompleted: Evolu.SqliteBoolean,
+});
+type TodoTable = Schema.Schema.To<typeof TodoTable>;
+
+const Database = Schema.struct({
+  todo: TodoTable,
+});
+
+export const {
+  useQuery,
+  useMutation,
+  useOwner,
+  useOwnerActions,
+  useEvoluError,
+} = Evolu.create(Database);
+```
+
+### Validate Data
+
+Learn more about [Schema](https://github.com/effect-ts/schema).
+
+```ts
+import * as Schema from "@effect/schema/Schema";
+import * as Evolu from "@evolu/react";
+
+Schema.parse(Evolu.String1000)(title);
+```
+
+### Mutate Data
+
+Mutation API is designed for local-first apps to ensure changes are always merged without conflicts.
+
+```ts
+const { create, update } = useMutation();
+
+create("todo", { title, isCompleted: false });
+update("todo", { id, isCompleted: true });
+```
+
+### Query Data
+
+Evolu uses type-safe TypeScript SQL query builder [kysely](https://github.com/koskimas/kysely), so autocompletion works out-of-the-box.
+
+```ts
+const { rows } = useQuery(
+  (db) => db.selectFrom("todo").select(["id", "title"]).orderBy("updatedAt"),
+  // (row) => row
+  ({ title, ...rest }) => title && { title, ...rest },
+);
+```
+
+### Protect Data
+
+Evolu encrypts data with Mnemonic, a safe autogenerated password based on [bip39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki).
+
+```ts
+const owner = useOwner();
+
+alert(owner.mnemonic);
+```
+
+### Delete Data
+
+Leave no traces on a device.
+
+```ts
+const ownerActions = useOwnerActions();
+
+if (confirm("Are you sure? It will delete all your local data."))
+  ownerActions.reset();
+```
+
+### Restore Data
+
+Restore data elsewhere. Encrypted data can only be restored with a Mnemonic.
+
+```ts
+const ownerActions = useOwnerActions();
+
+ownerActions.restore(mnemonic).then((either) => {
+  if (either._tag === "Left") alert(JSON.stringify(either.left, null, 2));
+});
+```
+
+### Handle Errors
+
+Evolu `useQuery` and `useMutation` never fail, it's the advantage of local first apps, but Evolu, in rare cases, can.
+
+```ts
+const evoluError = useEvoluError();
+
+useEffect(() => {
+  // eslint-disable-next-line no-console
+  if (evoluError) console.log(evoluError);
+}, [evoluError]);
+```
+
+And that's all. Minimal API is the key to a great developer experience.
+
+## Privacy
+
+Evolu uses end-to-end encryption and generates strong and safe passwords for you. Evolu sync and backup server see only userId and timestamps.
+
+## Trade-offs
+
+> “There are no solutions. There are only trade-offs.” ― Thomas Sowell
+
+Evolu is not P2P. For reliable syncing and backup, there needs to be a server. Evolu server is very minimal, and everyone can run their own. While it's theoretically possible to have P2P Evolu, I have yet to see a reliable solution. It's not only a technical problem; it's an economic problem. Someone has to be paid to keep your data safe. Evolu provides a free server for testing. Soon we will provide a paid server for production usage.
+
+All table columns except for ID are nullable by default. It's not a bug; it's a feature. Local-first data are meant to last forever, but schemas evolve. This design decision was inspired by GraphQL [nullability](https://graphql.org/learn/best-practices/#nullability) and [versioning](https://graphql.org/learn/best-practices/#versioning). Evolu provides a handy `filterMap` helper for queries.
+
+## Community
+
+The Evolu community is on GitHub Discussions, where you can ask questions and voice ideas.
+
+To chat with other community members, you can join the [Evolu Discord](https://discord.gg/2J8yyyyxtZ).
+
+[![Twitter URL](https://img.shields.io/twitter/url/https/twitter.com/evoluhq.svg?style=social&label=Follow%20%40evoluhq)](https://twitter.com/evoluhq)
+
+## FAQ
+
+### Is Evolu ready for production?
+
+Yes. Evolu is stable.
+
+### What is the SQLite database size limit?
+
+[Storage quotas and eviction criteria](https://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria)
+
+### How can I check the current database filesize?
+
+Use OPFS Explorer Chrome DevTools extension.
+
+## Contributing
+
+Evolu monorepo uses [pnpm](https://pnpm.io).
+
+Install the dependencies with:
+
+```
+pnpm install
+```
+
+Build Evolu monorepo:
+
+```
+pnpm build
+```
+
+Start developing and watch for code changes:
+
+```
+pnpm dev
+```
+
+Lint and tests:
+
+```
+pnpm lint test
+```
+
+Describe changes for release log:
+
+```
+pnpm changeset
+```

package/dist/src/Config.d.ts

@@ -0,0 +1,20 @@
+import { Context, Layer } from "effect";
+export interface Config {
+    /**
+     * Alternate URL to Evolu sync&backup server.
+     */
+    readonly syncUrl: string;
+    /**
+     * Alternate URL to reload browser tabs after `Owner` reset or restore.
+     * The default value is `/`.
+     */
+    readonly reloadUrl: string;
+    /**
+     * Maximum physical clock drift allowed in ms.
+     * The default value is 5 * 60 * 1000 (5 minutes).
+     */
+    readonly maxDrift: number;
+}
+export declare const Config: Context.Tag<Config, Config>;
+export declare const ConfigLive: (config?: Partial<Config>) => Layer.Layer<never, never, Config>;
+//# sourceMappingURL=Config.d.ts.map

package/dist/src/Config.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"Config.d.ts","sourceRoot":"","sources":["../../src/Config.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,KAAK,EAAE,MAAM,QAAQ,CAAC;AAExC,MAAM,WAAW,MAAM;IACrB;;OAEG;IACH,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB;;;OAGG;IACH,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B;;;OAGG;IACH,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;CAC3B;AAED,eAAO,MAAM,MAAM,6BAAsC,CAAC;AAE1D,eAAO,MAAM,UAAU,YACZ,QAAQ,MAAM,CAAC,KACvB,WAAW,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,CAShC,CAAC"}

package/dist/src/Config.js

@@ -0,0 +1,8 @@
+import { Context, Layer } from "effect";
+export const Config = Context.Tag("evolu/Config");
+export const ConfigLive = (config) => Layer.succeed(Config, Config.of({
+    syncUrl: "https://evolu.world",
+    maxDrift: 5 * 60 * 1000,
+    reloadUrl: "/",
+    ...config,
+}));

package/dist/src/Crypto.d.ts

@@ -0,0 +1,42 @@
+import * as Schema from "@effect/schema/Schema";
+import { Brand, Context, Effect, Layer } from "effect";
+export interface Bip39 {
+    readonly make: Effect.Effect<never, never, Mnemonic>;
+    readonly toSeed: (mnemonic: Mnemonic) => Effect.Effect<never, never, Uint8Array>;
+    readonly parse: (mnemonic: string) => Effect.Effect<never, InvalidMnemonicError, Mnemonic>;
+}
+export declare const Bip39: Context.Tag<Bip39, Bip39>;
+export interface InvalidMnemonicError {
+    readonly _tag: "InvalidMnemonicError";
+}
+/**
+ * Mnemonic is a password generated by Evolu in BIP39 format.
+ *
+ * A mnemonic, also known as a "seed phrase," is a set of 12 words in a
+ * specific order chosen from a predefined list. The purpose of the BIP39
+ * mnemonic is to provide a human-readable way of storing a private key.
+ */
+export type Mnemonic = string & Brand.Brand<"Mnemonic">;
+export interface NanoId {
+    readonly nanoid: Effect.Effect<never, never, string>;
+    readonly nanoidAsNodeId: Effect.Effect<never, never, NodeId>;
+}
+export declare const NanoId: Context.Tag<NanoId, NanoId>;
+export declare const NodeId: Schema.BrandSchema<string, string & Brand.Brand<"NodeId">>;
+export type NodeId = Schema.Schema.To<typeof NodeId>;
+export declare const NanoIdLive: Layer.Layer<never, never, NanoId>;
+/**
+ * SLIP-21 implementation
+ * https://github.com/satoshilabs/slips/blob/master/slip-0021.md
+ */
+export declare const slip21Derive: (seed: Uint8Array, path: string[]) => Effect.Effect<never, never, Uint8Array>;
+/**
+ * Alias to xsalsa20poly1305, for compatibility with libsodium / nacl
+ */
+export interface SecretBox {
+    readonly seal: (key: Uint8Array, plaintext: Uint8Array) => Effect.Effect<never, never, Uint8Array>;
+    readonly open: (key: Uint8Array, ciphertext: Uint8Array) => Effect.Effect<never, never, Uint8Array>;
+}
+export declare const SecretBox: Context.Tag<SecretBox, SecretBox>;
+export declare const SecretBoxLive: Layer.Layer<never, never, SecretBox>;
+//# sourceMappingURL=Crypto.d.ts.map

package/dist/src/Crypto.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"Crypto.d.ts","sourceRoot":"","sources":["../../src/Crypto.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,MAAM,MAAM,uBAAuB,CAAC;AAMhD,OAAO,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,QAAQ,CAAC;AAGvD,MAAM,WAAW,KAAK;IACpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,QAAQ,CAAC,CAAC;IAErD,QAAQ,CAAC,MAAM,EAAE,CACf,QAAQ,EAAE,QAAQ,KACf,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,UAAU,CAAC,CAAC;IAE7C,QAAQ,CAAC,KAAK,EAAE,CACd,QAAQ,EAAE,MAAM,KACb,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,oBAAoB,EAAE,QAAQ,CAAC,CAAC;CAC3D;AAED,eAAO,MAAM,KAAK,2BAAoC,CAAC;AAEvD,MAAM,WAAW,oBAAoB;IACnC,QAAQ,CAAC,IAAI,EAAE,sBAAsB,CAAC;CACvC;AAED;;;;;;GAMG;AACH,MAAM,MAAM,QAAQ,GAAG,MAAM,GAAG,KAAK,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;AAExD,MAAM,WAAW,MAAM;IACrB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC;IACrD,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,MAAM,CAAC,CAAC;CAC9D;AAED,eAAO,MAAM,MAAM,6BAAsC,CAAC;AAE1D,eAAO,MAAM,MAAM,EAAE,MAAM,CAAC,WAAW,CACrC,MAAM,EACN,MAAM,GAAG,KAAK,CAAC,KAAK,CAAC,QAAQ,CAAC,CAC6C,CAAC;AAC9E,MAAM,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,OAAO,MAAM,CAAC,CAAC;AAIrD,eAAO,MAAM,UAAU,mCAMtB,CAAC;AAEF;;;GAGG;AACH,eAAO,MAAM,YAAY,SACjB,UAAU,QACV,MAAM,EAAE,KACb,aAAa,CAAC,KAAK,EAAE,KAAK,EAAE,UAAU,CAWrC,CAAC;AAEL;;GAEG;AACH,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,IAAI,EAAE,CACb,GAAG,EAAE,UAAU,EACf,SAAS,EAAE,UAAU,KAClB,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,UAAU,CAAC,CAAC;IAE7C,QAAQ,CAAC,IAAI,EAAE,CACb,GAAG,EAAE,UAAU,EACf,UAAU,EAAE,UAAU,KACnB,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,EAAE,UAAU,CAAC,CAAC;CAC9C;AAED,eAAO,MAAM,SAAS,mCAA4C,CAAC;AAEnE,eAAO,MAAM,aAAa,sCAgBzB,CAAC"}

package/dist/src/Crypto.js

@@ -0,0 +1,44 @@
+import * as Schema from "@effect/schema/Schema";
+import { secretbox } from "@noble/ciphers/salsa";
+import { concatBytes } from "@noble/ciphers/utils";
+import { hmac } from "@noble/hashes/hmac";
+import { sha512 } from "@noble/hashes/sha512";
+import { randomBytes } from "@noble/hashes/utils";
+import { Context, Effect, Layer } from "effect";
+import { customAlphabet, nanoid } from "nanoid";
+export const Bip39 = Context.Tag("evolu/Bip39");
+export const NanoId = Context.Tag("evolu/NanoId");
+export const NodeId = Schema.string.pipe(Schema.pattern(/^[\w-]{16}$/), Schema.brand("NodeId"));
+const nanoidForNodeId = customAlphabet("0123456789abcdef", 16);
+export const NanoIdLive = Layer.succeed(NanoId, NanoId.of({
+    nanoid: Effect.sync(() => nanoid()),
+    nanoidAsNodeId: Effect.sync(() => nanoidForNodeId()),
+}));
+/**
+ * SLIP-21 implementation
+ * https://github.com/satoshilabs/slips/blob/master/slip-0021.md
+ */
+export const slip21Derive = (seed, path) => Effect.sync(() => {
+    let m = hmac(sha512, "Symmetric key seed", seed);
+    for (let i = 0; i < path.length; i++) {
+        const p = new TextEncoder().encode(path[i]);
+        const e = new Uint8Array(p.byteLength + 1);
+        e[0] = 0;
+        e.set(p, 1);
+        m = hmac(sha512, m.slice(0, 32), e);
+    }
+    return m.slice(32, 64);
+});
+export const SecretBox = Context.Tag("evolu/SecretBox");
+export const SecretBoxLive = Layer.succeed(SecretBox, SecretBox.of({
+    seal: (key, plaintext) => Effect.sync(() => {
+        const nonce = randomBytes(24);
+        const ciphertext = secretbox(key, nonce).seal(plaintext);
+        return concatBytes(nonce, ciphertext);
+    }),
+    open: (key, ciphertext) => Effect.sync(() => {
+        const nonce = ciphertext.subarray(0, 24);
+        const ciphertextWithoutNonce = ciphertext.subarray(24);
+        return secretbox(key, nonce).open(ciphertextWithoutNonce);
+    }),
+}));

package/dist/src/Db.d.ts

@@ -0,0 +1,60 @@
+import * as S from "@effect/schema/Schema";
+import { Brand, Context, Effect, ReadonlyRecord } from "effect";
+import * as Kysely from "kysely";
+import { Bip39, Mnemonic, NanoId } from "./Crypto.js";
+import { Id, SqliteBoolean, SqliteDate } from "./Model.js";
+import { Query, Row, Sqlite, Value } from "./Sqlite.js";
+export type TableSchema = ReadonlyRecord.ReadonlyRecord<Value> & {
+    readonly id: Id;
+};
+export type Schema = ReadonlyRecord.ReadonlyRecord<TableSchema>;
+export type CreateQuery<S extends Schema> = (queryCallback: QueryCallback<S, Row>) => Query;
+export type QueryCallback<S extends Schema, QueryRow> = (db: KyselyWithoutMutation<SchemaForQuery<S>>) => Kysely.SelectQueryBuilder<any, any, QueryRow>;
+type KyselyWithoutMutation<DB> = Pick<Kysely.Kysely<DB>, "selectFrom" | "fn">;
+type SchemaForQuery<S extends Schema> = {
+    readonly [Table in keyof S]: NullableExceptOfId<{
+        readonly [Column in keyof S[Table]]: S[Table][Column];
+    } & CommonColumns>;
+};
+export type NullableExceptOfId<T> = {
+    readonly [K in keyof T]: K extends "id" ? T[K] : T[K] | null;
+};
+export interface CommonColumns {
+    readonly createdAt: SqliteDate;
+    readonly updatedAt: SqliteDate;
+    readonly isDeleted: SqliteBoolean;
+}
+export interface Table {
+    readonly name: string;
+    readonly columns: ReadonlyArray<string>;
+}
+export type Tables = ReadonlyArray<Table>;
+export declare const makeCreateQuery: <T extends Schema>() => CreateQuery<T>;
+export declare const schemaToTables: (schema: S.Schema<any, any>) => Tables;
+/**
+ * `Owner` represents the Evolu database owner. Evolu auto-generates `Owner`
+ * on the first run. `Owner` can be reset on the current device and restored
+ * on a different one.
+ */
+export interface Owner {
+    /** The `Mnemonic` associated with `Owner`. */
+    readonly mnemonic: Mnemonic;
+    /** The unique identifier of `Owner` safely derived from its `Mnemonic`. */
+    readonly id: OwnerId;
+    readonly encryptionKey: Uint8Array;
+}
+export declare const Owner: Context.Tag<Owner, Owner>;
+/**
+ * The unique identifier of `Owner` safely derived from its `Mnemonic`.
+ */
+export type OwnerId = Id & Brand.Brand<"Owner">;
+export declare const transaction: <R, E, A>(effect: Effect.Effect<R, E, A>) => Effect.Effect<Sqlite | R, E, A>;
+export interface NoSuchTableOrColumnError {
+    readonly _tag: "NoSuchTableOrColumnError";
+}
+export declare const someDefectToNoSuchTableOrColumnError: <R, E, A>(self: Effect.Effect<R, E, A>) => Effect.Effect<R, NoSuchTableOrColumnError | E, A>;
+export declare const makeOwner: (mnemonic?: Mnemonic) => Effect.Effect<Bip39, never, Owner>;
+export declare const lazyInit: (mnemonic?: Mnemonic) => Effect.Effect<Sqlite | Bip39 | NanoId, never, Owner>;
+export declare const ensureSchema: (tables: Tables) => Effect.Effect<Sqlite, never, void>;
+export {};
+//# sourceMappingURL=Db.d.ts.map

package/dist/src/Db.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"Db.d.ts","sourceRoot":"","sources":["../../src/Db.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,CAAC,MAAM,uBAAuB,CAAC;AAG3C,OAAO,EACL,KAAK,EACL,OAAO,EACP,MAAM,EAKN,cAAc,EAGf,MAAM,QAAQ,CAAC;AAChB,OAAO,KAAK,MAAM,MAAM,QAAQ,CAAC;AAEjC,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,MAAM,EAAgB,MAAM,aAAa,CAAC;AAEpE,OAAO,EAAE,EAAE,EAAE,aAAa,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAO3D,OAAO,EACL,KAAK,EAEL,GAAG,EACH,MAAM,EACN,KAAK,EAEN,MAAM,aAAa,CAAC;AAGrB,MAAM,MAAM,WAAW,GAAG,cAAc,CAAC,cAAc,CAAC,KAAK,CAAC,GAAG;IAC/D,QAAQ,CAAC,EAAE,EAAE,EAAE,CAAC;CACjB,CAAC;AAEF,MAAM,MAAM,MAAM,GAAG,cAAc,CAAC,cAAc,CAAC,WAAW,CAAC,CAAC;AAEhE,MAAM,MAAM,WAAW,CAAC,CAAC,SAAS,MAAM,IAAI,CAC1C,aAAa,EAAE,aAAa,CAAC,CAAC,EAAE,GAAG,CAAC,KACjC,KAAK,CAAC;AAEX,MAAM,MAAM,aAAa,CAAC,CAAC,SAAS,MAAM,EAAE,QAAQ,IAAI,CACtD,EAAE,EAAE,qBAAqB,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,KAEzC,MAAM,CAAC,kBAAkB,CAAC,GAAG,EAAE,GAAG,EAAE,QAAQ,CAAC,CAAC;AAEnD,KAAK,qBAAqB,CAAC,EAAE,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,EAAE,YAAY,GAAG,IAAI,CAAC,CAAC;AAE9E,KAAK,cAAc,CAAC,CAAC,SAAS,MAAM,IAAI;IACtC,QAAQ,EAAE,KAAK,IAAI,MAAM,CAAC,GAAG,kBAAkB,CAC7C;QACE,QAAQ,EAAE,MAAM,IAAI,MAAM,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC;KACtD,GAAG,aAAa,CAClB;CACF,CAAC;AAEF,MAAM,MAAM,kBAAkB,CAAC,CAAC,IAAI;IAClC,QAAQ,EAAE,CAAC,IAAI,MAAM,CAAC,GAAG,CAAC,SAAS,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI;CAC7D,CAAC;AAEF,MAAM,WAAW,aAAa;IAC5B,QAAQ,CAAC,SAAS,EAAE,UAAU,CAAC;IAC/B,QAAQ,CAAC,SAAS,EAAE,UAAU,CAAC;IAC/B,QAAQ,CAAC,SAAS,EAAE,aAAa,CAAC;CACnC;AAED,MAAM,WAAW,KAAK;IACpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,OAAO,EAAE,aAAa,CAAC,MAAM,CAAC,CAAC;CACzC;AAED,MAAM,MAAM,MAAM,GAAG,aAAa,CAAC,KAAK,CAAC,CAAC;AAe1C,eAAO,MAAM,eAAe,wCAGmD,CAAC;AAkBhF,eAAO,MAAM,cAAc,WAEjB,QAAQ,CAAC,GAAG,EAAE,GAAG,CAAC,WAazB,CAAC;AAEJ;;;;GAIG;AACH,MAAM,WAAW,KAAK;IACpB,8CAA8C;IAC9C,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAC;IAC5B,2EAA2E;IAC3E,QAAQ,CAAC,EAAE,EAAE,OAAO,CAAC;IAErB,QAAQ,CAAC,aAAa,EAAE,UAAU,CAAC;CACpC;AAED,eAAO,MAAM,KAAK,2BAAoC,CAAC;AAEvD;;GAEG;AACH,MAAM,MAAM,OAAO,GAAG,EAAE,GAAG,KAAK,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC;AAEhD,eAAO,MAAM,WAAW,8EAUrB,CAAC;AAEJ,MAAM,WAAW,wBAAwB;IACvC,QAAQ,CAAC,IAAI,EAAE,0BAA0B,CAAC;CAC3C;AAED,eAAO,MAAM,oCAAoC,8FAoBhD,CAAC;AAEF,eAAO,MAAM,SAAS,cACT,QAAQ,KAClB,aAAa,CAAC,KAAK,EAAE,KAAK,EAAE,KAAK,CA0BhC,CAAC;AAEL,eAAO,MAAM,QAAQ,cACR,QAAQ,KAClB,aAAa,CAAC,MAAM,GAAG,KAAK,GAAG,MAAM,EAAE,KAAK,EAAE,KAAK,CA4BlD,CAAC;AA6DL,eAAO,MAAM,YAAY,sBAEtB,aAAa,CAAC,MAAM,EAAE,KAAK,EAAE,IAAI,CAUjC,CAAC"}

package/dist/src/Db.js

@@ -0,0 +1,118 @@
+import * as AST from "@effect/schema/AST";
+import { make } from "@effect/schema/Schema";
+import { bytesToHex } from "@noble/ciphers/utils";
+import { Context, Effect, Exit, Option, Predicate, ReadonlyArray, ReadonlyRecord, String, pipe, } from "effect";
+import * as Kysely from "kysely";
+import { urlAlphabet } from "nanoid";
+import { Bip39, slip21Derive } from "./Crypto.js";
+import { initialMerkleTree, merkleTreeToString } from "./MerkleTree.js";
+import { createMessageTable, createMessageTableIndex, createOwnerTable, insertOwner, } from "./Sql.js";
+import { Sqlite, queryObjectToQuery, } from "./Sqlite.js";
+import { makeInitialTimestamp, timestampToString } from "./Timestamp.js";
+const commonColumns = ["createdAt", "updatedAt", "isDeleted"];
+const kysely = new Kysely.Kysely({
+    dialect: {
+        createAdapter: () => new Kysely.SqliteAdapter(),
+        createDriver: () => new Kysely.DummyDriver(),
+        createIntrospector() {
+            throw "Not implemeneted";
+        },
+        createQueryCompiler: () => new Kysely.SqliteQueryCompiler(),
+    },
+});
+export const makeCreateQuery = () => (queryCallback) => queryObjectToQuery(queryCallback(kysely).compile());
+// https://github.com/Effect-TS/schema/releases/tag/v0.18.0
+// eslint-disable-next-line @typescript-eslint/no-explicit-any
+const getPropertySignatures = (schema) => {
+    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+    const out = {};
+    const propertySignatures = AST.getPropertySignatures(schema.ast);
+    for (let i = 0; i < propertySignatures.length; i++) {
+        const propertySignature = propertySignatures[i];
+        out[propertySignature.name] = make(propertySignature.type);
+    }
+    // eslint-disable-next-line @typescript-eslint/no-unsafe-return, @typescript-eslint/no-explicit-any
+    return out;
+};
+export const schemaToTables = (
+// eslint-disable-next-line @typescript-eslint/no-explicit-any
+schema) => pipe(getPropertySignatures(schema), ReadonlyRecord.toEntries, ReadonlyArray.map(([name, schema]) => ({
+    name,
+    columns: Object.keys(getPropertySignatures(schema)).concat(commonColumns),
+})));
+export const Owner = Context.Tag("evolu/Owner");
+export const transaction = (effect) => Effect.flatMap(Sqlite, (sqlite) => Effect.acquireUseRelease(sqlite.exec("BEGIN"), () => effect, (_, exit) => Exit.isFailure(exit) ? sqlite.exec("ROLLBACK") : sqlite.exec("END")));
+export const someDefectToNoSuchTableOrColumnError = Effect.catchSomeDefect((error) => {
+    if (typeof error === "object" &&
+        error != null &&
+        "message" in error &&
+        typeof error.message === "string" &&
+        error.message.includes("code 1") &&
+        (error.message.includes("no such table") ||
+            error.message.includes("no such column") ||
+            error.message.includes("has no column")))
+        return Option.some(Effect.fail({
+            _tag: "NoSuchTableOrColumnError",
+        }));
+    return Option.none();
+});
+export const makeOwner = (mnemonic) => Effect.gen(function* (_) {
+    const bip39 = yield* _(Bip39);
+    if (mnemonic == null)
+        mnemonic = yield* _(bip39.make);
+    const seed = yield* _(bip39.toSeed(mnemonic));
+    const id = yield* _(slip21Derive(seed, ["Evolu", "Owner Id"]).pipe(Effect.map((key) => {
+        // convert key to nanoid
+        let id = "";
+        for (let i = 0; i < 21; i++) {
+            id += urlAlphabet[key[i] & 63];
+        }
+        return id;
+    })));
+    const encryptionKey = yield* _(slip21Derive(seed, ["Evolu", "Encryption Key"]));
+    return { mnemonic, id, encryptionKey };
+});
+export const lazyInit = (mnemonic) => Effect.gen(function* (_) {
+    const [owner, sqlite, initialTimestampString] = yield* _(Effect.all([makeOwner(mnemonic), Sqlite, makeInitialTimestamp], {
+        concurrency: "unbounded",
+    }));
+    yield* _(Effect.all([
+        sqlite.exec(createMessageTable),
+        sqlite.exec(createMessageTableIndex),
+        sqlite.exec(createOwnerTable),
+        sqlite.exec({
+            sql: insertOwner,
+            parameters: [
+                owner.id,
+                owner.mnemonic,
+                // expo-sqlite 11.3.2 doesn't support Uint8Array
+                bytesToHex(owner.encryptionKey),
+                timestampToString(initialTimestampString),
+                merkleTreeToString(initialMerkleTree),
+            ],
+        }),
+    ]));
+    return owner;
+});
+const getTables = Sqlite.pipe(Effect.flatMap((sqlite) => sqlite.exec(`SELECT "name" FROM "sqlite_schema" WHERE "type" = 'table'`)), Effect.map((result) => result.rows), Effect.map(ReadonlyArray.map((row) => row.name + "")), Effect.map(ReadonlyArray.filter(Predicate.not(String.startsWith("__")))), Effect.map(ReadonlyArray.dedupeWith(String.Equivalence)));
+const updateTable = ({ name, columns, }) => Effect.gen(function* (_) {
+    const sqlite = yield* _(Sqlite);
+    const sql = yield* _(sqlite.exec(`PRAGMA table_info (${name})`), Effect.map((result) => result.rows), Effect.map(ReadonlyArray.map((row) => row.name)), Effect.map((existingColumns) => ReadonlyArray.differenceWith(String.Equivalence)(existingColumns)(columns)), Effect.map(ReadonlyArray.map((newColumn) => `ALTER TABLE "${name}" ADD COLUMN "${newColumn}" blob;`)), Effect.map(ReadonlyArray.join("")));
+    if (sql)
+        yield* _(sqlite.exec(sql));
+});
+const createTable = ({ name, columns, }) => Effect.flatMap(Sqlite, (sqlite) => sqlite.exec(`
+      CREATE TABLE ${name} (
+        "id" text primary key,
+        ${columns
+    .filter((c) => c !== "id")
+    // "A column with affinity BLOB does not prefer one storage class over another
+    // and no attempt is made to coerce data from one storage class into another."
+    // https://www.sqlite.org/datatype3.html
+    .map((name) => `"${name}" blob`)
+    .join(", ")}
+      );
+    `));
+export const ensureSchema = (tables) => Effect.flatMap(getTables, (existingTables) => Effect.forEach(tables, (tableDefinition) => existingTables.includes(tableDefinition.name)
+    ? updateTable(tableDefinition)
+    : createTable(tableDefinition), { discard: true }));

package/dist/src/DbWorker.d.ts

@@ -0,0 +1,78 @@
+import { Brand, Context, Layer, ReadonlyArray, ReadonlyRecord } from "effect";
+import { Config } from "./Config.js";
+import { Bip39, Mnemonic, NanoId } from "./Crypto.js";
+import { Owner, Tables } from "./Db.js";
+import { QueryPatches } from "./Diff.js";
+import { EvoluError } from "./Errors.js";
+import { Id, SqliteDate } from "./Model.js";
+import { Query, Row, Sqlite, Value } from "./Sqlite.js";
+import { SyncState, SyncWorker, SyncWorkerOutputSyncResponse } from "./SyncWorker.js";
+export interface DbWorker {
+    readonly postMessage: (input: DbWorkerInput) => void;
+    onMessage: (output: DbWorkerOutput) => void;
+}
+export declare const DbWorker: Context.Tag<DbWorker, DbWorker>;
+export type DbWorkerInput = DbWorkerInputInit | DbWorkerInputQuery | DbWorkerInputMutate | DbWorkerInputSync | DbWorkerInputReset | DbWorkerInputEnsureSchema | SyncWorkerOutputSyncResponse;
+interface DbWorkerInputInit {
+    readonly _tag: "init";
+    readonly config: Config;
+    readonly tables: Tables;
+}
+interface DbWorkerInputQuery {
+    readonly _tag: "query";
+    readonly queries: ReadonlyArray.NonEmptyReadonlyArray<Query>;
+}
+interface DbWorkerInputMutate {
+    readonly _tag: "mutate";
+    readonly items: ReadonlyArray.NonEmptyReadonlyArray<MutateItem>;
+    readonly queries: ReadonlyArray<Query>;
+}
+interface DbWorkerInputSync {
+    readonly _tag: "sync";
+    readonly queries: ReadonlyArray<Query>;
+}
+interface DbWorkerInputReset {
+    readonly _tag: "reset";
+    readonly mnemonic?: Mnemonic;
+}
+interface DbWorkerInputEnsureSchema {
+    readonly _tag: "ensureSchema";
+    readonly tables: Tables;
+}
+export type DbWorkerOutput = DbWorkerOutputOnError | DbWorkerOutputOnOwner | DbWorkerOutputOnQuery | DbWorkerOutputOnReceive | DbWorkerOutputOnResetOrRestore | DbWorkerOutputOnSyncState;
+interface DbWorkerOutputOnError {
+    readonly _tag: "onError";
+    readonly error: EvoluError;
+}
+interface DbWorkerOutputOnOwner {
+    readonly _tag: "onOwner";
+    readonly owner: Owner;
+}
+export interface DbWorkerOutputOnQuery {
+    readonly _tag: "onQuery";
+    readonly queriesPatches: ReadonlyArray<QueryPatches>;
+    readonly onCompleteIds: ReadonlyArray<OnCompleteId>;
+}
+export type OnCompleteId = string & Brand.Brand<"Id"> & Brand.Brand<"OnComplete">;
+interface DbWorkerOutputOnReceive {
+    readonly _tag: "onReceive";
+}
+interface DbWorkerOutputOnResetOrRestore {
+    readonly _tag: "onResetOrRestore";
+}
+interface DbWorkerOutputOnSyncState {
+    readonly _tag: "onSyncState";
+    readonly state: SyncState;
+}
+export interface MutateItem {
+    readonly table: string;
+    readonly id: Id;
+    readonly values: ReadonlyRecord.ReadonlyRecord<Value | Date | boolean | undefined>;
+    readonly isInsert: boolean;
+    readonly now: SqliteDate;
+    readonly onCompleteId: OnCompleteId | null;
+}
+export type RowsCacheMap = ReadonlyMap<Query, ReadonlyArray<Row>>;
+export declare const DbWorkerLive: Layer.Layer<Bip39 | NanoId | Sqlite | SyncWorker, never, DbWorker>;
+export {};
+//# sourceMappingURL=DbWorker.d.ts.map

package/dist/src/DbWorker.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"DbWorker.d.ts","sourceRoot":"","sources":["../../src/DbWorker.ts"],"names":[],"mappings":"AACA,OAAO,EACL,KAAK,EACL,OAAO,EAGP,KAAK,EAGL,aAAa,EACb,cAAc,EAGf,MAAM,QAAQ,CAAC;AAChB,OAAO,EAAE,MAAM,EAAc,MAAM,aAAa,CAAC;AACjD,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AACtD,OAAO,EACL,KAAK,EAGL,MAAM,EAKP,MAAM,SAAS,CAAC;AACjB,OAAO,EAAE,YAAY,EAAe,MAAM,WAAW,CAAC;AACtD,OAAO,EAAE,UAAU,EAAuB,MAAM,aAAa,CAAC;AAO9D,OAAO,EAAE,EAAE,EAAE,UAAU,EAAQ,MAAM,YAAY,CAAC;AAUlD,OAAO,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAwB,MAAM,aAAa,CAAC;AAC9E,OAAO,EAGL,SAAS,EACT,UAAU,EACV,4BAA4B,EAE7B,MAAM,iBAAiB,CAAC;AAiBzB,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,WAAW,EAAE,CAAC,KAAK,EAAE,aAAa,KAAK,IAAI,CAAC;IACrD,SAAS,EAAE,CAAC,MAAM,EAAE,cAAc,KAAK,IAAI,CAAC;CAC7C;AAED,eAAO,MAAM,QAAQ,iCAA0C,CAAC;AAEhE,MAAM,MAAM,aAAa,GACrB,iBAAiB,GACjB,kBAAkB,GAClB,mBAAmB,GACnB,iBAAiB,GACjB,kBAAkB,GAClB,yBAAyB,GACzB,4BAA4B,CAAC;AAEjC,UAAU,iBAAiB;IACzB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAED,UAAU,kBAAkB;IAC1B,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,aAAa,CAAC,qBAAqB,CAAC,KAAK,CAAC,CAAC;CAC9D;AAED,UAAU,mBAAmB;IAC3B,QAAQ,CAAC,IAAI,EAAE,QAAQ,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC,qBAAqB,CAAC,UAAU,CAAC,CAAC;IAChE,QAAQ,CAAC,OAAO,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC;CACxC;AAED,UAAU,iBAAiB;IACzB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,OAAO,EAAE,aAAa,CAAC,KAAK,CAAC,CAAC;CACxC;AAED,UAAU,kBAAkB;IAC1B,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB,QAAQ,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC;CAC9B;AAED,UAAU,yBAAyB;IACjC,QAAQ,CAAC,IAAI,EAAE,cAAc,CAAC;IAC9B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAQD,MAAM,MAAM,cAAc,GACtB,qBAAqB,GACrB,qBAAqB,GACrB,qBAAqB,GACrB,uBAAuB,GACvB,8BAA8B,GAC9B,yBAAyB,CAAC;AAE9B,UAAU,qBAAqB;IAC7B,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,KAAK,EAAE,UAAU,CAAC;CAC5B;AAED,UAAU,qBAAqB;IAC7B,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAC;CACvB;AAED,MAAM,WAAW,qBAAqB;IACpC,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IACzB,QAAQ,CAAC,cAAc,EAAE,aAAa,CAAC,YAAY,CAAC,CAAC;IACrD,QAAQ,CAAC,aAAa,EAAE,aAAa,CAAC,YAAY,CAAC,CAAC;CACrD;AAED,MAAM,MAAM,YAAY,GAAG,MAAM,GAC/B,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,GACjB,KAAK,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;AAE5B,UAAU,uBAAuB;IAC/B,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;CAC5B;AAED,UAAU,8BAA8B;IACtC,QAAQ,CAAC,IAAI,EAAE,kBAAkB,CAAC;CACnC;AAED,UAAU,yBAAyB;IACjC,QAAQ,CAAC,IAAI,EAAE,aAAa,CAAC;IAC7B,QAAQ,CAAC,KAAK,EAAE,SAAS,CAAC;CAC3B;AAED,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,EAAE,EAAE,EAAE,CAAC;IAChB,QAAQ,CAAC,MAAM,EAAE,cAAc,CAAC,cAAc,CAC5C,KAAK,GAAG,IAAI,GAAG,OAAO,GAAG,SAAS,CACnC,CAAC;IACF,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC;IAC3B,QAAQ,CAAC,GAAG,EAAE,UAAU,CAAC;IACzB,QAAQ,CAAC,YAAY,EAAE,YAAY,GAAG,IAAI,CAAC;CAC5C;AAwBD,MAAM,MAAM,YAAY,GAAG,WAAW,CAAC,KAAK,EAAE,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC;AAmXlE,eAAO,MAAM,YAAY,oEAkHxB,CAAC"}