defuss-db 1.1.4 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -3,150 +3,262 @@
3
3
  <img src="assets/defuss_mascott.png" width="100px" />
4
4
 
5
5
  <p align="center">
6
- <code>defuss</code>
6
+ <code>defuss-db</code>
7
7
  </p>
8
8
 
9
9
  <sup align="center">
10
10
 
11
- Database Abstraction
11
+ Isomorphic Multi-Backend Database Abstraction for Web and Node.js
12
12
 
13
13
  </sup>
14
14
 
15
15
  </h1>
16
16
 
17
+ > `defuss-db` is a schema-driven database abstraction for `defuss`. You declare a table once, define the fields you want indexed, and then use one CRUD API across all supported providers.
17
18
 
18
- > `defuss-db` abstracts interactions with databases, allowing you to focus on your application logic without worrying about the underlying database implementation, schema, or queries. It provides a consistent API for various databases, making it easier to switch between them or use multiple databases in a single application.
19
+ ## Supported Providers
19
20
 
21
+ - `DexieProvider` for browser IndexedDB.
22
+ - `LibsqlProvider` for SQLite/libSQL.
23
+ - `MongoProvider` for MongoDB.
24
+ - `JsonlProvider` for simple file-backed server-side storage.
20
25
 
21
- <h3 align="center">
26
+ ## Core Idea
22
27
 
23
- Supported Databases
28
+ Declare a table once and then query real value fields.
24
29
 
25
- </h3>
26
-
27
- - *Dexie*: A wrapper for IndexedDB that provides a simple and powerful API for working with client-side databases in web applications. It is designed to be easy to use and provides a consistent API across different browsers.
28
-
29
- - *MongoDB*: A NoSQL database that uses a flexible, JSON-like format for data storage. It is designed for scalability and performance, making it a popular choice for modern web applications.
30
-
31
- - *LibSQL*: A SQL database that is designed to be lightweight and easy to use. It is built on top of SQLite and provides a simple API for interacting with SQL databases.
30
+ ```ts
31
+ import { DefussTable, defineTable, type DefussRecord } from "defuss-db";
32
+ import { DexieProvider } from "defuss-db/client.js";
32
33
 
33
- Providing an isomorphic API for all three databases, `defuss-db` allows you to use the same codebase for both client-side and server-side database interactions. This means you can write your database queries once and use them in both environments without worrying about compatibility issues.
34
+ interface User extends DefussRecord {
35
+ name: string;
36
+ email: string;
37
+ age: number;
38
+ profile: {
39
+ city: string;
40
+ };
41
+ }
34
42
 
35
- <h3 align="center">
43
+ const userTable = defineTable<User>({
44
+ name: "users",
45
+ indexes: [
46
+ {
47
+ name: "email",
48
+ source: "email",
49
+ unique: true,
50
+ },
51
+ {
52
+ name: "city",
53
+ source: "profile.city",
54
+ },
55
+ ],
56
+ });
57
+
58
+ const provider = new DexieProvider("AppDatabase");
59
+ await provider.connect();
60
+
61
+ const users = new DefussTable(provider, userTable);
62
+ await users.init();
63
+ ```
36
64
 
37
- Integrating `defuss-db` in an existing defuss project
65
+ ## CRUD API
38
66
 
39
- </h3>
67
+ ```ts
68
+ const id = await users.insert({
69
+ name: "Alice",
70
+ email: "alice@example.com",
71
+ age: 30,
72
+ profile: {
73
+ city: "Berlin",
74
+ },
75
+ });
76
+
77
+ const byEmail = await users.findOne({ email: "alice@example.com" });
78
+ const byCity = await users.find({ "profile.city": "Berlin" });
79
+
80
+ await users.update({ id }, { age: 31 });
81
+ await users.delete({ id });
82
+
83
+ const upsertedId = await users.upsert(
84
+ { email: "alice@example.com" },
85
+ {
86
+ name: "Alice",
87
+ email: "alice@example.com",
88
+ age: 32,
89
+ profile: {
90
+ city: "Berlin",
91
+ },
92
+ },
93
+ );
94
+ ```
40
95
 
41
- **🚀 Looking for a template to start from?** `examples/notebooks` is an Astro project pre-configured to work with `defuss-db` out-of-the-box.
96
+ ## Aggregation API
42
97
 
43
- #### 2. Integrate `defuss-db`:
98
+ For read-heavy flows that need joins, projections, grouping, or sorting, use the provider-agnostic aggregation builder. It sits above `find()` so providers stay small while higher-level read pipelines remain portable.
44
99
 
45
- Just import `Table` from `defuss-db` and use it in your project:
100
+ You can start from a table:
46
101
 
47
102
  ```ts
48
- import { Table } from 'defuss-db';
103
+ const result = await users
104
+ .aggregate()
105
+ .project({
106
+ userId: "base.id",
107
+ email: "base.email",
108
+ })
109
+ .execute();
49
110
  ```
50
111
 
51
- A table needs a database storage provider:
52
-
53
- In-browser (client-side/frontend) use-case:
112
+ Or compose multiple sources directly:
54
113
 
55
114
  ```ts
56
- import { DexieProvider } from "defuss-db";
115
+ import {
116
+ avgBy,
117
+ countRows,
118
+ createAggregation,
119
+ sumBy,
120
+ } from "defuss-db";
121
+
122
+ const summary = await createAggregation({ table: orders, as: "orders" })
123
+ .join(
124
+ { table: users, as: "users" },
125
+ { type: "left", left: "orders.customerId", right: "id" },
126
+ )
127
+ .alias({
128
+ customerName: "users.name",
129
+ })
130
+ .groupBy(
131
+ {
132
+ customerId: "orders.customerId",
133
+ customerName: "customerName",
134
+ },
135
+ {
136
+ orderCount: countRows(),
137
+ revenue: sumBy("orders.total"),
138
+ averageOrderValue: avgBy("orders.total"),
139
+ },
140
+ )
141
+ .sortBy({ field: "revenue", direction: "desc" })
142
+ .execute();
57
143
  ```
58
144
 
59
- Using the database is as easy as wiring them up:
60
-
61
- ```ts
62
- // define a model
63
- interface User {
64
- name: string;
65
- age: number;
66
- email: string;
67
- }
145
+ Available pipeline steps:
68
146
 
69
- const dexieProvider = new DexieProvider(TEST_DB_NAME);
70
- await dexieProvider.connect(); // connects to the database
147
+ - `join()` supports `left`, `right`, and `inner` joins over table-backed or array-backed sources.
148
+ - `project()` reshapes rows into a new object.
149
+ - `alias()` adds derived fields while keeping the existing row.
150
+ - `compute()` and `computeMany()` append derived fields.
151
+ - `mapRows()` applies a custom row mapper.
152
+ - `removeFields()` removes one or more nested paths from each row.
153
+ - `mergeConsecutive()` merges adjacent rows after an explicit sort.
154
+ - `distinctBy()` keeps the first or last row for a computed key.
155
+ - `groupBy()` returns one row per group key with reducer outputs.
156
+ - `sortBy()` accepts field specs, resolver specs, or a custom comparator.
71
157
 
72
- const myTable = new DefussTable<User>(provider, "my-table");
73
- await myTable.init(); // runs a schema update
74
- ```
158
+ Reducer helpers exported from `defuss-db`:
75
159
 
160
+ - `countRows()`
161
+ - `sumBy()`
162
+ - `avgBy()`
163
+ - `minBy()`
164
+ - `maxBy()`
165
+ - `firstBy()`
166
+ - `lastBy()`
76
167
 
77
- #### 3. Use the API of `defuss-db`:
168
+ Notes:
78
169
 
79
- To interact with the table, create, read, update, and delete records:
170
+ - The base table alias defaults to `base`, or you can override it with `aggregate({ as: "users" })`.
171
+ - Source-level `where` selectors run before aggregation, so joins and grouping operate on filtered provider results.
172
+ - Join `left` paths are resolved against the current aggregated row, while `right` paths are resolved against the joined source row.
80
173
 
81
- ```ts
82
- const user: User = {
83
- name: "Alice",
84
- age: 30,
85
- email: "alice@example.com",
86
- };
174
+ ## Selector Rules
87
175
 
88
- // in defuss-db, indices are added explicitly, this allows for indices to diverge from the underlying from the data stored. This separation solves a common issue with performance in speed and size dimensions.
89
- const indexData = { email: user.email }; // data to find the user by later-on
176
+ - `find()`, `findOne()`, `update()`, and `delete()` accept selectors over real stored fields.
177
+ - If a selector field is indexed, the provider uses that index.
178
+ - If a selector field is not indexed, providers may fall back to a scan.
179
+ - `upsert()` is stricter: the selector must be non-empty and must target `id` or a declared unique index.
90
180
 
91
- // returns the primary key of the inserted user
92
- const pk = await table.insert(/* data */ user, /* indices */ indexData);
181
+ ## `id` Semantics
93
182
 
94
- // find a specific user
95
- const user = await table.findOne(/* indices */ { email: "alice@example.com" });
183
+ - `id` is always the public primary key.
184
+ - `id` is always implicitly indexed.
185
+ - MongoDB maps `id` to `_id`.
186
+ - LibSQL stores `id` as the primary key column.
187
+ - JSONL stores `id` as the row identity in memory and on disk.
188
+ - Dexie uses `id` as the table primary key field.
96
189
 
97
- // find many users of a specific TLD
98
- const users = await table.find(/* indices */ { email: "*@example.com" });
190
+ ## Nested And Derived Indexes
99
191
 
100
- // updates the user
101
- await table.update(/* update indices */ { email: user.email }, /* paertial data */ { age: 29 });
192
+ Use a path for nested values:
102
193
 
103
- // deletes the user
104
- await table.delete(/* indices */ { email: user.email });
194
+ ```ts
195
+ const table = defineTable({
196
+ name: "profiles",
197
+ indexes: [
198
+ { name: "city", source: "profile.city" },
199
+ ],
200
+ });
105
201
  ```
106
202
 
107
- Due to the nature of `defuss-db`, bridging the gap beween different database architectures and even between frontend and backend, the API cannot feature all the specific functionality of each underlaying provider. It is designed to simply provide the intersection feature-set. There are no foreign keys, no transactions, no joins, and no complex queries. You are supposed to design your data model in a semi-normalized way and use smart indices and data access patterns to get the most out of this.
203
+ Use a mapping function for derived indexes:
108
204
 
109
- <h3 align="center">
205
+ ```ts
206
+ const table = defineTable({
207
+ name: "users",
208
+ indexes: [
209
+ {
210
+ name: "emailDomain",
211
+ source: (value) => value.email.split("@")[1] ?? null,
212
+ },
213
+ ],
214
+ });
215
+ ```
110
216
 
111
- 🚀 How does `defuss-db` work?
217
+ ## JSONL Provider
112
218
 
113
- </h3>
219
+ The JSONL provider is exported from the server entry only.
114
220
 
115
- Inside this package, you'll find the following relevant folders and files:
221
+ ```ts
222
+ import { DefussTable, defineTable } from "defuss-db";
223
+ import { JsonlProvider } from "defuss-db/server.js";
116
224
 
117
- ```text
118
- /
119
- ├── provider/dexie.ts
120
- ├── provider/libsql.ts
121
- ├── provider/mongodb.ts
122
- ├── src/index.ts
123
- ├── src/types.ts
124
- ├── src/env.ts
125
- ├── src/table.ts
126
- ├── src/client.ts
127
- ├── src/server.ts
128
- ├── tsconfig.json
129
- ├── LICENSE
130
- ├── package.json
225
+ const provider = new JsonlProvider();
226
+ await provider.connect({
227
+ baseDir: "./.data",
228
+ });
131
229
  ```
132
230
 
133
- The `src/index.ts` file is the "entry point" for general and type imports, while `server.ts` and `client.ts` are the respective imports to use whenever a specific runtime environment implement is desired to be used. The `defuss-db/client` and `defuss-db/server` imports will resolve to these files automatically.
231
+ Behavior:
232
+
233
+ - One `.jsonl` file per table.
234
+ - Insert appends a new line.
235
+ - Update and delete rewrite the table file.
236
+ - All rows are loaded into memory on connect.
237
+
238
+ This provider is intended for prototyping, debugging, fixtures, and small local datasets.
134
239
 
135
- You'll find each specific provider implementation in the `provider` folder.
240
+ ## Runtime Entrypoints
136
241
 
137
- ## 🧞 Commands
242
+ - `defuss-db` exports shared types, `defineTable()`, `DefussTable`, `createAggregation()`, and aggregation reducer helpers.
243
+ - `defuss-db/client.js` exports `DexieProvider`.
244
+ - `defuss-db/server.js` exports `LibsqlProvider`, `MongoProvider`, and `JsonlProvider`.
138
245
 
139
- All commands are run from the root of the project, from a terminal:
246
+ ## Commands
140
247
 
141
- | Command | Action |
142
- | :------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
143
- | `npm build` | Build a new version of the integration. |
144
- | `npm mongodb:start` | Start the MongoDB server (requires Docker). |
145
- | `npm mongodb:stop` | Stop the MongoDB server (requires Docker). |
146
- | `npm test:integration` | Run the integration tests for the `defuss-db` package. |
248
+ All commands below are run from `packages/db`.
147
249
 
148
- ---
250
+ | Command | Action |
251
+ | :-- | :-- |
252
+ | `bun run build` | Build the package |
253
+ | `bun run example:dexie` | Run the Dexie / IndexedDB walkthrough |
254
+ | `bun run example:libsql` | Run the LibSQL walkthrough |
255
+ | `bun run example:jsonl` | Run the JSONL walkthrough |
256
+ | `bun run example:mongodb` | Run the MongoDB walkthrough |
257
+ | `bun run example:all` | Run all provider walkthroughs sequentially |
258
+ | `bun run mongodb:start` | Start the MongoDB test container |
259
+ | `bun run mongodb:stop` | Stop the MongoDB test container |
260
+ | `bun run test` | Runs the package test suite including all examples |
149
261
 
150
- <img src="https://raw.githubusercontent.com/kyr0/defuss/refs/heads/main/assets/defuss_comic.png" />
262
+ ## Architectural Decisions
151
263
 
152
- <caption><i><b>Come visit us on defuss island!</b></i></caption>
264
+ `defuss-db` intentionally keeps the provider contract small. CRUD, declared indexes, selectors over stored values, and predictable cross-provider behavior remain the core. Transactions, foreign keys, and provider-specific query DSLs are out of scope. Portable joins, grouping, sorting, and reshaping live in the separate aggregation builder above `find()` instead of expanding provider complexity.