datalevin-node 1.1.0 → 1.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,6 +2,8 @@
2
2
 
3
3
  Node.js bindings for Datalevin over the JVM interop bridge.
4
4
 
5
+ Visit [datalevin.org](https://datalevin.org) for Datalevin documentation.
6
+
5
7
  ## Install
6
8
 
7
9
  ```bash
@@ -69,6 +71,74 @@ identity comparison; the immutable snapshot, rather than structural `===`, is th
69
71
  composition contract. Non-structural objects such as backend handles are
70
72
  treated as atomic values.
71
73
 
74
+ ## Prepared reads and acknowledgement-only writes
75
+
76
+ `Connection` and `Database` expose `preparePull(selector, opts = null)` and
77
+ `prepareQuery(query)`. `KV` and `KVTransaction` expose
78
+ `prepareGetValue(dbiName, { kType, vType, ignoreKey = true })`; omitted types
79
+ default to `data`. The same names are exported as top-level helpers with the
80
+ connection, database, or KV handle as their first argument.
81
+
82
+ ```js
83
+ const readPerson = await conn.preparePull([":name", ":age"]);
84
+ const person = await readPerson.execute([":name", "Ada"]);
85
+ const lookup = await conn.prepareQuery(
86
+ "[:find ?age . :in $ ?name :where [?e :name ?name] [?e :age ?age]]"
87
+ );
88
+ const age = await lookup.execute(["Ada"]);
89
+ ```
90
+
91
+ Prepared queries accept EDN or typed `q` forms. Execute with an array of the
92
+ remaining `:in` values, excluding the database, or `[]` when there are none.
93
+ Results and input conversion match ordinary queries, including dynamic
94
+ `:limit` and `:offset` bindings. `executePrepared(prepared, input)` is an
95
+ equivalent top-level helper.
96
+
97
+ A local prepared pull or KV read accepts `execute(view, input)`, also exposed
98
+ as `executePrepared(prepared, view, input)`, to read an explicit view. Prepared
99
+ queries and remote preparations do not accept explicit views. Use a KV
100
+ transaction's `prepareGetValue` for transaction-bound reads. Keep the owning
101
+ store open; prepared reads require no separate close. Transaction-bound reads
102
+ retain the transaction's lifetime and thread restrictions.
103
+
104
+ `conn.transactAck(txData, txMeta = null)` and `transactAck(conn, ...)` return
105
+ `":transacted"` on success. Validation and durability match `transact`, and
106
+ listeners still receive full reports.
107
+
108
+ Use `schemaAttr({ noIndex: true })` for `:db/noindex`. Reads by entity and pull
109
+ remain available; query conditions on that attribute require
110
+ `await conn.indexAttr(attr)` (or `indexAttr(conn, attr)`) to atomically backfill AVE.
111
+
112
+ ## Atomic Updates and Database Notifications
113
+
114
+ `kv.update(dbiName, key, fn, { kType, vType, args })` and
115
+ `updateKv(kv, ...)` atomically replace one value and return `":transacted"`.
116
+ Types default to `":data"`. Embedded callbacks receive the old value (null for
117
+ a missing key), then any extra arguments. Use restricted Clojure function source
118
+ for remote updates:
119
+
120
+ ```js
121
+ await kv.update("counts", "visits", old => Number(old ?? 0) + 1);
122
+ await remoteKv.update("counts", "visits",
123
+ "(fn [old amount] (+ (long (or old 0)) (long amount)))",
124
+ { kType: ":string", vType: ":long", args: [1] });
125
+ ```
126
+
127
+ JVM long values arrive as `BigInt` in callbacks; use `BigInt` arithmetic or
128
+ convert small counters with `Number`, as in the example above.
129
+
130
+ Functions must be free of side effects because updates may retry. Invalid values
131
+ and callback exceptions abort the update; explicit transactions control commit.
132
+
133
+ `conn.listenDb(callback, key = null)` and `listenDb(conn, ...)` subscribe to
134
+ committed changes to a remote database, including writes from other clients.
135
+ Callbacks receive `{ ":type": ":db-changed", ":db-name": "app" }` and may await
136
+ queries on the connection to refresh data. Several commits may coalesce into one
137
+ event. Terminal failures deliver a `":subscription-error"` event with `":error"`.
138
+ Registering the same key replaces the callback. Call `conn.unlistenDb(key)` or
139
+ `unlistenDb(conn, key)` to stop a subscription; closing the connection stops all
140
+ its subscriptions. `conn.listen` continues to deliver transaction reports.
141
+
72
142
  ## Composing Queries
73
143
 
74
144
  Use normal JavaScript control flow to assemble clauses. Variables, attributes,
@@ -480,6 +550,107 @@ used explicitly. Typed descriptors are interned by normalized language, kind,
480
550
  id, and version, so equivalent descriptors also behave as one key in a
481
551
  JavaScript `Map`.
482
552
 
553
+ ## Ordered Custom Types
554
+
555
+ Register a database-wide type through `KV.registerType()` or
556
+ `Connection.registerType()`. Its order function returns an existing backing
557
+ type. Optional payload serializers return `Buffer` or `Uint8Array`, and
558
+ deserializers receive a `Buffer`. Nippy is the default payload codec.
559
+
560
+ ```javascript
561
+ import { UdfDescriptor, createUdfRegistry, openKv } from "datalevin-node";
562
+
563
+ const registry = await createUdfRegistry();
564
+ await registry.orderUdf("task/order", task => task.rank);
565
+ await registry.serializerUdf("task/encode", task =>
566
+ Buffer.from(JSON.stringify(task, (_key, value) =>
567
+ typeof value === "bigint" ? Number(value) : value)));
568
+ await registry.deserializerUdf("task/decode", payload => {
569
+ const task = JSON.parse(payload.toString("utf8"));
570
+ return { ...task, rank: BigInt(task.rank) };
571
+ });
572
+ const definition = {
573
+ index: { type: ":long", "order-fn": UdfDescriptor.orderFn("task/order") },
574
+ payload: {
575
+ serialize: UdfDescriptor.serializer("task/encode"),
576
+ deserialize: UdfDescriptor.deserializer("task/decode")
577
+ }
578
+ };
579
+ const kv = await openKv("/tmp/datalevin-custom-example", {
580
+ ":runtime-opts": { ":udf-registry": registry }
581
+ });
582
+ try {
583
+ await kv.registerType("app/task", definition);
584
+ await kv.openDbi("tasks", { ":key-type": ":app/task" });
585
+ const task = { rank: 2n, name: "Compile" };
586
+ await kv.transact([[":put", task, "queued"]], { dbiName: "tasks" });
587
+ console.log(await kv.getValue("tasks", task)); // queued
588
+ } finally {
589
+ await kv.close();
590
+ }
591
+ ```
592
+
593
+ For Datalog, register through the connection and set an attribute's
594
+ `:db/valueType` to `:app/task`. Ordered list items use the DBI option
595
+ `{ ":value-type": ":app/task" }`. Registrations persist and are shared by both
596
+ APIs. Supply runtime bindings again when reopening. Remote stores execute
597
+ functions using bindings installed on the server.
598
+
599
+ For native JavaScript classes, bind the class before opening the local
600
+ environment. Ordinary KV and Datalog operations then accept and return instances:
601
+
602
+ ```javascript
603
+ import { UdfDescriptor, createUdfRegistry, openKv } from "datalevin-node";
604
+
605
+ class Task {
606
+ constructor(rank, label) { this.rank = rank; this.label = label; }
607
+ }
608
+
609
+ const registry = await createUdfRegistry();
610
+ await registry.orderUdf("native/order", task => task.rank);
611
+ await registry.serializerUdf("native/encode", task =>
612
+ Buffer.from(JSON.stringify([task.rank.toString(), task.label])));
613
+ await registry.deserializerUdf("native/decode", payload => {
614
+ const [rank, label] = JSON.parse(payload.toString("utf8"));
615
+ return new Task(BigInt(rank), label);
616
+ });
617
+ const definition = {
618
+ index: { type: ":long", "order-fn": UdfDescriptor.orderFn("native/order") },
619
+ payload: { serialize: UdfDescriptor.serializer("native/encode"),
620
+ deserialize: UdfDescriptor.deserializer("native/decode") }
621
+ };
622
+ await registry.bindNativeType("app/task", Task, definition);
623
+ const kv = await openKv("/tmp/datalevin-js-native", {
624
+ ":runtime-opts": { ":udf-registry": registry }
625
+ });
626
+ try {
627
+ await kv.registerType("app/task", definition);
628
+ await kv.openDbi("tasks", { ":key-type": ":app/task" });
629
+ await kv.transact([[":put", new Task(1n, "Compile"), "queued"]], { dbiName: "tasks" });
630
+ console.log(await kv.getValue("tasks", new Task(1n, "Compile"))); // queued
631
+ console.log(await kv.getRange("tasks", [":all"])); // [[Task { ... }, "queued"]]
632
+ } finally {
633
+ await kv.close();
634
+ }
635
+ ```
636
+
637
+ Recreate the runtime class binding and UDF implementations on reopen. Serde
638
+ functions may be async. Exact matching defaults to `node:util.isDeepStrictEqual`
639
+ on decoded values, so equal instances may serialize to different bytes. For
640
+ classes with private fields, or a different value-equality rule, pass
641
+ `{ equals: (left, right) => /* boolean */ }` as the fourth argument to
642
+ `bindNativeType`. Equality may also be async and must remain stable and symmetric.
643
+
644
+ Native support covers local and remote operations, including query spilling to
645
+ disk. Remote handles use the caller registry in `:runtime-opts`; install matching
646
+ order/serde UDFs on the server through `datalevin.server/*server-runtime-opts-fn*`.
647
+ Only type names and payload bytes cross the network. The caller's deserializer
648
+ and equality function reconstruct returned values, including asynchronous hooks.
649
+ Untyped `:data` and eager helpers without a database handle remain unsupported.
650
+ Native values currently share a constant JVM hash,
651
+ so large hash joins and deduplication can be expensive. See the
652
+ [custom data guide](../../doc/custom-data.md#native-javascript-values).
653
+
483
654
  ## Fulltext Analyzer UDF Example
484
655
 
485
656
  Use analyzer UDFs when a Datalog fulltext domain needs host-language tokenizing.
@@ -592,6 +763,12 @@ Use `initDb()` and `fillDb()` when you already have Datom-shaped data and want
592
763
  the fast bulk-load path. Datoms can be compact arrays in
593
764
  `[entityId, attr, value]` shape, and `datom()` creates the same shape.
594
765
 
766
+ For a server database, pass its authenticated `dtlv://` URI as `dir` instead of
767
+ a local path. Both `fillDb(conn, datoms)` and `conn.fillDb(datoms)` support remote
768
+ connections. Remote loads preserve server WAL settings and commit in batches;
769
+ outside an explicit transaction, earlier batches remain committed if a later
770
+ batch fails. Use assigned entity IDs and trusted data matching the schema.
771
+
595
772
  ```js
596
773
  import { datom, fillDb, initDb } from "datalevin-node";
597
774
 
@@ -0,0 +1,4 @@
1
+ [diffend] Oversized file quarantined before diffing.
2
+ name: package/jars/datalevin-runtime-1.2.10.jar
3
+ size: 41127521 bytes
4
+ sha256: 61e97d4cd17f42f8e53332cbbdb2ced56ea91c895d7fceb858debd4c8fea3b15
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "datalevin-node",
3
- "version": "1.1.0",
3
+ "version": "1.2.10",
4
4
  "description": "Node.js bindings for Datalevin over the JVM interop bridge",
5
5
  "type": "module",
6
6
  "repository": {
package/src/connection.js CHANGED
@@ -1,3 +1,5 @@
1
+ import { randomUUID } from "node:crypto";
2
+
1
3
  import {
2
4
  toEdnForm,
3
5
  toJava,
@@ -7,10 +9,12 @@ import {
7
9
  } from "./convert.js";
8
10
  import { txReportToJs } from "./database.js";
9
11
  import { Form } from "./form.js";
12
+ import { preparePull, prepareQuery } from "./prepared.js";
10
13
  import { _BINDINGS } from "./interop.js";
11
14
  import { callJavaMethod, javaBridgeModule } from "./jvm.js";
12
15
  import { ResourceWrapper } from "./resource.js";
13
16
  import { toJsResult } from "./result.js";
17
+ import { bindNativeCallback, nativeMethods } from "./native.js";
14
18
 
15
19
  async function pullSelector(value) {
16
20
  if (typeof value === "string") {
@@ -67,37 +71,24 @@ function fulltextOpts(opts = null, { limit = null, offset = 0, pagingCachePages
67
71
  return (mapOpts ? merged.size === 0 : Object.keys(merged).length === 0) ? null : merged;
68
72
  }
69
73
 
70
- let listenerProxyEventLoopDepth = 0;
71
- let listenerProxyEventLoopPrevious = false;
72
-
73
- async function retainListenerProxyEventLoop() {
74
- const bridge = await javaBridgeModule();
75
- if (listenerProxyEventLoopDepth === 0) {
76
- listenerProxyEventLoopPrevious = bridge.config.runEventLoopWhenInterfaceProxyIsActive;
77
- bridge.config.runEventLoopWhenInterfaceProxyIsActive = true;
78
- }
79
- listenerProxyEventLoopDepth += 1;
80
- return () => {
81
- listenerProxyEventLoopDepth -= 1;
82
- if (listenerProxyEventLoopDepth === 0) {
83
- bridge.config.runEventLoopWhenInterfaceProxyIsActive = listenerProxyEventLoopPrevious;
84
- }
85
- };
86
- }
87
-
88
74
  async function createConsumerProxy(fn) {
89
75
  if (typeof fn !== "function") {
90
76
  throw new TypeError("callback must be a function.");
91
77
  }
92
78
  const { newProxy } = await javaBridgeModule();
93
79
  return newProxy("java.util.function.Consumer", {
94
- accept: async (value) => {
80
+ accept: bindNativeCallback(async (value) => {
95
81
  await fn(await toJs(value));
96
- }
82
+ })
97
83
  });
98
84
  }
99
85
 
100
86
  export class Connection extends ResourceWrapper {
87
+ /** Register a database-wide type for use as an attribute's :db/valueType. */
88
+ async registerType(typeName, definition) {
89
+ return toJs(await _BINDINGS.registerType(this.rawHandle(), typeName, definition));
90
+ }
91
+
101
92
  constructor(handle, { owned = true } = {}) {
102
93
  super(
103
94
  handle,
@@ -106,25 +97,43 @@ export class Connection extends ResourceWrapper {
106
97
  "connection"
107
98
  );
108
99
  this._listenerProxies = new Map();
100
+ this._dbListenerProxies = new Map();
109
101
  }
110
102
 
111
103
  async close() {
112
- this._resetListenerProxies();
113
104
  await super.close();
105
+ this._resetListenerProxies();
114
106
  }
115
107
 
116
108
  _resetListenerProxies() {
117
- for (const { proxy, release } of this._listenerProxies.values()) {
118
- proxy.reset?.();
119
- release?.();
109
+ for (const listeners of [this._listenerProxies, this._dbListenerProxies]) {
110
+ for (const { proxy } of listeners.values()) {
111
+ proxy.reset?.();
112
+ }
113
+ listeners.clear();
120
114
  }
121
- this._listenerProxies.clear();
122
115
  }
123
116
 
124
117
  async schema() {
125
118
  return toJsResult(await _BINDINGS.coreInvoke("schema", [this.rawHandle()]));
126
119
  }
127
120
 
121
+ async indexAttr(attr) {
122
+ return toJsResult(await _BINDINGS.connectionIndexAttr(this.rawHandle(), attr), { bridge: true });
123
+ }
124
+
125
+ async preparePull(selector, opts = null) {
126
+ return preparePull(this, selector, opts);
127
+ }
128
+
129
+ async prepareQuery(query) {
130
+ return prepareQuery(this, query);
131
+ }
132
+
133
+ async transactAck(txData, txMeta = null) {
134
+ return toJsResult(await _BINDINGS.connectionTransactAck(this.rawHandle(), txData, txMeta));
135
+ }
136
+
128
137
  async opts() {
129
138
  return toJsResult(await _BINDINGS.coreInvoke("opts", [this.rawHandle()]));
130
139
  }
@@ -299,11 +308,9 @@ export class Connection extends ResourceWrapper {
299
308
  }
300
309
  const previous = this._listenerProxies.get(registeredKey);
301
310
  previous?.proxy.reset?.();
302
- previous?.release?.();
303
- this._listenerProxies.set(registeredKey, {
304
- proxy,
305
- release: await retainListenerProxyEventLoop()
306
- });
311
+ // Transactions use async Java calls, so callbacks need no reentrant event
312
+ // loop. Enabling java-bridge's experimental loop can race proxy cleanup.
313
+ this._listenerProxies.set(registeredKey, { proxy });
307
314
  return registeredKey;
308
315
  }
309
316
 
@@ -311,10 +318,34 @@ export class Connection extends ResourceWrapper {
311
318
  await _BINDINGS.connectionUnlisten(this.rawHandle(), key);
312
319
  const existing = this._listenerProxies.get(key);
313
320
  existing?.proxy.reset?.();
314
- existing?.release?.();
315
321
  this._listenerProxies.delete(key);
316
322
  }
317
323
 
324
+ /** Subscribe to remote database commits; events may coalesce and callbacks may query. */
325
+ async listenDb(callback, key = null) {
326
+ const proxy = await createConsumerProxy(callback);
327
+ let registeredKey;
328
+ try {
329
+ // Generate string keys here so unlistenDb receives the same type as registration.
330
+ registeredKey = await toJsResult(
331
+ await _BINDINGS.connectionListenDb(this.rawHandle(), key ?? randomUUID(), proxy),
332
+ { bridge: true }
333
+ );
334
+ } catch (error) {
335
+ proxy.reset?.();
336
+ throw error;
337
+ }
338
+ this._dbListenerProxies.get(registeredKey)?.proxy.reset?.();
339
+ this._dbListenerProxies.set(registeredKey, { proxy });
340
+ return registeredKey;
341
+ }
342
+
343
+ async unlistenDb(key) {
344
+ await _BINDINGS.connectionUnlistenDb(this.rawHandle(), key);
345
+ this._dbListenerProxies.get(key)?.proxy.reset?.();
346
+ this._dbListenerProxies.delete(key);
347
+ }
348
+
318
349
  async fillDb(datoms) {
319
350
  await callJavaMethod(this.rawHandle(), "fillDb", await toJava(datoms));
320
351
  return this;
@@ -457,3 +488,5 @@ export class Connection extends ResourceWrapper {
457
488
  return toJsResult(await callJavaMethod(this.rawHandle(), "gcTxLogSegments"), { bridge: true });
458
489
  }
459
490
  }
491
+
492
+ nativeMethods(Connection);
package/src/convert.js CHANGED
@@ -1,5 +1,10 @@
1
1
  import { classes } from "./jvm.js";
2
2
  import { DatalogSymbol, EdnList, Form, Keyword, Uuid } from "./form.js";
3
+ import { currentNativeBindings, nativeCodecFor, nativeToJs } from "./native.js";
4
+
5
+ function nativeBindingsActive() {
6
+ return Boolean(currentNativeBindings()?.types.size);
7
+ }
3
8
 
4
9
  const INT64_MIN = -(2n ** 63n);
5
10
  const INT64_MAX = 2n ** 63n - 1n;
@@ -24,7 +29,15 @@ function toBufferView(value) {
24
29
  return Buffer.from(value.buffer, value.byteOffset, value.byteLength);
25
30
  }
26
31
 
27
- function materializeJavaCollection(collection) {
32
+ async function materializeJavaCollection(collection) {
33
+ if (nativeBindingsActive()) {
34
+ // Reading spilled collections can reconstruct maps/sets and call native
35
+ // equality. Leave the event loop available for those JVM callbacks.
36
+ const items = [];
37
+ const iterator = await collection.iterator();
38
+ while (await iterator.hasNext()) items.push(await iterator.next());
39
+ return items;
40
+ }
28
41
  if (typeof collection?.toArraySync === "function") {
29
42
  try {
30
43
  const items = collection.toArraySync();
@@ -58,7 +71,9 @@ async function toJavaMap(entries, converter) {
58
71
  const cls = await classes();
59
72
  const jmap = new cls.linkedHashMap();
60
73
  for (const [key, item] of entries) {
61
- jmap.putSync(await converter(key), await converter(item));
74
+ const k = await converter(key), v = await converter(item);
75
+ if (nativeBindingsActive()) await jmap.put(k, v);
76
+ else jmap.putSync(k, v);
62
77
  }
63
78
  return jmap;
64
79
  }
@@ -67,7 +82,9 @@ async function toJavaSet(items, converter) {
67
82
  const cls = await classes();
68
83
  const jset = new cls.linkedHashSet();
69
84
  for (const item of items) {
70
- jset.addSync(await converter(item));
85
+ const value = await converter(item);
86
+ if (nativeBindingsActive()) await jset.add(value);
87
+ else jset.addSync(value);
71
88
  }
72
89
  return jset;
73
90
  }
@@ -88,6 +105,8 @@ export function isJavaObject(value) {
88
105
  }
89
106
 
90
107
  export async function toJava(value) {
108
+ const native = nativeCodecFor(value);
109
+ if (native) return native.wrap(value);
91
110
  if (value === null || typeof value === "boolean" || typeof value === "string") {
92
111
  return value;
93
112
  }
@@ -136,7 +155,7 @@ export async function toJava(value) {
136
155
  for (const item of value) {
137
156
  items.addSync(await toJava(item));
138
157
  }
139
- return cls.interop.ednListSync(items);
158
+ return nativeBindingsActive() ? cls.interop.ednList(items) : cls.interop.ednListSync(items);
140
159
  }
141
160
 
142
161
  if (value instanceof Form) {
@@ -207,6 +226,11 @@ export async function toJs(value) {
207
226
 
208
227
  const cls = await classes();
209
228
 
229
+ if (typeof value.getClassSync === "function"
230
+ && value.getClassSync().getNameSync() === "datalevin.NativeValue") {
231
+ return nativeToJs(value);
232
+ }
233
+
210
234
  if (instanceOf(value, cls.uuid)) {
211
235
  return value.toStringSync();
212
236
  }
@@ -233,7 +257,8 @@ export async function toJs(value) {
233
257
 
234
258
  if (instanceOf(value, cls.mapType) || typeof value.entrySetSync === "function") {
235
259
  const entries = [];
236
- for (const entry of materializeJavaCollection(value.entrySetSync())) {
260
+ const entrySet = nativeBindingsActive() ? await value.entrySet() : value.entrySetSync();
261
+ for (const entry of await materializeJavaCollection(entrySet)) {
237
262
  entries.push([await toJs(entry.getKeySync()), await toJs(entry.getValueSync())]);
238
263
  }
239
264
 
@@ -245,7 +270,7 @@ export async function toJs(value) {
245
270
 
246
271
  if (instanceOf(value, cls.setType)) {
247
272
  const items = [];
248
- for (const item of materializeJavaCollection(value)) {
273
+ for (const item of await materializeJavaCollection(value)) {
249
274
  items.push(await toJs(item));
250
275
  }
251
276
  return new Set(items);
@@ -257,7 +282,7 @@ export async function toJs(value) {
257
282
  || typeof value.iteratorSync === "function"
258
283
  ) {
259
284
  const items = [];
260
- for (const item of materializeJavaCollection(value)) {
285
+ for (const item of await materializeJavaCollection(value)) {
261
286
  items.push(await toJs(item));
262
287
  }
263
288
  return items;
@@ -271,7 +296,7 @@ export async function toJsQueryResult(value) {
271
296
  const cls = await classes();
272
297
  if (instanceOf(value, cls.setType)) {
273
298
  const rows = [];
274
- for (const item of materializeJavaCollection(value)) {
299
+ for (const item of await materializeJavaCollection(value)) {
275
300
  rows.push(await toJs(item));
276
301
  }
277
302
  return rows;
@@ -284,12 +309,16 @@ async function toJavaFormMap(entries, converter) {
284
309
  const cls = await classes();
285
310
  const jmap = new cls.linkedHashMap();
286
311
  for (const [key, item] of entries) {
287
- jmap.putSync(await converter(key), await converter(item));
312
+ const k = await converter(key), v = await converter(item);
313
+ if (nativeBindingsActive()) await jmap.put(k, v);
314
+ else jmap.putSync(k, v);
288
315
  }
289
316
  return jmap;
290
317
  }
291
318
 
292
319
  export async function toEdnForm(value) {
320
+ const native = nativeCodecFor(value);
321
+ if (native) return native.wrap(value);
293
322
  if (value === null || isJavaObject(value)) {
294
323
  return value;
295
324
  }
@@ -345,6 +374,8 @@ export async function toEdnForm(value) {
345
374
  }
346
375
 
347
376
  export async function toQueryInput(value) {
377
+ const native = nativeCodecFor(value);
378
+ if (native) return native.wrap(value);
348
379
  if (value === null || isJavaObject(value)) {
349
380
  return value;
350
381
  }
package/src/database.js CHANGED
@@ -1,7 +1,9 @@
1
1
  import { toEdnForm, toJava } from "./convert.js";
2
2
  import { _BINDINGS } from "./interop.js";
3
+ import { preparePull, prepareQuery } from "./prepared.js";
3
4
  import { callJavaMethod } from "./jvm.js";
4
5
  import { toJsResult } from "./result.js";
6
+ import { currentNativeRegistry, nativeMethods } from "./native.js";
5
7
 
6
8
  async function pullSelector(value) {
7
9
  if (typeof value === "string") {
@@ -20,6 +22,7 @@ async function mapGet(map, key) {
20
22
 
21
23
  export class Database {
22
24
  constructor(handle) {
25
+ this._nativeRegistry = currentNativeRegistry();
23
26
  this._handle = handle;
24
27
  }
25
28
 
@@ -27,6 +30,14 @@ export class Database {
27
30
  return this._handle;
28
31
  }
29
32
 
33
+ async preparePull(selector, opts = null) {
34
+ return preparePull(this, selector, opts);
35
+ }
36
+
37
+ async prepareQuery(query) {
38
+ return prepareQuery(this, query);
39
+ }
40
+
30
41
  toString() {
31
42
  return "<Database>";
32
43
  }
@@ -83,3 +94,5 @@ export async function txReportToJs(report) {
83
94
  }
84
95
  return result;
85
96
  }
97
+
98
+ nativeMethods(Database);
package/src/entity.js CHANGED
@@ -1,6 +1,7 @@
1
1
  import { isJavaObject, toJs } from "./convert.js";
2
2
  import { _BINDINGS } from "./interop.js";
3
3
  import { classes } from "./jvm.js";
4
+ import { currentNativeRegistry, nativeMethods } from "./native.js";
4
5
 
5
6
  function materializeJavaCollection(collection) {
6
7
  if (typeof collection?.toArraySync === "function") {
@@ -123,6 +124,7 @@ async function entityValue(value) {
123
124
 
124
125
  export class Entity {
125
126
  constructor(handle) {
127
+ this._nativeRegistry = currentNativeRegistry();
126
128
  this.handle = handle;
127
129
  }
128
130
 
@@ -169,3 +171,5 @@ export class Entity {
169
171
  return "<Entity lazy>";
170
172
  }
171
173
  }
174
+
175
+ nativeMethods(Entity);
package/src/index.js CHANGED
@@ -1,6 +1,15 @@
1
1
  export { Client } from "./client.js";
2
2
  export { Connection } from "./connection.js";
3
3
  export { Database } from "./database.js";
4
+ export {
5
+ PreparedRead,
6
+ executePrepared,
7
+ indexAttr,
8
+ prepareGetValue,
9
+ preparePull,
10
+ prepareQuery,
11
+ transactAck
12
+ } from "./prepared.js";
4
13
  export { Entity } from "./entity.js";
5
14
  export {
6
15
  DatalevinConfigurationError,
@@ -34,6 +43,7 @@ export {
34
43
  initDb,
35
44
  jvmStarted,
36
45
  keyword,
46
+ listenDb,
37
47
  maxEid,
38
48
  newClient,
39
49
  newLlamaEmbedder,
@@ -58,6 +68,8 @@ export {
58
68
  txEnsure,
59
69
  txRetract,
60
70
  txRetractEntity,
71
+ unlistenDb,
72
+ updateKv,
61
73
  vectorAttr,
62
74
  vectorOptions,
63
75
  withTransaction,
package/src/interop.js CHANGED
@@ -1,6 +1,7 @@
1
1
  import { toEdnForm, toJava, toJs, toQueryInput } from "./convert.js";
2
2
  import { DatalevinError } from "./errors.js";
3
3
  import { callJavaMethod, classes, javaBridgeModule, jvmStarted, startJvm } from "./jvm.js";
4
+ import { bindNativeCallback, registryFromOpts, withNativeScope } from "./native.js";
4
5
 
5
6
  export const CONNECTION_WITH_TRANSACTION_UNSUPPORTED =
6
7
  "Connection withTransaction is not exposed by the JavaScript binding because Java interface callbacks deadlock when the callback calls back into Datalevin. Use transact() for a single Datalog transaction, or KV withTransaction for explicit KV transactions.";
@@ -41,10 +42,10 @@ async function normalizeInteropArgs(args = []) {
41
42
  async function createFunctionProxy(fn) {
42
43
  const { newProxy } = await javaBridgeModule();
43
44
  return newProxy("java.util.function.Function", {
44
- apply: async (value) => {
45
+ apply: bindNativeCallback(async (value) => {
45
46
  const result = await fn(value);
46
47
  return toJava(result === undefined ? null : result);
47
- }
48
+ })
48
49
  });
49
50
  }
50
51
 
@@ -69,6 +70,12 @@ async function withInterfaceProxyEventLoop(fn) {
69
70
  }
70
71
 
71
72
  class InteropBindings {
73
+ async registerType(handle, typeName, definition) {
74
+ const cls = await classes();
75
+ return callJavaMethod(cls.interop, "registerType",
76
+ await unwrapInteropHandle(handle), String(typeName), await toJava(definition));
77
+ }
78
+
72
79
  async apiInfoRaw() {
73
80
  const cls = await classes();
74
81
  return callJavaMethod(cls.datalevin, "apiInfo");
@@ -102,6 +109,34 @@ class InteropBindings {
102
109
  );
103
110
  }
104
111
 
112
+ async preparePull(db, selector, opts = null) {
113
+ const cls = await classes();
114
+ return callJavaMethod(cls.interop, "preparePull", db, selector, await toJava(opts));
115
+ }
116
+
117
+ async prepareQuery(db, query) {
118
+ const cls = await classes();
119
+ return callJavaMethod(cls.interop, "prepareQuery", db, query);
120
+ }
121
+
122
+ async prepareGetValue(kv, dbiName, kType, vType, ignoreKey) {
123
+ const cls = await classes();
124
+ return callJavaMethod(cls.interop, "prepareGetValue", kv, dbiName,
125
+ await toJava(kType), await toJava(vType), ignoreKey);
126
+ }
127
+
128
+ async connectionTransactAck(conn, txData, txMeta = null) {
129
+ const cls = await classes();
130
+ return callJavaMethod(cls.interop, "connectionTransactAck", await unwrapInteropHandle(conn),
131
+ await toJava(txData), await toJava(txMeta));
132
+ }
133
+
134
+ async connectionIndexAttr(conn, attr) {
135
+ const cls = await classes();
136
+ return callJavaMethod(cls.interop, "connectionIndexAttr", await unwrapInteropHandle(conn),
137
+ await toJava(attr));
138
+ }
139
+
105
140
  async clientInvoke(functionName, args = null) {
106
141
  const cls = await classes();
107
142
  return callJavaMethod(
@@ -368,6 +403,19 @@ class InteropBindings {
368
403
  );
369
404
  }
370
405
 
406
+ async connectionListenDb(handle, keyOrListener, listener = null) {
407
+ const cls = await classes();
408
+ const raw = await unwrapInteropHandle(handle);
409
+ return hasValue(listener)
410
+ ? callJavaMethod(cls.interop, "connectionListenDb", raw, await toJava(keyOrListener), listener)
411
+ : callJavaMethod(cls.interop, "connectionListenDb", raw, keyOrListener);
412
+ }
413
+
414
+ async connectionUnlistenDb(handle, key) {
415
+ const cls = await classes();
416
+ await callJavaMethod(cls.interop, "connectionUnlistenDb", await unwrapInteropHandle(handle), await toJava(key));
417
+ }
418
+
371
419
  async connectionDatoms(handle, index, c1 = null, c2 = null, c3 = null, limit = null) {
372
420
  const cls = await classes();
373
421
  return callJavaMethod(
@@ -531,9 +579,33 @@ class InteropBindings {
531
579
  return Boolean(await callJavaMethod(handle, "closed"));
532
580
  }
533
581
 
582
+ async keyValueUpdate(handle, dbi, key, fn, kType, vType, args) {
583
+ const cls = await classes();
584
+ const values = [await unwrapInteropHandle(handle), dbi, await toJava(key)];
585
+ const typesAndArgs = [await toJava(kType), await toJava(vType), await toJava(args)];
586
+ if (typeof fn === "string") {
587
+ return callJavaMethod(cls.interop, "keyValueUpdateSource", ...values, fn, ...typesAndArgs);
588
+ }
589
+ const { newProxy } = await javaBridgeModule();
590
+ const proxy = newProxy("datalevin.UdfFunction", {
591
+ invoke: bindNativeCallback(async (items) => {
592
+ const result = await fn(...await toJs(items));
593
+ return toJava(result === undefined ? null : result);
594
+ })
595
+ });
596
+ try {
597
+ return await callJavaMethod(cls.interop, "keyValueUpdate", ...values, proxy, ...typesAndArgs);
598
+ } finally {
599
+ proxy.reset?.();
600
+ }
601
+ }
602
+
534
603
  async keyValueBeginTransaction(handle) {
535
604
  const cls = await classes();
536
- return callJavaMethod(cls.interop, "keyValueBeginTransaction", await unwrapInteropHandle(handle));
605
+ const method = typeof cls.interop.keyValueBeginTransactionBridge === "function"
606
+ ? "keyValueBeginTransactionBridge"
607
+ : "keyValueBeginTransaction";
608
+ return callJavaMethod(cls.interop, method, await unwrapInteropHandle(handle));
537
609
  }
538
610
 
539
611
  async keyValueCommitTransaction(tx) {
@@ -1350,6 +1422,11 @@ class InteropBindings {
1350
1422
  return callJavaMethod(cls.interop, "registerUdf", registry, await toJava(descriptor), fn);
1351
1423
  }
1352
1424
 
1425
+ async bindNativeType(registry, typeName, deserialize) {
1426
+ const cls = await classes();
1427
+ return callJavaMethod(cls.interop, "bindNativeType", registry, typeName, await toJava(deserialize));
1428
+ }
1429
+
1353
1430
  async unregisterUdf(registry, descriptor) {
1354
1431
  const cls = await classes();
1355
1432
  return callJavaMethod(cls.interop, "unregisterUdf", registry, await toJava(descriptor));
@@ -1502,16 +1579,18 @@ export async function execJson(op, args = null) {
1502
1579
 
1503
1580
  export async function connect(dir = null, { schema = null, opts = null, shared = false } = {}) {
1504
1581
  const { Connection } = await import("./connection.js");
1505
- return new Connection(await _BINDINGS.createConnection(dir, schema, opts, { shared }));
1582
+ return withNativeScope(registryFromOpts(opts, dir), async () =>
1583
+ new Connection(await _BINDINGS.createConnection(dir, schema, opts, { shared })));
1506
1584
  }
1507
1585
 
1508
1586
  export async function initDb(datoms, { dir = null, schema = null, opts = null } = {}) {
1509
1587
  const { Connection } = await import("./connection.js");
1510
- return new Connection(await _BINDINGS.initDb(datoms, dir, schema, opts));
1588
+ return withNativeScope(registryFromOpts(opts, dir), async () =>
1589
+ new Connection(await _BINDINGS.initDb(datoms, dir, schema, opts)));
1511
1590
  }
1512
1591
 
1513
1592
  export async function fillDb(conn, datoms) {
1514
- await _BINDINGS.fillDb(conn, datoms);
1593
+ await withNativeScope(conn?._nativeRegistry ?? null, () => _BINDINGS.fillDb(conn, datoms));
1515
1594
  return conn;
1516
1595
  }
1517
1596
 
@@ -1531,6 +1610,18 @@ export async function txDataToSimulatedReport(conn, txData) {
1531
1610
  return conn.txDataToSimulatedReport(txData);
1532
1611
  }
1533
1612
 
1613
+ export async function listenDb(conn, callback, key = null) {
1614
+ return conn.listenDb(callback, key);
1615
+ }
1616
+
1617
+ export async function unlistenDb(conn, key) {
1618
+ return conn.unlistenDb(key);
1619
+ }
1620
+
1621
+ export async function updateKv(kv, dbiName, key, fn, opts = {}) {
1622
+ return kv.update(dbiName, key, fn, opts);
1623
+ }
1624
+
1534
1625
  export async function datalogKv(conn) {
1535
1626
  return conn.datalogKv();
1536
1627
  }
@@ -1599,6 +1690,7 @@ export function schemaAttr({
1599
1690
  cardinality = null,
1600
1691
  unique = null,
1601
1692
  index = null,
1693
+ noIndex = null,
1602
1694
  fulltext = null,
1603
1695
  isComponent = null,
1604
1696
  noHistory = null,
@@ -1633,6 +1725,9 @@ export function schemaAttr({
1633
1725
  if (index !== null && index !== undefined) {
1634
1726
  spec[":db/index"] = Boolean(index);
1635
1727
  }
1728
+ if (noIndex !== null && noIndex !== undefined) {
1729
+ spec[":db/noindex"] = Boolean(noIndex);
1730
+ }
1636
1731
  if (fulltext !== null && fulltext !== undefined) {
1637
1732
  spec[":db/fulltext"] = Boolean(fulltext);
1638
1733
  }
@@ -2066,7 +2161,8 @@ export function datomAdded(value) {
2066
2161
 
2067
2162
  export async function openKv(dir, opts = null) {
2068
2163
  const { KV } = await import("./kv.js");
2069
- return new KV(await _BINDINGS.openKeyValue(dir, opts));
2164
+ return withNativeScope(registryFromOpts(opts, dir), async () =>
2165
+ new KV(await _BINDINGS.openKeyValue(dir, opts)));
2070
2166
  }
2071
2167
 
2072
2168
  export async function newSearchEngine(kv, opts = null) {
package/src/kv.js CHANGED
@@ -4,6 +4,7 @@ import { _BINDINGS } from "./interop.js";
4
4
  import { callJavaMethod, javaBridgeModule } from "./jvm.js";
5
5
  import { ResourceWrapper } from "./resource.js";
6
6
  import { toJsResult } from "./result.js";
7
+ import { bindNativeCallback, nativeMethods } from "./native.js";
7
8
 
8
9
  const NO_TIMEOUT_OPTION = Symbol("no-timeout-option");
9
10
 
@@ -141,7 +142,9 @@ async function withInterfaceProxyEventLoop(fn) {
141
142
 
142
143
  async function withJavaProxy(interfaceName, implementation, fn) {
143
144
  const { newProxy } = await javaBridgeModule();
144
- const proxy = newProxy(interfaceName, implementation);
145
+ const methods = Object.fromEntries(Object.entries(implementation)
146
+ .map(([name, fn]) => [name, bindNativeCallback(fn)]));
147
+ const proxy = newProxy(interfaceName, methods);
145
148
  try {
146
149
  return await withInterfaceProxyEventLoop(() => fn(proxy));
147
150
  } finally {
@@ -261,6 +264,13 @@ export class KV extends ResourceWrapper {
261
264
  return toJsResult(await _BINDINGS.coreInvoke("dir", [this.rawHandle()]));
262
265
  }
263
266
 
267
+ /** Register an ordered type shared by this database's KV and Datalog APIs.
268
+ * Supply UDF descriptors and install bindings locally or on the server.
269
+ */
270
+ async registerType(typeName, definition) {
271
+ return toJs(await _BINDINGS.registerType(this.rawHandle(), typeName, definition));
272
+ }
273
+
264
274
  async openDbi(name, opts = null) {
265
275
  const args = [this.rawHandle(), name];
266
276
  if (opts !== null && opts !== undefined) {
@@ -413,6 +423,19 @@ export class KV extends ResourceWrapper {
413
423
  return toJsResult(await _BINDINGS.coreInvoke("gc-txlog-segments!", args));
414
424
  }
415
425
 
426
+ async prepareGetValue(dbiName, opts = {}) {
427
+ const { prepareGetValue } = await import("./prepared.js");
428
+ return prepareGetValue(this, dbiName, opts);
429
+ }
430
+
431
+ /** Atomically update a value; host callbacks are embedded-only, source also runs remotely. */
432
+ async update(dbiName, key, fn, { kType = ":data", vType = ":data", args = [] } = {}) {
433
+ if (typeof fn !== "function" && typeof fn !== "string") {
434
+ throw new TypeError("fn must be a function or restricted Clojure function source.");
435
+ }
436
+ return toJsResult(await _BINDINGS.keyValueUpdate(this.rawHandle(), dbiName, key, fn, kType, vType, args));
437
+ }
438
+
416
439
  async transact(txs, { dbiName = null, kType = null, vType = null } = {}) {
417
440
  if (dbiName === null && (kType !== null || vType !== null)) {
418
441
  throw new TypeError("kType and vType require dbiName for KV transact().");
@@ -1229,3 +1252,6 @@ export class KVTransaction extends KV {
1229
1252
  }
1230
1253
  }
1231
1254
  }
1255
+
1256
+ nativeMethods(KV);
1257
+ nativeMethods(KVTransaction);
package/src/native.js ADDED
@@ -0,0 +1,172 @@
1
+ import { AsyncLocalStorage } from "node:async_hooks";
2
+ import { randomUUID } from "node:crypto";
3
+ import { isDeepStrictEqual } from "node:util";
4
+ import { javaBridgeModule } from "./jvm.js";
5
+ import { UdfDescriptor, descriptorData } from "./udf-value.js";
6
+
7
+ const scope = new AsyncLocalStorage();
8
+ const codecs = new Map();
9
+ const cleanup = new FinalizationRegistry((id) => codecs.delete(id));
10
+ const builtins = new Set([Object, Array, Map, Set, Date, RegExp, Buffer, Uint8Array,
11
+ String, Number, Boolean, Function, Promise, WeakMap, WeakSet]);
12
+
13
+ export function descriptorKey(descriptor) {
14
+ const d = descriptorData(descriptor);
15
+ return [d[":udf/lang"], d[":udf/kind"], d[":udf/id"], d[":udf/version"] ?? ""].join("\u0000");
16
+ }
17
+
18
+ export function currentNativeRegistry() {
19
+ const registry = scope.getStore() ?? null;
20
+ return registry?.ownerRef?.deref() ?? registry;
21
+ }
22
+
23
+ export function withNativeScope(registry, fn) {
24
+ return scope.run(registry, fn);
25
+ }
26
+
27
+ export function bindNativeCallback(fn) {
28
+ const registry = currentNativeRegistry();
29
+ return (...args) => withNativeScope(registry, () => fn(...args));
30
+ }
31
+
32
+ export function nativeMethods(cls) {
33
+ for (const [name, property] of Object.entries(Object.getOwnPropertyDescriptors(cls.prototype))) {
34
+ if (name === "constructor" || name.startsWith("_") || typeof property.value !== "function") continue;
35
+ const fn = property.value;
36
+ Object.defineProperty(cls.prototype, name, {
37
+ ...property,
38
+ value: function (...args) {
39
+ const registry = this._nativeRegistry ?? null;
40
+ return registry === currentNativeRegistry()
41
+ ? fn.apply(this, args)
42
+ : withNativeScope(registry, () => fn.apply(this, args));
43
+ }
44
+ });
45
+ }
46
+ }
47
+
48
+ function field(map, key) {
49
+ return map instanceof Map ? (map.get(key) ?? map.get(`:${key}`)) : (map?.[key] ?? map?.[`:${key}`]);
50
+ }
51
+
52
+ export function registryFromOpts(opts, dir = null) {
53
+ const registry = field(field(opts, "runtime-opts"), "udf-registry");
54
+ if (!registry?._nativeBindings) return null;
55
+ return registry;
56
+ }
57
+
58
+ export function currentNativeBindings() {
59
+ const registry = currentNativeRegistry();
60
+ return registry?._nativeBindings ?? registry;
61
+ }
62
+
63
+ export function nativeCodecFor(value) {
64
+ if (value === null || typeof value !== "object") return null;
65
+ return currentNativeBindings()?.types.get(Object.getPrototypeOf(value)) ?? null;
66
+ }
67
+
68
+ function valueCodec(value) {
69
+ const codec = codecs.get(value.codecIdSync())?.deref();
70
+ if (!codec) throw new Error(`Missing native type binding for ${value.typeNameSync()}`);
71
+ return codec;
72
+ }
73
+
74
+ export async function nativeToJs(value) {
75
+ return valueCodec(value).decode(value.payloadSync());
76
+ }
77
+
78
+ // This callback captures no registry or proxy owner. The registry retains the
79
+ // proxy; codecs and UDF callbacks retain only host state and a weak proxy ref.
80
+ async function nativeEquals(left, right) {
81
+ const codec = valueCodec(left);
82
+ const equal = await codec.equals(await nativeToJs(left), await nativeToJs(right));
83
+ if (typeof equal !== "boolean") throw new TypeError("Native equality must return a boolean");
84
+ return equal;
85
+ }
86
+
87
+ export async function createNativeEqualityProxy() {
88
+ const { newProxy } = await javaBridgeModule();
89
+ return newProxy("java.util.function.BiPredicate", { test: nativeEquals });
90
+ }
91
+
92
+ let nativeClassPromise;
93
+ async function nativeClass() {
94
+ nativeClassPromise ??= javaBridgeModule().then((bridge) => bridge.importClass("datalevin.NativeValue"));
95
+ return nativeClassPromise;
96
+ }
97
+
98
+ export class NativeCodec {
99
+ constructor(bindings, typeName, nativeType, definition, { equals = isDeepStrictEqual } = {}) {
100
+ if (typeof nativeType !== "function" || !nativeType.prototype || builtins.has(nativeType)) {
101
+ throw new TypeError("nativeType must be a custom JavaScript class");
102
+ }
103
+ if (typeof equals !== "function") throw new TypeError("equals must be a function");
104
+ const name = String(typeName).replace(/^:/, "");
105
+ if (!/^[^/\s]+\/[^\s]+$/.test(name)) throw new TypeError("Native type name must be a namespaced keyword");
106
+ const payload = field(definition, "payload");
107
+ if (!payload || typeof payload !== "object") {
108
+ throw new TypeError("Native types require serialize and deserialize UDF descriptors");
109
+ }
110
+ this.serialize = UdfDescriptor.from(field(payload, "serialize"), { defaultLang: "javascript" });
111
+ this.deserialize = UdfDescriptor.from(field(payload, "deserialize"), { defaultLang: "javascript" });
112
+ if (this.serialize.kind !== ":serializer" || this.deserialize.kind !== ":deserializer") {
113
+ throw new TypeError("Native types require serializer and deserializer UDF kinds");
114
+ }
115
+ this.bindings = bindings;
116
+ this.nativeType = nativeType;
117
+ this.typeName = `:${name}`;
118
+ this.equals = equals;
119
+ this.id = randomUUID();
120
+ }
121
+
122
+ sameBinding(other) {
123
+ return this.typeName === other.typeName && this.nativeType === other.nativeType
124
+ && this.equals === other.equals
125
+ && descriptorKey(this.serialize) === descriptorKey(other.serialize)
126
+ && descriptorKey(this.deserialize) === descriptorKey(other.deserialize);
127
+ }
128
+
129
+ install() {
130
+ codecs.set(this.id, new WeakRef(this));
131
+ cleanup.register(this, this.id);
132
+ }
133
+
134
+ function(descriptor) {
135
+ const fn = this.bindings.functions.get(descriptorKey(descriptor));
136
+ if (!fn) throw new Error(`Missing native type UDF binding: ${descriptor.udfId}`);
137
+ return fn;
138
+ }
139
+
140
+ accepts(value) {
141
+ return value !== null && typeof value === "object" && Object.getPrototypeOf(value) === this.nativeType.prototype;
142
+ }
143
+
144
+ checkValue(value) {
145
+ if (!this.accepts(value)) {
146
+ throw new TypeError(`Deserializer for ${this.typeName} must return ${this.nativeType.name}`);
147
+ }
148
+ }
149
+
150
+ async decode(payload) {
151
+ const value = await this.function(this.deserialize)(Buffer.from(payload));
152
+ this.checkValue(value);
153
+ return value;
154
+ }
155
+
156
+ async wrap(value) {
157
+ const payload = await this.function(this.serialize)(value);
158
+ if (!Buffer.isBuffer(payload) && !(payload instanceof Uint8Array)) {
159
+ throw new TypeError("Custom serializer must return a byte array (Buffer or Uint8Array)");
160
+ }
161
+ return this.wrapPayload(Buffer.from(payload));
162
+ }
163
+
164
+ async wrapPayload(payload) {
165
+ const equality = this.bindings.equalityProxyRef?.deref();
166
+ if (!equality) throw new Error(`Missing native equality binding for ${this.typeName}`);
167
+ const NativeValue = await nativeClass();
168
+ const bytes = Buffer.from(payload);
169
+ return new NativeValue(this.id, this.typeName,
170
+ Array.from(new Int8Array(bytes.buffer, bytes.byteOffset, bytes.byteLength)), equality);
171
+ }
172
+ }
@@ -0,0 +1,88 @@
1
+ import { toEdnForm, toJava, toJs, toQueryInput } from "./convert.js";
2
+ import { Form } from "./form.js";
3
+ import { _BINDINGS } from "./interop.js";
4
+ import { callJavaMethod } from "./jvm.js";
5
+ import { currentNativeRegistry, nativeMethods, withNativeScope } from "./native.js";
6
+
7
+ async function viewHandle(view) {
8
+ const { Connection } = await import("./connection.js");
9
+ return view instanceof Connection
10
+ ? _BINDINGS.connectionDb(view.rawHandle())
11
+ : view.rawHandle();
12
+ }
13
+
14
+ async function form(value) {
15
+ return typeof value === "string" ? value : toEdnForm(value);
16
+ }
17
+
18
+ /** Reusable read metadata. No close is needed; the owning store must stay open. */
19
+ export class PreparedRead {
20
+ constructor(handle, { query = false, typed = false } = {}) {
21
+ this._handle = handle;
22
+ this._query = query;
23
+ this._typed = typed;
24
+ this._nativeRegistry = currentNativeRegistry();
25
+ }
26
+
27
+ rawHandle() {
28
+ return this._handle;
29
+ }
30
+
31
+ /** Execute with input, or with an explicit local view followed by input. */
32
+ async execute(...args) {
33
+ if (args.length !== 1 && args.length !== 2) {
34
+ throw new TypeError("execute expects input or view, input");
35
+ }
36
+ let value = args.at(-1);
37
+ if (this._query) {
38
+ if (!Array.isArray(value)) throw new TypeError("Prepared query input must be an array");
39
+ const { Connection } = await import("./connection.js");
40
+ const convert = this._typed ? toJava : toQueryInput;
41
+ value = await Promise.all(value.map(input => input instanceof Connection
42
+ ? _BINDINGS.connectionDb(input.rawHandle()) : convert(input)));
43
+ }
44
+ const normalized = await toJava(value);
45
+ const result = args.length === 2
46
+ ? await callJavaMethod(this._handle, "executeBridge", await viewHandle(args[0]), normalized)
47
+ : await callJavaMethod(this._handle, "executeBridge", normalized);
48
+ return toJs(result);
49
+ }
50
+ }
51
+
52
+ nativeMethods(PreparedRead);
53
+
54
+ /** Prepare a pull from a Connection or Database, with optional pull options. */
55
+ export async function preparePull(database, selector, opts = null) {
56
+ return withNativeScope(database._nativeRegistry, async () => new PreparedRead(
57
+ await _BINDINGS.preparePull(await viewHandle(database), await form(selector), opts)
58
+ ));
59
+ }
60
+
61
+ /** Prepare a query; execute with an array of the remaining :in values. */
62
+ export async function prepareQuery(database, query) {
63
+ return withNativeScope(database._nativeRegistry, async () => new PreparedRead(
64
+ await _BINDINGS.prepareQuery(await viewHandle(database), await form(query)),
65
+ { query: true, typed: query instanceof Form }
66
+ ));
67
+ }
68
+
69
+ /** Prepare a KV point read. Types default to data; results default to values. */
70
+ export async function prepareGetValue(kv, dbiName, {
71
+ kType = null, vType = null, ignoreKey = true
72
+ } = {}) {
73
+ return withNativeScope(kv._nativeRegistry, async () => new PreparedRead(
74
+ await _BINDINGS.prepareGetValue(kv.rawHandle(), dbiName, kType, vType, ignoreKey)
75
+ ));
76
+ }
77
+
78
+ export async function executePrepared(prepared, ...args) {
79
+ return prepared.execute(...args);
80
+ }
81
+
82
+ export async function transactAck(conn, txData, txMeta = null) {
83
+ return conn.transactAck(txData, txMeta);
84
+ }
85
+
86
+ export async function indexAttr(conn, attr) {
87
+ return conn.indexAttr(attr);
88
+ }
package/src/resource.js CHANGED
@@ -1,7 +1,9 @@
1
1
  import util from "node:util";
2
+ import { currentNativeRegistry } from "./native.js";
2
3
 
3
4
  export class ResourceWrapper {
4
5
  constructor(handle, closeFn, closedFn, label) {
6
+ this._nativeRegistry = currentNativeRegistry();
5
7
  this._handle = handle;
6
8
  this._closeFn = closeFn;
7
9
  this._closedFn = closedFn;
package/src/udf-value.js CHANGED
@@ -11,7 +11,10 @@ export const UDF_KINDS = Object.freeze(new Set([
11
11
  "predicate",
12
12
  "tx-fn",
13
13
  "analyzer",
14
- "query-analyzer"
14
+ "query-analyzer",
15
+ "order-fn",
16
+ "serializer",
17
+ "deserializer"
15
18
  ]));
16
19
 
17
20
  const ALIASES = new Map([
@@ -223,6 +226,18 @@ export class UdfDescriptor extends Form {
223
226
  return UdfDescriptor.of("query-analyzer", udfId, options);
224
227
  }
225
228
 
229
+ static orderFn(udfId, options = {}) {
230
+ return UdfDescriptor.of("order-fn", udfId, options);
231
+ }
232
+
233
+ static serializer(udfId, options = {}) {
234
+ return UdfDescriptor.of("serializer", udfId, options);
235
+ }
236
+
237
+ static deserializer(udfId, options = {}) {
238
+ return UdfDescriptor.of("deserializer", udfId, options);
239
+ }
240
+
226
241
  static from(value, { defaultLang = "javascript" } = {}) {
227
242
  return value instanceof UdfDescriptor
228
243
  ? value
package/src/udf.js CHANGED
@@ -3,22 +3,13 @@ import { Database } from "./database.js";
3
3
  import { _BINDINGS } from "./interop.js";
4
4
  import { javaBridgeModule } from "./jvm.js";
5
5
  import { UdfDescriptor, descriptorData } from "./udf-value.js";
6
+ import { NativeCodec, createNativeEqualityProxy, descriptorKey, withNativeScope } from "./native.js";
6
7
 
7
8
  const DATABASE_CLASSES = new Set([
8
9
  "datalevin.db.DB",
9
10
  "datalevin.DatabaseValue"
10
11
  ]);
11
12
 
12
- function descriptorKey(descriptor) {
13
- const normalized = descriptorData(descriptor);
14
- return [
15
- normalized[":udf/lang"],
16
- normalized[":udf/kind"],
17
- normalized[":udf/id"],
18
- normalized[":udf/version"] ?? ""
19
- ].join("\u0000");
20
- }
21
-
22
13
  function materializeJavaArgs(args) {
23
14
  if (Array.isArray(args)) {
24
15
  return args;
@@ -85,14 +76,28 @@ async function udfArgsToJs(args) {
85
76
  }));
86
77
  }
87
78
 
88
- async function createProxy(fn) {
79
+ async function createProxy(fn, descriptor, bindings) {
89
80
  const { newProxy } = await javaBridgeModule();
90
81
  return newProxy("datalevin.UdfFunction", {
91
- invoke: async (args) => {
82
+ invoke: (args) => withNativeScope(bindings, async () => {
92
83
  const values = await udfArgsToJs(args);
93
84
  const result = await fn(...values);
85
+ if (descriptor.kind === ":serializer"
86
+ && !Buffer.isBuffer(result) && !(result instanceof Uint8Array)) {
87
+ throw new TypeError("Custom serializer must return a byte array (Buffer or Uint8Array).");
88
+ }
89
+ if (descriptor.kind === ":deserializer") {
90
+ let expected;
91
+ for (const codec of bindings.types.values()) {
92
+ if (descriptorKey(codec.deserialize) === descriptorKey(descriptor)) {
93
+ expected ??= codec;
94
+ if (codec.accepts(result)) return codec.wrapPayload(values[0]);
95
+ }
96
+ }
97
+ expected?.checkValue(result);
98
+ }
94
99
  return toJava(result === undefined ? null : result);
95
- }
100
+ })
96
101
  });
97
102
  }
98
103
 
@@ -108,6 +113,8 @@ export class UdfRegistry {
108
113
  constructor(handle) {
109
114
  this._handle = handle;
110
115
  this._proxies = new Map();
116
+ this._nativeBindings = { types: new Map(), functions: new Map(), ownerRef: new WeakRef(this) };
117
+ this._nativeEqualityProxyPromise = null;
111
118
  }
112
119
 
113
120
  rawHandle() {
@@ -120,9 +127,10 @@ export class UdfRegistry {
120
127
  }
121
128
 
122
129
  const normalized = UdfDescriptor.from(descriptor, { defaultLang: "javascript" });
123
- const proxy = await createProxy(fn);
130
+ const proxy = await createProxy(fn, normalized, this._nativeBindings);
124
131
  await _BINDINGS.registerUdf(this._handle, normalized, proxy);
125
132
  this._proxies.set(descriptorKey(normalized), proxy);
133
+ this._nativeBindings.functions.set(descriptorKey(normalized), fn);
126
134
  return fn;
127
135
  }
128
136
 
@@ -130,6 +138,7 @@ export class UdfRegistry {
130
138
  const normalized = UdfDescriptor.from(descriptor, { defaultLang: "javascript" });
131
139
  await _BINDINGS.unregisterUdf(this._handle, normalized);
132
140
  const key = descriptorKey(normalized);
141
+ this._nativeBindings.functions.delete(key);
133
142
  const proxy = this._proxies.get(key);
134
143
  if (proxy !== undefined) {
135
144
  proxy.reset();
@@ -144,6 +153,46 @@ export class UdfRegistry {
144
153
  );
145
154
  }
146
155
 
156
+ /** Bind an exact native class to payload UDFs before opening a database.
157
+ * Equality defaults to node:util.isDeepStrictEqual; classes with private state
158
+ * can supply { equals: (left, right) => boolean }. This binding is not persisted.
159
+ */
160
+ async bindNativeType(typeName, nativeType, definition, options = {}) {
161
+ const codec = new NativeCodec(this._nativeBindings, typeName, nativeType, definition, options);
162
+ const existing = this._nativeBindings.types.get(nativeType.prototype);
163
+ if (existing) {
164
+ if (!existing.sameBinding(codec)) throw new TypeError("JavaScript class already has a different native type binding");
165
+ return this;
166
+ }
167
+ if ([...this._nativeBindings.types.values()].some((c) => c.typeName === codec.typeName)) {
168
+ throw new TypeError("Custom type already has a different JavaScript class binding");
169
+ }
170
+ this._nativeEqualityProxyPromise ??= createNativeEqualityProxy();
171
+ this._nativeBindings.equalityProxyRef = new WeakRef(await this._nativeEqualityProxyPromise);
172
+ // Recheck after the asynchronous proxy creation: concurrent conflicting
173
+ // bindings must not overwrite a class installed by another call.
174
+ const installed = this._nativeBindings.types.get(nativeType.prototype);
175
+ if (installed) {
176
+ if (!installed.sameBinding(codec)) throw new TypeError("JavaScript class already has a different native type binding");
177
+ return this;
178
+ }
179
+ if ([...this._nativeBindings.types.values()].some((c) => c.typeName === codec.typeName)) {
180
+ throw new TypeError("Custom type already has a different JavaScript class binding");
181
+ }
182
+ await _BINDINGS.bindNativeType(this._handle, codec.typeName, codec.deserialize);
183
+ const concurrent = this._nativeBindings.types.get(nativeType.prototype);
184
+ if (concurrent) {
185
+ if (!concurrent.sameBinding(codec)) throw new TypeError("JavaScript class already has a different native type binding");
186
+ return this;
187
+ }
188
+ if ([...this._nativeBindings.types.values()].some((c) => c.typeName === codec.typeName)) {
189
+ throw new TypeError("Custom type already has a different JavaScript class binding");
190
+ }
191
+ this._nativeBindings.types.set(nativeType.prototype, codec);
192
+ codec.install();
193
+ return this;
194
+ }
195
+
147
196
  async queryUdf(id, fn, options = {}) {
148
197
  return this.register(UdfDescriptor.queryFn(id, options), fn);
149
198
  }
@@ -163,6 +212,18 @@ export class UdfRegistry {
163
212
  async queryAnalyzerUdf(id, fn, options = {}) {
164
213
  return this.register(UdfDescriptor.queryAnalyzer(id, options), fn);
165
214
  }
215
+
216
+ async orderUdf(id, fn, options = {}) {
217
+ return this.register(UdfDescriptor.orderFn(id, options), fn);
218
+ }
219
+
220
+ async serializerUdf(id, fn, options = {}) {
221
+ return this.register(UdfDescriptor.serializer(id, options), fn);
222
+ }
223
+
224
+ async deserializerUdf(id, fn, options = {}) {
225
+ return this.register(UdfDescriptor.deserializer(id, options), fn);
226
+ }
166
227
  }
167
228
 
168
229
  export async function createUdfRegistry() {
@@ -1,4 +0,0 @@
1
- [diffend] Oversized file quarantined before diffing.
2
- name: package/jars/datalevin-runtime-1.1.0.jar
3
- size: 39587651 bytes
4
- sha256: 077a8f5fc0290acbf20ffbd3118ef21f3469cd2c4a5d3e923d4d6c704764bf92