w-orm-reladb 1.0.47 → 1.0.49

package/README.md

@@ -11,6 +11,62 @@ An operator for relational database in nodejs.
 ## Documentation
 To view documentation or get support, visit [docs](https://yuda-lyu.github.io/w-orm-reladb/WOrm.html).
 
+### `insert` with `option.returnList`
+
+An aggregate count answers "how many rows were new", but not **which ones** — and the latter is what deduplication actually needs, when only new rows should trigger an expensive downstream action. Setting `returnList` hands back the per row verdict that `insert` already computes internally:
+
+```alias
+let rs = await w.insert(data, { returnList: true })
+// => [ { n: 1, nInserted: 1, ok: 1 }, { n: 1, nInserted: 0, ok: 1 }, ... ]
+let fresh = data.filter((v, k) => rs[k].nInserted === 1)
+```
+
+The returned array is the same length as the input and in the same order. Each element carries `n: 1` and `ok: 1` — any failure in `insert` is a batch level error and rejects, so a per row element never reports a failure. `rs.filter(v => v.nInserted === 1).length` equals the `nInserted` of the default aggregate form, and invalid input returns `[]` instead of the aggregate empty result.
+
+The switch is static: the shape is decided solely by the value you write at the call site, never by the data or by what happened at run time. Treat the two values as two separate contracts and do not share result handling code between call sites that use different ones.
+
+### `insert` and `insertBulk`
+
+Both write rows that do not yet exist, and both return `{ n, nInserted, ok }`. They differ in what happens on a conflict, so `insertBulk` is **not** a faster `insert`:
+
+| Situation | `insert` | `insertBulk` |
+|---|---|---|
+| Primary key already exists | that row is skipped, the batch still returns `ok: 1` | the whole call **rejects, and not a single row is written** |
+| Duplicate primary keys inside one batch | only the first one counts toward `nInserted` | treated as a conflict, the whole call rejects |
+| `nInserted` | rows actually inserted, `0 ≤ nInserted ≤ n` | equals `n` whenever the call succeeds |
+| Use it for | ordinary writes against a table that may already hold data | bulk import where no conflict is expected |
+
+With no conflict the two are indistinguishable, so you may swap them under that assumption — and if the assumption is wrong, `insertBulk` tells you by rejecting rather than silently skipping.
+
+`insertBulk` is considerably faster here, because `insert` has to write row by row to report an exact `nInserted` (measured on this machine, Windows 11 / Node 24 / sequelize 6):
+
+| Rows | sqlite `insert` → `insertBulk` | mssql `insert` → `insertBulk` |
+|---|---|---|
+| 1000 | 5549 ms → 25 ms (226x) | 6802 ms → 190 ms (36x) |
+| 5000 | 29825 ms → 50 ms (592x) | 44116 ms → 489 ms (90x) |
+
+The all-or-nothing guarantee is provided by a transaction, not by the batch statement alone: mssql splits a large batch into several statements because of its bind parameter limit, so without one an interrupted batch would leave earlier rows behind. When you pass `option.transaction`, a SAVEPOINT is used instead, so a failure rolls back only this call and leaves the rest of your transaction untouched.
+
+### Closed instances
+
+`init()` hands you an instance you can share across calls through `option.instance`, and `instance.close()` ends it. Once closed, **every function rejects immediately** with a message naming the closed state — including `selectByPk`, which does not fall back to its usual `null`. This matters because a closed instance and "no such row" would otherwise look identical, and a deduplication caller reading that as "not present yet" would re-run the whole batch downstream. Do not keep using an instance after closing it; call `init()` again for a new one.
+
+Note also that a plain call made without `option.instance` opens and closes its own connection, so it ends any instance you are holding. Do not mix the two styles against the same `WOrmReladb` object.
+
+### Concurrency
+
+The scope of the atomicity guarantee, by range:
+
+| Range | Guaranteed | Provided by |
+|---|---|---|
+| Cross process — several processes against the same database | Yes | The database itself. `insert` and `save` write through single conditional statements, and the primary key unique constraint decides the winner. Independent of `opt.useStable`. |
+| Single process — several parallel calls in one process | Yes **when `opt.useStable` is `true`** (the default) | A built-in queue that runs one operation at a time. |
+| Single process, with `opt.useStable` set to `false` | **No** | — |
+
+**Do not issue parallel calls from one process while `opt.useStable` is `false`.** Each instance holds one shared connection, opened at the start of a call and closed at its end, so a second call running in parallel closes the connection the first one is still using. Measured on 30 parallel calls in a single process: sqlite raised `SQLITE_MISUSE: Database handle is closed` and rejected 12 of the batches, mssql reported `ConnectionManager.getConnection was called after the connection manager was closed!` as a per row failure; both ended with 29 of the 30 rows written, so writes are silently lost. Keep the default `true`, or serialize the calls yourself (for example `await` them one by one).
+
+Cross process was measured with 2 independent processes against the same 20 primary keys: `nInserted` summed to exactly 20 with 20 rows in the table, and with each process writing a different column to those keys all 40 operations returned `ok: 1` with both columns preserved on all 20 rows. sqlite was repeated over 6 rounds to rule out timing dependent `SQLITE_BUSY`. Platform: Windows 11, Node 24, sequelize 6, MSSQL 2022.
+
 ## Before Installation
 > If you need to use encrypted sqlite, you need to manually install `@journeyapps/sqlcipher`, as follows:
 >1. Open visual studio code by system administrator.
@@ -42,7 +98,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
 }
 
 let rs = [
@@ -106,8 +162,8 @@ async function test() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -196,10 +252,10 @@ test()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // select all [
 //   { id: 'id-peter', name: 'peter(modify)', value: 123 },
@@ -243,7 +299,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
 }
 
 let rs = [
@@ -307,8 +363,8 @@ async function testCommit() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -424,10 +480,10 @@ testCommit()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // change del
 // del then [
@@ -460,7 +516,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
 }
 
 let rs = [
@@ -524,8 +580,8 @@ async function testRollback() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -638,10 +694,10 @@ testRollback()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // change del
 // del then [
@@ -672,7 +728,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
     storage: './worm.sqlite',
 }
 
@@ -743,8 +799,8 @@ async function test() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -833,10 +889,10 @@ test()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // select all [
 //   { id: 'id-peter', name: 'peter(modify)', value: 123 },
@@ -881,7 +937,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
     storage: './worm.sqlite',
 }
 
@@ -951,8 +1007,8 @@ async function testCommit() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -1068,10 +1124,10 @@ testCommit()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // change del
 // del then [
@@ -1105,7 +1161,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
     storage: './worm.sqlite',
 }
 
@@ -1175,8 +1231,8 @@ async function testRollback() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -1289,10 +1345,10 @@ testRollback()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // change del
 // del then [
@@ -1323,7 +1379,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
     storage: './worm.sqlite',
     useEncryption: true,
 }
@@ -1401,8 +1457,8 @@ async function test() {
     w.on('change', function(mode, data, res) {
         console.log('change', mode)
     })
-    w.on('error', function(err) {
-        console.log('error', err)
+    w.on('error', function(mode, data, err) {
+        console.log('error', mode, err)
     })
 
 
@@ -1491,10 +1547,10 @@ test()
 // insert then { n: 3, nInserted: 3, ok: 1 }
 // change save
 // save then [
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 1, nModified: 1, ok: 1 },
-//   { n: 0, nModified: 0, ok: 1 } //autoInsert=false
-//   { n: 1, nInserted: 1, ok: 1 } //autoInsert=true
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 1, nInserted: 0, nModified: 1, ok: 1 },
+//   { n: 0, nInserted: 0, nModified: 0, ok: 1 } //autoInsert=false
+//   { n: 1, nInserted: 1, nModified: 0, ok: 1 } //autoInsert=true
 // ]
 // select all [
 //   { id: 'id-peter', name: 'peter(modify)', value: 123 },
@@ -1538,7 +1594,7 @@ let opt = {
     cl: 'users',
     fdModels: './models',
     // modelType: 'json',
-    // autoGenPK: false,
+    // autoGenPk: false,
 }
 
 let fd = opt.fdModels