nodejs-store 2.4.0 → 2.5.0

package/README.md

@@ -316,9 +316,26 @@ async function transfer() {
 await store.transaction('default', transfer);
 ```
 
-- `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `executeRaw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `runInTransaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
-- `store.executeRaw(source, sql, params, isWrite)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source throws `RawSqlError` (`store.RawSqlError`).
-- `isWrite=false` (default) returns `{ rows, affectedRows }` with the result-set rows; `isWrite=true` returns the affected-row count.
+- `store.transaction(source, fn)` opens a transaction scope on one source: every `executeRaw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `runInTransaction`). Mongo sources are probed at runtime (replica set / sharded) and wrapped in a session transaction; on standalone or probe failure `fn` runs as-is and emits `mongo_transaction_unsupported` (`deployment: standalone|unknown`) — it never pretends to be atomic. Executors without `withTransaction` also run `fn` as-is and emit a `transaction_not_atomic` feedback event (degradation is allowed, silent pretence is not). A nested same-source transaction opens a savepoint (an inner failure rolls back only that scope); without savepoint primitives it degrades by joining the outer transaction and emits `nested_savepoint_unsupported`.
+- `store.executeRaw(source, sql, params = null, isWrite = null)` runs raw SQL, compiled by the core `rawStmtCompile`. Two styles selected by the `params` type: **positional** (array/null) passes the SQL through as-is with native placeholders (`?` for MySQL / SQLite, `$1..$n` for PostgreSQL); **named** (object) compiles `:name` tokens in the SQL into dialect placeholders (same-name reuse, `::` casts / quotes / comments kept intact; missing or unused names throw `RawSqlError`). SQL sources only — a Mongo source throws `RawSqlError` (`store.RawSqlError`).
+- When `isWrite` is omitted it is inferred from the SQL's first word (SELECT / WITH / EXPLAIN / SHOW / PRAGMA / TABLE count as reads, everything else as a write — defaulting to write is the safe direction); passing it explicitly overrides the inference. Returns `{ rows, affectedRows }`: rows for reads, the affected-row count for writes.
+- `store.executeNative(source, collection, pipeline = [], options = null)` runs a native aggregation pipeline on a Mongo source (the Mongo counterpart of the SQL-side `executeRaw` escape hatch): `pipeline` is a native aggregation pipeline, `options` uses driver-native keys (`allowDiskUse` / `batchSize` / `hint` / `maxTimeMS`..., no host-side whitelist). Inside a transaction / session the session is injected automatically (owned by the transaction; `options.session` cannot override it); resolution always follows the read path, so `$merge` / `$out` write stages require you to open a transaction yourself. Mongo sources only — a SQL source throws `NativeCommandError` pointing to `executeRaw`; the MongoClient form requires a schema-declared namespace. Returns `{ rows }`.
+
+### Session (Unit of Work)
+
+```js
+await store.session(async (s) => {
+  await s.insert('Order', { ... });
+  await s.update('Account', cond, { ... });
+  await s.executeRaw('pg_main', 'SELECT ... FOR UPDATE', [1]);
+});
+```
+
+- Inside a session, **every command on the same SQL source lands on one transaction connection**: the session commits once on exit, and rolls back as one unit on any exception.
+- **Lazy transaction start**: a session with no commands never checks out a connection.
+- **Cross-source writes fail closed**: if a session writes to ≥2 datasources, it rolls everything back and throws `NonAtomicWriteError` on exit (no distributed transaction — it never commits a half-done unit of work).
+- Mongo sources are probed at runtime (replica set / sharded) and made transactional; on standalone or probe failure they run as-is (non-atomic) and emit one `mongo_transaction_unsupported` (`deployment: standalone|unknown`) feedback event.
+- Sessions nest: an inner scope opens a savepoint (`SAVEPOINT sp_<n>`) on the outer transaction and, on exit, `RELEASE`s it (success) or `ROLLBACK TO`s and releases it (failure) — **an inner failure rolls back only the inner scope while the outer one continues**. When the transaction handle has no savepoint primitives, the nested scope degrades by joining the outer one and emits one `nested_savepoint_unsupported` feedback event.
 
 ### DDL generation
 
@@ -530,11 +547,20 @@ await init({ default: db, pg_a: executors.createConnection('postgres', pgPool) }
 
 ## Transaction boundary
 
-- **Single SQL source**: `mutation` parent-child step sequences and `remove` (archive + delete) run inside one driver transaction on one checked-out connection — any step failure rolls back the whole sequence.
-- **Each SQL write command** is itself atomic: multi-statement plans (e.g. MySQL write + readback) are transaction-wrapped in the executor.
-- **Mongo sources**: single-document writes are atomic; multi-step `mutation` and `remove` execute sequentially and are **not** atomic across steps (Mongo transactions require a replica set). If your consistency requirement spans steps on Mongo, either use an SQL source for those models or add application-level compensation.
+| Scenario | Atomicity |
+|---|---|
+| Single-command API (`insert` / `insertMany` / `updateMany` / `upsert` / `remove` / `count` / `exists`) | Naturally atomic within one SQL source (a single statement); single documents are atomic on Mongo |
+| `store.transaction(source, fn)` | Atomic within one SQL source: every command in the scope shares one connection and one transaction; nested same-source scopes use a savepoint (an inner failure rolls back only that scope) |
+| `store.session(...)` | Atomic **across multiple calls** on one SQL source inside the session; cross-source writes are rejected explicitly (`NonAtomicWriteError`) |
+| Cross-source multi-write without a session | Not atomic (no 2PC / Saga), executed datasource by datasource, and declares `nonAtomic` via the feedback channel (event `non_atomic_write`, with the sources) |
+| Multi-step writes on Mongo | replica set / sharded: atomic on a single Mongo source (session transaction); standalone: non-atomic and explicitly declares `mongo_transaction_unsupported` |
+
+- **Mongo sources**: made transactional inside a session according to the runtime probe; non-transactable ones (standalone / probe failure) run as-is and emit a `mongo_transaction_unsupported` feedback event (`deployment: standalone|unknown`) (degradation is allowed, silent pretence is not).
+- **SQL executors without `openTransaction`**: commands run as-is inside a session and emit a `session_not_atomic` feedback event (degradation is allowed, silent pretence is not).
+- **SQL executors without `withTransaction`**: commands run as-is inside a transaction scope (or the top-level atomic envelope) and emit a `transaction_not_atomic` feedback event (symmetric with `session_not_atomic`; degradation is allowed, silent pretence is not).
 - **Archive idempotency**: `remove` archives with upsert-by-`_id` semantics, so a retry after partial failure no longer fails on duplicate `_id`.
-- **Cross-source steps** (parent and child bound to different datasources) cannot be atomic — they run sequentially by design.
+- Read consistency: only multiple reads inside an explicit session share one transaction connection; reads outside a session do not open an extra transaction.
+- **Cross-source writes (no session)**: a single write call touching ≥2 datasources **cannot be atomic**; it runs sequentially and emits one `non_atomic_write` feedback event (`code: nonAtomic`, with the source list) — degradation is allowed, silence is not. Converge writes onto a single source, or wrap them in `store.session()` (which fails closed on cross-source writes).
 
 ## FAQ
 

package/README.zh-CN.md

@@ -315,10 +315,29 @@ async function transfer() {
 await store.transaction('default', transfer);
 ```
 
-- `store.transaction(source, fn)` 在单个 SQL 源上开启事务作用域：`fn` 内的每个 `executeRaw` / CRUD 调用都落到该源的事务连接，`commit` / `rollback` 作为一个整体（复用内部的 `runInTransaction`）。Mongo 源或不支持事务的执行器会按原样执行 `fn` —— 绝不假装已原子。
+- `store.transaction(source, fn)` 在单个 SQL 源上开启事务作用域：`fn` 内的每个 `executeRaw` / CRUD 调用都落到该源的事务连接，`commit` / `rollback` 作为一个整体（复用内部的 `runInTransaction`）。Mongo 源按**运行时能力探测**（replica set / sharded）以 session 事务执行；standalone 或探测失败则按原样执行 `fn` 并发 `mongo_transaction_unsupported`（`deployment: standalone|unknown`）—— 绝不假装已原子。不支持事务（未实现 `withTransaction`）的执行器亦按原样执行，并发出一条 `transaction_not_atomic` 反馈（允许降级，绝不静默假装已事务化）。同源嵌套 transaction 会开保存点（内层失败只回滚本层）；句柄无保存点原语时降级并入外层并发 `nested_savepoint_unsupported`。
 - `store.executeRaw(source, sql, params, isWrite)` 执行原生 SQL，绕开 GQL 解析与方言翻译。占位符沿用各后端原生风格：MySQL / SQLite 用 `?`，PostgreSQL 用 `$1..$n`。仅限 SQL 源 —— Mongo 源会抛出 `RawSqlError`（`store.RawSqlError`）。
 - `isWrite=false`（默认）返回 `{ rows, affectedRows }` 含结果集行；`isWrite=true` 返回影响行数。
 
+### 会话（Session / 工作单元）
+
+```js
+await store.session(async (s) => {
+  await s.insert('Order', { ... });
+  await s.update('Account', cond, { ... });
+  await s.executeRaw('pg_main', 'SELECT ... FOR UPDATE', [1]);
+});
+```
+
+- 会话内同一 SQL 源的**全部命令落到同一事务连接**：退出统一提交，异常统一回滚；
+- **惰性开事务**：会话内没有任何命令时不占用连接；
+- **跨源写 fail-closed**：同一会话内写入了 ≥2 个数据源时，退出先全部回滚再抛
+  `NonAtomicWriteError`（跨源无分布式事务，绝不提交半截）；
+- Mongo 源按**运行时能力探测**（replica set / sharded）事务化；standalone 或探测失败则按原样执行（非原子），并发出一条 `mongo_transaction_unsupported`（`deployment: standalone|unknown`）反馈；
+- 会话可嵌套：内层作用域在已有事务上开保存点（`SAVEPOINT sp_<n>`），退出按成败
+  `RELEASE`（成功）/ `ROLLBACK TO` + `RELEASE`（失败）——**内层失败只回滚内层，外层可继续**；
+  事务句柄未提供保存点原语时降级并入外层，并发出一条 `nested_savepoint_unsupported` 反馈。
+
 ### DDL 生成
 
 ```js
@@ -527,11 +546,25 @@ await init({ default: db, pg_a: executors.createConnection('postgres', pgPool) }
 
 ## 事务边界
 
-- **单一 SQL 源**：`mutation` 的父子步骤序列与 `remove`（归档 + 删除）在一个已检出的连接上的单个驱动事务内执行 —— 任一步失败会回滚整个序列。
-- **每条 SQL 写入命令**本身即原子：多语句计划（例如 MySQL 的写入 + 回读）在执行器内被事务包裹。
-- **Mongo 源**：单文档写入是原子的；多步 `mutation` 与 `remove` 顺序执行，跨步骤**不**原子（Mongo 事务需要副本集）。若你的跨步骤一致性要求发生在 Mongo 上，请为这些模型改用 SQL 源，或补充应用层补偿。
+| 场景 | 原子性 |
+|---|---|
+| 单命令 API（`insert` / `insertMany` / `updateMany` / `upsert` / `remove` / `count` / `exists`） | 单 SQL 源内天然原子（单条 SQL）；Mongo 单文档原子 |
+| `store.transaction(source, fn)` | 单 SQL 源内原子：作用域内所有命令同连接、同事务；同源嵌套开保存点（内层失败只回滚本层） |
+| `store.session(...)` | 会话内单 SQL 源**跨多次调用**原子；跨源写被显式拦截（`NonAtomicWriteError`） |
+| 无会话的跨源多写 | 非原子（无 2PC / Saga 支持），按数据源顺序执行，并经反馈通道声明 `nonAtomic`（事件 `non_atomic_write`，含涉及源） |
+| Mongo 多步写 | replica set / sharded：单 Mongo 源原子（session 事务）；standalone：非原子并显式声明 `mongo_transaction_unsupported` |
+
+- **Mongo 源**：会话内按运行时能力探测结果事务化；不可事务（standalone / 探测失败）按原样执行，
+  并发出 `mongo_transaction_unsupported` 反馈（`deployment: standalone|unknown`）（允许降级，绝不静默假装已事务化）；
+- **未实现 `openTransaction` 的 SQL 执行器**：会话内按原样执行，并发出
+  `session_not_atomic` 反馈（允许降级，绝不静默假装已事务化）；
+- **未实现 `withTransaction` 的 SQL 执行器**：`store.transaction` / 顶层原子包络内按原样执行，
+  并发出 `transaction_not_atomic` 反馈（与 `session_not_atomic` 对称，允许降级，绝不静默假装已事务化）；
 - **归档幂等**：`remove` 的归档采用按 `_id` upsert 的语义，因此部分失败后的重试不会再因 `_id` 重复而失败。
-- **跨源步骤**（父与子绑定到不同数据源）无法原子 —— 按设计顺序执行。
+- 读一致性：只有在显式会话内的多条读才共享同一事务连接；会话外读不额外开启事务。
+- **跨源写（非会话）**：一次写调用涉及 ≥2 个数据源时**无法原子**，按顺序执行，并发出一条
+  `non_atomic_write` 反馈（`code: nonAtomic`，含涉及源列表）——允许降级、禁止静默。
+  把写收敛到单源，或放入 `store.session()` 内（后者对跨源写直接 fail-closed）。
 
 ## 常见问题
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "nodejs-store",
-  "version": "2.4.0",
+  "version": "2.5.0",
   "description": "Multi-backend data layer for Node.js (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control",
   "main": "src/index.js",
   "files": [

package/src/crud/exec.js

@@ -113,9 +113,14 @@ function _call(fn) {
 // ─── 命令执行（唯一 IO 边界） ────────────────────────────────
 
 /** 在指定数据源上执行命令（Mongo 走原生驱动，SQL 走 translate → exec；
- * 事务作用域内经 datasource.connectionFor 落到事务专用连接） */
+ * 事务 / 会话作用域内经 datasource.resolveConnection 落到事务专用连接） */
 async function _execOn(source, cmd) {
-  const connection = datasource.connectionFor(source);
+  const connection = await datasource.resolveConnection(source, datasource.isWriteCommand(cmd));
+  if (connection && connection.kind === 'mongo') {
+    // Mongo 事务视图：db 按命令 namespace 解析，session 透传给驱动
+    const db = datasource.mongoDb(connection.conn, source, cmd.namespace ?? null);
+    return execMongo(db, cmd, connection.session);
+  }
   const db = datasource.mongoDb(connection, source, cmd.namespace ?? null);
   if (db) {
     return execMongo(db, cmd);
@@ -128,6 +133,64 @@ async function _exec(cmd) {
   return _execOn(cmd.source || datasource.DEFAULT_SOURCE, cmd);
 }
 
+/** 从规划结果中提取数据源集合（探针 / 写 / 查 / 删 / mutation 步骤命令）
+ *
+ * `sources` 必须由规划结果提取，不得写死 `default`（多租户路由场景下的
+ * 源由 core 规划决定）。
+ */
+function sourcesOf(plan) {
+  const out = new Set();
+  for (const key of ['needsProbe', 'command', 'findCommand', 'deleteCommand']) {
+    const cmd = (plan || {})[key] || {};
+    if (cmd && Object.keys(cmd).length > 0) out.add(cmd.source || datasource.DEFAULT_SOURCE);
+  }
+  for (const step of (plan || {}).steps || []) {
+    const cmd = (step || {}).command || {};
+    if (cmd && Object.keys(cmd).length > 0) out.add(cmd.source || datasource.DEFAULT_SOURCE);
+  }
+  return out.size > 0 ? out : new Set([datasource.DEFAULT_SOURCE]);
+}
+
+/** 多源写：无法原子 → 程序化声明 nonAtomic（允许顺序执行，禁止静默） */
+function warnMultiSource(sources) {
+  const listed = Array.from(sources).sort();
+  _emitFeedback({
+    type: 'non_atomic_write',
+    code: 'nonAtomic',
+    layer: 'crud',
+    message: `本次写调用跨 ${listed.length} 个数据源（${listed.join(', ')}）：无法原子，按顺序执行（非原子）`,
+    hint: '把写操作收敛到单源；或在 store.session() 内执行以便跨源写被拒（fail-closed）',
+    sources: listed,
+  });
+}
+
+/**
+ * 顶层 API 调用的原子包络：无会话 + 单一源（SQL 或 Mongo）→ 包事务；否则原样执行
+ *
+ *   - 会话内：事务边界由会话统一管理，直接执行（不嵌套）；
+ *   - 单一 SQL 源：包事务（原子）；
+ *   - 多源：无法原子 → 程序化声明 `nonAtomic`（反馈通道），再按顺序原样执行；
+ *   - 单一 Mongo 源：按探测结果包 session 事务或降级声明（见 `runInTransaction`）；
+ *   - 未配置源：按原样执行；
+ *   - sources 由调用方从「规划结果」中提取（`sourcesOf`），命令源与事务源一致。
+ */
+async function runAtomic(sources, fn) {
+  if (datasource.currentSession() !== null) return fn();
+  const uniq = new Set(Array.from(sources, (s) => s || datasource.DEFAULT_SOURCE));
+  if (uniq.size === 1) {
+    const [source] = uniq;
+    if (
+      datasource.hasConnection(source) &&
+      (datasource.isSql(source) || datasource.isMongoSource(source))
+    ) {
+      return datasource.runInTransaction(source, fn);
+    }
+  } else if (uniq.size > 1) {
+    warnMultiSource(uniq);
+  }
+  return fn();
+}
+
 /** 深度替换命令中的占位符（命中 resolver 返回非字符串时替换） */
 function _substitute(value, resolver) {
   if (typeof value === 'string') return resolver(value);
@@ -172,6 +235,9 @@ module.exports = {
   _PROFILE_HINT,
   _exec,
   _execOn,
+  runAtomic,
+  warnMultiSource,
+  sourcesOf,
   _substitute,
   resolvePlaceholders,
 };

package/src/crud/mutation.js

@@ -5,9 +5,8 @@
  */
 
 const { core: _core, get: _getSchema } = require('../schema');
-const datasource = require('../datasource');
 const { emit: _emitFeedback } = require('../feedback');
-const { _call, _ctx, _exec, _nowFor, resolvePlaceholders } = require('./exec');
+const { _call, _ctx, _exec, _nowFor, resolvePlaceholders, runAtomic, sourcesOf } = require('./exec');
 const { _generateId, _newIdPool } = require('./id');
 
 /** mutation 单条：规划步骤序列 → 依序执行 + 父子 _id 占位符回填 */
@@ -36,11 +35,8 @@ async function _mutationOne(schemaName, data, now, routeOverride = null) {
 
   // 单一 SQL 源 → 步骤序列整体事务化（同连接同事务，任一步失败整体回滚）；
   // Mongo 源 / 跨源步骤按原样顺序执行（非原子边界见 README「事务边界」）
-  const sources = [...new Set(plan.steps.map((s) => s.command.source || datasource.DEFAULT_SOURCE))];
-  if (sources.length === 1 && datasource.isSql(sources[0])) {
-    return datasource.runInTransaction(sources[0], runSteps);
-  }
-  return runSteps();
+  const sources = sourcesOf(plan);
+  return runAtomic(sources, runSteps);
 }
 
 /**

package/src/crud/write.js

@@ -5,8 +5,7 @@
  */
 
 const { core: _core, get: _getSchema } = require('../schema');
-const datasource = require('../datasource');
-const { _call, _ctx, _exec, _nowFor } = require('./exec');
+const { _call, _ctx, _exec, _nowFor, runAtomic, sourcesOf } = require('./exec');
 const { _generateId } = require('./id');
 
 /** creator 写权限探针：先规划，若 needsProbe 则执行探针命令后重入 */
@@ -52,13 +51,31 @@ async function insertMany(schemaName, docs, routeOverride = null) {
  *
  * data 的 key 以 '$' 开头 → 原生 MongoDB 操作符（$set/$inc/$unset 等）直接透传。
  * 否则自动包装为 $set 模式。`routeOverride` 可选（多租户路由）。
+ *
+ * 「权限探针 + 写」整体纳入同一原子作用域（`runAtomic`）：单一 SQL 源时探针与写
+ * 同连接同事务，消除二者之间的并发窗口；`now` 只取一次，两次规划共用（调用级确定性）。
  */
 async function update(schemaName, condition, data, options = null, routeOverride = null) {
-  const out = await _planWithProbe((found, doc) => _call(() =>
-    _core.planUpdate(schemaName, condition ?? null, data ?? null, options ?? null, _nowFor(schemaName), _ctx(),
-      found, doc, routeOverride)));
-  const result = await _exec(out.command);
-  return result ? _call(() => _core.applyWriteDefaults(schemaName, result)) : null;
+  const now = _nowFor(schemaName);
+  const ctx = _ctx();
+  const first = _call(() =>
+    _core.planUpdate(schemaName, condition ?? null, data ?? null, options ?? null, now, ctx,
+      null, null, routeOverride));
+  const sources = sourcesOf(first);
+
+  const doRun = async () => {
+    let out = first;
+    if (out.needsProbe) {
+      const probeDoc = await _exec(out.needsProbe);
+      out = _call(() =>
+        _core.planUpdate(schemaName, condition ?? null, data ?? null, options ?? null, now, ctx,
+          probeDoc !== null && probeDoc !== undefined, probeDoc ?? null, routeOverride));
+    }
+    const result = await _exec(out.command);
+    return result ? _call(() => _core.applyWriteDefaults(schemaName, result)) : null;
+  };
+
+  return runAtomic(sources, doRun);
 }
 
 /** 批量更新（支持原生操作符） */
@@ -91,13 +108,8 @@ async function remove(schemaName, condition, routeOverride = null) {
     return { deletedCount: result.deletedCount, archivedCount };
   };
 
-  const sources = new Set([out.deleteCommand.source || datasource.DEFAULT_SOURCE]);
-  if (out.findCommand) sources.add(out.findCommand.source || datasource.DEFAULT_SOURCE);
-  const arr = [...sources];
-  if (arr.length === 1 && datasource.isSql(arr[0])) {
-    return datasource.runInTransaction(arr[0], doRemove);
-  }
-  return doRemove();
+  const sources = sourcesOf(out);
+  return runAtomic(sources, doRemove);
 }
 
 /** 判断是否存在 */