nodejs-store 2.5.0 → 2.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +173 -0
- package/README.zh-CN.md +162 -0
- package/llms-full.txt +71 -0
- package/llms.txt +1 -0
- package/package.json +2 -2
- package/src/ask.js +358 -0
- package/src/ask_knowledge.md +250 -0
- package/src/crud/write.js +61 -5
- package/src/ddl.js +250 -12
- package/src/executors/index.js +15 -2
- package/src/executors/mongo.js +18 -3
- package/src/executors/mysql.js +5 -1
- package/src/executors/sqlite.js +7 -2
- package/src/feedback.js +6 -1
- package/src/index.js +82 -25
- package/src/llm.js +157 -0
- package/src/permission.js +58 -0
- package/src/profile.js +35 -0
- package/src/schema.js +19 -2
- package/src/workflow.js +734 -0
package/README.md
CHANGED
|
@@ -35,6 +35,7 @@
|
|
|
35
35
|
- [Schema reference](#schema-reference)
|
|
36
36
|
- [Advanced API](#advanced-api)
|
|
37
37
|
- [Transactions](#transaction-boundary)
|
|
38
|
+
- [Transactional capabilities](#transactional-capabilities)
|
|
38
39
|
- [FAQ](#faq)
|
|
39
40
|
- [Related projects](#related-projects)
|
|
40
41
|
|
|
@@ -210,6 +211,9 @@ GQL tree queries compile to a single native query per backend — never hand-wri
|
|
|
210
211
|
- **Permission context** — `AsyncLocalStorage`-based roles (`super_admin`/`admin`/`guest`/`creator`...), schema/field-level read/write whitelists, automatic owner-condition injection.
|
|
211
212
|
- **Multi-datasource & multi-tenant** — locate a schema by `(source, namespace, collection)`; re-target per request with a route override.
|
|
212
213
|
- **Async-first, Rust core** — built on the `mongodb` Node.js driver and a shared Rust core with SQL dialects.
|
|
214
|
+
- **Relation predicates in mutations** — filter `update` / `remove` by related-table fields, pushed down to all four backends (previously a silent no-op on MongoDB).
|
|
215
|
+
- **Autoincrement primary keys** — declare `_id` as `{ type: 'int', strategy: 'autoincrement' }` for database-assigned integer IDs, with explicit errors where autoincrement is impossible.
|
|
216
|
+
- **Index DDL** — `schema.indexes` compiles to real `CREATE [UNIQUE] INDEX` statements (per backend, byte-identical); the generator still only emits text.
|
|
213
217
|
|
|
214
218
|
## GQL syntax
|
|
215
219
|
|
|
@@ -519,6 +523,97 @@ store.setFeedbackSink((e) => logger.warn({ code: e.code }, e.hint));
|
|
|
519
523
|
Non-pushdownable commands also throw `PushdownUnsupportedError` — catch it to re-run that
|
|
520
524
|
segment against a Mongo source.
|
|
521
525
|
|
|
526
|
+
### `store.ask(question, opts)`
|
|
527
|
+
|
|
528
|
+
AI ask — natural-language query (L1, read-only). The question plus a permission-filtered
|
|
529
|
+
schema summary (`store.describeForAi`) go to a pluggable LLM, which must answer with a
|
|
530
|
+
single `{"gql","params"}` JSON object; that query is then planned and executed inside the
|
|
531
|
+
hardened `text2query` profile (read-only, row/depth caps, route override disabled).
|
|
532
|
+
Structured failures are fed back to the LLM for retry (up to `maxRetries` rounds); on
|
|
533
|
+
exhaustion `AskExhausted` is thrown — ask never silently degrades and never returns
|
|
534
|
+
empty data for a failed query.
|
|
535
|
+
|
|
536
|
+
LLM output is treated as untrusted input: the guardrails (profile, context enforcement,
|
|
537
|
+
read-only) are hardcoded server-side and unreachable by the model.
|
|
538
|
+
|
|
539
|
+
| Option | Type | Meaning |
|
|
540
|
+
| --- | --- | --- |
|
|
541
|
+
| `llm` | `string \| Function` | required; registry name (see [`llm`](#llm-registry-llm) below) or an `async (messages) => string` client |
|
|
542
|
+
| `ctx` | object | required; server-side user context `{ userId, roles }`; a missing ctx is rejected (fail-secure) and the context never enters any LLM message |
|
|
543
|
+
| `maxRetries` | number (default `3`) | retries after a failed attempt (total attempts ≤ `1 + maxRetries`) |
|
|
544
|
+
| `knowledge` | string | override the system-prompt knowledge text (default: bundled `ask_knowledge.md`) |
|
|
545
|
+
|
|
546
|
+
Resolves to an `AskResult` `{ data, attempts, events }`. Each attempt is
|
|
547
|
+
`{ llmRaw, gql, params, rows }` — the successful round carries no `error` key; failed
|
|
548
|
+
rounds carry `{ error: { code, message } }` with `code` ∈ `badLlmOutput`,
|
|
549
|
+
`profileBlocked`, `permissionDenied`, `planError`. LLM client faults (`LlmError`:
|
|
550
|
+
network / HTTP status / empty content) propagate untouched — link failures fail
|
|
551
|
+
explicitly and are not retried.
|
|
552
|
+
|
|
553
|
+
```js
|
|
554
|
+
const { llm } = require('nodejs-store');
|
|
555
|
+
|
|
556
|
+
llm.registerLlm('deepseek', llm.makeOpenaiCompat({
|
|
557
|
+
baseUrl: 'https://api.deepseek.com/v1',
|
|
558
|
+
model: 'deepseek-chat',
|
|
559
|
+
apiKey: process.env.DEEPSEEK_API_KEY,
|
|
560
|
+
}));
|
|
561
|
+
|
|
562
|
+
const result = await store.ask('Total amount of my last 10 orders?', {
|
|
563
|
+
llm: 'deepseek',
|
|
564
|
+
ctx: { userId: 'u1', roles: ['viewer'] },
|
|
565
|
+
});
|
|
566
|
+
console.log(result.data); // query result rows
|
|
567
|
+
console.log(result.attempts.at(-1).gql); // the GQL the model produced
|
|
568
|
+
```
|
|
569
|
+
|
|
570
|
+
> The `text2query` profile is a process-wide core singleton: concurrent `ask()` calls in
|
|
571
|
+
> one process interleave. Serialize asks per process, or run each in its own worker.
|
|
572
|
+
|
|
573
|
+
### `store.describeForAi(ctx?)`
|
|
574
|
+
|
|
575
|
+
Permission-filtered schema summary for LLM prompts — a compact JSON array with one
|
|
576
|
+
`{ name, fields, relations, computes }` entry per model. Without a context only model
|
|
577
|
+
and field *names* are exposed (no types — probe-resistant). With a context, models,
|
|
578
|
+
fields, relations and computed columns are filtered by the caller's role
|
|
579
|
+
(`canRead` / `readableFields` / `readableRelations` / `readableComputes`), archive
|
|
580
|
+
tables (`*Deleted`) are dropped, and ops details (indexes / datasource / namespace)
|
|
581
|
+
never enter the prompt.
|
|
582
|
+
|
|
583
|
+
```js
|
|
584
|
+
const summary = store.describeForAi({ userId: 'u1', roles: ['viewer'] });
|
|
585
|
+
// [{ name: 'Order', fields: { _id: 'string', code: 'string', ... }, relations: { ... }, computes: { ... } }]
|
|
586
|
+
```
|
|
587
|
+
|
|
588
|
+
On native bindings that lack the `readableComputes` verdict, computed columns with a
|
|
589
|
+
`read` whitelist are conservatively excluded from the summary and an
|
|
590
|
+
`askSummaryComputeSkipped` feedback event is emitted — narrow the exposure, never
|
|
591
|
+
over-disclose.
|
|
592
|
+
|
|
593
|
+
### LLM registry (`llm`)
|
|
594
|
+
|
|
595
|
+
Pluggable LLM clients with no SDK bundled. A client is a single function
|
|
596
|
+
`async (messages: Array<{ role, content }>) => string`:
|
|
597
|
+
|
|
598
|
+
```js
|
|
599
|
+
const { llm } = require('nodejs-store');
|
|
600
|
+
|
|
601
|
+
llm.registerLlm('deepseek', client); // duplicate name → throws
|
|
602
|
+
llm.getLlm('deepseek'); // unregistered name → throws
|
|
603
|
+
|
|
604
|
+
llm.makeOpenaiCompat({ // OpenAI-compatible factory (global fetch, zero deps);
|
|
605
|
+
baseUrl, model, apiKey, // covers DeepSeek / OpenAI / Moonshot / Ollama / ...
|
|
606
|
+
jsonMode: true, // false → omit response_format (prompt convention + strict parse fallback)
|
|
607
|
+
effort: 'low', // null → omit reasoning_effort
|
|
608
|
+
maxTokens: 4096,
|
|
609
|
+
timeoutMs: 90000,
|
|
610
|
+
});
|
|
611
|
+
```
|
|
612
|
+
|
|
613
|
+
Client faults throw `LlmError` with a structured `.detail` (`code` ∈ `llmNetworkError`,
|
|
614
|
+
`llmHttpError`, `llmEmptyContent`, `llmJsonPromptMissing`); `store.ask` propagates them
|
|
615
|
+
untouched.
|
|
616
|
+
|
|
522
617
|
### Low-level modules
|
|
523
618
|
|
|
524
619
|
The package re-exports its building blocks for advanced hosts:
|
|
@@ -562,6 +657,84 @@ await init({ default: db, pg_a: executors.createConnection('postgres', pgPool) }
|
|
|
562
657
|
- Read consistency: only multiple reads inside an explicit session share one transaction connection; reads outside a session do not open an extra transaction.
|
|
563
658
|
- **Cross-source writes (no session)**: a single write call touching ≥2 datasources **cannot be atomic**; it runs sequentially and emits one `non_atomic_write` feedback event (`code: nonAtomic`, with the source list) — degradation is allowed, silence is not. Converge writes onto a single source, or wrap them in `store.session()` (which fails closed on cross-source writes).
|
|
564
659
|
|
|
660
|
+
## Transactional capabilities
|
|
661
|
+
|
|
662
|
+
Capabilities aimed at transactional workloads (orders, inventory — write contention plus
|
|
663
|
+
complex reads). Full details, semantics and the explicit-error list:
|
|
664
|
+
**[doc/transaction-capabilities.md](doc/transaction-capabilities.md)** ·
|
|
665
|
+
[中文](doc/transaction-capabilities.zh-CN.md).
|
|
666
|
+
|
|
667
|
+
- **Relation predicates in mutations** — `updateMany('Inventory', { product: { category: 'meat' } }, { $inc: { stock: 10 } })`: condition keys matching a declared relation become a semi/anti-join, normalized into a preCommand (aggregate fetching `_id`s) plus `_id $in`.
|
|
668
|
+
- **`$group by` one-relation paths** — `by: ['product.category']` compiles to `$lookup`+`$unwind` (Mongo) / `LEFT JOIN` (SQL); `many` paths fail explicitly (fan-out breaks count semantics).
|
|
669
|
+
- **Autoincrement PKs** — `_id: { type: 'int', strategy: 'autoincrement' }`; PG/SQLite read back via `INSERT…RETURNING`, MySQL via insertId; MongoDB and `insertMany` fail explicitly with `AUTOINCREMENT_NOT_SUPPORTED` (no silent ObjectId substitution).
|
|
670
|
+
- **Index DDL** — `schema.indexes` (MongoDB shape) → `CREATE [UNIQUE] INDEX idx_<table>_<cols>` in `ddl.generate`, byte-identical across MySQL/PostgreSQL/SQLite.
|
|
671
|
+
- **Declarative migration** — `ddl.diffDefs(old, new)` + `ddl.generateMigration(backend, old, new)`: whitelist-only (add table/column/index, type widening), per-dialect SQL, pure functions; destructive changes fail with `MIGRATION_UNSUPPORTED`.
|
|
672
|
+
|
|
673
|
+
## Workflow orchestration (first batch)
|
|
674
|
+
|
|
675
|
+
Express "orchestration of multi-step data operations" as data: a workflow definition (defn) is pure
|
|
676
|
+
JSON isomorphic to a schema defn, and each run is persisted to the built-in schema `__workflowRun`
|
|
677
|
+
(queryable with plain GQL — zero new observability endpoints). Execution generalizes the existing
|
|
678
|
+
mutation step-sequence mechanism: linear steps + per-step `when` guards + fail-fast. The engine
|
|
679
|
+
lives in the host layer (`src/workflow.js`), core unchanged; aligned with
|
|
680
|
+
`nodejs-store/src/workflow.js` (byte-identical outputs guarded by parity anchor tests).
|
|
681
|
+
|
|
682
|
+
```python
|
|
683
|
+
from py_store import workflow
|
|
684
|
+
|
|
685
|
+
store.registerWorkflow({
|
|
686
|
+
'name': 'placeOrder',
|
|
687
|
+
'run': ['admin', 'ops'], # three-tier whitelists read/write/run (run falls back to write)
|
|
688
|
+
'steps': [
|
|
689
|
+
{'op': 'query', 'as': 'inv',
|
|
690
|
+
'gql': 'Inventory($condition:@c0){_id, stock}',
|
|
691
|
+
'params': {'c0': {'productId': '{{input.productId}}', 'warehouse': '{{input.warehouse}}'}}},
|
|
692
|
+
{'op': 'fail', 'when': {'exists': '{{inv._id}}', 'is': None}, 'message': '库存记录不存在'},
|
|
693
|
+
{'op': 'fail', 'when': {'lt': '{{inv.stock}}', 'than': '{{input.qty}}'}, 'message': '库存不足'},
|
|
694
|
+
{'op': 'mutation', 'model': 'Inventory',
|
|
695
|
+
'data': {'_id': '{{inv._id}}', 'stock': '{{dec:{{inv.stock}},{{input.qty}}}}'}},
|
|
696
|
+
],
|
|
697
|
+
})
|
|
698
|
+
|
|
699
|
+
run = await store.runWorkflow('placeOrder', { productId: 'p1', warehouse: 'w1', qty: 30 })
|
|
700
|
+
# run['status'] ∈ succeeded | failed | rejected | drySucceeded | dryFailed
|
|
701
|
+
# Uniform contract: business failures never raise; the error lives in run['error']
|
|
702
|
+
# (set only on failure; always null on success — never `||`-masked downstream)
|
|
703
|
+
```
|
|
704
|
+
|
|
705
|
+
- **Step whitelist** (three kinds; anything else fails registration with `WORKFLOW_UNSUPPORTED`):
|
|
706
|
+
`query` (result must be unique — >1 row is an explicit error), `mutation` (store.mutation /
|
|
707
|
+
upsert), `fail` (explicit business assertion). Optional `when` guards (exists / is / eq / ne /
|
|
708
|
+
lt / lte / gt / gte) record `skipped` explicitly — never silently skipped.
|
|
709
|
+
- **Placeholders**: `{{input.<path>}}`, `{{<as>.<path>}}` (forward references only),
|
|
710
|
+
`{{dec:<a>,<b>}}`; full-string replacement keeps the value type. No placeholders inside gql
|
|
711
|
+
(bind via params — injection safety); no array-index path segments.
|
|
712
|
+
- **Permissions**: three-tier role whitelists embedded in the defn (same RBAC semantics: admin /
|
|
713
|
+
super_admin bypass, guest denied, internal bypass); runs inherit the caller's Context and every
|
|
714
|
+
step goes through core permission checks — no superuser. `require_context(true)` rejects
|
|
715
|
+
context-less runs (fail-secure wins over dry-run); rejected runs are persisted for audit.
|
|
716
|
+
- **Atomicity**: a single-source run is atomic across steps (outer `run_atomic` wraps the whole
|
|
717
|
+
loop, inner mutations nest into it); multi-source / prescan-failed runs execute sequentially and
|
|
718
|
+
emit feedback events (`workflow_non_atomic` / `workflow_prescan_failed`) — never silent. The run
|
|
719
|
+
record (running → terminal) is committed outside the business transaction so failed runs stay
|
|
720
|
+
queryable after rollback.
|
|
721
|
+
- **dry-run**: `store.runWorkflow(name, input, { dryRun: true })` — query steps execute for real (read-only
|
|
722
|
+
safe); mutation / fail are recorded as `wouldRun` (`drySucceeded | dryFailed`).
|
|
723
|
+
- **Run persistence**: `__workflowRun` is bootstrapped on import (idempotent); SQL backends need a
|
|
724
|
+
one-time `ddl.generate(backend, ['__workflowRun'])` (Mongo creates the collection on first write).
|
|
725
|
+
Its `write` whitelist is explicitly empty (GQL tampering with run audit is rejected by R2).
|
|
726
|
+
|
|
727
|
+
### Explicitly not in the first batch (detected → error; boundaries shipped with the same weight as features)
|
|
728
|
+
|
|
729
|
+
| Not supported | Why | Escape hatch |
|
|
730
|
+
|---|---|---|
|
|
731
|
+
| Loops / parallel / sub-workflows / human approval | DAG & wait semantics explode; linear + `when` covers the first batch | orchestrate in host code via the store API |
|
|
732
|
+
| Auto compensation (Saga) / auto retry | Inverse-operation burden; steps have no automatic idempotency | inspect run records and handle explicitly |
|
|
733
|
+
| Per-step host callbacks | Arbitrary code breaks whitelist governance | schema computes (read) / host code (write) |
|
|
734
|
+
| Workflow defn persistence / hot reload | Depends on schema versioning (next on the roadmap) | defn stays code-side JSON + register, like schemas today |
|
|
735
|
+
| Timers / event triggers | Scheduling is a resident-IO concern, orthogonal to pure orchestration | call `runWorkflow` from the application layer |
|
|
736
|
+
| Placeholders inside gql / array-index paths | Injection surface / per-row iteration semantics | params binding / host-code orchestration |
|
|
737
|
+
|
|
565
738
|
## FAQ
|
|
566
739
|
|
|
567
740
|
**How do I use one schema for both MongoDB and PostgreSQL in Node.js?**
|
package/README.zh-CN.md
CHANGED
|
@@ -34,6 +34,7 @@
|
|
|
34
34
|
- [Schema 参考](#schema-参考)
|
|
35
35
|
- [高级 API](#高级-api)
|
|
36
36
|
- [事务边界](#事务边界)
|
|
37
|
+
- [事务型能力](#事务型能力)
|
|
37
38
|
- [常见问题](#常见问题)
|
|
38
39
|
- [相关项目](#相关项目)
|
|
39
40
|
|
|
@@ -209,6 +210,9 @@ GQL 树查询会编译为每个后端一条原生查询 —— 再也不必手
|
|
|
209
210
|
- **权限上下文** —— 基于 `AsyncLocalStorage` 的角色(`super_admin`/`admin`/`guest`/`creator`...)、schema/字段级读写白名单、自动属主条件注入。
|
|
210
211
|
- **多数据源 & 多租户** —— 通过 `(source, namespace, collection)` 定位 schema;按请求用路由覆盖重新定向。
|
|
211
212
|
- **异步优先,Rust 核心** —— 基于 `mongodb` Node.js 驱动与共享的 Rust 核心(含 SQL 方言)。
|
|
213
|
+
- **mutation 关系谓词** —— `update` / `remove` 按关联表字段过滤,下推到全部四个后端(此前 MongoDB 侧是静默 no-op)。
|
|
214
|
+
- **自增主键** —— `_id` 声明 `{ type: 'int', strategy: 'autoincrement' }` 即用数据库自增整数 ID;做不到自增的场景显式报错。
|
|
215
|
+
- **索引 DDL** —— `schema.indexes` 编译为真实 `CREATE [UNIQUE] INDEX` 语句(按后端、逐字节一致);生成器仍只产文本。
|
|
212
216
|
|
|
213
217
|
## GQL 语法
|
|
214
218
|
|
|
@@ -518,6 +522,91 @@ store.setFeedbackSink((e) => logger.warn({ code: e.code }, e.hint));
|
|
|
518
522
|
不可下推的命令还会抛出 `PushdownUnsupportedError` —— 捕获它即可把该片段
|
|
519
523
|
改投到某个 Mongo 源重跑。
|
|
520
524
|
|
|
525
|
+
### `store.ask(question, opts)`
|
|
526
|
+
|
|
527
|
+
AI 问数——自然语言查询(L1,只读)。问题与一份按权限过滤的 schema 摘要
|
|
528
|
+
(`store.describeForAi`)一起交给可插拔的 LLM,LLM 必须返回唯一的
|
|
529
|
+
`{"gql","params"}` JSON 对象;该查询再经加固的 `text2query` 档位规划并执行
|
|
530
|
+
(只读、行数/深度硬限、路由覆盖禁用)。结构化失败会回喂 LLM 重试(至多
|
|
531
|
+
`maxRetries` 轮);耗尽抛 `AskExhausted`——问数失败就是失败,不静默降级、
|
|
532
|
+
不返回空结果。
|
|
533
|
+
|
|
534
|
+
LLM 输出永远当不可信输入:护栏(档位、上下文强制、只读)在服务端硬编码,
|
|
535
|
+
模型不可触达。
|
|
536
|
+
|
|
537
|
+
| 参数 | 类型 | 含义 |
|
|
538
|
+
| --- | --- | --- |
|
|
539
|
+
| `llm` | `string \| Function` | 必填;注册名(见下方 [`llm` 注册表](#llm-注册表))或符合 `async (messages) => string` 协议的客户端 |
|
|
540
|
+
| `ctx` | object | 必填;服务端构造的用户上下文 `{ userId, roles }`;缺失直接拒绝(fail-secure),且上下文绝不进入任何 LLM 消息 |
|
|
541
|
+
| `maxRetries` | number(默认 `3`) | 失败后的最大重试次数(总尝试 ≤ `1 + maxRetries`) |
|
|
542
|
+
| `knowledge` | string | 覆盖 system prompt 知识文本(缺省用包内 `ask_knowledge.md`) |
|
|
543
|
+
|
|
544
|
+
返回 `AskResult` `{ data, attempts, events }`。每次尝试为
|
|
545
|
+
`{ llmRaw, gql, params, rows }`——成功轮不含任何 `error` 键;失败轮携带
|
|
546
|
+
`{ error: { code, message } }`,`code` ∈ `badLlmOutput`、`profileBlocked`、
|
|
547
|
+
`permissionDenied`、`planError`。LLM 客户端自身故障(`LlmError`:网络 /
|
|
548
|
+
HTTP 状态 / 空 content)原样穿透——链路故障显式失败,不重试。
|
|
549
|
+
|
|
550
|
+
```js
|
|
551
|
+
const { llm } = require('nodejs-store');
|
|
552
|
+
|
|
553
|
+
llm.registerLlm('deepseek', llm.makeOpenaiCompat({
|
|
554
|
+
baseUrl: 'https://api.deepseek.com/v1',
|
|
555
|
+
model: 'deepseek-chat',
|
|
556
|
+
apiKey: process.env.DEEPSEEK_API_KEY,
|
|
557
|
+
}));
|
|
558
|
+
|
|
559
|
+
const result = await store.ask('我最近 10 笔订单的金额合计是多少?', {
|
|
560
|
+
llm: 'deepseek',
|
|
561
|
+
ctx: { userId: 'u1', roles: ['viewer'] },
|
|
562
|
+
});
|
|
563
|
+
console.log(result.data); // 查询结果行
|
|
564
|
+
console.log(result.attempts.at(-1).gql); // 模型产出的 GQL
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
> `text2query` 档位是进程级 core 单例:同一进程内并发调用 `ask()` 会互相串扰。
|
|
568
|
+
> 请在进程内串行化问数,或每个任务独享进程/worker。
|
|
569
|
+
|
|
570
|
+
### `store.describeForAi(ctx?)`
|
|
571
|
+
|
|
572
|
+
输出给 LLM prompt 用的权限过滤 schema 摘要——紧凑 JSON 数组,每模型一条
|
|
573
|
+
`{ name, fields, relations, computes }`。无上下文时仅暴露模型名与字段名
|
|
574
|
+
(不暴露类型细节,防探针)。有上下文时,模型/字段/关系/计算列按调用者角色过滤
|
|
575
|
+
(`canRead` / `readableFields` / `readableRelations` / `readableComputes`),
|
|
576
|
+
归档表(`*Deleted`)排除,运维细节(indexes / datasource / namespace)不进 prompt。
|
|
577
|
+
|
|
578
|
+
```js
|
|
579
|
+
const summary = store.describeForAi({ userId: 'u1', roles: ['viewer'] });
|
|
580
|
+
// [{ name: 'Order', fields: { _id: 'string', code: 'string', ... }, relations: { ... }, computes: { ... } }]
|
|
581
|
+
```
|
|
582
|
+
|
|
583
|
+
原生绑定未导出 `readableComputes` 判决时,配置了 `read` 白名单的计算列会保守
|
|
584
|
+
排除出摘要,并 emit `askSummaryComputeSkipped` 反馈事件——宁缺勿泄。
|
|
585
|
+
|
|
586
|
+
### `llm` 注册表
|
|
587
|
+
|
|
588
|
+
可插拔 LLM 客户端,不内置任何 SDK。客户端就是一个函数
|
|
589
|
+
`async (messages: Array<{ role, content }>) => string`:
|
|
590
|
+
|
|
591
|
+
```js
|
|
592
|
+
const { llm } = require('nodejs-store');
|
|
593
|
+
|
|
594
|
+
llm.registerLlm('deepseek', client); // 同名重复注册 → 报错
|
|
595
|
+
llm.getLlm('deepseek'); // 未注册 → 报错
|
|
596
|
+
|
|
597
|
+
llm.makeOpenaiCompat({ // OpenAI 兼容通用工厂(全局 fetch,零依赖),
|
|
598
|
+
baseUrl, model, apiKey, // 覆盖 DeepSeek / OpenAI / Moonshot / Ollama 等
|
|
599
|
+
jsonMode: true, // false → 不传 response_format(prompt 约定 + 严格解析回喂)
|
|
600
|
+
effort: 'low', // null → 不传 reasoning_effort
|
|
601
|
+
maxTokens: 4096,
|
|
602
|
+
timeoutMs: 90000,
|
|
603
|
+
});
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
客户端故障抛 `LlmError`(`.detail` 结构化,`code` ∈ `llmNetworkError`、
|
|
607
|
+
`llmHttpError`、`llmEmptyContent`、`llmJsonPromptMissing`);`store.ask`
|
|
608
|
+
对其原样穿透。
|
|
609
|
+
|
|
521
610
|
### 底层模块
|
|
522
611
|
|
|
523
612
|
该包会重导出其构建模块,供高级宿主使用:
|
|
@@ -553,6 +642,7 @@ await init({ default: db, pg_a: executors.createConnection('postgres', pgPool) }
|
|
|
553
642
|
| `store.session(...)` | 会话内单 SQL 源**跨多次调用**原子;跨源写被显式拦截(`NonAtomicWriteError`) |
|
|
554
643
|
| 无会话的跨源多写 | 非原子(无 2PC / Saga 支持),按数据源顺序执行,并经反馈通道声明 `nonAtomic`(事件 `non_atomic_write`,含涉及源) |
|
|
555
644
|
| Mongo 多步写 | replica set / sharded:单 Mongo 源原子(session 事务);standalone:非原子并显式声明 `mongo_transaction_unsupported` |
|
|
645
|
+
| 工作流 run(`runWorkflow`) | 单源 run **跨步骤**整体原子(外层 `runAtomic` 包住步骤循环、内层 mutation 嵌套并入);多源 / 源预扫失败按顺序执行并经反馈通道声明非原子 |
|
|
556
646
|
|
|
557
647
|
- **Mongo 源**:会话内按运行时能力探测结果事务化;不可事务(standalone / 探测失败)按原样执行,
|
|
558
648
|
并发出 `mongo_transaction_unsupported` 反馈(`deployment: standalone|unknown`)(允许降级,绝不静默假装已事务化);
|
|
@@ -566,6 +656,78 @@ await init({ default: db, pg_a: executors.createConnection('postgres', pgPool) }
|
|
|
566
656
|
`non_atomic_write` 反馈(`code: nonAtomic`,含涉及源列表)——允许降级、禁止静默。
|
|
567
657
|
把写收敛到单源,或放入 `store.session()` 内(后者对跨源写直接 fail-closed)。
|
|
568
658
|
|
|
659
|
+
## 事务型能力
|
|
660
|
+
|
|
661
|
+
面向事务型业务场景(订单、库存——写竞争 + 复杂读)的能力增补。完整语义、用法与显式报错清单:
|
|
662
|
+
**[doc/transaction-capabilities.zh-CN.md](doc/transaction-capabilities.zh-CN.md)** ·
|
|
663
|
+
[English](doc/transaction-capabilities.md).
|
|
664
|
+
|
|
665
|
+
- **mutation 关系谓词** —— `updateMany('Inventory', { product: { category: 'meat' } }, { $inc: { stock: 10 } })`:条件键命中已声明关系即 semi/anti-join,归一为 preCommand(aggregate 取 `_id`)+ `_id $in`。
|
|
666
|
+
- **`$group by` one 关系路径** —— `by: ['product.category']` 编译为 `$lookup`+`$unwind`(Mongo)/ `LEFT JOIN`(SQL);many 路径显式报错(扇出破坏计数语义)。
|
|
667
|
+
- **自增主键** —— `_id: { type: 'int', strategy: 'autoincrement' }`;PG/SQLite 经 `INSERT…RETURNING` 回读、MySQL 经 insertId;MongoDB 与 `insertMany` 显式报 `AUTOINCREMENT_NOT_SUPPORTED`(禁 ObjectId 静默顶替)。
|
|
668
|
+
- **索引 DDL** —— `schema.indexes`(Mongo 形态)→ `ddl.generate` 产出 `CREATE [UNIQUE] INDEX idx_<表>_<字段>`,MySQL/PostgreSQL/SQLite 三方言逐字节一致。
|
|
669
|
+
- **声明式迁移** —— `ddl.diffDefs(old, new)` + `ddl.generateMigration(backend, old, new)`:白名单制(加表/加列/加索引/类型放宽),纯函数按方言产 SQL;白名单外显式报 `MIGRATION_UNSUPPORTED`。
|
|
670
|
+
|
|
671
|
+
## 工作流编排(首批)
|
|
672
|
+
|
|
673
|
+
把「多步数据操作的编排」用数据表达:工作流定义(Workflow defn)是与 schema defn 同构的纯 JSON、
|
|
674
|
+
运行记录(run)落库为内建 schema `__workflowRun`(普通 GQL 即查,可观测性零新接口)。执行是既有
|
|
675
|
+
mutation 步骤序列机制的推广——线性步骤 + 步骤级 `when` 守卫 + fail-fast。引擎落在宿主层
|
|
676
|
+
(`src/workflow.js`),core 零改动;对齐 `py-store/src/py_store/workflow.py`(双宿主输出逐字节
|
|
677
|
+
一致由 parity 锚单测守护)。
|
|
678
|
+
|
|
679
|
+
```js
|
|
680
|
+
store.registerWorkflow({ // 注册即静态校验;白名单外显式 Err(WORKFLOW_UNSUPPORTED)
|
|
681
|
+
name: 'placeOrder',
|
|
682
|
+
run: ['admin', 'ops'], // 三级白名单 read/write/run(run 缺省回退 write)
|
|
683
|
+
steps: [
|
|
684
|
+
{ op: 'query', as: 'inv',
|
|
685
|
+
gql: 'Inventory($condition:@c0){_id, stock}',
|
|
686
|
+
params: { c0: { productId: '{{input.productId}}', warehouse: '{{input.warehouse}}' } } },
|
|
687
|
+
{ op: 'fail', when: { exists: '{{inv._id}}', is: null }, message: '库存记录不存在' },
|
|
688
|
+
{ op: 'fail', when: { lt: '{{inv.stock}}', than: '{{input.qty}}' }, message: '库存不足' },
|
|
689
|
+
{ op: 'mutation', model: 'Inventory',
|
|
690
|
+
data: { _id: '{{inv._id}}', stock: '{{dec:{{inv.stock}},{{input.qty}}}}' } },
|
|
691
|
+
],
|
|
692
|
+
});
|
|
693
|
+
|
|
694
|
+
const run = await store.runWorkflow('placeOrder', { productId: 'p1', warehouse: 'w1', qty: 30 });
|
|
695
|
+
// run.status ∈ succeeded | failed | rejected | drySucceeded | dryFailed
|
|
696
|
+
// 统一契约:业务失败不抛错,错误在 run.error(失败才有值;succeeded 态恒为 null)
|
|
697
|
+
```
|
|
698
|
+
|
|
699
|
+
- **步骤白名单**(首批仅三种,白名单外注册即 `WORKFLOW_UNSUPPORTED`):`query`(store.query;
|
|
700
|
+
结果单条化,>1 行显式 Err)、`mutation`(store.mutation / upsert,继承其步骤序列与占位符机制)、
|
|
701
|
+
`fail`(显式业务断言失败:run 记 failed + stepIndex + message)。步骤可选 `when` 守卫
|
|
702
|
+
(exists / is / eq / ne / lt / lte / gt / gte),不满足记 `skipped`——显式留痕,绝不静默跳过。
|
|
703
|
+
- **占位符**:`{{input.<path>}}`(本次 run 输入)、`{{<as>.<path>}}`(前序步骤结果,必须前向引用)、
|
|
704
|
+
`{{dec:<a>,<b>}}`(递减);整值替换保类型、内嵌替换字符串化。gql 内禁占位符(参数走 params
|
|
705
|
+
绑定,防注入);数组下标路径不支持(逐行处理请走宿主代码编排)。
|
|
706
|
+
- **权限**:defn 内嵌三级角色白名单(复用四级 RBAC 语义:admin / super_admin 放行、guest 拒绝、
|
|
707
|
+
internal 放行、creator 按 Missing 通过);run 继承触发者 Context,每步 query/mutation 都过 core
|
|
708
|
+
权限判定——工作流是「权限内的一次次普通调用」,不存在超级身份。`requireContext(true)` 开启时
|
|
709
|
+
无 ctx 拒跑(fail-secure 优先于 dry-run)。rejected 同样落库(拒绝可审计)。
|
|
710
|
+
- **原子性**:单源 run 整体原子(外层 `runAtomic` 包住整个步骤循环,内层 mutation 嵌套并入——
|
|
711
|
+
任一步失败整体回滚);多源 / 预扫失败按顺序执行并发反馈事件(`workflow_non_atomic` /
|
|
712
|
+
`workflow_prescan_failed`,禁静默)。run 记录时序:先落 `running`(进程崩溃可见)→ 步骤事务 →
|
|
713
|
+
终态在事务外独立提交(业务回滚不影响失败 run 可查)。
|
|
714
|
+
- **dry-run**:`store.runWorkflow(name, input, { dryRun: true })`——query 真实执行(只读安全),
|
|
715
|
+
mutation / fail 记 `wouldRun`;终态 `drySucceeded | dryFailed`。
|
|
716
|
+
- **run 落库**:`__workflowRun` 由模块加载即自举注册(幂等);SQL 后端首次启用工作流需执行
|
|
717
|
+
`ddl.generate(backend, ['__workflowRun'])` 建表(Mongo 无需,首次写入自动建集合)。
|
|
718
|
+
其 `write` 为显式空名单(普通角色 GQL 篡改 run 审计被 R2 拒绝;模块内部写入走 internal 上下文)。
|
|
719
|
+
|
|
720
|
+
### 首批明确不做(检出即 Err,边界与能力同权重)
|
|
721
|
+
|
|
722
|
+
| 不做 | 理由 | 出口 |
|
|
723
|
+
|---|---|---|
|
|
724
|
+
| 循环 / 并行 / 子工作流 / 人工审批 | DAG 与人工等待语义复杂度爆炸;线性 + `when` 覆盖首批场景 | 宿主代码用 store API 编排 |
|
|
725
|
+
| 自动补偿(Saga)/ 自动重试 | 反向操作语义负担大;步骤无自动幂等保证 | 人查 run 记录显式处置 |
|
|
726
|
+
| 步骤级宿主回调 | 任意代码击穿白名单治理 | schema computes(读)/ 宿主代码(写) |
|
|
727
|
+
| 工作流定义存库 / 热更 | 依赖 schema 版本化先行(路线图下一步) | defn 暂与 schema 同模式:代码内 JSON + register |
|
|
728
|
+
| 定时触发 / 事件触发 | 触发器是常驻 IO 职责,属调度层,与「纯编排」正交 | 应用层自行调用 `runWorkflow` |
|
|
729
|
+
| gql 内嵌占位符 / 数组下标路径 | 注入面 / 数组逐行处理语义 | params 绑定 / 宿主代码编排 |
|
|
730
|
+
|
|
569
731
|
## 常见问题
|
|
570
732
|
|
|
571
733
|
**如何在 Node.js 中让一份 schema 同时用于 MongoDB 和 PostgreSQL?**
|
package/llms-full.txt
CHANGED
|
@@ -379,6 +379,77 @@ store.setFeedbackSink((e) => logger.warn({ code: e.code }, e.hint));
|
|
|
379
379
|
|
|
380
380
|
Non-pushdownable commands also throw `PushdownUnsupportedError`.
|
|
381
381
|
|
|
382
|
+
### `store.ask(question, opts)`
|
|
383
|
+
|
|
384
|
+
AI ask — natural-language query (L1, read-only). The question plus a permission-filtered
|
|
385
|
+
schema summary go to a pluggable LLM, which must answer with a single `{"gql","params"}`
|
|
386
|
+
JSON object; the query is planned and executed inside the hardened `text2query` profile
|
|
387
|
+
(read-only, row/depth caps, route override disabled). Structured failures are fed back
|
|
388
|
+
to the LLM for retry (up to `maxRetries`); on exhaustion `AskExhausted` is thrown — no
|
|
389
|
+
silent degradation, no empty results.
|
|
390
|
+
|
|
391
|
+
| Option | Type | Meaning |
|
|
392
|
+
| --- | --- | --- |
|
|
393
|
+
| `llm` | `string \| Function` | required; registry name or an `async (messages) => string` client |
|
|
394
|
+
| `ctx` | object | required; server-side `{ userId, roles }`; missing ctx is rejected (fail-secure), never enters LLM messages |
|
|
395
|
+
| `maxRetries` | number (default `3`) | retries after a failed attempt |
|
|
396
|
+
| `knowledge` | string | override the system-prompt knowledge text |
|
|
397
|
+
|
|
398
|
+
Resolves to `{ data, attempts, events }`; each attempt is `{ llmRaw, gql, params, rows }`
|
|
399
|
+
(the successful round has no `error` key), failures carry
|
|
400
|
+
`{ error: { code, message } }` with `code` ∈ `badLlmOutput`, `profileBlocked`,
|
|
401
|
+
`permissionDenied`, `planError`. LLM client faults (`LlmError`) propagate untouched.
|
|
402
|
+
|
|
403
|
+
```js
|
|
404
|
+
const { llm } = require('nodejs-store');
|
|
405
|
+
|
|
406
|
+
llm.registerLlm('deepseek', llm.makeOpenaiCompat({
|
|
407
|
+
baseUrl: 'https://api.deepseek.com/v1',
|
|
408
|
+
model: 'deepseek-chat',
|
|
409
|
+
apiKey: process.env.DEEPSEEK_API_KEY,
|
|
410
|
+
}));
|
|
411
|
+
|
|
412
|
+
const result = await store.ask('Total amount of my last 10 orders?', {
|
|
413
|
+
llm: 'deepseek',
|
|
414
|
+
ctx: { userId: 'u1', roles: ['viewer'] },
|
|
415
|
+
});
|
|
416
|
+
console.log(result.data, result.attempts.at(-1).gql);
|
|
417
|
+
```
|
|
418
|
+
|
|
419
|
+
> The `text2query` profile is a process-wide core singleton: serialize `ask()` calls
|
|
420
|
+
> per process, or run each in its own worker.
|
|
421
|
+
|
|
422
|
+
### `store.describeForAi(ctx?)`
|
|
423
|
+
|
|
424
|
+
Permission-filtered schema summary for LLM prompts — a compact JSON array with one
|
|
425
|
+
`{ name, fields, relations, computes }` entry per model. Without a context only model
|
|
426
|
+
and field names are exposed (no types). With a context everything is filtered by the
|
|
427
|
+
caller's role (`canRead` / `readableFields` / `readableRelations` / `readableComputes`);
|
|
428
|
+
archive tables (`*Deleted`) and ops details (indexes / datasource / namespace) never
|
|
429
|
+
enter the prompt. Bindings lacking the `readableComputes` verdict conservatively exclude
|
|
430
|
+
`read`-whitelisted computed columns and emit `askSummaryComputeSkipped`.
|
|
431
|
+
|
|
432
|
+
### LLM registry (`llm`)
|
|
433
|
+
|
|
434
|
+
Pluggable LLM clients, no SDK bundled. A client is one function
|
|
435
|
+
`async (messages: Array<{ role, content }>) => string`.
|
|
436
|
+
|
|
437
|
+
```js
|
|
438
|
+
llm.registerLlm('deepseek', client); // duplicate name → throws
|
|
439
|
+
llm.getLlm('deepseek'); // unregistered name → throws
|
|
440
|
+
|
|
441
|
+
llm.makeOpenaiCompat({ // OpenAI-compatible factory (global fetch, zero deps)
|
|
442
|
+
baseUrl, model, apiKey,
|
|
443
|
+
jsonMode: true, // false → omit response_format
|
|
444
|
+
effort: 'low', // null → omit reasoning_effort
|
|
445
|
+
maxTokens: 4096,
|
|
446
|
+
timeoutMs: 90000,
|
|
447
|
+
});
|
|
448
|
+
```
|
|
449
|
+
|
|
450
|
+
Client faults throw `LlmError` with a structured `.detail` (`code` ∈ `llmNetworkError`,
|
|
451
|
+
`llmHttpError`, `llmEmptyContent`, `llmJsonPromptMissing`).
|
|
452
|
+
|
|
382
453
|
### Low-level modules
|
|
383
454
|
|
|
384
455
|
```js
|
package/llms.txt
CHANGED
|
@@ -37,6 +37,7 @@
|
|
|
37
37
|
- [Multi-datasource & multi-tenant](https://github.com/coenddt/nodejs-store#multi-datasource-connections): `(source, namespace, collection)` and route override
|
|
38
38
|
- [Schema reference](https://github.com/coenddt/nodejs-store#schema-reference): fields, relations, computes, indexes, read/write whitelists
|
|
39
39
|
- [Advanced API](https://github.com/coenddt/nodejs-store#advanced-api): `buildPipeline`, `syncSchema`, `setFeedbackSink`, low-level modules
|
|
40
|
+
- [AI ask (natural-language query)](https://github.com/coenddt/nodejs-store#storeaskquestion-opts): `store.ask` + `store.describeForAi` + pluggable LLM registry (`makeOpenaiCompat`), guarded by the read-only `text2query` profile
|
|
40
41
|
- [Transactions](https://github.com/coenddt/nodejs-store#transaction-boundary): per-source atomicity guarantees
|
|
41
42
|
|
|
42
43
|
## Optional
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "nodejs-store",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.7.0",
|
|
4
4
|
"description": "Multi-backend data layer for Node.js (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control",
|
|
5
5
|
"main": "src/index.js",
|
|
6
6
|
"files": [
|
|
@@ -65,7 +65,7 @@
|
|
|
65
65
|
"mongodb": "^6.21.0",
|
|
66
66
|
"mysql2": "^3.24.4",
|
|
67
67
|
"pg": "^8.23.0",
|
|
68
|
-
"rust-store-node": "^2.
|
|
68
|
+
"rust-store-node": "^2.7.0"
|
|
69
69
|
},
|
|
70
70
|
"devDependencies": {
|
|
71
71
|
"@eslint/js": "^9.39.2",
|