storepy 2.5.0__tar.gz → 2.7.0__tar.gz

{storepy-2.5.0 → storepy-2.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: storepy
-Version: 2.5.0
+Version: 2.7.0
 Summary: Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control
 Author-email: leo <coen_ddt@qq.com>
 License-Expression: MIT
@@ -85,6 +85,7 @@ from py_store import init, store
 - [Feedback events](#feedback-events)
 - [Schema reference](#schema-reference)
 - [Transactions](#transaction-boundary)
+- [Transactional capabilities](#transactional-capabilities)
 - [FAQ](#faq)
 - [Related projects](#related-projects)
 
@@ -265,6 +266,9 @@ GQL tree queries compile to a single native query per backend — never hand-wri
 - **Permission context** — `ContextVar`-based roles (`super_admin`/`admin`/`guest`/`creator`...), schema/field-level read/write whitelists, automatic owner-condition injection.
 - **Multi-datasource & multi-tenant** — locate a schema by `(source, namespace, collection)`; re-target per request with a route override.
 - **Async-first, Rust core** — built on PyMongo's `AsyncMongoClient` and a shared Rust core with SQL dialects.
+- **Relation predicates in mutations** — filter `update` / `remove` by related-table fields, pushed down to all four backends (previously a silent no-op on MongoDB).
+- **Autoincrement primary keys** — declare `_id` as `{"type": "int", "strategy": "autoincrement"}` for database-assigned integer IDs, with explicit errors where autoincrement is impossible.
+- **Index DDL** — `schema.indexes` compiles to real `CREATE [UNIQUE] INDEX` statements (per backend, byte-identical); the generator still only emits text.
 
 ## GQL syntax
 
@@ -548,6 +552,84 @@ Boundary rules worth knowing up front (all **fail explicitly**, never silently d
 - Read consistency: only multiple reads inside an explicit session share one transaction connection; reads outside a session do not open an extra transaction.
 - **Cross-source writes (no session)**: a single write call touching ≥2 datasources **cannot be atomic**; it runs sequentially and emits one `non_atomic_write` feedback event (`code: nonAtomic`, with the source list) — degradation is allowed, silence is not. Converge writes onto a single source, or wrap them in `store.session()` (which fails closed on cross-source writes).
 
+## Transactional capabilities
+
+Capabilities aimed at transactional workloads (orders, inventory — write contention plus
+complex reads). Full details, semantics and the explicit-error list:
+**[doc/transaction-capabilities.md](doc/transaction-capabilities.md)** ·
+[中文](doc/transaction-capabilities.zh-CN.md).
+
+- **Relation predicates in mutations** — `update_many('Inventory', {'product': {'category': 'meat'}}, {'$inc': {'stock': 10}})`: condition keys matching a declared relation become a semi/anti-join, normalized into a preCommand (aggregate fetching `_id`s) plus `_id $in`.
+- **`$group by` one-relation paths** — `by: ['product.category']` compiles to `$lookup`+`$unwind` (Mongo) / `LEFT JOIN` (SQL); `many` paths fail explicitly (fan-out breaks count semantics).
+- **Autoincrement PKs** — `_id: {'type': 'int', 'strategy': 'autoincrement'}`; PG/SQLite read back via `INSERT…RETURNING`, MySQL via last-insert-id; MongoDB and `insert_many` fail explicitly with `AUTOINCREMENT_NOT_SUPPORTED` (no silent ObjectId substitution).
+- **Index DDL** — `schema.indexes` (MongoDB shape) → `CREATE [UNIQUE] INDEX idx_<table>_<cols>` in `ddl.generate`, byte-identical across MySQL/PostgreSQL/SQLite.
+- **Declarative migration** — `ddl.diff_defs(old, new)` + `ddl.generate_migration(backend, old, new)`: whitelist-only (add table/column/index, type widening), per-dialect SQL, pure functions; destructive changes fail with `MIGRATION_UNSUPPORTED`.
+
+## Workflow orchestration (first batch)
+
+Express "orchestration of multi-step data operations" as data: a workflow definition (defn) is pure
+JSON isomorphic to a schema defn, and each run is persisted to the built-in schema `__workflowRun`
+(queryable with plain GQL — zero new observability endpoints). Execution generalizes the existing
+mutation step-sequence mechanism: linear steps + per-step `when` guards + fail-fast. The engine
+lives in the host layer (`py_store/workflow.py`), core unchanged; aligned with
+`nodejs-store/src/workflow.js` (byte-identical outputs guarded by parity anchor tests).
+
+```python
+from py_store import workflow
+
+workflow.register({
+    'name': 'placeOrder',
+    'run': ['admin', 'ops'],              # three-tier whitelists read/write/run (run falls back to write)
+    'steps': [
+        {'op': 'query', 'as': 'inv',
+         'gql': 'Inventory($condition:@c0){_id, stock}',
+         'params': {'c0': {'productId': '{{input.productId}}', 'warehouse': '{{input.warehouse}}'}}},
+        {'op': 'fail', 'when': {'exists': '{{inv._id}}', 'is': None}, 'message': '库存记录不存在'},
+        {'op': 'fail', 'when': {'lt': '{{inv.stock}}', 'than': '{{input.qty}}'}, 'message': '库存不足'},
+        {'op': 'mutation', 'model': 'Inventory',
+         'data': {'_id': '{{inv._id}}', 'stock': '{{dec:{{inv.stock}},{{input.qty}}}}'}},
+    ],
+})
+
+run = await workflow.run('placeOrder', {'productId': 'p1', 'warehouse': 'w1', 'qty': 30})
+# run['status'] ∈ succeeded | failed | rejected | drySucceeded | dryFailed
+# Uniform contract: business failures never raise; the error lives in run['error']
+# (set only on failure; always null on success — never `||`-masked downstream)
+```
+
+- **Step whitelist** (three kinds; anything else fails registration with `WORKFLOW_UNSUPPORTED`):
+  `query` (result must be unique — >1 row is an explicit error), `mutation` (store.mutation /
+  upsert), `fail` (explicit business assertion). Optional `when` guards (exists / is / eq / ne /
+  lt / lte / gt / gte) record `skipped` explicitly — never silently skipped.
+- **Placeholders**: `{{input.<path>}}`, `{{<as>.<path>}}` (forward references only),
+  `{{dec:<a>,<b>}}`; full-string replacement keeps the value type. No placeholders inside gql
+  (bind via params — injection safety); no array-index path segments.
+- **Permissions**: three-tier role whitelists embedded in the defn (same RBAC semantics: admin /
+  super_admin bypass, guest denied, internal bypass); runs inherit the caller's Context and every
+  step goes through core permission checks — no superuser. `require_context(true)` rejects
+  context-less runs (fail-secure wins over dry-run); rejected runs are persisted for audit.
+- **Atomicity**: a single-source run is atomic across steps (outer `run_atomic` wraps the whole
+  loop, inner mutations nest into it); multi-source / prescan-failed runs execute sequentially and
+  emit feedback events (`workflow_non_atomic` / `workflow_prescan_failed`) — never silent. The run
+  record (running → terminal) is committed outside the business transaction so failed runs stay
+  queryable after rollback.
+- **dry-run**: `workflow.run(name, input, dry_run=True)` — query steps execute for real (read-only
+  safe); mutation / fail are recorded as `wouldRun` (`drySucceeded | dryFailed`).
+- **Run persistence**: `__workflowRun` is bootstrapped on import (idempotent); SQL backends need a
+  one-time `ddl.generate(backend, ['__workflowRun'])` (Mongo creates the collection on first write).
+  Its `write` whitelist is explicitly empty (GQL tampering with run audit is rejected by R2).
+
+### Explicitly not in the first batch (detected → error; boundaries shipped with the same weight as features)
+
+| Not supported | Why | Escape hatch |
+|---|---|---|
+| Loops / parallel / sub-workflows / human approval | DAG & wait semantics explode; linear + `when` covers the first batch | orchestrate in host code via the store API |
+| Auto compensation (Saga) / auto retry | Inverse-operation burden; steps have no automatic idempotency | inspect run records and handle explicitly |
+| Per-step host callbacks | Arbitrary code breaks whitelist governance | schema computes (read) / host code (write) |
+| Workflow defn persistence / hot reload | Depends on schema versioning (next on the roadmap) | defn stays code-side JSON + register, like schemas today |
+| Timers / event triggers | Scheduling is a resident-IO concern, orthogonal to pure orchestration | call `run` from the application layer |
+| Placeholders inside gql / array-index paths | Injection surface / per-row iteration semantics | params binding / host-code orchestration |
+
 ## FAQ
 
 **How do I use one schema for both MongoDB and PostgreSQL in Python?**

{storepy-2.5.0 → storepy-2.7.0}/README.md

@@ -45,6 +45,7 @@ from py_store import init, store
 - [Feedback events](#feedback-events)
 - [Schema reference](#schema-reference)
 - [Transactions](#transaction-boundary)
+- [Transactional capabilities](#transactional-capabilities)
 - [FAQ](#faq)
 - [Related projects](#related-projects)
 
@@ -225,6 +226,9 @@ GQL tree queries compile to a single native query per backend — never hand-wri
 - **Permission context** — `ContextVar`-based roles (`super_admin`/`admin`/`guest`/`creator`...), schema/field-level read/write whitelists, automatic owner-condition injection.
 - **Multi-datasource & multi-tenant** — locate a schema by `(source, namespace, collection)`; re-target per request with a route override.
 - **Async-first, Rust core** — built on PyMongo's `AsyncMongoClient` and a shared Rust core with SQL dialects.
+- **Relation predicates in mutations** — filter `update` / `remove` by related-table fields, pushed down to all four backends (previously a silent no-op on MongoDB).
+- **Autoincrement primary keys** — declare `_id` as `{"type": "int", "strategy": "autoincrement"}` for database-assigned integer IDs, with explicit errors where autoincrement is impossible.
+- **Index DDL** — `schema.indexes` compiles to real `CREATE [UNIQUE] INDEX` statements (per backend, byte-identical); the generator still only emits text.
 
 ## GQL syntax
 
@@ -508,6 +512,84 @@ Boundary rules worth knowing up front (all **fail explicitly**, never silently d
 - Read consistency: only multiple reads inside an explicit session share one transaction connection; reads outside a session do not open an extra transaction.
 - **Cross-source writes (no session)**: a single write call touching ≥2 datasources **cannot be atomic**; it runs sequentially and emits one `non_atomic_write` feedback event (`code: nonAtomic`, with the source list) — degradation is allowed, silence is not. Converge writes onto a single source, or wrap them in `store.session()` (which fails closed on cross-source writes).
 
+## Transactional capabilities
+
+Capabilities aimed at transactional workloads (orders, inventory — write contention plus
+complex reads). Full details, semantics and the explicit-error list:
+**[doc/transaction-capabilities.md](doc/transaction-capabilities.md)** ·
+[中文](doc/transaction-capabilities.zh-CN.md).
+
+- **Relation predicates in mutations** — `update_many('Inventory', {'product': {'category': 'meat'}}, {'$inc': {'stock': 10}})`: condition keys matching a declared relation become a semi/anti-join, normalized into a preCommand (aggregate fetching `_id`s) plus `_id $in`.
+- **`$group by` one-relation paths** — `by: ['product.category']` compiles to `$lookup`+`$unwind` (Mongo) / `LEFT JOIN` (SQL); `many` paths fail explicitly (fan-out breaks count semantics).
+- **Autoincrement PKs** — `_id: {'type': 'int', 'strategy': 'autoincrement'}`; PG/SQLite read back via `INSERT…RETURNING`, MySQL via last-insert-id; MongoDB and `insert_many` fail explicitly with `AUTOINCREMENT_NOT_SUPPORTED` (no silent ObjectId substitution).
+- **Index DDL** — `schema.indexes` (MongoDB shape) → `CREATE [UNIQUE] INDEX idx_<table>_<cols>` in `ddl.generate`, byte-identical across MySQL/PostgreSQL/SQLite.
+- **Declarative migration** — `ddl.diff_defs(old, new)` + `ddl.generate_migration(backend, old, new)`: whitelist-only (add table/column/index, type widening), per-dialect SQL, pure functions; destructive changes fail with `MIGRATION_UNSUPPORTED`.
+
+## Workflow orchestration (first batch)
+
+Express "orchestration of multi-step data operations" as data: a workflow definition (defn) is pure
+JSON isomorphic to a schema defn, and each run is persisted to the built-in schema `__workflowRun`
+(queryable with plain GQL — zero new observability endpoints). Execution generalizes the existing
+mutation step-sequence mechanism: linear steps + per-step `when` guards + fail-fast. The engine
+lives in the host layer (`py_store/workflow.py`), core unchanged; aligned with
+`nodejs-store/src/workflow.js` (byte-identical outputs guarded by parity anchor tests).
+
+```python
+from py_store import workflow
+
+workflow.register({
+    'name': 'placeOrder',
+    'run': ['admin', 'ops'],              # three-tier whitelists read/write/run (run falls back to write)
+    'steps': [
+        {'op': 'query', 'as': 'inv',
+         'gql': 'Inventory($condition:@c0){_id, stock}',
+         'params': {'c0': {'productId': '{{input.productId}}', 'warehouse': '{{input.warehouse}}'}}},
+        {'op': 'fail', 'when': {'exists': '{{inv._id}}', 'is': None}, 'message': '库存记录不存在'},
+        {'op': 'fail', 'when': {'lt': '{{inv.stock}}', 'than': '{{input.qty}}'}, 'message': '库存不足'},
+        {'op': 'mutation', 'model': 'Inventory',
+         'data': {'_id': '{{inv._id}}', 'stock': '{{dec:{{inv.stock}},{{input.qty}}}}'}},
+    ],
+})
+
+run = await workflow.run('placeOrder', {'productId': 'p1', 'warehouse': 'w1', 'qty': 30})
+# run['status'] ∈ succeeded | failed | rejected | drySucceeded | dryFailed
+# Uniform contract: business failures never raise; the error lives in run['error']
+# (set only on failure; always null on success — never `||`-masked downstream)
+```
+
+- **Step whitelist** (three kinds; anything else fails registration with `WORKFLOW_UNSUPPORTED`):
+  `query` (result must be unique — >1 row is an explicit error), `mutation` (store.mutation /
+  upsert), `fail` (explicit business assertion). Optional `when` guards (exists / is / eq / ne /
+  lt / lte / gt / gte) record `skipped` explicitly — never silently skipped.
+- **Placeholders**: `{{input.<path>}}`, `{{<as>.<path>}}` (forward references only),
+  `{{dec:<a>,<b>}}`; full-string replacement keeps the value type. No placeholders inside gql
+  (bind via params — injection safety); no array-index path segments.
+- **Permissions**: three-tier role whitelists embedded in the defn (same RBAC semantics: admin /
+  super_admin bypass, guest denied, internal bypass); runs inherit the caller's Context and every
+  step goes through core permission checks — no superuser. `require_context(true)` rejects
+  context-less runs (fail-secure wins over dry-run); rejected runs are persisted for audit.
+- **Atomicity**: a single-source run is atomic across steps (outer `run_atomic` wraps the whole
+  loop, inner mutations nest into it); multi-source / prescan-failed runs execute sequentially and
+  emit feedback events (`workflow_non_atomic` / `workflow_prescan_failed`) — never silent. The run
+  record (running → terminal) is committed outside the business transaction so failed runs stay
+  queryable after rollback.
+- **dry-run**: `workflow.run(name, input, dry_run=True)` — query steps execute for real (read-only
+  safe); mutation / fail are recorded as `wouldRun` (`drySucceeded | dryFailed`).
+- **Run persistence**: `__workflowRun` is bootstrapped on import (idempotent); SQL backends need a
+  one-time `ddl.generate(backend, ['__workflowRun'])` (Mongo creates the collection on first write).
+  Its `write` whitelist is explicitly empty (GQL tampering with run audit is rejected by R2).
+
+### Explicitly not in the first batch (detected → error; boundaries shipped with the same weight as features)
+
+| Not supported | Why | Escape hatch |
+|---|---|---|
+| Loops / parallel / sub-workflows / human approval | DAG & wait semantics explode; linear + `when` covers the first batch | orchestrate in host code via the store API |
+| Auto compensation (Saga) / auto retry | Inverse-operation burden; steps have no automatic idempotency | inspect run records and handle explicitly |
+| Per-step host callbacks | Arbitrary code breaks whitelist governance | schema computes (read) / host code (write) |
+| Workflow defn persistence / hot reload | Depends on schema versioning (next on the roadmap) | defn stays code-side JSON + register, like schemas today |
+| Timers / event triggers | Scheduling is a resident-IO concern, orthogonal to pure orchestration | call `run` from the application layer |
+| Placeholders inside gql / array-index paths | Injection surface / per-row iteration semantics | params binding / host-code orchestration |
+
 ## FAQ
 
 **How do I use one schema for both MongoDB and PostgreSQL in Python?**

{storepy-2.5.0 → storepy-2.7.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "storepy"
-version = "2.5.0"
+version = "2.7.0"
 description = "Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -56,6 +56,10 @@ Issues = "https://github.com/coenddt/py-store/issues"
 [tool.setuptools.packages.find]
 where = ["src"]
 
+# ask() 的 system prompt 知识源（text-to-query 通用契约裁剪版）随包分发
+[tool.setuptools.package-data]
+py_store = ["ask_knowledge.md"]
+
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 pythonpath = ["src"]

{storepy-2.5.0 → storepy-2.7.0}/src/py_store/__init__.py

@@ -20,18 +20,19 @@ py-store — 轻量多后端数据层（Python 版，Rust 单核心架构；支
 """
 
 from collections.abc import Mapping
-from contextlib import contextmanager
 from typing import Any
 
 from pymongo.errors import PyMongoError
 
 from . import (
+    ask,
     crud,
     datasource,
     ddl,
     feedback,
     permission,
     schema,
+    workflow,
 )
 from . import (
     executors as executors,
@@ -39,10 +40,16 @@ from . import (
 from . import (
     introspect as introspect,
 )
+from .ask import AskExhausted as AskExhausted
+from .ask import AskResult as AskResult
 from .datasource import NativeCommandError as NativeCommandError
 from .datasource import NonAtomicWriteError as NonAtomicWriteError
 from .datasource import RawSqlError as RawSqlError
 from .datasource import Session as Session
+from .llm import get_llm as get_llm
+from .llm import make_openai_compat as make_openai_compat
+from .llm import register_llm as register_llm
+from .schema import text2query
 from .sync import sync_schema
 
 
@@ -52,21 +59,6 @@ def _build_pipeline(gql, params=None):
         gql, params if params is not None else {}, permission.get_context())
 
 
-@contextmanager
-def text2query():
-    """以 text2query 档执行（功能收缩 + 硬限制），退出恢复原档位。
-
-    AI 问数链路入口；与 ``permission.scoped_roles`` 同构（token-set/reset，嵌套安全）。
-    进入档位即等效强制携带用户上下文（core `ensure_profile_ctx`，见执行文档 §4.2）。
-    """
-    prev = schema.get_profile()
-    schema.set_profile('text2query')
-    try:
-        yield
-    finally:
-        schema.set_profile(prev)
-
-
 class Store:
     """以属性方式访问各 API（**全显式类方法，无动态查找**）
 
@@ -203,6 +195,35 @@ class Store:
         """从已注册 schema def 生成指定后端 DDL 文本（纯函数，不连库、不回写；铁律 6）"""
         return ddl.generate(backend, names)
 
+    # ── 工作流编排（首批：线性 + when 守卫 + fail-fast；见 workflow.py 与设计文档）──
+    def registerWorkflow(self, defn: dict) -> dict:
+        """注册工作流定义（注册即静态校验，白名单外显式 Err 含 WORKFLOW_UNSUPPORTED）"""
+        return workflow.register(defn)
+
+    def workflows(self, ctx: dict | None = None) -> list[str]:
+        """全部可见工作流名（read 白名单过滤）"""
+        return workflow.list(ctx)
+
+    def getWorkflow(self, name: str, ctx: dict | None = None) -> dict:
+        """按名取工作流定义（read 白名单过滤；不可见与不存在同形——防枚举）"""
+        return workflow.get(name, ctx)
+
+    async def runWorkflow(self, name: str, input: dict | None = None, *,
+                          dry_run: bool = False,
+                          route_override: dict | None = None) -> dict[str, Any]:
+        """触发工作流 → 完整 run 文档（终态 failed/rejected 不抛错，以 run.status + error 表达）"""
+        return await workflow.run(name, input, dry_run=dry_run, route_override=route_override)
+
+    # ── AI 问数（L1，只读）：自然语言 → LLM 翻译 → text2query 沙箱执行 → 结构化回喂 ──
+    # 护栏（档位/ctx/route_override）全部服务端硬编码于 ask.py，零暴露进 LLM 消息面（D5）
+    # （先于 ask 赋值取 describe_for_ai：赋值后类体命名空间的 ask 不再是模块）
+    describeForAi = staticmethod(ask.describe_for_ai)
+    describe_for_ai = describeForAi
+    ask = staticmethod(ask.ask)
+    # 问数结果/耗尽错误（实例可被 store.AskExhausted 捕获；对齐 PermissionError 先例）
+    AskResult = AskResult
+    AskExhausted = AskExhausted
+
     # ── 驼峰别名（与上方同名蛇形方法为**同一实现**，仅命名差异）──
     queryOne = query_one
     queryWithCount = query_with_count
@@ -214,6 +235,10 @@ class Store:
     executeRaw = execute_raw
     executeNative = execute_native
     generateDdl = generate_ddl
+    # 工作流编排（蛇形别名与上方驼峰同实现）
+    register_workflow = registerWorkflow
+    run_workflow = runWorkflow
+    get_workflow = getWorkflow
 
     # ── 其余 API 显式绑定（staticmethod：避免实例化后 self 注入）──
     # Schema 管理
@@ -243,6 +268,19 @@ class Store:
     set_require_context = staticmethod(schema.set_require_context)
     requireContext = staticmethod(schema.require_context)
     require_context = staticmethod(schema.require_context)
+    # RBAC 动态策略（判决唯一在 core；本层仅透传配置与查询面）
+    setRbac = staticmethod(permission.set_rbac)
+    set_rbac = staticmethod(permission.set_rbac)
+    rbacEnabled = staticmethod(permission.rbac_enabled)
+    rbac_enabled = staticmethod(permission.rbac_enabled)
+    rbacCan = staticmethod(permission.rbac_can)
+    rbac_can = staticmethod(permission.rbac_can)
+    rbacReadableFields = staticmethod(permission.rbac_readable_fields)
+    rbac_readable_fields = staticmethod(permission.rbac_readable_fields)
+    rbacWritableFields = staticmethod(permission.rbac_writable_fields)
+    rbac_writable_fields = staticmethod(permission.rbac_writable_fields)
+    rbacRowCondition = staticmethod(permission.rbac_row_condition)
+    rbac_row_condition = staticmethod(permission.rbac_row_condition)
     # 查询档位（判决唯一在 core）：standard 默认放开 / text2query 功能收缩
     # 进入档即等效强制 ctx；未知档由 core 抛 ValueError 上抛（禁静默回落默认档）
     setProfile = staticmethod(schema.set_profile)