storepy 2.1.0__tar.gz → 2.3.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {storepy-2.1.0/src/storepy.egg-info → storepy-2.3.1}/PKG-INFO +4 -1
- {storepy-2.1.0 → storepy-2.3.1}/README.md +2 -0
- {storepy-2.1.0 → storepy-2.3.1}/pyproject.toml +14 -2
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/__init__.py +37 -2
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/__init__.py +2 -1
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/exec.py +47 -1
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/query.py +36 -2
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/ddl.py +36 -6
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/mysql.py +10 -2
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/sqlite.py +10 -1
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/schema.py +65 -10
- {storepy-2.1.0 → storepy-2.3.1/src/storepy.egg-info}/PKG-INFO +4 -1
- {storepy-2.1.0 → storepy-2.3.1}/src/storepy.egg-info/SOURCES.txt +5 -1
- {storepy-2.1.0 → storepy-2.3.1}/src/storepy.egg-info/requires.txt +1 -0
- storepy-2.3.1/tests/test_coverage_margin.py +511 -0
- storepy-2.3.1/tests/test_host_paths.py +418 -0
- storepy-2.3.1/tests/test_profile.py +242 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_raw_sql_and_ddl.py +22 -4
- storepy-2.3.1/tests/test_sql_commit_and_registry.py +120 -0
- {storepy-2.1.0 → storepy-2.3.1}/LICENSE +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/setup.cfg +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/core.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/id.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/mutation.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/crud/write.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/datasource.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/__init__.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/_values.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/mongo.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/executors/postgres.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/feedback.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/introspect/__init__.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/introspect/mysql.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/introspect/postgres.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/introspect/sqlite.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/permission.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/py_store/sync.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/storepy.egg-info/dependency_links.txt +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/src/storepy.egg-info/top_level.txt +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_federation_e2e.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_host_contract.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_mongo_executor.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_multi_datasource.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_py_store.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_real_backends_e2e.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_require_context.py +0 -0
- {storepy-2.1.0 → storepy-2.3.1}/tests/test_scenario_course_platform.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: storepy
|
|
3
|
-
Version: 2.1
|
|
3
|
+
Version: 2.3.1
|
|
4
4
|
Summary: Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control
|
|
5
5
|
Author-email: leo <coen_ddt@qq.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -33,6 +33,7 @@ Provides-Extra: sqlite
|
|
|
33
33
|
Requires-Dist: aiosqlite>=0.19; extra == "sqlite"
|
|
34
34
|
Provides-Extra: dev
|
|
35
35
|
Requires-Dist: pytest>=8; extra == "dev"
|
|
36
|
+
Requires-Dist: pytest-cov>=5; extra == "dev"
|
|
36
37
|
Requires-Dist: ruff>=0.16; extra == "dev"
|
|
37
38
|
Requires-Dist: mypy>=1.0; extra == "dev"
|
|
38
39
|
Dynamic: license-file
|
|
@@ -372,6 +373,7 @@ await store.transaction("default", transfer)
|
|
|
372
373
|
- `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `execute_raw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `run_in_transaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
|
|
373
374
|
- `store.execute_raw(source, sql, params=None, is_write=False)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source raises `RawSqlError` (`py_store.RawSqlError` / `store.RawSqlError`).
|
|
374
375
|
- `is_write=False` (default) returns `{"rows", "affectedRows"}` with the result-set rows; `is_write=True` returns the affected-row count.
|
|
376
|
+
- **The non-transactional path commits explicitly**: on SQL sources, a write plan that runs outside `store.transaction` is committed by the executor (`commit` on success; `rollback` then re-raise on failure). `aiosqlite` is not autocommit by default, so without that commit the write would be visible only on the current connection while `execute_raw` still reported success — a silent data-loss hazard. Multi-statement writes that must be atomic as a group belong inside `store.transaction`.
|
|
375
377
|
|
|
376
378
|
### DDL generation
|
|
377
379
|
|
|
@@ -384,6 +386,7 @@ sql = store.generate_ddl("postgres", ["Course", "CourseDeleted"])
|
|
|
384
386
|
|
|
385
387
|
- Only scalar fields become columns; `object` / `array` fields do not.
|
|
386
388
|
- Every table gets the `__present` sentinel column; `timestamps` models also get `createdAt` / `updatedAt`; the `<collection>_deleted` archive table is generated like any other registered def.
|
|
389
|
+
- The `<Name>Deleted` archive def is **derived by the Rust core when the model is registered**; the Python host only mirrors it (`collection` `<c>_deleted`, the `deletedAt` field, empty `idPrefix`) and never re-registers it into the core. So `schema.list()` and `generate_ddl()` contain each archive table exactly once; if a duplicate ever appears (upstream regression), they deduplicate and emit a `schemaDuplicateName` / `ddlDuplicateTable` feedback event instead of silently emitting duplicate `CREATE TABLE`s.
|
|
387
390
|
- No `CREATE INDEX` is emitted — SQL backends keep indexes as metadata only.
|
|
388
391
|
- MySQL `__present` is `VARCHAR(255)`; a schema whose present-token string would overflow emits a `ddlPresentOverflow` feedback event rather than failing silently.
|
|
389
392
|
|
|
@@ -333,6 +333,7 @@ await store.transaction("default", transfer)
|
|
|
333
333
|
- `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `execute_raw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `run_in_transaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
|
|
334
334
|
- `store.execute_raw(source, sql, params=None, is_write=False)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source raises `RawSqlError` (`py_store.RawSqlError` / `store.RawSqlError`).
|
|
335
335
|
- `is_write=False` (default) returns `{"rows", "affectedRows"}` with the result-set rows; `is_write=True` returns the affected-row count.
|
|
336
|
+
- **The non-transactional path commits explicitly**: on SQL sources, a write plan that runs outside `store.transaction` is committed by the executor (`commit` on success; `rollback` then re-raise on failure). `aiosqlite` is not autocommit by default, so without that commit the write would be visible only on the current connection while `execute_raw` still reported success — a silent data-loss hazard. Multi-statement writes that must be atomic as a group belong inside `store.transaction`.
|
|
336
337
|
|
|
337
338
|
### DDL generation
|
|
338
339
|
|
|
@@ -345,6 +346,7 @@ sql = store.generate_ddl("postgres", ["Course", "CourseDeleted"])
|
|
|
345
346
|
|
|
346
347
|
- Only scalar fields become columns; `object` / `array` fields do not.
|
|
347
348
|
- Every table gets the `__present` sentinel column; `timestamps` models also get `createdAt` / `updatedAt`; the `<collection>_deleted` archive table is generated like any other registered def.
|
|
349
|
+
- The `<Name>Deleted` archive def is **derived by the Rust core when the model is registered**; the Python host only mirrors it (`collection` `<c>_deleted`, the `deletedAt` field, empty `idPrefix`) and never re-registers it into the core. So `schema.list()` and `generate_ddl()` contain each archive table exactly once; if a duplicate ever appears (upstream regression), they deduplicate and emit a `schemaDuplicateName` / `ddlDuplicateTable` feedback event instead of silently emitting duplicate `CREATE TABLE`s.
|
|
348
350
|
- No `CREATE INDEX` is emitted — SQL backends keep indexes as metadata only.
|
|
349
351
|
- MySQL `__present` is `VARCHAR(255)`; a schema whose present-token string would overflow emits a `ddlPresentOverflow` feedback event rather than failing silently.
|
|
350
352
|
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "storepy"
|
|
7
|
-
version = "2.1
|
|
7
|
+
version = "2.3.1"
|
|
8
8
|
description = "Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.10"
|
|
@@ -45,7 +45,7 @@ classifiers = [
|
|
|
45
45
|
mysql = ["asyncmy>=0.2.9"]
|
|
46
46
|
postgres = ["asyncpg>=0.29"]
|
|
47
47
|
sqlite = ["aiosqlite>=0.19"]
|
|
48
|
-
dev = ["pytest>=8", "ruff>=0.16", "mypy>=1.0"]
|
|
48
|
+
dev = ["pytest>=8", "pytest-cov>=5", "ruff>=0.16", "mypy>=1.0"]
|
|
49
49
|
|
|
50
50
|
[project.urls]
|
|
51
51
|
Homepage = "https://github.com/coenddt/py-store"
|
|
@@ -83,3 +83,15 @@ python_version = "3.10"
|
|
|
83
83
|
# 宽松起步:py_store 为动态薄适配层,注解覆盖不足,先只挡「导入失败/明显类型矛盾」
|
|
84
84
|
ignore_missing_imports = true
|
|
85
85
|
check_untyped_defs = true
|
|
86
|
+
|
|
87
|
+
# ── 覆盖率(门禁口径:语句 ≥90%,见 .github/workflows/ci.yml;分支单独出具) ──
|
|
88
|
+
[tool.coverage.run]
|
|
89
|
+
source = ["py_store"]
|
|
90
|
+
relative_files = true
|
|
91
|
+
|
|
92
|
+
[tool.coverage.report]
|
|
93
|
+
show_missing = true
|
|
94
|
+
exclude_lines = [
|
|
95
|
+
"pragma: no cover",
|
|
96
|
+
"if TYPE_CHECKING:",
|
|
97
|
+
]
|
|
@@ -20,6 +20,7 @@ py-store — 轻量多后端数据层(Python 版,Rust 单核心架构;支
|
|
|
20
20
|
"""
|
|
21
21
|
|
|
22
22
|
from collections.abc import Mapping
|
|
23
|
+
from contextlib import contextmanager
|
|
23
24
|
from typing import Any
|
|
24
25
|
|
|
25
26
|
from pymongo.errors import PyMongoError
|
|
@@ -48,6 +49,21 @@ def _build_pipeline(gql, params=None):
|
|
|
48
49
|
gql, params if params is not None else {}, permission.get_context())
|
|
49
50
|
|
|
50
51
|
|
|
52
|
+
@contextmanager
|
|
53
|
+
def text2query():
|
|
54
|
+
"""以 text2query 档执行(功能收缩 + 硬限制),退出恢复原档位。
|
|
55
|
+
|
|
56
|
+
AI 问数链路入口;与 ``permission.scoped_roles`` 同构(token-set/reset,嵌套安全)。
|
|
57
|
+
进入档位即等效强制携带用户上下文(core `ensure_profile_ctx`,见执行文档 §4.2)。
|
|
58
|
+
"""
|
|
59
|
+
prev = schema.get_profile()
|
|
60
|
+
schema.set_profile('text2query')
|
|
61
|
+
try:
|
|
62
|
+
yield
|
|
63
|
+
finally:
|
|
64
|
+
schema.set_profile(prev)
|
|
65
|
+
|
|
66
|
+
|
|
51
67
|
class Store:
|
|
52
68
|
"""以属性方式访问各 API(**全显式类方法,无动态查找**)
|
|
53
69
|
|
|
@@ -182,6 +198,16 @@ class Store:
|
|
|
182
198
|
set_require_context = staticmethod(schema.set_require_context)
|
|
183
199
|
requireContext = staticmethod(schema.require_context)
|
|
184
200
|
require_context = staticmethod(schema.require_context)
|
|
201
|
+
# 查询档位(判决唯一在 core):standard 默认放开 / text2query 功能收缩
|
|
202
|
+
# 进入档即等效强制 ctx;未知档由 core 抛 ValueError 上抛(禁静默回落默认档)
|
|
203
|
+
setProfile = staticmethod(schema.set_profile)
|
|
204
|
+
set_profile = staticmethod(schema.set_profile)
|
|
205
|
+
getProfile = staticmethod(schema.get_profile)
|
|
206
|
+
get_profile = staticmethod(schema.get_profile)
|
|
207
|
+
# 档位拒绝错误(实例可被 store.ProfileViolation 捕获;权限错误另见 PermissionError)
|
|
208
|
+
ProfileViolation = crud.ProfileViolation
|
|
209
|
+
# text2query 便捷上下文(进入设档、退出恢复;同 scoped_roles 的 token-set/reset)
|
|
210
|
+
text2query = staticmethod(text2query)
|
|
185
211
|
# 反馈事件通道(兜底/降级/拦截的统一出口,接入自动反馈闭环)
|
|
186
212
|
setFeedbackSink = staticmethod(feedback.set_sink)
|
|
187
213
|
set_feedback_sink = staticmethod(feedback.set_sink)
|
|
@@ -229,8 +255,17 @@ async def _create_indexes_if_needed():
|
|
|
229
255
|
|
|
230
256
|
await coll.create_index(list(keys.items()), **final_options)
|
|
231
257
|
except PyMongoError as e:
|
|
232
|
-
|
|
233
|
-
|
|
258
|
+
# 索引创建失败不阻塞 init(辅助动作),但必须走统一反馈通道:
|
|
259
|
+
# 无 sink 时由 feedback 默认落 stderr(不双份打印),宿主可 set_sink 接管。
|
|
260
|
+
# 对齐 nodejs-store/src/index.js(评审项 R7-m1)。
|
|
261
|
+
feedback.emit({
|
|
262
|
+
'type': 'index_create_failed',
|
|
263
|
+
'code': 'indexCreateFailed',
|
|
264
|
+
'layer': 'host',
|
|
265
|
+
'message': f'创建索引失败 {s["collection"]}: {e}',
|
|
266
|
+
'hint': ('检查该集合的索引定义与连接权限;'
|
|
267
|
+
'索引缺失不影响读写,相关查询将退化为全表扫描'),
|
|
268
|
+
})
|
|
234
269
|
|
|
235
270
|
|
|
236
271
|
async def init(connections):
|
|
@@ -7,12 +7,13 @@ CRUD 包 —— 薄 Host 适配层(对齐 nodejs-store/src/crud.js 的分工
|
|
|
7
7
|
- 写入时不补默认值(DB 存最少数据);读取时由 core 补默认值 + 计算列
|
|
8
8
|
"""
|
|
9
9
|
|
|
10
|
-
from .exec import _get_db, set_connections, set_db
|
|
10
|
+
from .exec import ProfileViolation, _get_db, set_connections, set_db
|
|
11
11
|
from .mutation import mutation, upsert
|
|
12
12
|
from .query import query, query_federated, query_one, query_with_count
|
|
13
13
|
from .write import count, exists, insert, insert_many, remove, update, update_many
|
|
14
14
|
|
|
15
15
|
__all__ = [
|
|
16
|
+
'ProfileViolation',
|
|
16
17
|
'_get_db',
|
|
17
18
|
'count',
|
|
18
19
|
'exists',
|
|
@@ -15,6 +15,7 @@ import time
|
|
|
15
15
|
|
|
16
16
|
from .. import datasource as _datasource
|
|
17
17
|
from ..executors.mongo import exec_mongo as _exec_mongo
|
|
18
|
+
from ..feedback import emit as _emit_feedback
|
|
18
19
|
from ..permission import PermissionError, get_context
|
|
19
20
|
from ..schema import get as _schema_get
|
|
20
21
|
|
|
@@ -26,6 +27,31 @@ _STEP_PH = re.compile(r'^\{\{step\.(\d+)\._id\}\}$')
|
|
|
26
27
|
# 可自由调整,映射不随文案漂移而静默失效。构造 PermissionError 时剥离前缀。
|
|
27
28
|
_PERM_PREFIX = 'ERR_PERMISSION:'
|
|
28
29
|
|
|
30
|
+
# 档位类错误识别:core text2query 档门禁统一携带 `ERR_TEXT2QUERY:` 稳定前缀
|
|
31
|
+
# (见 core `command/mod.rs::ERR_TEXT2QUERY`),同上按前缀映射。命中即 emit
|
|
32
|
+
# 反馈事件 `profile_blocked`(自动反馈原则:允许拦截,禁止静默)。
|
|
33
|
+
_PROFILE_PREFIX = 'ERR_TEXT2QUERY:'
|
|
34
|
+
|
|
35
|
+
# 从 core 文案 `... [$feature](功能收缩)` 中提取门禁项名;无 `[..]` 时留白(None),
|
|
36
|
+
# 不伪造 feature —— 缺值必须显式暴露(禁静默兜底)。
|
|
37
|
+
_FEATURE_RE = re.compile(r'\[(.+?)\]')
|
|
38
|
+
|
|
39
|
+
# 档位拦截反馈的统一提示(反馈事件契约 §4.6 的一部分;单点定义防文案漂移)
|
|
40
|
+
_PROFILE_HINT = ('上游(LLM 产出的 GQL / 调用方入参)越界;'
|
|
41
|
+
'text2query 档白名单见 SKILL.md §后端无关性与边界')
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
class ProfileViolation(Exception):
|
|
45
|
+
"""档位(profile)拒绝:text2query 档违反功能收缩 / 硬限制
|
|
46
|
+
|
|
47
|
+
与权限错误(``PermissionError``,403)区分:档位拒绝是**调用方合约违反**(400),
|
|
48
|
+
非授权问题(见执行文档 §4.4)。实参 ``status`` 供上层(HTTP 网关等)映射响应码。
|
|
49
|
+
"""
|
|
50
|
+
|
|
51
|
+
def __init__(self, message, status=400):
|
|
52
|
+
super().__init__(message)
|
|
53
|
+
self.status = status
|
|
54
|
+
|
|
29
55
|
|
|
30
56
|
def set_db(db):
|
|
31
57
|
"""单库简写:等价于 ``set_connections({'default': db})``"""
|
|
@@ -53,13 +79,33 @@ def _ctx():
|
|
|
53
79
|
|
|
54
80
|
|
|
55
81
|
def _call(fn):
|
|
56
|
-
"""
|
|
82
|
+
"""绑定层调用包装:
|
|
83
|
+
|
|
84
|
+
- 权限类错误(``ERR_PERMISSION:`` 前缀)→ ``PermissionError``
|
|
85
|
+
- 档位类错误(``ERR_TEXT2QUERY:`` 前缀)→ emit ``profile_blocked`` 反馈 + ``ProfileViolation``
|
|
86
|
+
|
|
87
|
+
按前缀映射而非具体文案(core 文案可自由调整,映射不随文案漂移而静默失效)。
|
|
88
|
+
其余异常原样上抛(不吞错)。
|
|
89
|
+
"""
|
|
57
90
|
try:
|
|
58
91
|
return fn()
|
|
59
92
|
except Exception as e:
|
|
60
93
|
msg = str(e)
|
|
61
94
|
if msg.startswith(_PERM_PREFIX):
|
|
62
95
|
raise PermissionError(msg[len(_PERM_PREFIX):]) from e
|
|
96
|
+
if msg.startswith(_PROFILE_PREFIX):
|
|
97
|
+
detail = msg[len(_PROFILE_PREFIX):]
|
|
98
|
+
m = _FEATURE_RE.search(detail)
|
|
99
|
+
_emit_feedback({
|
|
100
|
+
'type': 'profile_blocked',
|
|
101
|
+
'code': 'profileBlocked',
|
|
102
|
+
'layer': 'core',
|
|
103
|
+
'profile': 'text2query',
|
|
104
|
+
'feature': m.group(1) if m else None,
|
|
105
|
+
'message': detail,
|
|
106
|
+
'hint': _PROFILE_HINT,
|
|
107
|
+
})
|
|
108
|
+
raise ProfileViolation(detail) from e
|
|
63
109
|
raise
|
|
64
110
|
|
|
65
111
|
|
|
@@ -6,8 +6,39 @@ import inspect
|
|
|
6
6
|
|
|
7
7
|
from ..feedback import emit as _emit_feedback
|
|
8
8
|
from ..schema import core as _core
|
|
9
|
-
from ..schema import get_async_fn
|
|
10
|
-
from .exec import
|
|
9
|
+
from ..schema import get_async_fn, get_profile
|
|
10
|
+
from .exec import (
|
|
11
|
+
_PROFILE_HINT,
|
|
12
|
+
ProfileViolation,
|
|
13
|
+
_call,
|
|
14
|
+
_ctx,
|
|
15
|
+
_exec,
|
|
16
|
+
_exec_on,
|
|
17
|
+
resolve_placeholders,
|
|
18
|
+
)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def _guard_route_override(route_override):
|
|
22
|
+
"""Host 兜底:text2query 档禁用 ``route_override``(受信参数,禁 AI 侧指定)
|
|
23
|
+
|
|
24
|
+
判决唯一在 core(执行文档 §4.2 ⑤:各 ``plan_query*`` 入口已判并 Err);此为
|
|
25
|
+
第二层防护——即使 core 判决被绕过,Host 也不放行受信参数(CWE-639)。命中即
|
|
26
|
+
emit ``profile_blocked``,``layer='host'`` 本身即反馈:core 层未拦住,须回溯加固
|
|
27
|
+
(自动反馈原则:允许拦截,禁止静默)。
|
|
28
|
+
"""
|
|
29
|
+
if route_override is None or get_profile() != 'text2query':
|
|
30
|
+
return
|
|
31
|
+
detail = 'text2query 档禁用 route_override(受信参数,禁 AI 侧指定)'
|
|
32
|
+
_emit_feedback({
|
|
33
|
+
'type': 'profile_blocked',
|
|
34
|
+
'code': 'profileBlocked',
|
|
35
|
+
'layer': 'host',
|
|
36
|
+
'profile': 'text2query',
|
|
37
|
+
'feature': 'route_override',
|
|
38
|
+
'message': detail,
|
|
39
|
+
'hint': _PROFILE_HINT,
|
|
40
|
+
})
|
|
41
|
+
raise ProfileViolation(detail)
|
|
11
42
|
|
|
12
43
|
|
|
13
44
|
async def _run_query_plan(plan):
|
|
@@ -50,6 +81,7 @@ async def query(gql, params=None, route_override=None):
|
|
|
50
81
|
权限/计算列仍按结构 schema 判定(见 multi-datasource-routing-plan.md §6)。
|
|
51
82
|
注意:``route_override`` 为**受信服务端参数**,禁止透传用户输入(否则可被用于跨源路由,CWE-639)。
|
|
52
83
|
"""
|
|
84
|
+
_guard_route_override(route_override)
|
|
53
85
|
plan = _call(lambda: _core.plan_query(
|
|
54
86
|
gql, params if params is not None else {}, _ctx(), route_override))
|
|
55
87
|
return await _finalize(plan, await _run_query_plan(plan))
|
|
@@ -61,6 +93,7 @@ async def query_one(gql, params=None, route_override=None):
|
|
|
61
93
|
走 core ``plan_query_one``:用户未显式 ``$limit`` 时强制下推 ``$limit(1)``,
|
|
62
94
|
大集合不再全量取回后取首条(对齐 PyMongo ``find_one`` 的 limit-1 语义)。
|
|
63
95
|
"""
|
|
96
|
+
_guard_route_override(route_override)
|
|
64
97
|
plan = _call(lambda: _core.plan_query_one(
|
|
65
98
|
gql, params if params is not None else {}, _ctx(), route_override))
|
|
66
99
|
items = await _finalize(plan, await _run_query_plan(plan))
|
|
@@ -118,6 +151,7 @@ async def query_with_count(gql, params=None, route_override=None):
|
|
|
118
151
|
2. 传统 $skip/$limit — 从 GQL 参数推导 page/pageSize
|
|
119
152
|
pageSize 上限 5000,防止拖库。
|
|
120
153
|
"""
|
|
154
|
+
_guard_route_override(route_override)
|
|
121
155
|
plan = _call(lambda: _core.plan_query_with_count(
|
|
122
156
|
gql, params if params is not None else {}, _ctx(), None, route_override))
|
|
123
157
|
items = await _finalize(plan, await _run_query_plan(plan))
|
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
DDL 生成(schema def → CREATE TABLE 文本;纯函数,不连库、不回写)
|
|
3
3
|
|
|
4
4
|
与 core 契约严格对齐(schema→DDL 单向映射):
|
|
5
|
-
-
|
|
5
|
+
- 标量字段按声明类型建列;object/array 字段建 **JSON 列**(MySQL `JSON` / PG `jsonb` /
|
|
6
|
+
SQLite `TEXT`,同 core dialect::Backend::json_type_name)——落单列存 JSON 文本,
|
|
7
|
+
读侧由 core row::parse_json_col 还原为嵌套对象,跨后端对齐 Mongo 嵌套文档;
|
|
6
8
|
- 每表必建 __present 哨兵列(形态 ,f1,f2,;同 core write/insert.rs::present_value);
|
|
7
9
|
- timestamps !== false → 追加 createdAt / updatedAt(同 core schema/registry.rs::add_timestamp_fields);
|
|
8
10
|
- 归档表 <collection>_deleted 由 registry 自动派生,本模块按已注册 def 逐表生成(不特判);
|
|
@@ -32,6 +34,8 @@ _TYPES = {
|
|
|
32
34
|
'date': ('BIGINT', 'BIGINT', 'INTEGER'),
|
|
33
35
|
}
|
|
34
36
|
_NON_COLUMN = ('object', 'array')
|
|
37
|
+
# object/array 字段的列类型(JSON 文本列;同 core Backend::json_type_name)
|
|
38
|
+
_JSON_TYPE = ('JSON', 'jsonb', 'TEXT')
|
|
35
39
|
_ID_TYPE = ('VARCHAR(64)', 'TEXT', 'TEXT')
|
|
36
40
|
_PRESENT_TYPE = ('VARCHAR(255)', 'TEXT', 'TEXT')
|
|
37
41
|
_TIMESTAMP_FIELDS = ('createdAt', 'updatedAt')
|
|
@@ -56,17 +60,19 @@ def _declared_type(field_def):
|
|
|
56
60
|
|
|
57
61
|
|
|
58
62
|
def _columns(defn, backend):
|
|
59
|
-
"""返回 [(name, sql_type, pk)]
|
|
63
|
+
"""返回 [(name, sql_type, pk)],顺序:声明的字段(标量 / object·array JSON 列)→ timestamps → __present"""
|
|
60
64
|
i = _idx(backend)
|
|
61
65
|
cols = []
|
|
62
66
|
fields = defn.get('fields') or {}
|
|
63
67
|
for name, fdef in fields.items():
|
|
64
68
|
ftype = _declared_type(fdef)
|
|
65
|
-
if ftype in _NON_COLUMN:
|
|
66
|
-
continue
|
|
67
69
|
if name == '_id':
|
|
68
70
|
cols.append((name, _ID_TYPE[i], True))
|
|
69
71
|
continue
|
|
72
|
+
if ftype in _NON_COLUMN:
|
|
73
|
+
# object/array → 单列 JSON 文本(同 core field_column_ref::Json)
|
|
74
|
+
cols.append((name, _JSON_TYPE[i], False))
|
|
75
|
+
continue
|
|
70
76
|
if ftype not in _TYPES:
|
|
71
77
|
raise ValueError(
|
|
72
78
|
f'DDL 生成:字段 "{defn["name"]}.{name}" 类型 {ftype!r} 未知,'
|
|
@@ -117,8 +123,32 @@ def _create_table(defn, backend):
|
|
|
117
123
|
|
|
118
124
|
|
|
119
125
|
def generate(backend, names=None):
|
|
120
|
-
"""生成 DDL 文本(多表以空行分隔);backend ∈ mysql/postgres/sqlite
|
|
126
|
+
"""生成 DDL 文本(多表以空行分隔);backend ∈ mysql/postgres/sqlite
|
|
127
|
+
|
|
128
|
+
按表名去重:同名表只出一次 CREATE TABLE(防御 core 注册表出现重复名 ——
|
|
129
|
+
上游失守即告警,禁静默)。
|
|
130
|
+
"""
|
|
121
131
|
if backend not in _BACKENDS:
|
|
122
132
|
raise ValueError(f'DDL 生成:不支持的后端 {backend!r}(支持 {list(_BACKENDS)})')
|
|
123
133
|
targets = list(names) if names else list(_list_schema())
|
|
124
|
-
|
|
134
|
+
blocks = []
|
|
135
|
+
seen_tables = set()
|
|
136
|
+
dup = []
|
|
137
|
+
for n in targets:
|
|
138
|
+
defn = _get_schema(n)
|
|
139
|
+
table = defn.get('collection') or n
|
|
140
|
+
if table in seen_tables:
|
|
141
|
+
dup.append(table)
|
|
142
|
+
continue
|
|
143
|
+
seen_tables.add(table)
|
|
144
|
+
blocks.append(_create_table(defn, backend))
|
|
145
|
+
if dup:
|
|
146
|
+
_emit_feedback({
|
|
147
|
+
'type': 'ddl_duplicate_table',
|
|
148
|
+
'code': 'ddlDuplicateTable',
|
|
149
|
+
'layer': 'host',
|
|
150
|
+
'message': f'DDL 生成:表 {sorted(set(dup))} 重复注册,已去重',
|
|
151
|
+
'hint': 'schema 注册表出现重复名(见 schemaDuplicateName 告警);修复注册侧根因',
|
|
152
|
+
'backend': backend,
|
|
153
|
+
})
|
|
154
|
+
return '\n\n'.join(blocks)
|
|
@@ -96,9 +96,17 @@ def create(driver, options=None):
|
|
|
96
96
|
return {'docs': docs, 'rows': rows, 'affectedRows': affected_rows}
|
|
97
97
|
|
|
98
98
|
async def exec_(plan):
|
|
99
|
-
|
|
99
|
+
"""非事务路径:整个 plan 固定在同一连接执行,成功后显式 commit
|
|
100
|
+
(池 autocommit=True 时为幂等 no-op;单连接 autocommit=False 时杜绝「只执行不提交」);
|
|
101
|
+
失败则 rollback 后上抛,绝不提交半截写入"""
|
|
100
102
|
async with acquire(driver) as conn:
|
|
101
|
-
|
|
103
|
+
try:
|
|
104
|
+
out = await run_stmts(conn, plan)
|
|
105
|
+
except BaseException:
|
|
106
|
+
await conn.rollback()
|
|
107
|
+
raise
|
|
108
|
+
await conn.commit()
|
|
109
|
+
return out
|
|
102
110
|
|
|
103
111
|
async def with_transaction(body):
|
|
104
112
|
"""事务执行:显式 BEGIN + commit/rollback(对 autocommit 任意配置均确定成立);
|
|
@@ -48,7 +48,16 @@ def create(db, options=None):
|
|
|
48
48
|
return {'docs': docs, 'rows': rows, 'affectedRows': affected_rows}
|
|
49
49
|
|
|
50
50
|
async def exec_(plan):
|
|
51
|
-
|
|
51
|
+
"""非事务路径:plan 成功后显式提交(aiosqlite 默认非 autocommit,
|
|
52
|
+
不提交则仅当前连接可见 —— 落盘场景会丢数据/读到幻象,且无任何告警);
|
|
53
|
+
失败则回滚后上抛,绝不提交半截写入"""
|
|
54
|
+
try:
|
|
55
|
+
out = await run_stmts(plan)
|
|
56
|
+
except BaseException:
|
|
57
|
+
await db.rollback()
|
|
58
|
+
raise
|
|
59
|
+
await db.commit()
|
|
60
|
+
return out
|
|
52
61
|
|
|
53
62
|
async def with_transaction(body):
|
|
54
63
|
"""事务执行:显式 BEGIN + commit/rollback(先清理遗留隐式事务,保证 BEGIN 干净);
|
|
@@ -33,6 +33,7 @@ Schema 管理 — 薄适配层
|
|
|
33
33
|
"""
|
|
34
34
|
|
|
35
35
|
from .core import core
|
|
36
|
+
from .feedback import emit as _emit_feedback
|
|
36
37
|
|
|
37
38
|
# 缓存内置 list 类型(本模块的 list() 函数会遮蔽内置名)
|
|
38
39
|
_LIST_TYPES = (list, tuple)
|
|
@@ -74,7 +75,13 @@ def _to_core_defn(defn):
|
|
|
74
75
|
|
|
75
76
|
|
|
76
77
|
def register(defn):
|
|
77
|
-
"""注册一个 schema
|
|
78
|
+
"""注册一个 schema(core 注册 + Host 侧元数据镜像)
|
|
79
|
+
|
|
80
|
+
归档表 `<Name>Deleted` 由 core 在 register 内**自动派生并注册**
|
|
81
|
+
(rust-store/core/src/schema/registry.rs:52-55);Host 只补 Host 侧镜像,
|
|
82
|
+
**不再调用 core.register** —— 否则同名条目二次进入 core.order,使 list()/
|
|
83
|
+
generate_ddl() 出现重复表(基线实测 list=['User','UserDeleted','UserDeleted'])。
|
|
84
|
+
"""
|
|
78
85
|
core.register(_to_core_defn(defn))
|
|
79
86
|
|
|
80
87
|
# 计算列回调:fn → core 回调桥;asyncFn → Host 侧映射
|
|
@@ -106,19 +113,24 @@ def register(defn):
|
|
|
106
113
|
'datasource': defn.get('datasource'),
|
|
107
114
|
}
|
|
108
115
|
|
|
109
|
-
#
|
|
116
|
+
# 归档表镜像(形状对齐 core archive_defn:registry.rs:395-417)
|
|
117
|
+
# —— 字段 + deletedAt、collection=<c>_deleted、idPrefix=''、timestamps 默认 true
|
|
110
118
|
if not defn.get('_isArchive') and not defn['name'].endswith('Deleted'):
|
|
111
|
-
|
|
119
|
+
_schemas[f"{defn['name']}Deleted"] = {
|
|
112
120
|
'name': f"{defn['name']}Deleted",
|
|
113
121
|
'collection': f"{defn.get('collection') or defn['name']}_deleted",
|
|
114
|
-
'idPrefix': '',
|
|
115
|
-
'_isArchive': True,
|
|
116
|
-
# 归档表与原表同 (source, namespace)
|
|
117
|
-
'datasource': defn.get('datasource'),
|
|
118
122
|
'namespace': defn.get('namespace') or None,
|
|
123
|
+
'idPrefix': '',
|
|
124
|
+
'timestamps': True,
|
|
125
|
+
'timestampUnit': 'ms',
|
|
119
126
|
'fields': {**(defn.get('fields') or {}), 'deletedAt': {'type': 'number'}},
|
|
127
|
+
'relations': {},
|
|
128
|
+
'computes': {},
|
|
120
129
|
'indexes': defn.get('indexes') or [],
|
|
121
|
-
|
|
130
|
+
'read': None,
|
|
131
|
+
'write': None,
|
|
132
|
+
'datasource': defn.get('datasource'),
|
|
133
|
+
}
|
|
122
134
|
|
|
123
135
|
return _schemas[defn['name']]
|
|
124
136
|
|
|
@@ -136,9 +148,38 @@ def has(name):
|
|
|
136
148
|
return core.has(name)
|
|
137
149
|
|
|
138
150
|
|
|
151
|
+
# 去重告警签名(同一重复形态只告警一次,避免 list() 高频调用刷屏)
|
|
152
|
+
_dup_signatures: set = set()
|
|
153
|
+
|
|
154
|
+
|
|
139
155
|
def list():
|
|
140
|
-
"""所有已注册 schema 名称(core
|
|
141
|
-
|
|
156
|
+
"""所有已注册 schema 名称(core 侧,含归档表,按注册顺序;同名只保留首次出现)
|
|
157
|
+
|
|
158
|
+
去重是纵深防御的第二层:一旦检出重复即说明上游(register/core)失守,
|
|
159
|
+
去重同时 emit 告警(同签名只告警一次),禁静默。
|
|
160
|
+
"""
|
|
161
|
+
names = core.list()
|
|
162
|
+
seen = set()
|
|
163
|
+
out = []
|
|
164
|
+
for name in names:
|
|
165
|
+
if name in seen:
|
|
166
|
+
continue
|
|
167
|
+
seen.add(name)
|
|
168
|
+
out.append(name)
|
|
169
|
+
if len(out) != len(names):
|
|
170
|
+
sig = tuple(names)
|
|
171
|
+
if sig not in _dup_signatures:
|
|
172
|
+
_dup_signatures.add(sig)
|
|
173
|
+
_emit_feedback({
|
|
174
|
+
'type': 'schema_duplicate_name',
|
|
175
|
+
'code': 'schemaDuplicateName',
|
|
176
|
+
'layer': 'host',
|
|
177
|
+
'message': f'schema 注册表存在重复名(多 {len(names) - len(out)} 条),已顺序去重',
|
|
178
|
+
'hint': ('上游注册逻辑失守(core.order 同名两次);'
|
|
179
|
+
'核查 register 是否重复调用 core.register,或 core.register 未对同名去重'),
|
|
180
|
+
'names': names,
|
|
181
|
+
})
|
|
182
|
+
return out
|
|
142
183
|
|
|
143
184
|
|
|
144
185
|
def set_require_context(require=True):
|
|
@@ -157,6 +198,20 @@ def require_context():
|
|
|
157
198
|
return core.require_context()
|
|
158
199
|
|
|
159
200
|
|
|
201
|
+
def set_profile(profile):
|
|
202
|
+
"""设置查询档位:'standard'(默认,功能最大化 + 跨 DB 对齐)/
|
|
203
|
+
'text2query'(功能收缩 + 硬限制)
|
|
204
|
+
|
|
205
|
+
判决唯一在 core;未知档位由 core 抛 ValueError 上抛(禁静默回落到默认档)。
|
|
206
|
+
"""
|
|
207
|
+
core.set_profile(profile)
|
|
208
|
+
|
|
209
|
+
|
|
210
|
+
def get_profile():
|
|
211
|
+
"""当前档位字符串('standard' / 'text2query')"""
|
|
212
|
+
return core.profile()
|
|
213
|
+
|
|
214
|
+
|
|
160
215
|
def get_async_fn(fn_ref):
|
|
161
216
|
"""取 asyncFn 计算列实现(fnRef 缺省 = 计算列 key 名)"""
|
|
162
217
|
return _async_fns.get(fn_ref)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: storepy
|
|
3
|
-
Version: 2.1
|
|
3
|
+
Version: 2.3.1
|
|
4
4
|
Summary: Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control
|
|
5
5
|
Author-email: leo <coen_ddt@qq.com>
|
|
6
6
|
License-Expression: MIT
|
|
@@ -33,6 +33,7 @@ Provides-Extra: sqlite
|
|
|
33
33
|
Requires-Dist: aiosqlite>=0.19; extra == "sqlite"
|
|
34
34
|
Provides-Extra: dev
|
|
35
35
|
Requires-Dist: pytest>=8; extra == "dev"
|
|
36
|
+
Requires-Dist: pytest-cov>=5; extra == "dev"
|
|
36
37
|
Requires-Dist: ruff>=0.16; extra == "dev"
|
|
37
38
|
Requires-Dist: mypy>=1.0; extra == "dev"
|
|
38
39
|
Dynamic: license-file
|
|
@@ -372,6 +373,7 @@ await store.transaction("default", transfer)
|
|
|
372
373
|
- `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `execute_raw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `run_in_transaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
|
|
373
374
|
- `store.execute_raw(source, sql, params=None, is_write=False)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source raises `RawSqlError` (`py_store.RawSqlError` / `store.RawSqlError`).
|
|
374
375
|
- `is_write=False` (default) returns `{"rows", "affectedRows"}` with the result-set rows; `is_write=True` returns the affected-row count.
|
|
376
|
+
- **The non-transactional path commits explicitly**: on SQL sources, a write plan that runs outside `store.transaction` is committed by the executor (`commit` on success; `rollback` then re-raise on failure). `aiosqlite` is not autocommit by default, so without that commit the write would be visible only on the current connection while `execute_raw` still reported success — a silent data-loss hazard. Multi-statement writes that must be atomic as a group belong inside `store.transaction`.
|
|
375
377
|
|
|
376
378
|
### DDL generation
|
|
377
379
|
|
|
@@ -384,6 +386,7 @@ sql = store.generate_ddl("postgres", ["Course", "CourseDeleted"])
|
|
|
384
386
|
|
|
385
387
|
- Only scalar fields become columns; `object` / `array` fields do not.
|
|
386
388
|
- Every table gets the `__present` sentinel column; `timestamps` models also get `createdAt` / `updatedAt`; the `<collection>_deleted` archive table is generated like any other registered def.
|
|
389
|
+
- The `<Name>Deleted` archive def is **derived by the Rust core when the model is registered**; the Python host only mirrors it (`collection` `<c>_deleted`, the `deletedAt` field, empty `idPrefix`) and never re-registers it into the core. So `schema.list()` and `generate_ddl()` contain each archive table exactly once; if a duplicate ever appears (upstream regression), they deduplicate and emit a `schemaDuplicateName` / `ddlDuplicateTable` feedback event instead of silently emitting duplicate `CREATE TABLE`s.
|
|
387
390
|
- No `CREATE INDEX` is emitted — SQL backends keep indexes as metadata only.
|
|
388
391
|
- MySQL `__present` is `VARCHAR(255)`; a schema whose present-token string would overflow emits a `ddlPresentOverflow` feedback event rather than failing silently.
|
|
389
392
|
|
|
@@ -30,12 +30,16 @@ src/storepy.egg-info/SOURCES.txt
|
|
|
30
30
|
src/storepy.egg-info/dependency_links.txt
|
|
31
31
|
src/storepy.egg-info/requires.txt
|
|
32
32
|
src/storepy.egg-info/top_level.txt
|
|
33
|
+
tests/test_coverage_margin.py
|
|
33
34
|
tests/test_federation_e2e.py
|
|
34
35
|
tests/test_host_contract.py
|
|
36
|
+
tests/test_host_paths.py
|
|
35
37
|
tests/test_mongo_executor.py
|
|
36
38
|
tests/test_multi_datasource.py
|
|
39
|
+
tests/test_profile.py
|
|
37
40
|
tests/test_py_store.py
|
|
38
41
|
tests/test_raw_sql_and_ddl.py
|
|
39
42
|
tests/test_real_backends_e2e.py
|
|
40
43
|
tests/test_require_context.py
|
|
41
|
-
tests/test_scenario_course_platform.py
|
|
44
|
+
tests/test_scenario_course_platform.py
|
|
45
|
+
tests/test_sql_commit_and_registry.py
|