storepy 2.0.4__tar.gz → 2.3.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. {storepy-2.0.4/src/storepy.egg-info → storepy-2.3.1}/PKG-INFO +39 -5
  2. {storepy-2.0.4 → storepy-2.3.1}/README.md +37 -4
  3. {storepy-2.0.4 → storepy-2.3.1}/pyproject.toml +14 -2
  4. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/__init__.py +66 -2
  5. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/__init__.py +2 -1
  6. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/exec.py +47 -1
  7. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/query.py +36 -2
  8. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/datasource.py +27 -0
  9. storepy-2.3.1/src/py_store/ddl.py +154 -0
  10. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/mysql.py +10 -2
  11. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/postgres.py +3 -2
  12. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/sqlite.py +10 -1
  13. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/schema.py +65 -10
  14. {storepy-2.0.4 → storepy-2.3.1/src/storepy.egg-info}/PKG-INFO +39 -5
  15. {storepy-2.0.4 → storepy-2.3.1}/src/storepy.egg-info/SOURCES.txt +7 -1
  16. {storepy-2.0.4 → storepy-2.3.1}/src/storepy.egg-info/requires.txt +1 -0
  17. storepy-2.3.1/tests/test_coverage_margin.py +511 -0
  18. storepy-2.3.1/tests/test_host_paths.py +418 -0
  19. storepy-2.3.1/tests/test_profile.py +242 -0
  20. storepy-2.3.1/tests/test_raw_sql_and_ddl.py +224 -0
  21. {storepy-2.0.4 → storepy-2.3.1}/tests/test_scenario_course_platform.py +2 -2
  22. storepy-2.3.1/tests/test_sql_commit_and_registry.py +120 -0
  23. {storepy-2.0.4 → storepy-2.3.1}/LICENSE +0 -0
  24. {storepy-2.0.4 → storepy-2.3.1}/setup.cfg +0 -0
  25. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/core.py +0 -0
  26. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/id.py +0 -0
  27. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/mutation.py +0 -0
  28. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/crud/write.py +0 -0
  29. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/__init__.py +0 -0
  30. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/_values.py +0 -0
  31. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/executors/mongo.py +0 -0
  32. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/feedback.py +0 -0
  33. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/introspect/__init__.py +0 -0
  34. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/introspect/mysql.py +0 -0
  35. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/introspect/postgres.py +0 -0
  36. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/introspect/sqlite.py +0 -0
  37. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/permission.py +0 -0
  38. {storepy-2.0.4 → storepy-2.3.1}/src/py_store/sync.py +0 -0
  39. {storepy-2.0.4 → storepy-2.3.1}/src/storepy.egg-info/dependency_links.txt +0 -0
  40. {storepy-2.0.4 → storepy-2.3.1}/src/storepy.egg-info/top_level.txt +0 -0
  41. {storepy-2.0.4 → storepy-2.3.1}/tests/test_federation_e2e.py +0 -0
  42. {storepy-2.0.4 → storepy-2.3.1}/tests/test_host_contract.py +0 -0
  43. {storepy-2.0.4 → storepy-2.3.1}/tests/test_mongo_executor.py +0 -0
  44. {storepy-2.0.4 → storepy-2.3.1}/tests/test_multi_datasource.py +0 -0
  45. {storepy-2.0.4 → storepy-2.3.1}/tests/test_py_store.py +0 -0
  46. {storepy-2.0.4 → storepy-2.3.1}/tests/test_real_backends_e2e.py +0 -0
  47. {storepy-2.0.4 → storepy-2.3.1}/tests/test_require_context.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: storepy
3
- Version: 2.0.4
3
+ Version: 2.3.1
4
4
  Summary: Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control
5
5
  Author-email: leo <coen_ddt@qq.com>
6
6
  License-Expression: MIT
@@ -33,6 +33,7 @@ Provides-Extra: sqlite
33
33
  Requires-Dist: aiosqlite>=0.19; extra == "sqlite"
34
34
  Provides-Extra: dev
35
35
  Requires-Dist: pytest>=8; extra == "dev"
36
+ Requires-Dist: pytest-cov>=5; extra == "dev"
36
37
  Requires-Dist: ruff>=0.16; extra == "dev"
37
38
  Requires-Dist: mypy>=1.0; extra == "dev"
38
39
  Dynamic: license-file
@@ -147,7 +148,7 @@ Typical concrete scenarios (see [`doc/use-cases/`](doc/use-cases/) for full walk
147
148
 
148
149
  Being explicit about the boundary saves you time:
149
150
 
150
- - **You want a full ORM with a migration engine (Alembic, Django migrations).** `py-store` is a *data layer*, not a migration tool. It can **read** a SQL backend's physical structure (`sync_schema` → introspection) but it never writes DDL back.
151
+ - **You want a full ORM with a migration engine (Alembic, Django migrations).** `py-store` is a *data layer*, not a migration tool. It can **read** a SQL backend's physical structure (`sync_schema` → introspection) but it never writes DDL back. There is an optional `generate_ddl()` that renders `CREATE TABLE` text from your registered schemas — pure text, it never connects to or writes to the database.
151
152
  - **You want a Pydantic-model-centric ORM.** Schemas here are runtime JSON dicts, giving you cross-language parity (the same schema runs in Python and Node.js) rather than Pydantic type validation.
152
153
  - **You only ever use one database and rarely join.** A plain driver (or a single-database ODM/ORM) will be simpler.
153
154
  - **You need raw aggregation escape hatches.** `$pipeline` passthrough and `store.aggregate()` were deliberately removed. Use `$condition` / `$group` / `$having` / relations; anything that cannot be safely translated fails **explicitly** rather than silently.
@@ -165,7 +166,7 @@ General positioning, not a benchmark — always verify against each tool's curre
165
166
  | Built-in role / field-level RBAC + owner injection | ✅ | ➖ | ➖ | ➖ | ➖ | ➖ (permissions are app-level) |
166
167
  | Read-time computed columns (sync / async / relation-agg) | ✅ | ➖ (hybrid properties) | ➖ | ➖ | ➖ | ➖ |
167
168
  | Soft-delete archive table auto-provisioned | ✅ | ➖ | ➖ | ➖ | ➖ | ➖ |
168
- | Migration / DDL engine | ➖ (introspection read-only) | ✅ (Alembic) | ➖ | ✅ (Aerich) | ✅ (Alembic) | ✅ |
169
+ | Migration / DDL engine | ➖ (introspection read-only; optional `generate_ddl` text) | ✅ (Alembic) | ➖ | ✅ (Aerich) | ✅ (Alembic) | ✅ |
169
170
  | Framework coupling | none (asyncio) | none | none | none | none | Django |
170
171
  | Shared native core across Python & Node | ✅ (Rust `rust-store`) | ➖ | ➖ | ➖ | ➖ | ➖ |
171
172
 
@@ -175,7 +176,7 @@ Positioning only, based on those projects' public documentation at the time of w
175
176
 
176
177
  - **vs SQLAlchemy / SQLModel / Django ORM** — all SQL-only and model-class-centric: they do not target MongoDB, and none of them ships schema-declared role/field access control or read-time computed columns. `py-store` compiles one GQL to native MongoDB aggregation or to parameterized SQL.
177
178
  - **vs Beanie / Motor** — MongoDB-only. `py-store` uses the same MongoDB-flavoured query style but the identical query also runs on MySQL, SQLite and PostgreSQL.
178
- - **vs Tortoise ORM / pyloquent** — async Python ORMs over SQL backends, with model classes and (in Tortoise's case) a migration tool. `py-store` has no migration engine — introspection reads physical structure only — and describes models as plain dicts, which is exactly what makes a schema portable to the Node.js host.
179
+ - **vs Tortoise ORM / pyloquent** — async Python ORMs over SQL backends, with model classes and (in Tortoise's case) a migration tool. `py-store` has no migration engine — introspection only reads physical structure, and `generate_ddl()` only *renders* `CREATE TABLE` text without touching the database — and describes models as plain dicts, which is exactly what makes a schema portable to the Node.js host.
179
180
  - **vs `nodejs-store`** — the same engine and the same GQL, in JavaScript. Use whichever host matches your service; schemas and query semantics are interchangeable.
180
181
 
181
182
  Short version: use an ORM when you want **model classes, Pydantic validation and migrations**; use `py-store` when you want **one runtime schema + one query dialect spanning MongoDB and SQL**, with RBAC and computed columns built in.
@@ -356,6 +357,39 @@ Notes:
356
357
  - `update_many` / `remove` with an **empty condition** (`{}`, `None`, `{"$and": []}`) is rejected outright — it never falls through to a full-table write.
357
358
  - Snake-case aliases available: `query_one`, `insert_many`, `update_many`, `build_pipeline`, ...
358
359
 
360
+ ### Transactions and raw SQL
361
+
362
+ ```python
363
+ async def transfer():
364
+ rows = await store.execute_raw(
365
+ "default", "SELECT * FROM accounts WHERE _id = ? FOR UPDATE", [acc_id])
366
+ await store.execute_raw(
367
+ "default", "UPDATE accounts SET balance = ? WHERE _id = ?", [new_balance, acc_id],
368
+ is_write=True)
369
+
370
+ await store.transaction("default", transfer)
371
+ ```
372
+
373
+ - `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `execute_raw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `run_in_transaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
374
+ - `store.execute_raw(source, sql, params=None, is_write=False)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source raises `RawSqlError` (`py_store.RawSqlError` / `store.RawSqlError`).
375
+ - `is_write=False` (default) returns `{"rows", "affectedRows"}` with the result-set rows; `is_write=True` returns the affected-row count.
376
+ - **The non-transactional path commits explicitly**: on SQL sources, a write plan that runs outside `store.transaction` is committed by the executor (`commit` on success; `rollback` then re-raise on failure). `aiosqlite` is not autocommit by default, so without that commit the write would be visible only on the current connection while `execute_raw` still reported success — a silent data-loss hazard. Multi-statement writes that must be atomic as a group belong inside `store.transaction`.
377
+
378
+ ### DDL generation
379
+
380
+ ```python
381
+ sql = store.generate_ddl("mysql") # every registered model
382
+ sql = store.generate_ddl("postgres", ["Course", "CourseDeleted"])
383
+ ```
384
+
385
+ `store.generate_ddl(backend, names=None)` maps one registered schema def to one `CREATE TABLE` — the inverse of `sync_schema()`, which only *reads*. The generator is **pure text**: it never connects to, or writes to, the database (iron rule 6 still holds).
386
+
387
+ - Only scalar fields become columns; `object` / `array` fields do not.
388
+ - Every table gets the `__present` sentinel column; `timestamps` models also get `createdAt` / `updatedAt`; the `<collection>_deleted` archive table is generated like any other registered def.
389
+ - The `<Name>Deleted` archive def is **derived by the Rust core when the model is registered**; the Python host only mirrors it (`collection` `<c>_deleted`, the `deletedAt` field, empty `idPrefix`) and never re-registers it into the core. So `schema.list()` and `generate_ddl()` contain each archive table exactly once; if a duplicate ever appears (upstream regression), they deduplicate and emit a `schemaDuplicateName` / `ddlDuplicateTable` feedback event instead of silently emitting duplicate `CREATE TABLE`s.
390
+ - No `CREATE INDEX` is emitted — SQL backends keep indexes as metadata only.
391
+ - MySQL `__present` is `VARCHAR(255)`; a schema whose present-token string would overflow emits a `ddlPresentOverflow` feedback event rather than failing silently.
392
+
359
393
  ## Multi-datasource connections
360
394
 
361
395
  Every schema is located by the triple `(source, namespace, collection)` — the triple must be
@@ -513,7 +547,7 @@ Every registered model automatically gets a `<Model>Deleted` archive collection/
513
547
  Yes. Bind a schema to `(source, namespace, collection)` and pass a `{"source", "namespace"}` route override per request. Treat `route_override` as trusted server-side input only.
514
548
 
515
549
  **Does it run migrations?**
516
- No. `sync_schema()` only *reads* physical structure via introspection (introspect → merge overlay → register). Schema changes / DDL are your migration tool's job (Alembic, etc.).
550
+ No. `sync_schema()` only *reads* physical structure via introspection (introspect → merge overlay → register). Schema changes / DDL are your migration tool's job (Alembic, etc.). If you want a starting point, `store.generate_ddl(backend)` renders `CREATE TABLE` text from the registered schemas — but it is pure text generation: it never runs or writes DDL.
517
551
 
518
552
  **Can I see the generated query without running it?**
519
553
  Yes — `store.build_pipeline(gql, params)` returns the compiled plan with no execution and no permission/compute application.
@@ -108,7 +108,7 @@ Typical concrete scenarios (see [`doc/use-cases/`](doc/use-cases/) for full walk
108
108
 
109
109
  Being explicit about the boundary saves you time:
110
110
 
111
- - **You want a full ORM with a migration engine (Alembic, Django migrations).** `py-store` is a *data layer*, not a migration tool. It can **read** a SQL backend's physical structure (`sync_schema` → introspection) but it never writes DDL back.
111
+ - **You want a full ORM with a migration engine (Alembic, Django migrations).** `py-store` is a *data layer*, not a migration tool. It can **read** a SQL backend's physical structure (`sync_schema` → introspection) but it never writes DDL back. There is an optional `generate_ddl()` that renders `CREATE TABLE` text from your registered schemas — pure text, it never connects to or writes to the database.
112
112
  - **You want a Pydantic-model-centric ORM.** Schemas here are runtime JSON dicts, giving you cross-language parity (the same schema runs in Python and Node.js) rather than Pydantic type validation.
113
113
  - **You only ever use one database and rarely join.** A plain driver (or a single-database ODM/ORM) will be simpler.
114
114
  - **You need raw aggregation escape hatches.** `$pipeline` passthrough and `store.aggregate()` were deliberately removed. Use `$condition` / `$group` / `$having` / relations; anything that cannot be safely translated fails **explicitly** rather than silently.
@@ -126,7 +126,7 @@ General positioning, not a benchmark — always verify against each tool's curre
126
126
  | Built-in role / field-level RBAC + owner injection | ✅ | ➖ | ➖ | ➖ | ➖ | ➖ (permissions are app-level) |
127
127
  | Read-time computed columns (sync / async / relation-agg) | ✅ | ➖ (hybrid properties) | ➖ | ➖ | ➖ | ➖ |
128
128
  | Soft-delete archive table auto-provisioned | ✅ | ➖ | ➖ | ➖ | ➖ | ➖ |
129
- | Migration / DDL engine | ➖ (introspection read-only) | ✅ (Alembic) | ➖ | ✅ (Aerich) | ✅ (Alembic) | ✅ |
129
+ | Migration / DDL engine | ➖ (introspection read-only; optional `generate_ddl` text) | ✅ (Alembic) | ➖ | ✅ (Aerich) | ✅ (Alembic) | ✅ |
130
130
  | Framework coupling | none (asyncio) | none | none | none | none | Django |
131
131
  | Shared native core across Python & Node | ✅ (Rust `rust-store`) | ➖ | ➖ | ➖ | ➖ | ➖ |
132
132
 
@@ -136,7 +136,7 @@ Positioning only, based on those projects' public documentation at the time of w
136
136
 
137
137
  - **vs SQLAlchemy / SQLModel / Django ORM** — all SQL-only and model-class-centric: they do not target MongoDB, and none of them ships schema-declared role/field access control or read-time computed columns. `py-store` compiles one GQL to native MongoDB aggregation or to parameterized SQL.
138
138
  - **vs Beanie / Motor** — MongoDB-only. `py-store` uses the same MongoDB-flavoured query style but the identical query also runs on MySQL, SQLite and PostgreSQL.
139
- - **vs Tortoise ORM / pyloquent** — async Python ORMs over SQL backends, with model classes and (in Tortoise's case) a migration tool. `py-store` has no migration engine — introspection reads physical structure only — and describes models as plain dicts, which is exactly what makes a schema portable to the Node.js host.
139
+ - **vs Tortoise ORM / pyloquent** — async Python ORMs over SQL backends, with model classes and (in Tortoise's case) a migration tool. `py-store` has no migration engine — introspection only reads physical structure, and `generate_ddl()` only *renders* `CREATE TABLE` text without touching the database — and describes models as plain dicts, which is exactly what makes a schema portable to the Node.js host.
140
140
  - **vs `nodejs-store`** — the same engine and the same GQL, in JavaScript. Use whichever host matches your service; schemas and query semantics are interchangeable.
141
141
 
142
142
  Short version: use an ORM when you want **model classes, Pydantic validation and migrations**; use `py-store` when you want **one runtime schema + one query dialect spanning MongoDB and SQL**, with RBAC and computed columns built in.
@@ -317,6 +317,39 @@ Notes:
317
317
  - `update_many` / `remove` with an **empty condition** (`{}`, `None`, `{"$and": []}`) is rejected outright — it never falls through to a full-table write.
318
318
  - Snake-case aliases available: `query_one`, `insert_many`, `update_many`, `build_pipeline`, ...
319
319
 
320
+ ### Transactions and raw SQL
321
+
322
+ ```python
323
+ async def transfer():
324
+ rows = await store.execute_raw(
325
+ "default", "SELECT * FROM accounts WHERE _id = ? FOR UPDATE", [acc_id])
326
+ await store.execute_raw(
327
+ "default", "UPDATE accounts SET balance = ? WHERE _id = ?", [new_balance, acc_id],
328
+ is_write=True)
329
+
330
+ await store.transaction("default", transfer)
331
+ ```
332
+
333
+ - `store.transaction(source, fn)` opens a transaction scope on one SQL source: every `execute_raw` / CRUD call inside `fn` lands on that source's transaction connection, with `commit` / `rollback` as one unit (reuses the internal `run_in_transaction`). Mongo sources or executors without transactions run `fn` as-is — it never pretends to be atomic.
334
+ - `store.execute_raw(source, sql, params=None, is_write=False)` runs raw SQL, bypassing GQL parsing and dialect translation. Placeholders follow each backend's native style: `?` for MySQL / SQLite, `$1..$n` for PostgreSQL. SQL sources only — a Mongo source raises `RawSqlError` (`py_store.RawSqlError` / `store.RawSqlError`).
335
+ - `is_write=False` (default) returns `{"rows", "affectedRows"}` with the result-set rows; `is_write=True` returns the affected-row count.
336
+ - **The non-transactional path commits explicitly**: on SQL sources, a write plan that runs outside `store.transaction` is committed by the executor (`commit` on success; `rollback` then re-raise on failure). `aiosqlite` is not autocommit by default, so without that commit the write would be visible only on the current connection while `execute_raw` still reported success — a silent data-loss hazard. Multi-statement writes that must be atomic as a group belong inside `store.transaction`.
337
+
338
+ ### DDL generation
339
+
340
+ ```python
341
+ sql = store.generate_ddl("mysql") # every registered model
342
+ sql = store.generate_ddl("postgres", ["Course", "CourseDeleted"])
343
+ ```
344
+
345
+ `store.generate_ddl(backend, names=None)` maps one registered schema def to one `CREATE TABLE` — the inverse of `sync_schema()`, which only *reads*. The generator is **pure text**: it never connects to, or writes to, the database (iron rule 6 still holds).
346
+
347
+ - Only scalar fields become columns; `object` / `array` fields do not.
348
+ - Every table gets the `__present` sentinel column; `timestamps` models also get `createdAt` / `updatedAt`; the `<collection>_deleted` archive table is generated like any other registered def.
349
+ - The `<Name>Deleted` archive def is **derived by the Rust core when the model is registered**; the Python host only mirrors it (`collection` `<c>_deleted`, the `deletedAt` field, empty `idPrefix`) and never re-registers it into the core. So `schema.list()` and `generate_ddl()` contain each archive table exactly once; if a duplicate ever appears (upstream regression), they deduplicate and emit a `schemaDuplicateName` / `ddlDuplicateTable` feedback event instead of silently emitting duplicate `CREATE TABLE`s.
350
+ - No `CREATE INDEX` is emitted — SQL backends keep indexes as metadata only.
351
+ - MySQL `__present` is `VARCHAR(255)`; a schema whose present-token string would overflow emits a `ddlPresentOverflow` feedback event rather than failing silently.
352
+
320
353
  ## Multi-datasource connections
321
354
 
322
355
  Every schema is located by the triple `(source, namespace, collection)` — the triple must be
@@ -474,7 +507,7 @@ Every registered model automatically gets a `<Model>Deleted` archive collection/
474
507
  Yes. Bind a schema to `(source, namespace, collection)` and pass a `{"source", "namespace"}` route override per request. Treat `route_override` as trusted server-side input only.
475
508
 
476
509
  **Does it run migrations?**
477
- No. `sync_schema()` only *reads* physical structure via introspection (introspect → merge overlay → register). Schema changes / DDL are your migration tool's job (Alembic, etc.).
510
+ No. `sync_schema()` only *reads* physical structure via introspection (introspect → merge overlay → register). Schema changes / DDL are your migration tool's job (Alembic, etc.). If you want a starting point, `store.generate_ddl(backend)` renders `CREATE TABLE` text from the registered schemas — but it is pure text generation: it never runs or writes DDL.
478
511
 
479
512
  **Can I see the generated query without running it?**
480
513
  Yes — `store.build_pipeline(gql, params)` returns the compiled plan with no execution and no permission/compute application.
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "storepy"
7
- version = "2.0.4"
7
+ version = "2.3.1"
8
8
  description = "Multi-backend data layer for Python asyncio (MongoDB, MySQL, SQLite, PostgreSQL): pure JSON schemas, GQL tree queries compiled to a single native query, GROUP BY/HAVING aggregation, computed columns, soft-delete and role-based access control"
9
9
  readme = "README.md"
10
10
  requires-python = ">=3.10"
@@ -45,7 +45,7 @@ classifiers = [
45
45
  mysql = ["asyncmy>=0.2.9"]
46
46
  postgres = ["asyncpg>=0.29"]
47
47
  sqlite = ["aiosqlite>=0.19"]
48
- dev = ["pytest>=8", "ruff>=0.16", "mypy>=1.0"]
48
+ dev = ["pytest>=8", "pytest-cov>=5", "ruff>=0.16", "mypy>=1.0"]
49
49
 
50
50
  [project.urls]
51
51
  Homepage = "https://github.com/coenddt/py-store"
@@ -83,3 +83,15 @@ python_version = "3.10"
83
83
  # 宽松起步:py_store 为动态薄适配层,注解覆盖不足,先只挡「导入失败/明显类型矛盾」
84
84
  ignore_missing_imports = true
85
85
  check_untyped_defs = true
86
+
87
+ # ── 覆盖率(门禁口径:语句 ≥90%,见 .github/workflows/ci.yml;分支单独出具) ──
88
+ [tool.coverage.run]
89
+ source = ["py_store"]
90
+ relative_files = true
91
+
92
+ [tool.coverage.report]
93
+ show_missing = true
94
+ exclude_lines = [
95
+ "pragma: no cover",
96
+ "if TYPE_CHECKING:",
97
+ ]
@@ -20,6 +20,7 @@ py-store — 轻量多后端数据层(Python 版,Rust 单核心架构;支
20
20
  """
21
21
 
22
22
  from collections.abc import Mapping
23
+ from contextlib import contextmanager
23
24
  from typing import Any
24
25
 
25
26
  from pymongo.errors import PyMongoError
@@ -27,6 +28,7 @@ from pymongo.errors import PyMongoError
27
28
  from . import (
28
29
  crud,
29
30
  datasource,
31
+ ddl,
30
32
  feedback,
31
33
  permission,
32
34
  schema,
@@ -37,6 +39,7 @@ from . import (
37
39
  from . import (
38
40
  introspect as introspect,
39
41
  )
42
+ from .datasource import RawSqlError as RawSqlError
40
43
  from .sync import sync_schema
41
44
 
42
45
 
@@ -46,6 +49,21 @@ def _build_pipeline(gql, params=None):
46
49
  gql, params if params is not None else {}, permission.get_context())
47
50
 
48
51
 
52
+ @contextmanager
53
+ def text2query():
54
+ """以 text2query 档执行(功能收缩 + 硬限制),退出恢复原档位。
55
+
56
+ AI 问数链路入口;与 ``permission.scoped_roles`` 同构(token-set/reset,嵌套安全)。
57
+ 进入档位即等效强制携带用户上下文(core `ensure_profile_ctx`,见执行文档 §4.2)。
58
+ """
59
+ prev = schema.get_profile()
60
+ schema.set_profile('text2query')
61
+ try:
62
+ yield
63
+ finally:
64
+ schema.set_profile(prev)
65
+
66
+
49
67
  class Store:
50
68
  """以属性方式访问各 API(**全显式类方法,无动态查找**)
51
69
 
@@ -120,6 +138,29 @@ class Store:
120
138
  def build_pipeline(self, gql: str, params: dict | None = None) -> dict[str, Any]:
121
139
  return _build_pipeline(gql, params)
122
140
 
141
+ # ── 事务 + 原生 SQL(复用 datasource.run_in_transaction;见 README「事务边界」)──
142
+ async def transaction(self, source: str, fn) -> Any:
143
+ """事务作用域:单 SQL 源「同连接 + 同事务」执行 fn(复用 run_in_transaction)
144
+
145
+ fn 内 execute_raw / CRUD 均落到该源的事务连接(commit/rollback 一体);
146
+ Mongo 源或执行器未实现事务时按原样执行(跨源无法原子),绝不静默假装已事务化。
147
+ 单源场景 source 传 ``'default'``。
148
+ """
149
+ return await datasource.run_in_transaction(source, fn)
150
+
151
+ async def execute_raw(self, source: str, sql: str, params: list | None = None,
152
+ is_write: bool = False) -> dict[str, Any]:
153
+ """在指定 SQL 源执行原生 SQL(事务内可用;占位符按各后端原生风格)
154
+
155
+ mysql/sqlite 用 ``?``,postgres 用 ``$1..$n``;仅支持 SQL 源(Mongo 源抛
156
+ RawSqlError)。``is_write=False`` 取行(rows),``True`` 取影响行数(affectedRows)。
157
+ """
158
+ return await datasource.execute_raw(source, sql, params, is_write)
159
+
160
+ def generate_ddl(self, backend: str, names: list | None = None) -> str:
161
+ """从已注册 schema def 生成指定后端 DDL 文本(纯函数,不连库、不回写;铁律 6)"""
162
+ return ddl.generate(backend, names)
163
+
123
164
  # ── 驼峰别名(与上方同名蛇形方法为**同一实现**,仅命名差异)──
124
165
  queryOne = query_one
125
166
  queryWithCount = query_with_count
@@ -128,6 +169,8 @@ class Store:
128
169
  updateMany = update_many
129
170
  syncSchema = sync_schema
130
171
  buildPipeline = build_pipeline
172
+ executeRaw = execute_raw
173
+ generateDdl = generate_ddl
131
174
 
132
175
  # ── 其余 API 显式绑定(staticmethod:避免实例化后 self 注入)──
133
176
  # Schema 管理
@@ -148,11 +191,23 @@ class Store:
148
191
  run_as_internal = staticmethod(permission.run_as_internal)
149
192
  # 自定义权限错误(实例可被 store.PermissionError 捕获)
150
193
  PermissionError = permission.PermissionError
194
+ # 原生 SQL 入口错误(实例可被 store.RawSqlError 捕获)
195
+ RawSqlError = datasource.RawSqlError
151
196
  # 上下文强制开关(fail-secure:开启后 ctx 缺失报 ERR_NO_CONTEXT,内部调用走 run_as_internal)
152
197
  setRequireContext = staticmethod(schema.set_require_context)
153
198
  set_require_context = staticmethod(schema.set_require_context)
154
199
  requireContext = staticmethod(schema.require_context)
155
200
  require_context = staticmethod(schema.require_context)
201
+ # 查询档位(判决唯一在 core):standard 默认放开 / text2query 功能收缩
202
+ # 进入档即等效强制 ctx;未知档由 core 抛 ValueError 上抛(禁静默回落默认档)
203
+ setProfile = staticmethod(schema.set_profile)
204
+ set_profile = staticmethod(schema.set_profile)
205
+ getProfile = staticmethod(schema.get_profile)
206
+ get_profile = staticmethod(schema.get_profile)
207
+ # 档位拒绝错误(实例可被 store.ProfileViolation 捕获;权限错误另见 PermissionError)
208
+ ProfileViolation = crud.ProfileViolation
209
+ # text2query 便捷上下文(进入设档、退出恢复;同 scoped_roles 的 token-set/reset)
210
+ text2query = staticmethod(text2query)
156
211
  # 反馈事件通道(兜底/降级/拦截的统一出口,接入自动反馈闭环)
157
212
  setFeedbackSink = staticmethod(feedback.set_sink)
158
213
  set_feedback_sink = staticmethod(feedback.set_sink)
@@ -200,8 +255,17 @@ async def _create_indexes_if_needed():
200
255
 
201
256
  await coll.create_index(list(keys.items()), **final_options)
202
257
  except PyMongoError as e:
203
- import sys
204
- print(f'[py-store] 创建索引失败 {s["collection"]}: {e}', file=sys.stderr)
258
+ # 索引创建失败不阻塞 init(辅助动作),但必须走统一反馈通道:
259
+ # 无 sink 时由 feedback 默认落 stderr(不双份打印),宿主可 set_sink 接管。
260
+ # 对齐 nodejs-store/src/index.js(评审项 R7-m1)。
261
+ feedback.emit({
262
+ 'type': 'index_create_failed',
263
+ 'code': 'indexCreateFailed',
264
+ 'layer': 'host',
265
+ 'message': f'创建索引失败 {s["collection"]}: {e}',
266
+ 'hint': ('检查该集合的索引定义与连接权限;'
267
+ '索引缺失不影响读写,相关查询将退化为全表扫描'),
268
+ })
205
269
 
206
270
 
207
271
  async def init(connections):
@@ -7,12 +7,13 @@ CRUD 包 —— 薄 Host 适配层(对齐 nodejs-store/src/crud.js 的分工
7
7
  - 写入时不补默认值(DB 存最少数据);读取时由 core 补默认值 + 计算列
8
8
  """
9
9
 
10
- from .exec import _get_db, set_connections, set_db
10
+ from .exec import ProfileViolation, _get_db, set_connections, set_db
11
11
  from .mutation import mutation, upsert
12
12
  from .query import query, query_federated, query_one, query_with_count
13
13
  from .write import count, exists, insert, insert_many, remove, update, update_many
14
14
 
15
15
  __all__ = [
16
+ 'ProfileViolation',
16
17
  '_get_db',
17
18
  'count',
18
19
  'exists',
@@ -15,6 +15,7 @@ import time
15
15
 
16
16
  from .. import datasource as _datasource
17
17
  from ..executors.mongo import exec_mongo as _exec_mongo
18
+ from ..feedback import emit as _emit_feedback
18
19
  from ..permission import PermissionError, get_context
19
20
  from ..schema import get as _schema_get
20
21
 
@@ -26,6 +27,31 @@ _STEP_PH = re.compile(r'^\{\{step\.(\d+)\._id\}\}$')
26
27
  # 可自由调整,映射不随文案漂移而静默失效。构造 PermissionError 时剥离前缀。
27
28
  _PERM_PREFIX = 'ERR_PERMISSION:'
28
29
 
30
+ # 档位类错误识别:core text2query 档门禁统一携带 `ERR_TEXT2QUERY:` 稳定前缀
31
+ # (见 core `command/mod.rs::ERR_TEXT2QUERY`),同上按前缀映射。命中即 emit
32
+ # 反馈事件 `profile_blocked`(自动反馈原则:允许拦截,禁止静默)。
33
+ _PROFILE_PREFIX = 'ERR_TEXT2QUERY:'
34
+
35
+ # 从 core 文案 `... [$feature](功能收缩)` 中提取门禁项名;无 `[..]` 时留白(None),
36
+ # 不伪造 feature —— 缺值必须显式暴露(禁静默兜底)。
37
+ _FEATURE_RE = re.compile(r'\[(.+?)\]')
38
+
39
+ # 档位拦截反馈的统一提示(反馈事件契约 §4.6 的一部分;单点定义防文案漂移)
40
+ _PROFILE_HINT = ('上游(LLM 产出的 GQL / 调用方入参)越界;'
41
+ 'text2query 档白名单见 SKILL.md §后端无关性与边界')
42
+
43
+
44
+ class ProfileViolation(Exception):
45
+ """档位(profile)拒绝:text2query 档违反功能收缩 / 硬限制
46
+
47
+ 与权限错误(``PermissionError``,403)区分:档位拒绝是**调用方合约违反**(400),
48
+ 非授权问题(见执行文档 §4.4)。实参 ``status`` 供上层(HTTP 网关等)映射响应码。
49
+ """
50
+
51
+ def __init__(self, message, status=400):
52
+ super().__init__(message)
53
+ self.status = status
54
+
29
55
 
30
56
  def set_db(db):
31
57
  """单库简写:等价于 ``set_connections({'default': db})``"""
@@ -53,13 +79,33 @@ def _ctx():
53
79
 
54
80
 
55
81
  def _call(fn):
56
- """绑定层调用包装:权限类错误(``ERR_PERMISSION:`` 前缀)映射为 PermissionError"""
82
+ """绑定层调用包装:
83
+
84
+ - 权限类错误(``ERR_PERMISSION:`` 前缀)→ ``PermissionError``
85
+ - 档位类错误(``ERR_TEXT2QUERY:`` 前缀)→ emit ``profile_blocked`` 反馈 + ``ProfileViolation``
86
+
87
+ 按前缀映射而非具体文案(core 文案可自由调整,映射不随文案漂移而静默失效)。
88
+ 其余异常原样上抛(不吞错)。
89
+ """
57
90
  try:
58
91
  return fn()
59
92
  except Exception as e:
60
93
  msg = str(e)
61
94
  if msg.startswith(_PERM_PREFIX):
62
95
  raise PermissionError(msg[len(_PERM_PREFIX):]) from e
96
+ if msg.startswith(_PROFILE_PREFIX):
97
+ detail = msg[len(_PROFILE_PREFIX):]
98
+ m = _FEATURE_RE.search(detail)
99
+ _emit_feedback({
100
+ 'type': 'profile_blocked',
101
+ 'code': 'profileBlocked',
102
+ 'layer': 'core',
103
+ 'profile': 'text2query',
104
+ 'feature': m.group(1) if m else None,
105
+ 'message': detail,
106
+ 'hint': _PROFILE_HINT,
107
+ })
108
+ raise ProfileViolation(detail) from e
63
109
  raise
64
110
 
65
111
 
@@ -6,8 +6,39 @@ import inspect
6
6
 
7
7
  from ..feedback import emit as _emit_feedback
8
8
  from ..schema import core as _core
9
- from ..schema import get_async_fn
10
- from .exec import _call, _ctx, _exec, _exec_on, resolve_placeholders
9
+ from ..schema import get_async_fn, get_profile
10
+ from .exec import (
11
+ _PROFILE_HINT,
12
+ ProfileViolation,
13
+ _call,
14
+ _ctx,
15
+ _exec,
16
+ _exec_on,
17
+ resolve_placeholders,
18
+ )
19
+
20
+
21
+ def _guard_route_override(route_override):
22
+ """Host 兜底:text2query 档禁用 ``route_override``(受信参数,禁 AI 侧指定)
23
+
24
+ 判决唯一在 core(执行文档 §4.2 ⑤:各 ``plan_query*`` 入口已判并 Err);此为
25
+ 第二层防护——即使 core 判决被绕过,Host 也不放行受信参数(CWE-639)。命中即
26
+ emit ``profile_blocked``,``layer='host'`` 本身即反馈:core 层未拦住,须回溯加固
27
+ (自动反馈原则:允许拦截,禁止静默)。
28
+ """
29
+ if route_override is None or get_profile() != 'text2query':
30
+ return
31
+ detail = 'text2query 档禁用 route_override(受信参数,禁 AI 侧指定)'
32
+ _emit_feedback({
33
+ 'type': 'profile_blocked',
34
+ 'code': 'profileBlocked',
35
+ 'layer': 'host',
36
+ 'profile': 'text2query',
37
+ 'feature': 'route_override',
38
+ 'message': detail,
39
+ 'hint': _PROFILE_HINT,
40
+ })
41
+ raise ProfileViolation(detail)
11
42
 
12
43
 
13
44
  async def _run_query_plan(plan):
@@ -50,6 +81,7 @@ async def query(gql, params=None, route_override=None):
50
81
  权限/计算列仍按结构 schema 判定(见 multi-datasource-routing-plan.md §6)。
51
82
  注意:``route_override`` 为**受信服务端参数**,禁止透传用户输入(否则可被用于跨源路由,CWE-639)。
52
83
  """
84
+ _guard_route_override(route_override)
53
85
  plan = _call(lambda: _core.plan_query(
54
86
  gql, params if params is not None else {}, _ctx(), route_override))
55
87
  return await _finalize(plan, await _run_query_plan(plan))
@@ -61,6 +93,7 @@ async def query_one(gql, params=None, route_override=None):
61
93
  走 core ``plan_query_one``:用户未显式 ``$limit`` 时强制下推 ``$limit(1)``,
62
94
  大集合不再全量取回后取首条(对齐 PyMongo ``find_one`` 的 limit-1 语义)。
63
95
  """
96
+ _guard_route_override(route_override)
64
97
  plan = _call(lambda: _core.plan_query_one(
65
98
  gql, params if params is not None else {}, _ctx(), route_override))
66
99
  items = await _finalize(plan, await _run_query_plan(plan))
@@ -118,6 +151,7 @@ async def query_with_count(gql, params=None, route_override=None):
118
151
  2. 传统 $skip/$limit — 从 GQL 参数推导 page/pageSize
119
152
  pageSize 上限 5000,防止拖库。
120
153
  """
154
+ _guard_route_override(route_override)
121
155
  plan = _call(lambda: _core.plan_query_with_count(
122
156
  gql, params if params is not None else {}, _ctx(), None, route_override))
123
157
  items = await _finalize(plan, await _run_query_plan(plan))
@@ -239,3 +239,30 @@ async def exec_sql(source, connection, cmd):
239
239
  f'SQL 数据源 {source}({_kind_of(connection)}) 的执行器未接入(见执行文档 Phase 4)')
240
240
  out = await exec_fn(plan)
241
241
  return executors.shape_result(cmd, out)
242
+
243
+
244
+ class RawSqlError(RuntimeError):
245
+ """原生 SQL 入口的显式错误(非 SQL 源 / 执行器未接入)"""
246
+
247
+
248
+ async def execute_raw(source, sql, params=None, is_write=False):
249
+ """
250
+ 在指定 SQL 源上执行原生 SQL(Host 层逃生口,绕开 core 的 dialect_translate)
251
+
252
+ - 事务作用域内经 ``connection_for`` 落到事务专用连接 → 支持 SELECT ... FOR UPDATE;
253
+ - 占位符沿用各后端原生风格(mysql/sqlite 用 ``?``,postgres 用 ``$1..$n``);
254
+ - 仅支持 SQL 源;Mongo 源显式报错(绝不静默);
255
+ - ``is_write=False`` 视为读(取行);``True`` 视为写(取影响行数);
256
+ - 返回 ``{'rows': list|None, 'affectedRows': int}``。
257
+ """
258
+ conn = connection_for(source)
259
+ if not is_sql(conn):
260
+ raise RawSqlError(
261
+ f'数据源 {source} 不是 SQL 源(原生 SQL 入口仅支持 mysql/postgres/sqlite)')
262
+ exec_fn = _exec_of(conn)
263
+ if not callable(exec_fn):
264
+ raise RawSqlError(
265
+ f'SQL 数据源 {source}({_kind_of(conn)}) 的执行器未接入')
266
+ stmt = {'text': sql, 'params': list(params or []), 'isWrite': bool(is_write)}
267
+ out = await exec_fn({'stmts': [stmt]})
268
+ return {'rows': out.get('rows'), 'affectedRows': int(out.get('affectedRows') or 0)}