asterstore 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (139) hide show
  1. asterstore-0.1.0/CHANGELOG.md +16 -0
  2. asterstore-0.1.0/CONTRIBUTING.md +52 -0
  3. asterstore-0.1.0/LICENSE +202 -0
  4. asterstore-0.1.0/PKG-INFO +144 -0
  5. asterstore-0.1.0/README.md +128 -0
  6. asterstore-0.1.0/SECURITY.md +15 -0
  7. asterstore-0.1.0/benchmarks/README.md +15 -0
  8. asterstore-0.1.0/benchmarks/governance_scale.py +348 -0
  9. asterstore-0.1.0/benchmarks/managed_membership.py +88 -0
  10. asterstore-0.1.0/benchmarks/parquet_read.py +171 -0
  11. asterstore-0.1.0/docs/api.md +51 -0
  12. asterstore-0.1.0/docs/architecture.md +32 -0
  13. asterstore-0.1.0/docs/aster-integration.md +17 -0
  14. asterstore-0.1.0/docs/audit-fixes.md +21 -0
  15. asterstore-0.1.0/docs/automation.md +25 -0
  16. asterstore-0.1.0/docs/cleanup.md +63 -0
  17. asterstore-0.1.0/docs/compatibility.md +9 -0
  18. asterstore-0.1.0/docs/declarations.md +90 -0
  19. asterstore-0.1.0/docs/design.md +25 -0
  20. asterstore-0.1.0/docs/development.md +22 -0
  21. asterstore-0.1.0/docs/governance.md +115 -0
  22. asterstore-0.1.0/docs/integrations.md +17 -0
  23. asterstore-0.1.0/docs/protocol.md +114 -0
  24. asterstore-0.1.0/docs/publishing.md +75 -0
  25. asterstore-0.1.0/docs/release-scope.md +13 -0
  26. asterstore-0.1.0/docs/releasing.md +29 -0
  27. asterstore-0.1.0/docs/research.md +140 -0
  28. asterstore-0.1.0/docs/roadmap.md +11 -0
  29. asterstore-0.1.0/docs/spec/README.md +43 -0
  30. asterstore-0.1.0/docs/spec/aster-adapter.md +55 -0
  31. asterstore-0.1.0/docs/spec/conformance.md +88 -0
  32. asterstore-0.1.0/docs/spec/core.md +204 -0
  33. asterstore-0.1.0/docs/spec/metadata.md +148 -0
  34. asterstore-0.1.0/docs/validation/aster-integration.json +31 -0
  35. asterstore-0.1.0/examples/cleanup.py +48 -0
  36. asterstore-0.1.0/examples/declarations.py +57 -0
  37. asterstore-0.1.0/examples/governance.py +54 -0
  38. asterstore-0.1.0/examples/managed_declarations.py +46 -0
  39. asterstore-0.1.0/examples/parquet.py +48 -0
  40. asterstore-0.1.0/examples/registration.py +48 -0
  41. asterstore-0.1.0/pyproject.toml +79 -0
  42. asterstore-0.1.0/pyproject.toml.orig +48 -0
  43. asterstore-0.1.0/src/asterstore/__init__.py +84 -0
  44. asterstore-0.1.0/src/asterstore/errors.py +75 -0
  45. asterstore-0.1.0/src/asterstore/governance/__init__.py +13 -0
  46. asterstore-0.1.0/src/asterstore/governance/_abandonment.py +60 -0
  47. asterstore-0.1.0/src/asterstore/governance/_collection.py +178 -0
  48. asterstore-0.1.0/src/asterstore/governance/_inventory.py +389 -0
  49. asterstore-0.1.0/src/asterstore/governance/_references.py +163 -0
  50. asterstore-0.1.0/src/asterstore/governance/_service.py +109 -0
  51. asterstore-0.1.0/src/asterstore/governance/cleanup/__init__.py +5 -0
  52. asterstore-0.1.0/src/asterstore/governance/cleanup/_service.py +224 -0
  53. asterstore-0.1.0/src/asterstore/integrations/__init__.py +3 -0
  54. asterstore-0.1.0/src/asterstore/integrations/polars/__init__.py +5 -0
  55. asterstore-0.1.0/src/asterstore/integrations/polars/_parquet.py +34 -0
  56. asterstore-0.1.0/src/asterstore/metadata/__init__.py +18 -0
  57. asterstore-0.1.0/src/asterstore/metadata/capabilities/__init__.py +5 -0
  58. asterstore-0.1.0/src/asterstore/metadata/capabilities/_models.py +88 -0
  59. asterstore-0.1.0/src/asterstore/metadata/declarations/__init__.py +5 -0
  60. asterstore-0.1.0/src/asterstore/metadata/declarations/_models.py +33 -0
  61. asterstore-0.1.0/src/asterstore/metadata/identity/__init__.py +5 -0
  62. asterstore-0.1.0/src/asterstore/metadata/identity/_validation.py +43 -0
  63. asterstore-0.1.0/src/asterstore/metadata/membership/__init__.py +5 -0
  64. asterstore-0.1.0/src/asterstore/metadata/membership/_models.py +90 -0
  65. asterstore-0.1.0/src/asterstore/metadata/protocol/__init__.py +85 -0
  66. asterstore-0.1.0/src/asterstore/metadata/protocol/_cleanup.py +138 -0
  67. asterstore-0.1.0/src/asterstore/metadata/protocol/_codec.py +219 -0
  68. asterstore-0.1.0/src/asterstore/metadata/protocol/_governance.py +286 -0
  69. asterstore-0.1.0/src/asterstore/metadata/protocol/_json.py +66 -0
  70. asterstore-0.1.0/src/asterstore/metadata/protocol/_models.py +154 -0
  71. asterstore-0.1.0/src/asterstore/metadata/schemas/store.json +786 -0
  72. asterstore-0.1.0/src/asterstore/publishing/__init__.py +6 -0
  73. asterstore-0.1.0/src/asterstore/publishing/_candidate.py +321 -0
  74. asterstore-0.1.0/src/asterstore/publishing/_membership.py +77 -0
  75. asterstore-0.1.0/src/asterstore/publishing/_status.py +91 -0
  76. asterstore-0.1.0/src/asterstore/publishing/transactions/__init__.py +5 -0
  77. asterstore-0.1.0/src/asterstore/publishing/transactions/_commit.py +219 -0
  78. asterstore-0.1.0/src/asterstore/py.typed +0 -0
  79. asterstore-0.1.0/src/asterstore/reading/__init__.py +7 -0
  80. asterstore-0.1.0/src/asterstore/reading/_binding.py +63 -0
  81. asterstore-0.1.0/src/asterstore/reading/_catalog.py +31 -0
  82. asterstore-0.1.0/src/asterstore/reading/_loading.py +72 -0
  83. asterstore-0.1.0/src/asterstore/reading/_selection.py +18 -0
  84. asterstore-0.1.0/src/asterstore/reading/resources/__init__.py +5 -0
  85. asterstore-0.1.0/src/asterstore/reading/resources/_mapping.py +36 -0
  86. asterstore-0.1.0/src/asterstore/registration/__init__.py +7 -0
  87. asterstore-0.1.0/src/asterstore/registration/_operations.py +60 -0
  88. asterstore-0.1.0/src/asterstore/repository.py +129 -0
  89. asterstore-0.1.0/src/asterstore/storage/__init__.py +25 -0
  90. asterstore-0.1.0/src/asterstore/storage/_deletion.py +46 -0
  91. asterstore-0.1.0/src/asterstore/storage/_files.py +96 -0
  92. asterstore-0.1.0/src/asterstore/storage/_layout.py +8 -0
  93. asterstore-0.1.0/src/asterstore/storage/_locks.py +24 -0
  94. asterstore-0.1.0/src/asterstore/storage/registry/__init__.py +62 -0
  95. asterstore-0.1.0/src/asterstore/storage/registry/_cleanup.py +44 -0
  96. asterstore-0.1.0/src/asterstore/storage/registry/_governance.py +122 -0
  97. asterstore-0.1.0/src/asterstore/storage/registry/_initialize.py +48 -0
  98. asterstore-0.1.0/src/asterstore/storage/registry/_managed.py +37 -0
  99. asterstore-0.1.0/src/asterstore/storage/registry/_records.py +121 -0
  100. asterstore-0.1.0/tests/__init__.py +1 -0
  101. asterstore-0.1.0/tests/fixtures/protocol/cleanup/plan.json +1 -0
  102. asterstore-0.1.0/tests/fixtures/protocol/cleanup/progress.json +1 -0
  103. asterstore-0.1.0/tests/fixtures/protocol/governance/abandonment.json +1 -0
  104. asterstore-0.1.0/tests/fixtures/protocol/governance/plan.json +1 -0
  105. asterstore-0.1.0/tests/fixtures/protocol/governance/progress.json +1 -0
  106. asterstore-0.1.0/tests/fixtures/protocol/governance/repository.json +1 -0
  107. asterstore-0.1.0/tests/fixtures/protocol/governance/retention.json +1 -0
  108. asterstore-0.1.0/tests/fixtures/protocol/governance/retired.json +1 -0
  109. asterstore-0.1.0/tests/fixtures/protocol/invalid/boolean-generation.json +1 -0
  110. asterstore-0.1.0/tests/fixtures/protocol/invalid/duplicate-resource.json +1 -0
  111. asterstore-0.1.0/tests/fixtures/protocol/invalid/overflow-generation.json +1 -0
  112. asterstore-0.1.0/tests/fixtures/protocol/invalid/unclosed-membership.json +1 -0
  113. asterstore-0.1.0/tests/fixtures/protocol/invalid/unknown-feature.json +1 -0
  114. asterstore-0.1.0/tests/fixtures/protocol/managed/object.json +1 -0
  115. asterstore-0.1.0/tests/fixtures/protocol/managed/publication.json +1 -0
  116. asterstore-0.1.0/tests/fixtures/protocol/managed/repository.json +1 -0
  117. asterstore-0.1.0/tests/fixtures/protocol/managed/request.json +1 -0
  118. asterstore-0.1.0/tests/fixtures/protocol/object.json +1 -0
  119. asterstore-0.1.0/tests/fixtures/protocol/publication.json +1 -0
  120. asterstore-0.1.0/tests/fixtures/protocol/repository.json +1 -0
  121. asterstore-0.1.0/tests/test_audit_regressions.py +419 -0
  122. asterstore-0.1.0/tests/test_binding.py +152 -0
  123. asterstore-0.1.0/tests/test_cleanup.py +483 -0
  124. asterstore-0.1.0/tests/test_declarations.py +149 -0
  125. asterstore-0.1.0/tests/test_discovery.py +87 -0
  126. asterstore-0.1.0/tests/test_governance.py +523 -0
  127. asterstore-0.1.0/tests/test_governance_processes.py +189 -0
  128. asterstore-0.1.0/tests/test_io_contract.py +94 -0
  129. asterstore-0.1.0/tests/test_managed.py +450 -0
  130. asterstore-0.1.0/tests/test_managed_cleanup_processes.py +209 -0
  131. asterstore-0.1.0/tests/test_managed_processes.py +131 -0
  132. asterstore-0.1.0/tests/test_polars.py +179 -0
  133. asterstore-0.1.0/tests/test_registration.py +375 -0
  134. asterstore-0.1.0/tests/test_registration_processes.py +142 -0
  135. asterstore-0.1.0/tests/test_release_artifacts.py +76 -0
  136. asterstore-0.1.0/tests/test_supported_format.py +57 -0
  137. asterstore-0.1.0/tools/check_distribution.py +317 -0
  138. asterstore-0.1.0/tools/check_protocol.py +66 -0
  139. asterstore-0.1.0/tools/select_release_artifacts.py +53 -0
@@ -0,0 +1,16 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 — 2026-09-28
4
+
5
+ 首个公开版本,使用唯一的 `store` 格式(format_version=1)。
6
+
7
+ - Declaration / FileSet 描述精确成员;Binding 在内存中选取路径,无逐读扫描、stat 或 hash。
8
+ - 区分 registered 外部登记与 managed 受管写入,外部数据不获得隐式删除权。
9
+ - Candidate 提供原生写入路径、增量复用、原子提交、代际冲突和固定请求恢复。
10
+ - Governance 提供 metadata/objects 具名保留、条件释放、固定计划回收及失败候选清理。
11
+ - `Repository.list_datasets()`、`governance.list_retentions()` 和 CandidateStatus 归属字段提供显式控制记录查询,不增加普通读取成本。
12
+ - 可选 Polars 接口;核心无第三方运行依赖。uv 构建,Apache-2.0,Python 3.11/3.12。
13
+ - Aster 真实分钟行情接入已通过:2,505,120 行数据、10,414 条 VWAP 结果一致,验证完整本地生命周期。
14
+ - 手动发布流程复用完整 CI,只上传已验证的同一批 wheel/sdist,使用 PyPI Trusted Publishing。
15
+
16
+ 支持本地 Linux/POSIX。不承诺 NFS 写入、真实断电资格、全历史规模、发布依赖图、元数据压缩和完整审计。
@@ -0,0 +1,52 @@
1
+ # Contributing
2
+
3
+ 欢迎通过 [Issues](https://github.com/Aequiludium/asterstore/issues) 报告问题、讨论设计,通过 Pull Request 提交修改。漏洞请使用 [安全报告流程](SECURITY.md)。
4
+
5
+ asterstore 当前处于 `0.1.0.dev1` 开发阶段,只有一套实现,尚未冻结公开 API。变更须说明行为、权属与恢复影响,并更新当前协议与测试;不维护历史开发草稿的兼容层。范围见[首版验收](docs/release-scope.md)。
6
+
7
+ ## 开发环境
8
+
9
+ Python 3.11 或 3.12,本地 Linux/POSIX 环境,uv 0.10.7 或更高版本:
10
+
11
+ ```bash
12
+ git clone https://github.com/Aequiludium/asterstore.git
13
+ cd asterstore
14
+ uv sync --locked --extra polars
15
+ uv run --locked --extra polars ruff check .
16
+ uv run --locked --extra polars ruff format --check .
17
+ uv run --locked --extra polars mypy
18
+ uv run --locked --extra polars python tools/check_protocol.py
19
+ uv run --locked --extra polars pytest
20
+ ```
21
+
22
+ 构建与源码外隔离安装检查:
23
+
24
+ ```bash
25
+ uv run --locked --extra polars python tools/check_distribution.py --polars
26
+ ```
27
+
28
+ 依赖变化通过 `uv lock` 生成锁文件。普通代码修改无需更新锁文件。完整示例和规模 smoke 命令见 [开发指南](docs/development.md) 及 [CI](.github/workflows/ci.yml)。
29
+
30
+ ## 修改原则
31
+
32
+ - 核心保持业务无关和零第三方运行依赖;可选引擎放在 integrations。
33
+ - 普通 Binding 选择保持无治理 I/O;不要为每次读取增加扫描、保留写入、存在性或内容检查。
34
+ - 子包通过明确接口协作;私有实现不是公共兼容承诺。
35
+ - 发布、引用、删除的行为变化要说明生效点、锁顺序、异常后可见状态、原操作重试与恢复范围。
36
+ - 修改持久格式时同步检查 Schema、codec、golden fixtures、必要功能门槛和兼容文档;不能只改格式标记绕过版本限制。
37
+ - 回归测试验证可观察的语义与真实失败边界。性能结论提供规模、环境和成本分解,不依赖单次墙钟阈值。
38
+ - 试验使用临时或专用隔离目录。实际数据迁移和生产治理不是测试步骤。
39
+
40
+ PR 描述包括具体问题、变化后的行为、相关验证及仍存在的限制。请区分本地进程故障证据与真实断电/共享挂载资格,也区分开发版本通过验证与正式发行。
41
+
42
+ ## 问题报告
43
+
44
+ 普通问题请提供 Python/asterstore 版本、操作系统与文件系统、协议版本及必要功能、最小复现、预期和实际结果。涉及中断操作时附上脱敏后的操作阶段与异常,避免上传真实数据、访问凭据或未脱敏业务元数据。
45
+
46
+ ## 许可证
47
+
48
+ 本项目采用 [Apache-2.0](LICENSE)。提交贡献前请确认你有权以该许可证贡献相应代码和材料;第三方材料保留其原有归属与许可说明。
49
+
50
+ ## 分支与 CI
51
+
52
+ 通过 PR 合并到 main;提交前同步 main,并通过全部 CI。仓库采用 squash 合并。依赖 Bot 遵守同样的检查,不自动合并;配置和报告保留周期见[自动化说明](docs/automation.md)。
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
@@ -0,0 +1,144 @@
1
+ Metadata-Version: 2.4
2
+ Name: asterstore
3
+ Version: 0.1.0
4
+ Summary: Local file-based data lifecycle management
5
+ License-Expression: Apache-2.0
6
+ License-File: LICENSE
7
+ Requires-Dist: polars>=1.32,<2 ; extra == 'polars'
8
+ Maintainer: Aequiludium
9
+ Requires-Python: >=3.11
10
+ Project-URL: Repository, https://github.com/Aequiludium/asterstore
11
+ Project-URL: Documentation, https://github.com/Aequiludium/asterstore/blob/main/README.md
12
+ Project-URL: Issues, https://github.com/Aequiludium/asterstore/issues
13
+ Project-URL: Changelog, https://github.com/Aequiludium/asterstore/blob/main/CHANGELOG.md
14
+ Provides-Extra: polars
15
+ Description-Content-Type: text/markdown
16
+
17
+ <h1 align="center">asterstore</h1>
18
+
19
+ <p align="center">
20
+ 本地文件数据管理库<br>
21
+ <strong>轻量读取 · 显式发布 · 可恢复治理</strong>
22
+ </p>
23
+
24
+ <p align="center">
25
+ <a href="https://github.com/Aequiludium/asterstore/actions/workflows/ci.yml"><img src="https://github.com/Aequiludium/asterstore/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
26
+ <a href="pyproject.toml"><img src="https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square" alt="Python 3.11+"></a>
27
+ <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-52796F?style=flat-square" alt="Apache-2.0"></a>
28
+ </p>
29
+
30
+ <p align="center">
31
+ <a href="#快速开始">快速开始</a> ·
32
+ <a href="docs/api.md">API</a> ·
33
+ <a href="examples">示例</a> ·
34
+ <a href="docs/design.md">设计</a> ·
35
+ <a href="CONTRIBUTING.md">参与贡献</a>
36
+ </p>
37
+
38
+ ---
39
+
40
+ 数据仍然是普通文件。**asterstore 为它们管理发布、保留与回收**:哪些文件组成一次发布,哪些对象仍被任务使用,哪些残留可以清理。
41
+
42
+ 框架负责生产数据,Polars、Arrow 等引擎负责读取和计算。asterstore 连接这些环节,核心无第三方运行依赖,也不依赖 Aster 或任何业务框架。
43
+
44
+ ## 治理明确,读取轻量
45
+
46
+ - **发布有边界。** 写入候选后显式提交,以原子切换使新发布可见;未变化的受管对象可以直接复用。
47
+ - **读取走短路径。** 打开当前发布只读两份控制记录;绑定后反复选择文件路径,不扫描目录、不逐文件预检、不计算内容哈希。
48
+ - **保留由任务决定。** 需要的发布显式保留,使用完成后按引用版本释放;绑定或读取本身不产生保留记录。
49
+ - **回收可以恢复。** 先预览,再按固定计划退役与删除;中断后沿原计划恢复,清理范围不扩大。
50
+
51
+ ### 两种管理方式,同一套模型
52
+
53
+ | | 外部登记 · `registered` | 受管发布 · `managed` |
54
+ | --- | --- | --- |
55
+ | 数据由谁写入 | 现有程序或其他框架 | 通过候选写入与提交 |
56
+ | 如何接入 | 声明已有文件及资源位置 | 创建对象,或复用已提交对象 |
57
+ | 可以保留什么 | 发布元数据 | 发布元数据与受管文件 |
58
+ | 文件由谁回收 | 外部生产者 | asterstore 按权属和保留关系回收 |
59
+
60
+ 两者共用 `Declaration`、`Binding` 和治理接口。登记外部文件不会改变其生产流程,也不会赋予库删除权。详见[声明与资源绑定](docs/declarations.md)。
61
+
62
+ ## 快速开始
63
+
64
+ 从源码体验,需要 Python 3.11+ 和 uv:
65
+
66
+ ```bash
67
+ git clone https://github.com/Aequiludium/asterstore.git
68
+ cd asterstore
69
+ uv sync --locked
70
+ ```
71
+
72
+ **写入一次,显式发布。** 将下面代码保存为 `demo.py`,运行 `uv run python demo.py`:
73
+
74
+ ```python
75
+ from asterstore import Repository
76
+
77
+ store = Repository("./demo-store")
78
+ store.initialize(
79
+ resource_ids=["data"],
80
+ managed_resource_id="data",
81
+ lifecycle=True,
82
+ )
83
+
84
+ with store.prepare(
85
+ "measurements",
86
+ publication_id="run:1",
87
+ operation_id="write:1",
88
+ expected_generation=0,
89
+ ) as writer:
90
+ writer.write_bytes("readings", b"sensor,value\nA,21.5\n", relative_path="part.csv")
91
+ writer.commit()
92
+
93
+ # 捕获这次发布的成员与路径,交给文件读取或数据引擎。
94
+ binding = store.open("measurements")
95
+ print(binding.files(keys=["readings"])[0].read_text())
96
+ ```
97
+
98
+ `operation_id` 标识一次固定写入请求,`expected_generation` 用于检测并发更新。退出候选上下文不会自动提交;完整的复用与恢复流程见[发布示例](examples/managed_declarations.py)。
99
+
100
+ <details>
101
+ <summary><strong>任务需要保留数据时</strong></summary>
102
+
103
+ 在可能并发回收的场景中,先建立对象保留,再通过该引用读取。以下代码接续上面的示例:
104
+
105
+ ```python
106
+ from asterstore import RetentionScope
107
+
108
+ held = store.governance.retain("analysis", "measurements", "run:1", scope=RetentionScope.OBJECTS)
109
+ try:
110
+ inputs = store.governance.open("analysis", expected_revision=held.revision)
111
+ print(inputs.files()[0].read_text())
112
+ finally:
113
+ store.governance.release("analysis", expected_revision=held.revision)
114
+ ```
115
+
116
+ 释放引用不会立即删除文件。回收需显式调用,且仍保护当前发布及其他任务保留的对象。参见[治理示例](examples/governance.py)与[候选清理](examples/cleanup.py)。
117
+
118
+ </details>
119
+
120
+ **使用 Polars**,可直接运行增量发布、延迟查询与回收的完整示例:
121
+
122
+ ```bash
123
+ uv sync --locked --extra polars
124
+ uv run --locked --extra polars python examples/parquet.py
125
+ ```
126
+
127
+ ## 文档导航
128
+
129
+ | 想了解什么 | 从这里开始 |
130
+ | --- | --- |
131
+ | 使用接口与读取引擎 | [API](docs/api.md) · [引擎接入](docs/integrations.md) · [示例](examples) |
132
+ | 发布、保留与回收如何工作 | [发布](docs/publishing.md) · [治理](docs/governance.md) · [候选清理](docs/cleanup.md) |
133
+ | 模型与磁盘格式 | [设计原则](docs/design.md) · [项目结构](docs/architecture.md) · [协议](docs/protocol.md) |
134
+ | 开发、验证与贡献 | [开发指南](docs/development.md) · [性能验证](benchmarks/README.md) · [贡献指南](CONTRIBUTING.md) |
135
+
136
+ ## 项目状态
137
+
138
+ **`0.1.0` 首版发布准备中**。已完成 Aster 真实分钟行情的本地受管接入,验证发布、增量复用、读取、保留与回收;详见[接入验收](docs/aster-integration.md)。PyPI 上传完成前,请继续使用源码。
139
+
140
+ 当前验证范围为本地 Linux/POSIX;NFS 等共享挂载需要单独验证。核心以单个数据集发布为提交单位,控制历史尚未压缩。支持范围与后续工作见[首版范围](docs/release-scope.md)、[路线图](docs/roadmap.md)及[开发格式边界](docs/compatibility.md)。
141
+
142
+ ---
143
+
144
+ [Apache-2.0](LICENSE) · [变更记录](CHANGELOG.md) · [安全报告](SECURITY.md) · [问题反馈](https://github.com/Aequiludium/asterstore/issues)
@@ -0,0 +1,128 @@
1
+ <h1 align="center">asterstore</h1>
2
+
3
+ <p align="center">
4
+ 本地文件数据管理库<br>
5
+ <strong>轻量读取 · 显式发布 · 可恢复治理</strong>
6
+ </p>
7
+
8
+ <p align="center">
9
+ <a href="https://github.com/Aequiludium/asterstore/actions/workflows/ci.yml"><img src="https://github.com/Aequiludium/asterstore/actions/workflows/ci.yml/badge.svg?branch=main" alt="CI"></a>
10
+ <a href="pyproject.toml"><img src="https://img.shields.io/badge/Python-3.11%2B-3776AB?style=flat-square" alt="Python 3.11+"></a>
11
+ <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-52796F?style=flat-square" alt="Apache-2.0"></a>
12
+ </p>
13
+
14
+ <p align="center">
15
+ <a href="#快速开始">快速开始</a> ·
16
+ <a href="docs/api.md">API</a> ·
17
+ <a href="examples">示例</a> ·
18
+ <a href="docs/design.md">设计</a> ·
19
+ <a href="CONTRIBUTING.md">参与贡献</a>
20
+ </p>
21
+
22
+ ---
23
+
24
+ 数据仍然是普通文件。**asterstore 为它们管理发布、保留与回收**:哪些文件组成一次发布,哪些对象仍被任务使用,哪些残留可以清理。
25
+
26
+ 框架负责生产数据,Polars、Arrow 等引擎负责读取和计算。asterstore 连接这些环节,核心无第三方运行依赖,也不依赖 Aster 或任何业务框架。
27
+
28
+ ## 治理明确,读取轻量
29
+
30
+ - **发布有边界。** 写入候选后显式提交,以原子切换使新发布可见;未变化的受管对象可以直接复用。
31
+ - **读取走短路径。** 打开当前发布只读两份控制记录;绑定后反复选择文件路径,不扫描目录、不逐文件预检、不计算内容哈希。
32
+ - **保留由任务决定。** 需要的发布显式保留,使用完成后按引用版本释放;绑定或读取本身不产生保留记录。
33
+ - **回收可以恢复。** 先预览,再按固定计划退役与删除;中断后沿原计划恢复,清理范围不扩大。
34
+
35
+ ### 两种管理方式,同一套模型
36
+
37
+ | | 外部登记 · `registered` | 受管发布 · `managed` |
38
+ | --- | --- | --- |
39
+ | 数据由谁写入 | 现有程序或其他框架 | 通过候选写入与提交 |
40
+ | 如何接入 | 声明已有文件及资源位置 | 创建对象,或复用已提交对象 |
41
+ | 可以保留什么 | 发布元数据 | 发布元数据与受管文件 |
42
+ | 文件由谁回收 | 外部生产者 | asterstore 按权属和保留关系回收 |
43
+
44
+ 两者共用 `Declaration`、`Binding` 和治理接口。登记外部文件不会改变其生产流程,也不会赋予库删除权。详见[声明与资源绑定](docs/declarations.md)。
45
+
46
+ ## 快速开始
47
+
48
+ 从源码体验,需要 Python 3.11+ 和 uv:
49
+
50
+ ```bash
51
+ git clone https://github.com/Aequiludium/asterstore.git
52
+ cd asterstore
53
+ uv sync --locked
54
+ ```
55
+
56
+ **写入一次,显式发布。** 将下面代码保存为 `demo.py`,运行 `uv run python demo.py`:
57
+
58
+ ```python
59
+ from asterstore import Repository
60
+
61
+ store = Repository("./demo-store")
62
+ store.initialize(
63
+ resource_ids=["data"],
64
+ managed_resource_id="data",
65
+ lifecycle=True,
66
+ )
67
+
68
+ with store.prepare(
69
+ "measurements",
70
+ publication_id="run:1",
71
+ operation_id="write:1",
72
+ expected_generation=0,
73
+ ) as writer:
74
+ writer.write_bytes("readings", b"sensor,value\nA,21.5\n", relative_path="part.csv")
75
+ writer.commit()
76
+
77
+ # 捕获这次发布的成员与路径,交给文件读取或数据引擎。
78
+ binding = store.open("measurements")
79
+ print(binding.files(keys=["readings"])[0].read_text())
80
+ ```
81
+
82
+ `operation_id` 标识一次固定写入请求,`expected_generation` 用于检测并发更新。退出候选上下文不会自动提交;完整的复用与恢复流程见[发布示例](examples/managed_declarations.py)。
83
+
84
+ <details>
85
+ <summary><strong>任务需要保留数据时</strong></summary>
86
+
87
+ 在可能并发回收的场景中,先建立对象保留,再通过该引用读取。以下代码接续上面的示例:
88
+
89
+ ```python
90
+ from asterstore import RetentionScope
91
+
92
+ held = store.governance.retain("analysis", "measurements", "run:1", scope=RetentionScope.OBJECTS)
93
+ try:
94
+ inputs = store.governance.open("analysis", expected_revision=held.revision)
95
+ print(inputs.files()[0].read_text())
96
+ finally:
97
+ store.governance.release("analysis", expected_revision=held.revision)
98
+ ```
99
+
100
+ 释放引用不会立即删除文件。回收需显式调用,且仍保护当前发布及其他任务保留的对象。参见[治理示例](examples/governance.py)与[候选清理](examples/cleanup.py)。
101
+
102
+ </details>
103
+
104
+ **使用 Polars**,可直接运行增量发布、延迟查询与回收的完整示例:
105
+
106
+ ```bash
107
+ uv sync --locked --extra polars
108
+ uv run --locked --extra polars python examples/parquet.py
109
+ ```
110
+
111
+ ## 文档导航
112
+
113
+ | 想了解什么 | 从这里开始 |
114
+ | --- | --- |
115
+ | 使用接口与读取引擎 | [API](docs/api.md) · [引擎接入](docs/integrations.md) · [示例](examples) |
116
+ | 发布、保留与回收如何工作 | [发布](docs/publishing.md) · [治理](docs/governance.md) · [候选清理](docs/cleanup.md) |
117
+ | 模型与磁盘格式 | [设计原则](docs/design.md) · [项目结构](docs/architecture.md) · [协议](docs/protocol.md) |
118
+ | 开发、验证与贡献 | [开发指南](docs/development.md) · [性能验证](benchmarks/README.md) · [贡献指南](CONTRIBUTING.md) |
119
+
120
+ ## 项目状态
121
+
122
+ **`0.1.0` 首版发布准备中**。已完成 Aster 真实分钟行情的本地受管接入,验证发布、增量复用、读取、保留与回收;详见[接入验收](docs/aster-integration.md)。PyPI 上传完成前,请继续使用源码。
123
+
124
+ 当前验证范围为本地 Linux/POSIX;NFS 等共享挂载需要单独验证。核心以单个数据集发布为提交单位,控制历史尚未压缩。支持范围与后续工作见[首版范围](docs/release-scope.md)、[路线图](docs/roadmap.md)及[开发格式边界](docs/compatibility.md)。
125
+
126
+ ---
127
+
128
+ [Apache-2.0](LICENSE) · [变更记录](CHANGELOG.md) · [安全报告](SECURITY.md) · [问题反馈](https://github.com/Aequiludium/asterstore/issues)
@@ -0,0 +1,15 @@
1
+ # Security
2
+
3
+ asterstore 目前为开发版本 `0.1.0.dev1`,尚无稳定版本支持周期。安全修复优先进入当前开发分支;兼容范围见 [开发格式边界](docs/compatibility.md),不承诺独立的旧开发快照维护。
4
+
5
+ ## 报告问题
6
+
7
+ 疑似漏洞请通过 GitHub 的 [Report a vulnerability](https://github.com/Aequiludium/asterstore/security/advisories/new) 私下提交给维护者。普通 bug 使用 [Issues](https://github.com/Aequiludium/asterstore/issues)。如果私密报告入口不可用,请在公开 issue 中仅请求私密联系渠道,不公开漏洞细节或利用代码。
8
+
9
+ 报告建议包括受影响版本/提交、平台与挂载方式、最小复现、影响范围、必要前提,以及是否可能误删数据、越过受管路径或破坏保留关系。只提供人工构造的样本,不上传凭据或真实业务数据。维护者会按实际能力处理,不承诺尚未建立的响应时限。
10
+
11
+ ## 安全与治理边界
12
+
13
+ 仓库假设文件系统拥有者及使用库的进程相互协作,POSIX 锁用于协调;它不是针对同权限恶意进程的访问控制或沙箱。普通读取不逐文件验真,registered 声明不能阻止外部生产者修改数据。
14
+
15
+ 上述边界不使误删或授权判断错误成为可接受行为。受管/外部数据隔离、固定删除范围、对象归属、保留和恢复状态机的缺陷仍应报告。核心控制记录为严格 JSON,不执行应用 pickle/插件反序列化器。完整保证见 [首版范围](docs/release-scope.md) 和 [治理协议](docs/governance.md)。
@@ -0,0 +1,15 @@
1
+ # 性能验证
2
+
3
+ 当前脚本只针对唯一实现,不沿用旧开发协议的结果或容量结论。
4
+
5
+ ```bash
6
+ uv run --locked --extra polars python benchmarks/parquet_read.py --files 1 4 --rows 16 --repeats 2 --selections 10
7
+ uv run --locked python benchmarks/managed_membership.py --members 128 1024
8
+ uv run --locked python benchmarks/governance_scale.py --history 1 3 --datasets 2 --references 0 2 --logs 0 2 --repeats 1
9
+ ```
10
+
11
+ parquet_read 对同一文件集合比较原生 Polars 和 Binding 适配读取,并测量 open、成员选择及 Python 分配。managed_membership 测量候选增量构建与封存。governance_scale 分别改变发布历史、数据集、引用与回收日志数量,报告控制读取、同步、删除、耗时和 Python 内存。
12
+
13
+ 这些命令用于 smoke 验证,不代表生产容量。更大规模使用脚本参数和 --output 保存新结果,并记录存储介质和缓存条件。控制历史会积累;治理预览、GC 和完成态重试可能扫描与同步全仓。普通 current open 的两份控制记录边界不等于治理也是常数成本。
14
+
15
+ 共享挂载、多客户端和真实断电需要独立资格验证。