celestialflow 3.1.0__tar.gz → 3.1.2__tar.gz

{celestialflow-3.1.0 → celestialflow-3.1.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: celestialflow
-Version: 3.1.0
+Version: 3.1.2
 Summary: A flexible GRAPH-based task orchestration framework.
 Author-email: Mr-xiaotian <mingxiaomingtian@gmail.com>
 License: MIT
@@ -12,7 +12,7 @@ Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Framework :: FastAPI
 Classifier: Topic :: Software Development :: Libraries
-Requires-Python: >=3.8
+Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: tqdm
 Requires-Dist: loguru
@@ -130,7 +130,11 @@ flowchart LR
 安装 CelestialFlow:
 
 ```bash
-pip install celestialflow
+# 推荐使用 `uv` 管理依赖与环境
+uv pip install celestialflow
+
+# 不过也可以直接使用 `pip`
+uv pip install celestialflow
 ```
 
 一个简单的可运行代码:
@@ -172,19 +176,31 @@ if __name__ == "__main__":
 
 若你想了解框架的整体结构与核心组件，下面的参考文档会对你有帮助：
 
-- [🔧TaskExecutor概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_executor.md)
-- [🔧TaskStage概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_stage.md)
-- [🌐TaskGraph概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_graph.md)
-- [📚Go Worker概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/go_worker.md)
+- [stage/executor.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/executor.md)
+- [stage/stage.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/stage.md)
+- [graph/graph.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/graph.md)
+- [runtime/progress.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/progress.md)
+- [runtime/metrics.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/metrics.md)
+- [runtime/queue.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/queue.md)
+- [stage/nodes.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/nodes.md)
+- [observability/report.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/observability/report.md)
+- [graph/structure.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/structure.md)
+- [web/server.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/web/server.md)
+- [other/go_worker.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/go_worker.md)
 
 推荐阅读顺序:
 
 ```mermaid
 flowchart TD
-    classDef whiteNode fill:#ffffff,stroke:#000000,color:#000000;
+    classDef core fill:#e6efff,stroke:#3b82f6,color:#1e3a8a;
+    classDef runtime fill:#e9f8ef,stroke:#22c55e,color:#14532d;
+    classDef structure fill:#fff6e6,stroke:#f59e0b,color:#78350f;
+    classDef execution fill:#f3e8ff,stroke:#a855f7,color:#581c87;
+    classDef web fill:#ffeaea,stroke:#ef4444,color:#7f1d1d;
 
     TM[TaskExecutor.md] --> TS[TaskStage.md] --> TG[TaskGraph.md]
     TM --> TP[TaskProgress.md]
+    TM --> TME[TaskMetrics.md]
 
     TG --> TQ[TaskQueue.md]
     TG --> TN[TaskNodes.md]
@@ -194,14 +210,20 @@ flowchart TD
     TR --> TW[TaskWeb.md]
     TN --> GW[Go Worker.md]
 
-    class TM,TG,TP,TQ,TN,TR,TS,TW,GW,TSR whiteNode;
+    class TM,TS,TG core;
+    class TP,TME runtime;
+    class TSR structure;
+    class TQ,TN,GW execution;
+    class TR,TW web;
 ```
 
 以下三篇可以作为补充阅读:
 
-- [TaskTools](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_tools.md)
-- [TaskTypes](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_types.md)
-- [TaskLogging](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_logging.md)
+- [runtime/tools.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/tools.md)
+- [runtime/types.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/types.md)
+- [runtime/errors.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/errors.md)
+- [persistence/fail.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/fail.md)
+- [persistence/log.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/log.md)
 
 如果你更喜欢通过完整案例理解框架的运行方式，可以参考这篇从零开始构建 TaskGraph 的教程：
 
@@ -209,7 +231,7 @@ flowchart TD
 
 如果你对3.0.7版本加入的ctree_client与其功能感兴趣, 可以看看这一篇:
 
-[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/ctree_client.md)
+[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/ctree_client.md)
 
 ## 环境要求（Requirements）
 
@@ -231,100 +253,13 @@ flowchart TD
 
 ## 文件结构（File Structure）
 
-```
-📁 CelestialFlow	(224MB 407KB 500B)
-    📁 docs         	(34KB 944B)
-        📁 reference	(29KB 995B)
-            📝 ctree_client.md  	(6KB 997B)
-            📝 go_worker.md     	(6KB 378B)
-            📝 task_graph.md    	(4KB 989B)
-            📝 task_logging.md  	(0B)
-            📝 task_manage.md   	(7KB 499B)
-            📝 task_nodes.md    	(4KB 68B)
-            📝 task_progress.md 	(0B)
-            📝 task_queue.md    	(0B)
-            📝 task_report.md   	(0B)
-            📝 task_stage.md    	(0B)
-            📝 task_structure.md	(112B)
-            📝 task_tools.md    	(0B)
-            📝 task_types.md    	(0B)
-            📝 task_web.md      	(0B)
-        📝 quick_start.md	(4KB 973B)
-        📝 tutorial.md   	(0B)
-    📁 experiments  	(15KB 111B)
-        🐍 benchmark_datastructures.py	(6KB 52B)
-        🐍 benchmark_hash.py          	(1KB 284B)
-        🐍 benchmark_queue.py         	(5KB 504B)
-        🐍 benchmark_tqdm.py          	(1KB 160B)
-        🐍 experiment_tqdm.py         	(1KB 135B)
-    📁 go_worker    	(6MB 968KB 38B)
-        📁 worker	(5KB 746B)
-            🌀 parser.go   	(394B)
-            🌀 processor.go	(2KB 612B)
-            🌀 types.go    	(299B)
-            🌀 worker.go   	(2KB 465B)
-        ❓ go.mod       	(292B)
-        ❓ go.sum       	(1KB 68B)
-        ❓ go_worker.exe	(6MB 960KB)
-        🌀 main.go      	(980B)
-    📁 img          	(4MB 700KB 695B)
-        📷 logo(old).png  	(836KB 542B)
-        📷 logo.png       	(122KB 747B)
-        📷 web_display.png	(129KB 545B)
-        🎞️ web_ui.gif     	(3MB 635KB 909B)
-    📁 src          	(1MB 929KB 111B)
-        📁 celestialflow         	(1MB 911KB 486B)
-            📁 static     	(1MB 417KB 367B)
-                📁 css	(31KB 621B)
-                    🎨 base.css     	(6KB 181B)
-                    🎨 dashboard.css	(7KB 414B)
-                    🎨 errors.css   	(5KB 216B)
-                    🎨 inject.css   	(12KB 834B)
-                📁 js 	(33KB 672B)
-                    📜 main.js          	(5KB 65B)
-                    📜 task_errors.js   	(4KB 595B)
-                    📜 task_injection.js	(8KB 484B)
-                    📜 task_statuses.js 	(8KB 51B)
-                    📜 task_structure.js	(4KB 709B)
-                    📜 task_topology.js 	(1KB 375B)
-                    📜 utils.js         	(1KB 441B)
-                ❓ favicon.ico	(1MB 352KB 98B)
-            📁 templates  	(13KB 174B)
-                🌐 index.html	(13KB 174B)
-            📁 [1项排除的目录]	(339KB 553B)
-            🐍 task_graph.py    	(26KB 700B)
-            🐍 task_logging.py  	(10KB 282B)
-            🐍 task_manage.py   	(36KB 110B)
-            🐍 task_nodes.py    	(10KB 814B)
-            🐍 task_progress.py 	(1KB 477B)
-            🐍 task_queue.py    	(10KB 948B)
-            🐍 task_report.py   	(4KB 828B)
-            🐍 task_stage.py    	(6KB 577B)
-            🐍 task_structure.py	(7KB 255B)
-            🐍 task_tools.py    	(17KB 19B)
-            🐍 task_types.py    	(2KB 269B)
-            🐍 task_web.py      	(6KB 235B)
-            🐍 __init__.py      	(1KB 22B)
-        📁 [1项排除的目录]	(17KB 649B)
-    📁 tests        	(126KB 724B)
-        📁 [1项排除的目录]	(90KB 464B)
-        🐍 conftest.py      	(49B)
-        📝 README.md        	(6KB 324B)
-        🐍 test_graph.py    	(5KB 547B)
-        🐍 test_manage.py   	(3KB 90B)
-        🐍 test_nodes.py    	(11KB 218B)
-        🐍 test_structure.py	(10KB 56B)
-    📁 [6项排除的目录]	(210MB 685KB 450B)
-    ❓ .env          	(88B)
-    ❓ .gitignore    	(272B)
-    ❓ LICENSE       	(1KB 65B)
-    ❓ Makefile      	(501B)
-    ❓ pyproject.toml	(1KB 594B)
-    📝 README.md     	(16KB 3B)
-(celestial-flow 3.0.9)
-```
+<p align="center">
+  <img src="https://raw.githubusercontent.com/Mr-xiaotian/CelestialFlow/main/img/file_structure.svg" alt="FileStructure" />
+  <br/>
+  <em>celestial-flow 3.1.1</em>
+</p>
 
-(该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。)
+(该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。转换为图片则借助[Carbon](https://carbon.now.sh)。)
 
 ## 更新日志（Change Log）
 
@@ -352,8 +287,38 @@ flowchart TD
   - 在前端中添加error_id的显示, 为之后显示provenance_tree做准备; 
   - 增加大量warning与error, 用于提醒不规范设置; 
   - 优化前后端中错误数据的传输方式, 在大量错误数据时减少内存消耗; 
-  - 优化TaskLogger中log队列的准入机制; 
-  - 修改部分Bug
+  - 优化LogSinker中log队列的准入机制; 
+  - 修改部分Bug;
+- 3.1.0:
+  - 新增:
+    - 优化web端仪表盘页面的"总体状态摘要"卡片, 新增"总重复任务"与"总剩余市场", 后者由节点间拓扑关系计算而成, 后续还需要优化;
+  - 修复:
+    - 3.0.9版本下当web端与celestialflow运行端不同时, error数据无法传递的问题;
+    - 修复NullTaskReporter使用问题;
+- 3.1.1:
+  - 新增:
+    - [Important] CelestialTree中引入grpc, 这大大减少了emit操作的耗时;
+    - [Important] 为TerminationSignal添加id, 并可以像task一样通过CelestialTree进行跟踪;
+    - 优化task_graph与task_executor的log分级, 现在默认为"SUCCESS"级;
+    - 将go_worker部分分离为单独project: [celestialflow-goworker]https://github.com/Mr-xiaotian/celestialflow-goworker
+    - 在readme中使用svg图片来展示文件夹结构;
+    - 优化全局剩余时间的计算;
+    - 优化部分代码结构;
+  - 修复:
+    - 修复节点剩余时间在小于1s时显示0的问题(这很影响判断);
+    - 在task_graph中使用"staged"模式时会报错的问题;
+- 3.1.2:
+  - feat:
+    - [Important] 模仿已有Logger逻辑, 建立新的组合结构FailListener和FailSinker, 将原有绑定于TaskReporter的fail持久化在线程中实现; 
+    - 借此让TaskExecutor也可以进行fail持久化, 文件为fallback/{date_str}/{executor_errors}({time_str}).jsonl";
+    - 重构整体项目结构, 现在更加清晰;
+    - 补全docs/reference中全部文档, 并令其符合项目重构后结构
+    - 引入uv进行环境管理;
+    - 将executor中技术逻辑分离为runtime/metrics.py;
+    - 整合TaskQueue中重复逻辑;
+  - fix
+    - 修复前端renderNodeList中参数设置错误;
+    - 修复其他微小bug;
 
 ## Star 历史趋势（Star History）
 

{celestialflow-3.1.0 → celestialflow-3.1.2}/README.md

@@ -104,7 +104,11 @@ flowchart LR
 安装 CelestialFlow:
 
 ```bash
-pip install celestialflow
+# 推荐使用 `uv` 管理依赖与环境
+uv pip install celestialflow
+
+# 不过也可以直接使用 `pip`
+uv pip install celestialflow
 ```
 
 一个简单的可运行代码:
@@ -146,19 +150,31 @@ if __name__ == "__main__":
 
 若你想了解框架的整体结构与核心组件，下面的参考文档会对你有帮助：
 
-- [🔧TaskExecutor概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_executor.md)
-- [🔧TaskStage概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_stage.md)
-- [🌐TaskGraph概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_graph.md)
-- [📚Go Worker概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/go_worker.md)
+- [stage/executor.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/executor.md)
+- [stage/stage.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/stage.md)
+- [graph/graph.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/graph.md)
+- [runtime/progress.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/progress.md)
+- [runtime/metrics.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/metrics.md)
+- [runtime/queue.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/queue.md)
+- [stage/nodes.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/nodes.md)
+- [observability/report.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/observability/report.md)
+- [graph/structure.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/structure.md)
+- [web/server.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/web/server.md)
+- [other/go_worker.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/go_worker.md)
 
 推荐阅读顺序:
 
 ```mermaid
 flowchart TD
-    classDef whiteNode fill:#ffffff,stroke:#000000,color:#000000;
+    classDef core fill:#e6efff,stroke:#3b82f6,color:#1e3a8a;
+    classDef runtime fill:#e9f8ef,stroke:#22c55e,color:#14532d;
+    classDef structure fill:#fff6e6,stroke:#f59e0b,color:#78350f;
+    classDef execution fill:#f3e8ff,stroke:#a855f7,color:#581c87;
+    classDef web fill:#ffeaea,stroke:#ef4444,color:#7f1d1d;
 
     TM[TaskExecutor.md] --> TS[TaskStage.md] --> TG[TaskGraph.md]
     TM --> TP[TaskProgress.md]
+    TM --> TME[TaskMetrics.md]
 
     TG --> TQ[TaskQueue.md]
     TG --> TN[TaskNodes.md]
@@ -168,14 +184,20 @@ flowchart TD
     TR --> TW[TaskWeb.md]
     TN --> GW[Go Worker.md]
 
-    class TM,TG,TP,TQ,TN,TR,TS,TW,GW,TSR whiteNode;
+    class TM,TS,TG core;
+    class TP,TME runtime;
+    class TSR structure;
+    class TQ,TN,GW execution;
+    class TR,TW web;
 ```
 
 以下三篇可以作为补充阅读:
 
-- [TaskTools](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_tools.md)
-- [TaskTypes](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_types.md)
-- [TaskLogging](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_logging.md)
+- [runtime/tools.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/tools.md)
+- [runtime/types.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/types.md)
+- [runtime/errors.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/errors.md)
+- [persistence/fail.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/fail.md)
+- [persistence/log.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/log.md)
 
 如果你更喜欢通过完整案例理解框架的运行方式，可以参考这篇从零开始构建 TaskGraph 的教程：
 
@@ -183,7 +205,7 @@ flowchart TD
 
 如果你对3.0.7版本加入的ctree_client与其功能感兴趣, 可以看看这一篇:
 
-[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/ctree_client.md)
+[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/ctree_client.md)
 
 ## 环境要求（Requirements）
 
@@ -205,100 +227,13 @@ flowchart TD
 
 ## 文件结构（File Structure）
 
-```
-📁 CelestialFlow	(224MB 407KB 500B)
-    📁 docs         	(34KB 944B)
-        📁 reference	(29KB 995B)
-            📝 ctree_client.md  	(6KB 997B)
-            📝 go_worker.md     	(6KB 378B)
-            📝 task_graph.md    	(4KB 989B)
-            📝 task_logging.md  	(0B)
-            📝 task_manage.md   	(7KB 499B)
-            📝 task_nodes.md    	(4KB 68B)
-            📝 task_progress.md 	(0B)
-            📝 task_queue.md    	(0B)
-            📝 task_report.md   	(0B)
-            📝 task_stage.md    	(0B)
-            📝 task_structure.md	(112B)
-            📝 task_tools.md    	(0B)
-            📝 task_types.md    	(0B)
-            📝 task_web.md      	(0B)
-        📝 quick_start.md	(4KB 973B)
-        📝 tutorial.md   	(0B)
-    📁 experiments  	(15KB 111B)
-        🐍 benchmark_datastructures.py	(6KB 52B)
-        🐍 benchmark_hash.py          	(1KB 284B)
-        🐍 benchmark_queue.py         	(5KB 504B)
-        🐍 benchmark_tqdm.py          	(1KB 160B)
-        🐍 experiment_tqdm.py         	(1KB 135B)
-    📁 go_worker    	(6MB 968KB 38B)
-        📁 worker	(5KB 746B)
-            🌀 parser.go   	(394B)
-            🌀 processor.go	(2KB 612B)
-            🌀 types.go    	(299B)
-            🌀 worker.go   	(2KB 465B)
-        ❓ go.mod       	(292B)
-        ❓ go.sum       	(1KB 68B)
-        ❓ go_worker.exe	(6MB 960KB)
-        🌀 main.go      	(980B)
-    📁 img          	(4MB 700KB 695B)
-        📷 logo(old).png  	(836KB 542B)
-        📷 logo.png       	(122KB 747B)
-        📷 web_display.png	(129KB 545B)
-        🎞️ web_ui.gif     	(3MB 635KB 909B)
-    📁 src          	(1MB 929KB 111B)
-        📁 celestialflow         	(1MB 911KB 486B)
-            📁 static     	(1MB 417KB 367B)
-                📁 css	(31KB 621B)
-                    🎨 base.css     	(6KB 181B)
-                    🎨 dashboard.css	(7KB 414B)
-                    🎨 errors.css   	(5KB 216B)
-                    🎨 inject.css   	(12KB 834B)
-                📁 js 	(33KB 672B)
-                    📜 main.js          	(5KB 65B)
-                    📜 task_errors.js   	(4KB 595B)
-                    📜 task_injection.js	(8KB 484B)
-                    📜 task_statuses.js 	(8KB 51B)
-                    📜 task_structure.js	(4KB 709B)
-                    📜 task_topology.js 	(1KB 375B)
-                    📜 utils.js         	(1KB 441B)
-                ❓ favicon.ico	(1MB 352KB 98B)
-            📁 templates  	(13KB 174B)
-                🌐 index.html	(13KB 174B)
-            📁 [1项排除的目录]	(339KB 553B)
-            🐍 task_graph.py    	(26KB 700B)
-            🐍 task_logging.py  	(10KB 282B)
-            🐍 task_manage.py   	(36KB 110B)
-            🐍 task_nodes.py    	(10KB 814B)
-            🐍 task_progress.py 	(1KB 477B)
-            🐍 task_queue.py    	(10KB 948B)
-            🐍 task_report.py   	(4KB 828B)
-            🐍 task_stage.py    	(6KB 577B)
-            🐍 task_structure.py	(7KB 255B)
-            🐍 task_tools.py    	(17KB 19B)
-            🐍 task_types.py    	(2KB 269B)
-            🐍 task_web.py      	(6KB 235B)
-            🐍 __init__.py      	(1KB 22B)
-        📁 [1项排除的目录]	(17KB 649B)
-    📁 tests        	(126KB 724B)
-        📁 [1项排除的目录]	(90KB 464B)
-        🐍 conftest.py      	(49B)
-        📝 README.md        	(6KB 324B)
-        🐍 test_graph.py    	(5KB 547B)
-        🐍 test_manage.py   	(3KB 90B)
-        🐍 test_nodes.py    	(11KB 218B)
-        🐍 test_structure.py	(10KB 56B)
-    📁 [6项排除的目录]	(210MB 685KB 450B)
-    ❓ .env          	(88B)
-    ❓ .gitignore    	(272B)
-    ❓ LICENSE       	(1KB 65B)
-    ❓ Makefile      	(501B)
-    ❓ pyproject.toml	(1KB 594B)
-    📝 README.md     	(16KB 3B)
-(celestial-flow 3.0.9)
-```
+<p align="center">
+  <img src="https://raw.githubusercontent.com/Mr-xiaotian/CelestialFlow/main/img/file_structure.svg" alt="FileStructure" />
+  <br/>
+  <em>celestial-flow 3.1.1</em>
+</p>
 
-(该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。)
+(该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。转换为图片则借助[Carbon](https://carbon.now.sh)。)
 
 ## 更新日志（Change Log）
 
@@ -326,8 +261,38 @@ flowchart TD
   - 在前端中添加error_id的显示, 为之后显示provenance_tree做准备; 
   - 增加大量warning与error, 用于提醒不规范设置; 
   - 优化前后端中错误数据的传输方式, 在大量错误数据时减少内存消耗; 
-  - 优化TaskLogger中log队列的准入机制; 
-  - 修改部分Bug
+  - 优化LogSinker中log队列的准入机制; 
+  - 修改部分Bug;
+- 3.1.0:
+  - 新增:
+    - 优化web端仪表盘页面的"总体状态摘要"卡片, 新增"总重复任务"与"总剩余市场", 后者由节点间拓扑关系计算而成, 后续还需要优化;
+  - 修复:
+    - 3.0.9版本下当web端与celestialflow运行端不同时, error数据无法传递的问题;
+    - 修复NullTaskReporter使用问题;
+- 3.1.1:
+  - 新增:
+    - [Important] CelestialTree中引入grpc, 这大大减少了emit操作的耗时;
+    - [Important] 为TerminationSignal添加id, 并可以像task一样通过CelestialTree进行跟踪;
+    - 优化task_graph与task_executor的log分级, 现在默认为"SUCCESS"级;
+    - 将go_worker部分分离为单独project: [celestialflow-goworker]https://github.com/Mr-xiaotian/celestialflow-goworker
+    - 在readme中使用svg图片来展示文件夹结构;
+    - 优化全局剩余时间的计算;
+    - 优化部分代码结构;
+  - 修复:
+    - 修复节点剩余时间在小于1s时显示0的问题(这很影响判断);
+    - 在task_graph中使用"staged"模式时会报错的问题;
+- 3.1.2:
+  - feat:
+    - [Important] 模仿已有Logger逻辑, 建立新的组合结构FailListener和FailSinker, 将原有绑定于TaskReporter的fail持久化在线程中实现; 
+    - 借此让TaskExecutor也可以进行fail持久化, 文件为fallback/{date_str}/{executor_errors}({time_str}).jsonl";
+    - 重构整体项目结构, 现在更加清晰;
+    - 补全docs/reference中全部文档, 并令其符合项目重构后结构
+    - 引入uv进行环境管理;
+    - 将executor中技术逻辑分离为runtime/metrics.py;
+    - 整合TaskQueue中重复逻辑;
+  - fix
+    - 修复前端renderNodeList中参数设置错误;
+    - 修复其他微小bug;
 
 ## Star 历史趋势（Star History）
 
@@ -341,4 +306,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 ## 作者（Author）
 Author: Mr-xiaotian
 Email: mingxiaomingtian@gmail.com
-Project Link: [https://github.com/Mr-xiaotian/CelestialFlow](https://github.com/Mr-xiaotian/CelestialFlow)
+Project Link: [https://github.com/Mr-xiaotian/CelestialFlow](https://github.com/Mr-xiaotian/CelestialFlow)

{celestialflow-3.1.0 → celestialflow-3.1.2}/pyproject.toml

@@ -1,60 +1,71 @@
-[build-system]
-requires = ["setuptools>=61.0", "wheel"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "celestialflow"                   
-version = "3.1.0"                         
-description = "A flexible GRAPH-based task orchestration framework."
-readme = "README.md"               
-license = { text = "MIT" }    
-authors = [{ name = "Mr-xiaotian", email = "mingxiaomingtian@gmail.com" }]
-keywords = ["workflow", "task", "graph", "async", "CelestialFlow"]
-requires-python = ">=3.8"
-
-dependencies = [
-    "tqdm",
-    "loguru",
-    "fastapi",
-    "uvicorn",
-    "requests",
-    "networkx",
-    "redis",
-    "jinja2",
-    "celestialtree",
-]
-
-classifiers = [
-    "Programming Language :: Python :: 3",
-    "License :: OSI Approved :: MIT License",
-    "Operating System :: OS Independent",
-    "Framework :: FastAPI",
-    "Topic :: Software Development :: Libraries",
-]
-
-[project.urls]
-"Homepage" = "https://github.com/Mr-xiaotian/CelestialFlow"
-"Bug Tracker" = "https://github.com/Mr-xiaotian/CelestialFlow/issues"
-
-[project.scripts]
-celestialflow-web = "celestialflow.task_web:main_entry"
-
-[tool.setuptools]
-license-files = []
-
-[tool.setuptools.package-data]
-"celestialflow" = ["templates/*.html", "static/**/*"]
-
-[tool.pytest.ini_options]
-addopts = "-s"
-log_cli = true
-log_cli_level = "INFO"
-log_cli_format = "%(asctime)s [%(levelname)s] %(message)s"
-log_cli_date_format = "%Y-%m-%d %H:%M:%S"
-
-filterwarnings = [
-    "ignore::DeprecationWarning",
-    "ignore::pytest.PytestDeprecationWarning",
-]
-
-asyncio_default_fixture_loop_scope = "function"
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "celestialflow"                   
+version = "3.1.2"                         
+description = "A flexible GRAPH-based task orchestration framework."
+readme = "README.md"               
+license = { text = "MIT" }    
+authors = [{ name = "Mr-xiaotian", email = "mingxiaomingtian@gmail.com" }]
+keywords = ["workflow", "task", "graph", "async", "CelestialFlow"]
+requires-python = ">=3.10"
+
+dependencies = [
+    "tqdm",
+    "loguru",
+    "fastapi",
+    "uvicorn",
+    "requests",
+    "networkx",
+    "redis",
+    "jinja2",
+    "celestialtree",
+]
+
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Framework :: FastAPI",
+    "Topic :: Software Development :: Libraries",
+]
+
+[project.urls]
+"Homepage" = "https://github.com/Mr-xiaotian/CelestialFlow"
+"Bug Tracker" = "https://github.com/Mr-xiaotian/CelestialFlow/issues"
+
+[project.scripts]
+celestialflow-web = "celestialflow.web.server:main_entry"
+
+[tool.setuptools]
+license-files = []
+
+[tool.setuptools.package-data]
+"celestialflow" = ["templates/*.html", "static/**/*"]
+
+[tool.pytest.ini_options]
+addopts = "-s"
+log_cli = true
+log_cli_level = "INFO"
+log_cli_format = "%(asctime)s [%(levelname)s] %(message)s"
+log_cli_date_format = "%Y-%m-%d %H:%M:%S"
+
+filterwarnings = [
+    "ignore::DeprecationWarning",
+    "ignore::pytest.PytestDeprecationWarning",
+]
+
+asyncio_default_fixture_loop_scope = "function"
+
+[dependency-groups]
+dev = [
+    "build>=1.2.2.post1",
+    "twine>=6.1.0",
+    "pytest>=8.3.4",
+    "pytest-asyncio>=0.25.3",
+    "httpx>=0.28.1",
+    "python-dotenv>=1.2.2",
+    "black>=26.1.0",
+]

{celestialflow-3.1.0 → celestialflow-3.1.2}/src/celestialflow/__init__.py

@@ -1,14 +1,15 @@
-from .task_graph import TaskGraph
-from .task_executor import TaskExecutor
-from .task_stage import TaskStage
-from .task_nodes import (
+# __init__.py
+from .stage import (
+    TaskExecutor,
+    TaskStage,
     TaskSplitter,
     TaskRedisSink,
     TaskRedisSource,
     TaskRedisAck,
     TaskRouter,
 )
-from .task_structure import (
+from .graph import (
+    TaskGraph,
     TaskChain,
     TaskLoop,
     TaskCross,
@@ -16,15 +17,15 @@ from .task_structure import (
     TaskWheel,
     TaskGrid,
 )
-from .task_types import TerminationSignal
-from .task_tools import (
+from .runtime.types import TerminationSignal
+from .runtime.hash import make_hashable
+from .utils.format import format_table
+from .persistence.jsonl import (
     load_jsonl_logs,
     load_task_by_stage,
     load_task_by_error,
-    make_hashable,
-    format_table,
 )
-from .task_web import TaskWebServer
+from .web.server import TaskWebServer
 
 __all__ = [
     "TaskGraph",