celestialflow 3.1.1__tar.gz → 3.1.3__tar.gz

{celestialflow-3.1.1 → celestialflow-3.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: celestialflow
-Version: 3.1.1
+Version: 3.1.3
 Summary: A flexible GRAPH-based task orchestration framework.
 Author-email: Mr-xiaotian <mingxiaomingtian@gmail.com>
 License: MIT
@@ -12,17 +12,16 @@ Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Framework :: FastAPI
 Classifier: Topic :: Software Development :: Libraries
-Requires-Python: >=3.8
+Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 Requires-Dist: tqdm
-Requires-Dist: loguru
 Requires-Dist: fastapi
 Requires-Dist: uvicorn
 Requires-Dist: requests
 Requires-Dist: networkx
 Requires-Dist: redis
 Requires-Dist: jinja2
-Requires-Dist: celestialtree
+Requires-Dist: celestialtree>=0.1.2
 
 # CelestialFlow ——一个轻量级、可并行、基于图结构的 Python 任务调度框架
 
@@ -176,19 +175,31 @@ if __name__ == "__main__":
 
 若你想了解框架的整体结构与核心组件，下面的参考文档会对你有帮助：
 
-- [🔧TaskExecutor概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_executor.md)
-- [🔧TaskStage概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_stage.md)
-- [🌐TaskGraph概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_graph.md)
-- [📚Go Worker概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/go_worker.md)
+- [stage/executor.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/executor.md)
+- [stage/stage.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/stage.md)
+- [graph/graph.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/graph.md)
+- [runtime/progress.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/progress.md)
+- [runtime/metrics.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/metrics.md)
+- [runtime/queue.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/queue.md)
+- [stage/nodes.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/nodes.md)
+- [observability/report.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/observability/report.md)
+- [graph/structure.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/structure.md)
+- [web/server.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/web/server.md)
+- [other/go_worker.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/go_worker.md)
 
 推荐阅读顺序:
 
 ```mermaid
 flowchart TD
-    classDef whiteNode fill:#ffffff,stroke:#000000,color:#000000;
+    classDef core fill:#e6efff,stroke:#3b82f6,color:#1e3a8a;
+    classDef runtime fill:#e9f8ef,stroke:#22c55e,color:#14532d;
+    classDef structure fill:#fff6e6,stroke:#f59e0b,color:#78350f;
+    classDef execution fill:#f3e8ff,stroke:#a855f7,color:#581c87;
+    classDef web fill:#ffeaea,stroke:#ef4444,color:#7f1d1d;
 
     TM[TaskExecutor.md] --> TS[TaskStage.md] --> TG[TaskGraph.md]
     TM --> TP[TaskProgress.md]
+    TM --> TME[TaskMetrics.md]
 
     TG --> TQ[TaskQueue.md]
     TG --> TN[TaskNodes.md]
@@ -198,15 +209,20 @@ flowchart TD
     TR --> TW[TaskWeb.md]
     TN --> GW[Go Worker.md]
 
-    class TM,TG,TP,TQ,TN,TR,TS,TW,GW,TSR whiteNode;
+    class TM,TS,TG core;
+    class TP,TME runtime;
+    class TSR structure;
+    class TQ,TN,GW execution;
+    class TR,TW web;
 ```
 
 以下三篇可以作为补充阅读:
 
-- [TaskTools](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_tools.md)
-- [TaskTypes](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_types.md)
-- [TaskLogging](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_logging.md)
-- [TaskErrors](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_errors.md)
+- [runtime/tools.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/tools.md)
+- [runtime/types.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/types.md)
+- [runtime/errors.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/errors.md)
+- [persistence/fail.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/fail.md)
+- [persistence/log.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/log.md)
 
 如果你更喜欢通过完整案例理解框架的运行方式，可以参考这篇从零开始构建 TaskGraph 的教程：
 
@@ -214,7 +230,7 @@ flowchart TD
 
 如果你对3.0.7版本加入的ctree_client与其功能感兴趣, 可以看看这一篇:
 
-[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/ctree_client.md)
+[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/ctree_client.md)
 
 ## 环境要求（Requirements）
 
@@ -223,9 +239,8 @@ flowchart TD
 
 | 依赖包           | 说明 |
 | ----------------- | ---- |
-| **Python ≥ 3.8**  | 运行环境，建议使用 3.10 及以上版本 |
+| **Python ≥ 3.10**  | 运行环境，建议使用 3.10 及以上版本 |
 | **tqdm**          | 控制台进度条显示，用于任务执行可视化 |
-| **loguru**        | 高性能日志系统，支持多进程安全输出 |
 | **fastapi**       | Web 服务接口框架（用于任务可视化与远程控制） |
 | **uvicorn**       | FastAPI 的高性能 ASGI 服务器 |
 | **requests**      | HTTP 客户端库，用于任务状态上报与远程调用 |
@@ -239,7 +254,7 @@ flowchart TD
 <p align="center">
   <img src="https://raw.githubusercontent.com/Mr-xiaotian/CelestialFlow/main/img/file_structure.svg" alt="FileStructure" />
   <br/>
-  <em>celestial-flow 3.1.1</em>
+  <em>celestial-flow 3.1.3</em>
 </p>
 
 (该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。转换为图片则借助[Carbon](https://carbon.now.sh)。)
@@ -259,7 +274,7 @@ flowchart TD
 - 6/16/2025: 多轮评测后, 当前框架已支持完整有向图结构, 将TaskTree改名为TaskGraph
 - 3.0.1: 上线Pypi, 可喜可贺
 - 3.0.4: 新增一个抽象结构TaskQueue, 用于表示节点的所有"入边"与"出边"; 恢复未消费任务的保存功能
-- 3.0.5: 删除原有的TaskRedisTransfer节点, 并增添三种新的redis交互节点TaskRedisSink TaskRedisSource TaskRedisAck, 用于跨语言 跨进程 跨设备处理任务; 并在Web页面添加展示拓扑信息的卡片
+- 3.0.5: 删除原有的TaskRedisTransfer节点, 并增添三种新的redis交互节点TaskRedisTransport TaskRedisSource TaskRedisAck, 用于跨语言 跨进程 跨设备处理任务; 并在Web页面添加展示拓扑信息的卡片
 - 3.0.6: 添加对[CelestialTree](https://github.com/Mr-xiaotian/CelestialTree)系统的支持, 现在可以追踪单个任务的流向
 - 3.0.7: 将TaskStage从TaskExecutor中单独抽出来作为一个子类; 增加新节点TaskRouter, 可以将传入的任务选择的传给不同的下游节点, 而不是进行广播
 - 3.0.8: 在ctree逻辑上将"任务重试"事件后的"任务成功/失败/重试"事件视为因果关系, 而非之前的并行关系; 重构错误搜集部分逻辑; 修复大量3.0.6与3.07版本引入的bug; 优化部分log表现
@@ -270,7 +285,7 @@ flowchart TD
   - 在前端中添加error_id的显示, 为之后显示provenance_tree做准备; 
   - 增加大量warning与error, 用于提醒不规范设置; 
   - 优化前后端中错误数据的传输方式, 在大量错误数据时减少内存消耗; 
-  - 优化TaskLogger中log队列的准入机制; 
+  - 优化LogSinker中log队列的准入机制; 
   - 修改部分Bug;
 - 3.1.0:
   - 新增:
@@ -290,6 +305,27 @@ flowchart TD
   - 修复:
     - 修复节点剩余时间在小于1s时显示0的问题(这很影响判断);
     - 在task_graph中使用"staged"模式时会报错的问题;
+- 3.1.2:
+  - feat:
+    - [Important] 模仿已有Logger逻辑, 建立新的组合结构FailListener和FailSinker, 将原有绑定于TaskReporter的fail持久化在线程中实现; 
+    - 借此让TaskExecutor也可以进行fail持久化, 文件为fallback/{date_str}/{executor_errors}({time_str}).jsonl";
+    - 重构整体项目结构, 现在更加清晰;
+    - 补全docs/reference中全部文档, 并令其符合项目重构后结构
+    - 引入uv进行环境管理;
+    - 将executor中技术逻辑分离为runtime/metrics.py;
+    - 整合TaskQueue中重复逻辑;
+  - fix
+    - 修复前端renderNodeList中参数设置错误;
+    - 修复其他微小bug;
+- 3.1.3
+  - feat:
+    - 抽象出BaseListener与BaseSinker;
+    - 移除loguru, 完全由LogListener和LogSinker实现log记录; 
+    - 将bench相关代码从TaskExecutor和TaskGraph中抽离, 不再作为方法, 而是单独bench函数; 
+    - 重构TaskExecutor部分代码, 以尽量瘦身; 
+    - 优化log处理代码中对时间戳的处理, 现在更加准确;
+  - fix:
+    - 修复一些影响性能的小问题;
 
 ## Star 历史趋势（Star History）
 

{celestialflow-3.1.1 → celestialflow-3.1.3}/README.md

@@ -150,19 +150,31 @@ if __name__ == "__main__":
 
 若你想了解框架的整体结构与核心组件，下面的参考文档会对你有帮助：
 
-- [🔧TaskExecutor概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_executor.md)
-- [🔧TaskStage概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_stage.md)
-- [🌐TaskGraph概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_graph.md)
-- [📚Go Worker概念](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/go_worker.md)
+- [stage/executor.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/executor.md)
+- [stage/stage.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/stage.md)
+- [graph/graph.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/graph.md)
+- [runtime/progress.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/progress.md)
+- [runtime/metrics.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/metrics.md)
+- [runtime/queue.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/queue.md)
+- [stage/nodes.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/stage/nodes.md)
+- [observability/report.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/observability/report.md)
+- [graph/structure.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/graph/structure.md)
+- [web/server.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/web/server.md)
+- [other/go_worker.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/go_worker.md)
 
 推荐阅读顺序:
 
 ```mermaid
 flowchart TD
-    classDef whiteNode fill:#ffffff,stroke:#000000,color:#000000;
+    classDef core fill:#e6efff,stroke:#3b82f6,color:#1e3a8a;
+    classDef runtime fill:#e9f8ef,stroke:#22c55e,color:#14532d;
+    classDef structure fill:#fff6e6,stroke:#f59e0b,color:#78350f;
+    classDef execution fill:#f3e8ff,stroke:#a855f7,color:#581c87;
+    classDef web fill:#ffeaea,stroke:#ef4444,color:#7f1d1d;
 
     TM[TaskExecutor.md] --> TS[TaskStage.md] --> TG[TaskGraph.md]
     TM --> TP[TaskProgress.md]
+    TM --> TME[TaskMetrics.md]
 
     TG --> TQ[TaskQueue.md]
     TG --> TN[TaskNodes.md]
@@ -172,15 +184,20 @@ flowchart TD
     TR --> TW[TaskWeb.md]
     TN --> GW[Go Worker.md]
 
-    class TM,TG,TP,TQ,TN,TR,TS,TW,GW,TSR whiteNode;
+    class TM,TS,TG core;
+    class TP,TME runtime;
+    class TSR structure;
+    class TQ,TN,GW execution;
+    class TR,TW web;
 ```
 
 以下三篇可以作为补充阅读:
 
-- [TaskTools](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_tools.md)
-- [TaskTypes](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_types.md)
-- [TaskLogging](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_logging.md)
-- [TaskErrors](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/task_errors.md)
+- [runtime/tools.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/tools.md)
+- [runtime/types.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/types.md)
+- [runtime/errors.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/runtime/errors.md)
+- [persistence/fail.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/fail.md)
+- [persistence/log.md](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/persistence/log.md)
 
 如果你更喜欢通过完整案例理解框架的运行方式，可以参考这篇从零开始构建 TaskGraph 的教程：
 
@@ -188,7 +205,7 @@ flowchart TD
 
 如果你对3.0.7版本加入的ctree_client与其功能感兴趣, 可以看看这一篇:
 
-[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/ctree_client.md)
+[📚CelestialTreeClient](https://github.com/Mr-xiaotian/CelestialFlow/blob/main/docs/reference/other/ctree_client.md)
 
 ## 环境要求（Requirements）
 
@@ -197,9 +214,8 @@ flowchart TD
 
 | 依赖包           | 说明 |
 | ----------------- | ---- |
-| **Python ≥ 3.8**  | 运行环境，建议使用 3.10 及以上版本 |
+| **Python ≥ 3.10**  | 运行环境，建议使用 3.10 及以上版本 |
 | **tqdm**          | 控制台进度条显示，用于任务执行可视化 |
-| **loguru**        | 高性能日志系统，支持多进程安全输出 |
 | **fastapi**       | Web 服务接口框架（用于任务可视化与远程控制） |
 | **uvicorn**       | FastAPI 的高性能 ASGI 服务器 |
 | **requests**      | HTTP 客户端库，用于任务状态上报与远程调用 |
@@ -213,7 +229,7 @@ flowchart TD
 <p align="center">
   <img src="https://raw.githubusercontent.com/Mr-xiaotian/CelestialFlow/main/img/file_structure.svg" alt="FileStructure" />
   <br/>
-  <em>celestial-flow 3.1.1</em>
+  <em>celestial-flow 3.1.3</em>
 </p>
 
 (该视图由我的另一个项目[CelestialVault](https://github.com/Mr-xiaotian/CelestialVault)中inst_file.FileTree.print_tree()生成。转换为图片则借助[Carbon](https://carbon.now.sh)。)
@@ -233,7 +249,7 @@ flowchart TD
 - 6/16/2025: 多轮评测后, 当前框架已支持完整有向图结构, 将TaskTree改名为TaskGraph
 - 3.0.1: 上线Pypi, 可喜可贺
 - 3.0.4: 新增一个抽象结构TaskQueue, 用于表示节点的所有"入边"与"出边"; 恢复未消费任务的保存功能
-- 3.0.5: 删除原有的TaskRedisTransfer节点, 并增添三种新的redis交互节点TaskRedisSink TaskRedisSource TaskRedisAck, 用于跨语言 跨进程 跨设备处理任务; 并在Web页面添加展示拓扑信息的卡片
+- 3.0.5: 删除原有的TaskRedisTransfer节点, 并增添三种新的redis交互节点TaskRedisTransport TaskRedisSource TaskRedisAck, 用于跨语言 跨进程 跨设备处理任务; 并在Web页面添加展示拓扑信息的卡片
 - 3.0.6: 添加对[CelestialTree](https://github.com/Mr-xiaotian/CelestialTree)系统的支持, 现在可以追踪单个任务的流向
 - 3.0.7: 将TaskStage从TaskExecutor中单独抽出来作为一个子类; 增加新节点TaskRouter, 可以将传入的任务选择的传给不同的下游节点, 而不是进行广播
 - 3.0.8: 在ctree逻辑上将"任务重试"事件后的"任务成功/失败/重试"事件视为因果关系, 而非之前的并行关系; 重构错误搜集部分逻辑; 修复大量3.0.6与3.07版本引入的bug; 优化部分log表现
@@ -244,7 +260,7 @@ flowchart TD
   - 在前端中添加error_id的显示, 为之后显示provenance_tree做准备; 
   - 增加大量warning与error, 用于提醒不规范设置; 
   - 优化前后端中错误数据的传输方式, 在大量错误数据时减少内存消耗; 
-  - 优化TaskLogger中log队列的准入机制; 
+  - 优化LogSinker中log队列的准入机制; 
   - 修改部分Bug;
 - 3.1.0:
   - 新增:
@@ -264,6 +280,27 @@ flowchart TD
   - 修复:
     - 修复节点剩余时间在小于1s时显示0的问题(这很影响判断);
     - 在task_graph中使用"staged"模式时会报错的问题;
+- 3.1.2:
+  - feat:
+    - [Important] 模仿已有Logger逻辑, 建立新的组合结构FailListener和FailSinker, 将原有绑定于TaskReporter的fail持久化在线程中实现; 
+    - 借此让TaskExecutor也可以进行fail持久化, 文件为fallback/{date_str}/{executor_errors}({time_str}).jsonl";
+    - 重构整体项目结构, 现在更加清晰;
+    - 补全docs/reference中全部文档, 并令其符合项目重构后结构
+    - 引入uv进行环境管理;
+    - 将executor中技术逻辑分离为runtime/metrics.py;
+    - 整合TaskQueue中重复逻辑;
+  - fix
+    - 修复前端renderNodeList中参数设置错误;
+    - 修复其他微小bug;
+- 3.1.3
+  - feat:
+    - 抽象出BaseListener与BaseSinker;
+    - 移除loguru, 完全由LogListener和LogSinker实现log记录; 
+    - 将bench相关代码从TaskExecutor和TaskGraph中抽离, 不再作为方法, 而是单独bench函数; 
+    - 重构TaskExecutor部分代码, 以尽量瘦身; 
+    - 优化log处理代码中对时间戳的处理, 现在更加准确;
+  - fix:
+    - 修复一些影响性能的小问题;
 
 ## Star 历史趋势（Star History）
 
@@ -277,4 +314,4 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 ## 作者（Author）
 Author: Mr-xiaotian
 Email: mingxiaomingtian@gmail.com
-Project Link: [https://github.com/Mr-xiaotian/CelestialFlow](https://github.com/Mr-xiaotian/CelestialFlow)
+Project Link: [https://github.com/Mr-xiaotian/CelestialFlow](https://github.com/Mr-xiaotian/CelestialFlow)

{celestialflow-3.1.1 → celestialflow-3.1.3}/pyproject.toml

@@ -4,24 +4,23 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "celestialflow"                   
-version = "3.1.1"                         
+version = "3.1.3"
 description = "A flexible GRAPH-based task orchestration framework."
 readme = "README.md"               
 license = { text = "MIT" }    
 authors = [{ name = "Mr-xiaotian", email = "mingxiaomingtian@gmail.com" }]
 keywords = ["workflow", "task", "graph", "async", "CelestialFlow"]
-requires-python = ">=3.8"
+requires-python = ">=3.10"
 
 dependencies = [
     "tqdm",
-    "loguru",
     "fastapi",
     "uvicorn",
     "requests",
     "networkx",
     "redis",
     "jinja2",
-    "celestialtree",
+    "celestialtree>=0.1.2",
 ]
 
 classifiers = [
@@ -37,7 +36,7 @@ classifiers = [
 "Bug Tracker" = "https://github.com/Mr-xiaotian/CelestialFlow/issues"
 
 [project.scripts]
-celestialflow-web = "celestialflow.task_web:main_entry"
+celestialflow-web = "celestialflow.web.server:main_entry"
 
 [tool.setuptools]
 license-files = []
@@ -63,4 +62,9 @@ asyncio_default_fixture_loop_scope = "function"
 dev = [
     "build>=1.2.2.post1",
     "twine>=6.1.0",
+    "pytest>=8.3.4",
+    "pytest-asyncio>=0.25.3",
+    "httpx>=0.28.1",
+    "python-dotenv>=1.2.2",
+    "black>=26.1.0",
 ]

{celestialflow-3.1.1 → celestialflow-3.1.3}/src/celestialflow/__init__.py

@@ -1,14 +1,15 @@
-from .task_graph import TaskGraph
-from .task_executor import TaskExecutor
-from .task_stage import TaskStage
-from .task_nodes import (
+# __init__.py
+from .stage import (
+    TaskExecutor,
+    TaskStage,
     TaskSplitter,
-    TaskRedisSink,
+    TaskRedisTransport,
     TaskRedisSource,
     TaskRedisAck,
     TaskRouter,
 )
-from .task_structure import (
+from .graph import (
+    TaskGraph,
     TaskChain,
     TaskLoop,
     TaskCross,
@@ -16,15 +17,16 @@ from .task_structure import (
     TaskWheel,
     TaskGrid,
 )
-from .task_types import TerminationSignal
-from .task_tools import (
+from .persistence.jsonl import (
     load_jsonl_logs,
     load_task_by_stage,
     load_task_by_error,
-    make_hashable,
-    format_table,
 )
-from .task_web import TaskWebServer
+from .runtime.types import TerminationSignal
+from .runtime.hash import make_hashable
+from .utils.format import format_table
+from .utils.benchmark import benchmark_graph, benchmark_executor
+from .web.server import TaskWebServer
 
 __all__ = [
     "TaskGraph",
@@ -37,7 +39,7 @@ __all__ = [
     "TaskExecutor",
     "TaskStage",
     "TaskSplitter",
-    "TaskRedisSink",
+    "TaskRedisTransport",
     "TaskRedisSource",
     "TaskRedisAck",
     "TaskRouter",
@@ -48,4 +50,6 @@ __all__ = [
     "load_task_by_error",
     "make_hashable",
     "format_table",
+    "benchmark_graph",
+    "benchmark_executor",
 ]

celestialflow-3.1.3/src/celestialflow/graph/__init__.py

@@ -0,0 +1,20 @@
+# graph/__init__.py
+from .graph import TaskGraph
+from .structure import (
+    TaskChain,
+    TaskLoop,
+    TaskCross,
+    TaskComplete,
+    TaskWheel,
+    TaskGrid,
+)
+
+__all__ = [
+    "TaskGraph",
+    "TaskChain",
+    "TaskLoop",
+    "TaskCross",
+    "TaskComplete",
+    "TaskWheel",
+    "TaskGrid",
+]

celestialflow-3.1.3/src/celestialflow/graph/analysis.py

@@ -0,0 +1,50 @@
+# graph/analysis.py
+import networkx as nx
+from networkx import is_directed_acyclic_graph
+from typing import Dict, Any, List
+
+
+# ======== (图论分析) ========
+def format_networkx_graph(structure_graph: List[Dict[str, Any]]) -> nx.DiGraph:
+    """
+    将结构图（由 build_structure_graph 生成）转换为 networkx 有向图（DiGraph）
+
+    :param structure_graph: JSON 格式的任务结构图，List[Dict]
+    :return: 构建好的 networkx.DiGraph
+    """
+    G = nx.DiGraph()
+
+    def add_node_and_edges(node: Dict[str, Any]):
+        node_id = f'{node["name"]}[{node["func_name"]}]'
+        G.add_node(node_id, **{"mode": node.get("stage_mode")})
+
+        for child in node.get("next_stages", []):
+            child_id = f'{child["name"]}[{child["func_name"]}]'
+            G.add_edge(node_id, child_id)
+            # 递归添加子节点
+            add_node_and_edges(child)
+
+    for root in structure_graph:
+        add_node_and_edges(root)
+
+    return G
+
+
+def compute_node_levels(G: nx.DiGraph) -> Dict[str, int]:
+    """
+    计算 DAG 中每个节点的层级（最早执行阶段）
+    前提：图必须是有向无环图（DAG）
+
+    :param G: networkx 有向图（DiGraph）
+    :return: dict[node] = level (int)
+    """
+    if not nx.is_directed_acyclic_graph(G):
+        raise ValueError("该图不是 DAG，无法进行层级划分")
+
+    level = {node: 0 for node in G.nodes}  # 初始层级为 0
+
+    for node in nx.topological_sort(G):  # 按拓扑顺序遍历
+        for succ in G.successors(node):
+            level[succ] = max(level[succ], level[node] + 1)
+
+    return level