PyPI - mindstudio-probe - Versions diffs - 1.3.0__tar.gz → 8.1.0__tar.gz - Mend

mindstudio-probe 1.3.0tar.gz → 8.1.0tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (502) hide show

{mindstudio_probe-1.3.0 → mindstudio_probe-8.1.0}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mindstudio-probe
-Version: 1.3.0
+Version: 8.1.0
 Summary: Ascend Probe Utils
 Home-page: https://gitee.com/ascend/mstt/tree/master/debug/accuracy_tools/msprobe
 Author: Ascend Team
@@ -33,5 +33,6 @@ Requires-Dist: twisted
 Requires-Dist: matplotlib
 Requires-Dist: tensorboard
 Requires-Dist: tabulate
+Requires-Dist: pwinput
 MindStudio-Probe is a set of tools for diagnosing and improving model accuracy on Ascend NPU, including API acc checker, ptdbg, grad tool etc.

{mindstudio_probe-1.3.0 → mindstudio_probe-8.1.0}/mindstudio_probe.egg-info/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: mindstudio-probe
-Version: 1.3.0
+Version: 8.1.0
 Summary: Ascend Probe Utils
 Home-page: https://gitee.com/ascend/mstt/tree/master/debug/accuracy_tools/msprobe
 Author: Ascend Team
@@ -33,5 +33,6 @@ Requires-Dist: twisted
 Requires-Dist: matplotlib
 Requires-Dist: tensorboard
 Requires-Dist: tabulate
+Requires-Dist: pwinput
 MindStudio-Probe is a set of tools for diagnosing and improving model accuracy on Ascend NPU, including API acc checker, ptdbg, grad tool etc.

{mindstudio_probe-1.3.0 → mindstudio_probe-8.1.0}/mindstudio_probe.egg-info/SOURCES.txt RENAMED Viewed

@@ -15,6 +15,8 @@ msprobe/config.json
 msprobe/msprobe.py
 msprobe/core/__init__.py
 msprobe/core/common_config.py
+msprobe/core/hook_manager.py
+msprobe/core/service.py
 msprobe/core/advisor/advisor.py
 msprobe/core/advisor/advisor_const.py
 msprobe/core/advisor/advisor_result.py
@@ -22,14 +24,19 @@ msprobe/core/common/const.py
 msprobe/core/common/decorator.py
 msprobe/core/common/exceptions.py
 msprobe/core/common/file_utils.py
+msprobe/core/common/framework_adapter.py
+msprobe/core/common/global_lock.py
 msprobe/core/common/inplace_op_checker.py
 msprobe/core/common/inplace_ops.yaml
 msprobe/core/common/log.py
+msprobe/core/common/runtime.py
 msprobe/core/common/utils.py
 msprobe/core/compare/acc_compare.py
 msprobe/core/compare/check.py
 msprobe/core/compare/compare_cli.py
+msprobe/core/compare/config.py
 msprobe/core/compare/highlight.py
+msprobe/core/compare/ms_to_pt_api.yaml
 msprobe/core/compare/multiprocessing_compute.py
 msprobe/core/compare/npy_compare.py
 msprobe/core/compare/utils.py
@@ -40,6 +47,26 @@ msprobe/core/compare/layer_mapping/postprocess_pass.py
 msprobe/core/compare/merge_result/merge_result.py
 msprobe/core/compare/merge_result/merge_result_cli.py
 msprobe/core/compare/merge_result/utils.py
+msprobe/core/config_check/__init__.py
+msprobe/core/config_check/config_check_cli.py
+msprobe/core/config_check/config_checker.py
+msprobe/core/config_check/checkers/__init__.py
+msprobe/core/config_check/checkers/base_checker.py
+msprobe/core/config_check/checkers/dataset_checker.py
+msprobe/core/config_check/checkers/env_args_checker.py
+msprobe/core/config_check/checkers/hyperparameter_checker.py
+msprobe/core/config_check/checkers/pip_checker.py
+msprobe/core/config_check/checkers/random_checker.py
+msprobe/core/config_check/checkers/weights_checker.py
+msprobe/core/config_check/ckpt_compare/ckpt_comparator.py
+msprobe/core/config_check/ckpt_compare/megatron_loader.py
+msprobe/core/config_check/ckpt_compare/metrics.py
+msprobe/core/config_check/ckpt_compare/name_mapping.yaml
+msprobe/core/config_check/resource/dependency.yaml
+msprobe/core/config_check/resource/env.yaml
+msprobe/core/config_check/resource/hyperparameter.yaml
+msprobe/core/config_check/utils/hyperparameter_parser.py
+msprobe/core/config_check/utils/utils.py
 msprobe/core/data_dump/api_registry.py
 msprobe/core/data_dump/data_collector.py
 msprobe/core/data_dump/json_writer.py
@@ -48,10 +75,14 @@ msprobe/core/data_dump/data_processor/base.py
 msprobe/core/data_dump/data_processor/factory.py
 msprobe/core/data_dump/data_processor/mindspore_processor.py
 msprobe/core/data_dump/data_processor/pytorch_processor.py
+msprobe/core/debugger/precision_debugger.py
 msprobe/core/grad_probe/__init__.py
 msprobe/core/grad_probe/constant.py
 msprobe/core/grad_probe/grad_compare.py
 msprobe/core/grad_probe/utils.py
+msprobe/core/kernel_dump/kernel_config.py
+msprobe/core/monitor/__init__.py
+msprobe/core/monitor/anomaly_processor.py
 msprobe/core/overflow_check/abnormal_scene.py
 msprobe/core/overflow_check/api_info.py
 msprobe/core/overflow_check/checker.py
@@ -59,6 +90,9 @@ msprobe/core/overflow_check/filter.py
 msprobe/core/overflow_check/ignore_rules.yaml
 msprobe/core/overflow_check/level.py
 msprobe/core/overflow_check/utils.py
+msprobe/core/single_save/__init__.py
+msprobe/core/single_save/single_comparator.py
+msprobe/core/single_save/single_saver.py
 msprobe/docs/01.installation.md
 msprobe/docs/02.config_introduction.md
 msprobe/docs/03.config_examples.md
@@ -90,6 +124,11 @@ msprobe/docs/28.debugger_save_instruction.md
 msprobe/docs/28.kernel_dump_MindSpore.md
 msprobe/docs/29.data_dump_MSAdapter.md
 msprobe/docs/30.overflow_check_MSAdapter.md
+msprobe/docs/31.config_check.md
+msprobe/docs/32.ckpt_compare.md
+msprobe/docs/33.generate_operator_MindSpore.md
+msprobe/docs/34.RL_collect.md
+msprobe/docs/35.nan_analyze.md
 msprobe/docs/FAQ.md
 msprobe/docs/S02.report_free_benchmarking_validation_performance_baseline.md
 msprobe/docs/accuracy_checker_MindSpore/accuracy_checker_MindSpore_baseline.md
@@ -130,10 +169,12 @@ msprobe/docs/img/module_compare.png
 msprobe/docs/img/ms_dump.png
 msprobe/docs/img/ms_layer.png
 msprobe/docs/img/pt_dump.png
+msprobe/docs/img/save_compare_result_sample.png
 msprobe/docs/img/monitor/cpu_info.png
 msprobe/docs/img/monitor/step_count_per_record.png
 msprobe/docs/img/visualization/fuzzy_match_ms.png
 msprobe/docs/img/visualization/fuzzy_match_pt.png
+msprobe/docs/img/visualization/proxy.png
 msprobe/docs/img/visualization/tensorboard_1.png
 msprobe/docs/img/visualization/tensorboard_2.png
 msprobe/docs/img/visualization/vis_browser_1.png
@@ -157,9 +198,8 @@ msprobe/docs/visualization/no_mapping_analyze.png
 msprobe/docs/visualization/top_layer.png
 msprobe/mindspore/__init__.py
 msprobe/mindspore/cell_processor.py
+msprobe/mindspore/mindspore_service.py
 msprobe/mindspore/ms_config.py
-msprobe/mindspore/runtime.py
-msprobe/mindspore/service.py
 msprobe/mindspore/task_handler_factory.py
 msprobe/mindspore/api_accuracy_checker/__init__.py
 msprobe/mindspore/api_accuracy_checker/api_accuracy_checker.py
@@ -178,6 +218,9 @@ msprobe/mindspore/api_accuracy_checker/type_mapping.py
 msprobe/mindspore/api_accuracy_checker/utils.py
 msprobe/mindspore/api_accuracy_checker/bench_functions/flash_attention_score.py
 msprobe/mindspore/api_accuracy_checker/bench_functions/fusion_operator.py
+msprobe/mindspore/api_accuracy_checker/generate_op_script/config_op.json
+msprobe/mindspore/api_accuracy_checker/generate_op_script/op_generator.py
+msprobe/mindspore/api_accuracy_checker/generate_op_script/operator_replication.template
 msprobe/mindspore/code_mapping/__init__.py
 msprobe/mindspore/code_mapping/bind.py
 msprobe/mindspore/code_mapping/cmd_parser.py
@@ -188,24 +231,29 @@ msprobe/mindspore/code_mapping/processor.py
 msprobe/mindspore/common/const.py
 msprobe/mindspore/common/log.py
 msprobe/mindspore/common/utils.py
+msprobe/mindspore/compare/common_dir_compare.py
 msprobe/mindspore/compare/distributed_compare.py
 msprobe/mindspore/compare/ms_compare.py
 msprobe/mindspore/compare/ms_graph_compare.py
-msprobe/mindspore/compare/ms_to_pt_api.yaml
+msprobe/mindspore/compare/utils.py
 msprobe/mindspore/debugger/__init__.py
 msprobe/mindspore/debugger/debugger_config.py
 msprobe/mindspore/debugger/precision_debugger.py
 msprobe/mindspore/dump/__init__.py
+msprobe/mindspore/dump/cell_dump_process.py
+msprobe/mindspore/dump/cell_dump_with_insert_gradient.py
 msprobe/mindspore/dump/dump_tool_factory.py
+msprobe/mindspore/dump/graph_mode_cell_dump.py
+msprobe/mindspore/dump/graph_tensor_dump.py
 msprobe/mindspore/dump/jit_dump.py
 msprobe/mindspore/dump/kernel_graph_dump.py
 msprobe/mindspore/dump/kernel_kbyk_dump.py
 msprobe/mindspore/dump/hook_cell/api_register.py
 msprobe/mindspore/dump/hook_cell/hook_cell.py
+msprobe/mindspore/dump/hook_cell/ms_hook_manager.py
 msprobe/mindspore/dump/hook_cell/primitive_hooks.py
 msprobe/mindspore/dump/hook_cell/support_wrap_ops.yaml
-msprobe/mindspore/dump/kernel_dump/kernel_config.py
-msprobe/mindspore/dym_loader/hook_dynamic_loader.cc
+msprobe/mindspore/dym_loader/hook_dynamic_loader.cpp
 msprobe/mindspore/dym_loader/hook_dynamic_loader.h
 msprobe/mindspore/free_benchmark/__init__.py
 msprobe/mindspore/free_benchmark/api_pynative_self_check.py
@@ -236,10 +284,11 @@ msprobe/mindspore/grad_probe/hook.py
 msprobe/mindspore/grad_probe/utils.py
 msprobe/mindspore/mindtorch/__init__.py
 msprobe/mindspore/mindtorch/mindtorch_adaptor.py
-msprobe/mindspore/monitor/anomaly_detect.py
+msprobe/mindspore/monitor/common_func.py
+msprobe/mindspore/monitor/data_writers.py
 msprobe/mindspore/monitor/features.py
 msprobe/mindspore/monitor/module_hook.py
-msprobe/mindspore/monitor/module_spec_verifier.py
+msprobe/mindspore/monitor/optimizer_collect.py
 msprobe/mindspore/monitor/utils.py
 msprobe/mindspore/monitor/distributed/__init__.py
 msprobe/mindspore/monitor/distributed/distributed_ops.yaml
@@ -248,10 +297,15 @@ msprobe/mindspore/monitor/distributed/wrap_distributed.py
 msprobe/mindspore/overflow_check/__init__.py
 msprobe/mindspore/overflow_check/kernel_graph_overflow_check.py
 msprobe/mindspore/overflow_check/overflow_check_tool_factory.py
+msprobe/nan_analyze/__init__.py
+msprobe/nan_analyze/analyzer.py
+msprobe/nan_analyze/graph.py
+msprobe/nan_analyze/utils.py
 msprobe/pytorch/__init__.py
+msprobe/pytorch/attl_manager.py
 msprobe/pytorch/function_factory.py
 msprobe/pytorch/pt_config.py
-msprobe/pytorch/service.py
+msprobe/pytorch/pytorch_service.py
 msprobe/pytorch/api_accuracy_checker/.keep
 msprobe/pytorch/api_accuracy_checker/__init__.py
 msprobe/pytorch/api_accuracy_checker/config.yaml
@@ -322,11 +376,12 @@ msprobe/pytorch/compare/distributed_compare.py
 msprobe/pytorch/compare/mapping.yaml
 msprobe/pytorch/compare/match.py
 msprobe/pytorch/compare/pt_compare.py
+msprobe/pytorch/compare/utils.py
 msprobe/pytorch/debugger/__init__.py
 msprobe/pytorch/debugger/debugger_config.py
 msprobe/pytorch/debugger/precision_debugger.py
-msprobe/pytorch/dump/kernel_dump/kernel_config.py
 msprobe/pytorch/dump/module_dump/__init__.py
+msprobe/pytorch/dump/module_dump/hook_wrapper.py
 msprobe/pytorch/dump/module_dump/module_dump.py
 msprobe/pytorch/dump/module_dump/module_processer.py
 msprobe/pytorch/free_benchmark/__init__.py
@@ -362,18 +417,18 @@ msprobe/pytorch/grad_probe/grad_stat_csv.py
 msprobe/pytorch/hook_module/__init__.py
 msprobe/pytorch/hook_module/api_register.py
 msprobe/pytorch/hook_module/hook_module.py
+msprobe/pytorch/hook_module/jit_script_wrapper.py
+msprobe/pytorch/hook_module/pt_hook_manager.py
 msprobe/pytorch/hook_module/register_optimizer_hook.py
 msprobe/pytorch/hook_module/support_wrap_ops.yaml
 msprobe/pytorch/hook_module/utils.py
 msprobe/pytorch/hook_module/wrap_aten.py
 msprobe/pytorch/monitor/__init__.py
-msprobe/pytorch/monitor/anomaly_analyse.py
-msprobe/pytorch/monitor/anomaly_detect.py
 msprobe/pytorch/monitor/csv2tb.py
+msprobe/pytorch/monitor/data_writers.py
 msprobe/pytorch/monitor/features.py
 msprobe/pytorch/monitor/module_hook.py
 msprobe/pytorch/monitor/module_metric.py
-msprobe/pytorch/monitor/module_spec_verifier.py
 msprobe/pytorch/monitor/optimizer_collect.py
 msprobe/pytorch/monitor/utils.py
 msprobe/pytorch/monitor/visualizer.py
@@ -381,8 +436,6 @@ msprobe/pytorch/monitor/distributed/__init__.py
 msprobe/pytorch/monitor/distributed/distributed_ops.yaml
 msprobe/pytorch/monitor/distributed/stack_blacklist.yaml
 msprobe/pytorch/monitor/distributed/wrap_distributed.py
-msprobe/pytorch/monitor/unittest/__init__.py
-msprobe/pytorch/monitor/unittest/test_monitor.py
 msprobe/pytorch/online_dispatch/__init__.py
 msprobe/pytorch/online_dispatch/compare.py
 msprobe/pytorch/online_dispatch/dispatch.py

{mindstudio_probe-1.3.0 → mindstudio_probe-8.1.0}/mindstudio_probe.egg-info/requires.txt RENAMED Viewed

@@ -11,3 +11,4 @@ twisted
 matplotlib
 tensorboard
 tabulate
+pwinput

{mindstudio_probe-1.3.0 → mindstudio_probe-8.1.0}/msprobe/README.md RENAMED Viewed

@@ -54,7 +54,9 @@ export MSPROBE_LOG_LEVEL={x}
 **2. 工具读写的所有路径，如config_path、dump_path等，只允许包含大小写字母、数字、下划线、斜杠、点和短横线。**
-## ⚙️ [安装](./docs/01.installation.md)
+## ⚙️ 安装
+请参见[安装指导说明](./docs/01.installation.md)。
 ## 🌟 新版本特性
@@ -138,6 +140,8 @@ MindSpore 动态图场景的[离线预检](./docs/09.accuracy_checker_MindSpore.
 [PyTorch 单算子API自动生成脚本](./docs/23.generate_operator_PyTorch.md)
+[MindSpore 单算子API自动生成脚本](./docs/33.generate_operator_MindSpore.md)
 ### 11 数码关联
 该功能只支持 MindSpore 静态图场景，用于将IR图与dump数据进行关联，获取dump数据和代码调用栈的关联关系。
@@ -155,6 +159,33 @@ MindSpore 动态图场景的[离线预检](./docs/09.accuracy_checker_MindSpore.
 [MSAdapter 场景的溢出检测](./docs/30.overflow_check_MSAdapter.md)
+### 13 训练检查
+该工具主要包括：
+训练前或精度比对前，对比两个环境下可能影响训练精度的配置差异。
+[PyTorch 训练前配置检查](./docs/31.config_check.md)
+训练过程中或结束后，比较两个不同的checkpoint，评估模型相似度。
+[checkpoint比对](./docs/32.ckpt_compare.md)
+### 14 强化学习数据采集
+主要能力：
+灵活采集强化学习中重要关键过程数据，并支持比对。
+[强化学习数据采集](./docs/34.RL_collect.md)
+### 15 整网首个溢出节点分析
+多rank场景下通过dump数据找到首个出现Nan或Inf的节点。
+[PyTorch 场景整网首个溢出节点分析](./docs/35.nan_analyze.md)
 ## 📑 补充材料
 [无标杆比对功能在 PyTorch 场景的性能基线报告](./docs/S02.report_free_benchmarking_validation_performance_baseline.md)

mindstudio_probe-8.1.0/msprobe/core/__init__.py ADDED Viewed

@@ -0,0 +1,17 @@
+# Copyright (c) 2025-2025, Huawei Technologies Co., Ltd.
+# All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0  (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from msprobe.core.single_save.single_saver import SingleSave
+from msprobe.core.single_save.single_comparator import SingleComparator

mindstudio-probe 1.3.0__tar.gz → 8.1.0__tar.gz

mindstudio-probe 1.3.0tar.gz → 8.1.0tar.gz