mindspore 2.3.0rc1__cp39-none-any.whl → 2.3.0rc2__cp39-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of mindspore might be problematic. Click here for more details.
- mindspore/.commit_id +1 -1
- mindspore/__init__.py +1 -1
- mindspore/_akg/akg/utils/tbe_codegen_utils.py +13 -3
- mindspore/_c_dataengine.cpython-39-aarch64-linux-gnu.so +0 -0
- mindspore/_c_expression.cpython-39-aarch64-linux-gnu.so +0 -0
- mindspore/_checkparam.py +20 -0
- mindspore/_extends/parse/parser.py +1 -1
- mindspore/_extends/parse/standard_method.py +6 -5
- mindspore/_mindspore_offline_debug.cpython-39-aarch64-linux-gnu.so +0 -0
- mindspore/amp.py +5 -5
- mindspore/boost/boost_cell_wrapper.py +1 -1
- mindspore/boost/group_loss_scale_manager.py +1 -1
- mindspore/common/__init__.py +4 -2
- mindspore/common/_register_for_recompute.py +48 -0
- mindspore/common/_stub_tensor.py +1 -0
- mindspore/common/api.py +56 -4
- mindspore/common/dtype.py +5 -3
- mindspore/common/dump.py +2 -2
- mindspore/common/hook_handle.py +51 -4
- mindspore/common/initializer.py +1 -1
- mindspore/common/jit_config.py +17 -6
- mindspore/common/parameter.py +7 -2
- mindspore/common/recompute.py +247 -0
- mindspore/common/sparse_tensor.py +2 -2
- mindspore/common/symbol.py +1 -1
- mindspore/common/tensor.py +74 -36
- mindspore/communication/__init__.py +3 -3
- mindspore/communication/management.py +30 -30
- mindspore/context.py +28 -15
- mindspore/dataset/__init__.py +5 -5
- mindspore/dataset/audio/__init__.py +2 -2
- mindspore/dataset/audio/transforms.py +51 -51
- mindspore/dataset/callback/ds_callback.py +2 -2
- mindspore/dataset/engine/cache_client.py +1 -1
- mindspore/dataset/engine/datasets.py +3 -3
- mindspore/dataset/engine/datasets_audio.py +14 -14
- mindspore/dataset/engine/datasets_standard_format.py +3 -3
- mindspore/dataset/engine/datasets_text.py +38 -38
- mindspore/dataset/engine/datasets_user_defined.py +3 -3
- mindspore/dataset/engine/datasets_vision.py +68 -68
- mindspore/dataset/text/__init__.py +3 -3
- mindspore/dataset/text/transforms.py +26 -26
- mindspore/dataset/transforms/__init__.py +1 -1
- mindspore/dataset/vision/__init__.py +3 -3
- mindspore/dataset/vision/transforms.py +92 -92
- mindspore/dataset/vision/utils.py +1 -1
- mindspore/experimental/optim/adadelta.py +2 -2
- mindspore/experimental/optim/adagrad.py +2 -2
- mindspore/experimental/optim/adam.py +2 -2
- mindspore/experimental/optim/adamax.py +2 -2
- mindspore/experimental/optim/adamw.py +2 -2
- mindspore/experimental/optim/asgd.py +2 -2
- mindspore/experimental/optim/lr_scheduler.py +24 -20
- mindspore/experimental/optim/nadam.py +2 -2
- mindspore/experimental/optim/optimizer.py +1 -1
- mindspore/experimental/optim/radam.py +2 -2
- mindspore/experimental/optim/rmsprop.py +2 -2
- mindspore/experimental/optim/rprop.py +2 -2
- mindspore/experimental/optim/sgd.py +2 -2
- mindspore/hal/stream.py +2 -0
- mindspore/include/mindapi/base/types.h +5 -0
- mindspore/lib/libdnnl.so.2 +0 -0
- mindspore/lib/libmindspore.so +0 -0
- mindspore/lib/libmindspore_backend.so +0 -0
- mindspore/lib/libmindspore_common.so +0 -0
- mindspore/lib/libmindspore_core.so +0 -0
- mindspore/lib/libmindspore_gpr.so.15 +0 -0
- mindspore/lib/libmindspore_grpc++.so.1 +0 -0
- mindspore/lib/libmindspore_grpc.so.15 +0 -0
- mindspore/lib/libmindspore_shared_lib.so +0 -0
- mindspore/lib/libopencv_core.so.4.5 +0 -0
- mindspore/lib/libopencv_imgcodecs.so.4.5 +0 -0
- mindspore/lib/libopencv_imgproc.so.4.5 +0 -0
- mindspore/lib/plugin/ascend/custom_aicpu_ops/op_impl/cpu/aicpu_kernel/impl/libcust_cpu_kernels.so +0 -0
- mindspore/lib/plugin/ascend/custom_aicpu_ops/op_impl/cpu/config/cust_aicpu_kernel.json +6 -6
- mindspore/lib/plugin/ascend/custom_aicpu_ops/op_proto/libcust_op_proto.so +0 -0
- mindspore/lib/plugin/ascend/libdvpp_utils.so +0 -0
- mindspore/lib/plugin/ascend/liblowlatency_collective.so +0 -0
- mindspore/lib/plugin/ascend/libmindspore_cpu_kernels.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/bin/DeviceBin +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/bin/PkgInspect +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/bin/op_man +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/device/ascend910b/bin/ascend910b.bin +101787 -98559
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/host/libasdops_cann_host.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/host/libasdops_host.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/base/op_register.h +2 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/params/mix.h +8 -1
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/params/norm.h +5 -3
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/params/reduce.h +2 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/utils/rt/backend/backend.h +3 -3
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/utils/rt/backend/rtbackend.h +3 -3
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/utils/rt/base/types.h +0 -1
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/utils/rt/module/module.h +3 -3
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/include/asdops/utils/svector/svector.h +3 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/lib/libasdops.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/asdops/lib/libasdops_static.a +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/add/tiling/add_tiling.h +9 -9
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/apply_rotary_pos_emb_impl.h +2 -6
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb.h +2 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb_base.h +460 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb_bf16.h +217 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb_fp16.h +116 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb_tiling.h +16 -24
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/apply_rotary_pos_emb/kernel/apply_rotary_pos_emb_value.h +27 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/asdop/asd_op_impl.h +0 -4
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{attention/FlashAttentionScore_impl.h → flash_attention_score/flash_attention_score_impl.h} +2 -1
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{attention/bs_attention_tiling.h → flash_attention_score/flash_attention_score_tiling.h} +15 -19
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/gelu/tiling/gelu_tiling.h +7 -9
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/lccl/lccl_wrapper.h +58 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/matmul/matmul_impl.h +19 -8
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{matmul → matmul_common}/pp_matmul_common_tiling.h +18 -8
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{matmul → matmul_common}/pp_matmul_info.h +7 -4
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{matmul → matmul_common}/tiling_data.h +44 -6
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/matmul_common/tiling_utils.h +65 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/matmul_stridedslice/matmul_stridedslice_fusion_impl.h +10 -6
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/op_param.h +4 -1
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/paged_attention/kernel/paged_attention_mix_hwsync.h +41 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{attention/PagedAttention_impl.h → paged_attention/paged_attention_impl.h} +1 -1
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/paged_attention/paged_attention_tiling.h +63 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/param/add_param.h +2 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{attention_param.h → param/attention_param.h} +11 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/param/matmul_ext_param.h +37 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/param/sub_param.h +45 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/reshape_and_cache/reshape_and_cache_tiling.h +1 -2
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/kernel/rms_norm.h +23 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/kernel/rms_norm_base.h +175 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/kernel/rms_norm_normal.h +276 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/kernel/rms_norm_split_d.h +280 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/kernel/tiling_data.h +35 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/rms_norm/rms_norm_impl.h +45 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/sub/kernel/sub_kernel.h +20 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/sub/sub_impl.h +47 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/sub/sub_tiling.h +25 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/tune_repo/matmul_table.h +323 -23
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/types.h +15 -4
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/utils/log/log_tiling.h +8 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libAdd_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libSub_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libadd_layernorm_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libadd_rms_norm_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libapply_rotary_pos_emb_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libcast_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libgelu_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libmatmul_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libmatmul_stridedslice_fusion_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libms_kernels_internal.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libnot_equal_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/libreshape_and_cache_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/lib/librms_norm_impl.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_bf16_bnsd_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_bf16_bnsd_tri_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_bf16_bsh_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_bf16_bsh_tri_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_fp16_bnsd_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_fp16_bnsd_tri_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_fp16_bsh_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/flash_attention_score_fp16_bsh_tri_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/paged_attention_bf16_bnsd_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/paged_attention_bf16_bsh_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/paged_attention_fp16_bnsd_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/paged_attention_fp16_bsh_full_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lcal.h +22 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lcal_comm.h +70 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lcal_types.h +103 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lccl.h +47 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lccl_wrapper.h +58 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/include/lcoc.h +154 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/lib/liblcal.so +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/lccl/lib/liblccl_wrapper.so +0 -0
- mindspore/lib/plugin/libmindspore_ascend.so.2 +0 -0
- mindspore/log.py +2 -2
- mindspore/mint/__init__.py +457 -0
- mindspore/mint/nn/__init__.py +430 -0
- mindspore/mint/nn/functional.py +424 -0
- mindspore/mint/optim/__init__.py +24 -0
- mindspore/mint/optim/adamw.py +186 -0
- mindspore/multiprocessing/__init__.py +4 -0
- mindspore/nn/__init__.py +3 -0
- mindspore/nn/cell.py +51 -47
- mindspore/nn/extend/__init__.py +29 -0
- mindspore/nn/extend/basic.py +140 -0
- mindspore/nn/extend/embedding.py +143 -0
- mindspore/nn/extend/layer/__init__.py +27 -0
- mindspore/nn/extend/layer/normalization.py +107 -0
- mindspore/nn/extend/pooling.py +117 -0
- mindspore/nn/generator.py +297 -0
- mindspore/nn/layer/basic.py +109 -1
- mindspore/nn/layer/container.py +2 -2
- mindspore/nn/layer/conv.py +6 -6
- mindspore/nn/layer/embedding.py +1 -1
- mindspore/nn/layer/normalization.py +21 -43
- mindspore/nn/layer/padding.py +4 -0
- mindspore/nn/optim/ada_grad.py +2 -2
- mindspore/nn/optim/adadelta.py +1 -1
- mindspore/nn/optim/adafactor.py +1 -1
- mindspore/nn/optim/adam.py +7 -7
- mindspore/nn/optim/adamax.py +2 -2
- mindspore/nn/optim/adasum.py +2 -2
- mindspore/nn/optim/asgd.py +2 -2
- mindspore/nn/optim/ftrl.py +1 -1
- mindspore/nn/optim/lamb.py +3 -3
- mindspore/nn/optim/lars.py +1 -1
- mindspore/nn/optim/lazyadam.py +2 -2
- mindspore/nn/optim/momentum.py +2 -2
- mindspore/nn/optim/optimizer.py +2 -2
- mindspore/nn/optim/proximal_ada_grad.py +2 -2
- mindspore/nn/optim/rmsprop.py +2 -2
- mindspore/nn/optim/rprop.py +2 -2
- mindspore/nn/optim/sgd.py +2 -2
- mindspore/nn/optim/thor.py +2 -2
- mindspore/nn/wrap/cell_wrapper.py +9 -9
- mindspore/nn/wrap/grad_reducer.py +5 -5
- mindspore/ops/_grad_experimental/grad_comm_ops.py +4 -2
- mindspore/ops/_vmap/vmap_grad_nn_ops.py +41 -2
- mindspore/ops/_vmap/vmap_math_ops.py +27 -8
- mindspore/ops/_vmap/vmap_nn_ops.py +66 -8
- mindspore/ops/auto_generate/cpp_create_prim_instance_helper.py +73 -1
- mindspore/ops/auto_generate/gen_arg_dtype_cast.py +12 -3
- mindspore/ops/auto_generate/gen_arg_handler.py +24 -0
- mindspore/ops/auto_generate/gen_extend_func.py +274 -0
- mindspore/ops/auto_generate/gen_ops_def.py +889 -22
- mindspore/ops/auto_generate/gen_ops_prim.py +3541 -253
- mindspore/ops/auto_generate/pyboost_inner_prim.py +282 -0
- mindspore/ops/composite/multitype_ops/_compile_utils.py +2 -1
- mindspore/ops/composite/multitype_ops/_constexpr_utils.py +9 -0
- mindspore/ops/extend/__init__.py +9 -1
- mindspore/ops/extend/array_func.py +134 -27
- mindspore/ops/extend/math_func.py +3 -3
- mindspore/ops/extend/nn_func.py +363 -2
- mindspore/ops/function/__init__.py +19 -2
- mindspore/ops/function/array_func.py +463 -439
- mindspore/ops/function/clip_func.py +7 -18
- mindspore/ops/function/grad/grad_func.py +5 -5
- mindspore/ops/function/linalg_func.py +4 -4
- mindspore/ops/function/math_func.py +260 -243
- mindspore/ops/function/nn_func.py +825 -62
- mindspore/ops/function/random_func.py +73 -4
- mindspore/ops/function/sparse_unary_func.py +1 -1
- mindspore/ops/function/vmap_func.py +1 -1
- mindspore/ops/functional.py +2 -2
- mindspore/ops/op_info_register.py +1 -31
- mindspore/ops/operations/__init__.py +2 -3
- mindspore/ops/operations/_grad_ops.py +2 -107
- mindspore/ops/operations/_inner_ops.py +5 -5
- mindspore/ops/operations/_sequence_ops.py +2 -2
- mindspore/ops/operations/array_ops.py +11 -233
- mindspore/ops/operations/comm_ops.py +32 -32
- mindspore/ops/operations/custom_ops.py +7 -89
- mindspore/ops/operations/manually_defined/ops_def.py +329 -4
- mindspore/ops/operations/math_ops.py +13 -163
- mindspore/ops/operations/nn_ops.py +9 -316
- mindspore/ops/operations/random_ops.py +1 -1
- mindspore/ops/operations/sparse_ops.py +3 -3
- mindspore/ops/primitive.py +2 -2
- mindspore/ops_generate/arg_dtype_cast.py +12 -3
- mindspore/ops_generate/arg_handler.py +24 -0
- mindspore/ops_generate/gen_ops_inner_prim.py +2 -0
- mindspore/ops_generate/gen_pyboost_func.py +13 -6
- mindspore/ops_generate/pyboost_utils.py +2 -17
- mindspore/parallel/__init__.py +3 -2
- mindspore/parallel/_auto_parallel_context.py +106 -1
- mindspore/parallel/_parallel_serialization.py +34 -2
- mindspore/parallel/_utils.py +16 -0
- mindspore/parallel/algo_parameter_config.py +4 -4
- mindspore/parallel/checkpoint_transform.py +249 -77
- mindspore/parallel/cluster/process_entity/_api.py +1 -1
- mindspore/parallel/parameter_broadcast.py +1 -1
- mindspore/parallel/shard.py +1 -1
- mindspore/profiler/parser/ascend_analysis/fwk_cann_parser.py +1 -0
- mindspore/profiler/parser/ascend_analysis/profiler_info_parser.py +17 -5
- mindspore/profiler/parser/ascend_msprof_exporter.py +3 -3
- mindspore/profiler/parser/ascend_msprof_generator.py +10 -3
- mindspore/profiler/parser/ascend_op_generator.py +26 -9
- mindspore/profiler/parser/ascend_timeline_generator.py +7 -4
- mindspore/profiler/parser/profiler_info.py +11 -1
- mindspore/profiler/profiling.py +13 -5
- mindspore/rewrite/api/node.py +12 -12
- mindspore/rewrite/api/symbol_tree.py +11 -11
- mindspore/run_check/_check_version.py +1 -1
- mindspore/safeguard/rewrite_obfuscation.py +2 -2
- mindspore/train/amp.py +4 -4
- mindspore/train/anf_ir_pb2.py +8 -2
- mindspore/train/callback/_backup_and_restore.py +2 -2
- mindspore/train/callback/_callback.py +4 -4
- mindspore/train/callback/_checkpoint.py +2 -2
- mindspore/train/callback/_early_stop.py +2 -2
- mindspore/train/callback/_landscape.py +4 -4
- mindspore/train/callback/_loss_monitor.py +2 -2
- mindspore/train/callback/_on_request_exit.py +2 -2
- mindspore/train/callback/_reduce_lr_on_plateau.py +2 -2
- mindspore/train/callback/_summary_collector.py +2 -2
- mindspore/train/callback/_time_monitor.py +2 -2
- mindspore/train/dataset_helper.py +8 -3
- mindspore/train/loss_scale_manager.py +2 -2
- mindspore/train/metrics/metric.py +3 -3
- mindspore/train/mind_ir_pb2.py +22 -17
- mindspore/train/model.py +15 -15
- mindspore/train/serialization.py +18 -18
- mindspore/train/summary/summary_record.py +7 -7
- mindspore/train/train_thor/convert_utils.py +3 -3
- mindspore/version.py +1 -1
- {mindspore-2.3.0rc1.dist-info → mindspore-2.3.0rc2.dist-info}/METADATA +1 -1
- {mindspore-2.3.0rc1.dist-info → mindspore-2.3.0rc2.dist-info}/RECORD +307 -260
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/matmul_stridedslice/tiling_data.h +0 -59
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/FlashAttentionScore_bf16_BNSD_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/FlashAttentionScore_bf16_BSH_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/FlashAttentionScore_fp16_BNSD_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/FlashAttentionScore_fp16_BSH_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/PagedAttention_bf16_BNSD_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/PagedAttention_bf16_BSH_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/PagedAttention_fp16_BNSD_mix.o +0 -0
- mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/op_kernels/ascend910b/BSAttention/PagedAttention_fp16_BSH_mix.o +0 -0
- /mindspore/lib/plugin/ascend/ms_kernels_internal/internal_kernel/include/{attention/bs_attention_mix_hwsync.h → flash_attention_score/kernel/flash_attention_score_mix_hwsync.h} +0 -0
- {mindspore-2.3.0rc1.dist-info → mindspore-2.3.0rc2.dist-info}/WHEEL +0 -0
- {mindspore-2.3.0rc1.dist-info → mindspore-2.3.0rc2.dist-info}/entry_points.txt +0 -0
- {mindspore-2.3.0rc1.dist-info → mindspore-2.3.0rc2.dist-info}/top_level.txt +0 -0
|
@@ -156,7 +156,7 @@ class Caltech101Dataset(GeneratorDataset):
|
|
|
156
156
|
|
|
157
157
|
Tutorial Examples:
|
|
158
158
|
- `Load & Process Data With Dataset Pipeline
|
|
159
|
-
<https://www.mindspore.cn/docs/en/
|
|
159
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
160
160
|
|
|
161
161
|
Note:
|
|
162
162
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -287,7 +287,7 @@ class Caltech256Dataset(MappableDataset, VisionBaseDataset):
|
|
|
287
287
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
288
288
|
argument can only be specified when `num_shards` is also specified.
|
|
289
289
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
290
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
290
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
291
291
|
Default: ``None`` , which means no cache is used.
|
|
292
292
|
|
|
293
293
|
Raises:
|
|
@@ -302,7 +302,7 @@ class Caltech256Dataset(MappableDataset, VisionBaseDataset):
|
|
|
302
302
|
|
|
303
303
|
Tutorial Examples:
|
|
304
304
|
- `Load & Process Data With Dataset Pipeline
|
|
305
|
-
<https://www.mindspore.cn/docs/en/
|
|
305
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
306
306
|
|
|
307
307
|
Note:
|
|
308
308
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -399,7 +399,7 @@ class CelebADataset(MappableDataset, VisionBaseDataset):
|
|
|
399
399
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
400
400
|
argument can only be specified when `num_shards` is also specified.
|
|
401
401
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
402
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
402
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
403
403
|
Default: ``None`` , which means no cache is used.
|
|
404
404
|
decrypt (callable, optional): Image decryption function, which accepts the path of the encrypted image file
|
|
405
405
|
and returns the decrypted bytes data. Default: ``None`` , no decryption.
|
|
@@ -416,7 +416,7 @@ class CelebADataset(MappableDataset, VisionBaseDataset):
|
|
|
416
416
|
|
|
417
417
|
Tutorial Examples:
|
|
418
418
|
- `Load & Process Data With Dataset Pipeline
|
|
419
|
-
<https://www.mindspore.cn/docs/en/
|
|
419
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
420
420
|
|
|
421
421
|
Note:
|
|
422
422
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -552,7 +552,7 @@ class Cifar10Dataset(MappableDataset, VisionBaseDataset):
|
|
|
552
552
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
553
553
|
argument can only be specified when `num_shards` is also specified.
|
|
554
554
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
555
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
555
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
556
556
|
Default: ``None`` , which means no cache is used.
|
|
557
557
|
|
|
558
558
|
Raises:
|
|
@@ -567,7 +567,7 @@ class Cifar10Dataset(MappableDataset, VisionBaseDataset):
|
|
|
567
567
|
|
|
568
568
|
Tutorial Examples:
|
|
569
569
|
- `Load & Process Data With Dataset Pipeline
|
|
570
|
-
<https://www.mindspore.cn/docs/en/
|
|
570
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
571
571
|
|
|
572
572
|
Note:
|
|
573
573
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -666,7 +666,7 @@ class Cifar100Dataset(MappableDataset, VisionBaseDataset):
|
|
|
666
666
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
667
667
|
argument can only be specified when `num_shards` is also specified.
|
|
668
668
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
669
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
669
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
670
670
|
Default: ``None`` , which means no cache is used.
|
|
671
671
|
|
|
672
672
|
Raises:
|
|
@@ -681,7 +681,7 @@ class Cifar100Dataset(MappableDataset, VisionBaseDataset):
|
|
|
681
681
|
|
|
682
682
|
Tutorial Examples:
|
|
683
683
|
- `Load & Process Data With Dataset Pipeline
|
|
684
|
-
<https://www.mindspore.cn/docs/en/
|
|
684
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
685
685
|
|
|
686
686
|
Note:
|
|
687
687
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -780,7 +780,7 @@ class CityscapesDataset(MappableDataset, VisionBaseDataset):
|
|
|
780
780
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
781
781
|
argument can only be specified when `num_shards` is also specified.
|
|
782
782
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
783
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
783
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
784
784
|
Default: ``None`` , which means no cache is used.
|
|
785
785
|
|
|
786
786
|
Raises:
|
|
@@ -798,7 +798,7 @@ class CityscapesDataset(MappableDataset, VisionBaseDataset):
|
|
|
798
798
|
|
|
799
799
|
Tutorial Examples:
|
|
800
800
|
- `Load & Process Data With Dataset Pipeline
|
|
801
|
-
<https://www.mindspore.cn/docs/en/
|
|
801
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
802
802
|
|
|
803
803
|
Note:
|
|
804
804
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -931,7 +931,7 @@ class CocoDataset(MappableDataset, VisionBaseDataset):
|
|
|
931
931
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
932
932
|
argument can only be specified when `num_shards` is also specified.
|
|
933
933
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
934
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
934
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
935
935
|
Default: ``None`` , which means no cache is used.
|
|
936
936
|
extra_metadata(bool, optional): Flag to add extra meta-data to row. If True, an additional column will be
|
|
937
937
|
output at the end :py:obj:`[_meta-filename, dtype=string]` . Default: ``False``.
|
|
@@ -994,7 +994,7 @@ class CocoDataset(MappableDataset, VisionBaseDataset):
|
|
|
994
994
|
|
|
995
995
|
Tutorial Examples:
|
|
996
996
|
- `Load & Process Data With Dataset Pipeline
|
|
997
|
-
<https://www.mindspore.cn/docs/en/
|
|
997
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
998
998
|
|
|
999
999
|
Note:
|
|
1000
1000
|
- Column '[_meta-filename, dtype=string]' won't be output unless an explicit rename dataset op is added
|
|
@@ -1173,7 +1173,7 @@ class DIV2KDataset(MappableDataset, VisionBaseDataset):
|
|
|
1173
1173
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
1174
1174
|
argument can only be specified when `num_shards` is also specified.
|
|
1175
1175
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
1176
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
1176
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
1177
1177
|
Default: ``None`` , which means no cache is used.
|
|
1178
1178
|
|
|
1179
1179
|
Raises:
|
|
@@ -1194,7 +1194,7 @@ class DIV2KDataset(MappableDataset, VisionBaseDataset):
|
|
|
1194
1194
|
|
|
1195
1195
|
Tutorial Examples:
|
|
1196
1196
|
- `Load & Process Data With Dataset Pipeline
|
|
1197
|
-
<https://www.mindspore.cn/docs/en/
|
|
1197
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1198
1198
|
|
|
1199
1199
|
Note:
|
|
1200
1200
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -1341,7 +1341,7 @@ class EMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
1341
1341
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
1342
1342
|
argument can only be specified when `num_shards` is also specified.
|
|
1343
1343
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
1344
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
1344
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
1345
1345
|
Default: ``None`` , which means no cache is used.
|
|
1346
1346
|
|
|
1347
1347
|
Raises:
|
|
@@ -1353,7 +1353,7 @@ class EMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
1353
1353
|
|
|
1354
1354
|
Tutorial Examples:
|
|
1355
1355
|
- `Load & Process Data With Dataset Pipeline
|
|
1356
|
-
<https://www.mindspore.cn/docs/en/
|
|
1356
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1357
1357
|
|
|
1358
1358
|
Note:
|
|
1359
1359
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -1456,7 +1456,7 @@ class FakeImageDataset(MappableDataset, VisionBaseDataset):
|
|
|
1456
1456
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
1457
1457
|
argument can only be specified when `num_shards` is also specified.
|
|
1458
1458
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
1459
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
1459
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
1460
1460
|
Default: ``None`` , which means no cache is used.
|
|
1461
1461
|
|
|
1462
1462
|
Raises:
|
|
@@ -1469,7 +1469,7 @@ class FakeImageDataset(MappableDataset, VisionBaseDataset):
|
|
|
1469
1469
|
|
|
1470
1470
|
Tutorial Examples:
|
|
1471
1471
|
- `Load & Process Data With Dataset Pipeline
|
|
1472
|
-
<https://www.mindspore.cn/docs/en/
|
|
1472
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1473
1473
|
|
|
1474
1474
|
Note:
|
|
1475
1475
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -1527,7 +1527,7 @@ class FashionMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
1527
1527
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
1528
1528
|
argument can only be specified when `num_shards` is also specified.
|
|
1529
1529
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
1530
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
1530
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
1531
1531
|
Default: ``None`` , which means no cache is used.
|
|
1532
1532
|
|
|
1533
1533
|
Raises:
|
|
@@ -1541,7 +1541,7 @@ class FashionMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
1541
1541
|
|
|
1542
1542
|
Tutorial Examples:
|
|
1543
1543
|
- `Load & Process Data With Dataset Pipeline
|
|
1544
|
-
<https://www.mindspore.cn/docs/en/
|
|
1544
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1545
1545
|
|
|
1546
1546
|
Note:
|
|
1547
1547
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -1632,7 +1632,7 @@ class FlickrDataset(MappableDataset, VisionBaseDataset):
|
|
|
1632
1632
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
1633
1633
|
argument can only be specified when `num_shards` is also specified.
|
|
1634
1634
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
1635
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
1635
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
1636
1636
|
Default: ``None`` , which means no cache is used.
|
|
1637
1637
|
|
|
1638
1638
|
Raises:
|
|
@@ -1648,7 +1648,7 @@ class FlickrDataset(MappableDataset, VisionBaseDataset):
|
|
|
1648
1648
|
|
|
1649
1649
|
Tutorial Examples:
|
|
1650
1650
|
- `Load & Process Data With Dataset Pipeline
|
|
1651
|
-
<https://www.mindspore.cn/docs/en/
|
|
1651
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1652
1652
|
|
|
1653
1653
|
Note:
|
|
1654
1654
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -1871,7 +1871,7 @@ class Flowers102Dataset(GeneratorDataset):
|
|
|
1871
1871
|
|
|
1872
1872
|
Tutorial Examples:
|
|
1873
1873
|
- `Load & Process Data With Dataset Pipeline
|
|
1874
|
-
<https://www.mindspore.cn/docs/en/
|
|
1874
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
1875
1875
|
|
|
1876
1876
|
Note:
|
|
1877
1877
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2005,7 +2005,7 @@ class Food101Dataset(MappableDataset, VisionBaseDataset):
|
|
|
2005
2005
|
shard_id (int, optional): The shard ID within `num_shards` . This argument can only be specified
|
|
2006
2006
|
when `num_shards` is also specified. Default: ``None`` .
|
|
2007
2007
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2008
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2008
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2009
2009
|
Default: ``None`` , which means no cache is used.
|
|
2010
2010
|
|
|
2011
2011
|
Raises:
|
|
@@ -2021,7 +2021,7 @@ class Food101Dataset(MappableDataset, VisionBaseDataset):
|
|
|
2021
2021
|
|
|
2022
2022
|
Tutorial Examples:
|
|
2023
2023
|
- `Load & Process Data With Dataset Pipeline
|
|
2024
|
-
<https://www.mindspore.cn/docs/en/
|
|
2024
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2025
2025
|
|
|
2026
2026
|
Note:
|
|
2027
2027
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2126,7 +2126,7 @@ class ImageFolderDataset(MappableDataset, VisionBaseDataset):
|
|
|
2126
2126
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
2127
2127
|
argument can only be specified when `num_shards` is also specified.
|
|
2128
2128
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2129
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2129
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2130
2130
|
Default: ``None`` , which means no cache is used.
|
|
2131
2131
|
decrypt (callable, optional): Image decryption function, which accepts the path of the encrypted image file
|
|
2132
2132
|
and returns the decrypted bytes data. Default: ``None`` , no decryption.
|
|
@@ -2143,7 +2143,7 @@ class ImageFolderDataset(MappableDataset, VisionBaseDataset):
|
|
|
2143
2143
|
|
|
2144
2144
|
Tutorial Examples:
|
|
2145
2145
|
- `Load & Process Data With Dataset Pipeline
|
|
2146
|
-
<https://www.mindspore.cn/docs/en/
|
|
2146
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2147
2147
|
|
|
2148
2148
|
Note:
|
|
2149
2149
|
- The shape of the image column is [image_size] if `decode` flag is ``False``, or [H,W,C] otherwise.
|
|
@@ -2270,7 +2270,7 @@ class KITTIDataset(MappableDataset, VisionBaseDataset):
|
|
|
2270
2270
|
shard_id (int, optional): The shard ID within `num_shards`. Default: ``None`` . This
|
|
2271
2271
|
argument can only be specified when `num_shards` is also specified.
|
|
2272
2272
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2273
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2273
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2274
2274
|
Default: ``None`` , which means no cache is used.
|
|
2275
2275
|
|
|
2276
2276
|
Raises:
|
|
@@ -2283,7 +2283,7 @@ class KITTIDataset(MappableDataset, VisionBaseDataset):
|
|
|
2283
2283
|
|
|
2284
2284
|
Tutorial Examples:
|
|
2285
2285
|
- `Load & Process Data With Dataset Pipeline
|
|
2286
|
-
<https://www.mindspore.cn/docs/en/
|
|
2286
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2287
2287
|
|
|
2288
2288
|
Note:
|
|
2289
2289
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2390,7 +2390,7 @@ class KMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
2390
2390
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
2391
2391
|
argument can only be specified when `num_shards` is also specified.
|
|
2392
2392
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2393
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2393
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2394
2394
|
Default: ``None`` , which means no cache is used.
|
|
2395
2395
|
|
|
2396
2396
|
Raises:
|
|
@@ -2404,7 +2404,7 @@ class KMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
2404
2404
|
|
|
2405
2405
|
Tutorial Examples:
|
|
2406
2406
|
- `Load & Process Data With Dataset Pipeline
|
|
2407
|
-
<https://www.mindspore.cn/docs/en/
|
|
2407
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2408
2408
|
|
|
2409
2409
|
Note:
|
|
2410
2410
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2500,7 +2500,7 @@ class LFWDataset(MappableDataset, VisionBaseDataset):
|
|
|
2500
2500
|
shard_id (int, optional): The shard ID within `num_shards`. Default: ``None`` . This
|
|
2501
2501
|
argument can only be specified when `num_shards` is also specified.
|
|
2502
2502
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2503
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2503
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2504
2504
|
Default: ``None`` , which means no cache is used.
|
|
2505
2505
|
|
|
2506
2506
|
Raises:
|
|
@@ -2513,7 +2513,7 @@ class LFWDataset(MappableDataset, VisionBaseDataset):
|
|
|
2513
2513
|
|
|
2514
2514
|
Tutorial Examples:
|
|
2515
2515
|
- `Load & Process Data With Dataset Pipeline
|
|
2516
|
-
<https://www.mindspore.cn/docs/en/
|
|
2516
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2517
2517
|
|
|
2518
2518
|
Note:
|
|
2519
2519
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2639,7 +2639,7 @@ class LSUNDataset(MappableDataset, VisionBaseDataset):
|
|
|
2639
2639
|
shard_id (int, optional): The shard ID within `num_shards`. Default: ``None`` . This
|
|
2640
2640
|
argument can only be specified when `num_shards` is also specified.
|
|
2641
2641
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2642
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2642
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2643
2643
|
Default: ``None`` , which means no cache is used.
|
|
2644
2644
|
|
|
2645
2645
|
Raises:
|
|
@@ -2653,7 +2653,7 @@ class LSUNDataset(MappableDataset, VisionBaseDataset):
|
|
|
2653
2653
|
|
|
2654
2654
|
Tutorial Examples:
|
|
2655
2655
|
- `Load & Process Data With Dataset Pipeline
|
|
2656
|
-
<https://www.mindspore.cn/docs/en/
|
|
2656
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2657
2657
|
|
|
2658
2658
|
Note:
|
|
2659
2659
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2760,7 +2760,7 @@ class ManifestDataset(MappableDataset, VisionBaseDataset):
|
|
|
2760
2760
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
2761
2761
|
argument can only be specified when `num_shards` is also specified.
|
|
2762
2762
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2763
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2763
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2764
2764
|
Default: ``None`` , which means no cache is used.
|
|
2765
2765
|
|
|
2766
2766
|
Raises:
|
|
@@ -2775,7 +2775,7 @@ class ManifestDataset(MappableDataset, VisionBaseDataset):
|
|
|
2775
2775
|
|
|
2776
2776
|
Tutorial Examples:
|
|
2777
2777
|
- `Load & Process Data With Dataset Pipeline
|
|
2778
|
-
<https://www.mindspore.cn/docs/en/
|
|
2778
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2779
2779
|
|
|
2780
2780
|
Note:
|
|
2781
2781
|
- If `decode` is ``False`` , the "image" column will get the 1D raw bytes of the image.
|
|
@@ -2881,7 +2881,7 @@ class MnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
2881
2881
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
2882
2882
|
argument can only be specified when `num_shards` is also specified.
|
|
2883
2883
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2884
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2884
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2885
2885
|
Default: ``None`` , which means no cache is used.
|
|
2886
2886
|
|
|
2887
2887
|
Raises:
|
|
@@ -2896,7 +2896,7 @@ class MnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
2896
2896
|
|
|
2897
2897
|
Tutorial Examples:
|
|
2898
2898
|
- `Load & Process Data With Dataset Pipeline
|
|
2899
|
-
<https://www.mindspore.cn/docs/en/
|
|
2899
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
2900
2900
|
|
|
2901
2901
|
Note:
|
|
2902
2902
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -2986,7 +2986,7 @@ class OmniglotDataset(MappableDataset, VisionBaseDataset):
|
|
|
2986
2986
|
shard_id (int, optional): The shard ID within `num_shards`. Default: ``None`` . This
|
|
2987
2987
|
argument can only be specified when `num_shards` is also specified.
|
|
2988
2988
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
2989
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
2989
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
2990
2990
|
Default: ``None`` , which means no cache is used.
|
|
2991
2991
|
|
|
2992
2992
|
Raises:
|
|
@@ -2999,7 +2999,7 @@ class OmniglotDataset(MappableDataset, VisionBaseDataset):
|
|
|
2999
2999
|
|
|
3000
3000
|
Tutorial Examples:
|
|
3001
3001
|
- `Load & Process Data With Dataset Pipeline
|
|
3002
|
-
<https://www.mindspore.cn/docs/en/
|
|
3002
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3003
3003
|
|
|
3004
3004
|
Note:
|
|
3005
3005
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3106,7 +3106,7 @@ class PhotoTourDataset(MappableDataset, VisionBaseDataset):
|
|
|
3106
3106
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
3107
3107
|
argument can only be specified when `num_shards` is also specified.
|
|
3108
3108
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3109
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3109
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3110
3110
|
Default: ``None`` , which means no cache is used.
|
|
3111
3111
|
|
|
3112
3112
|
Raises:
|
|
@@ -3124,7 +3124,7 @@ class PhotoTourDataset(MappableDataset, VisionBaseDataset):
|
|
|
3124
3124
|
|
|
3125
3125
|
Tutorial Examples:
|
|
3126
3126
|
- `Load & Process Data With Dataset Pipeline
|
|
3127
|
-
<https://www.mindspore.cn/docs/en/
|
|
3127
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3128
3128
|
|
|
3129
3129
|
Note:
|
|
3130
3130
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3234,7 +3234,7 @@ class Places365Dataset(MappableDataset, VisionBaseDataset):
|
|
|
3234
3234
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
3235
3235
|
argument can only be specified when `num_shards` is also specified.
|
|
3236
3236
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3237
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3237
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3238
3238
|
Default: ``None`` , which means no cache is used.
|
|
3239
3239
|
|
|
3240
3240
|
Raises:
|
|
@@ -3249,7 +3249,7 @@ class Places365Dataset(MappableDataset, VisionBaseDataset):
|
|
|
3249
3249
|
|
|
3250
3250
|
Tutorial Examples:
|
|
3251
3251
|
- `Load & Process Data With Dataset Pipeline
|
|
3252
|
-
<https://www.mindspore.cn/docs/en/
|
|
3252
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3253
3253
|
|
|
3254
3254
|
Note:
|
|
3255
3255
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3356,7 +3356,7 @@ class QMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
3356
3356
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
3357
3357
|
argument can only be specified when `num_shards` is also specified.
|
|
3358
3358
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3359
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3359
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3360
3360
|
Default: ``None`` , which means no cache is used.
|
|
3361
3361
|
|
|
3362
3362
|
Raises:
|
|
@@ -3370,7 +3370,7 @@ class QMnistDataset(MappableDataset, VisionBaseDataset):
|
|
|
3370
3370
|
|
|
3371
3371
|
Tutorial Examples:
|
|
3372
3372
|
- `Load & Process Data With Dataset Pipeline
|
|
3373
|
-
<https://www.mindspore.cn/docs/en/
|
|
3373
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3374
3374
|
|
|
3375
3375
|
Note:
|
|
3376
3376
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3454,7 +3454,7 @@ class RandomDataset(SourceDataset, VisionBaseDataset):
|
|
|
3454
3454
|
Default: ``None`` , will use global default workers(8), it can be set
|
|
3455
3455
|
by :func:`mindspore.dataset.config.set_num_parallel_workers` .
|
|
3456
3456
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3457
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3457
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3458
3458
|
Default: ``None`` , which means no cache is used.
|
|
3459
3459
|
shuffle (bool, optional): Whether or not to perform shuffle on the dataset.
|
|
3460
3460
|
Default: ``None`` , expected order behavior shown in the table below.
|
|
@@ -3477,7 +3477,7 @@ class RandomDataset(SourceDataset, VisionBaseDataset):
|
|
|
3477
3477
|
|
|
3478
3478
|
Tutorial Examples:
|
|
3479
3479
|
- `Load & Process Data With Dataset Pipeline
|
|
3480
|
-
<https://www.mindspore.cn/docs/en/
|
|
3480
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3481
3481
|
|
|
3482
3482
|
Examples:
|
|
3483
3483
|
>>> from mindspore import dtype as mstype
|
|
@@ -3539,7 +3539,7 @@ class RenderedSST2Dataset(MappableDataset, VisionBaseDataset):
|
|
|
3539
3539
|
shard_id (int, optional): The shard ID within `num_shards` . This
|
|
3540
3540
|
argument can only be specified when `num_shards` is also specified. Default: ``None`` .
|
|
3541
3541
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3542
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3542
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3543
3543
|
Default: ``None`` , which means no cache is used.
|
|
3544
3544
|
|
|
3545
3545
|
Raises:
|
|
@@ -3554,7 +3554,7 @@ class RenderedSST2Dataset(MappableDataset, VisionBaseDataset):
|
|
|
3554
3554
|
|
|
3555
3555
|
Tutorial Examples:
|
|
3556
3556
|
- `Load & Process Data With Dataset Pipeline
|
|
3557
|
-
<https://www.mindspore.cn/docs/en/
|
|
3557
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3558
3558
|
|
|
3559
3559
|
Note:
|
|
3560
3560
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3750,7 +3750,7 @@ class SBDataset(GeneratorDataset):
|
|
|
3750
3750
|
|
|
3751
3751
|
Tutorial Examples:
|
|
3752
3752
|
- `Load & Process Data With Dataset Pipeline
|
|
3753
|
-
<https://www.mindspore.cn/docs/en/
|
|
3753
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3754
3754
|
|
|
3755
3755
|
Note:
|
|
3756
3756
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3847,7 +3847,7 @@ class SBUDataset(MappableDataset, VisionBaseDataset):
|
|
|
3847
3847
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
3848
3848
|
argument can only be specified when `num_shards` is also specified.
|
|
3849
3849
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3850
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3850
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3851
3851
|
Default: ``None`` , which means no cache is used.
|
|
3852
3852
|
|
|
3853
3853
|
Raises:
|
|
@@ -3861,7 +3861,7 @@ class SBUDataset(MappableDataset, VisionBaseDataset):
|
|
|
3861
3861
|
|
|
3862
3862
|
Tutorial Examples:
|
|
3863
3863
|
- `Load & Process Data With Dataset Pipeline
|
|
3864
|
-
<https://www.mindspore.cn/docs/en/
|
|
3864
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3865
3865
|
|
|
3866
3866
|
Note:
|
|
3867
3867
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -3944,7 +3944,7 @@ class SemeionDataset(MappableDataset, VisionBaseDataset):
|
|
|
3944
3944
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
3945
3945
|
argument can only be specified when `num_shards` is also specified.
|
|
3946
3946
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
3947
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
3947
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
3948
3948
|
Default: ``None`` , which means no cache is used.
|
|
3949
3949
|
|
|
3950
3950
|
Raises:
|
|
@@ -3958,7 +3958,7 @@ class SemeionDataset(MappableDataset, VisionBaseDataset):
|
|
|
3958
3958
|
|
|
3959
3959
|
Tutorial Examples:
|
|
3960
3960
|
- `Load & Process Data With Dataset Pipeline
|
|
3961
|
-
<https://www.mindspore.cn/docs/en/
|
|
3961
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
3962
3962
|
|
|
3963
3963
|
Note:
|
|
3964
3964
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -4054,7 +4054,7 @@ class STL10Dataset(MappableDataset, VisionBaseDataset):
|
|
|
4054
4054
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
4055
4055
|
argument can only be specified when `num_shards` is also specified.
|
|
4056
4056
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
4057
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
4057
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
4058
4058
|
Default: ``None`` , which means no cache is used.
|
|
4059
4059
|
|
|
4060
4060
|
Raises:
|
|
@@ -4069,7 +4069,7 @@ class STL10Dataset(MappableDataset, VisionBaseDataset):
|
|
|
4069
4069
|
|
|
4070
4070
|
Tutorial Examples:
|
|
4071
4071
|
- `Load & Process Data With Dataset Pipeline
|
|
4072
|
-
<https://www.mindspore.cn/docs/en/
|
|
4072
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4073
4073
|
|
|
4074
4074
|
Note:
|
|
4075
4075
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -4167,7 +4167,7 @@ class SUN397Dataset(MappableDataset, VisionBaseDataset):
|
|
|
4167
4167
|
shard_id (int, optional): The shard ID within `num_shards` . This
|
|
4168
4168
|
argument can only be specified when `num_shards` is also specified. Default: ``None`` .
|
|
4169
4169
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
4170
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
4170
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
4171
4171
|
Default: ``None`` , which means no cache is used.
|
|
4172
4172
|
|
|
4173
4173
|
Raises:
|
|
@@ -4181,7 +4181,7 @@ class SUN397Dataset(MappableDataset, VisionBaseDataset):
|
|
|
4181
4181
|
|
|
4182
4182
|
Tutorial Examples:
|
|
4183
4183
|
- `Load & Process Data With Dataset Pipeline
|
|
4184
|
-
<https://www.mindspore.cn/docs/en/
|
|
4184
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4185
4185
|
|
|
4186
4186
|
Note:
|
|
4187
4187
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -4340,7 +4340,7 @@ class SVHNDataset(GeneratorDataset):
|
|
|
4340
4340
|
|
|
4341
4341
|
Tutorial Examples:
|
|
4342
4342
|
- `Load & Process Data With Dataset Pipeline
|
|
4343
|
-
<https://www.mindspore.cn/docs/en/
|
|
4343
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4344
4344
|
|
|
4345
4345
|
Note:
|
|
4346
4346
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -4428,7 +4428,7 @@ class USPSDataset(SourceDataset, VisionBaseDataset):
|
|
|
4428
4428
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
4429
4429
|
argument can only be specified when `num_shards` is also specified.
|
|
4430
4430
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
4431
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
4431
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
4432
4432
|
Default: ``None`` , which means no cache is used.
|
|
4433
4433
|
|
|
4434
4434
|
Raises:
|
|
@@ -4441,7 +4441,7 @@ class USPSDataset(SourceDataset, VisionBaseDataset):
|
|
|
4441
4441
|
|
|
4442
4442
|
Tutorial Examples:
|
|
4443
4443
|
- `Load & Process Data With Dataset Pipeline
|
|
4444
|
-
<https://www.mindspore.cn/docs/en/
|
|
4444
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4445
4445
|
|
|
4446
4446
|
Examples:
|
|
4447
4447
|
>>> import mindspore.dataset as ds
|
|
@@ -4536,7 +4536,7 @@ class VOCDataset(MappableDataset, VisionBaseDataset):
|
|
|
4536
4536
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` . This
|
|
4537
4537
|
argument can only be specified when `num_shards` is also specified.
|
|
4538
4538
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
4539
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
4539
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
4540
4540
|
Default: ``None`` , which means no cache is used.
|
|
4541
4541
|
extra_metadata(bool, optional): Flag to add extra meta-data to row. If True, an additional column named
|
|
4542
4542
|
:py:obj:`[_meta-filename, dtype=string]` will be output at the end. Default: ``False``.
|
|
@@ -4560,7 +4560,7 @@ class VOCDataset(MappableDataset, VisionBaseDataset):
|
|
|
4560
4560
|
|
|
4561
4561
|
Tutorial Examples:
|
|
4562
4562
|
- `Load & Process Data With Dataset Pipeline
|
|
4563
|
-
<https://www.mindspore.cn/docs/en/
|
|
4563
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4564
4564
|
|
|
4565
4565
|
Note:
|
|
4566
4566
|
- Column '[_meta-filename, dtype=string]' won't be output unless an explicit rename dataset op
|
|
@@ -4718,7 +4718,7 @@ class WIDERFaceDataset(MappableDataset, VisionBaseDataset):
|
|
|
4718
4718
|
shard_id (int, optional): The shard ID within `num_shards` . Default: ``None`` .
|
|
4719
4719
|
This argument can only be specified when `num_shards` is also specified.
|
|
4720
4720
|
cache (DatasetCache, optional): Use tensor caching service to speed up dataset processing. More details:
|
|
4721
|
-
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/
|
|
4721
|
+
`Single-Node Data Cache <https://www.mindspore.cn/tutorials/experts/en/master/dataset/cache.html>`_ .
|
|
4722
4722
|
Default: ``None`` , which means no cache is used.
|
|
4723
4723
|
|
|
4724
4724
|
Raises:
|
|
@@ -4735,7 +4735,7 @@ class WIDERFaceDataset(MappableDataset, VisionBaseDataset):
|
|
|
4735
4735
|
|
|
4736
4736
|
Tutorial Examples:
|
|
4737
4737
|
- `Load & Process Data With Dataset Pipeline
|
|
4738
|
-
<https://www.mindspore.cn/docs/en/
|
|
4738
|
+
<https://www.mindspore.cn/docs/en/master/api_python/samples/dataset/dataset_gallery.html>`_
|
|
4739
4739
|
|
|
4740
4740
|
Note:
|
|
4741
4741
|
- The parameters `num_samples` , `shuffle` , `num_shards` , `shard_id` can be used to control the sampler
|
|
@@ -25,7 +25,7 @@ Common imported modules in corresponding API examples are as follows:
|
|
|
25
25
|
import mindspore.dataset.text as text
|
|
26
26
|
|
|
27
27
|
See `Text Transforms
|
|
28
|
-
<https://www.mindspore.cn/tutorials/en/
|
|
28
|
+
<https://www.mindspore.cn/tutorials/en/master/beginner/transforms.html#text-transforms>`_ tutorial for more details.
|
|
29
29
|
|
|
30
30
|
Descriptions of common data processing terms are as follows:
|
|
31
31
|
|
|
@@ -35,10 +35,10 @@ Descriptions of common data processing terms are as follows:
|
|
|
35
35
|
The data transform operation can be executed in the data processing pipeline or in the eager mode:
|
|
36
36
|
|
|
37
37
|
- Pipeline mode is generally used to process big datasets. Examples refer to
|
|
38
|
-
`introduction to data processing pipeline <https://www.mindspore.cn/docs/en/
|
|
38
|
+
`introduction to data processing pipeline <https://www.mindspore.cn/docs/en/master/api_python/
|
|
39
39
|
mindspore.dataset.html#introduction-to-data-processing-pipeline>`_ .
|
|
40
40
|
- Eager mode is more like a function call to process data. Examples refer to
|
|
41
|
-
`Lightweight Data Processing <https://www.mindspore.cn/tutorials/en/
|
|
41
|
+
`Lightweight Data Processing <https://www.mindspore.cn/tutorials/en/master/advanced/dataset/eager.html>`_ .
|
|
42
42
|
"""
|
|
43
43
|
import platform
|
|
44
44
|
|