mysphinx-forge 0.2.3__tar.gz → 0.3.0__tar.gz

{mysphinx_forge-0.2.3/mysphinx_forge.egg-info → mysphinx_forge-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mysphinx-forge
-Version: 0.2.3
+Version: 0.3.0
 Summary: Data and model workflow toolkit for cleaning, clustering, generation, and evaluation
 Keywords: data-cleaning,deduplication,clustering,nlp,cli
 Classifier: Development Status :: 3 - Alpha
@@ -27,6 +27,10 @@ Requires-Dist: sentence-transformers>=5.1.0; extra == "embeddings"
 Provides-Extra: llm-local
 Requires-Dist: torch>=2.8.0; extra == "llm-local"
 Requires-Dist: transformers>=4.55.0; extra == "llm-local"
+Provides-Extra: train
+Requires-Dist: torch>=2.8.0; extra == "train"
+Requires-Dist: transformers>=4.55.0; extra == "train"
+Requires-Dist: scikit-learn>=1.7.0; extra == "train"
 Provides-Extra: all
 Requires-Dist: numpy>=2.2.0; extra == "all"
 Requires-Dist: scikit-learn>=1.7.0; extra == "all"
@@ -66,6 +70,7 @@ Requires-Dist: transformers>=4.55.0; extra == "all"
 | `split` | 切分 train / valid / test | `*_split_train.*` 等 |
 | `model-test` | 批量执行模型推理或单条烟雾测试，含预期结果列时自动输出评估报告 | `*_model_tested.*` 或终端输出 |
 | `convert-sft` | 转换表格数据为 SFT 数据（`alpaca` / `pa` 格式） | `*_alpaca.json` / `*_pa.jsonl`（超 10000 条自动切分） |
+| `train` | 微调本地预训练分类基座（BERT 等）为意图分类模型，含验证集时自动输出评估报告 | `output_models/<基模名>-<日期>/` + `*_train_eval.csv` |
 
 ## 项目结构
 
@@ -128,6 +133,7 @@ uv sync --extra all --group dev
 - `embeddings`：语义去重
 - `ml + embeddings`：聚类
 - `llm-local`：本地模型 `model-test`
+- `train`：意图分类模型 `train`（BERT 等基座），以及分类模型的 `model-test`（`local` 模式自动识别）
 - `all`：安装全部能力
 
 ### 从源码安装当前项目
@@ -137,6 +143,7 @@ uv pip install -e .
 uv pip install -e '.[embeddings]'
 uv pip install -e '.[ml,embeddings]'
 uv pip install -e '.[llm-local]'
+uv pip install -e '.[train]'
 uv pip install -e '.[all]'
 ```
 
@@ -654,6 +661,38 @@ F1,0.75000,0.75000,0.75000,0.75000,8
 
 本地批量测试会按可见 GPU 数自动分配 worker；没有 GPU 时自动退化为单 worker CPU 模式。
 
+### 训练意图分类模型
+
+`train` 基于一个本地预训练分类基座（默认 BERT，也兼容其它带 `*ForSequenceClassification` 实现的 encoder 架构），微调出意图分类模型。输入表需要同时包含文本列（默认自动探测 `text/用户问题/客户问题/用户输入`）和标签列（默认自动探测 `category/label/intent/.../预期结果`）。需要先安装训练依赖：
+
+```bash
+uv sync --extra train --group dev
+```
+
+```bash
+# 直接在切分后的训练集上微调（默认基础模型 models/bert-base-chinese）
+mysphinx-forge --action train --input-file data/input_split_train.xlsx --base-model-path models/bert-base-chinese
+```
+
+验证集的选取优先级：`--valid-file` 指定的独立文件 > Excel 中的 `valid` 注入表 > 按 `--validation-ratio` 从训练集随机切分。存在验证集时会输出评估报告：
+
+```bash
+mysphinx-forge --action train --input-file data/input_split_train.xlsx \
+  --valid-file data/input_split_valid.xlsx \
+  --label-column category --num-train-epochs 3 --learning-rate 2e-5 --train-batch-size 16
+```
+
+常用训练超参：`--num-train-epochs`、`--learning-rate`、`--train-batch-size`、`--max-length`、`--weight-decay`、`--warmup-ratio`、`--train-seed`。
+
+默认情况下，训练产物会写到当前工作目录下的 `output_models/`，目录按「基模名-日期」命名（例如 `output_models/bert-base-chinese-20260624`）；同一天重复训练会自动追加序号（`-2`、`-3`……，首次不带序号）。可用 `--output-dir` 显式指定模型目录路径覆盖该默认行为。
+
+模型目录是一个标准的 transformers 模型目录（含权重、tokenizer 和 `label_map.json`），可直接用 `model-test`（`local` 模式）在测试集上复跑评估——检测到目录内的 `label_map.json` 时会自动按文本分类模型推理，无需额外指定：
+
+```bash
+mysphinx-forge --action model-test --input-file data/input_split_test.xlsx \
+  --test-model-path output_models/bert-base-chinese-20260624
+```
+
 ## 输出文件规则
 
 | Action | 主要输出 | 附加输出 |
@@ -665,6 +704,7 @@ F1,0.75000,0.75000,0.75000,0.75000,8
 | `split` | `*_split_train.*`、`*_split_valid.*`、`*_split_test.*` | `*_split.meta.json`、`mysphinx-forge.log` |
 | `model-test` 文件模式 | `*_model_tested.*` | 含 `预期结果` 列时额外生成 `*_model_tested_eval.csv`；同时写 `mysphinx-forge.log` |
 | `model-test` 单条模式 | 终端输出 | 当前工作目录下的 `mysphinx-forge.log` |
+| `train` | `output_models/<基模名>-<日期>/`（模型目录，可用 `--output-dir` 覆盖） | 含验证集时额外生成同级 `<模型目录名>_train_eval.csv`；同时写同级 `<模型目录名>.meta.json` 与 `output_models/mysphinx-forge.log` |
 
 补充说明：
 

mysphinx_forge-0.2.3/PKG-INFO → mysphinx_forge-0.3.0/README.md

@@ -1,40 +1,3 @@
-Metadata-Version: 2.4
-Name: mysphinx-forge
-Version: 0.2.3
-Summary: Data and model workflow toolkit for cleaning, clustering, generation, and evaluation
-Keywords: data-cleaning,deduplication,clustering,nlp,cli
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Environment :: Console
-Requires-Python: >=3.12
-Description-Content-Type: text/markdown
-Requires-Dist: pandas>=2.2.3
-Requires-Dist: tqdm>=4.67.1
-Requires-Dist: openpyxl>=3.1.5
-Requires-Dist: xlrd>=2.0.1
-Requires-Dist: pyyaml>=6.0.0
-Provides-Extra: ml
-Requires-Dist: numpy>=2.2.0; extra == "ml"
-Requires-Dist: scikit-learn>=1.7.0; extra == "ml"
-Provides-Extra: embeddings
-Requires-Dist: faiss-cpu>=1.11.0; extra == "embeddings"
-Requires-Dist: sentence-transformers>=5.1.0; extra == "embeddings"
-Provides-Extra: llm-local
-Requires-Dist: torch>=2.8.0; extra == "llm-local"
-Requires-Dist: transformers>=4.55.0; extra == "llm-local"
-Provides-Extra: all
-Requires-Dist: numpy>=2.2.0; extra == "all"
-Requires-Dist: scikit-learn>=1.7.0; extra == "all"
-Requires-Dist: faiss-cpu>=1.11.0; extra == "all"
-Requires-Dist: sentence-transformers>=5.1.0; extra == "all"
-Requires-Dist: torch>=2.8.0; extra == "all"
-Requires-Dist: transformers>=4.55.0; extra == "all"
-
 # MySphinx Forge
 
 `MySphinx Forge` 是一个面向表格文本数据的 Python CLI 工具，重点解决语料预处理和模型验证这两类工作：
@@ -66,6 +29,7 @@ Requires-Dist: transformers>=4.55.0; extra == "all"
 | `split` | 切分 train / valid / test | `*_split_train.*` 等 |
 | `model-test` | 批量执行模型推理或单条烟雾测试，含预期结果列时自动输出评估报告 | `*_model_tested.*` 或终端输出 |
 | `convert-sft` | 转换表格数据为 SFT 数据（`alpaca` / `pa` 格式） | `*_alpaca.json` / `*_pa.jsonl`（超 10000 条自动切分） |
+| `train` | 微调本地预训练分类基座（BERT 等）为意图分类模型，含验证集时自动输出评估报告 | `output_models/<基模名>-<日期>/` + `*_train_eval.csv` |
 
 ## 项目结构
 
@@ -128,6 +92,7 @@ uv sync --extra all --group dev
 - `embeddings`：语义去重
 - `ml + embeddings`：聚类
 - `llm-local`：本地模型 `model-test`
+- `train`：意图分类模型 `train`（BERT 等基座），以及分类模型的 `model-test`（`local` 模式自动识别）
 - `all`：安装全部能力
 
 ### 从源码安装当前项目
@@ -137,6 +102,7 @@ uv pip install -e .
 uv pip install -e '.[embeddings]'
 uv pip install -e '.[ml,embeddings]'
 uv pip install -e '.[llm-local]'
+uv pip install -e '.[train]'
 uv pip install -e '.[all]'
 ```
 
@@ -654,6 +620,38 @@ F1,0.75000,0.75000,0.75000,0.75000,8
 
 本地批量测试会按可见 GPU 数自动分配 worker；没有 GPU 时自动退化为单 worker CPU 模式。
 
+### 训练意图分类模型
+
+`train` 基于一个本地预训练分类基座（默认 BERT，也兼容其它带 `*ForSequenceClassification` 实现的 encoder 架构），微调出意图分类模型。输入表需要同时包含文本列（默认自动探测 `text/用户问题/客户问题/用户输入`）和标签列（默认自动探测 `category/label/intent/.../预期结果`）。需要先安装训练依赖：
+
+```bash
+uv sync --extra train --group dev
+```
+
+```bash
+# 直接在切分后的训练集上微调（默认基础模型 models/bert-base-chinese）
+mysphinx-forge --action train --input-file data/input_split_train.xlsx --base-model-path models/bert-base-chinese
+```
+
+验证集的选取优先级：`--valid-file` 指定的独立文件 > Excel 中的 `valid` 注入表 > 按 `--validation-ratio` 从训练集随机切分。存在验证集时会输出评估报告：
+
+```bash
+mysphinx-forge --action train --input-file data/input_split_train.xlsx \
+  --valid-file data/input_split_valid.xlsx \
+  --label-column category --num-train-epochs 3 --learning-rate 2e-5 --train-batch-size 16
+```
+
+常用训练超参：`--num-train-epochs`、`--learning-rate`、`--train-batch-size`、`--max-length`、`--weight-decay`、`--warmup-ratio`、`--train-seed`。
+
+默认情况下，训练产物会写到当前工作目录下的 `output_models/`，目录按「基模名-日期」命名（例如 `output_models/bert-base-chinese-20260624`）；同一天重复训练会自动追加序号（`-2`、`-3`……，首次不带序号）。可用 `--output-dir` 显式指定模型目录路径覆盖该默认行为。
+
+模型目录是一个标准的 transformers 模型目录（含权重、tokenizer 和 `label_map.json`），可直接用 `model-test`（`local` 模式）在测试集上复跑评估——检测到目录内的 `label_map.json` 时会自动按文本分类模型推理，无需额外指定：
+
+```bash
+mysphinx-forge --action model-test --input-file data/input_split_test.xlsx \
+  --test-model-path output_models/bert-base-chinese-20260624
+```
+
 ## 输出文件规则
 
 | Action | 主要输出 | 附加输出 |
@@ -665,6 +663,7 @@ F1,0.75000,0.75000,0.75000,0.75000,8
 | `split` | `*_split_train.*`、`*_split_valid.*`、`*_split_test.*` | `*_split.meta.json`、`mysphinx-forge.log` |
 | `model-test` 文件模式 | `*_model_tested.*` | 含 `预期结果` 列时额外生成 `*_model_tested_eval.csv`；同时写 `mysphinx-forge.log` |
 | `model-test` 单条模式 | 终端输出 | 当前工作目录下的 `mysphinx-forge.log` |
+| `train` | `output_models/<基模名>-<日期>/`（模型目录，可用 `--output-dir` 覆盖） | 含验证集时额外生成同级 `<模型目录名>_train_eval.csv`；同时写同级 `<模型目录名>.meta.json` 与 `output_models/mysphinx-forge.log` |
 
 补充说明：
 

{mysphinx_forge-0.2.3 → mysphinx_forge-0.3.0}/mysphinx_forge/cli.py

@@ -56,6 +56,12 @@ from mysphinx_forge.model_testing import (
     model_test_dataframe,
     run_model_test,
 )
+from mysphinx_forge.model_training import (
+    ClassifierTrainingConfig,
+    DEFAULT_BASE_MODEL_PATH,
+    TextClassificationStats,
+    train_intent_classifier,
+)
 from mysphinx_forge.progress import ProgressBar, run_stage
 from mysphinx_forge.semantic_deduplication import (
     DEFAULT_EMBEDDING_MODEL_PATH,
@@ -87,6 +93,7 @@ _ACTION_CHOICES = [
     "model-test",
     "split",
     "convert-sft",
+    "train",
 ]
 
 
@@ -171,6 +178,24 @@ def main() -> int:
     if args.repetition_penalty <= 0:
         print("--repetition-penalty 必须是大于 0 的数值。")
         return 1
+    if args.num_train_epochs <= 0:
+        print("--num-train-epochs 必须是大于 0 的整数。")
+        return 1
+    if args.learning_rate <= 0:
+        print("--learning-rate 必须是大于 0 的数值。")
+        return 1
+    if args.train_batch_size <= 0:
+        print("--train-batch-size 必须是大于 0 的整数。")
+        return 1
+    if args.max_length <= 0:
+        print("--max-length 必须是大于 0 的整数。")
+        return 1
+    if args.weight_decay < 0:
+        print("--weight-decay 不能小于 0。")
+        return 1
+    if not 0 <= args.warmup_ratio < 1:
+        print("--warmup-ratio 必须在 0 到 1 之间，且不能等于 1。")
+        return 1
     if args.split_random_seed < 0:
         print("--split-random-seed 不能小于 0。")
         return 1
@@ -315,6 +340,23 @@ def main() -> int:
             args.sft_user_query_as_instruction,
             args.sft_max_records_per_file,
         )
+    if args.action == "train":
+        return _run_train(
+            input_file=args.input_file,
+            output_arg=args.output_dir,
+            target_column=args.target_column,
+            label_column=args.label_column,
+            valid_file=args.valid_file or None,
+            base_model_path=args.base_model_path,
+            num_train_epochs=args.num_train_epochs,
+            learning_rate=args.learning_rate,
+            train_batch_size=args.train_batch_size,
+            max_length=args.max_length,
+            weight_decay=args.weight_decay,
+            warmup_ratio=args.warmup_ratio,
+            train_seed=args.train_seed,
+            validation_ratio=args.validation_ratio,
+        )
 
     parser.print_help()
     return 1
@@ -416,6 +458,12 @@ def _build_parser(
         default=config_defaults.get("output"),
         help="输出文件路径。未指定时，默认在原文件旁生成 *_cleaned 文件。",
     )
+    parser.add_argument(
+        "--output-dir",
+        dest="output_dir",
+        default=config_defaults.get("output_dir"),
+        help="train 模型输出目录。未指定时默认写到 cwd 下的 output_models/，按「基模名-日期」命名。",
+    )
     parser.add_argument(
         "--chunk-size",
         type=int,
@@ -458,7 +506,7 @@ def _build_parser(
         "--model-test-mode",
         choices=["local", "openai", "http"],
         default=config_defaults.get("model_test_mode", DEFAULT_MODEL_TEST_MODE),
-        help="模型测试模式。local 为本地模型推理，openai 为兼容 OpenAI Chat Completions 的接口调用，http 为通用 HTTP POST 接口调用。",
+        help="模型测试模式。local 为本地模型推理，openai 为兼容 OpenAI Chat Completions 的接口调用，http 为通用 HTTP POST 接口调用。local 模式下若模型目录含 label_map.json，会自动按文本分类模型推理。",
     )
     parser.add_argument(
         "--test-model-path",
@@ -722,6 +770,64 @@ def _build_parser(
         default=config_defaults.get("sft_max_records_per_file", PA_MAX_RECORDS_PER_FILE),
         help=f"pa 格式每个 JSONL 文件最大记录数，超出时自动切分为多个文件，默认 {PA_MAX_RECORDS_PER_FILE}。",
     )
+    parser.add_argument(
+        "--base-model-path",
+        default=config_defaults.get("base_model_path", DEFAULT_BASE_MODEL_PATH),
+        help=f"train 训练使用的本地预训练分类基座模型路径（BERT 等 encoder 架构），默认 {DEFAULT_BASE_MODEL_PATH}。",
+    )
+    parser.add_argument(
+        "--label-column",
+        default=config_defaults.get("label_column", ""),
+        help="train 训练使用的标签列名。未指定时自动探测 category/label/intent/.../预期结果。",
+    )
+    parser.add_argument(
+        "--valid-file",
+        dest="valid_file",
+        default=config_defaults.get("valid_file", ""),
+        help="train 训练使用的独立验证集文件。未指定时优先使用 valid 注入表，否则按 --validation-ratio 从训练集切分。",
+    )
+    parser.add_argument(
+        "--num-train-epochs",
+        type=int,
+        default=config_defaults.get("num_train_epochs", 3),
+        help="train 训练轮数，默认 3。",
+    )
+    parser.add_argument(
+        "--learning-rate",
+        type=float,
+        default=config_defaults.get("learning_rate", 2e-5),
+        help="train 训练学习率，默认 2e-5。",
+    )
+    parser.add_argument(
+        "--train-batch-size",
+        type=int,
+        default=config_defaults.get("train_batch_size", 16),
+        help="train 单设备训练批大小，默认 16。",
+    )
+    parser.add_argument(
+        "--max-length",
+        type=int,
+        default=config_defaults.get("max_length", 128),
+        help="train 文本分词的最大长度，默认 128。",
+    )
+    parser.add_argument(
+        "--weight-decay",
+        type=float,
+        default=config_defaults.get("weight_decay", 0.01),
+        help="train 训练权重衰减系数，默认 0.01。",
+    )
+    parser.add_argument(
+        "--warmup-ratio",
+        type=float,
+        default=config_defaults.get("warmup_ratio", 0.1),
+        help="train 训练学习率预热比例，默认 0.1。",
+    )
+    parser.add_argument(
+        "--train-seed",
+        type=int,
+        default=config_defaults.get("train_seed", 42),
+        help="train 训练随机种子，默认 42。",
+    )
     return parser
 
 
@@ -1918,6 +2024,156 @@ def _run_convert_sft(
     return 0
 
 
+def _run_train(
+    input_file: str,
+    output_arg: str | None,
+    target_column: str,
+    label_column: str,
+    valid_file: str | None,
+    base_model_path: str,
+    num_train_epochs: int,
+    learning_rate: float,
+    train_batch_size: int,
+    max_length: int,
+    weight_decay: float,
+    warmup_ratio: float,
+    train_seed: int,
+    validation_ratio: float,
+) -> int:
+    input_path = Path(input_file)
+    output_dir = _resolve_train_output_dir(output_arg, base_model_path)
+    # 评估报告与模型目录同级，保证整套训练产物自成一体地落在 output_models/ 下。
+    eval_csv_path = output_dir.with_name(f"{output_dir.name}_train_eval.csv")
+    logger = configure_logger(_resolve_log_path(output_dir))
+    logger.info(
+        "开始执行 action=train input=%s output=%s base_model=%s",
+        input_path,
+        output_dir,
+        base_model_path,
+    )
+
+    try:
+        run_stage("读取文件", logger=logger)
+        main_df, _train_inject_df, valid_inject_df, _test_inject_df = load_split_dataframes(input_file)
+        valid_explicit_df = load_dataframe(valid_file) if valid_file else None
+        train_df, valid_df = _resolve_train_valid_frames(
+            main_df=main_df,
+            valid_inject_df=valid_inject_df,
+            valid_explicit_df=valid_explicit_df,
+            validation_ratio=validation_ratio,
+            seed=train_seed,
+        )
+    except ValueError as exc:
+        _emit_error(str(exc), logger)
+        close_logger()
+        return 1
+
+    config = ClassifierTrainingConfig(
+        base_model_path=base_model_path,
+        num_train_epochs=num_train_epochs,
+        learning_rate=learning_rate,
+        per_device_batch_size=train_batch_size,
+        max_length=max_length,
+        weight_decay=weight_decay,
+        warmup_ratio=warmup_ratio,
+        seed=train_seed,
+    )
+
+    total_steps = num_train_epochs * math.ceil(max(len(train_df), 1) / train_batch_size)
+    progress_bar = ProgressBar(total=total_steps, description="训练模型", logger=logger)
+    try:
+        stats, eval_report = train_intent_classifier(
+            train_df,
+            valid_df,
+            config=config,
+            target_column=target_column,
+            label_column=label_column,
+            model_output_dir=output_dir,
+            progress_callback=progress_bar.advance,
+        )
+    except ValueError as exc:
+        progress_bar.close()
+        _emit_error(str(exc), logger)
+        close_logger()
+        return 1
+    except Exception as exc:
+        progress_bar.close()
+        logger.exception("执行模型训练失败")
+        _emit_error(f"执行模型训练失败：{type(exc).__name__}: {exc}", logger)
+        close_logger()
+        return 1
+    finally:
+        progress_bar.close()
+
+    run_stage("写出结果", logger=logger)
+    if eval_report is not None:
+        eval_csv_path.write_text(eval_report.format_csv(), encoding="utf-8")
+    _write_meta(
+        output_path=output_dir,
+        action="train",
+        input_path=input_path,
+        parameters={
+            "target_column": stats.target_column,
+            "label_column": stats.label_column,
+            "valid_file": valid_file,
+            "base_model_path": base_model_path,
+            "num_train_epochs": num_train_epochs,
+            "learning_rate": learning_rate,
+            "train_batch_size": train_batch_size,
+            "max_length": max_length,
+            "weight_decay": weight_decay,
+            "warmup_ratio": warmup_ratio,
+            "train_seed": train_seed,
+            "validation_ratio": validation_ratio,
+        },
+        training_stats=stats,
+        extra_output_files={"eval_file": eval_csv_path} if eval_report is not None else None,
+    )
+    _print_training_stats(stats, eval_report, output_dir, eval_csv_path if eval_report else None, logger)
+    close_logger()
+    return 0
+
+
+def _resolve_train_valid_frames(
+    *,
+    main_df: pd.DataFrame,
+    valid_inject_df: pd.DataFrame | None,
+    valid_explicit_df: pd.DataFrame | None,
+    validation_ratio: float,
+    seed: int,
+) -> tuple[pd.DataFrame, pd.DataFrame | None]:
+    if valid_explicit_df is not None and not valid_explicit_df.empty:
+        return main_df, valid_explicit_df
+    if valid_inject_df is not None and not valid_inject_df.empty:
+        return main_df, valid_inject_df
+    if validation_ratio > 0 and len(main_df) >= 2:
+        valid_df = main_df.sample(frac=validation_ratio, random_state=seed)
+        if not valid_df.empty:
+            train_df = main_df.drop(valid_df.index)
+            return train_df, valid_df
+    return main_df, None
+
+
+DEFAULT_TRAIN_OUTPUT_ROOT = "output_models"
+
+
+def _resolve_train_output_dir(output_arg: str | None, base_model_path: str) -> Path:
+    if output_arg:
+        return Path(output_arg)
+    # 默认产物落在 cwd 下的 output_models/，按「基模名-日期」命名；
+    # 同一天重复训练时追加序号 -2、-3……（首次不带序号）。
+    base_name = Path(base_model_path).name or "model"
+    date_str = datetime.now().strftime("%Y%m%d")
+    root = Path(DEFAULT_TRAIN_OUTPUT_ROOT)
+    candidate = root / f"{base_name}-{date_str}"
+    if not candidate.exists():
+        return candidate
+    index = 2
+    while (root / f"{base_name}-{date_str}-{index}").exists():
+        index += 1
+    return root / f"{base_name}-{date_str}-{index}"
+
+
 def _resolve_output_path(input_path: Path, output_arg: str | None) -> Path:
     if output_arg:
         return Path(output_arg)
@@ -1987,6 +2243,10 @@ def _resolve_cluster_report_html_output_path(output_path: Path) -> Path:
 
 
 def _resolve_meta_output_path(output_path: Path) -> Path:
+    # 目录型产物（如 train 的模型目录）用完整名派生，避免 .stem 把
+    # 含点的名字（如 Qwen2.5-0.5B-20260624）从最后一个点处截断。
+    if output_path.is_dir():
+        return output_path.with_name(f"{output_path.name}.meta.json")
     return output_path.with_name(f"{output_path.stem}.meta.json")
 
 
@@ -2026,6 +2286,39 @@ def _print_deduplication_stats(
     _emit_message(f"去重后总行数：{stats.total_after}", logger)
 
 
+def _print_training_stats(
+    stats: TextClassificationStats,
+    eval_report: EvalReport | None,
+    output_dir: Path,
+    eval_csv_path: Path | None,
+    logger: Logger,
+) -> None:
+    _emit_message(f"模型训练完成，输出目录：{output_dir}", logger)
+    _emit_message(f"基础模型：{stats.base_model_path}", logger)
+    _emit_message(f"使用目标列：{stats.target_column}", logger)
+    _emit_message(f"使用标签列：{stats.label_column}", logger)
+    _emit_message(f"标签数量：{stats.num_labels}", logger)
+    _emit_message(f"标签列表：{', '.join(stats.label_names)}", logger)
+    _emit_message(f"训练样本数：{stats.train_rows}", logger)
+    _emit_message(f"验证样本数：{stats.valid_rows}", logger)
+    _emit_message(f"训练设备：{stats.device}", logger)
+    _emit_message(
+        "训练参数："
+        f"epochs={stats.num_train_epochs}, "
+        f"batch_size={stats.per_device_batch_size}, "
+        f"learning_rate={stats.learning_rate}, "
+        f"max_length={stats.max_length}",
+        logger,
+    )
+    _emit_message(f"最终训练损失：{stats.final_train_loss}", logger)
+    if stats.best_metric is not None:
+        _emit_message(f"验证集 Macro F1：{stats.best_metric}", logger)
+    if eval_report is not None:
+        _emit_message(eval_report.format_summary(), logger)
+    if eval_csv_path is not None:
+        _emit_message(f"评估报告（CSV）：{eval_csv_path}", logger)
+
+
 def _print_clustering_stats(
     stats: ClusteringStats,
     output_path: Path,
@@ -2179,6 +2472,7 @@ def _write_meta(
     clustering_stats: ClusteringStats | None = None,
     split_stats: SplitStats | None = None,
     sft_conversion_stats: SftConversionStats | None = None,
+    training_stats: TextClassificationStats | None = None,
     match_output_path: Path | None = None,
     cluster_summary_path: Path | None = None,
     projection_path: Path | None = None,
@@ -2259,6 +2553,24 @@ def _write_meta(
             "skipped_blank_output_rows": sft_conversion_stats.skipped_blank_output_rows,
             "skipped_rows": sft_conversion_stats.skipped_rows,
         }
+    if training_stats is not None:
+        meta["training_stats"] = {
+            "base_model_path": training_stats.base_model_path,
+            "model_output_dir": training_stats.model_output_dir,
+            "target_column": training_stats.target_column,
+            "label_column": training_stats.label_column,
+            "num_labels": training_stats.num_labels,
+            "label_names": training_stats.label_names,
+            "train_rows": training_stats.train_rows,
+            "valid_rows": training_stats.valid_rows,
+            "num_train_epochs": training_stats.num_train_epochs,
+            "per_device_batch_size": training_stats.per_device_batch_size,
+            "learning_rate": training_stats.learning_rate,
+            "max_length": training_stats.max_length,
+            "device": training_stats.device,
+            "final_train_loss": training_stats.final_train_loss,
+            "best_metric": training_stats.best_metric,
+        }
     if match_output_path is not None and match_output_path.exists():
         meta["match_file"] = str(match_output_path)
     if cluster_summary_path is not None and cluster_summary_path.exists():

{mysphinx_forge-0.2.3 → mysphinx_forge-0.3.0}/mysphinx_forge/config.py

@@ -22,6 +22,8 @@ _PATH_LIKE_KEYS = {
     "embedding_model_path",
     "train_model_path",
     "test_model_path",
+    "base_model_path",
+    "valid_file",
     "system_prompt_file",
     "sft_system_prompt_file",
 }

{mysphinx_forge-0.2.3 → mysphinx_forge-0.3.0}/mysphinx_forge/model_testing.py

@@ -37,6 +37,7 @@ EXPECTED_RESULT_COLUMN = "预期结果"
 MATCH_EXPECTED_COLUMN = "匹配预期"
 MODEL_CALL_TIME_COLUMN = "模型调用时间"
 DEFAULT_MODEL_TEST_MODE = "local"
+DEFAULT_CLASSIFICATION_MAX_LENGTH = 128
 DEFAULT_MODEL_TEST_API_BASE_URL = "https://api.openai.com/v1"
 DEFAULT_HTTP_API_KEY_ENV_VAR = "HTTP_API_KEY"
 DEFAULT_HTTP_API_KEY_HEADER = "api_key"
@@ -496,7 +497,19 @@ def model_test_dataframe(
     expected_results = (
         dataframe[expected_result_column].tolist() if has_expected_result else [None] * len(dataframe)
     )
-    if runtime_config.mode in {"openai", "http"}:
+    effective_mode = runtime_config.mode
+    if effective_mode == "local" and _is_classification_model_dir(model_path):
+        effective_mode = "classification"
+
+    if effective_mode == "classification":
+        model_results, model_call_times, device_used = _run_classification_batches(
+            prompts=prompts,
+            model_path=model_path,
+            runtime_config=runtime_config,
+            progress_callback=progress_callback,
+        )
+        worker_count = 1
+    elif effective_mode in {"openai", "http"}:
         model_results, model_call_times, device_used = _run_openai_batches(
             prompts=prompts,
             model_path=model_path,
@@ -597,6 +610,43 @@ def _run_openai_batches(
     return model_results, model_call_times, tester.device
 
 
+def _is_classification_model_dir(model_path: str | Path) -> bool:
+    # 延迟导入以避免与 model_training -> model_eval -> model_testing 形成循环依赖。
+    from mysphinx_forge.model_training import is_classification_model_dir
+
+    candidate = Path(model_path)
+    return candidate.is_dir() and is_classification_model_dir(candidate)
+
+
+def _run_classification_batches(
+    prompts: list[object],
+    model_path: str | Path,
+    runtime_config: ModelTestRuntimeConfig,
+    progress_callback: Callable[[int], None] | None = None,
+) -> tuple[list[str], list[float], str]:
+    from mysphinx_forge.model_training import _resolve_training_device, predict_intent
+
+    try:
+        import torch
+    except ImportError as exc:
+        raise ValueError("未安装分类推理所需依赖，请先执行 uv sync --extra train。") from exc
+
+    device = _resolve_training_device(torch)
+    started_at = time.perf_counter()
+    model_results = predict_intent(
+        model_path,
+        list(prompts),
+        batch_size=runtime_config.batch_size,
+        max_length=DEFAULT_CLASSIFICATION_MAX_LENGTH,
+        device=device,
+        progress_callback=progress_callback,
+    )
+    elapsed_seconds = time.perf_counter() - started_at
+    per_row_seconds = round(elapsed_seconds / len(model_results), 4) if model_results else 0.0
+    model_call_times = [per_row_seconds] * len(model_results)
+    return model_results, model_call_times, device
+
+
 def _build_model_tester(
     model_path: str | Path,
     runtime_config: ModelTestRuntimeConfig,