bertuner 0.1.1__tar.gz → 0.2.0__tar.gz

{bertuner-0.1.1 → bertuner-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bertuner
-Version: 0.1.1
+Version: 0.2.0
 Summary: Hyperparameter optimization and fine-tuning for BERT-style text classifiers (Optuna + MLflow), with long-context ModernBERT support
 Author-email: elemets <alafunnell@gmail.com>
 License: MIT
@@ -24,6 +24,7 @@ Requires-Dist: torch>=2.0
 Requires-Dist: transformers>=4.48
 Requires-Dist: numpy>=1.24
 Requires-Dist: pandas>=2.0
+Requires-Dist: matplotlib>=3.7
 Requires-Dist: scikit-learn>=1.3
 Requires-Dist: optuna>=3.0
 Requires-Dist: mlflow>=2.9
@@ -81,6 +82,9 @@ classifier = BERTuneClassifier(
     text_feature="text_col",           # column containing the text
     target_cols=["label_col"],         # one column = single-label
     max_length=512,
+    precision="auto",                  # BF16/FP16 on supported CUDA, else FP32
+    retry_nonfinite_in_fp32=True,       # retry verified NaN/Inf failures once
+    max_grad_norm=1.0,                  # explicit gradient clipping threshold
 )
 
 # 2. Configure (optional: uses defaults if called without arguments)
@@ -105,6 +109,24 @@ Multi-label classification: pass several target columns — `target_cols=["l1",
 
 Grouped data (e.g. multiple notes per patient): pass `group_key="patient_id"` and the train/val/test split guarantees no group leaks across splits.
 
+### Numerical-stability recovery
+
+Every attempt checks logits, loss, gradients before the optimizer step, and evaluation predictions for NaN/Inf. When a mixed-precision attempt becomes non-finite, BERTuner deletes its checkpoints, reloads the pretrained model, resets the seed, and retries the same Optuna trial and hyperparameters once in FP32. A second numerical failure prunes an optimization trial; final-model training raises `NonFiniteTrainingError`. Invalid labels, class weights, OOM errors, and other exceptions never trigger the retry.
+
+`precision` accepts `"auto"`, `"fp32"`, `"bf16"`, or `"fp16"`. Explicit unsupported precision raises before training. Set `retry_nonfinite_in_fp32=False` to disable recovery, `max_grad_norm=None` to disable clipping, or `class_weight_warning_threshold=None` to disable warnings for large finite class weights. Saved `bertuner_config.json` records requested/effective precision and fallback status.
+
+### Metrics logged to MLflow
+
+Final runs log one canonical metric set for both `Validation_*` and `Test_*`:
+
+- **Binary:** accuracy, balanced accuracy, precision, recall, specificity, F1, Matthews correlation coefficient (MCC), average precision, AUROC, Brier score, and log loss.
+- **Multiclass:** accuracy, balanced accuracy, macro and weighted precision/recall/F1, MCC, macro and weighted average precision/AUROC, and log loss.
+- **Multi-label:** subset accuracy, Hamming loss/accuracy, micro/macro/sample precision/recall/F1 and Jaccard, micro MCC, micro/macro average precision and AUROC, Brier score, and log loss.
+
+The optimized binary or per-label decision threshold is logged as a parameter. Multiclass runs log `decision_rule=argmax`. Training-time `eval_*` metrics are not duplicated in MLflow; their losses remain visible in the `plots/training_vs_evaluation_loss.png` artifact.
+
+When optimizing a lower-is-better metric such as `log_loss`, pass `greater_is_better=False`; Optuna and best-checkpoint selection will both minimize it.
+
 ## Customizing the hyperparameter search
 
 Two things are configurable: **which models** are searched and **which hyperparameters** with what ranges.

{bertuner-0.1.1 → bertuner-0.2.0}/README.md

@@ -41,6 +41,9 @@ classifier = BERTuneClassifier(
     text_feature="text_col",           # column containing the text
     target_cols=["label_col"],         # one column = single-label
     max_length=512,
+    precision="auto",                  # BF16/FP16 on supported CUDA, else FP32
+    retry_nonfinite_in_fp32=True,       # retry verified NaN/Inf failures once
+    max_grad_norm=1.0,                  # explicit gradient clipping threshold
 )
 
 # 2. Configure (optional: uses defaults if called without arguments)
@@ -65,6 +68,24 @@ Multi-label classification: pass several target columns — `target_cols=["l1",
 
 Grouped data (e.g. multiple notes per patient): pass `group_key="patient_id"` and the train/val/test split guarantees no group leaks across splits.
 
+### Numerical-stability recovery
+
+Every attempt checks logits, loss, gradients before the optimizer step, and evaluation predictions for NaN/Inf. When a mixed-precision attempt becomes non-finite, BERTuner deletes its checkpoints, reloads the pretrained model, resets the seed, and retries the same Optuna trial and hyperparameters once in FP32. A second numerical failure prunes an optimization trial; final-model training raises `NonFiniteTrainingError`. Invalid labels, class weights, OOM errors, and other exceptions never trigger the retry.
+
+`precision` accepts `"auto"`, `"fp32"`, `"bf16"`, or `"fp16"`. Explicit unsupported precision raises before training. Set `retry_nonfinite_in_fp32=False` to disable recovery, `max_grad_norm=None` to disable clipping, or `class_weight_warning_threshold=None` to disable warnings for large finite class weights. Saved `bertuner_config.json` records requested/effective precision and fallback status.
+
+### Metrics logged to MLflow
+
+Final runs log one canonical metric set for both `Validation_*` and `Test_*`:
+
+- **Binary:** accuracy, balanced accuracy, precision, recall, specificity, F1, Matthews correlation coefficient (MCC), average precision, AUROC, Brier score, and log loss.
+- **Multiclass:** accuracy, balanced accuracy, macro and weighted precision/recall/F1, MCC, macro and weighted average precision/AUROC, and log loss.
+- **Multi-label:** subset accuracy, Hamming loss/accuracy, micro/macro/sample precision/recall/F1 and Jaccard, micro MCC, micro/macro average precision and AUROC, Brier score, and log loss.
+
+The optimized binary or per-label decision threshold is logged as a parameter. Multiclass runs log `decision_rule=argmax`. Training-time `eval_*` metrics are not duplicated in MLflow; their losses remain visible in the `plots/training_vs_evaluation_loss.png` artifact.
+
+When optimizing a lower-is-better metric such as `log_loss`, pass `greater_is_better=False`; Optuna and best-checkpoint selection will both minimize it.
+
 ## Customizing the hyperparameter search
 
 Two things are configurable: **which models** are searched and **which hyperparameters** with what ranges.