bertuner 0.1.2__tar.gz → 0.2.1__tar.gz

{bertuner-0.1.2 → bertuner-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: bertuner
-Version: 0.1.2
+Version: 0.2.1
 Summary: Hyperparameter optimization and fine-tuning for BERT-style text classifiers (Optuna + MLflow), with long-context ModernBERT support
 Author-email: elemets <alafunnell@gmail.com>
 License: MIT
@@ -82,6 +82,9 @@ classifier = BERTuneClassifier(
     text_feature="text_col",           # column containing the text
     target_cols=["label_col"],         # one column = single-label
     max_length=512,
+    precision="auto",                  # BF16/FP16 on supported CUDA, else FP32
+    retry_nonfinite_in_fp32=True,       # retry verified NaN/Inf failures once
+    max_grad_norm=1.0,                  # explicit gradient clipping threshold
 )
 
 # 2. Configure (optional: uses defaults if called without arguments)
@@ -106,6 +109,12 @@ Multi-label classification: pass several target columns — `target_cols=["l1",
 
 Grouped data (e.g. multiple notes per patient): pass `group_key="patient_id"` and the train/val/test split guarantees no group leaks across splits.
 
+### Numerical-stability recovery
+
+Every attempt checks logits, loss, gradients before the optimizer step, and evaluation predictions for NaN/Inf. When a mixed-precision attempt becomes non-finite, BERTuner deletes its checkpoints, reloads the pretrained model, resets the seed, and retries the same Optuna trial and hyperparameters once in FP32. A second numerical failure prunes an optimization trial; final-model training raises `NonFiniteTrainingError`. Invalid labels, class weights, OOM errors, and other exceptions never trigger the retry.
+
+`precision` accepts `"auto"`, `"fp32"`, `"bf16"`, or `"fp16"`. Explicit unsupported precision raises before training. Set `retry_nonfinite_in_fp32=False` to disable recovery, `max_grad_norm=None` to disable clipping, or `class_weight_warning_threshold=None` to disable warnings for large finite class weights. Saved `bertuner_config.json` records requested/effective precision and fallback status.
+
 ### Metrics logged to MLflow
 
 Final runs log one canonical metric set for both `Validation_*` and `Test_*`:

{bertuner-0.1.2 → bertuner-0.2.1}/README.md

@@ -41,6 +41,9 @@ classifier = BERTuneClassifier(
     text_feature="text_col",           # column containing the text
     target_cols=["label_col"],         # one column = single-label
     max_length=512,
+    precision="auto",                  # BF16/FP16 on supported CUDA, else FP32
+    retry_nonfinite_in_fp32=True,       # retry verified NaN/Inf failures once
+    max_grad_norm=1.0,                  # explicit gradient clipping threshold
 )
 
 # 2. Configure (optional: uses defaults if called without arguments)
@@ -65,6 +68,12 @@ Multi-label classification: pass several target columns — `target_cols=["l1",
 
 Grouped data (e.g. multiple notes per patient): pass `group_key="patient_id"` and the train/val/test split guarantees no group leaks across splits.
 
+### Numerical-stability recovery
+
+Every attempt checks logits, loss, gradients before the optimizer step, and evaluation predictions for NaN/Inf. When a mixed-precision attempt becomes non-finite, BERTuner deletes its checkpoints, reloads the pretrained model, resets the seed, and retries the same Optuna trial and hyperparameters once in FP32. A second numerical failure prunes an optimization trial; final-model training raises `NonFiniteTrainingError`. Invalid labels, class weights, OOM errors, and other exceptions never trigger the retry.
+
+`precision` accepts `"auto"`, `"fp32"`, `"bf16"`, or `"fp16"`. Explicit unsupported precision raises before training. Set `retry_nonfinite_in_fp32=False` to disable recovery, `max_grad_norm=None` to disable clipping, or `class_weight_warning_threshold=None` to disable warnings for large finite class weights. Saved `bertuner_config.json` records requested/effective precision and fallback status.
+
 ### Metrics logged to MLflow
 
 Final runs log one canonical metric set for both `Validation_*` and `Test_*`: