nkululeko 0.88.12__tar.gz → 0.89.0__tar.gz

{nkululeko-0.88.12 → nkululeko-0.89.0}/CHANGELOG.md

@@ -1,8 +1,13 @@
 Changelog
 =========
 
-Version 0.88.12
+Version 0.89.0
 --------------
+* added Roc plots and classification report on Debug
+
+
+Version 0.88.12
+---------------
 * added n_jobs for sklearn processing
 * re_named num_workers n_jobs
 
@@ -475,9 +480,9 @@ Version 0.66.3
 
 Version 0.66.2
 --------------
-* enabled data-pathes with quotes 
+* enabled data-pacthes with quotes 
 * enabled missing category labels
-* used tgdm for progress display
+* used tqdm for progress display
 
 Version 0.66.1
 --------------

{nkululeko-0.88.12/nkululeko.egg-info → nkululeko-0.89.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: nkululeko
-Version: 0.88.12
+Version: 0.89.0
 Summary: Machine learning audio prediction experiments based on templates
 Home-page: https://github.com/felixbur/nkululeko
 Author: Felix Burkhardt
@@ -204,7 +204,7 @@ All of them take *--config <my_config.ini>* as an argument.
   * *configurations*: which experiments to combine
   * *--method* (optional): majority_voting, mean (default), max, sum, uncertainty, uncertainty_weighted, confidence_weighted, performance_weighted  
   * *--threshold*: uncertainty threshold (1.0 means no threshold)
-  * *--weightes*: weights for performance_weighted method (could be from previous UAR, ACC)
+  * *--weights*: weights for performance_weighted method (could be from previous UAR, ACC)
   * *--outfile* (optional): name of CSV file for output (default: ensemble_result.csv)
   * *--no_labels* (optional): indicate that no ground truth is given 
 * **nkululeko.multidb**: do [multiple experiments](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/), comparing several databases cross and in itself
@@ -220,14 +220,11 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
-* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
-  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
-                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
- 
-
-
-
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line. Usage:
 
+  ```bash
+  $ python -m nkululeko.nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET] [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+  ```
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)
@@ -359,8 +356,13 @@ F. Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben and Björn Schulle
 Changelog
 =========
 
-Version 0.88.12
+Version 0.89.0
 --------------
+* added Roc plots and classification report on Debug
+
+
+Version 0.88.12
+---------------
 * added n_jobs for sklearn processing
 * re_named num_workers n_jobs
 
@@ -833,9 +835,9 @@ Version 0.66.3
 
 Version 0.66.2
 --------------
-* enabled data-pathes with quotes 
+* enabled data-pacthes with quotes 
 * enabled missing category labels
-* used tgdm for progress display
+* used tqdm for progress display
 
 Version 0.66.1
 --------------

{nkululeko-0.88.12 → nkululeko-0.89.0}/README.md

@@ -160,7 +160,7 @@ All of them take *--config <my_config.ini>* as an argument.
   * *configurations*: which experiments to combine
   * *--method* (optional): majority_voting, mean (default), max, sum, uncertainty, uncertainty_weighted, confidence_weighted, performance_weighted  
   * *--threshold*: uncertainty threshold (1.0 means no threshold)
-  * *--weightes*: weights for performance_weighted method (could be from previous UAR, ACC)
+  * *--weights*: weights for performance_weighted method (could be from previous UAR, ACC)
   * *--outfile* (optional): name of CSV file for output (default: ensemble_result.csv)
   * *--no_labels* (optional): indicate that no ground truth is given 
 * **nkululeko.multidb**: do [multiple experiments](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/), comparing several databases cross and in itself
@@ -176,14 +176,11 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
-* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
-  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
-                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
- 
-
-
-
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line. Usage:
 
+  ```bash
+  $ python -m nkululeko.nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET] [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+  ```
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)

{nkululeko-0.88.12 → nkululeko-0.89.0}/nkululeko/constants.py

@@ -1,2 +1,2 @@
-VERSION="0.88.12"
+VERSION="0.89.0"
 SAMPLING_RATE = 16000

{nkululeko-0.88.12 → nkululeko-0.89.0}/nkululeko/models/model_tree.py

@@ -12,4 +12,6 @@ class Tree_model(Model):
     def __init__(self, df_train, df_test, feats_train, feats_test):
         super().__init__(df_train, df_test, feats_train, feats_test)
         self.name = "tree"
-        self.clf = DecisionTreeClassifier()  # set up the classifier
+        self.clf = DecisionTreeClassifier(
+            random_state=42
+        )  # set up the classifier

{nkululeko-0.88.12 → nkululeko-0.89.0}/nkululeko/reporting/reporter.py

@@ -27,6 +27,7 @@ from sklearn.metrics import (
     r2_score,
     roc_auc_score,
     roc_curve,
+    RocCurveDisplay,
 )
 
 import nkululeko.glob_conf as glob_conf
@@ -75,6 +76,7 @@ class Reporter:
         self.result = Result(0, 0, 0, 0, "unknown")
         self.run = run
         self.epoch = epoch
+        self.model_type = self.util.get_model_type()
         self._set_metric()
         self.filenameadd = ""
         self.cont_to_cat = False
@@ -387,6 +389,7 @@ class Reporter:
             epoch = self.epoch
         """Print all evaluation values to text file."""
         res_dir = self.util.get_path("res_dir")
+        fig_dir = self.util.get_path("fig_dir")
         file_name = f"{res_dir}{self.util.get_exp_name()}_{epoch}{self.filenameadd}.txt"
         if self.util.exp_is_classification():
             labels = glob_conf.labels
@@ -397,6 +400,10 @@ class Reporter:
                     target_names=labels,
                     output_dict=True,
                 )
+                # print classifcation report in console
+                self.util.debug(
+                    f"\n {classification_report(self.truths, self.preds, target_names=labels)}"
+                )
             except ValueError as e:
                 self.util.debug(
                     "Reporter: caught a ValueError when trying to get"
@@ -415,6 +422,17 @@ class Reporter:
                 if len(np.unique(self.truths)) == 2:
                     fpr, tpr, _ = roc_curve(self.truths, self.preds)
                     auc_score = auc(fpr, tpr)
+                    display = RocCurveDisplay(
+                        fpr=fpr,
+                        tpr=tpr,
+                        roc_auc=auc_score,
+                        estimator_name=f"{self.model_type} estimator",
+                    )
+                    # save plot
+                    plot_path = f"{fig_dir}{self.util.get_exp_name()}_{epoch}{self.filenameadd}_roc.{self.format}"
+                    display.plot(ax=None)
+                    plt.savefig(plot_path)
+                    self.util.debug(f"Saved ROC curve to {plot_path}")
                     pauc_score = roc_auc_score(self.truths, self.preds, max_fpr=0.1)
                     auc_pauc = f"auc: {auc_score:.3f}, pauc: {pauc_score:.3f} from epoch: {epoch}"
                     self.util.debug(auc_pauc)

{nkululeko-0.88.12 → nkululeko-0.89.0}/nkululeko/utils/util.py

@@ -175,7 +175,7 @@ class Util:
         """Get the experiment directory."""
         root = os.path.join(self.config["EXP"]["root"], "")
         name = self.config["EXP"]["name"]
-        dir_name = f"{root}{name}"
+        dir_name = f"{root}/{name}"
         audeer.mkdir(dir_name)
         return dir_name
 

{nkululeko-0.88.12 → nkululeko-0.89.0/nkululeko.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: nkululeko
-Version: 0.88.12
+Version: 0.89.0
 Summary: Machine learning audio prediction experiments based on templates
 Home-page: https://github.com/felixbur/nkululeko
 Author: Felix Burkhardt
@@ -204,7 +204,7 @@ All of them take *--config <my_config.ini>* as an argument.
   * *configurations*: which experiments to combine
   * *--method* (optional): majority_voting, mean (default), max, sum, uncertainty, uncertainty_weighted, confidence_weighted, performance_weighted  
   * *--threshold*: uncertainty threshold (1.0 means no threshold)
-  * *--weightes*: weights for performance_weighted method (could be from previous UAR, ACC)
+  * *--weights*: weights for performance_weighted method (could be from previous UAR, ACC)
   * *--outfile* (optional): name of CSV file for output (default: ensemble_result.csv)
   * *--no_labels* (optional): indicate that no ground truth is given 
 * **nkululeko.multidb**: do [multiple experiments](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/), comparing several databases cross and in itself
@@ -220,14 +220,11 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
-* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
-  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
-                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
- 
-
-
-
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line. Usage:
 
+  ```bash
+  $ python -m nkululeko.nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET] [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+  ```
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)
@@ -359,8 +356,13 @@ F. Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben and Björn Schulle
 Changelog
 =========
 
-Version 0.88.12
+Version 0.89.0
 --------------
+* added Roc plots and classification report on Debug
+
+
+Version 0.88.12
+---------------
 * added n_jobs for sklearn processing
 * re_named num_workers n_jobs
 
@@ -833,9 +835,9 @@ Version 0.66.3
 
 Version 0.66.2
 --------------
-* enabled data-pathes with quotes 
+* enabled data-pacthes with quotes 
 * enabled missing category labels
-* used tgdm for progress display
+* used tqdm for progress display
 
 Version 0.66.1
 --------------