nkululeko 0.81.7__tar.gz → 0.82.1__tar.gz

{nkululeko-0.81.7 → nkululeko-0.82.1}/CHANGELOG.md

@@ -1,6 +1,14 @@
 Changelog
 =========
 
+Version 0.82.1
+--------------
+* Add more test for GC action
+
+Version 0.82.0
+--------------
+* added nkuluflag module
+
 Version 0.81.7
 --------------
 * bugfixes

{nkululeko-0.81.7/nkululeko.egg-info → nkululeko-0.82.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: nkululeko
-Version: 0.81.7
+Version: 0.82.1
 Summary: Machine learning audio prediction experiments based on templates
 Home-page: https://github.com/felixbur/nkululeko
 Author: Felix Burkhardt
@@ -42,6 +42,7 @@ Requires-Dist: umap-learn
 Requires-Dist: xgboost
 Requires-Dist: pylatex
 
+
 - [Overview](#overview)
   - [Confusion matrix](#confusion-matrix)
   - [Epoch progression](#epoch-progression)
@@ -203,6 +204,14 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
+  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
+                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+ 
+
+
+
+
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)
@@ -246,6 +255,7 @@ There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Oversample the training set](http://blog.syntheticspeech.de/2023/11/16/nkululeko-oversample-the-training-set/)  
 * [Compare several databases](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/)
 * [Tweak the target variable for database comparison](http://blog.syntheticspeech.de/2024/03/13/nkululeko-how-to-tweak-the-target-variable-for-database-comparison/)
+* [How to run multiple experiments in one go](http://blog.syntheticspeech.de/2022/03/28/how-to-run-multiple-experiments-in-one-go-with-nkululeko/)
 
 ### <a name="helloworld">Hello World example</a>
 * NEW: [Here's a Google colab that runs this example out-of-the-box](https://colab.research.google.com/drive/1GYNBd5cdZQ1QC3Jm58qoeMaJg3UuPhjw?usp=sharing#scrollTo=4G_SjuF9xeQf), and here is the same [with Kaggle](https://www.kaggle.com/felixburk/nkululeko-hello-world-example)
@@ -323,6 +333,14 @@ F. Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben and Björn Schulle
 Changelog
 =========
 
+Version 0.82.1
+--------------
+* Add more test for GC action
+
+Version 0.82.0
+--------------
+* added nkuluflag module
+
 Version 0.81.7
 --------------
 * bugfixes

{nkululeko-0.81.7 → nkululeko-0.82.1}/README.md

@@ -1,3 +1,4 @@
+
 - [Overview](#overview)
   - [Confusion matrix](#confusion-matrix)
   - [Epoch progression](#epoch-progression)
@@ -159,6 +160,14 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
+  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
+                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+ 
+
+
+
+
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)
@@ -202,6 +211,7 @@ There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Oversample the training set](http://blog.syntheticspeech.de/2023/11/16/nkululeko-oversample-the-training-set/)  
 * [Compare several databases](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/)
 * [Tweak the target variable for database comparison](http://blog.syntheticspeech.de/2024/03/13/nkululeko-how-to-tweak-the-target-variable-for-database-comparison/)
+* [How to run multiple experiments in one go](http://blog.syntheticspeech.de/2022/03/28/how-to-run-multiple-experiments-in-one-go-with-nkululeko/)
 
 ### <a name="helloworld">Hello World example</a>
 * NEW: [Here's a Google colab that runs this example out-of-the-box](https://colab.research.google.com/drive/1GYNBd5cdZQ1QC3Jm58qoeMaJg3UuPhjw?usp=sharing#scrollTo=4G_SjuF9xeQf), and here is the same [with Kaggle](https://www.kaggle.com/felixburk/nkululeko-hello-world-example)

nkululeko-0.82.1/meta/demos/multiple_exeriments/do_experiments.py

@@ -0,0 +1,35 @@
+import os
+
+
+classifiers = [
+    {"--model": "mlp", "--layers": "\"{'l1':64,'l2':16}\"", "--epochs": 100},
+    {
+        "--model": "mlp",
+        "--layers": "\"{'l1':128,'l2':64,'l3':16}\"",
+        "--learning_rate": ".01",
+        "--drop": ".3",
+        "--epochs": 100,
+    },
+    {"--model": "xgb", "--epochs": 1},
+    {"--model": "svm", "--epochs": 1},
+]
+
+features = [
+    {"--feat": "os"},
+    # {'--feat': 'os',
+    # '--set': 'ComParE_2016',
+    # },
+    {"--feat": "praat"},
+]
+
+
+for c in classifiers:
+    for f in features:
+        cmd = "python -m nkululeko.nkuluflag --config exp.ini "
+        for item in c:
+            cmd += f"{item} {c[item]} "
+        for item in f:
+            cmd += f"{item} {f[item]} "
+        print(cmd)
+        os.system(cmd)
+        # print(f"results: {result}, {last_epoch}")

{nkululeko-0.81.7 → nkululeko-0.82.1}/nkululeko/constants.py

@@ -1,2 +1,2 @@
-VERSION="0.81.7"
+VERSION="0.82.1"
 SAMPLING_RATE = 16000

{nkululeko-0.81.7 → nkululeko-0.82.1}/nkululeko/multidb.py

@@ -12,18 +12,15 @@ import numpy as np
 import pandas as pd
 import seaborn as sn
 
-import nkululeko.glob_conf as glob_conf
 from nkululeko.aug_train import doit as aug_train
-from nkululeko.experiment import Experiment
 from nkululeko.nkululeko import doit as nkulu
-from nkululeko.utils.util import Util
 
 
 def main(src_dir):
     parser = argparse.ArgumentParser(
-        description="Call the nkululeko MULTIDB framework.")
-    parser.add_argument("--config", default="exp.ini",
-                        help="The base configuration")
+        description="Call the nkululeko MULTIDB framework."
+    )
+    parser.add_argument("--config", default="exp.ini", help="The base configuration")
     args = parser.parse_args()
     if args.config is not None:
         config_file = args.config
@@ -58,8 +55,7 @@ def main(src_dir):
                 dataset = datasets[i]
                 print(f"running {dataset}")
                 if extra_trains:
-                    extra_trains_1 = extra_trains.removeprefix(
-                        "[").removesuffix("]")
+                    extra_trains_1 = extra_trains.removeprefix("[").removesuffix("]")
                     config["DATA"]["databases"] = f"['{dataset}', {extra_trains_1}]"
                     extra_trains_2 = ast.literal_eval(extra_trains)
                     for extra_train in extra_trains_2:
@@ -72,8 +68,7 @@ def main(src_dir):
                 test = datasets[j]
                 print(f"running train: {train}, test: {test}")
                 if extra_trains:
-                    extra_trains_1 = extra_trains.removeprefix(
-                        "[").removesuffix("]")
+                    extra_trains_1 = extra_trains.removeprefix("[").removesuffix("]")
                     config["DATA"][
                         "databases"
                     ] = f"['{train}', '{test}', {extra_trains_1}]"

nkululeko-0.82.1/nkululeko/nkuluflag.py

@@ -0,0 +1,95 @@
+import argparse
+import configparser
+import os
+import os.path
+
+from nkululeko.nkululeko import doit as nkulu
+
+
+def do_it(src_dir):
+    parser = argparse.ArgumentParser(description="Call the nkululeko framework.")
+    parser.add_argument("--config", help="The base configuration")
+    parser.add_argument("--data", help="The databases", nargs="*", action="append")
+    parser.add_argument(
+        "--label", nargs="*", help="The labels for the target", action="append"
+    )
+    parser.add_argument(
+        "--tuning_params", nargs="*", help="parameters to be tuned", action="append"
+    )
+    parser.add_argument(
+        "--layers",
+        nargs="*",
+        help="layer config for mlp, e.g. l1:128 ",
+        action="append",
+    )
+    parser.add_argument("--model", default="xgb", help="The model type")
+    parser.add_argument("--feat", default="['os']", help="The feature type")
+    parser.add_argument("--set", help="The opensmile set")
+    parser.add_argument("--with_os", help="To add os features")
+    parser.add_argument("--target", help="The target designation")
+    parser.add_argument("--epochs", help="The number of epochs")
+    parser.add_argument("--runs", help="The number of runs")
+    parser.add_argument("--learning_rate", help="The learning rate")
+    parser.add_argument("--drop", help="The dropout rate [0:1]")
+
+    args = parser.parse_args()
+
+    if args.config is not None:
+        config_file = args.config
+    else:
+        print("ERROR: need config file")
+        quit(-1)
+    # test if config is there
+    if not os.path.isfile(config_file):
+        print(f"ERROR: no such file {config_file}")
+
+    config = configparser.ConfigParser()
+    config.read(config_file)
+    # fill the config
+
+    if args.data is not None:
+        databases = []
+        for t in args.data:
+            databases.append(t[0])
+        print(f"got databases: {databases}")
+        config["DATA"]["databases"] = str(databases)
+    if args.label is not None:
+        labels = []
+        for label in args.label:
+            labels.append(label[0])
+        print(f"got labels: {labels}")
+        config["DATA"]["labels"] = str(labels)
+    if args.tuning_params is not None:
+        tuning_params = []
+        for tp in args.tuning_params:
+            tuning_params.append(tp[0])
+        config["MODEL"]["tuning_params"] = str(tuning_params)
+    if args.layers is not None:
+        config["MODEL"]["layers"] = args.layers[0][0]
+    if args.target is not None:
+        config["DATA"]["target"] = args.target
+    if args.epochs is not None:
+        config["EXP"]["epochs"] = args.epochs
+    if args.runs is not None:
+        config["EXP"]["runs"] = args.runs
+    if args.learning_rate is not None:
+        config["MODEL"]["learning_rate"] = args.learning_rate
+    if args.drop is not None:
+        config["MODEL"]["drop"] = args.drop
+    if args.model is not None:
+        config["MODEL"]["type"] = args.model
+    if args.feat is not None:
+        config["FEATS"]["type"] = f"['{args.feat}']"
+    if args.set is not None:
+        config["FEATS"]["set"] = args.set
+    tmp_config = "tmp.ini"
+    with open(tmp_config, "w") as tmp_file:
+        config.write(tmp_file)
+
+    result, last_epoch = nkulu(tmp_config)
+    return result, last_epoch
+
+
+if __name__ == "__main__":
+    cwd = os.path.dirname(os.path.abspath(__file__))
+    do_it(cwd)  # sys.argv[1])

{nkululeko-0.81.7 → nkululeko-0.82.1}/nkululeko/nkululeko.py

@@ -1,12 +1,14 @@
 # nkululeko.py
 # Entry script to do a Nkululeko experiment
-import numpy as np
-import os.path
-import configparser
 import argparse
+import configparser
+import os.path
+
+import numpy as np
+
+from nkululeko.constants import VERSION
 import nkululeko.experiment as exp
 from nkululeko.utils.util import Util
-from nkululeko.constants import VERSION
 
 
 def doit(config_file):

{nkululeko-0.81.7 → nkululeko-0.82.1/nkululeko.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: nkululeko
-Version: 0.81.7
+Version: 0.82.1
 Summary: Machine learning audio prediction experiments based on templates
 Home-page: https://github.com/felixbur/nkululeko
 Author: Felix Burkhardt
@@ -42,6 +42,7 @@ Requires-Dist: umap-learn
 Requires-Dist: xgboost
 Requires-Dist: pylatex
 
+
 - [Overview](#overview)
   - [Confusion matrix](#confusion-matrix)
   - [Epoch progression](#epoch-progression)
@@ -203,6 +204,14 @@ All of them take *--config <my_config.ini>* as an argument.
 * **nkululeko.predict**: [predict features](http://blog.syntheticspeech.de/2023/08/16/nkululeko-how-to-predict-labels-for-your-data-from-existing-models-and-check-them/) like SNR, MOS, arousal/valence, age/gender, with DNN models
 * **nkululeko.segment**: [segment a database](http://blog.syntheticspeech.de/2023/07/14/nkululeko-segmenting-a-database/) based on VAD (voice activity detection)
 * **nkululeko.resample**: check on all [sampling rates and change](http://blog.syntheticspeech.de/2023/08/31/how-to-fix-different-sampling-rates-in-a-dataset-with-nkululeko/) to 16kHz 
+* **nkululeko.nkuluflag**: a convenient module to specify configuration parameters on the command-line.
+  * usage: nkuluflag.py [-h] [--config CONFIG] [--data [DATA ...]] [--label [LABEL ...]] [--tuning_params [TUNING_PARAMS ...]] [--layers [LAYERS ...]] [--model MODEL] [--feat FEAT] [--set SET]
+                    [--with_os WITH_OS] [--target TARGET] [--epochs EPOCHS] [--runs RUNS] [--learning_rate LEARNING_RATE] [--drop DROP]
+ 
+
+
+
+
 
 There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Introduction](http://blog.syntheticspeech.de/2021/08/04/machine-learning-experiment-framework/)
@@ -246,6 +255,7 @@ There's my [blog](http://blog.syntheticspeech.de/?s=nkululeko) with tutorials:
 * [Oversample the training set](http://blog.syntheticspeech.de/2023/11/16/nkululeko-oversample-the-training-set/)  
 * [Compare several databases](http://blog.syntheticspeech.de/2024/01/02/nkululeko-compare-several-databases/)
 * [Tweak the target variable for database comparison](http://blog.syntheticspeech.de/2024/03/13/nkululeko-how-to-tweak-the-target-variable-for-database-comparison/)
+* [How to run multiple experiments in one go](http://blog.syntheticspeech.de/2022/03/28/how-to-run-multiple-experiments-in-one-go-with-nkululeko/)
 
 ### <a name="helloworld">Hello World example</a>
 * NEW: [Here's a Google colab that runs this example out-of-the-box](https://colab.research.google.com/drive/1GYNBd5cdZQ1QC3Jm58qoeMaJg3UuPhjw?usp=sharing#scrollTo=4G_SjuF9xeQf), and here is the same [with Kaggle](https://www.kaggle.com/felixburk/nkululeko-hello-world-example)
@@ -323,6 +333,14 @@ F. Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben and Björn Schulle
 Changelog
 =========
 
+Version 0.82.1
+--------------
+* Add more test for GC action
+
+Version 0.82.0
+--------------
+* added nkuluflag module
+
 Version 0.81.7
 --------------
 * bugfixes

{nkululeko-0.81.7 → nkululeko-0.82.1}/nkululeko.egg-info/SOURCES.txt

@@ -52,7 +52,6 @@ meta/demos/my_experiment.py
 meta/demos/my_experiment_local.py
 meta/demos/plot_faster_anim.py
 meta/demos/multiple_exeriments/do_experiments.py
-meta/demos/multiple_exeriments/parse_nkulu.py
 nkululeko/__init__.py
 nkululeko/aug_train.py
 nkululeko/augment.py
@@ -70,6 +69,7 @@ nkululeko/filter_data.py
 nkululeko/glob_conf.py
 nkululeko/modelrunner.py
 nkululeko/multidb.py
+nkululeko/nkuluflag.py
 nkululeko/nkululeko.py
 nkululeko/plots.py
 nkululeko/predict.py

nkululeko-0.81.7/meta/demos/multiple_exeriments/do_experiments.py

@@ -1,48 +0,0 @@
-import os
-
-src_path = 'demos/multiple_exeriments/'
-
-classifiers = [
-    {'--model': 'mlp',
-    '--layers': '\"{\'l1\':64,\'l2\':16}\"'},
-    {'--model': 'mlp',
-    '--layers': '\"{\'l1\':64,\'l2\':16}\"',
-    '--learning_rate': '.1',},
-    {'--model': 'mlp',
-    '--layers': '\"{\'l1\':64,\'l2\':16}\"',
-    '--learning_rate': '.0001',
-    '--drop': '.3',
-    },
-    {'--model': 'xgb',
-    '--epochs':1},
-    {'--model': 'svm',
-    '--epochs':1},
-]
-
-features = [
-    {'--feat': 'os'},
-    # {'--feat': 'os', 
-    # '--set': 'ComParE_2016',
-    # },
-    # {'--feat': 'mld'},
-    # {'--feat': 'mld',
-    # '--with_os': 'True',
-    # },
-    # {'--feat': 'xbow'},
-    # {'--feat': 'xbow',
-    # '--with_os': 'True',
-    # },
-    # {'--feat': 'trill'},
-    # {'--feat': 'wav2vec'},
-]
-
-
-for c in classifiers:
-    for f in features:
-        cmd = f'python {src_path}parse_nkulu.py '
-        for item in c:
-            cmd += f'{item} {c[item]} '
-        for item in f:
-            cmd += f'{item} {f[item]} '
-        print(cmd)
-        os.system(cmd)

nkululeko-0.81.7/meta/demos/multiple_exeriments/parse_nkulu.py

@@ -1,112 +0,0 @@
-
-import sys
-#sys.path.append("../../src")
-sys.path.append("../nkululeko/src")
-import constants
-import numpy as np
-import experiment as exp
-import configparser
-from util import Util
-import argparse
-import os.path
-src_path = './demos/multiple_experiments/'
-
-
-def main():
-    parser = argparse.ArgumentParser(description='Call the nkululeko framework.')
-    parser.add_argument('--config', default='exp.ini', help='The base configuration')
-    parser.add_argument('--data', help='The databases', nargs='*', \
-        action='append')
-    parser.add_argument('--label', nargs='*', help='The labels for the target', \
-        action='append')
-    parser.add_argument('--tuning_params', nargs='*', help='parameters to be tuned', \
-        action='append')
-    parser.add_argument('--layers', nargs='*', help='layer config for mlp, e.g. l1:128 ', \
-        action='append')
-    parser.add_argument('--model', default='xgb', help='The model type')
-    parser.add_argument('--feat', default='os', help='The feature type')
-    parser.add_argument('--set', help='The opensmile set')
-    parser.add_argument('--with_os', help='To add os features')
-    parser.add_argument('--target', help='The target designation')
-    parser.add_argument('--epochs', help='The number of epochs')
-    parser.add_argument('--runs', help='The number of runs')
-    parser.add_argument('--learning_rate', help='The learning rate')
-    parser.add_argument('--drop', help='The dropout rate [0:1]')
-    
-    args = parser.parse_args()
-
-    if args.config is not None:
-        config_file = args.config    
-    else:
-        config_file = f'{src_path}exp.ini'
-    # test if config is there
-    if not os.path.isfile(config_file):
-        print(f'ERROR: no such file {config_file}')
-
-    config = configparser.ConfigParser()
-    config.read(config_file)
-    # fill the config
-
-
-    if args.data is not None:
-        databases = []
-        for t in args.data:
-            databases.append(t[0])
-        print(f'got databases: {databases}')
-        config['DATA']['databases'] = str(databases)
-    if args.label is not None:
-        labels = []
-        for l in args.label:
-            labels.append(l[0])
-        print(f'got labels: {labels}')
-        config['DATA']['labels'] = str(labels)
-    if args.tuning_params is not None:
-        tuning_params = []
-        for tp in args.tuning_params:
-            tuning_params.append(tp[0])
-        config['MODEL']['tuning_params'] = str(tuning_params)
-    if args.layers is not None:
-        config['MODEL']['layers'] = args.layers[0][0]
-    if args.target is not None:
-        config['DATA']['target'] = args.target
-    if args.epochs is not None:
-        config['EXP']['epochs'] = args.epochs
-    if args.runs is not None:
-        config['EXP']['runs'] = args.runs
-    if args.learning_rate is not None:
-        config['MODEL']['learning_rate'] = args.learning_rate
-    if args.drop is not None:
-        config['MODEL']['drop'] = args.drop
-    if args.model is not None:
-        config['MODEL']['type'] = args.model
-    if args.feat is not None:
-        config['FEATS']['type'] = f'[\'{args.feat}\']'
-    if args.with_os is not None:
-        config['FEATS']['with_os'] = args.with_os
-    if args.set is not None:
-        config['FEATS']['set'] = args.set
-
-
-    name = config['EXP']['name']
-
-    # init the experiment
-    expr = exp.Experiment(config)
-    util = Util()
-    util.debug(f'running {name}, Nkululeko version {constants.VERSION}')
-
-    # load the data
-    expr.load_datasets()
-    # split into train and test
-    expr.fill_train_and_tests()
-    # extract features
-    expr.extract_feats()
-    # initialize a run manager
-    expr.init_runmanager()
-    # run the experiment
-    reports = expr.run()
-    result = reports[-1].result.test
-    # report result
-    util.debug(f'result for {expr.get_name()} is {result}')
-
-if __name__ == "__main__":
-    main()