pyerualjetwork 5.0.3__py3-none-any.whl → 5.2__py3-none-any.whl

pyerualjetwork/neu_cpu.py

@@ -6,10 +6,27 @@ NEU (Neural Networks) on CPU
 ============================
 This module hosts functions for training and evaluating artificial neural networks on CPU for labeled classification tasks (for now).
 
-Currently, two types of models can be trained:
+Currently, 3 types of models can be trained:
 
     PLAN (Potentiation Learning Artificial Neural Network)
+        * Training Time for Small Projects: fast
+        * Training Time for Big Projects: fast
+        * Explainability: high
+        * Learning Capacity: medium (compared to single perceptrons)
+
     MLP (Multi-Layer Perceptron → Deep Learning) -- With non-bias
+        * Training Time for Small Projects: fast
+        * Training Time for Big Projects: slow
+        * Explainability: low
+        * Learning Capacity: high
+
+    PTNN (Potentiation Transfer Neural Network) -- With non-bias
+        * Training Time for Small Projects: fast
+        * Training Time for Big Projects: fast
+        * Explainability: low
+        * Learning Capacity: high
+
+Read learn function docstring for know how to use of these model architectures.
 
 For more information about PLAN: https://github.com/HCB06/PyerualJetwork/blob/main/Welcome_to_PLAN/PLAN.pdf
 
@@ -32,6 +49,7 @@ PyerualJetwork document: https://github.com/HCB06/PyerualJetwork/blob/main/Welco
 
 import numpy as np
 import copy
+import random
 
 ### LIBRARY IMPORTS ###
 from .ui import loading_bars, initialize_loading_bar
@@ -98,23 +116,26 @@ def plan_fit(
     return normalization(weight, dtype=dtype)
 
 
-def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, pop_size=None,
+def learn(x_train, y_train, optimizer, gen, pop_size, fit_start=True, batch_size=1,
            weight_evolve=True, neural_web_history=False, show_current_activations=False, auto_normalization=False,
            neurons_history=False, early_stop=False, show_history=False, target_loss=None,
            interval=33.33, target_acc=None, loss='categorical_crossentropy', acc_impact=0.9, loss_impact=0.1,
            start_this_act=None, start_this_W=None, neurons=[], activation_functions=[], dtype=np.float32):
     """
     Optimizes the activation functions for a neural network by leveraging train data to find 
-    the most accurate combination of activation potentiation(or activation function) & weight values for the given dataset.
+    the most accurate combination of activation potentiation(or activation function) & weight values for the labeled classificaiton dataset.
 
     Why genetic optimization ENE(Eugenic NeuroEvolution) and not backpropagation?
     Because PLAN is different from other neural network architectures. In PLAN, the learnable parameters are not the weights; instead, the learnable parameters are the activation functions.
     Since activation functions are not differentiable, we cannot use gradient descent or backpropagation. However, I developed a more powerful genetic optimization algorithm: ENE.
 
+    * This function also able to train classic MLP model architectures.
+    * And my newest innovative architecture: PTNN (Potentiation Transfer Neural Network).
+
     :Args:
     :param x_train: (array-like): Training input data.
     :param y_train: (array-like): Labels for training data. one-hot encoded.
-    :param optimizer: (function): Optimization technique with hyperparameters. (PLAN and MLP (both) using ENE for optimization.) Please use this: from pyerualjetwork.ene import evolver (and) optimizer = lambda *args, **kwargs: evolver(*args, 'here give your neat hyperparameters for example:  activation_add_prob=0.85', **kwargs) Example:
+    :param optimizer: (function): Optimization technique with hyperparameters. (PLAN, MLP & PTNN (all) using ENE for optimization. Gradient based technique's will added in the future.) Please use this: from pyerualjetwork.ene_cpu import evolver (and) optimizer = lambda *args, **kwargs: evolver(*args, 'here give your hyperparameters for example:  activation_add_prob=0.85', **kwargs) Example:
     ```python
              optimizer = lambda *args, **kwargs: ene_cpu.evolver(*args,
                                                             activation_add_prob=0.05,
@@ -131,11 +152,11 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                             batch_size=0.05,
                             interval=16.67)
     ```
-    :param fit_start: (bool, optional): If the fit_start parameter is set to True, the initial generation population undergoes a simple short training process using the PLAN algorithm. This allows for a very robust starting point, especially for large and complex datasets. However, for small or relatively simple datasets, it may result in unnecessary computational overhead. When fit_start is True, completing the first generation may take slightly longer (this increase in computational cost applies only to the first generation and does not affect subsequent generations). If fit_start is set to False, the initial population will be entirely random. Options: True or False. Default: True
-    :param gen: (int, optional): The generation count for genetic optimization.
+    :param fit_start: (bool, optional): If the fit_start parameter is set to True, the initial generation population undergoes a simple short training process using the PLAN algorithm. This allows for a very robust starting point, especially for large and complex datasets. However, for small or relatively simple datasets, it may result in unnecessary computational overhead. When fit_start is True, completing the first generation may take slightly longer (this increase in computational cost applies only to the first generation and does not affect subsequent generations). If fit_start is set to False, the initial population will be entirely random. Additonaly if you want to train PTNN model you must be give True. Options: True or False. Default: True
+    :param gen: (int or list): The generation count for genetic optimization. If you want to train PTNN model you must give a list of two number. First number for PLAN model training second number for MLP.
     :param batch_size: (float, optional): Batch size is used in the prediction process to receive train feedback by dividing the train data into chunks and selecting activations based on randomly chosen partitions. This process reduces computational cost and time while still covering the entire train set due to random selection, so it doesn't significantly impact accuracy. For example, a batch size of 0.08 means each train batch represents %8 of the train set. Default is 1. (%100 of train)
-    :param pop_size: (int, optional): Population size of each generation. Default: count of activation functions
-    :param weight_evolve: (bool, optional): Activation combinations already optimizes by PLANEAT genetic search algorithm. Should the weight parameters also evolve or should the weights be determined according to the aggregating learning principle of the PLAN algorithm? Default: True (Evolves Weights)
+    :param pop_size: (int): Population size of each generation.
+    :param weight_evolve: (bool, optional): Activation combinations already optimizes by ENE genetic search algorithm. Should the weight parameters also evolve or should the weights be determined according to the aggregating learning principle of the PLAN algorithm? Default: True (Evolves Weights)
     :param neural_web_history: (bool, optional): Draws history of neural web. Default is False. [ONLY FOR PLAN MODELS]
     :param show_current_activations: (bool, optional): Should it display the activations selected according to the current strategies during learning, or not? (True or False) This can be very useful if you want to cancel the learning process and resume from where you left off later. After canceling, you will need to view the live training activations in order to choose the activations to be given to the 'start_this' parameter. Default is False
     :param auto_normalization: (bool, optional): Normalization may solves overflow problem. Default: False
@@ -150,8 +171,8 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
     :param loss_impact: (float, optional): Impact of loss for optimization [0-1]. Default: 0.1
     :param start_this_act: (list, optional): To resume a previously canceled or interrupted training from where it left off, or to continue from that point with a different strategy, provide the list of activation functions selected up to the learned portion to this parameter. Default is None
     :param start_this_W: (numpy.array, optional): To resume a previously canceled or interrupted training from where it left off, or to continue from that point with a different strategy, provide the weight matrix of this genome. Default is None
-    :param neurons: (list[int], optional): If you dont want train PLAN model this parameter represents neuron count of each hidden layer for MLP. Number of elements --> Layer count. Default: [] (No hidden layer) --> architecture setted to PLAN, if not --> architecture setted to MLP.
-    :param activation_functions: (list[str], optional): If you dont want train PLAN model this parameter represents activation function of each hidden layer for MLP. if neurons is not [] --> uses default: ['linear'] * len(neurons). if neurons is [] --> uses [].
+    :param neurons: (list[int], optional): If you dont want train PLAN model this parameter represents neuron count of each hidden layer for MLP or PTNN. Number of elements --> Layer count. Default: [] (No hidden layer) --> architecture setted to PLAN, if not --> architecture setted to MLP.
+    :param activation_functions: (list[str], optional): If you dont want train PLAN model this parameter represents activation function of each hidden layer for MLP or PTNN. if neurons is not [] --> uses default: ['linear'] * len(neurons). if neurons is [] --> uses [].
     :param dtype: (numpy.dtype): Data type for the Weight matrices. np.float32 by default. Example: np.float64 or np.float16.
 
     Returns:
@@ -165,28 +186,58 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
     except_this = ['spiral', 'circular']
     activations = [item for item in all_activations() if item not in except_this]
     activations_len = len(activations)
-   
+    
     # Pre-checks
 
-    if pop_size is None: pop_size = activations_len
+    if pop_size > activations_len and fit_start is True:
+        for _ in range(pop_size - len(activations)):
+            random_index_all_act = random.randint(0, len(activations)-1)
+            activations.append(activations[random_index_all_act])
+
     y_train = optimize_labels(y_train, cuda=False)
 
     if pop_size < activations_len: raise ValueError(f"pop_size must be higher or equal to {activations_len}")
 
-    if gen is None: gen = activations_len
-
     if target_acc is not None and (target_acc < 0 or target_acc > 1): raise ValueError('target_acc must be in range 0 and 1')
     if fit_start is not True and fit_start is not False: raise ValueError('fit_start parameter only be True or False. Please read doc-string')
 
     if neurons != []:
-        fit_start = False
         weight_evolve = True
-        is_mlp = True
+        
         if activation_functions == []: activation_functions = ['linear'] * len(neurons)
-        activations = activation_functions
+        
+        if fit_start is False:
+            # MLP
+            activations = activation_functions
+            model_type = 'MLP'
+            activation_potentiations = [0] * pop_size
+            activation_potentiation = None
+            is_mlp = True
+            transfer_learning = False
+        else:
+            # PTNN
+            model_type = 'PLAN' # First generation index gen[0] is PLAN, other index gen[1] it will change to PTNN (PLAN Connects to MLP and will transfer the learned information).
+            transfer_learning = True
+
+            neurons_copy = neurons.copy()
+            neurons = []
+            gen_copy = gen.copy()
+            gen = gen[0] + gen[1]
+            activation_potentiations = [0] * pop_size
+            activation_potentiation = None
+            is_mlp = False # it will change
+        
     else:
+        # PLAN
+        model_type = 'PLAN'
+        transfer_learning = False
+
+        activation_potentiations = [0] * pop_size # NOTE: For PLAN models, activation_potentiations is needed BUT activations variable already mirros activation_potentiation values.
+                                      # So, we don't need to use activation_potentiations variable. activation_potentiations variable is only for PTNN models.
+        activation_potentiation = None
         is_mlp = False
 
+
     # Initialize visualization components
     viz_objects = initialize_visualization_for_learner(show_history, neurons_history, neural_web_history, x_train, y_train)
 
@@ -201,12 +252,12 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
 
     progress = initialize_loading_bar(total=activations_len, desc="", ncols=77, bar_format=bar_format_learner)
 
-    if fit_start is False or pop_size > activations_len:
-        weight_pop, act_pop = define_genomes(input_shape=len(x_train[0]), output_shape=len(y_train[0]), neurons=neurons, activation_functions=activation_functions, population_size=pop_size, dtype=dtype)
+    if fit_start is False:
+        weight_pop, act_pop = define_genomes(input_shape=len(x_train[0]), output_shape=len(y_train[0]), neurons=neurons, activation_functions=activations, population_size=pop_size, dtype=dtype)
 
     else:
-        weight_pop = [0] * pop_size
-        act_pop = [0] * pop_size
+        weight_pop = [0] * len(activations)
+        act_pop = [0] * len(activations)
 
     if start_this_act is not None and start_this_W is not None:
         weight_pop[0] = start_this_W
@@ -214,6 +265,41 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
 
     # LEARNING STARTED
     for i in range(gen):
+
+        # TRANSFORMATION PLAN TO MLP FOR PTNN (in later generations)
+        if model_type == 'PLAN' and transfer_learning:
+            if i == gen_copy[0]:
+
+                model_type = 'PTNN'
+                neurons = neurons_copy
+
+                for individual in range(len(weight_pop)):
+                    weight_pop[individual] = np.copy(best_weight)
+                    activation_potentiations[individual] = final_activations.copy() if isinstance(final_activations, list) else final_activations
+
+                activation_potentiation = activation_potentiations[0]
+
+                neurons_copy = [len(y_train[0])] + neurons_copy
+                activation_functions = ['linear'] + activation_functions
+                
+                weight_pop, act_pop = define_genomes(input_shape=len(x_train[0]), output_shape=len(y_train[0]), neurons=neurons_copy, activation_functions=activation_functions, population_size=pop_size, dtype=dtype)
+ 
+                # 0 indexed individual will keep PLAN's learned informations and in later generations it will share other individuals.
+                for l in range(1, len(weight_pop[0])):
+                    original_shape = weight_pop[0][l].shape
+
+                    identity_matrix = np.eye(original_shape[0], original_shape[1], dtype=weight_pop[0][l].dtype)
+                    weight_pop[0][l] = identity_matrix
+                    
+                for l in range(len(weight_pop)):
+                    weight_pop[l][0] = np.copy(best_weight)
+
+                best_weight = np.array(weight_pop[0], dtype=object)
+                final_activations = act_pop[0]
+                is_mlp = True
+                fit_start = False
+
+
         postfix_dict["Gen"] = str(i+1) + '/' + str(gen)
         progress.set_postfix(postfix_dict)
 
@@ -225,18 +311,21 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
 
             x_train_batch, y_train_batch = batcher(x_train, y_train, batch_size=batch_size)
             
-            if fit_start is True and i == 0 and j < activations_len:
+            if fit_start is True and i == 0:
                 if start_this_act is not None and j == 0:
                     pass
                 else:
+           
                     act_pop[j] = activations[j]
                     W = plan_fit(x_train_batch, y_train_batch, activations=act_pop[j], auto_normalization=auto_normalization, dtype=dtype)
                     weight_pop[j] = W
-            
+
+
             if weight_evolve is False:
                 weight_pop[j] = plan_fit(x_train_batch, y_train_batch, activations=act_pop[j], auto_normalization=auto_normalization, dtype=dtype)
 
-            model = evaluate(x_train_batch, y_train_batch, W=weight_pop[j], activations=act_pop[j], auto_normalization=auto_normalization, is_mlp=is_mlp)
+
+            model = evaluate(x_train_batch, y_train_batch, W=weight_pop[j], activations=act_pop[j], activation_potentiations=activation_potentiations[j], auto_normalization=auto_normalization, model_type=model_type)
             acc = model[get_acc()]
             
             if loss == 'categorical_crossentropy':
@@ -246,7 +335,7 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                 train_loss = binary_crossentropy(y_true_batch=y_train_batch, 
                                             y_pred_batch=model[get_preds_softmax()])
 
-            fitness  = wals(acc, train_loss, acc_impact, loss_impact)
+            fitness = wals(acc, train_loss, acc_impact, loss_impact)
             target_pop.append(fitness)
 
             if fitness >= best_fitness:
@@ -254,11 +343,11 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                 best_fitness = fitness
                 best_acc = acc
                 best_loss = train_loss
-                best_weight = np.copy(weight_pop[j]) if is_mlp is False else copy.deepcopy(weight_pop[j])
+                best_weight = np.copy(weight_pop[j]) if model_type == 'PLAN' else copy.deepcopy(weight_pop[j])
                 best_model = model
 
                 final_activations = act_pop[j].copy() if isinstance(act_pop[j], list) else act_pop[j]
-                if is_mlp is False: final_activations = [final_activations[0]] if len(set(final_activations)) == 1 else final_activations # removing if all same
+                if model_type == 'PLAN': final_activations = [final_activations[0]] if len(set(final_activations)) == 1 else final_activations # removing if all same
 
                 if batch_size == 1:
                     postfix_dict[f"{data} Accuracy"] = np.round(best_acc, 4)
@@ -293,7 +382,7 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                 if target_acc is not None and best_acc >= target_acc:
                     progress.close()
                     train_model = evaluate(x_train, y_train, W=best_weight, 
-                                        activations=final_activations, auto_normalization=auto_normalization, is_mlp=is_mlp)
+                                        activations=final_activations, activation_potentiations=activation_potentiation, auto_normalization=auto_normalization, model_type=model_type)
                     if loss == 'categorical_crossentropy':
                         train_loss = categorical_crossentropy(y_true_batch=y_train, 
                                                            y_pred_batch=train_model[get_preds_softmax()])
@@ -302,18 +391,20 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                                                        y_pred_batch=train_model[get_preds_softmax()])
 
                     print('\nActivations: ', final_activations)
-                    print(f'Train Accuracy:', train_model[get_acc()])
-                    print(f'Train Loss: ', train_loss, '\n')
+                    print('Activation Potentiation: ', activation_potentiation)
+                    print('Train Accuracy:', train_model[get_acc()])
+                    print('Train Loss: ', train_loss, '\n')
+                    print('Model Type:', model_type)
                     
                     display_visualizations_for_learner(viz_objects, best_weight, data, best_acc, 
                                               best_loss, y_train, interval)
-                    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations
+                    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations, None, None, None, None, None, None, None, activation_potentiation
             
                 # Check target loss
                 if target_loss is not None and best_loss <= target_loss:
                     progress.close()
                     train_model = evaluate(x_train, y_train, W=best_weight,
-                                        activations=final_activations, auto_normalization=auto_normalization, is_mlp=is_mlp)
+                                        activations=final_activations, activation_potentiations=activation_potentiation, auto_normalization=auto_normalization, model_type=model_type)
 
                     if loss == 'categorical_crossentropy':
                         train_loss = categorical_crossentropy(y_true_batch=y_train, 
@@ -323,19 +414,21 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                                                        y_pred_batch=train_model[get_preds_softmax()])
 
                     print('\nActivations: ', final_activations)
-                    print(f'Train Accuracy :', train_model[get_acc()])
-                    print(f'Train Loss : ', train_loss, '\n')
+                    print('Activation Potentiation: ', activation_potentiation)
+                    print('Train Accuracy:', train_model[get_acc()])
+                    print('Train Loss: ', train_loss, '\n')
+                    print('Model Type:', model_type)
 
                     # Display final visualizations
                     display_visualizations_for_learner(viz_objects, best_weight, data, best_acc, 
                                               train_loss, y_train, interval)
-                    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations
+                    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations, None, None, None, None, None, None, None, activation_potentiation
 
             
             progress.update(1)
 
         if batch_size != 1:
-            train_model = evaluate(x_train, y_train, best_weight, final_activations, auto_normalization=auto_normalization, is_mlp=is_mlp)
+            train_model = evaluate(x_train, y_train, W=best_weight, activations=final_activations, activation_potentiations=activation_potentiation, auto_normalization=auto_normalization, model_type=model_type)
     
             if loss == 'categorical_crossentropy':
                 train_loss = categorical_crossentropy(y_true_batch=y_train, 
@@ -355,7 +448,7 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
             best_acc_per_gen_list.append(best_acc)
             loss_list.append(best_loss)
 
-        if is_mlp is False: weight_pop = np.array(weight_pop, copy=False, dtype=dtype)
+        if model_type == 'PLAN': weight_pop = np.array(weight_pop, copy=False, dtype=dtype)
         else: weight_pop = np.array(weight_pop, copy=False, dtype=object)
 
         weight_pop, act_pop = optimizer(weight_pop, act_pop, i, np.array(target_pop, dtype=dtype, copy=False), weight_evolve=weight_evolve, is_mlp=is_mlp, bar_status=False)
@@ -366,7 +459,7 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
             if best_acc_per_gen_list[i] == best_acc_per_gen_list[i-1]:
                 progress.close()
                 train_model = evaluate(x_train, y_train, W=best_weight, 
-                                    activations=final_activations, auto_normalization=auto_normalization, is_mlp=is_mlp)
+                                    activations=final_activations, activation_potentiations=activation_potentiation, auto_normalization=auto_normalization, model_type=model_type)
                 
             if loss == 'categorical_crossentropy':
                 train_loss = categorical_crossentropy(y_true_batch=y_train, 
@@ -376,19 +469,21 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                                                 y_pred_batch=train_model[get_preds_softmax()])
 
             print('\nActivations: ', final_activations)
-            print(f'Train Accuracy:', train_model[get_acc()])
-            print(f'Train Loss: ', train_loss, '\n')
+            print('Activation Potentiation: ', activation_potentiation)
+            print('Train Accuracy:', train_model[get_acc()])
+            print('Train Loss: ', train_loss, '\n')
+            print('Model Type:', model_type)
 
             # Display final visualizations
             display_visualizations_for_learner(viz_objects, best_weight, data, best_acc, 
                                         train_loss, y_train, interval)
-            return best_weight, best_model[get_preds_softmax()], best_acc, final_activations
+            return best_weight, best_model[get_preds_softmax()], best_acc, final_activations, None, None, None, None, None, None, None, activation_potentiation
 
     # Final evaluation
     progress.close()
 
     train_model = evaluate(x_train, y_train, W=best_weight,
-                        activations=final_activations, auto_normalization=auto_normalization, is_mlp=is_mlp)
+                        activations=final_activations, activation_potentiations=activation_potentiation, auto_normalization=auto_normalization, model_type=model_type)
 
     if loss == 'categorical_crossentropy':
         train_loss = categorical_crossentropy(y_true_batch=y_train, 
@@ -398,21 +493,24 @@ def learn(x_train, y_train, optimizer, fit_start=True, gen=None, batch_size=1, p
                                         y_pred_batch=train_model[get_preds_softmax()])
         
     print('\nActivations: ', final_activations)
-    print(f'Train Accuracy:', train_model[get_acc()])
-    print(f'Train Loss: ', train_loss, '\n')
+    print('Activation Potentiation: ', activation_potentiation)
+    print('Train Accuracy:', train_model[get_acc()])
+    print('Train Loss: ', train_loss, '\n')
+    print('Model Type:', model_type)
 
     # Display final visualizations
     display_visualizations_for_learner(viz_objects, best_weight, data, best_acc, train_loss, y_train, interval)
-    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations
+    return best_weight, best_model[get_preds_softmax()], best_acc, final_activations, None, None, None, None, None, None, None, activation_potentiation
 
 
 def evaluate(
     x_test,
     y_test,
+    model_type,
     W,
     activations=['linear'],
-    auto_normalization=False,
-    is_mlp=False
+    activation_potentiations=[],
+    auto_normalization=False
 ) -> tuple:
     """
     Evaluates the neural network model using the given test data.
@@ -422,13 +520,15 @@ def evaluate(
 
         y_test (np.ndarray): Test labels (one-hot encoded).
 
+        model_type: (str): Type of the model. Options: 'PLAN', 'MLP', 'PTNN'.
+
         W (np.ndarray): Neural net weight matrix.
         
-        activations (list, optional): Activation list. Default = ['linear'].
+        activations (list, optional): Activation list for PLAN or MLP models (MLP layers activations if it PTNN model). Default = ['linear'].
+
+        activation_potentiations (list, optional): Extra activation potentiation list (PLAN layers activations) for PTNN models. Default = [].
 
         auto_normalization (bool, optional): Normalization for x_test ? Default = False.
-        
-        is_mlp (bool, optional): Evaluate PLAN model or MLP model ? Default: False (PLAN)
 
     Returns:
         tuple: Model (list).
@@ -441,7 +541,7 @@ def evaluate(
     elif isinstance(activations, list):
         activations = [item if isinstance(item, list) or isinstance(item, str) else [item] for item in activations]
 
-    if is_mlp:
+    if model_type == 'MLP':
         layer = x_test
         for i in range(len(W)):
             if i != len(W) - 1 and i != 0: layer = apply_activation(layer, activations[i])
@@ -450,11 +550,28 @@ def evaluate(
 
         result = layer
 
-    else:
+    if model_type == 'PLAN':
 
         x_test = apply_activation(x_test, activations)
         result = x_test @ W.T
     
+    if model_type == 'PTNN':
+
+        if isinstance(activation_potentiations, str):
+            activation_potentiations = [activation_potentiations]
+        elif isinstance(activation_potentiations, list):
+            activation_potentiations = [item if isinstance(item, list) or isinstance(item, str) else [item] for item in activation_potentiations]
+
+        x_test = apply_activation(x_test, activation_potentiations)
+        layer = x_test @ W[0].T
+
+        for i in range(1, len(W)):
+            if i != len(W) - 1: layer = apply_activation(layer, activations[i])
+
+            layer = layer @ W[i].T
+
+        result = layer
+
     max_vals = np.max(result, axis=1, keepdims=True)
     
     softmax_preds = np.exp(result - max_vals) / np.sum(np.exp(result - max_vals), axis=1, keepdims=True)