congrads 0.2.0__py3-none-any.whl → 0.3.0__py3-none-any.whl

congrads-0.3.0.dist-info/METADATA

@@ -0,0 +1,234 @@
+Metadata-Version: 2.3
+Name: congrads
+Version: 0.3.0
+Summary: A toolbox for using Constraint Guided Gradient Descent when training neural networks.
+Author: Wout Rombouts, Quinten Van Baelen, Peter Karsmakers
+Author-email: Wout Rombouts <wout.rombouts@kuleuven.be>, Quinten Van Baelen <quinten.vanbaelen@kuleuven.be>, Peter Karsmakers <peter.karsmakers@kuleuven.be>
+License: Copyright 2024 DTAI - KU Leuven
+         
+         Redistribution and use in source and binary forms, with or without modification, 
+         are permitted provided that the following conditions are met:
+         
+         1. Redistributions of source code must retain the above copyright notice, 
+         this list of conditions and the following disclaimer.
+         
+         2. Redistributions in binary form must reproduce the above copyright notice, 
+         this list of conditions and the following disclaimer in the documentation 
+         and/or other materials provided with the distribution.
+         
+         3. Neither the name of the copyright holder nor the names of its 
+         contributors may be used to endorse or promote products derived from 
+         this software without specific prior written permission.
+         
+         THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” 
+         AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
+         IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
+         ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE 
+         LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 
+         DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
+         SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 
+         CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 
+         OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 
+         OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=1.5.0
+Requires-Dist: torch>=2.0.0
+Requires-Dist: torchvision>=0.15.1
+Requires-Dist: tqdm>=4.65.0
+Requires-Dist: matplotlib>=3.7.0 ; extra == 'examples'
+Requires-Dist: tensorboard>=2.18.0 ; extra == 'examples'
+Requires-Python: >=3.11
+Provides-Extra: examples
+Description-Content-Type: text/markdown
+
+<div align="center">
+	<img src="https://github.com/ML-KULeuven/congrads/blob/main/docs/_static/congrads_export.png?raw=true" height="200">
+	<p>
+	<b>Incorporate constraints into neural network training for more reliable and robust models.</b>
+	</p>
+	<br/>
+
+[![PyPi](https://img.shields.io/pypi/v/congrads.svg)](https://pypi.org/project/congrads)
+[![Read the Docs](https://img.shields.io/readthedocs/congrads/latest.svg?label=Read%20the%20Docs)](https://congrads.readthedocs.io)
+[![Python Version: 3.11+](https://img.shields.io/badge/Python-3.11+-blue.svg)](https://pypi.org/project/congrads)
+[![Downloads](https://img.shields.io/pypi/dm/congrads.svg)](https://pypistats.org/packages/congrads)
+[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
+
+<br/>
+<br/>
+</div>
+
+**Congrads** is a Python toolbox that brings **constraint-guided gradient descent** capabilities to your machine learning projects. Built with seamless integration into PyTorch, Congrads empowers you to enhance the training and optimization process by incorporating constraints into your training pipeline.
+
+Whether you're working with simple inequality constraints, combinations of input-output relations, or custom constraint formulations, Congrads provides the tools and flexibility needed to build more robust and generalized models.
+
+
+> **Notice:** All previous `v1.x` releases are **yanked**.  
+> The library is still in active development, and backwards compatibility is not guaranteed.  
+> Please use the new `v0.x` series for ongoing updates.
+
+## Key Features
+
+- **Constraint-Guided Training**: Add constraints to guide the optimization process, ensuring that your model generalizes better by trying to satisfy the constraints.
+- **Flexible Constraint Definition**: Define constraints on inputs, outputs, or combinations thereof, using an intuitive and extendable interface. Make use of pre-programmed constraint classes or write your own.
+- **Seamless PyTorch Integration**: Use Congrads within your existing PyTorch workflows with minimal setup.
+- **Flexible and extendible**: Write your own custom networks, constraints and dataset classes to easily extend the functionality of the toolbox.
+
+## Getting Started
+
+### 1. **Installation**
+
+First, make sure to install PyTorch since Congrads heavily relies on its deep learning framework. Please refer to the [PyTorch's getting started guide](https://pytorch.org/get-started/locally/). Make sure to install with CUDA support for GPU training.
+
+Next, install the Congrads toolbox. The recommended way to install it is to use pip:
+
+```bash
+pip install congrads
+```
+
+You can also install Congrads together with extra packages required to run the examples:
+
+```bash
+pip install congrads[examples]
+```
+
+This should automatically install all required dependencies for you. If you would like to install dependencies manually, Congrads depends on the following:
+
+- Python 3.11 - 3.13
+- **PyTorch** (install with CUDA support for GPU training, refer to [PyTorch's getting started guide](https://pytorch.org/get-started/locally/))
+- **NumPy** (install with `pip install numpy`, or refer to [NumPy's install guide](https://numpy.org/install/).)
+- **Pandas** (install with `pip install pandas`, or refer to [Panda's install guide](https://pandas.pydata.org/docs/getting_started/install.html).)
+- **Tqdm** (install with `pip install tqdm`)
+- **Torchvision** (install with `pip install torchvision`)
+- Optional: **Tensorboard** (install with `pip install tensorboard`)
+
+### 2. **Core concepts**
+
+Before diving into the toolbox, it is recommended to familiarize yourself with Congrads's core concept and topics.
+Please read the documentation at https://congrads.readthedocs.io/en/latest/ to get up-to-date.
+
+### 3. **Basic Usage**
+
+Below, a basic example can be found that illustrates how to work with the Congrads toolbox.
+For additional examples, refer to the [examples](https://github.com/ML-KULeuven/congrads/tree/main/examples) and [notebooks](https://github.com/ML-KULeuven/congrads/tree/main/notebooks) folders in the repository.
+
+#### 1. First, select the device to run your code on with.
+
+```python
+use_cuda = torch.cuda.is_available()
+device = torch.device("cuda:0" if use_cuda else "cpu")
+```
+
+#### 2. Next, load your data and split it into training, validation and testing subsets.
+
+```python
+data = BiasCorrection(
+    "./datasets", preprocess_BiasCorrection, download=True
+)
+loaders = split_data_loaders(
+    data,
+    loader_args={"batch_size": 100, "shuffle": True},
+    valid_loader_args={"shuffle": False},
+    test_loader_args={"shuffle": False},
+)
+```
+
+#### 3. Instantiate your neural network, make sure the dimensions match up with your data.
+
+```python
+network = MLPNetwork(25, 2, n_hidden_layers=3, hidden_dim=35)
+network = network.to(device)
+```
+
+#### 4. Choose your loss function and optimizer.
+
+```python
+criterion = MSELoss()
+optimizer = Adam(network.parameters(), lr=0.001)
+```
+
+#### 5. Then, setup the descriptor, that will attach names to specific parts of your network.
+
+```python
+descriptor = Descriptor()
+descriptor.add("output", 0, "Tmax")
+descriptor.add("output", 1, "Tmin")
+```
+
+#### 6. Define your constraints on the network.
+
+```python
+Constraint.descriptor = descriptor
+Constraint.device = device
+constraints = [
+    ScalarConstraint("Tmin", ge, 0),
+    ScalarConstraint("Tmin", le, 1),
+    ScalarConstraint("Tmax", ge, 0),
+    ScalarConstraint("Tmax", le, 1),
+    BinaryConstraint("Tmax", gt, "Tmin"),
+]
+```
+
+#### 7. Instantiate metric manager and core, and start the training.
+
+```python
+metric_manager = MetricManager()
+core = CongradsCore(
+    descriptor,
+    constraints,
+    loaders,
+    network,
+    criterion,
+    optimizer,
+    metric_manager,
+    device,
+    checkpoint_manager,
+)
+
+core.fit(max_epochs=50)
+```
+
+## Example Use Cases
+
+- **Optimization with Domain Knowledge**: Ensure outputs meet real-world restrictions or safety standards.
+- **Improve Training Process**: Inject domain knowledge in the training stage, increasing learning efficiency.
+- **Physics-Informed Neural Networks (PINNs)**: Coming soon, Enforce physical laws as constraints in your models.
+
+## Planned changes / Roadmap
+
+- [ ] Add ODE/PDE constraints to support PINNs
+- [x] Rework callback system
+- [ ] Add support for constraint parser that can interpret equations
+
+## Research
+
+If you make use of this package or it's concepts in your research, please consider citing the following papers.
+
+- Van Baelen, Q., & Karsmakers, P. (2023). **Constraint guided gradient descent: Training with inequality constraints with applications in regression and semantic segmentation.**
+  Neurocomputing, 556, 126636. doi:10.1016/j.neucom.2023.126636 <br/>[ [pdf](https://www.sciencedirect.com/science/article/abs/pii/S0925231223007592) | [bibtex](https://raw.githubusercontent.com/ML-KULeuven/congrads/main/docs/_static/VanBaelen2023.bib) ]
+
+## Contributing
+
+We welcome contributions to Congrads! Whether you want to report issues, suggest features, or contribute code via issues and pull requests.
+
+## License
+
+Congrads is licensed under the [The 3-Clause BSD License](LICENSE). We encourage companies that are interested in a collaboration for a specific topic to contact the authors for more information or to set up joint research projects.
+
+## Contacts
+
+Feel free to contact any of the below contact persons for more information or details about the project. Companies interested in a collaboration, or to set up joint research projects are also encouraged to get in touch with us.
+
+- Peter Karsmakers [ [email](mailto:peter.karsmakers@kuleuven.be) | [website](https://www.kuleuven.be/wieiswie/en/person/00047893) ]
+- Quinten Van Baelen [ [email](mailto:quinten.vanbaelen@kuleuven.be) | [website](https://www.kuleuven.be/wieiswie/en/person/00125540) ]
+
+## Contributors
+
+Below you find a list of people who contributed in making the toolbox. Feel free to contact them for any repository- or code-specific questions, suggestions or remarks.
+
+- Wout Rombouts [ [email](mailto:wout.rombouts@kuleuven.be) | [github profile](https://github.com/rombie18) ]
+- Quinten Van Baelen [ [email](mailto:quinten.vanbaelen@kuleuven.be) | [github profile](https://github.com/quinten-vb) ]
+
+---
+
+Elevate your neural networks with Congrads! 🚀

congrads-0.3.0.dist-info/RECORD

@@ -0,0 +1,23 @@
+congrads/__init__.py,sha256=XJKWRteSvmTYawgS1Pon8kWhhd3haKo4RGAsyjEGS8Q,383
+congrads/callbacks/base.py,sha256=OChXls-tndgJQOXNfqavnPywHZHn3N87yLKD4kbHDHk,13714
+congrads/callbacks/registry.py,sha256=KkzjDqMS3CkE__PpGrmAEYwRngqGSdQNE8NVWl7ogeA,3898
+congrads/checkpoints.py,sha256=V79n3mqjB48nbNkBELqKDg9iou0b1vc5eRrlcu8aIA4,7228
+congrads/constraints/base.py,sha256=k9OyPS2A4bP3fSEAEANGuw7zofiWlGIxqb5ows1LQWs,10105
+congrads/constraints/registry.py,sha256=k__RfcXle-qDL9OJ-nfwgL9zeM6-ISwgQyIzmx-lsgc,52302
+congrads/core/batch_runner.py,sha256=emc7smJLDHq0J8_J9t9X0RtqrXaYwOP9mmhlX_M78e4,6967
+congrads/core/congradscore.py,sha256=9ZKUVMB9RbmudtuC-MQcNColBYUjb6XLqb0eISzfrGk,12070
+congrads/core/constraint_engine.py,sha256=UEt-tmtJeJX0Wu3ol17Z0A9hacL0F8oouJUwHgIIoDE,8994
+congrads/core/epoch_runner.py,sha256=l0x3uLXQ5I5o1C63wXgL4_QkhFmXxW-jeejNJK6sf18,4093
+congrads/datasets/registry.py,sha256=RfffRiA7Qijc69cJTBJhItTZ8x9B-p1kXMjvcfEC_nA,31102
+congrads/descriptor.py,sha256=tUHF4vvyNzJP5vpq1xn0uhKnOlAkElwG2R9gG4glHvQ,6914
+congrads/metrics.py,sha256=e52QC8yNKsxAndjC3U4WMUnQ_0GmiSlExKtxRRShHao,4625
+congrads/networks/registry.py,sha256=UPzPDU0wI2zoOEvi697QBSDOtaa3Rc0rgCb-tCxbjak,2252
+congrads/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+congrads/transformations/base.py,sha256=KZQkloaDGcqAp8EhlUnHL8VfZqSq8OCqp_Iy_a2Nfns,900
+congrads/transformations/registry.py,sha256=p2cLnt3X1bspEPfR7IVd31qPXQimVe_bRu2VhUOIZj0,2607
+congrads/utils/preprocessors.py,sha256=oqW3hV_yoUd-6I-NSoE61e_JDNEPnBJvvvdsuKd9Ekg,18190
+congrads/utils/utility.py,sha256=zvOAVjQjtmsvyuJm0rF0cy_jApR6qluQsFxk9ItalzE,18893
+congrads/utils/validation.py,sha256=Jj8ZJGJrrH9B02cIaScsQpne3zjyarkPldDdT1pejVA,6208
+congrads-0.3.0.dist-info/WHEEL,sha256=eh7sammvW2TypMMMGKgsM83HyA_3qQ5Lgg3ynoecH3M,79
+congrads-0.3.0.dist-info/METADATA,sha256=UCutFzaiD6CaeSr9BqtfiyMS-LsDs6Iwpw8QcXSS2jc,10748
+congrads-0.3.0.dist-info/RECORD,,

congrads-0.3.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: uv 0.8.24
+Root-Is-Purelib: true
+Tag: py3-none-any

congrads/constraints.py

@@ -1,389 +0,0 @@
-from abc import ABC, abstractmethod
-from numbers import Number
-import random
-import string
-from typing import Callable, Dict
-from torch import (
-    Tensor,
-    ge,
-    gt,
-    lt,
-    le,
-    reshape,
-    stack,
-    ones,
-    tensor,
-    zeros_like,
-)
-import logging
-from torch.nn.functional import normalize
-
-from .descriptor import Descriptor
-
-
-class Constraint(ABC):
-
-    descriptor: Descriptor = None
-    device = None
-
-    def __init__(
-        self,
-        neurons: set[str],
-        name: str = None,
-        rescale_factor: float = 1.5,
-    ) -> None:
-
-        # Init parent class
-        super().__init__()
-
-        # Init object variables
-        self.neurons = neurons
-        self.rescale_factor = rescale_factor
-
-        # Perform checks
-        if rescale_factor <= 1:
-            logging.warning(
-                f"Rescale factor for constraint {name} is <= 1. The network will favor general loss over the constraint-adjusted loss. Is this intended behaviour? Normally, the loss should always be larger than 1."
-            )
-
-        # If no constraint_name is set, generate one based on the class name and a random suffix
-        if name:
-            self.name = name
-        else:
-            random_suffix = "".join(
-                random.choices(string.ascii_uppercase + string.digits, k=6)
-            )
-            self.name = f"{self.__class__.__name__}_{random_suffix}"
-            logging.warning(f"Name for constraint is not set. Using {self.name}.")
-
-        # If rescale factor is not larger than 1, warn user and adjust
-        if not rescale_factor > 1:
-            self.rescale_factor = abs(rescale_factor) + 1.5
-            logging.warning(
-                f"Rescale factor for constraint {name} is < 1, adjusted value {rescale_factor} to {self.rescale_factor}."
-            )
-        else:
-            self.rescale_factor = rescale_factor
-
-        # Infer layers from descriptor and neurons
-        self.layers = set()
-        for neuron in self.neurons:
-            if neuron not in self.descriptor.neuron_to_layer.keys():
-                raise ValueError(
-                    f'The neuron name {neuron} used with constraint {self.name} is not defined in the descriptor. Please add it to the correct layer using descriptor.add("layer", ...).'
-                )
-
-            self.layers.add(self.descriptor.neuron_to_layer[neuron])
-
-    # TODO only denormalize if required for efficiency
-    def _denormalize(self, input: Tensor, neuron_names: list[str]):
-        # Extract min and max for each neuron
-        min_values = tensor(
-            [self.descriptor.neuron_to_minmax[name][0] for name in neuron_names],
-            device=input.device,
-        )
-        max_values = tensor(
-            [self.descriptor.neuron_to_minmax[name][1] for name in neuron_names],
-            device=input.device,
-        )
-
-        # Apply vectorized denormalization
-        return input * (max_values - min_values) + min_values
-
-    @abstractmethod
-    def check_constraint(self, prediction: dict[str, Tensor]) -> Tensor:
-        raise NotImplementedError
-
-    @abstractmethod
-    def calculate_direction(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-        raise NotImplementedError
-
-
-class ScalarConstraint(Constraint):
-
-    def __init__(
-        self,
-        neuron_name: str,
-        comparator: Callable[[Tensor, Number], Tensor],
-        scalar: Number,
-        name: str = None,
-        rescale_factor: float = 1.5,
-    ) -> None:
-
-        # Compose constraint name
-        name = f"{neuron_name}_{comparator.__name__}_{str(scalar)}"
-
-        # Init parent class
-        super().__init__({neuron_name}, name, rescale_factor)
-
-        # Init variables
-        self.comparator = comparator
-        self.scalar = scalar
-
-        # Get layer name and feature index from neuron_name
-        self.layer = self.descriptor.neuron_to_layer[neuron_name]
-        self.index = self.descriptor.neuron_to_index[neuron_name]
-
-        # If comparator function is not supported, raise error
-        if comparator not in [ge, le, gt, lt]:
-            raise ValueError(
-                f"Comparator {str(comparator)} used for constraint {name} is not supported. Only ge, le, gt, lt are allowed."
-            )
-
-        # Calculate directions based on constraint operator
-        if self.comparator in [lt, le]:
-            self.direction = -1
-        elif self.comparator in [gt, ge]:
-            self.direction = 1
-
-    def check_constraint(self, prediction: dict[str, Tensor]) -> Tensor:
-
-        return ~self.comparator(prediction[self.layer][:, self.index], self.scalar)
-
-    def calculate_direction(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-        # NOTE currently only works for dense layers due to neuron to index translation
-
-        output = {}
-
-        for layer in self.layers:
-            output[layer] = zeros_like(prediction[layer][0])
-
-        output[self.layer][self.index] = self.direction
-
-        for layer in self.layers:
-            output[layer] = normalize(reshape(output[layer], [1, -1]), dim=1)
-
-        return output
-
-
-class BinaryConstraint(Constraint):
-
-    def __init__(
-        self,
-        neuron_name_left: str,
-        comparator: Callable[[Tensor, Number], Tensor],
-        neuron_name_right: str,
-        name: str = None,
-        rescale_factor: float = 1.5,
-    ) -> None:
-
-        # Compose constraint name
-        name = f"{neuron_name_left}_{comparator.__name__}_{neuron_name_right}"
-
-        # Init parent class
-        super().__init__(
-            {neuron_name_left, neuron_name_right},
-            name,
-            rescale_factor,
-        )
-
-        # Init variables
-        self.comparator = comparator
-
-        # Get layer name and feature index from neuron_name
-        self.layer_left = self.descriptor.neuron_to_layer[neuron_name_left]
-        self.layer_right = self.descriptor.neuron_to_layer[neuron_name_right]
-        self.index_left = self.descriptor.neuron_to_index[neuron_name_left]
-        self.index_right = self.descriptor.neuron_to_index[neuron_name_right]
-
-        # If comparator function is not supported, raise error
-        if comparator not in [ge, le, gt, lt]:
-            raise RuntimeError(
-                f"Comparator {str(comparator)} used for constraint {name} is not supported. Only ge, le, gt, lt are allowed."
-            )
-
-        # Calculate directions based on constraint operator
-        if self.comparator in [lt, le]:
-            self.direction_left = -1
-            self.direction_right = 1
-        else:
-            self.direction_left = 1
-            self.direction_right = -1
-
-    def check_constraint(self, prediction: dict[str, Tensor]) -> Tensor:
-
-        return ~self.comparator(
-            prediction[self.layer_left][:, self.index_left],
-            prediction[self.layer_right][:, self.index_right],
-        )
-
-    def calculate_direction(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-        # NOTE currently only works for dense layers due to neuron to index translation
-
-        output = {}
-
-        for layer in self.layers:
-            output[layer] = zeros_like(prediction[layer][0])
-
-        output[self.layer_left][self.index_left] = self.direction_left
-        output[self.layer_right][self.index_right] = self.direction_right
-
-        for layer in self.layers:
-            output[layer] = normalize(reshape(output[layer], [1, -1]), dim=1)
-
-        return output
-
-
-class SumConstraint(Constraint):
-    def __init__(
-        self,
-        neuron_names_left: list[str],
-        comparator: Callable[[Tensor, Number], Tensor],
-        neuron_names_right: list[str],
-        weights_left: list[float] = None,
-        weights_right: list[float] = None,
-        name: str = None,
-        rescale_factor: float = 1.5,
-    ) -> None:
-
-        # Init parent class
-        neuron_names = set(neuron_names_left) | set(neuron_names_right)
-        super().__init__(neuron_names, name, rescale_factor)
-
-        # Init variables
-        self.comparator = comparator
-        self.neuron_names_left = neuron_names_left
-        self.neuron_names_right = neuron_names_right
-
-        # If comparator function is not supported, raise error
-        if comparator not in [ge, le, gt, lt]:
-            raise ValueError(
-                f"Comparator {str(comparator)} used for constraint {name} is not supported. Only ge, le, gt, lt are allowed."
-            )
-
-        # If feature list dimensions don't match weight list dimensions, raise error
-        if weights_left and (len(neuron_names_left) != len(weights_left)):
-            raise ValueError(
-                "The dimensions of neuron_names_left don't match with the dimensions of weights_left."
-            )
-        if weights_right and (len(neuron_names_right) != len(weights_right)):
-            raise ValueError(
-                "The dimensions of neuron_names_right don't match with the dimensions of weights_right."
-            )
-
-        # If weights are provided for summation, transform them to Tensors
-        if weights_left:
-            self.weights_left = tensor(weights_left, device=self.device)
-        else:
-            self.weights_left = ones(len(neuron_names_left), device=self.device)
-        if weights_right:
-            self.weights_right = tensor(weights_right, device=self.device)
-        else:
-            self.weights_right = ones(len(neuron_names_right), device=self.device)
-
-        # Calculate directions based on constraint operator
-        if self.comparator in [lt, le]:
-            self.direction_left = -1
-            self.direction_right = 1
-        else:
-            self.direction_left = 1
-            self.direction_right = -1
-
-    def check_constraint(self, prediction: dict[str, Tensor]) -> Tensor:
-
-        def compute_weighted_sum(neuron_names: list[str], weights: tensor) -> tensor:
-            layers = [
-                self.descriptor.neuron_to_layer[neuron_name]
-                for neuron_name in neuron_names
-            ]
-            indices = [
-                self.descriptor.neuron_to_index[neuron_name]
-                for neuron_name in neuron_names
-            ]
-
-            # Extract predictions for all neurons and apply weights in bulk
-            predictions = stack(
-                [prediction[layer][:, index] for layer, index in zip(layers, indices)],
-                dim=1,
-            )
-
-            # Denormalize if required
-            predictions_denorm = self._denormalize(predictions, neuron_names)
-
-            # Calculate weighted sum
-            weighted_sum = (predictions_denorm * weights.unsqueeze(0)).sum(dim=1)
-
-            return weighted_sum
-
-        weighted_sum_left = compute_weighted_sum(
-            self.neuron_names_left, self.weights_left
-        )
-        weighted_sum_right = compute_weighted_sum(
-            self.neuron_names_right, self.weights_right
-        )
-
-        # Apply the comparator and calculate the result
-        return ~self.comparator(weighted_sum_left, weighted_sum_right)
-
-    def calculate_direction(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-        # NOTE currently only works for dense layers due to neuron to index translation
-
-        output = {}
-
-        for layer in self.layers:
-            output[layer] = zeros_like(prediction[layer][0])
-
-        for neuron_name_left in self.neuron_names_left:
-            layer = self.descriptor.neuron_to_layer[neuron_name_left]
-            index = self.descriptor.neuron_to_index[neuron_name_left]
-            output[layer][index] = self.direction_left
-
-        for neuron_name_right in self.neuron_names_right:
-            layer = self.descriptor.neuron_to_layer[neuron_name_right]
-            index = self.descriptor.neuron_to_index[neuron_name_right]
-            output[layer][index] = self.direction_right
-
-        for layer in self.layers:
-            output[layer] = normalize(reshape(output[layer], [1, -1]), dim=1)
-
-        return output
-
-
-# class MonotonicityConstraint(Constraint):
-#     # TODO docstring
-
-#     def __init__(
-#         self,
-#         neuron_name: str,
-#         name: str = None,
-#         descriptor: Descriptor = None,
-#         rescale_factor: float = 1.5,
-#     ) -> None:
-
-#         # Compose constraint name
-#         name = f"Monotonicity_{neuron_name}"
-
-#         # Init parent class
-#         super().__init__({neuron_name}, name, rescale_factor)
-
-#         # Init variables
-#         if descriptor != None:
-#             self.descriptor = descriptor
-#             self.run_init_descriptor()
-
-#         # Get layer name and feature index from neuron_name
-#         self.layer = self.descriptor.neuron_to_layer[neuron_name]
-#         self.index = self.descriptor.neuron_to_index[neuron_name]
-
-#     def check_constraint(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-#         # Check if values for column in batch are only increasing
-#         result = ~ge(
-#             diff(
-#                 prediction[self.layer][:, self.index],
-#                 prepend=zeros_like(
-#                     prediction[self.layer][:, self.index][:1],
-#                     device=prediction[self.layer].device,
-#                 ),
-#             ),
-#             0,
-#         )
-
-#         return {self.layer: result}
-
-#     def calculate_direction(self, prediction: dict[str, Tensor]) -> Dict[str, Tensor]:
-#         # TODO implement
-
-#         output = {self.layer: zeros_like(prediction[self.layer][0])}
-#         output[self.layer][self.index] = 1
-
-#         return output