ssad 0.1.2__tar.gz

ssad-0.1.2/LICENSE

@@ -0,0 +1,7 @@
+Copyright (c) 2026 Orange SA
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ssad-0.1.2/PKG-INFO

@@ -0,0 +1,74 @@
+Metadata-Version: 2.4
+Name: ssad
+Version: 0.1.2
+Summary: Framework for self-supervised training of reconstruction-based autoencoder models for anomaly detection.
+Author-email: Samuel Berlemont <samuel.berlemont@orange.com>
+Maintainer: Mohammed Achraf El Khamlichi
+Maintainer-email: Julien Cumin <julien1.cumin@orange.com>
+License: MIT
+Project-URL: Homepage, https://github.com/Orange-OpenSource/SSAD
+Project-URL: Issues, https://github.com/Orange-OpenSource/SSAD/issues
+Project-URL: Repository, https://github.com/Orange-OpenSource/SSAD
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: <3.15,>=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: category_encoders<3.0,>=2.8
+Requires-Dist: numpy<3.0,>=1.26
+Requires-Dist: pandas<3.0,>=2.2
+Requires-Dist: scipy<2.0,>=1.15
+Requires-Dist: scikit-learn<2.0,>=1.9
+Requires-Dist: torch<3.0,>=2.10
+Requires-Dist: lightning<3.0,>=2.5
+Requires-Dist: mlflow<4.0,>=3.12
+Requires-Dist: psutil<8.0,>=7.0
+Requires-Dist: nvidia-ml-py<14.0,>=13.610
+Provides-Extra: dev
+Requires-Dist: pandas-stubs==2.2.2.240603; extra == "dev"
+Requires-Dist: torchvision<0.29,>=0.25; extra == "dev"
+Requires-Dist: types-pytz>=2026.2; extra == "dev"
+Requires-Dist: mypy<2.0,>=1.11; extra == "dev"
+Requires-Dist: pylint<4.0,>=3.2; extra == "dev"
+Dynamic: license-file
+
+# SSAD — Self-Supervised Anomaly Detection Library
+
+A Python library for autoencoder-based **anomaly detection** with self-supervised training and dynamic per-sample **confidence** updates.
+
+## Key Features
+
+- Compute per-sample anomaly scores
+- Estimate confidence from score distributions
+- Recalibrate confidence intervals during training
+- Apply confidence-aware losses (normal / abnormal / uncertain)
+- Track experiments and artifacts with **MLflow**
+
+## Installation
+
+```bash
+pip install ssad
+```
+
+For development setup:
+
+```bash
+pip install -e .[dev]
+```
+
+## Quick Links
+
+- **Repository**: https://github.com/Orange-OpenSource/SSAD
+- **Examples**: https://github.com/Orange-OpenSource/SSAD/tree/main/examples
+- **Issues**: https://github.com/Orange-OpenSource/SSAD/issues
+
+
+## References
+
+1. N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, C. Garcia,  
+   *Robust Variational Autoencoders and Normalizing Flows for Unsupervised Network Anomaly Detection*,  
+   AINA 2022, doi: 10.1007/978-3-030-99587-4_24
+
+2. N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, C. Garcia,  
+   *RADON: Robust Autoencoder for Unsupervised Anomaly Detection*,  
+   SIN 2021, doi: 10.1109/SIN54109.2021.9699174

ssad-0.1.2/README.md

@@ -0,0 +1,208 @@
+# SSAD — Self-Supervised Anomaly Detection Library
+
+A Python library for autoencoder-based **anomaly detection** based on self-supervised training with dynamic **sample confidence** updates.
+
+## Purpose
+
+This library is designed to:
+
+- train a model that produces an **anomaly score**;
+- estimate per-sample **confidence** from that score;
+- analyze score distributions to periodically recalibrate confidence intervals corresponding to normal, abnormal and unknown samples;
+- apply different losses depending on confidence regions, which can take confidence and intervals into account to reweight samples;
+- track experiments, metrics, and artifacts with **MLflow** and an SQL backend store.
+
+NB: examples are proposed in the [examples folder](examples). They correspond to the implementation of the RADON and GRAnD anomaly detection models.
+
+[1] N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, et C. Garcia, « Robust Variational Autoencoders and Normalizing Flows for Unsupervised Network Anomaly Detection », in Advanced Information Networking and Applications, vol. 450, L. Barolli, F. Hussain, et T. Enokido, Éd., in Lecture Notes in Networks and Systems, vol. 450. , Cham: Springer International Publishing, 2022, p. 281‑292. doi: 10.1007/978-3-030-99587-4_24.
+
+[2] N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, et C. Garcia, « RADON: Robust Autoencoder for Unsupervised Anomaly Detection », in 2021 14th International Conference on Security of Information and Networks (SIN), déc. 2021, p. 1‑8. doi: 10.1109/SIN54109.2021.9699174.
+
+---
+
+## Installation
+
+```
+pip install -e .
+```
+
+or, to launch examples,
+
+```
+pip install -e .[dev]
+```
+
+---
+
+## Visualize results
+Define the tracking URI in the MLflow configuration of the experiment:
+
+```
+tracking_uri=f"sqlite:///<path-to>/mlflow.db",
+```
+
+then start the server to visualize the results and artefacts:
+
+```
+mlflow ui --backend-store-uri sqlite:////<path-to>/mlflow.db
+```
+
+## High-Level Architecture
+
+The codebase is split into focused modules:
+
+- `confidence_estimators`: confidence estimation logic from model scores.
+- `distribution_analyzers`: score distribution analysis and interval extraction.
+- `datamodules`: Lightning data wrapping with confidence-aware datasets.
+- `datasets`: dataset types and confidence I/O helpers.
+- `modules`: self-supervision training module and callback.
+- `models`: PyTorch model definitions (e.g., autoencoder).
+- `loggers`: MLflow utility functions for artifacts and metrics logging.
+
+Consolidated class diagram:  
+![Complete class diagram](docs/diagrams/complete.svg)
+
+---
+
+## Class Diagrams
+
+All PlantUML source files are in:
+
+- `docs/diagrams/*.plantuml`
+
+### 1) Confidence Estimators
+- UML source: `docs/diagrams/confidence_estimators.plantuml`
+- Figure:  
+  ![confidence_estimators](docs/diagrams/confidence_estimators.svg)
+
+Main elements:
+
+- `SupportsConfidenceEstimation` (Protocol)
+- `BaseConfidenceEstimator` (abstract)
+- `ConfidenceIntervalsConfiguration`
+- `Interval` (extends `pandas.Interval`)
+
+---
+
+### 2) Data Modules
+- UML source: `docs/diagrams/datamodules.plantuml`
+- Figure:  
+  ![datamodules](docs/diagrams/datamodules.svg)
+
+Main elements:
+
+- `SelfSupervisionDataModule` wrapping a Lightning datamodule
+- Integration with `DatasetWithConfidence`
+
+---
+
+### 3) Datasets
+- UML source: `docs/diagrams/datasets.plantuml`
+- Figure:  
+  ![datasets](docs/diagrams/datasets.svg)
+
+Main elements:
+
+- `DataFrameWithLabels`
+- `DatasetWithLabels` / `DatasetWithInputDim` (Protocols)
+- `DatasetWithConfidence`
+- Utility functions:
+  - `init_confidence_from_csv`
+  - `save_confidence_to_csv`
+
+---
+
+### 4) Distribution Analyzers
+- UML source: `docs/diagrams/distribution_analyzers.plantuml`
+- Figure:  
+  ![distribution_analyzers](docs/diagrams/distribution_analyzers.svg)
+
+Main elements:
+
+- `SupportsDistributionAnalysis` (Protocol)
+- Concrete analyzer implementations (e.g., thresholding strategies)
+
+---
+
+### 5) Models
+- UML source: `docs/diagrams/models.plantuml`
+- Figure:  
+  ![models](docs/diagrams/models.svg)
+
+Main elements:
+
+- `torch.nn.Module`
+- `Autoencoder`
+
+---
+
+### 6) Self-Supervision Modules
+- UML source: `docs/diagrams/modules.plantuml`
+- Figure:  
+  ![modules](docs/diagrams/modules.svg)
+
+Main elements:
+
+- `SupportsSelfSupervision` (Protocol)
+- `SelfSupervisionModule` (abstract Lightning module)
+- `SelfSupervisionCallback`
+- Dependency injection of:
+  - `SupportsConfidenceEstimation`
+  - `SupportsDistributionAnalysis`
+
+---
+
+## Training Loop (Conceptual)
+
+1. The model computes per-sample scores (`score` / `_prediction_score`).
+2. Distribution analysis derives confidence intervals.
+3. Confidence estimator maps scores to confidence values.
+4. Training dataset is refreshed with updated confidence.
+5. Loss computation uses confidence-aware behavior (normal/abnormal/uncertain).
+6. Confidence and intervals are recalibrated every `every_n_epochs`.
+7. Metrics and artifacts are logged.
+
+---
+
+## Global workflow
+
+![workflow](docs/diagrams/ssad_workflow_light.svg)
+
+---
+
+## Logging
+
+`ssad/loggers/mlflow_logger.py` provides helper functions to log:
+
+- confidence CSV snapshots (`confidence_epoch_*.csv`)
+- confidence interval JSON files (`confidence_intervals_epoch_*.json`)
+- distribution analysis figures (`confidence_analysis_epoch_*.svg`)
+- system metrics (CPU / RAM / GPU)
+- test metrics by threshold (`test_metrics_threshold=*.json`)
+
+---
+
+## Main Dependencies
+
+- Python 3.10+
+- PyTorch
+- Lightning
+- NumPy / pandas / scikit-learn / matplotlib
+- MLflow
+- psutil
+
+---
+
+## Typical Usage (High-Level)
+
+1. Prepare a dataset compatible with:
+   - `DatasetWithLabels`
+   - `DatasetWithInputDim`
+2. Build your base `LightningDataModule`.
+3. Wrap it with `SelfSupervisionDataModule`.
+4. Instantiate:
+   - a model (`nn.Module`)
+   - a confidence estimator (`SupportsConfidenceEstimation`)
+   - a distribution analyzer (`SupportsDistributionAnalysis`)
+   - a concrete `SelfSupervisionModule`
+5. Train/evaluate with Lightning `Trainer`.

ssad-0.1.2/README_PYPI.md

@@ -0,0 +1,40 @@
+# SSAD — Self-Supervised Anomaly Detection Library
+
+A Python library for autoencoder-based **anomaly detection** with self-supervised training and dynamic per-sample **confidence** updates.
+
+## Key Features
+
+- Compute per-sample anomaly scores
+- Estimate confidence from score distributions
+- Recalibrate confidence intervals during training
+- Apply confidence-aware losses (normal / abnormal / uncertain)
+- Track experiments and artifacts with **MLflow**
+
+## Installation
+
+```bash
+pip install ssad
+```
+
+For development setup:
+
+```bash
+pip install -e .[dev]
+```
+
+## Quick Links
+
+- **Repository**: https://github.com/Orange-OpenSource/SSAD
+- **Examples**: https://github.com/Orange-OpenSource/SSAD/tree/main/examples
+- **Issues**: https://github.com/Orange-OpenSource/SSAD/issues
+
+
+## References
+
+1. N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, C. Garcia,  
+   *Robust Variational Autoencoders and Normalizing Flows for Unsupervised Network Anomaly Detection*,  
+   AINA 2022, doi: 10.1007/978-3-030-99587-4_24
+
+2. N. Najari, S. Berlemont, G. Lefebvre, S. Duffner, C. Garcia,  
+   *RADON: Robust Autoencoder for Unsupervised Anomaly Detection*,  
+   SIN 2021, doi: 10.1109/SIN54109.2021.9699174

ssad-0.1.2/pyproject.toml

@@ -0,0 +1,111 @@
+[project]
+name = "ssad"
+version = "0.1.2"
+description = "Framework for self-supervised training of reconstruction-based autoencoder models for anomaly detection."
+readme = "README_PYPI.md"
+requires-python = ">=3.10,<3.15"
+license = { text = "MIT" }
+authors = [
+  { name = "Samuel Berlemont", email = "samuel.berlemont@orange.com" }
+]
+maintainers = [
+  { name = "Julien Cumin", email = "julien1.cumin@orange.com" },
+  { name = "Mohammed Achraf El Khamlichi" },
+]
+classifiers = [
+  "Programming Language :: Python :: 3",
+  "Operating System :: OS Independent",
+]
+dependencies = [
+  "category_encoders>=2.8,<3.0",
+  "numpy>=1.26,<3.0",
+  "pandas>=2.2,<3.0",
+  "scipy>=1.15,<2.0",
+  "scikit-learn>=1.9,<2.0",
+  "torch>=2.10,<3.0",
+  "lightning>=2.5,<3.0",
+  "mlflow>=3.12,<4.0",
+  "psutil>=7.0,<8.0",
+  "nvidia-ml-py>=13.610,<14.0"
+]
+
+[project.urls]
+Homepage = "https://github.com/Orange-OpenSource/SSAD"
+Issues = "https://github.com/Orange-OpenSource/SSAD/issues"
+Repository = "https://github.com/Orange-OpenSource/SSAD"
+
+[project.optional-dependencies]
+dev = [
+  "pandas-stubs==2.2.2.240603",
+  "torchvision>=0.25,<0.29",
+  "types-pytz>=2026.2",
+  "mypy>=1.11,<2.0",
+  "pylint>=3.2,<4.0",
+]
+
+[build-system]
+requires = ["setuptools >= 77.0.3", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["ssad*"]
+
+[tool.setuptools.package-data]
+ssad = ["py.typed"]
+
+[tool.mypy]
+python_version = "3.10"
+files = ["ssad"]
+pretty = true
+show_error_codes = true
+warn_unused_configs = true
+
+# Progressive strictness - active baseline (Palier 1)
+ignore_missing_imports = true
+check_untyped_defs = true
+no_implicit_optional = true
+
+# warn_redundant_casts = true
+# warn_unused_ignores = true
+# strict_equality = true
+# disallow_incomplete_defs = true
+
+# disallow_untyped_defs = true
+# warn_return_any = true
+# disallow_any_generics = true
+
+[[tool.mypy.overrides]]
+module = [
+  "category_encoders.*",
+  "lightning.*",
+  "mlflow.*",
+  "torchvision.*",
+]
+ignore_missing_imports = true
+
+[tool.pylint.main]
+py-version = "3.10"
+jobs = 0
+recursive = true
+ignore = ["build", "dist", ".venv"]
+
+[tool.pylint.format]
+max-line-length = 100
+
+[tool.pylint."messages control"]
+disable = [
+  "C0114", # missing-module-docstring
+  "C0115", # missing-class-docstring
+  "C0116", # missing-function-docstring
+  "R0903", # too-few-public-methods
+]
+
+[tool.pylint.design]
+max-args = 10
+max-locals = 25
+max-branches = 20
+max-statements = 80
+
+[tool.pylint.typecheck]
+ignored-modules = ["torch", "lightning", "mlflow"]

ssad-0.1.2/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ssad-0.1.2/src/ssad/__init__.py

@@ -0,0 +1,39 @@
+# Software Name : Self-Supervised Anomaly Detection
+# SPDX-FileCopyrightText: Copyright (c) Orange SA
+# SPDX-License-Identifier: MIT
+#
+# This software is distributed under the MIT License,
+# see the "LICENSE.txt" file for more details or https://spdx.org/licenses/MIT.html
+#
+# Authors: see CONTRIBUTORS
+# Software description: A Python library for autoencoder-based anomaly detection
+#          based on self-supervised training with dynamic sample confidence updates.
+"""
+Initializes the main package API by exposing public interfaces from submodules.
+
+This module re-exports the core components of the package to simplify access and
+maintain a clean and consistent public API.
+
+Available namespaces:
+- confidence_estimators
+- datamodules
+- datasets
+- distribution_analyzers
+- models
+- modules
+- loggers
+
+Usage:
+    from mypackage import SomeModel, SomeDataModule, ConfidenceEstimator
+
+Note:
+    This file uses wildcard imports (`*`) to expose only the public symbols defined
+    in each submodule's `__all__` list.
+"""
+from .confidence_estimators.api import *
+from .datamodules.api import *
+from .datasets.api import *
+from .distribution_analyzers.api import *
+from .models.api import *
+from .modules.api import *
+from .loggers.api import *

ssad-0.1.2/src/ssad/confidence_estimators/__init__.py

@@ -0,0 +1,15 @@
+# Software Name : Self-Supervised Anomaly Detection
+# SPDX-FileCopyrightText: Copyright (c) Orange SA
+# SPDX-License-Identifier: MIT
+#
+# This software is distributed under the MIT License,
+# see the "LICENSE.txt" file for more details or https://spdx.org/licenses/MIT.html
+#
+# Authors: see CONTRIBUTORS
+# Software description: A Python library for autoencoder-based anomaly detection
+#          based on self-supervised training with dynamic sample confidence updates.
+r"""
+# What is confidence estimator
+A confidence estimator is a module that, given a value such as a reconstruction error,
+or a gradient value, returns a confidence score comprised between -1 and 1.
+"""

ssad-0.1.2/src/ssad/confidence_estimators/api.py

@@ -0,0 +1,16 @@
+# Software Name : Self-Supervised Anomaly Detection
+# SPDX-FileCopyrightText: Copyright (c) Orange SA
+# SPDX-License-Identifier: MIT
+#
+# This software is distributed under the MIT License,
+# see the "LICENSE.txt" file for more details or https://spdx.org/licenses/MIT.html
+#
+# Authors: see CONTRIBUTORS
+# Software description: A Python library for autoencoder-based anomaly detection
+#          based on self-supervised training with dynamic sample confidence updates.
+"""Public API for self-supervision confidence estimators."""
+from .binary_confidence import BinaryConfidence
+from .supports_confidence_estimation import SupportsConfidenceEstimation
+from .confidence_intervals_configuration import ConfidenceIntervalsConfiguration
+
+__all__ = ["SupportsConfidenceEstimation", "BinaryConfidence", "ConfidenceIntervalsConfiguration"]

ssad-0.1.2/src/ssad/confidence_estimators/binary_confidence.py

@@ -0,0 +1,30 @@
+# Software Name : Self-Supervised Anomaly Detection
+# SPDX-FileCopyrightText: Copyright (c) Orange SA
+# SPDX-License-Identifier: MIT
+#
+# This software is distributed under the MIT License,
+# see the "LICENSE.txt" file for more details or https://spdx.org/licenses/MIT.html
+#
+# Authors: see CONTRIBUTORS
+# Software description: A Python library for autoencoder-based anomaly detection
+#          based on self-supervised training with dynamic sample confidence updates.
+"""
+Implements a binary confidence estimator.
+A sample is either normal or abnormal, otherwise it is omitted (zero confidence).
+"""
+
+import torch
+from .confidence_estimator import BaseConfidenceEstimator
+
+
+class BinaryConfidence(BaseConfidenceEstimator):
+    """Binary confidence estimator"""
+
+    def _confidence_normal(self, score):
+        return torch.ones_like(score)
+
+    def _confidence_abnormal(self, score):
+        return torch.full_like(score, -1)
+
+    def _confidence_unknown(self, score):
+        return torch.zeros_like(score)

ssad-0.1.2/src/ssad/confidence_estimators/confidence_estimator.py

@@ -0,0 +1,94 @@
+# Software Name : Self-Supervised Anomaly Detection
+# SPDX-FileCopyrightText: Copyright (c) Orange SA
+# SPDX-License-Identifier: MIT
+#
+# This software is distributed under the MIT License,
+# see the "LICENSE.txt" file for more details or https://spdx.org/licenses/MIT.html
+#
+# Authors: see CONTRIBUTORS
+# Software description: A Python library for autoencoder-based anomaly detection
+#          based on self-supervised training with dynamic sample confidence updates.
+"""
+Provides the base class for confidence estimators.
+"""
+
+from abc import ABC, abstractmethod
+from typing import Optional
+import torch
+
+from .confidence_intervals_configuration import (
+    ConfidenceIntervalsConfiguration,
+)
+from .supports_confidence_estimation import SupportsConfidenceEstimation
+
+
+class BaseConfidenceEstimator(ABC, SupportsConfidenceEstimation):
+    """Base class for confidence estimators.
+    A confidence estimator relies on four intervals with associated scoring functions
+    to provide a confidence score given a criterion score for a sample.
+
+    The criterion score can be for instance a reconstruction score, or the norm of
+    the gradient of the reconstruction error.
+
+    The intervals define the domain for model scores for the four different confidence behaviors:
+    - normal: samples with model scores in this interval are considered as normal
+    - abnormal: samples with model scores in this interval are considered as abnormal
+    - unknown_positive: samples with model scores in this interval are
+        considered as unknown, but leaning towards a normal sample.
+    - unknown_negative: samples with model scores in this interval are
+        considered as unknown, but leaning towards an abnormal sample.
+
+    Each interval is associated with a "criterion score to confidence score" conversion function.
+    These functions should be implemented in the _estimate_confidence_from_model_score method.
+    """
+
+    def __init__(self):
+        super().__init__()
+        self.configuration: Optional[ConfidenceIntervalsConfiguration] = None
+        self.distribution: Optional[torch.Tensor] = None
+
+    @abstractmethod
+    def _confidence_normal(self, score):
+        raise NotImplementedError()
+
+    @abstractmethod
+    def _confidence_abnormal(self, score) -> torch.Tensor:
+        raise NotImplementedError()
+
+    @abstractmethod
+    def _confidence_unknown(self, score) -> torch.Tensor:
+        raise NotImplementedError()
+
+    @torch.no_grad()
+    def estimate_confidence(self, scores_batch: torch.Tensor) -> torch.Tensor:
+        """Estimates the confidence in a batch by retrieving the criterion score
+        and translating into a confidence score.
+
+        Args:
+            scores_batch (torch.Tensor): batch whose confidence is to be estimated.
+
+        Returns:
+            torch.Tensor: confidence score
+        """
+        # TODO: check this order of computation
+        confidence = self._confidence_unknown(scores_batch)
+
+        if self.configuration is None:
+            raise ValueError("Confidence estimator configuration is None")
+
+        # TODO: rework signatures of confidence normal/abnormal/unknown
+        normal_confidences = self._confidence_normal(scores_batch)
+        abnormal_confidences = self._confidence_abnormal(scores_batch)
+
+        confidence = torch.where(
+            self.configuration.normal.contains_tensor_mask(scores_batch),
+            normal_confidences,
+            confidence,
+        )
+        confidence = torch.where(
+            self.configuration.abnormal.contains_tensor_mask(scores_batch),
+            abnormal_confidences,
+            confidence,
+        )
+
+        return confidence