deepsuite 1.0.0__py3-none-any.whl

deepsuite/README.md

@@ -0,0 +1,261 @@
+# DeepSuite
+
+This is the main directory for the DeepSuite source code.
+
+## Module Overview
+
+Each subdirectory contains its own README.md with detailed documentation.
+
+### 🧠 Language Models & NLP
+
+- **[layers/](layers/)** - Neural Network Layers
+
+  - Attention Mechanisms (MLA, RoPE, KV-Compression)
+  - Mixture-of-Experts (MoE)
+  - Specialized Layers (Complex, Hermite, Laguerre)
+
+- **[modules/](modules/)** - PyTorch Lightning Modules
+
+  - GPT-2/GPT-3
+  - DeepSeek-V3 with MLA and MoE
+  - YOLO, CenterNet
+
+- **[heads/](heads/)** - Output Heads
+
+  - Language Model Head
+  - Multi-Token Prediction Head
+  - Classification, Detection, Heatmap Heads
+
+- **[loss/](loss/)** - Loss Functions
+  - Language Modeling Losses
+  - Multi-Token Prediction Loss
+  - Detection Losses (Focal, GIoU, DFL)
+  - Knowledge Distillation
+
+### 👁️ Computer Vision
+
+- **[model/](model/)** - Model Architectures
+
+  - Object Detection (YOLO, CenterNet, EfficientDet)
+  - Feature Extraction (ResNet, EfficientNet, DarkNet, FPN)
+  - Tracking
+
+- **[tracker/](tracker/)** - Multi-Object Tracking
+
+  - SORT, DeepSORT, ByteTrack
+  - Re-Identification
+  - Tracking Pipeline
+
+- **[metric/](metric/)** - Evaluation Metrics
+  - Mean Average Precision (mAP)
+  - Detection Metrics
+  - Confusion Matrix
+
+### 🎵 Audio Processing
+
+- **[model/beamforming/](model/beamforming/)** - Audio Beamforming
+- See also: `model/complex.py`, `model/doa.py`, `model/rnn.py`
+
+### ⚙️ Training & Utilities
+
+- **[lightning_base/](lightning_base/)** - PyTorch Lightning Base
+
+  - Base Lightning Module
+  - Dataset Loaders (Text, Image, Audio)
+  - Continual Learning Manager
+
+- **[callbacks/](callbacks/)** - Training Callbacks
+
+  - TorchScript, TensorRT Export
+  - Embedding Logger
+  - t-SNE Visualization
+
+- **[utils/](utils/)** - Utility Functions
+
+  - Bounding Box Operations
+  - Image Processing
+  - Tensor Utilities
+  - Device Management
+
+- **[viz/](viz/)** - Visualization
+  - Embedding Visualization (t-SNE, UMAP, PCA)
+
+### 🔧 Specialized
+
+- **[config/](config/)** - Configuration Management
+- **[svm/](svm/)** - Support Vector Machines
+- **[conv/](conv/)** - Convolutional Neural Networks
+
+## Quick Access
+
+### Key Files
+
+```
+src/deepsuite/
+├── modules/
+│   ├── gpt.py              # GPT-2/GPT-3 Implementation
+│   ├── deepseek.py         # DeepSeek-V3 Implementation
+│   ├── yolo.py             # YOLO Object Detection
+│   └── centernet.py        # CenterNet Detection
+│
+├── layers/
+│   ├── attention/          # Attention Mechanisms
+│   │   ├── mla.py          # Multi-Head Latent Attention
+│   │   └── rope.py         # Rotary Position Embeddings
+│   └── moe.py              # Mixture-of-Experts
+│
+├── lightning_base/dataset/
+│   ├── text_loader.py      # Text Dataset for LLMs
+│   ├── image_loader.py     # Image Dataset
+│   └── audio_loader.py     # Audio Dataset
+│
+└── loss/
+    ├── classification.py   # Cross-Entropy, Focal Loss
+    ├── mtp_loss.py         # Multi-Token Prediction Loss
+    └── detection.py        # Detection Losses
+```
+
+## Usage
+
+### Import Examples
+
+```python
+# Language Models
+from deepsuite.modules import GPTModule, DeepSeekModule
+from deepsuite.layers.attention import MultiHeadLatentAttention
+from deepsuite.layers import DeepSeekMoE
+
+# Object Detection
+from deepsuite.modules import YOLOModule, CenterNetModule
+from deepsuite.model.detection import YOLO, CenterNet
+
+# Datasets
+from deepsuite.lightning_base.dataset import TextDataLoader, ImageDataLoader
+
+# Losses
+from deepsuite.loss import CrossEntropyLoss, MTPLoss, FocalLoss, GIoULoss
+
+# Metrics
+from deepsuite.metric import MeanAveragePrecision, DetectionMetrics
+
+# Tracking
+from deepsuite.tracker import ObjectTracker, DeepSORTTracker
+
+# Utils
+from deepsuite.utils import bbox, image, device
+
+# Visualization
+from deepsuite.viz import EmbeddingVisualizer
+```
+
+## Documentation
+
+### READMEs
+
+Each module has its own README.md:
+
+- [callbacks/README.md](callbacks/README.md) - Training Callbacks
+- [heads/README.md](heads/README.md) - Output Heads
+- [layers/README.md](layers/README.md) - Neural Network Layers
+- [lightning_base/README.md](lightning_base/README.md) - Lightning Base Components
+- [loss/README.md](loss/README.md) - Loss Functions
+- [metric/README.md](metric/README.md) - Evaluation Metrics
+- [model/README.md](model/README.md) - Model Architectures
+- [modules/README.md](modules/README.md) - Lightning Modules
+- [tracker/README.md](tracker/README.md) - Object Tracking
+- [utils/README.md](utils/README.md) - Utility Functions
+- [viz/README.md](viz/README.md) - Visualization Tools
+
+### Complete Documentation
+
+See [docs/modules_overview.md](../../docs/modules_overview.md) for a complete overview of all modules.
+
+### Specific Topics
+
+- [docs/llm_modules.md](../../docs/llm_modules.md) - Language Models (GPT, DeepSeek-V3)
+- [docs/llm_loss_head.md](../../docs/llm_loss_head.md) - LLM Losses & Heads
+- [docs/moe.md](../../docs/moe.md) - Mixture-of-Experts
+- [docs/text_dataset.md](../../docs/text_dataset.md) - Text Data Loading
+
+## Development
+
+### Code Style
+
+```bash
+# Format
+ruff format src/deepsuite
+
+# Lint
+ruff check src/deepsuite
+
+# Type Checking
+mypy src/deepsuite
+```
+
+### Testing
+
+```bash
+# All tests
+pytest tests/
+
+# Specific module
+pytest tests/test_tensor_rt_export_callback.py
+```
+
+## Architecture Principles
+
+### 1. Modularity
+
+Each component is independently usable:
+
+```python
+# Layer alone
+from deepsuite.layers import DeepSeekMoE
+moe = DeepSeekMoE(d_model=2048, ...)
+
+# In custom model
+class MyModel(nn.Module):
+    def __init__(self):
+        self.moe = DeepSeekMoE(...)
+```
+
+### 2. Lightning Integration
+
+All main models are Lightning Modules:
+
+```python
+from deepsuite.modules import DeepSeekModule
+import pytorch_lightning as pl
+
+model = DeepSeekModule(...)
+trainer = pl.Trainer(...)
+trainer.fit(model, datamodule)
+```
+
+### 3. Composability
+
+Components can be freely combined:
+
+```python
+from deepsuite.heads import LMHead
+from deepsuite.loss import CrossEntropyLoss
+from deepsuite.metric import Perplexity
+
+class CustomModule(pl.LightningModule):
+    def __init__(self):
+        self.backbone = ...
+        self.head = LMHead(...)
+        self.loss_fn = CrossEntropyLoss(...)
+        self.metric = Perplexity()
+```
+
+## More Information
+
+- **Main Project**: [README.md](../../README.md)
+- **Documentation**: [docs/](../../docs/)
+- **Examples**: [examples/](../../examples/)
+- **Tests**: [tests/](../../tests/)
+
+---
+
+**Version**: 0.2.0 | **License**: Apache 2.0

deepsuite/__init__.py

@@ -0,0 +1,42 @@
+"""APTT - Advanced PyTorch Training Toolkit.
+
+A PyTorch Lightning-based framework for deep learning with focus on:
+- Object Detection (YOLO, CenterNet)
+- Object Tracking
+- Continual Learning
+- Audio/Signal Processing
+"""
+
+__version__ = "1.0.3"
+
+# Core Lightning Modules
+from deepsuite.lightning_base.continual_learning_manager import ContinualLearningManager
+
+# Base Classes
+from deepsuite.lightning_base.module import BaseModule
+from deepsuite.lightning_base.trainer import BaseTrainer
+from deepsuite.model.backend_adapter import BackboneAdapter
+from deepsuite.model.detection.centernet import CenterNetModel
+
+# Model Architectures
+from deepsuite.model.detection.yolo import YOLO
+from deepsuite.modules.centernet import CenterNetModule
+from deepsuite.modules.tracking import TrackingModule
+from deepsuite.modules.yolo import Yolo
+
+__all__ = [
+    # Version
+    "__version__",
+    # Lightning Modules (end-to-end trainable)
+    "Yolo",
+    "CenterNetModule",
+    "TrackingModule",
+    # Base Classes
+    "BaseModule",
+    "BaseTrainer",
+    "ContinualLearningManager",
+    # Model Architectures
+    "YOLO",
+    "CenterNetModel",
+    "BackboneAdapter",
+]

deepsuite/callbacks/README.md

@@ -0,0 +1,79 @@
+# Callbacks
+
+PyTorch Lightning callbacks for export, optimization, and visualization.
+
+## Modules
+
+### Export & Optimization
+
+- **`torchscript.py`** - TorchScript export for production deployment
+- **`tensor_rt.py`** - TensorRT optimization for NVIDIA GPUs
+
+### Logging & Visualization
+
+- **`embedding_logger.py`** - Embedding visualization during training
+- **`tsne_laplace_callback.py`** - t-SNE and Laplace eigenmap visualization
+
+### Base
+
+- **`base.py`** - Base callback class and shared functionality
+
+## Usage
+
+### TorchScript Export
+
+```python
+from deepsuite.callbacks import TorchScriptCallback
+
+trainer = pl.Trainer(
+    callbacks=[
+        TorchScriptCallback(
+            export_path="model.pt",
+            method="script"  # oder "trace"
+        )
+    ]
+)
+```
+
+### TensorRT Optimization
+
+```python
+from deepsuite.callbacks import TensorRTCallback
+
+trainer = pl.Trainer(
+    callbacks=[
+        TensorRTCallback(
+            export_path="model.engine",
+            precision="fp16",
+            workspace_size=1 << 30  # 1GB
+        )
+    ]
+)
+```
+
+### Embedding Visualization
+
+```python
+from deepsuite.callbacks import EmbeddingLoggerCallback
+
+trainer = pl.Trainer(
+    callbacks=[
+        EmbeddingLoggerCallback(
+            log_every_n_epochs=5,
+            num_samples=1000
+        )
+    ]
+)
+```
+
+## Features
+
+- ✅ Automatischer Export am Ende des Trainings
+- ✅ Validierung der exportierten Modelle
+- ✅ Integration mit TensorBoard und Weights & Biases
+- ✅ Flexible Konfiguration per Callback-Parameter
+- ✅ Fehlerbehandlung und Logging
+
+## Weitere Informationen
+
+Siehe Hauptdokumentation: [docs/modules_overview.md](../../../docs/modules_overview.md)

deepsuite/callbacks/__init__.py

@@ -0,0 +1,17 @@
+"""Export-Callback Paket."""
+
+from .base import ExportBaseCallback
+from .kendryte import KendryteExportCallback
+from .onnx import ONNXExportCallback
+from .tensor_rt import TensorRTExportCallback
+from .tflite import TFLiteExportCallback
+from .torchscript import TorchScriptExportCallback
+
+__all__ = [
+    "ExportBaseCallback",
+    "KendryteExportCallback",
+    "ONNXExportCallback",
+    "TFLiteExportCallback",
+    "TensorRTExportCallback",
+    "TorchScriptExportCallback",
+]

deepsuite/callbacks/base.py

@@ -0,0 +1,111 @@
+"""Basisklasse für Modell-Export-Callbacks.
+
+Stellt Hilfsfunktionen bereit, um automatisch Beispielbatches zu laden und
+auf das korrekte Gerät zu verschieben, sodass Export-Callbacks (z. B. ONNX,
+TorchScript, TensorRT) konsistent arbeiten können.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+
+from loguru import logger
+import pytorch_lightning as pl
+import torch
+
+from deepsuite.utils.device import get_best_device
+
+
+class ExportBaseCallback(pl.callbacks.ModelCheckpoint):
+    """Basisklasse für Export-Callbacks mit Batch-Autoloading."""
+
+    output_dir: Path
+    example_input: Any | None
+
+    def __init__(self, output_dir: str = "models", **kwargs: Any) -> None:
+        """Initialisiert den Export-Callback.
+
+        Args:
+            output_dir: Zielverzeichnis für Exports.
+            **kwargs: Zusätzliche Argumente, werden an ModelCheckpoint weitergegeben.
+        """
+        super().__init__(**kwargs)
+        self.output_dir = Path(output_dir)
+        self.output_dir.mkdir(parents=True, exist_ok=True)
+        self.example_input = None  # Wird später automatisch gesetzt
+
+    def get_example_input(self, trainer: pl.Trainer) -> Any | None:
+        """Lädt einen Beispielbatch und speichert ihn in `example_input`.
+
+        Args:
+            trainer: PyTorch-Lightning-Trainer.
+
+        Returns:
+            Optional[Any]: Beispielinputbatch (ggf. Tensor oder Strukturen) oder None.
+
+        Beispiel:
+            ```python
+            from pytorch_lightning import Trainer
+            from deepsuite.callbacks.base import ExportBaseCallback
+
+            trainer = Trainer(callbacks=[ExportBaseCallback()])
+            trainer.fit(model)
+            ```
+        """
+        if self.example_input is None:  # Nur einmal laden
+            try:
+                datamodule = trainer.datamodule
+                if datamodule is None:
+                    logger.error("❌ No DataModule found.")
+                    return None
+
+                dataloader = datamodule.train_dataloader()
+                if dataloader is None:
+                    logger.error("❌ No train_dataloader() found.")
+                    return None
+
+                batch = next(iter(dataloader))  # Einen Batch entnehmen
+
+                if isinstance(batch, (tuple, list)):
+                    self.example_input = batch[0]
+                else:
+                    self.example_input = batch
+
+                # Auf dasselbe Gerät wie das Modell verschieben
+                if trainer.model is not None:
+                    device = trainer.model.device
+                    self.example_input = self._move_batch_to_device(self.example_input, device)
+
+                logger.info(
+                    "✅ Example batch automatically loaded and moved to the correct device."
+                )
+
+            except Exception as e:
+                logger.error(f"❌ Could not load example batch: {e}")
+                return None
+
+        return self.example_input
+
+    def _move_batch_to_device(self, batch: Any, device: str | torch.device | None = None) -> Any:
+        """Verschiebt einen Batch rekursiv auf das angegebene Gerät.
+
+        Args:
+            batch: Tensor oder (verschachtelte) Liste/Tuple/Dict aus Tensors.
+            device: Zielgerät als str ('cuda', 'cpu') oder torch.device, oder None.
+
+        Returns:
+            Any: Der auf das Zielgerät verschobene Batch.
+        """
+        if device is None:
+            device_str: str | torch.device = get_best_device()
+        else:
+            device_str = device
+
+        if isinstance(batch, torch.Tensor):
+            return batch.to(device_str)
+        if isinstance(batch, (tuple, list)):
+            return type(batch)(self._move_batch_to_device(b, device_str) for b in batch)
+        if isinstance(batch, dict):
+            return {k: self._move_batch_to_device(v, device_str) for k, v in batch.items()}
+        return batch

deepsuite/callbacks/embedding_logger.py

@@ -0,0 +1,101 @@
+"""Embedding Logger module."""
+
+from pytorch_lightning import Callback
+from pytorch_lightning.loggers.tensorboard import TensorBoardLogger
+import torch
+
+from deepsuite.viz.embedding import log_embedding_plot_to_mlflow
+
+
+class EmbeddingLoggerCallback(Callback):
+    """Logs embedding vectors during validation to TensorBoard and optionally MLflow.
+
+    This callback collects embeddings and corresponding labels from a limited
+    number of validation batches. At the end of the epoch, it visualizes them
+    in TensorBoard and optionally logs a TSNE plot to MLflow.
+
+    Example:
+        >>> callback = EmbeddingLoggerCallback(num_batches=3, log_to_mlflow=True)
+        >>> trainer = Trainer(callbacks=[callback])
+    """
+
+    def __init__(self, num_batches: int = 1, log_to_mlflow: bool = True) -> None:
+        """Initializes the embedding logging callback.
+
+        Args:
+            num_batches (int): Number of validation batches to collect for embedding visualization.
+            log_to_mlflow (bool): Whether to log a TSNE plot of embeddings to MLflow.
+        """
+        self.num_batches = num_batches
+        self.embeddings = []
+        self.labels = []
+        self.log_to_mlflow = log_to_mlflow
+
+    def on_validation_batch_end(
+        self, trainer, pl_module, outputs, batch, batch_idx, dataloader_idx=0
+    ):
+        """Collects embeddings and labels from the current batch during validation.
+
+        Args:
+            trainer (Trainer): The PyTorch Lightning trainer.
+            pl_module (LightningModule): The current Lightning model.
+            outputs: The outputs from the validation step (unused).
+            batch (Tuple[Tensor, Tensor]): The input batch, typically (x, y).
+            batch_idx (int): The index of the current batch.
+            dataloader_idx (int): Index of the current dataloader (unused).
+        """
+        if batch_idx >= self.num_batches:
+            return
+
+        x, y = batch
+        with torch.no_grad():
+            if hasattr(pl_module, "extract_features"):
+                features = pl_module.extract_features(x.to(pl_module.device))
+            else:
+                features = pl_module(x.to(pl_module.device))
+
+        self.embeddings.append(features.cpu())
+        self.labels.append(y.cpu())
+
+    def on_validation_epoch_end(self, trainer, pl_module):
+        """Logs collected embeddings to TensorBoard and optionally to MLflow.
+
+        At the end of the validation epoch, this method concatenates the stored
+        embeddings and labels, then logs them as an embedding projector to
+        TensorBoard. If `log_to_mlflow` is True and `pl_module.use_mlflow` is set,
+        it also logs a TSNE image to MLflow.
+
+        Args:
+            trainer (Trainer): The PyTorch Lightning trainer.
+            pl_module (LightningModule): The current Lightning model.
+        """
+        if not self.embeddings or not self.labels:
+            return
+
+        embeddings = torch.cat(self.embeddings, dim=0)
+        labels = torch.cat(self.labels, dim=0)
+
+        # TensorBoard Logging
+        tb_logger = None
+        for logger in trainer.loggers if isinstance(trainer.loggers, list) else [trainer.logger]:
+            if isinstance(logger, TensorBoardLogger):
+                tb_logger = logger.experiment  # SummaryWriter
+                break
+
+        if tb_logger:
+            tb_logger.add_embedding(
+                mat=embeddings,
+                metadata=labels.tolist(),
+                global_step=trainer.global_step,
+                tag="val_embeddings",
+            )
+
+        # MLflow Logging
+        if self.log_to_mlflow and getattr(pl_module, "use_mlflow", False):
+            log_embedding_plot_to_mlflow(
+                embeddings=embeddings, labels=labels, step=trainer.global_step
+            )
+
+        # Puffer leeren
+        self.embeddings.clear()
+        self.labels.clear()