ldif-model 0.1.0__tar.gz

ldif_model-0.1.0/License

@@ -0,0 +1,3 @@
+MIT
+
+Copyright (c) 2026 [Muhammad Muhaimin, Muhammad Waseem Ashraf, Shahzadi Tayyaba] Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, do its origin, subject to the following conditions:The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ldif_model-0.1.0/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: ldif-model
+Version: 0.1.0
+Summary: Latent Dual Interaction Flow neural architecture with response-conditioned gating and adaptive spectrum pruning.
+Author: Muhammad Muhaimin, Muhammad Waseem Ashraf, Shahzadi Tayyaba
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/engrdrwaseem
+Project-URL: Repository, https://github.com/yourusername/ldif
+Project-URL: Issues, https://github.com/yourusername/ldif/issues
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: License
+Requires-Dist: torch>=1.9.0
+Requires-Dist: numpy>=1.19.0
+Dynamic: license-file
+
+# LDIF: Latent Dual Interaction Flow
+
+**LDIF** is a PyTorch implementation of the Latent Dual Interaction Flow. It is a continuous time neural architecture with response conditioned gating and adaptive spectrum pruning.
+
+## Installation
+
+pip install ldif-model
+
+## Quick Start
+
+### Static (tabular) data
+
+LDIF can be implemented in python for static data using this example code.
+
+import torch
+from ldif import LDIFStatic
+model = LDIFStatic(
+    input_dim=128,
+    output_dim=1,
+    task_type='binary_classification',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.01
+)
+
+x = torch.randn(32, 128)
+logits = model(x)   # shape (32,)
+
+For regression, use task_type='regression'; for multiclass, task_type='multiclass' and set output_dim accordingly.
+
+### Sequential (time‑series) data
+
+LDIF can be implemented in python for sequential (temporal) data using this example code.
+
+from ldif import LDIFSequential
+model = LDIFSequential(
+    input_dim=8,
+    output_dim=1,
+    task_type='regression',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.2
+)
+
+# x: (batch, sequence_length, features)
+x = torch.randn(16, 50, 8)
+out = model(x)                 # shape (16,)
+
+## Default Hyperparameters
+
+The default hyperparameters for both `LDIFStatic` and `LDIFSequential` are explained here.
+
+### `LDIFStatic`
+
+LDIFStatic is tuned for static tabular datasets (e.g., regression or classification on feature vectors). The defaults are chosen to balance capacity and stability for moderate sized data (up to ~10k samples):
+
+num_layers=2: Two stacked LDIF blocks provide sufficient depth to capture non‑linear interactions without overfitting.
+k_max=16: The adaptive spectrum has a maximum rank of 16, which is ample for most tabular problems; the L1 penalty will prune unnecessary components.
+r=8: The gate projections use a rank of 8, offering a good trade‑off between routing expressiveness and parameter efficiency.
+gamma=0.01: A mild decay coefficient prevents unbounded growth of the hidden state while allowing long‑term information retention.
+input_scale=2.0: A relatively strong input injection boosts the signal from the raw features, helping the model quickly adapt to the data.
+z_init_mean=1.0: The spectrum logits start with a mean of 1.0, corresponding to sigmoid values around 0.73 – this encourages most components to be initially active, giving the model full capacity from the start.
+z_init_std=0.01: Very tight initialization ensures that the spectrum does not vary wildly early in training, promoting stable gradient flow.
+MU is a hyper parameter it is L1 penalty and explained in the paper. The mu is dependent on the dataset samples. It is preferred to define your N_train like:
+N_train = len(train_loader.dataset)   # if using DataLoader
+# or
+N_train = len(train_dataset)          # if using TensorDataset directly
+The number of samples calculate the value of mu, for proper implementation it is preferred to make the mu factor trained using optuna, some data may even require mu factor as low as 0.07 or lower so optuna is prefered to be used to find best mu factor or if optuna is not available or computationally expensive simply using grid search to find best mu factor with validation metrics being monitored is recommended. The default mu_factor in the code is 1.0 (the default argument in compute_mu(N_train, mu_factor=1.0)).
+This value corresponds to the base scaling recommended by the paper – i.e., mu = 1.82 * N_train^(-0.55). Users can easily override it by passing a different mu_factor (e.g., from Optuna tuning) when calling compute_mu().
+
+Important: mu itself is not hardcoded anywhere in the model; it must be computed during training and applied to the loss. The helper function simply provides the recommended formula. Users should call it before training, e.g.:
+
+mu = compute_mu(N_train, mu_factor=1.0)   # or any tuned value
+
+mu_factor=1.0 is a sensible starting point for most datasets.
+
+### LDIFSequential
+
+LDIFSequential is designed for time‑series data (e.g., sensor readings, financial sequences). Its defaults reflect the need for stable recurrent dynamics and awareness of temporal order:
+
+num_layers=2: Same as static – two layers are sufficient for most sequence lengths up to ~100 time steps.
+k_max=16: The spectrum capacity is kept at 16, which is enough for typical sequential patterns; the L1 penalty will still prune irrelevant components.
+r=8: Gate projection rank remains 8, offering sufficient flexibility for mixing symmetric and skew‑symmetric responses.
+gamma=0.2: A stronger decay coefficient is used to dissipate energy more quickly, preventing the hidden state from becoming unstable over long sequences.
+input_scale=0.05: Input forcing is much weaker compared to the static variant – this ensures that new observations gently influence the state without overwhelming the recurrent dynamics.
+z_init_mean=-2.0: The spectrum logits start with a mean of -2.0, corresponding to sigmoid values around 0.12 – this highly conservative initialization avoids sudden large updates early in training, which is crucial for stable recurrent learning.
+z_init_std=0.01: As with the static variant, the tight standard deviation keeps initial spectrum values close to each other, reducing variance in early gradients.
+use_positional_encoding=True: Positional encoding is enabled to inject temporal order information, helping the model distinguish between different time steps and better capture sequential patterns.
+mu is as above.
+
+## Hyperparameter Tuning
+
+The defaults work well for moderate‑sized datasets (up to ~10k samples for static, sequences up to length ~100). For larger or more complex data, we recommend tuning the following with Optuna or similar:
+
+k_max: 12–64 (larger for high‑capacity tasks)
+r: 6–24 (larger for richer gate interactions)
+num_layers: 1–4 (more layers for long‑range dependencies)
+gamma: 0.001–0.1 (static) or 0.05–0.5 (sequential) 
+lr (optimiser): typically 1e‑4 to 1e‑3
+weight_decay: 1e‑5 to 1e‑2
+mu (spectrum penalty coefficient): use mu = 1.82 * (N_train ** (-0.55)) * mu_factor, where mu_factor is tuned (e.g., 0.01–10).
+
+A good starting point for the L1 penalty is `mu = 0.01` for datasets with >10k samples, scaling down for smaller datasets.
+
+The initial draft of paper explaining the model is titled: 'LDIF: Latent dual interaction flow' currently published on arxiv. For more detail check the paper.
+
+## Difference between LDIFStatic and LDIFSequential
+The model architecture is same only difference between the import is that LDIFStatic is designed for static/tabular data. It uses a stronger input scaling (2.0) and lower decay (gamma=0.01) to capture feature interactions, with positional encoding disabled.
+LDIFSequential targets time-series data. It uses weaker input scaling (0.05), stronger decay (gamma=0.2) for stable recurrent dynamics, and enables positional encoding to preserve temporal order.
+For z_init_mean value or the initialization of the spectrum values the static varient has a value of 1.0 so the spectrum starts near 0.73, for sequential to promote stability and prevent any type of exploding gradients the value is set -2.0 so spectrum value starts near 0.12.  
+In our experiments these configuration for the initialization and input scaling were found to be properly working for respective data type.

ldif_model-0.1.0/README.md

@@ -0,0 +1,113 @@
+# LDIF: Latent Dual Interaction Flow
+
+**LDIF** is a PyTorch implementation of the Latent Dual Interaction Flow. It is a continuous time neural architecture with response conditioned gating and adaptive spectrum pruning.
+
+## Installation
+
+pip install ldif-model
+
+## Quick Start
+
+### Static (tabular) data
+
+LDIF can be implemented in python for static data using this example code.
+
+import torch
+from ldif import LDIFStatic
+model = LDIFStatic(
+    input_dim=128,
+    output_dim=1,
+    task_type='binary_classification',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.01
+)
+
+x = torch.randn(32, 128)
+logits = model(x)   # shape (32,)
+
+For regression, use task_type='regression'; for multiclass, task_type='multiclass' and set output_dim accordingly.
+
+### Sequential (time‑series) data
+
+LDIF can be implemented in python for sequential (temporal) data using this example code.
+
+from ldif import LDIFSequential
+model = LDIFSequential(
+    input_dim=8,
+    output_dim=1,
+    task_type='regression',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.2
+)
+
+# x: (batch, sequence_length, features)
+x = torch.randn(16, 50, 8)
+out = model(x)                 # shape (16,)
+
+## Default Hyperparameters
+
+The default hyperparameters for both `LDIFStatic` and `LDIFSequential` are explained here.
+
+### `LDIFStatic`
+
+LDIFStatic is tuned for static tabular datasets (e.g., regression or classification on feature vectors). The defaults are chosen to balance capacity and stability for moderate sized data (up to ~10k samples):
+
+num_layers=2: Two stacked LDIF blocks provide sufficient depth to capture non‑linear interactions without overfitting.
+k_max=16: The adaptive spectrum has a maximum rank of 16, which is ample for most tabular problems; the L1 penalty will prune unnecessary components.
+r=8: The gate projections use a rank of 8, offering a good trade‑off between routing expressiveness and parameter efficiency.
+gamma=0.01: A mild decay coefficient prevents unbounded growth of the hidden state while allowing long‑term information retention.
+input_scale=2.0: A relatively strong input injection boosts the signal from the raw features, helping the model quickly adapt to the data.
+z_init_mean=1.0: The spectrum logits start with a mean of 1.0, corresponding to sigmoid values around 0.73 – this encourages most components to be initially active, giving the model full capacity from the start.
+z_init_std=0.01: Very tight initialization ensures that the spectrum does not vary wildly early in training, promoting stable gradient flow.
+MU is a hyper parameter it is L1 penalty and explained in the paper. The mu is dependent on the dataset samples. It is preferred to define your N_train like:
+N_train = len(train_loader.dataset)   # if using DataLoader
+# or
+N_train = len(train_dataset)          # if using TensorDataset directly
+The number of samples calculate the value of mu, for proper implementation it is preferred to make the mu factor trained using optuna, some data may even require mu factor as low as 0.07 or lower so optuna is prefered to be used to find best mu factor or if optuna is not available or computationally expensive simply using grid search to find best mu factor with validation metrics being monitored is recommended. The default mu_factor in the code is 1.0 (the default argument in compute_mu(N_train, mu_factor=1.0)).
+This value corresponds to the base scaling recommended by the paper – i.e., mu = 1.82 * N_train^(-0.55). Users can easily override it by passing a different mu_factor (e.g., from Optuna tuning) when calling compute_mu().
+
+Important: mu itself is not hardcoded anywhere in the model; it must be computed during training and applied to the loss. The helper function simply provides the recommended formula. Users should call it before training, e.g.:
+
+mu = compute_mu(N_train, mu_factor=1.0)   # or any tuned value
+
+mu_factor=1.0 is a sensible starting point for most datasets.
+
+### LDIFSequential
+
+LDIFSequential is designed for time‑series data (e.g., sensor readings, financial sequences). Its defaults reflect the need for stable recurrent dynamics and awareness of temporal order:
+
+num_layers=2: Same as static – two layers are sufficient for most sequence lengths up to ~100 time steps.
+k_max=16: The spectrum capacity is kept at 16, which is enough for typical sequential patterns; the L1 penalty will still prune irrelevant components.
+r=8: Gate projection rank remains 8, offering sufficient flexibility for mixing symmetric and skew‑symmetric responses.
+gamma=0.2: A stronger decay coefficient is used to dissipate energy more quickly, preventing the hidden state from becoming unstable over long sequences.
+input_scale=0.05: Input forcing is much weaker compared to the static variant – this ensures that new observations gently influence the state without overwhelming the recurrent dynamics.
+z_init_mean=-2.0: The spectrum logits start with a mean of -2.0, corresponding to sigmoid values around 0.12 – this highly conservative initialization avoids sudden large updates early in training, which is crucial for stable recurrent learning.
+z_init_std=0.01: As with the static variant, the tight standard deviation keeps initial spectrum values close to each other, reducing variance in early gradients.
+use_positional_encoding=True: Positional encoding is enabled to inject temporal order information, helping the model distinguish between different time steps and better capture sequential patterns.
+mu is as above.
+
+## Hyperparameter Tuning
+
+The defaults work well for moderate‑sized datasets (up to ~10k samples for static, sequences up to length ~100). For larger or more complex data, we recommend tuning the following with Optuna or similar:
+
+k_max: 12–64 (larger for high‑capacity tasks)
+r: 6–24 (larger for richer gate interactions)
+num_layers: 1–4 (more layers for long‑range dependencies)
+gamma: 0.001–0.1 (static) or 0.05–0.5 (sequential) 
+lr (optimiser): typically 1e‑4 to 1e‑3
+weight_decay: 1e‑5 to 1e‑2
+mu (spectrum penalty coefficient): use mu = 1.82 * (N_train ** (-0.55)) * mu_factor, where mu_factor is tuned (e.g., 0.01–10).
+
+A good starting point for the L1 penalty is `mu = 0.01` for datasets with >10k samples, scaling down for smaller datasets.
+
+The initial draft of paper explaining the model is titled: 'LDIF: Latent dual interaction flow' currently published on arxiv. For more detail check the paper.
+
+## Difference between LDIFStatic and LDIFSequential
+The model architecture is same only difference between the import is that LDIFStatic is designed for static/tabular data. It uses a stronger input scaling (2.0) and lower decay (gamma=0.01) to capture feature interactions, with positional encoding disabled.
+LDIFSequential targets time-series data. It uses weaker input scaling (0.05), stronger decay (gamma=0.2) for stable recurrent dynamics, and enables positional encoding to preserve temporal order.
+For z_init_mean value or the initialization of the spectrum values the static varient has a value of 1.0 so the spectrum starts near 0.73, for sequential to promote stability and prevent any type of exploding gradients the value is set -2.0 so spectrum value starts near 0.12.  
+In our experiments these configuration for the initialization and input scaling were found to be properly working for respective data type.

ldif_model-0.1.0/ldif_model/__init__.py

@@ -0,0 +1 @@
+from .model import LDIFStatic, LDIFSequential, LDIFModel, LDIFBlock, compute_mu, compile_model

ldif_model-0.1.0/ldif_model/model.py

@@ -0,0 +1,321 @@
+"""
+LDIF: Latent Dual Interaction Flow – PyTorch implementation.
+
+Provides LDIFBlock, LDIFModel, LDIFStatic, and LDIFSequential.
+"""
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import math
+import os
+
+# Performance tweaks
+if torch.cuda.is_available():
+    torch.set_float32_matmul_precision('high')
+    torch.backends.cudnn.benchmark = True
+else:
+    try:
+        n_threads = max(1, os.cpu_count() - 1)
+        torch.set_num_threads(n_threads)
+    except:
+        pass
+
+
+class LDIFBlock(nn.Module):
+    """
+    Single LDIF layer with full gate.
+    """
+    def __init__(self, d, k_max, r, gamma=0.01, z_init_mean=1.0, z_init_std=0.01):
+        super().__init__()
+        self.d = d
+        self.k_max = k_max
+        self.r = r
+        self.gamma = gamma
+
+        # Spectrum: s = sigma(z)
+        self.z = nn.Parameter(torch.randn(k_max) * z_init_std + z_init_mean)
+
+        # Symmetric branch
+        U_S_raw = torch.empty(k_max, d)
+        V_S_raw = torch.empty(k_max, d)
+        nn.init.kaiming_uniform_(U_S_raw, a=math.sqrt(5))
+        nn.init.kaiming_uniform_(V_S_raw, a=math.sqrt(5))
+        self.U_S = nn.Parameter(U_S_raw.T)
+        self.V_S = nn.Parameter(V_S_raw.T)
+
+        # Skew-symmetric branch
+        U_A_raw = torch.empty(k_max, d)
+        V_A_raw = torch.empty(k_max, d)
+        nn.init.kaiming_uniform_(U_A_raw, a=math.sqrt(5))
+        nn.init.kaiming_uniform_(V_A_raw, a=math.sqrt(5))
+        self.U_A = nn.Parameter(U_A_raw.T)
+        self.V_A = nn.Parameter(V_A_raw.T)
+
+        # Gate projections
+        V_h_raw = torch.empty(r, d)
+        V_Sr_raw = torch.empty(r, d)
+        V_Ar_raw = torch.empty(r, d)
+        nn.init.kaiming_uniform_(V_h_raw, a=math.sqrt(5))
+        nn.init.kaiming_uniform_(V_Sr_raw, a=math.sqrt(5))
+        nn.init.kaiming_uniform_(V_Ar_raw, a=math.sqrt(5))
+        self.V_h = nn.Parameter(V_h_raw.T)
+        self.V_Sr = nn.Parameter(V_Sr_raw.T)
+        self.V_Ar = nn.Parameter(V_Ar_raw.T)
+
+        U_h_raw = torch.empty(d, r)
+        U_Sr_raw = torch.empty(d, r)
+        U_Ar_raw = torch.empty(d, r)
+        std_h = 1.0 / math.sqrt(r)
+        bound_h = math.sqrt(3.0) * std_h
+        U_h_raw.uniform_(-bound_h, bound_h)
+        U_Sr_raw.uniform_(-bound_h, bound_h)
+        U_Ar_raw.uniform_(-bound_h, bound_h)
+        self.U_h = nn.Parameter(U_h_raw)
+        self.U_Sr = nn.Parameter(U_Sr_raw)
+        self.U_Ar = nn.Parameter(U_Ar_raw)
+        self.b_g = nn.Parameter(torch.zeros(d))
+
+        # Bounded step-size parameters
+        self.alpha_raw = nn.Parameter(torch.ones(d))
+        self.lambda_raw = nn.Parameter(torch.tensor(0.0))
+
+    @property
+    def s(self):
+        return torch.sigmoid(self.z)
+
+    @property
+    def S2(self):
+        return self.s ** 2
+
+    @property
+    def alpha(self):
+        return torch.clamp(F.softplus(self.alpha_raw), min=0.5, max=2.0)
+
+    @property
+    def lambda_(self):
+        return 0.05 + 0.45 * torch.sigmoid(self.lambda_raw)
+
+    def _ensure_batch(self, h):
+        if h.dim() == 1:
+            return h.unsqueeze(0), True
+        return h, False
+
+    def _restore_batch(self, h, was_1d):
+        if was_1d:
+            return h.squeeze(0)
+        return h
+
+    def forward(self, h):
+        h, was_1d = self._ensure_batch(h)
+        s2 = self.S2
+
+        # Symmetric response
+        U_S_h = torch.matmul(h, self.U_S)
+        V_S_h = torch.matmul(h, self.V_S)
+        u_scaled = U_S_h * s2.unsqueeze(0)
+        v_scaled = V_S_h * s2.unsqueeze(0)
+        r_S = torch.matmul(u_scaled, self.V_S.T) + torch.matmul(v_scaled, self.U_S.T)
+
+        # Skew-symmetric response
+        U_A_h = torch.matmul(h, self.U_A)
+        V_A_h = torch.matmul(h, self.V_A)
+        uA_scaled = U_A_h * s2.unsqueeze(0)
+        vA_scaled = V_A_h * s2.unsqueeze(0)
+        r_A = torch.matmul(uA_scaled, self.V_A.T) - torch.matmul(vA_scaled, self.U_A.T)
+
+        # Gating
+        gate_in = torch.matmul(h, self.V_h) @ self.U_h.T
+        gate_in += torch.matmul(r_S, self.V_Sr) @ self.U_Sr.T
+        gate_in += torch.matmul(r_A, self.V_Ar) @ self.U_Ar.T
+        gate_in += self.b_g
+        g = torch.sigmoid(gate_in)
+        R = g * r_S + (1 - g) * r_A
+
+        # Master update
+        lambda_ = self.lambda_
+        alpha = self.alpha
+        h_next = h + lambda_ * alpha * torch.tanh(R) - lambda_ * self.gamma * h
+
+        intermediates = {
+            'r_S': r_S, 'r_A': r_A, 'gate': g, 'R': R,
+            's': self.s, 'alpha': alpha, 'lambda': lambda_,
+            'z': self.z, 'h_next': h_next,
+        }
+        return self._restore_batch(h_next, was_1d), intermediates
+
+    def spectrum_penalty(self):
+        return torch.sum(self.s)
+
+    def mean_spectrum(self):
+        return self.s.mean().item()
+
+    def effective_rank(self, s_threshold=0.1):
+        return (self.s > s_threshold).sum().item()
+
+
+class LDIFModel(nn.Module):
+    """
+    Base LDIF model supporting both static and sequential data.
+    """
+    def __init__(self, input_dim, output_dim, task_type,
+                 num_layers=2, k_max=16, r=8, gamma=0.01,
+                 input_scale=2.0, z_init_mean=1.0, z_init_std=0.01,
+                 use_positional_encoding=True, use_random_init=True):
+        super().__init__()
+        self.input_dim = input_dim
+        self.output_dim = output_dim
+        self.task_type = task_type
+        self.num_layers = num_layers
+        self.use_positional_encoding = use_positional_encoding
+        self.use_random_init = use_random_init
+
+        self.input_proj = nn.Linear(input_dim, input_dim)
+        self.input_scale = nn.Parameter(torch.ones(1) * input_scale)
+
+        if use_positional_encoding:
+            self.pos_embed = nn.Linear(1, input_dim)
+            nn.init.normal_(self.pos_embed.weight, std=0.02)
+            nn.init.zeros_(self.pos_embed.bias)
+        else:
+            self.pos_embed = None
+
+        self.blocks = nn.ModuleList([
+            LDIFBlock(input_dim, k_max, r,
+                      gamma=gamma,
+                      z_init_mean=z_init_mean,
+                      z_init_std=z_init_std)
+            for _ in range(num_layers)
+        ])
+
+        if task_type == 'regression':
+            self.head = nn.Linear(input_dim, 1)
+        elif task_type in ['binary_classification', 'imbalanced_binary']:
+            self.head = nn.Linear(input_dim, 1)
+        elif task_type == 'multiclass':
+            self.head = nn.Linear(input_dim, output_dim)
+        else:
+            self.head = nn.Linear(input_dim, output_dim)
+
+        nn.init.xavier_uniform_(self.head.weight)
+        if self.head.bias is not None:
+            nn.init.zeros_(self.head.bias)
+
+    def forward(self, x, return_intermediates=False):
+        is_sequential = x.dim() == 3
+        all_intermediates = {}
+
+        if not is_sequential:
+            h = x
+            for idx, block in enumerate(self.blocks):
+                h, inter = block(h)
+                if return_intermediates:
+                    all_intermediates[f'layer_{idx}'] = {k: v.detach() if isinstance(v, torch.Tensor) else v for k, v in inter.items()}
+            out = self.head(h)
+            if self.task_type in ['regression', 'binary_classification', 'imbalanced_binary']:
+                out = out.view(-1)
+            return (out, all_intermediates) if return_intermediates else out
+
+        # Sequential path
+        B, T, D = x.shape
+        if self.use_random_init:
+            h = torch.randn(B, D, device=x.device) * 0.01
+        else:
+            h = torch.zeros(B, D, device=x.device)
+
+        if self.use_positional_encoding and self.pos_embed is not None:
+            time_indices = torch.arange(1, T+1, device=x.device).float().view(1, T, 1)
+            pos_enc = self.pos_embed(time_indices)
+            pos_enc_exp = pos_enc.expand(B, -1, -1)
+        else:
+            pos_enc_exp = None
+
+        for t in range(T):
+            x_t = x[:, t, :]
+            if pos_enc_exp is not None:
+                x_t = x_t + pos_enc_exp[:, t, :]
+            x_t = torch.tanh(self.input_proj(x_t))
+            h = h + self.input_scale * x_t
+
+            for idx, block in enumerate(self.blocks):
+                h, inter = block(h)
+                if return_intermediates and (t == 0 or t == T-1):
+                    time_tag = 't0' if t == 0 else 't_end'
+                    all_intermediates[f'{time_tag}_layer_{idx}'] = {
+                        k: v.detach() if isinstance(v, torch.Tensor) else v for k, v in inter.items()
+                    }
+        out = self.head(h)
+        if self.task_type in ['regression', 'binary_classification', 'imbalanced_binary']:
+            out = out.view(-1)
+        return (out, all_intermediates) if return_intermediates else out
+
+    def spectrum_penalty(self):
+        return torch.stack([block.spectrum_penalty() for block in self.blocks]).sum()
+
+    def get_mean_spectrum(self):
+        return [block.mean_spectrum() for block in self.blocks]
+
+
+class LDIFStatic(LDIFModel):
+    """
+    LDIF for static (tabular) data with recommended defaults.
+    """
+    def __init__(self, input_dim, output_dim, task_type,
+                 num_layers=2, k_max=16, r=8, gamma=0.01):
+        super().__init__(
+            input_dim=input_dim,
+            output_dim=output_dim,
+            task_type=task_type,
+            num_layers=num_layers,
+            k_max=k_max,
+            r=r,
+            gamma=gamma,
+            input_scale=2.0,
+            z_init_mean=1.0,
+            z_init_std=0.01,
+            use_positional_encoding=False,
+            use_random_init=True
+        )
+
+
+class LDIFSequential(LDIFModel):
+    """
+    LDIF for sequential (time-series) data with recommended defaults.
+    """
+    def __init__(self, input_dim, output_dim, task_type,
+                 num_layers=2, k_max=16, r=8, gamma=0.2):
+        super().__init__(
+            input_dim=input_dim,
+            output_dim=output_dim,
+            task_type=task_type,
+            num_layers=num_layers,
+            k_max=k_max,
+            r=r,
+            gamma=gamma,
+            input_scale=0.05,
+            z_init_mean=-2.0,
+            z_init_std=0.01,
+            use_positional_encoding=True,
+            use_random_init=False
+        )
+
+
+def compute_mu(N_train, mu_factor=1.0):
+    """
+    Compute L1 penalty coefficient using: mu = 1.82 * N_train^(-0.55) * mu_factor
+    """
+    return 1.82 * (N_train ** (-0.55)) * mu_factor
+
+
+def compile_model(model, mode='default'):
+    """
+    Compile the model with torch.compile for faster execution.
+    """
+    if hasattr(torch, 'compile'):
+        try:
+            return torch.compile(model, mode=mode, fullgraph=False)
+        except Exception as e:
+            print(f"torch.compile failed: {e}. Using uncompiled model.")
+            return model
+    else:
+        return model

ldif_model-0.1.0/ldif_model.egg-info/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: ldif-model
+Version: 0.1.0
+Summary: Latent Dual Interaction Flow neural architecture with response-conditioned gating and adaptive spectrum pruning.
+Author: Muhammad Muhaimin, Muhammad Waseem Ashraf, Shahzadi Tayyaba
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/engrdrwaseem
+Project-URL: Repository, https://github.com/yourusername/ldif
+Project-URL: Issues, https://github.com/yourusername/ldif/issues
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: License
+Requires-Dist: torch>=1.9.0
+Requires-Dist: numpy>=1.19.0
+Dynamic: license-file
+
+# LDIF: Latent Dual Interaction Flow
+
+**LDIF** is a PyTorch implementation of the Latent Dual Interaction Flow. It is a continuous time neural architecture with response conditioned gating and adaptive spectrum pruning.
+
+## Installation
+
+pip install ldif-model
+
+## Quick Start
+
+### Static (tabular) data
+
+LDIF can be implemented in python for static data using this example code.
+
+import torch
+from ldif import LDIFStatic
+model = LDIFStatic(
+    input_dim=128,
+    output_dim=1,
+    task_type='binary_classification',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.01
+)
+
+x = torch.randn(32, 128)
+logits = model(x)   # shape (32,)
+
+For regression, use task_type='regression'; for multiclass, task_type='multiclass' and set output_dim accordingly.
+
+### Sequential (time‑series) data
+
+LDIF can be implemented in python for sequential (temporal) data using this example code.
+
+from ldif import LDIFSequential
+model = LDIFSequential(
+    input_dim=8,
+    output_dim=1,
+    task_type='regression',
+    num_layers=2,
+    k_max=16,
+    r=8,
+    gamma=0.2
+)
+
+# x: (batch, sequence_length, features)
+x = torch.randn(16, 50, 8)
+out = model(x)                 # shape (16,)
+
+## Default Hyperparameters
+
+The default hyperparameters for both `LDIFStatic` and `LDIFSequential` are explained here.
+
+### `LDIFStatic`
+
+LDIFStatic is tuned for static tabular datasets (e.g., regression or classification on feature vectors). The defaults are chosen to balance capacity and stability for moderate sized data (up to ~10k samples):
+
+num_layers=2: Two stacked LDIF blocks provide sufficient depth to capture non‑linear interactions without overfitting.
+k_max=16: The adaptive spectrum has a maximum rank of 16, which is ample for most tabular problems; the L1 penalty will prune unnecessary components.
+r=8: The gate projections use a rank of 8, offering a good trade‑off between routing expressiveness and parameter efficiency.
+gamma=0.01: A mild decay coefficient prevents unbounded growth of the hidden state while allowing long‑term information retention.
+input_scale=2.0: A relatively strong input injection boosts the signal from the raw features, helping the model quickly adapt to the data.
+z_init_mean=1.0: The spectrum logits start with a mean of 1.0, corresponding to sigmoid values around 0.73 – this encourages most components to be initially active, giving the model full capacity from the start.
+z_init_std=0.01: Very tight initialization ensures that the spectrum does not vary wildly early in training, promoting stable gradient flow.
+MU is a hyper parameter it is L1 penalty and explained in the paper. The mu is dependent on the dataset samples. It is preferred to define your N_train like:
+N_train = len(train_loader.dataset)   # if using DataLoader
+# or
+N_train = len(train_dataset)          # if using TensorDataset directly
+The number of samples calculate the value of mu, for proper implementation it is preferred to make the mu factor trained using optuna, some data may even require mu factor as low as 0.07 or lower so optuna is prefered to be used to find best mu factor or if optuna is not available or computationally expensive simply using grid search to find best mu factor with validation metrics being monitored is recommended. The default mu_factor in the code is 1.0 (the default argument in compute_mu(N_train, mu_factor=1.0)).
+This value corresponds to the base scaling recommended by the paper – i.e., mu = 1.82 * N_train^(-0.55). Users can easily override it by passing a different mu_factor (e.g., from Optuna tuning) when calling compute_mu().
+
+Important: mu itself is not hardcoded anywhere in the model; it must be computed during training and applied to the loss. The helper function simply provides the recommended formula. Users should call it before training, e.g.:
+
+mu = compute_mu(N_train, mu_factor=1.0)   # or any tuned value
+
+mu_factor=1.0 is a sensible starting point for most datasets.
+
+### LDIFSequential
+
+LDIFSequential is designed for time‑series data (e.g., sensor readings, financial sequences). Its defaults reflect the need for stable recurrent dynamics and awareness of temporal order:
+
+num_layers=2: Same as static – two layers are sufficient for most sequence lengths up to ~100 time steps.
+k_max=16: The spectrum capacity is kept at 16, which is enough for typical sequential patterns; the L1 penalty will still prune irrelevant components.
+r=8: Gate projection rank remains 8, offering sufficient flexibility for mixing symmetric and skew‑symmetric responses.
+gamma=0.2: A stronger decay coefficient is used to dissipate energy more quickly, preventing the hidden state from becoming unstable over long sequences.
+input_scale=0.05: Input forcing is much weaker compared to the static variant – this ensures that new observations gently influence the state without overwhelming the recurrent dynamics.
+z_init_mean=-2.0: The spectrum logits start with a mean of -2.0, corresponding to sigmoid values around 0.12 – this highly conservative initialization avoids sudden large updates early in training, which is crucial for stable recurrent learning.
+z_init_std=0.01: As with the static variant, the tight standard deviation keeps initial spectrum values close to each other, reducing variance in early gradients.
+use_positional_encoding=True: Positional encoding is enabled to inject temporal order information, helping the model distinguish between different time steps and better capture sequential patterns.
+mu is as above.
+
+## Hyperparameter Tuning
+
+The defaults work well for moderate‑sized datasets (up to ~10k samples for static, sequences up to length ~100). For larger or more complex data, we recommend tuning the following with Optuna or similar:
+
+k_max: 12–64 (larger for high‑capacity tasks)
+r: 6–24 (larger for richer gate interactions)
+num_layers: 1–4 (more layers for long‑range dependencies)
+gamma: 0.001–0.1 (static) or 0.05–0.5 (sequential) 
+lr (optimiser): typically 1e‑4 to 1e‑3
+weight_decay: 1e‑5 to 1e‑2
+mu (spectrum penalty coefficient): use mu = 1.82 * (N_train ** (-0.55)) * mu_factor, where mu_factor is tuned (e.g., 0.01–10).
+
+A good starting point for the L1 penalty is `mu = 0.01` for datasets with >10k samples, scaling down for smaller datasets.
+
+The initial draft of paper explaining the model is titled: 'LDIF: Latent dual interaction flow' currently published on arxiv. For more detail check the paper.
+
+## Difference between LDIFStatic and LDIFSequential
+The model architecture is same only difference between the import is that LDIFStatic is designed for static/tabular data. It uses a stronger input scaling (2.0) and lower decay (gamma=0.01) to capture feature interactions, with positional encoding disabled.
+LDIFSequential targets time-series data. It uses weaker input scaling (0.05), stronger decay (gamma=0.2) for stable recurrent dynamics, and enables positional encoding to preserve temporal order.
+For z_init_mean value or the initialization of the spectrum values the static varient has a value of 1.0 so the spectrum starts near 0.73, for sequential to promote stability and prevent any type of exploding gradients the value is set -2.0 so spectrum value starts near 0.12.  
+In our experiments these configuration for the initialization and input scaling were found to be properly working for respective data type.

ldif_model-0.1.0/ldif_model.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+License
+README.md
+pyproject.toml
+ldif_model/__init__.py
+ldif_model/model.py
+ldif_model.egg-info/PKG-INFO
+ldif_model.egg-info/SOURCES.txt
+ldif_model.egg-info/dependency_links.txt
+ldif_model.egg-info/requires.txt
+ldif_model.egg-info/top_level.txt

ldif_model-0.1.0/ldif_model.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

ldif_model-0.1.0/ldif_model.egg-info/requires.txt

@@ -0,0 +1,2 @@
+torch>=1.9.0
+numpy>=1.19.0

ldif_model-0.1.0/ldif_model.egg-info/top_level.txt

@@ -0,0 +1 @@
+ldif_model

ldif_model-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ldif-model"
+version = "0.1.0"
+description = "Latent Dual Interaction Flow neural architecture with response-conditioned gating and adaptive spectrum pruning."
+readme = "README.md"
+authors = [
+    {name = "Muhammad Muhaimin"},
+    {name = "Muhammad Waseem Ashraf"},
+    {name = "Shahzadi Tayyaba"}
+]
+license = "MIT"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence"
+]
+requires-python = ">=3.8"
+dependencies = [
+    "torch>=1.9.0",
+    "numpy>=1.19.0"
+]
+
+[project.urls]
+Homepage = "https://github.com/engrdrwaseem"
+Repository = "https://github.com/yourusername/ldif"
+Issues = "https://github.com/yourusername/ldif/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["ldif*"]

ldif_model-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+