l0-python 0.4.0__py3-none-any.whl → 0.4.2__py3-none-any.whl

l0/calibration.py

@@ -444,9 +444,9 @@ class SparseCalibrationWeights(nn.Module):
                                 rel_errors[group_mask].mean().item()
                             )
                             group_losses.append(group_mean_err)
-                        mean_group_loss = np.mean(group_losses)
+                        mean_group_mare = np.mean(group_losses)
                     else:
-                        mean_group_loss = mean_rel_err
+                        mean_group_mare = mean_rel_err
 
                     # Calculate sparsity percentage
                     sparsity_pct = 100 * (
@@ -461,7 +461,7 @@ class SparseCalibrationWeights(nn.Module):
                     if target_groups is not None:
                         print(
                             f"Epoch {epoch+1:4d}: "
-                            f"mean_group_loss={mean_group_loss:.1%}, "
+                            f"mean_group_mare={mean_group_mare:.4%}, "
                             f"max_error={max_rel_err:.1%}, "
                             f"total_loss={actual_total_loss:.3f}, "
                             f"active={active_info['count']:4d}/{self.n_features} ({sparsity_pct:.1f}% sparse)"
@@ -469,7 +469,7 @@ class SparseCalibrationWeights(nn.Module):
                     else:
                         print(
                             f"Epoch {epoch+1:4d}: "
-                            f"mean_error={mean_rel_err:.1%}, "
+                            f"mean_error={mean_rel_err:.4%}, "
                             f"max_error={max_rel_err:.1%}, "
                             f"total_loss={actual_total_loss:.3f}, "
                             f"active={active_info['count']:4d}/{self.n_features} ({sparsity_pct:.1f}% sparse)"

{l0_python-0.4.0.dist-info → l0_python-0.4.2.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: l0-python
-Version: 0.4.0
+Version: 0.4.2
 Summary: L0 regularization for sparse neural networks and intelligent sampling
 Author-email: PolicyEngine <hello@policyengine.org>
 License: MIT License
@@ -51,7 +51,7 @@ Requires-Dist: scikit-learn>=1.0; extra == "dev"
 Requires-Dist: yaml-changelog>=0.3.0; extra == "dev"
 Requires-Dist: twine>=4.0.0; extra == "dev"
 Provides-Extra: docs
-Requires-Dist: mystmd>=1.3; extra == "docs"
+Requires-Dist: jupyter-book>=1.0; extra == "docs"
 Requires-Dist: jupyter>=1.0; extra == "docs"
 Requires-Dist: notebook>=7.0; extra == "docs"
 Requires-Dist: ipykernel>=6.0; extra == "docs"
@@ -65,21 +65,29 @@ Dynamic: license-file
 
 # L0 Regularization
 
-A PyTorch implementation of L0 regularization for neural network sparsification and intelligent sampling, based on [Louizos, Welling, & Kingma (2017)](https://arxiv.org/abs/1712.01312).
+[![PyPI version](https://badge.fury.io/py/l0-python.svg)](https://pypi.org/project/l0-python/)
+[![CI](https://github.com/PolicyEngine/L0/actions/workflows/push.yml/badge.svg)](https://github.com/PolicyEngine/L0/actions)
 
-## Features
+A PyTorch implementation of L0 regularization based on [Louizos, Welling, & Kingma (2017)](https://arxiv.org/abs/1712.01312), with a **critical bug fix** to the original authors' implementation.
 
-- **Hard Concrete Distribution**: Differentiable approximation of L0 norm
-- **Sparse Neural Network Layers**: L0Linear, L0Conv2d with automatic pruning
-- **Intelligent Sampling**: Sample/feature selection gates for calibration
-- **L0L2 Combined Penalty**: Recommended approach to prevent overfitting
-- **Temperature Scheduling**: Annealing for improved convergence
-- **TDD Development**: Comprehensive test coverage
+## Why This Package?
+
+The original L0 implementation from AMLab-Amsterdam contains a bug in test-time gate computation where the temperature parameter is incorrectly omitted. This prevents proper sparsity and degrades performance. Our implementation corrects this error:
+
+```python
+# Original (buggy): gates never fully close
+pi = sigmoid(log_alpha)
+
+# Corrected: temperature division required for proper sparsity
+pi = sigmoid(log_alpha / temperature)
+```
+
+This fix enables gates to achieve true 0/1 values, producing exact sparsity as intended by the L0 formulation.
 
 ## Installation
 
 ```bash
-pip install l0
+pip install l0-python
 ```
 
 For development:
@@ -89,21 +97,76 @@ cd L0
 pip install -e .[dev]
 ```
 
-## Quick Start
+## Primary Use Case: Survey Calibration
+
+This package was developed for PolicyEngine's survey calibration system, where we select a sparse subset of survey households while matching population targets.
+
+```python
+import numpy as np
+from scipy import sparse as sp
+from l0.calibration import SparseCalibrationWeights
+
+# Setup: Q targets, N households
+Q, N = 200, 10000
+M = sp.random(Q, N, density=0.3, format="csr")  # Household characteristics
+y = np.random.uniform(1e6, 1e8, size=Q)          # Population targets
+
+# Initialize model
+model = SparseCalibrationWeights(
+    n_features=N,
+    beta=0.35,
+    gamma=-0.1,
+    zeta=1.1,
+    init_keep_prob=0.5,      # Start with 50% active probability
+    init_weights=1.0,        # Or pass array of initial weights
+    log_weight_jitter_sd=0.05,
+    device="cuda",           # GPU acceleration
+)
+
+# Train with L0+L2 regularization
+model.fit(
+    M=M,
+    y=y,
+    lambda_l0=1e-6,          # Controls sparsity level
+    lambda_l2=1e-8,          # Prevents weight explosion
+    lr=0.15,
+    epochs=2000,
+    loss_type="relative",    # Scale-invariant loss
+    verbose=True,
+    verbose_freq=100,
+)
+
+# Get results
+active = model.get_active_weights()
+print(f"Selected {active['count']} of {N} households ({100*active['count']/N:.1f}%)")
+print(f"Sparsity: {model.get_sparsity():.1%}")
+
+# Predict calibrated totals
+y_pred = model.predict(M)
+```
+
+### Key Features for Calibration
+
+- **Non-negative weights**: All weights constrained to be positive via log-space parameterization
+- **Sparse solutions**: L0 penalty directly minimizes the count of active weights
+- **Relative loss**: Scale-invariant loss for targets spanning orders of magnitude
+- **Group-wise averaging**: Balance loss contributions across target groups with different cardinalities
+- **GPU support**: CUDA acceleration for large-scale problems
+
+## Neural Network Sparsification
 
-### Neural Network Sparsification
+The package also supports traditional neural network pruning:
 
 ```python
 import torch
 from l0 import L0Linear, compute_l0l2_penalty, TemperatureScheduler, update_temperatures
 
-# Create a sparse model
 class SparseModel(torch.nn.Module):
     def __init__(self):
         super().__init__()
         self.fc1 = L0Linear(784, 256, init_sparsity=0.5)
         self.fc2 = L0Linear(256, 10, init_sparsity=0.7)
-    
+
     def forward(self, x):
         x = torch.relu(self.fc1(x))
         return self.fc2(x)
@@ -112,95 +175,71 @@ model = SparseModel()
 optimizer = torch.optim.Adam(model.parameters())
 scheduler = TemperatureScheduler(initial_temp=1.0, final_temp=0.1)
 
-# Training loop
 for epoch in range(100):
-    # Update temperature
     temp = scheduler.get_temperature(epoch)
     update_temperatures(model, temp)
-    
-    # Forward pass
+
     output = model(input_data)
     ce_loss = criterion(output, target)
-    
-    # Add L0L2 penalty
     penalty = compute_l0l2_penalty(model, l0_lambda=1e-3, l2_lambda=1e-4)
     loss = ce_loss + penalty
-    
-    # Backward pass
+
     optimizer.zero_grad()
     loss.backward()
     optimizer.step()
 ```
 
-### Intelligent Sample Selection
+### Available Layers
+
+- `L0Linear`: Fully connected layer with L0 gates on weights
+- `L0Conv2d`: 2D convolution with channel-wise L0 gates
+- `L0DepthwiseConv2d`: Depthwise convolution with L0 gates
+- `SparseMLP`: Multi-layer perceptron with built-in L0 regularization
+
+## Intelligent Sampling Gates
+
+Standalone gates for sample/feature selection:
 
 ```python
-from l0 import SampleGate, HybridGate
+from l0 import SampleGate, FeatureGate, HybridGate
 
-# Pure L0 selection
+# Select samples via learned gates
 gate = SampleGate(n_samples=10000, target_samples=1000)
 selected_data, indices = gate.select_samples(data)
 
-# Hybrid selection (25% L0, 75% random)
+# Select features
+gate = FeatureGate(n_features=1000, max_features=50)
+selected_data, indices = gate.select_features(data)
+
+# Hybrid: combine L0 selection with random sampling
 hybrid = HybridGate(
     n_items=10000,
-    l0_fraction=0.25,
-    random_fraction=0.75,
-    target_items=1000
+    l0_fraction=0.25,    # 25% via learned L0 gates
+    random_fraction=0.75 # 75% random for coverage
 )
 selected, indices, types = hybrid.select(data)
 ```
 
-### Feature Selection
-
-```python
-from l0 import FeatureGate
-
-# Select top features
-gate = FeatureGate(n_features=1000, max_features=50)
-selected_data, feature_indices = gate.select_features(data)
-
-# Get feature importance
-importance = gate.get_feature_importance()
-```
-
-## Integration with PolicyEngine
-
-This package is designed to work with PolicyEngine's calibration system:
-
-```python
-# In policyengine-us-data or similar
-from l0 import HardConcrete
-
-# Use for household selection in CPS calibration
-gates = HardConcrete(
-    len(household_weights),
-    temperature=0.25,
-    init_mean=0.999  # Start with most households
-)
+## How L0 Regularization Works
 
-# Apply gates during reweighting
-masked_weights = weights * gates()
-```
+Unlike post-hoc pruning (setting small weights to zero), L0 regularization integrates sparsity into the training objective:
 
-## Documentation
+1. **Stochastic gates**: Each weight has a learned gate parameter controlling activation probability
+2. **Hard Concrete distribution**: Enables differentiable 0/1 gate values during training
+3. **Expected L0 penalty**: Minimizes the expected number of active gates
+4. **Temperature annealing**: Gradually sharpens gates from soft to hard decisions
 
-Full documentation available at: https://policyengine.github.io/L0/
+The result: the network learns *which* weights should be zero as part of optimization, not as a post-processing step.
 
 ## Testing
 
-Run tests with:
 ```bash
 pytest tests/ -v --cov=l0
 ```
 
-## Acknowledgments
-
-This implementation is inspired by and builds upon the [original L0 regularization code](https://github.com/AMLab-Amsterdam/L0_regularization) by AMLab Amsterdam, which accompanied the paper by Louizos et al. (2018).
-
 ## Citation
 
-If you use this package, please cite:
+If you use this package, please cite the original paper:
 
 ```bibtex
 @article{louizos2017learning,
@@ -213,4 +252,4 @@ If you use this package, please cite:
 
 ## License
 
-MIT License - see [LICENSE](LICENSE) file for details.
+MIT License - see [LICENSE](LICENSE) for details.

{l0_python-0.4.0.dist-info → l0_python-0.4.2.dist-info}/RECORD

@@ -1,12 +1,12 @@
 l0/__init__.py,sha256=ezGVLGbdU_jAElAJSdLKjSWCnZRILAZi8qMBPjcHwQc,1192
-l0/calibration.py,sha256=EnIhQPkC6yvRp0-o1qG3SCVUp6pIqwkqkx3DX3uWdG0,17838
+l0/calibration.py,sha256=3VM-28M1yp-DM6AlHX7sqZCaE7OJevpY9Ey-syjSPik,17838
 l0/distributions.py,sha256=GvVoDasdxoDlPPdzt33lFzkuJgQsfkEn9e-abxru3nM,6325
 l0/gates.py,sha256=54OORMg9qSsdbIyzYIL9t9CCHWsKbumSKl_d4-YFuNU,11443
 l0/layers.py,sha256=j_DPEFdfUc37ojtoK2ufGzVYzZ_DwusqTcJXCVxZS68,13785
 l0/penalties.py,sha256=okqXgX2eGUmg5prcDclicTQALPPPF9FA5eQP09V4GTE,9569
 l0/sparse.py,sha256=UVpkrrJBnQ9iSNDoWK5F_sr43tQareWXzYHyMq_cKss,9726
-l0_python-0.4.0.dist-info/licenses/LICENSE,sha256=LoyFJ8qmpwWPp5RmxFGHhdBxsemvp_BqGyiCgNAZHlk,1069
-l0_python-0.4.0.dist-info/METADATA,sha256=tVMaa0UIafPSIv9V9WWoxga7x2xtIVMhQ_egiX3C3y4,6833
-l0_python-0.4.0.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-l0_python-0.4.0.dist-info/top_level.txt,sha256=brP2SoElgmGhNJk-qRFEQZ5UE-d1e87Tr60hSSh8n34,3
-l0_python-0.4.0.dist-info/RECORD,,
+l0_python-0.4.2.dist-info/licenses/LICENSE,sha256=LoyFJ8qmpwWPp5RmxFGHhdBxsemvp_BqGyiCgNAZHlk,1069
+l0_python-0.4.2.dist-info/METADATA,sha256=czuRwTuF5iOn6f-rsFSxk_gHXWuKXAaYAmDugkGw4rA,9074
+l0_python-0.4.2.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+l0_python-0.4.2.dist-info/top_level.txt,sha256=brP2SoElgmGhNJk-qRFEQZ5UE-d1e87Tr60hSSh8n34,3
+l0_python-0.4.2.dist-info/RECORD,,