@wentorai/research-plugins 1.0.0 → 1.2.0

package/skills/domains/ai-ml/pytorch-lightning-guide/SKILL.md

@@ -0,0 +1,244 @@
+---
+name: pytorch-lightning-guide
+description: "PyTorch Lightning framework for scalable model training and research"
+metadata:
+  openclaw:
+    emoji: "⚡"
+    category: "domains"
+    subcategory: "ai-ml"
+    keywords: ["pytorch-lightning", "training", "distributed", "finetuning", "scalability", "research"]
+    source: "https://github.com/Lightning-AI/pytorch-lightning"
+---
+
+# PyTorch Lightning Guide
+
+## Overview
+
+PyTorch Lightning is a deep learning framework with over 31,000 GitHub stars that provides a high-level interface for PyTorch, enabling researchers to focus on model design rather than engineering boilerplate. Developed by Lightning AI, it decouples the science (model architecture, loss functions, data processing) from the engineering (distributed training, mixed precision, gradient accumulation, checkpointing) through a structured `LightningModule` abstraction.
+
+For academic researchers, Lightning eliminates the need to write repetitive training loops, device management code, and distributed training logic. You define your model, training step, and data loaders, and Lightning handles everything else -- from single GPU to multi-node distributed training, from FP32 to mixed precision, from local development to cloud deployment. This means faster iteration on research ideas with production-quality training infrastructure.
+
+Lightning is used extensively in AI research labs and has become a standard tool for reproducible deep learning experiments. It integrates seamlessly with experiment tracking tools like Weights & Biases, MLflow, and TensorBoard, and supports all PyTorch-compatible model architectures.
+
+## Installation and Setup
+
+```bash
+# Install PyTorch Lightning
+pip install lightning
+
+# Or install with specific extras
+pip install lightning[extra]
+
+# For development/research with all features
+pip install lightning[all]
+```
+
+Lightning requires Python 3.9+ and PyTorch 2.1+. For GPU training, ensure your PyTorch installation includes CUDA support:
+
+```bash
+# Check GPU availability
+python -c "import torch; print(torch.cuda.is_available())"
+```
+
+Verify your installation:
+
+```python
+import lightning as L
+print(L.__version__)
+```
+
+## Core Architecture
+
+### The LightningModule
+
+The `LightningModule` is the central abstraction. It organizes your PyTorch code into clearly defined methods:
+
+```python
+import lightning as L
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+class ResearchModel(L.LightningModule):
+    def __init__(self, input_dim, hidden_dim, output_dim, lr=1e-3):
+        super().__init__()
+        self.save_hyperparameters()
+        self.encoder = nn.Sequential(
+            nn.Linear(input_dim, hidden_dim),
+            nn.ReLU(),
+            nn.Dropout(0.2),
+            nn.Linear(hidden_dim, hidden_dim),
+            nn.ReLU(),
+        )
+        self.classifier = nn.Linear(hidden_dim, output_dim)
+        self.lr = lr
+
+    def forward(self, x):
+        features = self.encoder(x)
+        return self.classifier(features)
+
+    def training_step(self, batch, batch_idx):
+        x, y = batch
+        logits = self(x)
+        loss = F.cross_entropy(logits, y)
+        acc = (logits.argmax(dim=-1) == y).float().mean()
+        self.log("train_loss", loss, prog_bar=True)
+        self.log("train_acc", acc, prog_bar=True)
+        return loss
+
+    def validation_step(self, batch, batch_idx):
+        x, y = batch
+        logits = self(x)
+        loss = F.cross_entropy(logits, y)
+        acc = (logits.argmax(dim=-1) == y).float().mean()
+        self.log("val_loss", loss, prog_bar=True)
+        self.log("val_acc", acc, prog_bar=True)
+
+    def configure_optimizers(self):
+        optimizer = torch.optim.AdamW(self.parameters(), lr=self.lr)
+        scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(
+            optimizer, T_max=self.trainer.max_epochs
+        )
+        return [optimizer], [scheduler]
+```
+
+### The LightningDataModule
+
+Encapsulate all data processing in a reusable `LightningDataModule`:
+
+```python
+class ResearchDataModule(L.LightningDataModule):
+    def __init__(self, data_dir, batch_size=32, num_workers=4):
+        super().__init__()
+        self.data_dir = data_dir
+        self.batch_size = batch_size
+        self.num_workers = num_workers
+
+    def setup(self, stage=None):
+        # Load and split data
+        dataset = load_research_dataset(self.data_dir)
+        self.train_data, self.val_data, self.test_data = random_split(
+            dataset, [0.8, 0.1, 0.1]
+        )
+
+    def train_dataloader(self):
+        return DataLoader(self.train_data, batch_size=self.batch_size,
+                         shuffle=True, num_workers=self.num_workers)
+
+    def val_dataloader(self):
+        return DataLoader(self.val_data, batch_size=self.batch_size,
+                         num_workers=self.num_workers)
+```
+
+### The Trainer
+
+The `Trainer` orchestrates everything with a rich set of configuration options:
+
+```python
+trainer = L.Trainer(
+    max_epochs=100,
+    accelerator="gpu",
+    devices=4,
+    strategy="ddp",
+    precision="16-mixed",
+    gradient_clip_val=1.0,
+    accumulate_grad_batches=4,
+    callbacks=[
+        L.callbacks.EarlyStopping(monitor="val_loss", patience=10),
+        L.callbacks.ModelCheckpoint(monitor="val_loss", save_top_k=3),
+        L.callbacks.LearningRateMonitor(),
+    ],
+    logger=L.loggers.WandbLogger(project="my-research"),
+)
+
+# Train the model
+trainer.fit(model, datamodule=data_module)
+
+# Test with best checkpoint
+trainer.test(model, datamodule=data_module, ckpt_path="best")
+```
+
+## Advanced Research Features
+
+### Distributed Training Strategies
+
+Lightning supports multiple distributed training strategies out of the box:
+
+- **DDP (Distributed Data Parallel)**: Standard multi-GPU training
+- **FSDP (Fully Sharded Data Parallel)**: Memory-efficient training for large models
+- **DeepSpeed**: ZeRO optimization stages 1, 2, and 3
+
+```python
+# FSDP for large model training
+trainer = L.Trainer(
+    strategy="fsdp",
+    devices=8,
+    precision="bf16-mixed",
+)
+```
+
+### Custom Training Loops
+
+Override the training loop for non-standard research workflows like GANs, reinforcement learning, or meta-learning:
+
+```python
+class GANModule(L.LightningModule):
+    def training_step(self, batch, batch_idx):
+        optimizer_g, optimizer_d = self.optimizers()
+
+        # Train discriminator
+        real_loss = self.discriminator_loss(batch, real=True)
+        fake_loss = self.discriminator_loss(batch, real=False)
+        d_loss = (real_loss + fake_loss) / 2
+        optimizer_d.zero_grad()
+        self.manual_backward(d_loss)
+        optimizer_d.step()
+
+        # Train generator
+        g_loss = self.generator_loss(batch)
+        optimizer_g.zero_grad()
+        self.manual_backward(g_loss)
+        optimizer_g.step()
+
+    @property
+    def automatic_optimization(self):
+        return False
+```
+
+### Profiling and Debugging
+
+Built-in profiling tools help identify bottlenecks:
+
+```python
+trainer = L.Trainer(
+    profiler="advanced",  # or "simple", "pytorch"
+    detect_anomaly=True,
+    overfit_batches=10,   # Quick sanity check
+)
+```
+
+## Experiment Reproducibility
+
+Lightning has built-in support for reproducibility, which is critical for academic research:
+
+```python
+# Seed everything for reproducibility
+L.seed_everything(42, workers=True)
+
+# Hyperparameters are automatically saved
+model = ResearchModel(input_dim=768, hidden_dim=256, output_dim=10)
+# model.hparams is automatically populated and logged
+
+# Checkpoints include full training state
+# Resume training from a checkpoint
+trainer.fit(model, ckpt_path="path/to/checkpoint.ckpt")
+```
+
+The `save_hyperparameters()` call in your module's `__init__` automatically tracks all constructor arguments, making experiment comparison straightforward.
+
+## References
+
+- Repository: https://github.com/Lightning-AI/pytorch-lightning
+- Documentation: https://lightning.ai/docs/pytorch/stable/
+- Lightning AI platform: https://lightning.ai/
+- Migration guide from vanilla PyTorch: https://lightning.ai/docs/pytorch/stable/starter/converting.html

package/skills/domains/ai-ml/responsible-ai-guide/SKILL.md

@@ -0,0 +1,126 @@
+---
+name: responsible-ai-guide
+description: "Resources for trustworthy, fair, and ethical AI research"
+metadata:
+  openclaw:
+    emoji: "⚖️"
+    category: "domains"
+    subcategory: "ai-ml"
+    keywords: ["responsible AI", "AI ethics", "fairness", "trustworthy AI", "AI safety", "bias"]
+    source: "https://github.com/AthenaCore/AwesomeResponsibleAI"
+---
+
+# Responsible AI Guide
+
+## Overview
+
+A comprehensive collection of resources for building trustworthy, fair, and ethical AI systems. Covers fairness metrics, bias detection and mitigation, explainability methods, privacy-preserving techniques, robustness testing, and governance frameworks. Essential reading for researchers working on AI safety, alignment, and deploying models in high-stakes domains.
+
+## Topic Taxonomy
+
+```
+Responsible AI
+├── Fairness
+│   ├── Bias detection (data, model, outcome)
+│   ├── Fairness metrics (demographic parity, equalized odds)
+│   ├── Bias mitigation (pre/in/post-processing)
+│   └── Intersectional fairness
+├── Explainability
+│   ├── Feature attribution (SHAP, LIME, IG)
+│   ├── Concept-based (TCAV, concept bottleneck)
+│   ├── Counterfactual explanations
+│   └── Mechanistic interpretability
+├── Privacy
+│   ├── Differential privacy
+│   ├── Federated learning
+│   ├── Membership inference attacks
+│   └── Machine unlearning
+├── Robustness
+│   ├── Adversarial attacks/defenses
+│   ├── Distribution shift
+│   ├── Uncertainty quantification
+│   └── Out-of-distribution detection
+├── Safety & Alignment
+│   ├── RLHF and preference learning
+│   ├── Constitutional AI
+│   ├── Red teaming
+│   └── Guardrails and filters
+└── Governance
+    ├── Model cards
+    ├── Datasheets for datasets
+    ├── AI impact assessments
+    └── Regulatory compliance (EU AI Act)
+```
+
+## Key Tools
+
+| Tool | Category | Purpose |
+|------|----------|---------|
+| **Fairlearn** | Fairness | Bias assessment + mitigation |
+| **AI Fairness 360** | Fairness | IBM fairness toolkit |
+| **SHAP** | Explainability | Shapley value explanations |
+| **Captum** | Explainability | PyTorch interpretability |
+| **Opacus** | Privacy | Differential privacy for PyTorch |
+| **ART** | Robustness | Adversarial robustness toolbox |
+| **Alibi** | Explainability | ML model explanations |
+
+## Fairness Assessment
+
+```python
+from fairlearn.metrics import MetricFrame
+from sklearn.metrics import accuracy_score, recall_score
+
+# Assess fairness across demographic groups
+metrics = MetricFrame(
+    metrics={
+        "accuracy": accuracy_score,
+        "recall": recall_score,
+    },
+    y_true=y_test,
+    y_pred=y_pred,
+    sensitive_features=demographics,
+)
+
+print("Overall:")
+print(metrics.overall)
+print("\nBy group:")
+print(metrics.by_group)
+print("\nDifference (max - min):")
+print(metrics.difference())
+```
+
+## Reading Roadmap
+
+```markdown
+### Foundations
+1. "Fairness and Machine Learning" (Barocas, Hardt, Narayanan)
+2. "Datasheets for Datasets" (Gebru et al., 2021)
+3. "Model Cards for Model Reporting" (Mitchell et al., 2019)
+
+### Fairness
+4. "On Fairness and Calibration" (Pleiss et al., 2017)
+5. "Fairness Through Awareness" (Dwork et al., 2012)
+
+### Explainability
+6. "A Unified Approach to Interpreting Model Predictions" (SHAP)
+7. "Why Should I Trust You?" (LIME, Ribeiro et al., 2016)
+
+### Safety
+8. "Constitutional AI" (Bai et al., 2022)
+9. "Red Teaming Language Models" (Perez et al., 2022)
+10. "Scaling Monosemanticity" (Anthropic, 2024)
+```
+
+## Use Cases
+
+1. **Bias auditing**: Check models for demographic biases
+2. **Compliance**: EU AI Act and regulatory requirements
+3. **Model documentation**: Model cards and impact assessments
+4. **Research ethics**: Ethical considerations for AI research
+5. **Course material**: Teach responsible AI principles
+
+## References
+
+- [AwesomeResponsibleAI](https://github.com/AthenaCore/AwesomeResponsibleAI)
+- [Fairlearn](https://fairlearn.org/)
+- [EU AI Act](https://artificialintelligenceact.eu/)

package/skills/domains/ai-ml/tensorflow-guide/SKILL.md

@@ -0,0 +1,241 @@
+---
+name: tensorflow-guide
+description: "TensorFlow best practices for tf.function, GPU memory, and deployment"
+metadata:
+  openclaw:
+    emoji: "🧮"
+    category: "domains"
+    subcategory: "ai-ml"
+    keywords: ["TensorFlow", "tf.function", "GPU", "SavedModel", "distributed training", "XLA"]
+    source: "https://github.com/tensorflow/tensorflow"
+---
+
+# TensorFlow Guide
+
+## Overview
+
+TensorFlow is a production-grade machine learning framework that excels at deployment, distributed training, and hardware acceleration. While PyTorch dominates pure research prototyping, TensorFlow remains the standard in industry ML systems and is heavily used in applied research where models must move from experiment to production.
+
+TensorFlow 2.x unified eager execution with graph-mode performance through `tf.function`, but this hybrid approach introduces subtle pitfalls. Understanding when and how TensorFlow traces functions, manages GPU memory, and distributes computation is essential for writing correct and efficient code.
+
+This guide covers the key patterns that trip up researchers: `tf.function` tracing semantics, GPU memory management, distributed strategies, model export, and the ecosystem of tools (TFX, TensorBoard, TF Serving) that make TensorFlow uniquely powerful for end-to-end ML workflows.
+
+## tf.function: The Critical Abstraction
+
+### How Tracing Works
+
+```python
+import tensorflow as tf
+
+@tf.function
+def add(a, b):
+    print("Tracing!")  # Runs only during tracing, NOT every call
+    tf.print("Executing!")  # Runs every call (TF op)
+    return a + b
+
+# First call with float32 shape (2,) -- traces
+add(tf.constant([1.0, 2.0]), tf.constant([3.0, 4.0]))  # Prints "Tracing!" + "Executing!"
+
+# Second call with same signature -- reuses trace
+add(tf.constant([5.0, 6.0]), tf.constant([7.0, 8.0]))  # Prints only "Executing!"
+
+# Third call with different dtype -- re-traces!
+add(tf.constant([1, 2]), tf.constant([3, 4]))  # Prints "Tracing!" + "Executing!"
+```
+
+### Common tf.function Pitfalls
+
+```python
+# PITFALL 1: Python side effects in tf.function
+counter = 0
+
+@tf.function
+def increment():
+    global counter
+    counter += 1  # Only runs during tracing! counter stays at 1 forever.
+    return counter
+
+# FIX: Use tf.Variable for mutable state
+counter = tf.Variable(0)
+
+@tf.function
+def increment():
+    counter.assign_add(1)
+    return counter
+
+# PITFALL 2: Creating variables inside tf.function
+@tf.function
+def bad_function(x):
+    w = tf.Variable(tf.random.normal([3, 3]))  # ERROR on second call!
+    return x @ w
+
+# FIX: Create variables outside, pass as arguments or use Keras layers
+w = tf.Variable(tf.random.normal([3, 3]))
+
+@tf.function
+def good_function(x):
+    return x @ w
+
+# PITFALL 3: Python lists that grow
+@tf.function
+def bad_accumulate(dataset):
+    results = []
+    for x in dataset:
+        results.append(x * 2)  # Creates new trace on every iteration!
+    return results
+
+# FIX: Use tf.TensorArray
+@tf.function
+def good_accumulate(dataset):
+    results = tf.TensorArray(tf.float32, size=0, dynamic_size=True)
+    for i, x in enumerate(dataset):
+        results = results.write(i, x * 2)
+    return results.stack()
+```
+
+### Input Signatures for Stable Tracing
+
+```python
+@tf.function(input_signature=[
+    tf.TensorSpec(shape=[None, 224, 224, 3], dtype=tf.float32),
+    tf.TensorSpec(shape=[None], dtype=tf.int64),
+])
+def train_step(images, labels):
+    """Fixed signature prevents re-tracing on different batch sizes."""
+    with tf.GradientTape() as tape:
+        predictions = model(images, training=True)
+        loss = loss_fn(labels, predictions)
+    gradients = tape.gradient(loss, model.trainable_variables)
+    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
+    return loss
+```
+
+## GPU Memory Management
+
+```python
+# Problem: TensorFlow grabs ALL GPU memory by default
+# Solution: Enable memory growth
+
+gpus = tf.config.list_physical_devices("GPU")
+for gpu in gpus:
+    tf.config.experimental.set_memory_growth(gpu, True)
+
+# Alternative: Set a hard memory limit
+tf.config.set_logical_device_configuration(
+    gpus[0],
+    [tf.config.LogicalDeviceConfiguration(memory_limit=8192)]  # 8 GB
+)
+
+# Monitor memory usage
+print(tf.config.experimental.get_memory_info("GPU:0"))
+```
+
+## Distributed Training Strategies
+
+| Strategy | GPUs | Machines | Sync | Use Case |
+|----------|------|----------|------|----------|
+| `MirroredStrategy` | Multiple | 1 | Sync | Most common multi-GPU |
+| `MultiWorkerMirroredStrategy` | Multiple | Multiple | Sync | Multi-node training |
+| `TPUStrategy` | TPU cores | 1 pod | Sync | TPU training |
+| `ParameterServerStrategy` | Multiple | Multiple | Async | Very large models |
+
+```python
+# Multi-GPU training with MirroredStrategy
+strategy = tf.distribute.MirroredStrategy()
+print(f"Number of devices: {strategy.num_replicas_in_sync}")
+
+with strategy.scope():
+    model = build_model()
+    model.compile(
+        optimizer=tf.keras.optimizers.Adam(learning_rate=0.001 * strategy.num_replicas_in_sync),
+        loss="sparse_categorical_crossentropy",
+        metrics=["accuracy"],
+    )
+
+# Global batch size = per_replica_batch * num_replicas
+global_batch_size = 32 * strategy.num_replicas_in_sync
+dataset = dataset.batch(global_batch_size)
+model.fit(dataset, epochs=10)
+```
+
+## Model Export and Serving
+
+```python
+# SavedModel: The universal export format
+model.save("saved_model/my_model")
+
+# Load with full TF capabilities
+loaded = tf.saved_model.load("saved_model/my_model")
+infer = loaded.signatures["serving_default"]
+
+# TF Lite for mobile/edge deployment
+converter = tf.lite.TFLiteConverter.from_saved_model("saved_model/my_model")
+converter.optimizations = [tf.lite.Optimize.DEFAULT]
+tflite_model = converter.convert()
+
+with open("model.tflite", "wb") as f:
+    f.write(tflite_model)
+
+# TensorFlow.js for browser deployment
+# Command line:
+# tensorflowjs_converter --input_format=tf_saved_model saved_model/my_model web_model/
+```
+
+## Performance Optimization with XLA
+
+```python
+# XLA (Accelerated Linear Algebra) compiles tf.functions for hardware
+@tf.function(jit_compile=True)
+def fast_matmul(a, b):
+    return tf.matmul(a, b)
+
+# Enable XLA globally for Keras
+tf.config.optimizer.set_jit(True)
+
+# Benchmark XLA vs non-XLA
+import time
+a = tf.random.normal([1024, 1024])
+b = tf.random.normal([1024, 1024])
+
+# Warm up
+fast_matmul(a, b)
+
+start = time.time()
+for _ in range(1000):
+    fast_matmul(a, b)
+print(f"XLA matmul: {time.time() - start:.3f}s")
+```
+
+## Debugging and Profiling
+
+```python
+# Enable eager mode for debugging
+tf.config.run_functions_eagerly(True)
+
+# TensorBoard profiler integration
+log_dir = "logs/profile"
+tf.profiler.experimental.start(log_dir)
+# ... run training steps ...
+tf.profiler.experimental.stop()
+# View: tensorboard --logdir logs/profile
+
+# Check for numerical issues
+tf.debugging.enable_check_numerics()  # Raises on NaN/Inf
+```
+
+## Best Practices
+
+- **Set memory growth before any TF operations.** It must be the first GPU-related call.
+- **Use `tf.function` with explicit `input_signature`** to prevent re-tracing in production.
+- **Avoid Python control flow inside `tf.function`** unless you use `tf.cond` / `tf.while_loop`.
+- **Profile with TensorBoard** before optimizing; identify whether you are CPU-bound, GPU-bound, or I/O-bound.
+- **Use mixed precision** via `tf.keras.mixed_precision.set_global_policy("mixed_float16")` for modern GPUs.
+- **Pin TF version in Docker images** for reproducible research -- different versions can produce different numerical results.
+
+## References
+
+- [TensorFlow documentation](https://www.tensorflow.org/guide) -- Official guides and API reference
+- [Better performance with tf.function](https://www.tensorflow.org/guide/function) -- Tracing semantics deep dive
+- [Distributed training guide](https://www.tensorflow.org/guide/distributed_training) -- Multi-GPU and multi-node patterns
+- [TensorFlow Model Garden](https://github.com/tensorflow/models) -- Reference implementations of SOTA models
+- [XLA documentation](https://www.tensorflow.org/xla) -- Hardware-accelerated compilation