omgkit 2.19.3 → 2.21.0

package/plugin/commands/omgdeploy/edge.md

@@ -0,0 +1,93 @@
+---
+description: Deploy model to edge devices including TFLite Micro, Jetson, Raspberry Pi, and mobile platforms
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <target> [--model <path>] [--optimize]
+---
+
+# Edge Deployment: $ARGUMENTS
+
+Deploy to edge: **$ARGUMENTS**
+
+## Agent
+Uses **deployment-agent** for edge deployment.
+
+## Parameters
+- **target**: tflite_micro | jetson | raspberry_pi | mobile_ios | mobile_android
+- **model**: Path to model
+- **optimize**: Apply target-specific optimization (default: true)
+
+## Edge Targets
+
+### TFLite Micro
+- Microcontroller deployment
+- <1MB Flash, <256KB RAM
+- INT8 quantization required
+- C++ runtime
+
+### Jetson (Nano/Xavier/Orin)
+- NVIDIA GPU accelerated
+- TensorRT optimization
+- FP16/INT8 support
+- CUDA runtime
+
+### Raspberry Pi
+- ARM CPU deployment
+- ONNX Runtime
+- INT8 quantization
+- Python/C++ runtime
+
+### Mobile iOS
+- CoreML format
+- Neural Engine
+- On-device inference
+- Swift/Obj-C integration
+
+### Mobile Android
+- TFLite format
+- NNAPI delegate
+- GPU delegate
+- Kotlin/Java integration
+
+## Code Template
+```python
+from omgkit.deployment import EdgeDeployer
+
+deployer = EdgeDeployer()
+
+# Deploy to TFLite Micro (TinyML)
+tflite_model = deployer.deploy(
+    model_path="models/best_model.pt",
+    target="tflite_micro",
+    optimize=True,
+    constraints={
+        "max_flash_kb": 512,
+        "max_ram_kb": 128,
+        "target_latency_ms": 10
+    },
+    output_dir="edge/tflite_micro/"
+)
+
+# Deploy to Jetson
+jetson_model = deployer.deploy(
+    model_path="models/best_model.pt",
+    target="jetson",
+    optimize=True,
+    tensorrt_precision="fp16",
+    output_dir="edge/jetson/"
+)
+```
+
+## Optimization Applied
+- Quantization (INT8/FP16)
+- Pruning
+- Operator fusion
+- Memory optimization
+
+## Progress
+- [ ] Target validated
+- [ ] Model converted
+- [ ] Optimization applied
+- [ ] Size verified
+- [ ] Package created
+
+Deploy ML models to resource-constrained devices.

package/plugin/commands/omgdeploy/package.md

@@ -0,0 +1,91 @@
+---
+description: Package model for deployment in TorchServe, TF Serving, ONNX, TFLite, or Docker formats
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <format> [--model <path>]
+---
+
+# Model Packaging: $ARGUMENTS
+
+Package model: **$ARGUMENTS**
+
+## Agent
+Uses **deployment-agent** for model packaging.
+
+## Parameters
+- **format**: torchserve | tensorflow_serving | onnx | tflite | docker
+- **model**: Path to trained model
+
+## Package Formats
+
+### TorchServe
+- .mar archive file
+- Custom handler support
+- Multi-model serving
+- Batch inference
+
+### TensorFlow Serving
+- SavedModel format
+- gRPC/REST endpoints
+- Model versioning
+- Warm-up support
+
+### ONNX
+- Cross-framework format
+- Hardware agnostic
+- Wide runtime support
+- Optimization passes
+
+### TFLite
+- Mobile/embedded ready
+- Quantization built-in
+- Delegates support
+- Minimal footprint
+
+### Docker
+- Containerized deployment
+- All dependencies included
+- Kubernetes ready
+- Easy scaling
+
+## Code Template
+```python
+from omgkit.deployment import ModelPackager
+
+packager = ModelPackager()
+
+# Package for TorchServe
+package = packager.package(
+    model_path="models/best_model.pt",
+    format="torchserve",
+    model_name="churn_predictor",
+    version="1.0",
+    handler="src/serving/handler.py",
+    requirements=["torch", "numpy", "pandas"],
+    output_dir="artifacts/"
+)
+
+# Package as Docker
+docker_image = packager.package(
+    model_path="models/best_model.pt",
+    format="docker",
+    base_image="pytorch/pytorch:2.0.0-cuda11.7-cudnn8-runtime",
+    expose_port=8080,
+    output_dir="docker/"
+)
+```
+
+## Output Artifacts
+- Packaged model file
+- Dockerfile (if docker)
+- Requirements file
+- Handler script
+- Config files
+
+## Progress
+- [ ] Model loaded
+- [ ] Format validated
+- [ ] Dependencies resolved
+- [ ] Package created
+- [ ] Artifacts saved
+
+Create deployment-ready model packages.

package/plugin/commands/omgdeploy/serve.md

@@ -0,0 +1,92 @@
+---
+description: Deploy model serving endpoint on local, Kubernetes, AWS SageMaker, GCP Vertex, or Azure ML
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <platform> [--config <config>]
+---
+
+# Model Serving: $ARGUMENTS
+
+Deploy serving: **$ARGUMENTS**
+
+## Agent
+Uses **deployment-agent** for model serving deployment.
+
+## Parameters
+- **platform**: local | kubernetes | aws_sagemaker | gcp_vertex | azure_ml
+- **config**: Path to deployment configuration
+
+## Deployment Platforms
+
+### Local
+- FastAPI server
+- Development/testing
+- Quick iteration
+- localhost access
+
+### Kubernetes
+- Seldon Core / KServe
+- Auto-scaling
+- Canary deployments
+- Production grade
+
+### AWS SageMaker
+- Managed endpoints
+- Auto-scaling
+- Multi-model serving
+- A/B testing
+
+### GCP Vertex AI
+- Managed prediction
+- Explanation support
+- Batch prediction
+- AutoML integration
+
+### Azure ML
+- Managed endpoints
+- Blue-green deployment
+- Container support
+- Enterprise features
+
+## Code Template
+```python
+from omgkit.deployment import ModelServer
+
+server = ModelServer()
+
+# Deploy to Kubernetes
+endpoint = server.deploy(
+    model_path="artifacts/churn_predictor.mar",
+    platform="kubernetes",
+    config={
+        "replicas": 3,
+        "resources": {
+            "requests": {"cpu": "500m", "memory": "1Gi"},
+            "limits": {"cpu": "2", "memory": "4Gi"}
+        },
+        "autoscaling": {
+            "min_replicas": 2,
+            "max_replicas": 10,
+            "target_cpu_utilization": 70
+        }
+    },
+    namespace="ml-serving"
+)
+
+print(f"Endpoint deployed: {endpoint.url}")
+```
+
+## Features
+- Health checks
+- Metrics collection
+- Request logging
+- Error handling
+- Graceful shutdown
+
+## Progress
+- [ ] Config validated
+- [ ] Platform connected
+- [ ] Model deployed
+- [ ] Health verified
+- [ ] Endpoint active
+
+Deploy scalable model serving infrastructure.

package/plugin/commands/omgfeature/embed.md

@@ -0,0 +1,93 @@
+---
+description: Create embeddings for text, images, and categorical features using pretrained or custom models
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <data_type> [--model <model>]
+---
+
+# Feature Embeddings: $ARGUMENTS
+
+Create embeddings: **$ARGUMENTS**
+
+## Agent
+Uses **feature-engineer-agent** for embedding generation.
+
+## Parameters
+- **data_type**: text | image | categorical
+- **model**: Embedding model name (e.g., sentence-transformers, CLIP)
+
+## Embedding Types
+
+### Text Embeddings
+- Sentence Transformers
+- OpenAI embeddings
+- Cohere embeddings
+- Custom fine-tuned models
+- Word2Vec/FastText
+
+### Image Embeddings
+- CLIP
+- ResNet features
+- EfficientNet
+- ViT (Vision Transformer)
+
+### Categorical Embeddings
+- Entity embeddings
+- Neural network learned
+- Co-occurrence based
+
+## Code Template
+```python
+from omgkit.features import EmbeddingGenerator
+
+generator = EmbeddingGenerator()
+
+# Text embeddings
+text_embeddings = generator.embed(
+    data_path="data/processed/texts.parquet",
+    data_type="text",
+    model="sentence-transformers/all-MiniLM-L6-v2",
+    column="description",
+    batch_size=32,
+    output_path="data/features/text_embeddings.parquet"
+)
+
+# Image embeddings
+image_embeddings = generator.embed(
+    data_path="data/processed/images/",
+    data_type="image",
+    model="openai/clip-vit-base-patch32",
+    batch_size=16,
+    output_path="data/features/image_embeddings.parquet"
+)
+
+# Categorical embeddings
+cat_embeddings = generator.embed(
+    data_path="data/processed/categories.parquet",
+    data_type="categorical",
+    model="entity_embedding",
+    columns=["category", "brand", "seller"],
+    embedding_dim=32,
+    output_path="data/features/cat_embeddings.parquet"
+)
+```
+
+## Model Options
+- OpenAI: text-embedding-ada-002
+- Sentence Transformers: all-MiniLM-L6-v2
+- CLIP: openai/clip-vit-base-patch32
+- Custom: fine-tuned models
+
+## Output
+- Embedding vectors
+- Metadata mapping
+- Dimensionality info
+- Quality metrics
+
+## Progress
+- [ ] Model loaded
+- [ ] Data processed
+- [ ] Embeddings generated
+- [ ] Quality validated
+- [ ] Output saved
+
+Generate dense vector representations for ML tasks.

package/plugin/commands/omgfeature/extract.md

@@ -0,0 +1,93 @@
+---
+description: Extract features from numerical, categorical, text, image, and temporal data with configurable pipelines
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <feature_type> [--config <config>]
+---
+
+# Feature Extraction: $ARGUMENTS
+
+Extract features: **$ARGUMENTS**
+
+## Agent
+Uses **feature-engineer-agent** for feature extraction.
+
+## Parameters
+- **feature_type**: numerical | categorical | text | image | temporal | custom
+- **config**: Path to feature extraction config
+
+## Feature Operations
+
+### Numerical
+- Scaling (standard, minmax, robust)
+- Log transform
+- Polynomial features
+- Binning/discretization
+- Outlier handling
+
+### Categorical
+- One-hot encoding
+- Label encoding
+- Target encoding
+- Frequency encoding
+- Hash encoding
+
+### Text
+- TF-IDF vectors
+- Word2Vec/FastText
+- BERT embeddings
+- Text statistics
+- N-gram features
+
+### Image
+- CNN feature extraction
+- Histogram features
+- Edge detection
+- Color features
+
+### Temporal
+- Datetime components
+- Cyclical encoding (sin/cos)
+- Lag features
+- Rolling statistics
+- Trend extraction
+
+## Code Template
+```python
+from omgkit.features import FeatureExtractor
+
+extractor = FeatureExtractor()
+
+feature_pipeline = extractor.create_pipeline({
+    "numerical": {
+        "columns": ["age", "income", "tenure"],
+        "operations": ["scaling", "log_transform"]
+    },
+    "categorical": {
+        "columns": ["city", "occupation"],
+        "operations": ["target_encoding"]
+    },
+    "text": {
+        "columns": ["description"],
+        "operations": ["tfidf", "text_statistics"]
+    },
+    "temporal": {
+        "columns": ["signup_date", "last_login"],
+        "operations": ["datetime_components", "cyclical_encoding"]
+    }
+})
+
+features = extractor.extract(
+    data_path="data/processed/users.parquet",
+    pipeline=feature_pipeline,
+    output_path="data/features/user_features.parquet"
+)
+```
+
+## Progress
+- [ ] Config loaded
+- [ ] Pipeline created
+- [ ] Features extracted
+- [ ] Quality validated
+- [ ] Output saved
+
+Create reproducible feature engineering pipelines.

package/plugin/commands/omgfeature/select.md

@@ -0,0 +1,85 @@
+---
+description: Select most important features using filter, wrapper, embedded, or hybrid methods
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <method> [--n_features <n>]
+---
+
+# Feature Selection: $ARGUMENTS
+
+Select features: **$ARGUMENTS**
+
+## Agent
+Uses **feature-engineer-agent** for feature selection.
+
+## Parameters
+- **method**: filter | wrapper | embedded | hybrid (default: hybrid)
+- **n_features**: Number of features to select
+
+## Selection Methods
+
+### Filter Methods
+- Mutual information
+- Chi-square test
+- ANOVA F-test
+- Correlation analysis
+- Variance threshold
+
+### Wrapper Methods
+- Recursive feature elimination (RFE)
+- Forward selection
+- Backward elimination
+- Exhaustive search
+
+### Embedded Methods
+- LASSO importance (L1)
+- Tree-based importance
+- Permutation importance
+- SHAP values
+
+### Hybrid
+- Combines multiple methods
+- Voting/consensus approach
+- Best of both worlds
+
+## Code Template
+```python
+from omgkit.features import FeatureSelector
+
+selector = FeatureSelector()
+
+selected_features = selector.select(
+    data_path="data/features/all_features.parquet",
+    target_column="label",
+    methods={
+        "filter": ["mutual_information", "correlation"],
+        "embedded": ["tree_importance"]
+    },
+    n_features=50,
+    output_path="data/features/selected_features.parquet"
+)
+
+# Generate importance report
+selector.report(output="reports/feature_importance.html")
+```
+
+## Output
+- Selected feature list
+- Feature importance scores
+- Selection rationale
+- Visualization plots
+- HTML report
+
+## Best Practices
+- Start with filter methods (fast)
+- Use embedded for final selection
+- Validate on holdout set
+- Consider domain knowledge
+
+## Progress
+- [ ] Features loaded
+- [ ] Methods applied
+- [ ] Selection complete
+- [ ] Report generated
+- [ ] Output saved
+
+Reduce dimensionality while preserving predictive power.

package/plugin/commands/omgfeature/store.md

@@ -0,0 +1,97 @@
+---
+description: Feature store operations - register, materialize, and serve features for training and inference
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <action> [--feature_view <view>]
+---
+
+# Feature Store: $ARGUMENTS
+
+Feature store operation: **$ARGUMENTS**
+
+## Agent
+Uses **feature-engineer-agent** for feature store management.
+
+## Parameters
+- **action**: register | materialize | serve | list
+- **feature_view**: Feature view name
+
+## Actions
+
+### Register
+- Define feature definitions
+- Specify entities and joins
+- Set TTL and freshness
+- Configure data sources
+
+### Materialize
+- Compute features for training
+- Populate online store
+- Batch feature computation
+- Incremental updates
+
+### Serve
+- Get features for inference
+- Point-in-time correct retrieval
+- Low-latency serving
+- Batch retrieval
+
+### List
+- Show all feature views
+- Display feature definitions
+- Check freshness status
+
+## Code Template
+```python
+from omgkit.features import FeatureStore
+from feast import Entity, FeatureView, Field
+from feast.types import Float32, Int64
+
+store = FeatureStore(repo_path="feature_repo/")
+
+# Define entity
+user = Entity(name="user", join_keys=["user_id"])
+
+# Define feature view
+user_features = FeatureView(
+    name="user_features",
+    entities=[user],
+    schema=[
+        Field(name="total_purchases", dtype=Float32),
+        Field(name="avg_order_value", dtype=Float32),
+        Field(name="days_since_last_order", dtype=Int64),
+    ],
+    source=user_source,
+    online=True,
+    ttl=timedelta(days=1)
+)
+
+# Register
+store.register([user, user_features])
+
+# Materialize
+store.materialize(
+    start_date=datetime(2024, 1, 1),
+    end_date=datetime.now()
+)
+
+# Serve features
+features = store.get_online_features(
+    features=["user_features:total_purchases", "user_features:avg_order_value"],
+    entity_rows=[{"user_id": 123}]
+).to_dict()
+```
+
+## Benefits
+- Training-serving consistency
+- Point-in-time correctness
+- Feature reuse across models
+- Feature discovery
+
+## Progress
+- [ ] Action validated
+- [ ] Feature store connected
+- [ ] Operation executed
+- [ ] Cache updated
+- [ ] Status verified
+
+Ensure consistent features across training and serving.

package/plugin/commands/omgml/init.md

@@ -0,0 +1,60 @@
+---
+description: Initialize ML project with standard structure, configs, experiment tracking, and best practices
+allowed-tools: Task, Read, Write, Bash, Grep, Glob
+argument-hint: <project_type> [--template <template>]
+---
+
+# ML Project Init: $ARGUMENTS
+
+Initialize ML project: **$ARGUMENTS**
+
+## Agent
+Uses **ml-engineer** agent with data engineering skills.
+
+## Parameters
+- **project_type**: classification | regression | detection | segmentation | nlp | timeseries | recommendation
+- **template**: minimal | standard | production | research (default: standard)
+
+## Project Structure Created
+```
+project/
+├── config/
+│   ├── config.yaml           # Main config
+│   ├── model_config.yaml     # Model hyperparameters
+│   └── data_config.yaml      # Data processing config
+├── data/
+│   ├── raw/                  # Raw data
+│   ├── processed/            # Processed data
+│   ├── features/             # Feature store
+│   └── .dvc/                 # DVC tracking
+├── src/
+│   ├── data/                 # Data processing
+│   ├── features/             # Feature engineering
+│   ├── models/               # Model definitions
+│   ├── training/             # Training scripts
+│   └── serving/              # Inference code
+├── notebooks/                # Jupyter notebooks
+├── tests/                    # Unit tests
+├── pipelines/                # CI/CD pipelines
+├── mlruns/                   # MLflow tracking
+└── artifacts/                # Model artifacts
+```
+
+## Actions
+1. Create standard directory structure
+2. Generate configuration files
+3. Setup experiment tracking (MLflow/W&B)
+4. Initialize DVC for data versioning
+5. Create README with project guidelines
+6. Setup virtual environment
+7. Install base dependencies
+
+## Progress
+- [ ] Directory structure created
+- [ ] Config files generated
+- [ ] Experiment tracking setup
+- [ ] DVC initialized
+- [ ] README created
+- [ ] Environment configured
+
+Initialize with best practices for production ML.