featcopilot 0.1.0__tar.gz → 0.3.0__tar.gz

{featcopilot-0.1.0 → featcopilot-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: featcopilot
-Version: 0.1.0
+Version: 0.3.0
 Summary: Next-generation LLM-powered auto feature engineering framework with GitHub Copilot SDK
 Author: FeatCopilot Contributors
 License: MIT
@@ -28,11 +28,27 @@ Requires-Dist: pydantic>=2.0.0
 Requires-Dist: joblib>=1.1.0
 Provides-Extra: llm
 Requires-Dist: github-copilot-sdk>=0.1.0; extra == "llm"
+Requires-Dist: nest_asyncio>=1.5.0; extra == "llm"
+Provides-Extra: litellm
+Requires-Dist: litellm>=1.0.0; extra == "litellm"
+Requires-Dist: nest_asyncio>=1.5.0; extra == "litellm"
 Provides-Extra: timeseries
 Requires-Dist: statsmodels>=0.13.0; extra == "timeseries"
+Provides-Extra: feast
+Requires-Dist: feast>=0.30.0; extra == "feast"
 Provides-Extra: full
 Requires-Dist: github-copilot-sdk>=0.1.0; extra == "full"
+Requires-Dist: litellm>=1.0.0; extra == "full"
 Requires-Dist: statsmodels>=0.13.0; extra == "full"
+Requires-Dist: feast>=0.30.0; extra == "full"
+Requires-Dist: nest_asyncio>=1.5.0; extra == "full"
+Provides-Extra: benchmark
+Requires-Dist: github-copilot-sdk>=0.1.0; extra == "benchmark"
+Requires-Dist: statsmodels>=0.13.0; extra == "benchmark"
+Requires-Dist: flaml[automl,blendsearch]>=2.0.0; extra == "benchmark"
+Requires-Dist: autogluon.tabular[fastai]>=1.5.0; extra == "benchmark"
+Requires-Dist: h2o>=3.40.0; extra == "benchmark"
+Requires-Dist: numpy<2; extra == "benchmark"
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
@@ -44,32 +60,39 @@ Requires-Dist: pre-commit>=3.6.0; extra == "dev"
 
 # FeatCopilot 🚀
 
-**Next-Generation LLM-Powered Auto Feature Engineering with GitHub Copilot SDK**
+**Next-Generation LLM-Powered Auto Feature Engineering Framework**
 
-FeatCopilot is a unified feature engineering framework that combines the best approaches from existing libraries (Featuretools, TSFresh, AutoFeat, OpenFE) with novel LLM-powered capabilities via GitHub Copilot SDK.
+FeatCopilot automatically generates, selects, and explains predictive features using semantic understanding. It analyzes column meanings, applies domain-aware transformations, and provides human-readable explanations—turning raw data into ML-ready features in seconds.
+
+## 🎬 Introduction Video
+
+[![FeatCopilot Introduction](https://img.youtube.com/vi/H7m50TLGHFk/0.jpg)](https://www.youtube.com/watch?v=H7m50TLGHFk)
 
 ## 📊 Benchmark Highlights
 
-### Tabular Engine (Fast Mode - <1s)
+### Simple Models Benchmark (42 Datasets)
+
+| Configuration | Improved | Avg Improvement | Best Improvement |
+|---------------|----------|-----------------|------------------|
+| **Tabular Engine** | 20 (48%) | +4.54% | +197% (delays_zurich) |
+| **Tabular + LLM** | 23 (55%) | +6.12% | +420% (delays_zurich) |
+
+Models: RandomForest (n_estimators=200, max_depth=20), LogisticRegression/Ridge
 
-| Task Type | Average Improvement | Best Case |
-|-----------|--------------------:|----------:|
-| **Text Classification** | **+12.44%** | +49.02% (News Headlines) |
-| Time Series | +1.51% | +12.12% (Retail Demand) |
-| Classification | +0.54% | +4.35% |
-| Regression | +0.65% | +5.57% |
+### AutoML Benchmark (FLAML, 120s budget)
 
-### LLM Engine (With Copilot - 30-60s)
+| Metric | Value |
+|--------|-------|
+| **Datasets** | 41 |
+| **Improved** | 19 (46%) |
+| **Best Improvement** | +8.55% (abalone) |
 
-| Task Type | Average Improvement | Best Case |
-|-----------|--------------------:|----------:|
-| **Regression** | **+7.79%** | +19.66% (Retail Demand) |
-| Classification | +2.38% | +2.87% |
+### Key Results
 
-- ✅ **12/12 wins** on text classification (tabular mode)
-- 🧠 **+19.66% max improvement** with LLM-powered features
-- ⚡ **<1 second** (tabular) or **30-60s** (with LLM) processing time
-- 📈 Largest gains with simple models (LogisticRegression, Ridge)
+- ✅ **+197% improvement** on delays_zurich (tabular only)
+- 🧠 **+420% improvement** with LLM-enhanced features
+- 📈 **+8.98%** on abalone regression task
+- 🚀 **+5.68%** on complex_classification
 
 [View Full Benchmark Results](https://thinkall.github.io/featcopilot/user-guide/benchmarks/)
 
@@ -87,7 +110,7 @@ FeatCopilot is a unified feature engineering framework that combines the best ap
 # Basic installation
 pip install featcopilot
 
-# With LLM capabilities (requires GitHub Copilot)
+# With LLM capabilities
 pip install featcopilot[llm]
 
 # Full installation
@@ -111,12 +134,12 @@ X_transformed = engineer.fit_transform(X, y)  # <1 second
 print(f"Features: {X.shape[1]} -> {X_transformed.shape[1]}")
 ```
 
-### LLM Mode (With Copilot)
+### LLM Mode (With LiteLLM)
 
 ```python
 from featcopilot import AutoFeatureEngineer
 
-# LLM-powered semantic features (+19.66% max improvement)
+# LLM-powered semantic features (+420% max improvement)
 engineer = AutoFeatureEngineer(
     engines=['tabular', 'llm'],
     max_features=50
@@ -164,16 +187,24 @@ engine = TimeSeriesEngine(
 ```
 
 ### LLM Engine
-Uses GitHub Copilot SDK for intelligent feature generation.
+Uses GitHub Copilot SDK (default) or LiteLLM (100+ providers) for intelligent feature generation.
 
 ```python
 from featcopilot.llm import SemanticEngine
 
+# Default: GitHub Copilot SDK
 engine = SemanticEngine(
-    model='gpt-5',
+    model='gpt-5.2',
     max_suggestions=20,
     validate_features=True
 )
+
+# Alternative: LiteLLM backend
+engine = SemanticEngine(
+    model='gpt-4o',
+    backend='litellm',
+    max_suggestions=20
+)
 ```
 
 ## Feature Selection
@@ -211,7 +242,7 @@ X_selected = selector.fit_transform(X, y)
 
 - Python 3.9+
 - NumPy, Pandas, Scikit-learn
-- GitHub Copilot CLI (for LLM features)
+- GitHub Copilot SDK (default) or LiteLLM (for 100+ LLM providers)
 
 ## License
 

{featcopilot-0.1.0 → featcopilot-0.3.0}/README.md

@@ -1,31 +1,38 @@
 # FeatCopilot 🚀
 
-**Next-Generation LLM-Powered Auto Feature Engineering with GitHub Copilot SDK**
+**Next-Generation LLM-Powered Auto Feature Engineering Framework**
 
-FeatCopilot is a unified feature engineering framework that combines the best approaches from existing libraries (Featuretools, TSFresh, AutoFeat, OpenFE) with novel LLM-powered capabilities via GitHub Copilot SDK.
+FeatCopilot automatically generates, selects, and explains predictive features using semantic understanding. It analyzes column meanings, applies domain-aware transformations, and provides human-readable explanations—turning raw data into ML-ready features in seconds.
+
+## 🎬 Introduction Video
+
+[![FeatCopilot Introduction](https://img.youtube.com/vi/H7m50TLGHFk/0.jpg)](https://www.youtube.com/watch?v=H7m50TLGHFk)
 
 ## 📊 Benchmark Highlights
 
-### Tabular Engine (Fast Mode - <1s)
+### Simple Models Benchmark (42 Datasets)
+
+| Configuration | Improved | Avg Improvement | Best Improvement |
+|---------------|----------|-----------------|------------------|
+| **Tabular Engine** | 20 (48%) | +4.54% | +197% (delays_zurich) |
+| **Tabular + LLM** | 23 (55%) | +6.12% | +420% (delays_zurich) |
+
+Models: RandomForest (n_estimators=200, max_depth=20), LogisticRegression/Ridge
 
-| Task Type | Average Improvement | Best Case |
-|-----------|--------------------:|----------:|
-| **Text Classification** | **+12.44%** | +49.02% (News Headlines) |
-| Time Series | +1.51% | +12.12% (Retail Demand) |
-| Classification | +0.54% | +4.35% |
-| Regression | +0.65% | +5.57% |
+### AutoML Benchmark (FLAML, 120s budget)
 
-### LLM Engine (With Copilot - 30-60s)
+| Metric | Value |
+|--------|-------|
+| **Datasets** | 41 |
+| **Improved** | 19 (46%) |
+| **Best Improvement** | +8.55% (abalone) |
 
-| Task Type | Average Improvement | Best Case |
-|-----------|--------------------:|----------:|
-| **Regression** | **+7.79%** | +19.66% (Retail Demand) |
-| Classification | +2.38% | +2.87% |
+### Key Results
 
-- ✅ **12/12 wins** on text classification (tabular mode)
-- 🧠 **+19.66% max improvement** with LLM-powered features
-- ⚡ **<1 second** (tabular) or **30-60s** (with LLM) processing time
-- 📈 Largest gains with simple models (LogisticRegression, Ridge)
+- ✅ **+197% improvement** on delays_zurich (tabular only)
+- 🧠 **+420% improvement** with LLM-enhanced features
+- 📈 **+8.98%** on abalone regression task
+- 🚀 **+5.68%** on complex_classification
 
 [View Full Benchmark Results](https://thinkall.github.io/featcopilot/user-guide/benchmarks/)
 
@@ -43,7 +50,7 @@ FeatCopilot is a unified feature engineering framework that combines the best ap
 # Basic installation
 pip install featcopilot
 
-# With LLM capabilities (requires GitHub Copilot)
+# With LLM capabilities
 pip install featcopilot[llm]
 
 # Full installation
@@ -67,12 +74,12 @@ X_transformed = engineer.fit_transform(X, y)  # <1 second
 print(f"Features: {X.shape[1]} -> {X_transformed.shape[1]}")
 ```
 
-### LLM Mode (With Copilot)
+### LLM Mode (With LiteLLM)
 
 ```python
 from featcopilot import AutoFeatureEngineer
 
-# LLM-powered semantic features (+19.66% max improvement)
+# LLM-powered semantic features (+420% max improvement)
 engineer = AutoFeatureEngineer(
     engines=['tabular', 'llm'],
     max_features=50
@@ -120,16 +127,24 @@ engine = TimeSeriesEngine(
 ```
 
 ### LLM Engine
-Uses GitHub Copilot SDK for intelligent feature generation.
+Uses GitHub Copilot SDK (default) or LiteLLM (100+ providers) for intelligent feature generation.
 
 ```python
 from featcopilot.llm import SemanticEngine
 
+# Default: GitHub Copilot SDK
 engine = SemanticEngine(
-    model='gpt-5',
+    model='gpt-5.2',
     max_suggestions=20,
     validate_features=True
 )
+
+# Alternative: LiteLLM backend
+engine = SemanticEngine(
+    model='gpt-4o',
+    backend='litellm',
+    max_suggestions=20
+)
 ```
 
 ## Feature Selection
@@ -167,7 +182,7 @@ X_selected = selector.fit_transform(X, y)
 
 - Python 3.9+
 - NumPy, Pandas, Scikit-learn
-- GitHub Copilot CLI (for LLM features)
+- GitHub Copilot SDK (default) or LiteLLM (for 100+ LLM providers)
 
 ## License
 

{featcopilot-0.1.0 → featcopilot-0.3.0}/featcopilot/__init__.py

@@ -5,11 +5,16 @@ A unified feature engineering framework combining traditional approaches
 with novel LLM-powered capabilities via GitHub Copilot SDK.
 """
 
-__version__ = "0.1.0"
+from importlib.metadata import version
+
+__version__ = version("featcopilot")
 __author__ = "FeatCopilot Contributors"
 
 from featcopilot.core.base import BaseEngine, BaseSelector
 from featcopilot.core.feature import Feature, FeatureSet
+from featcopilot.core.transform_rule import TransformRule
+from featcopilot.llm.transform_rule_generator import TransformRuleGenerator
+from featcopilot.stores.rule_store import TransformRuleStore
 from featcopilot.transformers.sklearn_compat import (
     AutoFeatureEngineer,
     FeatureEngineerTransformer,
@@ -21,6 +26,10 @@ __all__ = [
     "BaseSelector",
     "Feature",
     "FeatureSet",
+    # Transform Rules
+    "TransformRule",
+    "TransformRuleStore",
+    "TransformRuleGenerator",
     # Main API
     "AutoFeatureEngineer",
     "FeatureEngineerTransformer",

{featcopilot-0.1.0 → featcopilot-0.3.0}/featcopilot/core/__init__.py

@@ -3,6 +3,7 @@
 from featcopilot.core.base import BaseEngine, BaseSelector
 from featcopilot.core.feature import Feature, FeatureSet
 from featcopilot.core.registry import FeatureRegistry
+from featcopilot.core.transform_rule import TransformRule
 
 __all__ = [
     "BaseEngine",
@@ -10,4 +11,5 @@ __all__ = [
     "Feature",
     "FeatureSet",
     "FeatureRegistry",
+    "TransformRule",
 ]

{featcopilot-0.1.0 → featcopilot-0.3.0}/featcopilot/core/feature.py

@@ -7,6 +7,10 @@ from typing import Any, Optional
 import numpy as np
 import pandas as pd
 
+from featcopilot.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
 
 class FeatureType(Enum):
     """Types of features."""
@@ -220,5 +224,5 @@ class FeatureSet:
                     result[feature.name] = feature.compute(df)
                 except Exception as e:
                     # Log warning but continue
-                    print(f"Warning: Could not compute feature {feature.name}: {e}")
+                    logger.warning(f"Could not compute feature {feature.name}: {e}")
         return result

featcopilot-0.3.0/featcopilot/core/transform_rule.py

@@ -0,0 +1,276 @@
+"""Transform rule model for reusable feature transformations.
+
+Defines TransformRule - a reusable transformation that can be created from
+natural language descriptions and applied across different datasets.
+"""
+
+import re
+import uuid
+from datetime import datetime, timezone
+from typing import Any, Optional
+
+import numpy as np
+import pandas as pd
+from pydantic import BaseModel, Field
+
+from featcopilot.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class TransformRule(BaseModel):
+    """
+    A reusable feature transformation rule.
+
+    Transform rules capture feature engineering logic that can be generated
+    from natural language descriptions and reused across different datasets.
+
+    Parameters
+    ----------
+    id : str, optional
+        Unique identifier for the rule
+    name : str
+        Human-readable name for the rule
+    description : str
+        Natural language description of what the rule does
+    code : str
+        Python code that implements the transformation
+    input_columns : list[str]
+        Column names or patterns this rule expects as input
+    output_name : str, optional
+        Name for the output feature (default: derived from rule name)
+    output_type : str
+        Expected output data type ('numeric', 'categorical', 'boolean')
+    tags : list[str]
+        Tags for categorization and search
+    column_patterns : list[str]
+        Regex patterns for matching columns (e.g., 'price.*', '.*_amount')
+    usage_count : int
+        Number of times this rule has been applied
+    created_at : str
+        ISO timestamp of rule creation
+    metadata : dict
+        Additional metadata
+
+    Examples
+    --------
+    >>> rule = TransformRule(
+    ...     name="ratio_calculation",
+    ...     description="Calculate ratio of two numeric columns",
+    ...     code="result = df['{col1}'] / (df['{col2}'] + 1e-8)",
+    ...     input_columns=["col1", "col2"],
+    ...     tags=["ratio", "numeric"]
+    ... )
+    >>> result = rule.apply(df, column_mapping={"col1": "price", "col2": "quantity"})
+    """
+
+    id: str = Field(default_factory=lambda: str(uuid.uuid4())[:8], description="Unique rule identifier")
+    name: str = Field(description="Human-readable rule name")
+    description: str = Field(description="Natural language description of the transformation")
+    code: str = Field(description="Python code implementing the transformation")
+    input_columns: list[str] = Field(default_factory=list, description="Expected input column names or placeholders")
+    output_name: Optional[str] = Field(default=None, description="Output feature name")
+    output_type: str = Field(default="numeric", description="Output data type")
+    tags: list[str] = Field(default_factory=list, description="Tags for categorization")
+    column_patterns: list[str] = Field(default_factory=list, description="Regex patterns for column matching")
+    usage_count: int = Field(default=0, description="Number of times applied")
+    created_at: str = Field(
+        default_factory=lambda: datetime.now(timezone.utc).isoformat(), description="Creation timestamp"
+    )
+    metadata: dict[str, Any] = Field(default_factory=dict, description="Additional metadata")
+
+    def get_output_name(self, column_mapping: Optional[dict[str, str]] = None) -> str:
+        """
+        Get the output feature name.
+
+        Parameters
+        ----------
+        column_mapping : dict, optional
+            Mapping from placeholder columns to actual column names
+
+        Returns
+        -------
+        str
+            Output feature name
+        """
+        if self.output_name:
+            return self.output_name
+
+        # Generate name from input columns
+        if column_mapping and self.input_columns:
+            cols = [column_mapping.get(c, c) for c in self.input_columns[:2]]
+            return f"{'_'.join(cols)}_{self.name}"
+
+        return f"rule_{self.name}"
+
+    def matches_columns(self, columns: list[str]) -> tuple[bool, dict[str, str]]:
+        """
+        Check if this rule can be applied to the given columns.
+
+        Parameters
+        ----------
+        columns : list[str]
+            Available column names
+
+        Returns
+        -------
+        matches : bool
+            Whether the rule can be applied
+        mapping : dict
+            Suggested mapping from rule's input_columns to actual columns
+        """
+        if not self.input_columns:
+            return True, {}
+
+        mapping = {}
+
+        for input_col in self.input_columns:
+            # Try exact match first
+            if input_col in columns:
+                mapping[input_col] = input_col
+                continue
+
+            # Try pattern matching
+            matched = False
+            for pattern in self.column_patterns:
+                regex = re.compile(pattern, re.IGNORECASE)
+                for col in columns:
+                    if regex.match(col) and col not in mapping.values():
+                        mapping[input_col] = col
+                        matched = True
+                        break
+                if matched:
+                    break
+
+            # Try fuzzy matching by checking if input_col is substring
+            if not matched:
+                for col in columns:
+                    if input_col.lower() in col.lower() and col not in mapping.values():
+                        mapping[input_col] = col
+                        matched = True
+                        break
+
+            if not matched:
+                return False, {}
+
+        return len(mapping) == len(self.input_columns), mapping
+
+    def apply(
+        self,
+        df: pd.DataFrame,
+        column_mapping: Optional[dict[str, str]] = None,
+        validate: bool = True,
+    ) -> pd.Series:
+        """
+        Apply the transformation rule to a DataFrame.
+
+        Parameters
+        ----------
+        df : DataFrame
+            Input data
+        column_mapping : dict, optional
+            Mapping from rule's input_columns to actual column names
+        validate : bool, default=True
+            Whether to validate before execution
+
+        Returns
+        -------
+        Series
+            Transformed feature values
+
+        Raises
+        ------
+        ValueError
+            If required columns are missing or code execution fails
+        """
+        column_mapping = column_mapping or {}
+
+        # Prepare the code with actual column names
+        code = self._prepare_code(column_mapping)
+
+        if validate:
+            # Check required columns exist
+            for input_col in self.input_columns:
+                actual_col = column_mapping.get(input_col, input_col)
+                if actual_col not in df.columns:
+                    raise ValueError(f"Required column '{actual_col}' not found in DataFrame")
+
+        # Execute the code in a restricted environment
+        local_vars: dict[str, Any] = {"df": df, "np": np, "pd": pd}
+        try:
+            exec(self._get_safe_code(code), {"__builtins__": self._get_safe_builtins()}, local_vars)
+
+            if "result" not in local_vars:
+                raise ValueError("Code did not produce a 'result' variable")
+
+            result = local_vars["result"]
+
+            # Increment usage count
+            self.usage_count += 1
+
+            return result
+
+        except Exception as e:
+            logger.error(f"Failed to apply rule '{self.name}': {e}")
+            raise ValueError(f"Rule execution failed: {e}") from e
+
+    def _prepare_code(self, column_mapping: dict[str, str]) -> str:
+        """Substitute column placeholders with actual column names."""
+        code = self.code
+
+        # Replace {col} style placeholders
+        for placeholder, actual in column_mapping.items():
+            code = code.replace(f"{{{{ '{placeholder}' }}}}", f"'{actual}'")
+            code = code.replace(f"{{{placeholder}}}", actual)
+            code = code.replace(f"df['{placeholder}']", f"df['{actual}']")
+            code = code.replace(f'df["{placeholder}"]', f'df["{actual}"]')
+
+        return code
+
+    def _get_safe_code(self, code: str) -> str:
+        """Wrap code for safe execution."""
+        return code
+
+    def _get_safe_builtins(self) -> dict[str, Any]:
+        """Get restricted builtins for safe code execution."""
+        return {
+            "len": len,
+            "sum": sum,
+            "max": max,
+            "min": min,
+            "int": int,
+            "float": float,
+            "str": str,
+            "bool": bool,
+            "abs": abs,
+            "round": round,
+            "pow": pow,
+            "range": range,
+            "list": list,
+            "dict": dict,
+            "set": set,
+            "tuple": tuple,
+            "sorted": sorted,
+            "reversed": reversed,
+            "enumerate": enumerate,
+            "zip": zip,
+            "any": any,
+            "all": all,
+            "map": map,
+            "filter": filter,
+            "isinstance": isinstance,
+            "hasattr": hasattr,
+            "getattr": getattr,
+        }
+
+    def to_dict(self) -> dict[str, Any]:
+        """Convert rule to dictionary for serialization."""
+        return self.model_dump()
+
+    @classmethod
+    def from_dict(cls, data: dict[str, Any]) -> "TransformRule":
+        """Create rule from dictionary."""
+        return cls(**data)
+
+    def __repr__(self) -> str:
+        return f"TransformRule(name='{self.name}', description='{self.description[:50]}...')"

{featcopilot-0.1.0 → featcopilot-0.3.0}/featcopilot/engines/relational.py

@@ -11,6 +11,9 @@ from pydantic import Field
 
 from featcopilot.core.base import BaseEngine, EngineConfig
 from featcopilot.core.feature import FeatureSet
+from featcopilot.utils.logger import get_logger
+
+logger = get_logger(__name__)
 
 
 class RelationalEngineConfig(EngineConfig):
@@ -141,7 +144,7 @@ class RelationalEngine(BaseEngine):
         self._primary_columns = X.columns.tolist()
 
         if self.config.verbose:
-            print(f"RelationalEngine: {len(self._relationships)} relationships defined")
+            logger.info(f"RelationalEngine: {len(self._relationships)} relationships defined")
 
         self._is_fitted = True
         return self
@@ -191,7 +194,7 @@ class RelationalEngine(BaseEngine):
         self._feature_names = [c for c in result.columns if c not in X.columns]
 
         if self.config.verbose:
-            print(f"RelationalEngine: Generated {len(self._feature_names)} features")
+            logger.info(f"RelationalEngine: Generated {len(self._feature_names)} features")
 
         return result