equiaudit 0.1.1__tar.gz

equiaudit-0.1.1/PKG-INFO

@@ -0,0 +1,161 @@
+Metadata-Version: 2.4
+Name: equiaudit
+Version: 0.1.1
+Summary: An Agentic AI Framework for Fairness Auditing
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/pchmelo/EquiAudit
+Project-URL: Repository, https://github.com/pchmelo/EquiAudit
+Project-URL: Issues, https://github.com/pchmelo/EquiAudit/issues
+Keywords: fairness,bias,auditing,agentic-ai,machine-learning,tabular-data
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: aif360==0.6.1
+Requires-Dist: imbalanced-learn==0.14.1
+Requires-Dist: matplotlib>=3.10
+Requires-Dist: numpy>=2.2
+Requires-Dist: pandas>=2.2
+Requires-Dist: python-dotenv>=1.2
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: reportlab>=4.2
+Requires-Dist: requests>=2.32
+Requires-Dist: scikit-learn>=1.8
+Requires-Dist: scipy>=1.14
+Requires-Dist: seaborn>=0.13
+Requires-Dist: google-genai>=1.0
+Provides-Extra: gui
+Requires-Dist: streamlit>=1.52; extra == "gui"
+
+# EquiAudit — Dataset Quality & Fairness Evaluation System
+
+An AI-agent pipeline for evaluating datasets on data quality and fairness concerns, with an interactive Streamlit GUI, a headless CLI, and a Python API.
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/simple_diagram.png" alt="System overview" width="320"/>
+
+*A dataset is fed into EquiAudit, which coordinates AI agents under optional user supervision to produce an audit report and a bias-mitigated dataset.*
+</div>
+
+## How It Works
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/data_preparation_fairness_analysis.png" alt="Agent-tool-data interaction" width="600"/>
+
+*AI agents interact bidirectionally with the analyst, invoke tools to query and compute statistics on the dataset, and synthesise findings into an audit report.*
+</div>
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/mitigation.png" alt="Bias mitigation workflow" width="600"/>
+
+*Each selected mitigation technique produces a transformed dataset variant; tools recompute fairness statistics on every variant; the AI agent compares them against the original statistics and produces a comparative mitigation report.*
+</div>
+
+## Demo
+
+A video demonstration of the full GUI workflow is available on YouTube:
+[https://youtu.be/_USTmBhzDkI](https://youtu.be/_USTmBhzDkI)
+
+## Documentation
+
+| Guide | Description |
+|-------|-------------|
+| [USAGE.md](docs/USAGE.md) | Installation, configuration reference, all execution modes (GUI / CLI / Python API), and worked examples |
+| [EXTENDING.md](docs/EXTENDING.md) | How to add new Tools, Agent types, LLM backends, and Pipeline stages |
+
+## Installation
+
+**From PyPI (recommended):**
+```bash
+pip install equiaudit            # core pipeline
+pip install "equiaudit[gui]"     # optional: Streamlit GUI
+```
+
+**From source:**
+```bash
+git clone https://github.com/pchmelo/EquiAudit.git
+cd EquiAudit
+pip install -r requirements.txt          # core pipeline
+pip install -r requirements-gui.txt      # optional: Streamlit GUI
+```
+
+## Configuration
+
+Copy `examples/config.yml` and set your API keys as environment variables or in a `.env` file at the project root:
+
+```bash
+GOOGLE_API_KEY=your-google-gemini-api-key-here
+OPENROUTER_API_KEY=your-openrouter-api-key-here
+```
+
+At least one cloud API key is required unless running a local model via Ollama (`ollama serve`).
+
+See [USAGE.md — Configuration File](docs/USAGE.md#configuration-file) for the full annotated config reference.
+
+## Running the Application
+
+The recommended entry point is `examples/example_usage.py`. Set `mode: gui` or `mode: quick` in `examples/config.yml`, then run:
+
+```bash
+python examples/example_usage.py
+```
+
+The script dispatches to the GUI or the headless evaluator based on the config:
+
+```python
+# examples/example_usage.py
+import os, yaml
+from dotenv import load_dotenv
+load_dotenv()
+
+CONFIG_PATH  = os.path.join(os.path.dirname(__file__), "config.yml")
+DATASET_PATH = os.path.join(os.path.dirname(__file__), "adult-all.csv")
+
+with open(CONFIG_PATH, encoding="utf-8") as f:
+    _mode = (yaml.safe_load(f) or {}).get("mode", "quick")
+
+if _mode == "gui":
+    from equiaudit.gui import launch
+    launch(config_path=CONFIG_PATH, dataset_path=DATASET_PATH)
+else:
+    from equiaudit.cli import FairnessEvaluator
+    evaluator = FairnessEvaluator(config_path=CONFIG_PATH)
+    result = evaluator.evaluate(
+        data=DATASET_PATH,
+        # target="Income",                                          # overrides target_column in config
+        # sensitive_columns=["Sex", "Race", "Age"],                # overrides sensitive_attribute_analysis
+        # sensitive_pairs=[["Sex", "Race"], ["Age", "Education"]], # overrides pair_evaluation
+        # mitigation_techniques=["reweighting", "smote"],          # overrides mitigation_techniques
+    )
+    if result.success:
+        print(f"Report: {result.report_dir}")
+    else:
+        print(f"Error:  {result.error}")
+```
+
+See [USAGE.md — Execution Modes](docs/USAGE.md#execution-modes) for the full CLI flag reference and Python API usage.
+
+## Datasets
+Example datasets are provided in `src/equiaudit/data/` for testing. The default example is `adult-all.csv` (UCI Adult Census Income, target column `Income`).
+
+## Reports
+Generated reports are saved under `reports/<dataset>_<timestamp>/` and include a Markdown narrative, a PDF, per-group fairness metrics, and raw JSON stage data. A sample report is available in `reports/`.
+
+## GUI Screenshots
+
+![Framework configuration](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_1.png)
+*Framework configuration (dataset selection, model backend, target column, and pipeline options).*
+
+![Sensitive attribute identification](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_2.png)
+*Sensitive attribute identification (agent-generated candidate list pending analyst confirmation).*
+
+![Proxy model fairness results](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_3.png)
+*Proxy model fairness results (per-group metric bar charts from the outcome disparity stage).*
+
+![Previous results browser](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_4.png)
+*Previous results browser (inspect and compare past audit runs).*

equiaudit-0.1.1/README.md

@@ -0,0 +1,127 @@
+# EquiAudit — Dataset Quality & Fairness Evaluation System
+
+An AI-agent pipeline for evaluating datasets on data quality and fairness concerns, with an interactive Streamlit GUI, a headless CLI, and a Python API.
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/simple_diagram.png" alt="System overview" width="320"/>
+
+*A dataset is fed into EquiAudit, which coordinates AI agents under optional user supervision to produce an audit report and a bias-mitigated dataset.*
+</div>
+
+## How It Works
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/data_preparation_fairness_analysis.png" alt="Agent-tool-data interaction" width="600"/>
+
+*AI agents interact bidirectionally with the analyst, invoke tools to query and compute statistics on the dataset, and synthesise findings into an audit report.*
+</div>
+
+<div align="center">
+<img src="https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/mitigation.png" alt="Bias mitigation workflow" width="600"/>
+
+*Each selected mitigation technique produces a transformed dataset variant; tools recompute fairness statistics on every variant; the AI agent compares them against the original statistics and produces a comparative mitigation report.*
+</div>
+
+## Demo
+
+A video demonstration of the full GUI workflow is available on YouTube:
+[https://youtu.be/_USTmBhzDkI](https://youtu.be/_USTmBhzDkI)
+
+## Documentation
+
+| Guide | Description |
+|-------|-------------|
+| [USAGE.md](docs/USAGE.md) | Installation, configuration reference, all execution modes (GUI / CLI / Python API), and worked examples |
+| [EXTENDING.md](docs/EXTENDING.md) | How to add new Tools, Agent types, LLM backends, and Pipeline stages |
+
+## Installation
+
+**From PyPI (recommended):**
+```bash
+pip install equiaudit            # core pipeline
+pip install "equiaudit[gui]"     # optional: Streamlit GUI
+```
+
+**From source:**
+```bash
+git clone https://github.com/pchmelo/EquiAudit.git
+cd EquiAudit
+pip install -r requirements.txt          # core pipeline
+pip install -r requirements-gui.txt      # optional: Streamlit GUI
+```
+
+## Configuration
+
+Copy `examples/config.yml` and set your API keys as environment variables or in a `.env` file at the project root:
+
+```bash
+GOOGLE_API_KEY=your-google-gemini-api-key-here
+OPENROUTER_API_KEY=your-openrouter-api-key-here
+```
+
+At least one cloud API key is required unless running a local model via Ollama (`ollama serve`).
+
+See [USAGE.md — Configuration File](docs/USAGE.md#configuration-file) for the full annotated config reference.
+
+## Running the Application
+
+The recommended entry point is `examples/example_usage.py`. Set `mode: gui` or `mode: quick` in `examples/config.yml`, then run:
+
+```bash
+python examples/example_usage.py
+```
+
+The script dispatches to the GUI or the headless evaluator based on the config:
+
+```python
+# examples/example_usage.py
+import os, yaml
+from dotenv import load_dotenv
+load_dotenv()
+
+CONFIG_PATH  = os.path.join(os.path.dirname(__file__), "config.yml")
+DATASET_PATH = os.path.join(os.path.dirname(__file__), "adult-all.csv")
+
+with open(CONFIG_PATH, encoding="utf-8") as f:
+    _mode = (yaml.safe_load(f) or {}).get("mode", "quick")
+
+if _mode == "gui":
+    from equiaudit.gui import launch
+    launch(config_path=CONFIG_PATH, dataset_path=DATASET_PATH)
+else:
+    from equiaudit.cli import FairnessEvaluator
+    evaluator = FairnessEvaluator(config_path=CONFIG_PATH)
+    result = evaluator.evaluate(
+        data=DATASET_PATH,
+        # target="Income",                                          # overrides target_column in config
+        # sensitive_columns=["Sex", "Race", "Age"],                # overrides sensitive_attribute_analysis
+        # sensitive_pairs=[["Sex", "Race"], ["Age", "Education"]], # overrides pair_evaluation
+        # mitigation_techniques=["reweighting", "smote"],          # overrides mitigation_techniques
+    )
+    if result.success:
+        print(f"Report: {result.report_dir}")
+    else:
+        print(f"Error:  {result.error}")
+```
+
+See [USAGE.md — Execution Modes](docs/USAGE.md#execution-modes) for the full CLI flag reference and Python API usage.
+
+## Datasets
+Example datasets are provided in `src/equiaudit/data/` for testing. The default example is `adult-all.csv` (UCI Adult Census Income, target column `Income`).
+
+## Reports
+Generated reports are saved under `reports/<dataset>_<timestamp>/` and include a Markdown narrative, a PDF, per-group fairness metrics, and raw JSON stage data. A sample report is available in `reports/`.
+
+## GUI Screenshots
+
+![Framework configuration](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_1.png)
+*Framework configuration (dataset selection, model backend, target column, and pipeline options).*
+
+![Sensitive attribute identification](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_2.png)
+*Sensitive attribute identification (agent-generated candidate list pending analyst confirmation).*
+
+![Proxy model fairness results](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_3.png)
+*Proxy model fairness results (per-group metric bar charts from the outcome disparity stage).*
+
+![Previous results browser](https://raw.githubusercontent.com/pchmelo/EquiAudit/master/docs/img/new_4.png)
+*Previous results browser (inspect and compare past audit runs).*

equiaudit-0.1.1/pyproject.toml

@@ -0,0 +1,56 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "equiaudit"
+version = "0.1.1"
+description = "An Agentic AI Framework for Fairness Auditing"
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.10"
+keywords = ["fairness", "bias", "auditing", "agentic-ai", "machine-learning", "tabular-data"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+
+dependencies = [
+    "aif360==0.6.1",
+    "imbalanced-learn==0.14.1",
+    "matplotlib>=3.10",
+    "numpy>=2.2",
+    "pandas>=2.2",
+    "python-dotenv>=1.2",
+    "pyyaml>=6.0",
+    "reportlab>=4.2",
+    "requests>=2.32",
+    "scikit-learn>=1.8",
+    "scipy>=1.14",
+    "seaborn>=0.13",
+    "google-genai>=1.0",
+]
+
+[project.optional-dependencies]
+gui = ["streamlit>=1.52"]
+
+[project.scripts]
+equiaudit = "equiaudit.cli.__main__:main"
+
+[project.urls]
+Homepage = "https://github.com/pchmelo/EquiAudit"
+Repository = "https://github.com/pchmelo/EquiAudit"
+Issues = "https://github.com/pchmelo/EquiAudit/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+"equiaudit.models" = ["config.yml"]
+"equiaudit.pipeline" = ["pipeline_config.yml"]
+"equiaudit.data" = ["*.csv"]

equiaudit-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

equiaudit-0.1.1/src/equiaudit/__init__.py

@@ -0,0 +1,3 @@
+"""EquiAudit: An Agentic AI Framework for Fairness Auditing."""
+
+__version__ = "0.1.1"

equiaudit-0.1.1/src/equiaudit/cli/__init__.py

@@ -0,0 +1,7 @@
+from equiaudit.cli.evaluator import FairnessEvaluator, EvaluationResult, VerificationResult
+
+__all__ = [
+    "FairnessEvaluator",
+    "EvaluationResult",
+    "VerificationResult",
+]

equiaudit-0.1.1/src/equiaudit/cli/__main__.py

@@ -0,0 +1,204 @@
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+
+def main(args: argparse.Namespace = None) -> int:
+    """Main entry point for CLI."""
+    
+    if args is None:
+        args = parse_args()
+    
+    # Handle environment variable loading from .env file
+    if args.env_file:
+        load_env_file(args.env_file)
+    
+    # Import after env loading to ensure API keys are available
+    from equiaudit.cli.evaluator import FairnessEvaluator
+    
+    # Create evaluator
+    evaluator = FairnessEvaluator(
+        config_path=args.config,
+        output_dir=args.output,
+        model=args.model,
+        verbose=not args.quiet,
+    )
+    
+    # Verify mode
+    if args.verify:
+        result = evaluator.verify()
+        return 0 if result.success else 1
+    
+    # Evaluate mode - requires data
+    if not args.data:
+        print("Error: --data is required for evaluation mode")
+        print("Use --verify to check configuration only")
+        return 1
+    
+    # Run evaluation
+    result = evaluator.evaluate(
+        data=args.data,
+        target=args.target,
+        objective=args.objective,
+        sensitive_columns=args.sensitive.split(",") if args.sensitive else None,
+        generate_pdf=not args.no_pdf,
+    )
+    
+    if result.success:
+        print("\n" + "=" * 60)
+        print("EVALUATION COMPLETE")
+        print("=" * 60)
+        print(f"Dataset:    {result.dataset}")
+        print(f"Target:     {result.target_column or 'auto-detected'}")
+        print(f"Report Dir: {result.report_dir}")
+        if result.pdf_path:
+            print(f"PDF Report: {result.pdf_path}")
+        if result.markdown_path:
+            print(f"MD Report:  {result.markdown_path}")
+        print(f"Stages:     {', '.join(result.stages_completed)}")
+        if result.warnings:
+            print(f"\nWarnings ({len(result.warnings)}):")
+            for w in result.warnings:
+                print(f"  - {w}")
+        return 0
+    else:
+        print("\n" + "=" * 60)
+        print("EVALUATION FAILED")
+        print("=" * 60)
+        print(f"Error: {result.error}")
+        return 1
+
+
+def parse_args() -> argparse.Namespace:
+    """Parse command-line arguments."""
+    
+    parser = argparse.ArgumentParser(
+        prog="fairness-eval",
+        description="Evaluate datasets for fairness and bias issues.",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""
+Examples:
+  # Basic evaluation
+  equiaudit --data adult-all.csv --target Income
+
+  # Verify configuration
+  equiaudit --verify
+
+  # Custom config and output
+  equiaudit --config my_config.yml --data data.csv --output ./reports
+
+  # Specify model and sensitive columns
+  equiaudit --data data.csv --model gemini-flash --sensitive "age,sex,race"
+
+Environment Variables:
+  OPENROUTER_API_KEY  API key for OpenRouter models
+  GOOGLE_API_KEY      API key for Gemini models
+        """,
+    )
+    
+    # Data input
+    parser.add_argument(
+        "--data", "-d",
+        type=str,
+        help="Path to CSV dataset file",
+    )
+    
+    parser.add_argument(
+        "--target", "-t",
+        type=str,
+        default=None,
+        help="Target column name for classification (auto-detected if not specified)",
+    )
+    
+    parser.add_argument(
+        "--objective", "-o",
+        type=str,
+        default=None,
+        help="Custom evaluation objective/prompt",
+    )
+    
+    parser.add_argument(
+        "--sensitive", "-s",
+        type=str,
+        default=None,
+        help="Comma-separated list of sensitive columns (auto-detected if not specified)",
+    )
+    
+    # Configuration
+    parser.add_argument(
+        "--config", "-c",
+        type=str,
+        default=None,
+        help="Path to YAML configuration file (uses bundled default if omitted)",
+    )
+    
+    parser.add_argument(
+        "--model", "-m",
+        type=str,
+        default=None,
+        help="Override default model from config",
+    )
+    
+    parser.add_argument(
+        "--env-file", "-e",
+        type=str,
+        default=None,
+        help="Path to .env file for loading API keys",
+    )
+    
+    # Output
+    parser.add_argument(
+        "--output", "-O",
+        type=str,
+        default=None,
+        help="Output directory for reports (default: reports/)",
+    )
+    
+    parser.add_argument(
+        "--no-pdf",
+        action="store_true",
+        help="Skip PDF generation",
+    )
+    
+    # Modes
+    parser.add_argument(
+        "--verify", "-V",
+        action="store_true",
+        help="Verify configuration only (don't run evaluation)",
+    )
+    
+    parser.add_argument(
+        "--quiet", "-q",
+        action="store_true",
+        help="Suppress progress messages",
+    )
+    
+    return parser.parse_args()
+
+
+def load_env_file(env_path: str) -> None:
+    """Load environment variables from a .env file."""
+    env_path = Path(env_path)
+    if not env_path.exists():
+        print(f"Warning: .env file not found: {env_path}")
+        return
+    
+    print(f"Loading environment from: {env_path}")
+    
+    with open(env_path, "r") as f:
+        for line in f:
+            line = line.strip()
+            if not line or line.startswith("#"):
+                continue
+            if "=" in line:
+                key, value = line.split("=", 1)
+                key = key.strip()
+                value = value.strip().strip('"').strip("'")
+                os.environ[key] = value
+
+
+if __name__ == "__main__":
+    sys.exit(main())