greenmining 1.0.5__py3-none-any.whl → 1.0.7__py3-none-any.whl

{greenmining-1.0.5.dist-info → greenmining-1.0.7.dist-info}/METADATA

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: greenmining
-Version: 1.0.5
-Summary: Analyze GitHub repositories to identify green software engineering patterns and energy-efficient practices
+Version: 1.0.7
+Summary: An empirical Python library for Mining Software Repositories (MSR) in Green IT research
 Author-email: Adam Bouafia <a.bouafia@student.vu.nl>
 License: MIT
 Project-URL: Homepage, https://github.com/adam-bouafia/greenmining
@@ -9,7 +9,7 @@ Project-URL: Documentation, https://github.com/adam-bouafia/greenmining#readme
 Project-URL: Repository, https://github.com/adam-bouafia/greenmining
 Project-URL: Issues, https://github.com/adam-bouafia/greenmining/issues
 Project-URL: Changelog, https://github.com/adam-bouafia/greenmining/blob/main/CHANGELOG.md
-Keywords: green-software,gsf,sustainability,carbon-footprint,microservices,mining,repository-analysis,energy-efficiency,github-analysis
+Keywords: green-software,gsf,msr,mining-software-repositories,green-it,sustainability,carbon-footprint,energy-efficiency,repository-analysis,github-analysis,pydriller,empirical-software-engineering
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: Intended Audience :: Science/Research
@@ -45,6 +45,11 @@ Requires-Dist: ruff>=0.1.9; extra == "dev"
 Requires-Dist: mypy>=1.8.0; extra == "dev"
 Requires-Dist: build>=1.0.5; extra == "dev"
 Requires-Dist: twine>=4.0.2; extra == "dev"
+Provides-Extra: energy
+Requires-Dist: psutil>=5.9.0; extra == "energy"
+Requires-Dist: codecarbon>=2.3.0; extra == "energy"
+Provides-Extra: dashboard
+Requires-Dist: flask>=3.0.0; extra == "dashboard"
 Provides-Extra: docs
 Requires-Dist: sphinx>=7.2.0; extra == "docs"
 Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
@@ -53,15 +58,30 @@ Dynamic: license-file
 
 # greenmining
 
-Green mining for microservices repositories.
+An empirical Python library for Mining Software Repositories (MSR) in Green IT research.
 
 [![PyPI](https://img.shields.io/pypi/v/greenmining)](https://pypi.org/project/greenmining/)
 [![Python](https://img.shields.io/pypi/pyversions/greenmining)](https://pypi.org/project/greenmining/)
 [![License](https://img.shields.io/github/license/adam-bouafia/greenmining)](LICENSE)
+[![Documentation](https://img.shields.io/badge/docs-readthedocs-blue)](https://greenmining.readthedocs.io/)
 
 ## Overview
 
-`greenmining` is a Python library for analyzing GitHub repositories to identify green software engineering practices and energy-efficient patterns. It detects sustainable software patterns across cloud, web, AI, database, networking, and general categories.
+`greenmining` is a research-grade Python library designed for **empirical Mining Software Repositories (MSR)** studies in **Green IT**. It enables researchers and practitioners to:
+
+- **Mine repositories at scale** - Fetch and analyze GitHub repositories via GraphQL API with configurable filters
+- **Batch analysis with parallelism** - Analyze multiple repositories concurrently with configurable worker pools
+- **Classify green commits** - Detect 122 sustainability patterns from the Green Software Foundation (GSF) catalog
+- **Analyze any repository by URL** - Direct PyDriller-based analysis with support for private repositories
+- **Measure energy consumption** - RAPL, CodeCarbon, and CPU Energy Meter backends for power profiling
+- **Carbon footprint reporting** - CO2 emissions calculation with 20+ country profiles and cloud region support
+- **Power regression detection** - Identify commits that increased energy consumption
+- **Method-level analysis** - Per-method complexity and metrics via Lizard integration
+- **Version power comparison** - Compare power consumption across software versions
+- **Generate research datasets** - Statistical analysis, temporal trends, and publication-ready reports
+- **Web dashboard** - Flask-based interactive visualization of analysis results
+
+Whether you're conducting MSR research, analyzing green software adoption, or measuring the energy footprint of codebases, GreenMining provides the empirical toolkit you need.
 
 ## Installation
 
@@ -310,7 +330,137 @@ print(f"Top patterns: {stats['top_patterns'][:5]}")
 aggregator.export_to_csv(results, "output.csv")
 ```
 
-#### Batch Analysis
+#### URL-Based Repository Analysis
+
+```python
+from greenmining.services.local_repo_analyzer import LocalRepoAnalyzer
+
+analyzer = LocalRepoAnalyzer(
+    max_commits=200,
+    cleanup_after=True,
+)
+
+result = analyzer.analyze_repository("https://github.com/pallets/flask")
+
+print(f"Repository: {result.name}")
+print(f"Commits analyzed: {result.total_commits}")
+print(f"Green-aware: {result.green_commits} ({result.green_commit_rate:.1%})")
+
+for commit in result.commits[:5]:
+    if commit.green_aware:
+        print(f"  {commit.message[:60]}...")
+```
+
+#### Batch Analysis with Parallelism
+
+```python
+from greenmining import analyze_repositories
+
+results = analyze_repositories(
+    urls=[
+        "https://github.com/kubernetes/kubernetes",
+        "https://github.com/istio/istio",
+        "https://github.com/envoyproxy/envoy",
+    ],
+    max_commits=100,
+    parallel_workers=3,
+    energy_tracking=True,
+    energy_backend="auto",
+)
+
+for result in results:
+    print(f"{result.name}: {result.green_commit_rate:.1%} green")
+```
+
+#### Private Repository Analysis
+
+```python
+from greenmining.services.local_repo_analyzer import LocalRepoAnalyzer
+
+# HTTPS with token
+analyzer = LocalRepoAnalyzer(github_token="ghp_xxxx")
+result = analyzer.analyze_repository("https://github.com/company/private-repo")
+
+# SSH with key
+analyzer = LocalRepoAnalyzer(ssh_key_path="~/.ssh/id_rsa")
+result = analyzer.analyze_repository("git@github.com:company/private-repo.git")
+```
+
+#### Power Regression Detection
+
+```python
+from greenmining.analyzers import PowerRegressionDetector
+
+detector = PowerRegressionDetector(
+    test_command="pytest tests/ -x",
+    energy_backend="rapl",
+    threshold_percent=5.0,
+    iterations=5,
+)
+
+regressions = detector.detect(
+    repo_path="/path/to/repo",
+    baseline_commit="v1.0.0",
+    target_commit="HEAD",
+)
+
+for regression in regressions:
+    print(f"Commit {regression.sha[:8]}: +{regression.power_increase:.1f}%")
+```
+
+#### Version Power Comparison
+
+```python
+from greenmining.analyzers import VersionPowerAnalyzer
+
+analyzer = VersionPowerAnalyzer(
+    test_command="pytest tests/",
+    energy_backend="rapl",
+    iterations=10,
+    warmup_iterations=2,
+)
+
+report = analyzer.analyze_versions(
+    repo_path="/path/to/repo",
+    versions=["v1.0", "v1.1", "v1.2", "v2.0"],
+)
+
+print(report.summary())
+print(f"Trend: {report.trend}")
+print(f"Most efficient: {report.most_efficient}")
+```
+
+#### Metrics-to-Power Correlation
+
+```python
+from greenmining.analyzers import MetricsPowerCorrelator
+
+correlator = MetricsPowerCorrelator()
+correlator.fit(
+    metrics=["complexity", "nloc", "code_churn"],
+    metrics_values={
+        "complexity": [10, 20, 30, 40],
+        "nloc": [100, 200, 300, 400],
+        "code_churn": [50, 100, 150, 200],
+    },
+    power_measurements=[5.0, 8.0, 12.0, 15.0],
+)
+
+print(f"Pearson: {correlator.pearson}")
+print(f"Spearman: {correlator.spearman}")
+print(f"Feature importance: {correlator.feature_importance}")
+```
+
+#### Web Dashboard
+
+```python
+from greenmining.dashboard import run_dashboard
+
+# Launch interactive dashboard (requires pip install greenmining[dashboard])
+run_dashboard(data_dir="./data", host="127.0.0.1", port=5000)
+```
+
+#### Pipeline Batch Analysis
 
 ```python
 from greenmining.controllers.repository_controller import RepositoryController
@@ -531,17 +681,24 @@ config = Config(
 
 ### Core Capabilities
 
-- **Pattern Detection**: Automatically identifies 122 sustainability patterns across 15 categories
-- **Keyword Analysis**: Scans commit messages using 321 green software keywords
-- **Custom Repository Fetching**: Fetch repositories with custom search keywords (not limited to microservices)
-- **Repository Analysis**: Analyzes repositories from GitHub with flexible filtering
-- **Batch Processing**: Analyze hundreds of repositories and thousands of commits
-- **Multi-format Output**: Generates Markdown reports, CSV exports, and JSON data
-- **Statistical Analysis**: Calculates green-awareness metrics, pattern distribution, and trends
+- **Pattern Detection**: 122 sustainability patterns across 15 categories from the GSF catalog
+- **Keyword Analysis**: 321 green software detection keywords
+- **Repository Fetching**: GraphQL API with date, star, and language filters
+- **URL-Based Analysis**: Direct PyDriller analysis from GitHub URLs (HTTPS and SSH)
+- **Batch Processing**: Parallel analysis of multiple repositories with configurable workers
+- **Private Repository Support**: Authentication via SSH keys or GitHub tokens
+- **Energy Measurement**: RAPL, CodeCarbon, and CPU Energy Meter backends
+- **Carbon Footprint Reporting**: CO2 emissions with 20+ country profiles and cloud region support (AWS, GCP, Azure)
+- **Power Regression Detection**: Identify commits that increased energy consumption
+- **Metrics-to-Power Correlation**: Pearson and Spearman analysis between code metrics and power
+- **Version Power Comparison**: Compare power consumption across software versions with trend detection
+- **Method-Level Analysis**: Per-method complexity metrics via Lizard integration
+- **Source Code Access**: Before/after source code for refactoring detection
+- **Full Process Metrics**: All 8 PyDriller process metrics (ChangeSet, CodeChurn, CommitsCount, ContributorsCount, ContributorsExperience, HistoryComplexity, HunksCount, LinesCount)
+- **Statistical Analysis**: Correlations, effect sizes, and temporal trends
+- **Multi-format Output**: Markdown reports, CSV exports, JSON data
+- **Web Dashboard**: Flask-based interactive visualization (`pip install greenmining[dashboard]`)
 - **Docker Support**: Pre-built images for containerized analysis
-- **Programmatic API**: Full Python API for custom workflows and integrations
-- **Clean Architecture**: Modular design with services layer (Fetcher, Extractor, Analyzer, Aggregator, Reports)
-- **Energy Measurement**: Real-time energy consumption tracking via RAPL (Linux) or CodeCarbon (cross-platform)
 
 ### Energy Measurement
 
@@ -553,38 +710,44 @@ greenmining includes built-in energy measurement capabilities for tracking the c
 |---------|----------|---------|--------------|
 | **RAPL** | Linux (Intel/AMD) | CPU/RAM energy (Joules) | `/sys/class/powercap/` access |
 | **CodeCarbon** | Cross-platform | Energy + Carbon emissions (gCO2) | `pip install codecarbon` |
+| **CPU Meter** | All platforms | Estimated CPU energy (Joules) | Optional: `pip install psutil` |
+| **Auto** | All platforms | Best available backend | Automatic detection |
 
 #### Python API
 
 ```python
-from greenmining.energy import RAPLEnergyMeter, CodeCarbonMeter
-
-# RAPL (Linux only)
-rapl = RAPLEnergyMeter()
-if rapl.is_available():
-    rapl.start()
-    # ... run analysis ...
-    result = rapl.stop()
-    print(f"Energy: {result.energy_joules:.2f} J")
-
-# CodeCarbon (cross-platform)
-cc = CodeCarbonMeter()
-if cc.is_available():
-    cc.start()
-    # ... run analysis ...
-    result = cc.stop()
-    print(f"Energy: {result.energy_joules:.2f} J")
-    print(f"Carbon: {result.carbon_grams:.4f} gCO2")
+from greenmining.energy import RAPLEnergyMeter, CPUEnergyMeter, get_energy_meter
+
+# Auto-detect best backend
+meter = get_energy_meter("auto")
+meter.start()
+# ... run analysis ...
+result = meter.stop()
+print(f"Energy: {result.joules:.2f} J")
+print(f"Power: {result.watts_avg:.2f} W")
+
+# Integrated energy tracking during analysis
+from greenmining.services.local_repo_analyzer import LocalRepoAnalyzer
+
+analyzer = LocalRepoAnalyzer(energy_tracking=True, energy_backend="auto")
+result = analyzer.analyze_repository("https://github.com/pallets/flask")
+print(f"Analysis energy: {result.energy_metrics['joules']:.2f} J")
 ```
 
-#### Experiment Results
+#### Carbon Footprint Reporting
 
-CodeCarbon was verified with a real experiment:
-- **Repository**: flask (pallets/flask)
-- **Commits analyzed**: 10
-- **Energy measured**: 160.6 J
-- **Carbon emissions**: 0.0119 gCO2
-- **Duration**: 11.28 seconds
+```python
+from greenmining.energy import CarbonReporter
+
+reporter = CarbonReporter(
+    country_iso="USA",
+    cloud_provider="aws",
+    region="us-east-1",
+)
+report = reporter.generate_report(total_joules=3600.0)
+print(f"CO2: {report.total_emissions_kg * 1000:.4f} grams")
+print(f"Equivalent: {report.tree_months:.2f} tree-months to offset")
+```
 
 ### Pattern Database
 
@@ -687,8 +850,14 @@ ruff check greenmining/ tests/
 - PyGithub >= 2.1.1
 - PyDriller >= 2.5
 - pandas >= 2.2.0
-- click >= 8.1.7
-- codecarbon >= 2.0.0 (optional, for cross-platform energy measurement)
+
+**Optional dependencies:**
+
+```bash
+pip install greenmining[energy]      # psutil, codecarbon (energy measurement)
+pip install greenmining[dashboard]   # flask (web dashboard)
+pip install greenmining[dev]         # pytest, black, ruff, mypy (development)
+```
 
 ## License
 

{greenmining-1.0.5.dist-info → greenmining-1.0.7.dist-info}/RECORD

@@ -1,20 +1,27 @@
-greenmining/__init__.py,sha256=wlLxbv7hzKNy1yWAEZckcSpYkC1rYDIHU9bjQZHs9es,992
+greenmining/__init__.py,sha256=C9NT6lUmL91LkE0eGUFPrMLvZJYpdhTNYePqZxn1fpg,2909
 greenmining/__main__.py,sha256=NYOVS7D4w2XDLn6SyXHXPKE5GrNGOeoWSTb_KazgK5c,590
 greenmining/__version__.py,sha256=xZc02a8bS3vUJlzh8k9RoxemB1irQmq_SpVVj6Cg5M0,62
-greenmining/config.py,sha256=00v1Ln8eZE0RxMrLxvonf8XOWqeRYaIme_iC1yDLR90,8228
+greenmining/config.py,sha256=M4a7AwM1ErCmOY0n5Vmyoo9HPblSkTZ-HD3k2YHzs4A,8340
 greenmining/gsf_patterns.py,sha256=hnd9GuWB8GEflrusEib5hjvl8CD5TSbGcBtb0gfxFp4,54193
 greenmining/utils.py,sha256=dSFwQzQwbS8rYZSgwLIxM_geLqbldwqVOIXMqEg08Qs,5609
-greenmining/analyzers/__init__.py,sha256=VI22zb3TnhZrwEuBy0J3pIbqNVVZl1wx8NYRlhR6Wok,362
+greenmining/analyzers/__init__.py,sha256=rTgpDfFE6za4QAHW59ncnS6TW02omn-TZMnYNVUIZp0,753
 greenmining/analyzers/code_diff_analyzer.py,sha256=1dk68R3O0RZG8gx1cm9B_UlZ1Uwyb_Q3oScRbCVx4tM,10950
+greenmining/analyzers/metrics_power_correlator.py,sha256=qMKr4hSTzT0Un3vsGZNkPCp9TxyzdFwrhjw5M1IKOgk,5964
+greenmining/analyzers/power_regression.py,sha256=5pxs7IoTtGcwwX5KzGeM5hOm2I9Axr-0X4N_4007iMw,7387
 greenmining/analyzers/qualitative_analyzer.py,sha256=RcjOMLj_DPH869ey9J0uI7JK_krCefMhNkPLOJUDFF8,15391
 greenmining/analyzers/statistical_analyzer.py,sha256=DzWAcCyw42Ig3FIxTwPPBikgt2uzMdktxklonOYfnOk,7166
 greenmining/analyzers/temporal_analyzer.py,sha256=JfTcAoI20oCFMehGrSRnDqhJTXI-RUbdCTMwDOTW9-g,14259
+greenmining/analyzers/version_power_analyzer.py,sha256=2P6zOqBg-ButtIhF-4cutiwD2Q1geMY49VFUghHXXoI,8119
 greenmining/controllers/__init__.py,sha256=UiAT6zBvC1z_9cJWfzq1cLA0I4r9b2vURHipj8oDczI,180
 greenmining/controllers/repository_controller.py,sha256=fyL6Y8xpoixDplP4_rKWiwak42M2DaIihzyKVaBlivA,9680
-greenmining/energy/__init__.py,sha256=Y9RkNuZ3T6npEBxOZJhVc8wy6feXQePdXojLaZxkfGM,308
-greenmining/energy/base.py,sha256=s0yyRDhnEkrkCE5cgp2yHOrIhKbCpU9V7n4Rf1ejWLM,5559
-greenmining/energy/codecarbon_meter.py,sha256=z4T63qunEsU2R2qylZdGCtk3e-y_HYaBjBMD3nuFRU0,5102
-greenmining/energy/rapl.py,sha256=nZoVmdZshSsFLEYWNQxYyNg8fhhzgNME02bpQuIQL_U,5584
+greenmining/dashboard/__init__.py,sha256=Ig_291-hLrH9k3rV0whhQ1EkhiaRR8ciHiJ5s5OCBf4,141
+greenmining/dashboard/app.py,sha256=Hk6_i2qmcg6SGW7UzxglEIvUBJiloRA-hMYI-YSORcA,8604
+greenmining/energy/__init__.py,sha256=GoCYh7hitWBoPMtan1HF1yezCHi7o4sa_YUJgGkeJc8,558
+greenmining/energy/base.py,sha256=mVdp-E04KWu3UnHFL61pzrI-OP-KsUshAmXHwgsJkRU,5749
+greenmining/energy/carbon_reporter.py,sha256=bKIFlLhHfYzI4DBu_ff4GW1Psz4oSCAF4NmzQb-EShA,8298
+greenmining/energy/codecarbon_meter.py,sha256=HyQptyEaS1ZMu_qdxg0Tyuly1PCmmbbNwwYX8qYsTs4,4927
+greenmining/energy/cpu_meter.py,sha256=mhEG3Y7fjz3wV5lojcYeFXvCXXgmelGQaBfN2q7yTNc,5007
+greenmining/energy/rapl.py,sha256=b63M1mS7uF9Uo0vFi0z7Qwdo56U1TqxIYQXINhYp9Jo,5292
 greenmining/models/__init__.py,sha256=2hkB0quhMePvvA1AkYfj5uiF_HyGtXVxn0BU-5m_oSg,302
 greenmining/models/aggregated_stats.py,sha256=N-ZGcQO7IJ33Joa8luMVjtHhKYzNe48VW8hFqs9a5Jc,1016
 greenmining/models/analysis_result.py,sha256=YICTCEcrJxZ1R8Xaio3AZOjCGwMzC_62BMAL0J_XY1w,1509
@@ -22,16 +29,16 @@ greenmining/models/commit.py,sha256=mnRDWSiIyGtJeGXI8sav9hukWUyVFpoNe6GixRlZjY4,
 greenmining/models/repository.py,sha256=SKjS01onOptpMioumtAPZxKpKheHAeVXnXyvatl7CfM,2856
 greenmining/presenters/__init__.py,sha256=d1CMtqtUAHYHYNzigPyjtGOUtnH1drtUwf7-bFQq2B8,138
 greenmining/presenters/console_presenter.py,sha256=XOahvlcr4qLbUdhk8cGq1ZWagvemEd3Wgriu8T5EI3s,4896
-greenmining/services/__init__.py,sha256=UhjS2X9x2v5iH991UDPazP3dTPuSgylMq4kQJaueQYs,481
-greenmining/services/commit_extractor.py,sha256=3EfUVBwd8hGSbl7pS-_jAL8gX8RxIASXTX5EZBbKQPI,8387
+greenmining/services/__init__.py,sha256=ZEMOVut0KRdume_vz58beSNps3YgeoGBXmUjEqNgIhc,690
+greenmining/services/commit_extractor.py,sha256=Fz2WTWjIZ_vQhSfkJKnWpJnBpI2nm0KacA4qYAvCpSE,8451
 greenmining/services/data_aggregator.py,sha256=TsFT0oGOnnHk0QGZ1tT6ZhKGc5X1H1D1u7-7OpiPo7Y,19566
 greenmining/services/data_analyzer.py,sha256=f0nlJkPAclHHCzzTyQW5bjhYrgE0XXiR1x7_o3fJaDs,9732
 greenmining/services/github_fetcher.py,sha256=mUcmQevhdDRYX72O-M7Vi-s3y4ZwNyKewleti838cqU,8285
 greenmining/services/github_graphql_fetcher.py,sha256=p76vp5EgStzkmTcws__jb90za8m61toW0CBrwrm5Ew4,11972
-greenmining/services/local_repo_analyzer.py,sha256=IrfqY1L6peGO78zufEj4uAU1N7nskc0edAYVzE0Ew_w,14785
-greenmining/services/reports.py,sha256=7Smc7a4KtpmkAJ8UoMlzH5BZerC_iO_jMyQw3_42n1s,23387
-greenmining-1.0.5.dist-info/licenses/LICENSE,sha256=M7ma3JHGeiIZIs3ea0HTcFl_wLFPX2NZElUliYs4bCA,1083
-greenmining-1.0.5.dist-info/METADATA,sha256=UIp-sji0KZ4GOtLaZOsPjaT9Qb0uDfuzFSIpw-UEcjQ,24899
-greenmining-1.0.5.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
-greenmining-1.0.5.dist-info/top_level.txt,sha256=nreXgXxZIWI-42yQknQ0HXtUrFnzZ8N1ra4Mdy2KcsI,12
-greenmining-1.0.5.dist-info/RECORD,,
+greenmining/services/local_repo_analyzer.py,sha256=5DMN9RIyGXNdsOlIDV4Mp0fPavbB69oBA9us17P5cNo,24668
+greenmining/services/reports.py,sha256=Vrw_pBNmVw2mTAf1dpcAqjBe6gXv-O4w_XweoVTt7L8,23392
+greenmining-1.0.7.dist-info/licenses/LICENSE,sha256=M7ma3JHGeiIZIs3ea0HTcFl_wLFPX2NZElUliYs4bCA,1083
+greenmining-1.0.7.dist-info/METADATA,sha256=p0Go_jSsrZ04Lvi0-I2HQY1ycWloEbD3agXZnioCFB4,30913
+greenmining-1.0.7.dist-info/WHEEL,sha256=wUyA8OaulRlbfwMtmQsvNngGrxQHAvkKcvRmdizlJi0,92
+greenmining-1.0.7.dist-info/top_level.txt,sha256=nreXgXxZIWI-42yQknQ0HXtUrFnzZ8N1ra4Mdy2KcsI,12
+greenmining-1.0.7.dist-info/RECORD,,