greenmining 0.1.10__tar.gz → 0.1.12__tar.gz

{greenmining-0.1.10 → greenmining-0.1.12}/CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog
 
+## [0.1.12] - 2025-12-03
+
+### Added
+- Custom search keywords for repository fetching (`--keywords` option)
+- `fetch_repositories()` function exposed in public API
+- Users can now search for any topic (kubernetes, docker, serverless, etc.)
+
+### Changed
+- README updated to reflect 122 patterns (was showing 76 in PyPI description)
+- CLI `fetch` command now accepts `--keywords` parameter
+- Repository fetching no longer hardcoded to "microservices"
+
+### Fixed
+- Outdated pattern count in PyPI package description
+
+## [0.1.11] - 2025-12-03
+
+### Added
+- Expanded pattern database from 76 to 122 patterns
+- Added 9 new categories (Resource, Caching, Data, Async, Code, Monitoring, Network, Microservices, Infrastructure)
+- Expanded keywords from 190 to 321
+- VU Amsterdam 2024 research patterns for ML systems
+
+### Changed
+- README with comprehensive feature documentation
+- Detection rate improved to 37.15% (up from 33.79%)
+
 ## [0.1.7] - 2025-12-02
 
 ### Added

{greenmining-0.1.10/greenmining.egg-info → greenmining-0.1.12}/PKG-INFO

@@ -1,15 +1,14 @@
 Metadata-Version: 2.4
 Name: greenmining
-Version: 0.1.10
+Version: 0.1.12
 Summary: Analyze GitHub repositories to identify green software engineering patterns and energy-efficient practices
-Author-email: Your Name <your.email@example.com>
-Maintainer-email: Your Name <your.email@example.com>
+Author-email: Adam Bouafia <a.bouafia@student.vu.nl>
 License: MIT
-Project-URL: Homepage, https://github.com/yourusername/greenmining
-Project-URL: Documentation, https://github.com/yourusername/greenmining#readme
-Project-URL: Repository, https://github.com/yourusername/greenmining
-Project-URL: Issues, https://github.com/yourusername/greenmining/issues
-Project-URL: Changelog, https://github.com/yourusername/greenmining/blob/main/CHANGELOG.md
+Project-URL: Homepage, https://github.com/adam-bouafia/greenmining
+Project-URL: Documentation, https://github.com/adam-bouafia/greenmining#readme
+Project-URL: Repository, https://github.com/adam-bouafia/greenmining
+Project-URL: Issues, https://github.com/adam-bouafia/greenmining/issues
+Project-URL: Changelog, https://github.com/adam-bouafia/greenmining/blob/main/CHANGELOG.md
 Keywords: green-software,gsf,sustainability,carbon-footprint,microservices,mining,repository-analysis,energy-efficiency,github-analysis
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
@@ -63,11 +62,11 @@ Green mining for microservices repositories.
 
 ## Overview
 
-`greenmining` is a Python library and CLI tool for analyzing GitHub repositories to identify green software engineering practices and energy-efficient patterns. It detects 76 sustainable software patterns across cloud, web, AI, database, networking, and general categories.
+`greenmining` is a Python library and CLI tool for analyzing GitHub repositories to identify green software engineering practices and energy-efficient patterns. It detects 122 sustainable software patterns across cloud, web, AI, database, networking, and general categories, including advanced patterns from VU Amsterdam 2024 research on green architectural tactics for ML systems.
 
 ## Features
 
-- 🔍 **76 Sustainability Patterns**: Detect energy-efficient and environmentally conscious coding practices
+- 🔍 **122 Sustainability Patterns**: Detect energy-efficient and environmentally conscious coding practices across 15 categories (expanded from 76)
 - 📊 **Repository Mining**: Analyze 100+ microservices repositories from GitHub
 - 📈 **Green Awareness Detection**: Identify sustainability-focused commits
 - 📄 **Comprehensive Reports**: Generate analysis reports in multiple formats
@@ -107,7 +106,10 @@ export GITHUB_TOKEN="your_github_token"
 # Run full analysis pipeline
 greenmining pipeline --max-repos 100
 
-# Fetch repositories
+# Fetch repositories with custom keywords
+greenmining fetch --max-repos 100 --min-stars 100 --keywords "kubernetes docker cloud-native"
+
+# Fetch with default (microservices)
 greenmining fetch --max-repos 100 --min-stars 100
 
 # Extract commits
@@ -128,7 +130,7 @@ greenmining report
 from greenmining import GSF_PATTERNS, is_green_aware, get_pattern_by_keywords
 
 # Check available patterns
-print(f"Total patterns: {len(GSF_PATTERNS)}")  # 76
+print(f"Total patterns: {len(GSF_PATTERNS)}")  # 122 patterns across 15 categories
 
 # Detect green awareness in commit messages
 commit_msg = "Optimize Redis caching to reduce energy consumption"
@@ -138,22 +140,42 @@ if is_green_aware(commit_msg):
     # Output: ['Cache Static Data', 'Use Efficient Cache Strategies']
 ```
 
+#### Fetch Repositories with Custom Keywords (NEW)
+
+```python
+from greenmining import fetch_repositories
+
+# Fetch repositories with custom search keywords
+repos = fetch_repositories(
+    github_token="your_github_token",
+    max_repos=50,
+    min_stars=500,
+    keywords="kubernetes cloud-native",
+    languages=["Python", "Go"]
+)
+
+print(f"Found {len(repos)} repositories")
+for repo in repos[:5]:
+    print(f"- {repo.full_name} ({repo.stars} stars)")
+```
+
 #### Analyze Repository Commits
 
 ```python
-from greenmining.services.github_fetcher import GitHubFetcher
 from greenmining.services.commit_extractor import CommitExtractor
 from greenmining.services.data_analyzer import DataAnalyzer
-from greenmining.config import Config
+from greenmining import fetch_repositories
 
-# Initialize services
-config = Config()
-fetcher = GitHubFetcher(config)
-extractor = CommitExtractor(config)
-analyzer = DataAnalyzer(config)
+# Fetch repositories with custom keywords
+repos = fetch_repositories(
+    github_token="your_token",
+    max_repos=10,
+    keywords="serverless edge-computing"
+)
 
-# Fetch repositories
-repos = fetcher.fetch_repositories(max_repos=10, min_stars=100)
+# Initialize services
+extractor = CommitExtractor()
+analyzer = DataAnalyzer()
 
 # Extract commits from first repo
 commits = extractor.extract_commits(repos[0], max_commits=50)
@@ -173,12 +195,18 @@ for commit in commits:
 ```python
 from greenmining import GSF_PATTERNS
 
-# Get all cloud patterns
+# Get all patterns by category
 cloud_patterns = {
     pid: pattern for pid, pattern in GSF_PATTERNS.items()
     if pattern['category'] == 'cloud'
 }
-print(f"Cloud patterns: {len(cloud_patterns)}")
+print(f"Cloud patterns: {len(cloud_patterns)}")  # 40 patterns
+
+ai_patterns = {
+    pid: pattern for pid, pattern in GSF_PATTERNS.items()
+    if pattern['category'] == 'ai'
+}
+print(f"AI/ML patterns: {len(ai_patterns)}")  # 19 patterns
 
 # Get pattern details
 cache_pattern = GSF_PATTERNS['gsf_001']
@@ -186,6 +214,13 @@ print(f"Pattern: {cache_pattern['name']}")
 print(f"Category: {cache_pattern['category']}")
 print(f"Keywords: {cache_pattern['keywords']}")
 print(f"Impact: {cache_pattern['sci_impact']}")
+
+# List all available categories
+categories = set(p['category'] for p in GSF_PATTERNS.values())
+print(f"Available categories: {sorted(categories)}")
+# Output: ['ai', 'async', 'caching', 'cloud', 'code', 'data', 
+#          'database', 'general', 'infrastructure', 'microservices',
+#          'monitoring', 'network', 'networking', 'resource', 'web']
 ```
 
 #### Generate Custom Reports
@@ -256,26 +291,127 @@ COMMITS_PER_REPO=50
 OUTPUT_DIR=./data
 ```
 
+## Features
+
+### Core Capabilities
+
+- **Pattern Detection**: Automatically identifies 122 sustainability patterns across 15 categories
+- **Keyword Analysis**: Scans commit messages using 321 green software keywords
+- **Custom Repository Fetching**: Fetch repositories with custom search keywords (not limited to microservices)
+- **Repository Analysis**: Analyzes repositories from GitHub with flexible filtering
+- **Batch Processing**: Analyze hundreds of repositories and thousands of commits
+- **Multi-format Output**: Generates Markdown reports, CSV exports, and JSON data
+- **Statistical Analysis**: Calculates green-awareness metrics, pattern distribution, and trends
+- **Docker Support**: Pre-built images for containerized analysis
+- **Programmatic API**: Full Python API for custom workflows and integrations
+- **Clean Architecture**: Modular design with services layer (Fetcher, Extractor, Analyzer, Aggregator, Reports)
+
+### Pattern Database
+
+**122 green software patterns based on:**
+- Green Software Foundation (GSF) Patterns Catalog
+- VU Amsterdam 2024 research on ML system sustainability
+- ICSE 2024 conference papers on sustainable software
+
+### Detection Performance
+
+- **Coverage**: 67% of patterns actively detect in real-world commits
+- **Accuracy**: 100% true positive rate for green-aware commits
+- **Categories**: 15 distinct sustainability domains covered
+- **Keywords**: 321 detection terms across all patterns
+
 ## GSF Pattern Categories
 
-- **Cloud** (40 patterns): Autoscaling, serverless, right-sizing, region selection
-- **Web** (15 patterns): CDN, caching, lazy loading, compression
-- **AI/ML** (8 patterns): Model optimization, pruning, quantization
-- **Database** (6 patterns): Indexing, query optimization, connection pooling
-- **Networking** (4 patterns): Protocol optimization, connection reuse
-- **General** (3 patterns): Code efficiency, resource management
+**122 patterns across 15 categories:**
+
+### 1. Cloud (40 patterns)
+Auto-scaling, serverless computing, right-sizing instances, region selection for renewable energy, spot instances, idle resource detection, cloud-native architectures
+
+### 2. Web (17 patterns)
+CDN usage, caching strategies, lazy loading, asset compression, image optimization, minification, code splitting, tree shaking, prefetching
+
+### 3. AI/ML (19 patterns)
+Model optimization, pruning, quantization, edge inference, batch optimization, efficient training, model compression, hardware acceleration, green ML pipelines
+
+### 4. Database (5 patterns)
+Indexing strategies, query optimization, connection pooling, prepared statements, database views, denormalization for efficiency
+
+### 5. Networking (8 patterns)
+Protocol optimization, connection reuse, HTTP/2, gRPC, efficient serialization, compression, persistent connections
+
+### 6. Network (6 patterns)
+Request batching, GraphQL optimization, API gateway patterns, circuit breakers, rate limiting, request deduplication
+
+### 7. Caching (2 patterns)
+Multi-level caching, cache invalidation strategies, data deduplication, distributed caching
+
+### 8. Resource (2 patterns)
+Resource limits, dynamic allocation, memory management, CPU throttling
+
+### 9. Data (3 patterns)
+Efficient serialization formats, pagination, streaming, data compression
+
+### 10. Async (3 patterns)
+Event-driven architecture, reactive streams, polling elimination, non-blocking I/O
+
+### 11. Code (4 patterns)
+Algorithm optimization, code efficiency, garbage collection tuning, memory profiling
+
+### 12. Monitoring (3 patterns)
+Energy monitoring, performance profiling, APM tools, observability patterns
+
+### 13. Microservices (4 patterns)
+Service decomposition, colocation strategies, graceful shutdown, service mesh optimization
+
+### 14. Infrastructure (4 patterns)
+Alpine containers, Infrastructure as Code, renewable energy regions, container optimization
+
+### 15. General (8 patterns)
+Feature flags, incremental processing, precomputation, background jobs, workflow optimization
 
 ## CLI Commands
 
-| Command | Description |
-|---------|-------------|
-| `fetch` | Fetch microservices repositories from GitHub |
-| `extract` | Extract commit history from repositories |
-| `analyze` | Analyze commits for green patterns |
-| `aggregate` | Aggregate analysis results |
-| `report` | Generate comprehensive report |
-| `pipeline` | Run complete analysis pipeline |
-| `status` | Show current analysis status |
+| Command | Description | Key Options |
+|---------|-------------|-------------|
+| `fetch` | Fetch repositories from GitHub with custom keywords | `--max-repos`, `--min-stars`, `--languages`, `--keywords` |
+| `extract` | Extract commit history from repositories | `--max-commits` per repository |
+| `analyze` | Analyze commits for green patterns | Auto-detects patterns from 122-pattern database |
+| `aggregate` | Aggregate analysis results | Generates statistics and summaries |
+| `report` | Generate comprehensive report | Creates Markdown and CSV outputs |
+| `pipeline` | Run complete analysis pipeline | `--max-repos`, `--max-commits` (all-in-one) |
+| `status` | Show current analysis status | Displays progress and file statistics |
+
+### Command Details
+
+#### Fetch Repositories
+```bash
+# Fetch with custom search keywords
+greenmining fetch --max-repos 100 --min-stars 50 --languages Python --keywords "kubernetes docker"
+
+# Fetch microservices (default)
+greenmining fetch --max-repos 100 --min-stars 50 --languages Python
+```
+Options:
+- `--max-repos`: Maximum repositories to fetch (default: 100)
+- `--min-stars`: Minimum GitHub stars (default: 100)
+- `--languages`: Filter by programming languages (default: "Python,Java,Go,JavaScript,TypeScript")
+- `--keywords`: Custom search keywords (default: "microservices")
+
+#### Extract Commits
+```bash
+greenmining extract --max-commits 50
+```
+Options:
+- `--max-commits`: Maximum commits per repository (default: 50)
+
+#### Run Pipeline
+```bash
+greenmining pipeline --max-repos 50 --max-commits 100
+```
+Options:
+- `--max-repos`: Repositories to analyze
+- `--max-commits`: Commits per repository
+- Executes: fetch → extract → analyze → aggregate → report
 
 ## Output Files