prescale-agent 0.2.0__tar.gz

prescale_agent-0.2.0/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: prescale-agent
+Version: 0.2.0
+Summary: Metrics collection agent for Prescale - Predictive Infrastructure Intelligence Platform
+Author-email: Prescale Platform <maintainers@prescale.dev>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/pyjeebz/prescale
+Project-URL: Repository, https://github.com/pyjeebz/prescale
+Keywords: kubernetes,metrics,monitoring,prometheus,observability
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: httpx>=0.24.0
+Requires-Dist: click>=8.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: psutil>=5.9.0
+Provides-Extra: prometheus
+Requires-Dist: prometheus-client>=0.17.0; extra == "prometheus"
+Provides-Extra: datadog
+Requires-Dist: datadog-api-client>=2.0.0; extra == "datadog"
+Provides-Extra: aws
+Requires-Dist: boto3>=1.26.0; extra == "aws"
+Provides-Extra: azure
+Requires-Dist: azure-identity>=1.12.0; extra == "azure"
+Requires-Dist: azure-mgmt-monitor>=6.0.0; extra == "azure"
+Provides-Extra: gcp
+Requires-Dist: google-cloud-monitoring>=2.15.0; extra == "gcp"
+Provides-Extra: kubernetes
+Requires-Dist: kubernetes>=28.0.0; extra == "kubernetes"
+Provides-Extra: all
+Requires-Dist: prescale-agent[aws,azure,datadog,gcp,kubernetes,prometheus]; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+
+# Prescale Agent
+
+Metrics collection agent for [Prescale](https://github.com/pyjeebz/prescale) - Predictive Infrastructure Intelligence Platform.
+
+## Installation
+
+```bash
+# Base installation (system metrics + Prometheus)
+pip install prescale-platform-agent
+
+# With specific backends
+pip install prescale-platform-agent[gcp]      # + GCP Cloud Monitoring
+pip install prescale-platform-agent[aws]      # + AWS CloudWatch
+pip install prescale-platform-agent[azure]    # + Azure Monitor
+pip install prescale-platform-agent[datadog]  # + Datadog
+pip install prescale-platform-agent[all]      # All backends
+```
+
+## Quick Start
+
+```bash
+# Generate configuration file
+prescale-agent init
+
+# List available metric sources
+prescale-agent sources
+
+# Test configured sources
+prescale-agent test
+
+# Run the agent
+prescale-agent run --config prescale-agent.yaml
+```
+
+## Configuration
+
+Create a `prescale-agent.yaml` file:
+
+```yaml
+agent:
+  collection_interval: 60
+  log_level: INFO
+
+sources:
+  # System metrics (always available)
+  - type: system
+    enabled: true
+    config:
+      collect_cpu: true
+      collect_memory: true
+
+  # GCP Cloud Monitoring
+  - type: gcp-monitoring
+    enabled: true
+    config:
+      project_id: your-gcp-project
+      metrics:
+        - kubernetes.io/container/cpu/limit_utilization
+        - kubernetes.io/container/memory/limit_utilization
+
+  # Prometheus
+  - type: prometheus
+    enabled: false
+    config:
+      url: http://prometheus:9090
+      queries:
+        - name: cpu_usage
+          query: rate(container_cpu_usage_seconds_total[5m])
+
+prescale:
+  endpoint: http://prescale-inference:8080
+```
+
+## Supported Sources
+
+| Source | Description | Extra Install |
+|--------|-------------|---------------|
+| `system` | Local CPU, memory, disk via psutil | Built-in |
+| `prometheus` | Query Prometheus server | Built-in |
+| `gcp-monitoring` | GCP Cloud Monitoring | `[gcp]` |
+| `cloudwatch` | AWS CloudWatch | `[aws]` |
+| `azure-monitor` | Azure Monitor | `[azure]` |
+| `datadog` | Datadog API | `[datadog]` |
+
+## CLI Commands
+
+```bash
+prescale-agent init              # Generate config file
+prescale-agent run               # Start collecting metrics
+prescale-agent run --once        # Single collection (testing)
+prescale-agent run --deployment my-deployment  # Associate with deployment
+prescale-agent sources           # List available sources
+prescale-agent test              # Test source connections
+prescale-agent status            # Show agent status
+```
+
+## Environment Variables
+
+| Variable | Description |
+|----------|-------------|
+| `PRESCALE_CONFIG_FILE` | Path to config file (default: `./prescale-agent.yaml`) |
+| `PRESCALE_ENDPOINT` | Prescale inference endpoint |
+| `PRESCALE_API_KEY` | API key for authentication |
+
+## License
+
+Apache 2.0 - See [LICENSE](../LICENSE)

prescale_agent-0.2.0/README.md

@@ -0,0 +1,107 @@
+# Prescale Agent
+
+Metrics collection agent for [Prescale](https://github.com/pyjeebz/prescale) - Predictive Infrastructure Intelligence Platform.
+
+## Installation
+
+```bash
+# Base installation (system metrics + Prometheus)
+pip install prescale-platform-agent
+
+# With specific backends
+pip install prescale-platform-agent[gcp]      # + GCP Cloud Monitoring
+pip install prescale-platform-agent[aws]      # + AWS CloudWatch
+pip install prescale-platform-agent[azure]    # + Azure Monitor
+pip install prescale-platform-agent[datadog]  # + Datadog
+pip install prescale-platform-agent[all]      # All backends
+```
+
+## Quick Start
+
+```bash
+# Generate configuration file
+prescale-agent init
+
+# List available metric sources
+prescale-agent sources
+
+# Test configured sources
+prescale-agent test
+
+# Run the agent
+prescale-agent run --config prescale-agent.yaml
+```
+
+## Configuration
+
+Create a `prescale-agent.yaml` file:
+
+```yaml
+agent:
+  collection_interval: 60
+  log_level: INFO
+
+sources:
+  # System metrics (always available)
+  - type: system
+    enabled: true
+    config:
+      collect_cpu: true
+      collect_memory: true
+
+  # GCP Cloud Monitoring
+  - type: gcp-monitoring
+    enabled: true
+    config:
+      project_id: your-gcp-project
+      metrics:
+        - kubernetes.io/container/cpu/limit_utilization
+        - kubernetes.io/container/memory/limit_utilization
+
+  # Prometheus
+  - type: prometheus
+    enabled: false
+    config:
+      url: http://prometheus:9090
+      queries:
+        - name: cpu_usage
+          query: rate(container_cpu_usage_seconds_total[5m])
+
+prescale:
+  endpoint: http://prescale-inference:8080
+```
+
+## Supported Sources
+
+| Source | Description | Extra Install |
+|--------|-------------|---------------|
+| `system` | Local CPU, memory, disk via psutil | Built-in |
+| `prometheus` | Query Prometheus server | Built-in |
+| `gcp-monitoring` | GCP Cloud Monitoring | `[gcp]` |
+| `cloudwatch` | AWS CloudWatch | `[aws]` |
+| `azure-monitor` | Azure Monitor | `[azure]` |
+| `datadog` | Datadog API | `[datadog]` |
+
+## CLI Commands
+
+```bash
+prescale-agent init              # Generate config file
+prescale-agent run               # Start collecting metrics
+prescale-agent run --once        # Single collection (testing)
+prescale-agent run --deployment my-deployment  # Associate with deployment
+prescale-agent sources           # List available sources
+prescale-agent test              # Test source connections
+prescale-agent status            # Show agent status
+```
+
+## Environment Variables
+
+| Variable | Description |
+|----------|-------------|
+| `PRESCALE_CONFIG_FILE` | Path to config file (default: `./prescale-agent.yaml`) |
+| `PRESCALE_ENDPOINT` | Prescale inference endpoint |
+| `PRESCALE_API_KEY` | API key for authentication |
+
+## License
+
+Apache 2.0 - See [LICENSE](../LICENSE)

prescale_agent-0.2.0/pyproject.toml

@@ -0,0 +1,85 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "prescale-agent"
+version = "0.2.0"
+description = "Metrics collection agent for Prescale - Predictive Infrastructure Intelligence Platform"
+readme = "README.md"
+license = {text = "Apache-2.0"}
+requires-python = ">=3.9"
+authors = [
+    {name = "Prescale Platform", email = "maintainers@prescale.dev"}
+]
+keywords = [
+    "kubernetes",
+    "metrics",
+    "monitoring",
+    "prometheus",
+    "observability"
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Intended Audience :: System Administrators",
+    "License :: OSI Approved :: Apache Software License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: System :: Monitoring",
+]
+
+dependencies = [
+    "httpx>=0.24.0",
+    "click>=8.0.0",
+    "rich>=13.0.0",
+    "pyyaml>=6.0",
+    "psutil>=5.9.0",
+]
+
+[project.optional-dependencies]
+prometheus = [
+    "prometheus-client>=0.17.0",
+]
+datadog = [
+    "datadog-api-client>=2.0.0",
+]
+aws = [
+    "boto3>=1.26.0",
+]
+azure = [
+    "azure-identity>=1.12.0",
+    "azure-mgmt-monitor>=6.0.0",
+]
+gcp = [
+    "google-cloud-monitoring>=2.15.0",
+]
+kubernetes = [
+    "kubernetes>=28.0.0",
+]
+all = [
+    "prescale-agent[prometheus,datadog,aws,azure,gcp,kubernetes]",
+]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-asyncio>=0.21.0",
+    "ruff>=0.1.0",
+]
+
+[project.scripts]
+prescale-agent = "prescale_agent.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/pyjeebz/prescale"
+Repository = "https://github.com/pyjeebz/prescale"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-dir]
+"" = "src"

prescale_agent-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

prescale_agent-0.2.0/src/prescale_agent/__init__.py

@@ -0,0 +1,3 @@
+"""Prescale Agent - Metrics Collection for Predictive Infrastructure Intelligence."""
+
+__version__ = "0.2.0"

prescale_agent-0.2.0/src/prescale_agent/agent.py

@@ -0,0 +1,245 @@
+"""Prescale Agent - Main agent runner with unified metrics sources."""
+
+import asyncio
+import logging
+import signal
+from datetime import datetime, timezone
+from typing import Optional
+
+from .client import PrescaleClient
+from .sources import MetricsSource, MetricSample, SourceRegistry
+from .config import AgentConfig
+
+logger = logging.getLogger(__name__)
+
+
+class Agent:
+    """
+    Main Prescale metrics collection agent.
+    
+    Uses a unified source interface to collect metrics from any backend:
+    - system: Local host metrics via psutil
+    - prometheus: Prometheus server
+    - datadog: Datadog API
+    - cloudwatch: AWS CloudWatch
+    - azure_monitor: Azure Monitor
+    - gcp_monitoring: Google Cloud Monitoring
+    """
+    
+    def __init__(self, config: AgentConfig):
+        self.config = config
+        self.sources: list[MetricsSource] = []
+        self.client: Optional[PrescaleClient] = None
+        self._running = False
+        self._paused = False
+        self._interval_override: Optional[int] = None  # Server-controlled interval
+        self._metrics_buffer: list[MetricSample] = []
+        self._last_flush = datetime.now(timezone.utc)
+    
+    async def setup(self):
+        """Initialize sources and client."""
+        # Setup Prescale client
+        self.client = PrescaleClient(
+            endpoint=self.config.endpoint.url,
+            api_key=self.config.endpoint.api_key,
+            timeout=self.config.endpoint.timeout,
+            retry_attempts=self.config.endpoint.retry_attempts,
+            retry_delay=self.config.endpoint.retry_delay,
+        )
+        
+        # Setup sources from config
+        for source_config in self.config.sources:
+            if not source_config.enabled:
+                continue
+            
+            source = SourceRegistry.create(source_config)
+            if source is None:
+                logger.warning(f"Unknown source type: {source_config.type}")
+                continue
+            
+            # Initialize the source
+            try:
+                if await source.initialize():
+                    self.sources.append(source)
+                    logger.info(f"Initialized source: {source.name} (type: {source_config.type})")
+                else:
+                    logger.warning(f"Failed to initialize source: {source.name}")
+            except Exception as e:
+                logger.error(f"Error initializing source {source.name}: {e}")
+        
+        logger.info(f"Agent initialized with {len(self.sources)} sources")
+        logger.info(f"Available source types: {SourceRegistry.list_types()}")
+    
+    async def run(self):
+        """Run the agent main loop."""
+        self._running = True
+        logger.info("Starting Prescale Agent...")
+        
+        # Setup signal handlers
+        loop = asyncio.get_running_loop()
+        for sig in (signal.SIGINT, signal.SIGTERM):
+            try:
+                loop.add_signal_handler(sig, lambda: asyncio.create_task(self.stop()))
+            except NotImplementedError:
+                # Windows doesn't support add_signal_handler
+                pass
+        
+        # Check Prescale API health
+        if self.client:
+            healthy = await self.client.check_health()
+            if healthy:
+                logger.info(f"Connected to Prescale at {self.config.endpoint.url}")
+            else:
+                logger.warning(f"Could not connect to Prescale at {self.config.endpoint.url}")
+        
+        # Start source collection tasks
+        tasks = []
+        for source in self.sources:
+            if source.is_enabled():
+                task = asyncio.create_task(self._run_source(source))
+                tasks.append(task)
+        
+        # Start flush task
+        flush_task = asyncio.create_task(self._flush_loop())
+        tasks.append(flush_task)
+        
+        # Wait for all tasks
+        try:
+            await asyncio.gather(*tasks)
+        except asyncio.CancelledError:
+            logger.info("Agent tasks cancelled")
+    
+    async def stop(self):
+        """Stop the agent gracefully."""
+        logger.info("Stopping Prescale Agent...")
+        self._running = False
+        
+        # Flush remaining metrics
+        await self._flush_metrics()
+        
+        # Close sources
+        for source in self.sources:
+            try:
+                await source.close()
+            except Exception as e:
+                logger.warning(f"Error closing source {source.name}: {e}")
+        
+        # Close client
+        if self.client:
+            await self.client.close()
+    
+    async def _run_source(self, source: MetricsSource):
+        """Run a source collection loop."""
+        while self._running:
+            # Check if paused
+            if self._paused:
+                logger.debug(f"Agent paused, skipping collection from {source.name}")
+                await asyncio.sleep(5)  # Check pause state every 5s
+                continue
+            
+            try:
+                result = await source.collect()
+                
+                if result.success:
+                    # Convert source metrics to buffer format
+                    self._metrics_buffer.extend(result.metrics)
+                    logger.debug(
+                        f"Collected {len(result.metrics)} metrics from {source.name} "
+                        f"in {result.duration_ms:.1f}ms"
+                    )
+                else:
+                    logger.warning(f"Source {source.name} error: {result.error}")
+                
+            except Exception as e:
+                logger.error(f"Error running source {source.name}: {e}")
+            
+            # Use server-controlled interval if available, else source default
+            interval = self._interval_override or source.config.interval
+            await asyncio.sleep(interval)
+    
+    async def _flush_loop(self):
+        """Periodically flush metrics to Prescale."""
+        while self._running:
+            await asyncio.sleep(self.config.flush_interval)
+            await self._flush_metrics()
+    
+    async def _flush_metrics(self):
+        """Flush buffered metrics to Prescale."""
+        if not self._metrics_buffer:
+            return
+        
+        if not self.client:
+            logger.warning("No Prescale client configured, discarding metrics")
+            self._metrics_buffer.clear()
+            return
+        
+        # Get metrics to send
+        metrics_to_send = self._metrics_buffer[:self.config.batch_size]
+        self._metrics_buffer = self._metrics_buffer[self.config.batch_size:]
+        
+        # Send metrics
+        result = await self.client.send_metrics(metrics_to_send)
+        
+        if result is not None:
+            logger.info(f"Sent {len(metrics_to_send)} metrics to Prescale")
+            
+            # Apply control commands from server
+            commands = result.get("commands")
+            if commands:
+                self._apply_commands(commands)
+        else:
+            # Re-add failed metrics to buffer (at the front)
+            self._metrics_buffer = metrics_to_send + self._metrics_buffer
+            # Trim buffer if too large
+            max_buffer = self.config.batch_size * 10
+            if len(self._metrics_buffer) > max_buffer:
+                dropped = len(self._metrics_buffer) - max_buffer
+                self._metrics_buffer = self._metrics_buffer[:max_buffer]
+                logger.warning(f"Buffer full, dropped {dropped} oldest metrics")
+    
+    def _apply_commands(self, commands: dict):
+        """Apply control commands received from the server."""
+        if "paused" in commands:
+            new_paused = bool(commands["paused"])
+            if new_paused != self._paused:
+                self._paused = new_paused
+                state = "PAUSED" if new_paused else "RESUMED"
+                logger.info(f"Agent {state} by server command")
+        
+        if "collection_interval" in commands:
+            new_interval = int(commands["collection_interval"])
+            if new_interval != self._interval_override:
+                old = self._interval_override or "default"
+                self._interval_override = new_interval
+                logger.info(f"Collection interval changed: {old} -> {new_interval}s")
+    
+    async def collect_once(self) -> list[MetricSample]:
+        """Run all sources once and return metrics."""
+        all_metrics = []
+        
+        for source in self.sources:
+            if source.is_enabled():
+                result = await source.collect()
+                if result.success:
+                    all_metrics.extend(result.metrics)
+        
+        return all_metrics
+    
+    async def health_check(self) -> dict:
+        """Check health of all sources and client."""
+        status = {
+            "sources": {},
+            "client": False,
+            "metrics_buffered": len(self._metrics_buffer),
+        }
+        
+        for source in self.sources:
+            try:
+                status["sources"][source.name] = await source.health_check()
+            except Exception:
+                status["sources"][source.name] = False
+        
+        if self.client:
+            status["client"] = await self.client.check_health()
+        
+        return status