holmesgpt 0.11.5__py3-none-any.whl

holmes/plugins/toolsets/azure_sql/azure_sql_instructions.jinja2

@@ -0,0 +1,137 @@
+# Azure SQL Database Troubleshooting Guidelines
+
+## Goal
+Your primary goal when using these tools is to diagnose Azure SQL Database performance, health, and operational issues. Use these tools systematically to identify problems and provide actionable insights.
+
+## Diagnostic Workflow
+
+### 1. Start with Health and Alert Assessment
+- **Always begin** with `analyze_database_health_status` to get overall database health
+- **Immediately follow** with `get_active_alerts` to check for any active monitoring alerts
+- Look for:
+  - Active operations that might indicate ongoing maintenance or issues
+  - Resource usage warnings (high CPU, memory, storage usage >90%)
+  - Active Azure Monitor alerts indicating problems
+  - Any error messages in operations or usage retrieval
+
+### 2. Performance Analysis
+- Use `analyze_database_performance` to examine:
+  - **Automatic tuning status** - check if desired and actual states match
+  - **Performance advisors** - look for active recommendations
+  - **Performance recommendations** - pay attention to high-impact suggestions
+- Focus on recommendations with "High" impact and "Active" or "Pending" status
+
+### 3. Query Performance Deep Dive
+When performance issues are detected, use these tools to identify problematic queries:
+- `get_top_cpu_queries` - Start here for CPU performance issues
+- `get_slow_queries` - For response time problems
+- `get_top_data_io_queries` - For storage I/O bottlenecks
+- `get_top_log_io_queries` - For transaction log performance issues
+
+**Query Analysis Best Practices:**
+- Use `hours_back=24` for trend analysis, `hours_back=2` for recent issues
+- Look for queries with high execution counts AND high resource usage
+- Pay attention to query patterns that might indicate inefficient application logic
+
+### 4. Alert and Monitoring Analysis
+- **Always check alerts** with `get_active_alerts` to understand current system health
+- Look for:
+  - Active Azure Monitor alerts related to the database
+  - Alert patterns and frequencies
+  - Critical or error-level alerts requiring immediate attention
+- Use alert information to prioritize further investigation
+
+### 5. Connection Issues Investigation
+- Use `analyze_database_connections` for general connection analysis:
+  - Connection pool exhaustion
+  - Blocking sessions
+  - High concurrent user loads
+- Use `analyze_connection_failures` for connection failure patterns:
+  - Authentication failures
+  - Firewall blocks
+  - Connection timeout issues
+  - Failed connection trends and analysis
+- Look for blocked connections and identify blocking sessions
+
+### 6. Storage Problems
+- Use `analyze_database_storage` for:
+  - Storage capacity planning
+  - Growth rate analysis
+  - File-level storage issues
+  - TempDB usage problems
+
+## Key Indicators to Flag
+
+### Critical Issues (🚨)
+- **Active critical alerts** (Sev0/Critical level) in monitoring
+- **High connection failure rates** (>5% failed connections)
+- **Firewall blocks** indicating unauthorized access attempts
+- **Blocked connections** with blocking_session_id > 0
+- **Storage usage > 90%** in any file
+- **Active performance recommendations** with High impact
+- **Resource alarms** (memory, disk space)
+- **Auto-tuning mismatches** (desired ≠ actual state)
+
+### Warning Signs (⚠️)
+- **Warning-level alerts** (Sev1/Sev2) that may indicate developing issues
+- **Connection failure trends** showing increasing failures over time
+- **Storage usage 70-90%**
+- **High daily growth rates** (>100MB/day)
+- **Many concurrent connections** without proper pooling
+- **Long-running queries** (>10 seconds average duration)
+
+## Recommendations Framework
+
+### Performance Issues
+1. **Index recommendations**: Apply suggested indexes from performance advisors
+2. **Query optimization**: Rewrite queries identified in top CPU/IO reports
+3. **Auto-tuning**: Enable automatic tuning options if they're disabled
+
+### Storage Issues
+1. **Immediate**: Address files >90% full
+2. **Planning**: Monitor growth trends and plan capacity increases
+3. **Optimization**: Review top space-consuming tables for archiving opportunities
+
+### Connection Issues
+1. **Connection failures**: Investigate and resolve authentication/firewall issues
+2. **Connection pooling**: Implement proper connection pooling in applications
+3. **Blocking resolution**: Identify and optimize long-running transactions
+4. **Security**: Review firewall rules if blocked connections are detected
+5. **Monitoring**: Set up alerts for connection count and failure rate thresholds
+
+### Alert Management
+1. **Critical alerts**: Address immediately - these indicate active problems
+2. **Alert patterns**: Look for recurring alerts that indicate systemic issues
+3. **Proactive monitoring**: Set up additional alerts based on discovered patterns
+
+## Analysis Structure
+Present findings in this order:
+1. **Executive Summary** - High-level status and critical issues
+2. **Detailed Findings** - Tool-by-tool analysis with specific metrics
+3. **Prioritized Recommendations** - Critical first, then important
+4. **Monitoring Suggestions** - Ongoing observability improvements
+
+## Available Tools Reference
+
+### Health and Monitoring Tools
+- `analyze_database_health_status` - Overall database health and resource usage
+- `get_active_alerts` - Active Azure Monitor alerts and alert patterns
+- `analyze_database_performance` - Performance advisors and automatic tuning status
+
+### Query Performance Tools
+- `get_top_cpu_queries` - Highest CPU-consuming queries from Query Store
+- `get_slow_queries` - Longest-running queries causing response time issues
+- `get_top_data_io_queries` - Queries with highest data I/O (logical reads/writes)
+- `get_top_log_io_queries` - Queries consuming most transaction log I/O
+
+### Connection and Infrastructure Tools
+- `analyze_database_connections` - Connection pooling, blocking sessions, concurrent users
+- `analyze_connection_failures` - Connection failure patterns, authentication issues, firewall blocks
+- `analyze_database_storage` - Storage capacity, growth trends, file-level analysis
+
+## Important Notes
+- **Time ranges**: Adjust `hours_back` based on issue timeline (2h for immediate, 24h+ for trends)
+- **Multiple tools**: Use complementary tools together (e.g., health + alerts + performance reports)
+- **Resource context**: Always consider database tier and limits when interpreting metrics
+- **Correlation**: Look for patterns across different metrics (CPU high + IO high = query optimization needed)
+- **Security focus**: Use connection failure analysis to detect potential security issues

holmes/plugins/toolsets/azure_sql/azure_sql_toolset.py

@@ -0,0 +1,183 @@
+import os
+import logging
+from typing import Any, Dict, Tuple, Union
+
+from azure.identity import DefaultAzureCredential, ClientSecretCredential
+
+from holmes.core.tools import (
+    CallablePrerequisite,
+    ToolsetTag,
+)
+from holmes.plugins.toolsets.azure_sql.apis.azure_sql_api import AzureSQLAPIClient
+from holmes.plugins.toolsets.consts import TOOLSET_CONFIG_MISSING_ERROR
+from holmes.plugins.toolsets.azure_sql.azure_base_toolset import (
+    BaseAzureSQLToolset,
+    AzureSQLConfig,
+    AzureSQLDatabaseConfig,
+)
+
+# Import all tool classes
+from holmes.plugins.toolsets.azure_sql.tools.analyze_database_health_status import (
+    AnalyzeDatabaseHealthStatus,
+)
+from holmes.plugins.toolsets.azure_sql.tools.analyze_database_performance import (
+    AnalyzeDatabasePerformance,
+)
+from holmes.plugins.toolsets.azure_sql.tools.analyze_database_connections import (
+    AnalyzeDatabaseConnections,
+)
+from holmes.plugins.toolsets.azure_sql.tools.analyze_database_storage import (
+    AnalyzeDatabaseStorage,
+)
+from holmes.plugins.toolsets.azure_sql.tools.get_top_cpu_queries import GetTopCPUQueries
+from holmes.plugins.toolsets.azure_sql.tools.get_slow_queries import GetSlowQueries
+from holmes.plugins.toolsets.azure_sql.tools.get_top_data_io_queries import (
+    GetTopDataIOQueries,
+)
+from holmes.plugins.toolsets.azure_sql.tools.get_top_log_io_queries import (
+    GetTopLogIOQueries,
+)
+from holmes.plugins.toolsets.azure_sql.tools.get_active_alerts import GetActiveAlerts
+from holmes.plugins.toolsets.azure_sql.tools.analyze_connection_failures import (
+    AnalyzeConnectionFailures,
+)
+
+
+class AzureSQLToolset(BaseAzureSQLToolset):
+    def __init__(self):
+        # Reduce Azure SDK HTTP logging verbosity
+        logging.getLogger("azure.core.pipeline.policies.http_logging_policy").setLevel(
+            logging.WARNING
+        )
+        logging.getLogger("azure.identity").setLevel(logging.WARNING)
+        logging.getLogger("azure.mgmt").setLevel(logging.WARNING)
+        logging.getLogger("azure.monitor").setLevel(logging.WARNING)
+
+        super().__init__(
+            name="azure/sql",
+            description="Analyzes Azure SQL Database performance, health, and operational issues using Azure REST APIs and Query Store data",
+            prerequisites=[CallablePrerequisite(callable=self.prerequisites_callable)],
+            docs_url="https://kagi.com/proxy/png-clipart-microsoft-sql-server-microsoft-azure-sql-database-microsoft-text-logo-thumbnail.png?c=4Sg1bvcUGOrhnDzXgoBBa0G0j27ykgskX4a8cLrZp_quzqlpVGVG02OqQtezTxy7lB6ydmTKgbVAn_F7BxofxK6LKKUZSpjJ1huIAsXPVaXyakO4sWXFiX0Wz_8WjkA0AIlO_oFfW31AKaj5RcvGcr3siy0n5kW-GcqdpeBWsmm_huxUT6RycULFCDFBwuUzHvVl5TW3cYqlMxT8ecPZfg%3D%3D",
+            icon_url="https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/Azure_SQL_Database_logo.svg/1200px-Azure_SQL_Database_logo.svg.png",
+            tags=[ToolsetTag.CORE],
+            experimental=True,
+            tools=[
+                AnalyzeDatabaseHealthStatus(self),
+                AnalyzeDatabasePerformance(self),
+                AnalyzeDatabaseConnections(self),
+                AnalyzeDatabaseStorage(self),
+                GetTopCPUQueries(self),
+                GetSlowQueries(self),
+                GetTopDataIOQueries(self),
+                GetTopLogIOQueries(self),
+                GetActiveAlerts(self),
+                AnalyzeConnectionFailures(self),
+            ],
+        )
+        self._reload_llm_instructions()
+
+    def prerequisites_callable(self, config: Dict[str, Any]) -> Tuple[bool, str]:
+        if not config:
+            return False, TOOLSET_CONFIG_MISSING_ERROR
+
+        errors = []
+        try:
+            azure_sql_config = AzureSQLConfig(**config)
+
+            # Set up Azure credentials
+            try:
+                credential: Union[ClientSecretCredential, DefaultAzureCredential]
+                if (
+                    azure_sql_config.tenant_id
+                    and azure_sql_config.client_id
+                    and azure_sql_config.client_secret
+                ):
+                    logging.info(
+                        "Using ClientSecretCredential for Azure authentication"
+                    )
+                    credential = ClientSecretCredential(
+                        tenant_id=azure_sql_config.tenant_id,
+                        client_id=azure_sql_config.client_id,
+                        client_secret=azure_sql_config.client_secret,
+                    )
+                else:
+                    logging.info(
+                        "Using DefaultAzureCredential for Azure authentication"
+                    )
+                    credential = DefaultAzureCredential()
+
+                # Test the credential by attempting to get tokens for both required scopes
+                mgmt_token = credential.get_token(
+                    "https://management.azure.com/.default"
+                )
+                if not mgmt_token.token:
+                    raise Exception("Failed to obtain Azure management token")
+
+                # Test SQL database token as well
+                sql_token = credential.get_token(
+                    "https://database.windows.net/.default"
+                )
+                if not sql_token.token:
+                    raise Exception("Failed to obtain Azure SQL database token")
+
+            except Exception as e:
+                message = f"Failed to set up Azure authentication: {str(e)}"
+                logging.error(message)
+                errors.append(message)
+                return False, message
+
+            # Store single database configuration and create API client
+            self._database_config = azure_sql_config.database
+            self._api_client = AzureSQLAPIClient(
+                credential, azure_sql_config.database.subscription_id
+            )
+            logging.info(
+                f"Configured Azure SQL database: {azure_sql_config.database.server_name}/{azure_sql_config.database.database_name}"
+            )
+
+            # Validate each tool's configuration requirements
+            # tool_validation_errors = []
+            # for tool in self.tools:
+            #     if isinstance(tool, BaseAzureSQLTool):
+            #         azure_tool = cast(BaseAzureSQLTool, tool)
+            #         try:
+            #             is_valid, error_msg = azure_tool.validate_config(
+            #                 self._api_client, self._database_config
+            #             )
+            #             if not is_valid:
+            #                 tool_validation_errors.append(
+            #                     f"Tool '{azure_tool.name}' validation failed: {error_msg}"
+            #                 )
+            #         except Exception as e:
+            #             tool_validation_errors.append(
+            #                 f"Tool '{azure_tool.name}' validation error: {str(e)}"
+            #             )
+
+            # Combine all errors
+            all_errors = errors  # + tool_validation_errors
+
+            return len(all_errors) == 0, "\n".join(all_errors)
+        except Exception as e:
+            logging.exception("Failed to set up Azure SQL toolset")
+            return False, str(e)
+
+    def get_example_config(self) -> Dict[str, Any]:
+        example_config = AzureSQLConfig(
+            tenant_id="{{ env.AZURE_TENANT_ID }}",
+            client_id="{{ env.AZURE_CLIENT_ID }}",
+            client_secret="{{ env.AZURE_CLIENT_SECRET }}",
+            database=AzureSQLDatabaseConfig(
+                subscription_id="12345678-1234-1234-1234-123456789012",
+                resource_group="my-resource-group",
+                server_name="myserver",
+                database_name="mydatabase",
+            ),
+        )
+        return example_config.model_dump()
+
+    def _reload_llm_instructions(self):
+        """Load Azure SQL specific troubleshooting instructions."""
+        template_file_path = os.path.abspath(
+            os.path.join(os.path.dirname(__file__), "azure_sql_instructions.jinja2")
+        )
+        self._load_llm_instructions(jinja_template=f"file://{template_file_path}")

holmes/plugins/toolsets/azure_sql/install.md

@@ -0,0 +1,66 @@
+# The dockerfile contains the odbc driver.
+
+Supported authentication include Azure AD Workload Identity as well as Service Principal.
+
+## Configuration
+
+### Azure AD Workload Identity
+
+```yaml
+holmes:
+  toolsets:
+    azure/sql:
+      enabled: True
+      config:
+        database:
+          subscription_id: "2f90e3c5-xxxx-xxxx-xxxx-9783a7a5dea7"
+          resource_group: "<...azure resource group...>"
+          server_name: "<azure sql server name>"
+          database_name: "<azure sql database name>"
+```
+
+With AD workload identity,
+
+### Service Principal
+
+```yaml
+holmes:
+  toolsets:
+    azure/sql:
+      enabled: True
+      config:
+        tenant_id: e5317b2d-xxxx-xxxx-xxxx-875841d00831
+        client_id: 73bacf7a-xxxx-xxxx-xxxx-110360f79d16
+        client_secret: "xxxx"
+        database:
+          subscription_id: "2f90e3c5-xxxx-xxxx-xxxx-9783a7a5dea7"
+          resource_group: "<...azure resource group...>"
+          server_name: "<azure sql server name>"
+          database_name: "<azure sql database name>"
+```
+
+
+### Roles / Access controls
+
+The service principal requires these roles:
+
+#### 1. Azure
+
+```
+Azure Level (RBAC):
+├── Monitoring Reader (subscription)
+├── SQL DB Contributor (resource group)
+```
+
+#### 2. SQL
+
+```
+Database Level (SQL permissions):
+├── CREATE USER [holmes-service-principal] FROM EXTERNAL PROVIDER
+├── GRANT VIEW SERVER STATE TO [holmes-service-principal]
+└── ALTER ROLE db_datareader ADD MEMBER [holmes-service-principal]
+```
+
+#### 3. Query Store
+
+In addition, Query Store should be enabled on target databases

holmes/plugins/toolsets/azure_sql/tools/__init__.py

@@ -0,0 +1 @@
+# Azure SQL Tool Classes