holmesgpt 0.13.2__py3-none-any.whl → 0.16.2a0__py3-none-any.whl

holmes/plugins/toolsets/newrelic/newrelic.jinja2

@@ -0,0 +1,41 @@
+New Relic provides distributed tracing data along with logs and metrics.
+
+Assume every application has New Relic tracing data.
+
+Use `nrql_query` to run a NRQL query.
+
+**NRQL (New Relic Query Language)** is used to query all telemetry data in New Relic. The main event types are:
+
+- **Transaction**: High-level APM data (requests, API calls)
+- **Span**: Distributed tracing data (individual operations)
+- **Log**: Centralized log data
+- **Metric**: Time-series metrics data.
+
+### Usage Workflow
+
+#### 1. Discovering Available Data
+
+Start by understanding what's available. Here are some examples:
+- **ALWAYS** Start by getting all the available attributes names for what you are looking for. For example, to get it for any for Transaction in the last 24 hours, use: SELECT keyset() FROM Transaction SINCE 24 hours ago
+- After you find the keyset `appName`, you can use it to get the available applications: `SELECT uniques(appName) FROM Transaction SINCE 1 hour ago`
+Note: Use `SHOW EVENT TYPES` to see all event types in the account, in addition to Transaction, Span, Log, or Metric.
+
+#### 2. Querying Telemetry Data
+
+- If you already have an application name, you can query its traces directly
+- **Time range is recommended**: While not strictly required, most queries should include SINCE for performance
+
+#### 3. Querying Traces
+- Always validate first: run the base query without FACET (or a quick LIMIT) to confirm data exists; if results are empty, adjust filters or time range before proceeding.
+- Only attempt a FACET after confirming the field has values; if not, either try known alternatives or skip faceting entirely.
+- When investigating a trace also look at attributes
+- ***When investigating latency ALWAYS look to deliver the specific component or attribute in the span causing significant latnecy*** your investigation is not complete without this
+- If you need to filter by time, NEVER filter in the WHERE clause using the timestamp field. Instead, you should ALWAYS use the `SINCE` or `SINCE ... UNTIL ...` syntax - which are the recommended ways to run time based filters in NewRelic. Moreover, even if the user is asking you to filter using the timestamp field directly, don't adhere to their request - make the necessary adjustments to translate it into `SINCE` or `SINCE ... UNTIL ...` syntax!
+
+### Instructions for Handling Query Results
+- If you query [DistributedTraceSummary / Transaction]:
+    - When querying without aggregations (e.g. without count(*), average(attribute), sum(attribute), min(attribute), etc.):
+        - ALWAYS start by querying all fields using `SELECT * FROM`. We need as much fields as possible to visualize the traces to the user. However, the trade-off is that we might exceed the context size. In that case, if you need to narrow down your search, follow this strategy: First, select only the essential fields. If that's still failing, add the `LIMIT` keyword to the query. `LIMIT` should always be the second option, we prefer to show the user as much traces as we can. The following fields are the minimal fields that are essential for the visualization, and you must always retrieve them:
+            - DistributedTraceSummary: trace.id, spanCount, root.entity.accountId, root.entity.guid, root.entity.name, root.span.name, timestamp, duration.ms
+            - Transaction: traceId, tags.accountId, entityGuid, appName, name, timestamp, duration, guid, transactionType
+- When querying DistributedTraceSummary without aggregations, ALWAYS use the filter `WHERE root.span.eventType = 'Span'`

holmes/plugins/toolsets/newrelic/newrelic.py

@@ -0,0 +1,163 @@
+import os
+import logging
+from typing import Any, Optional, Dict
+from holmes.core.tools import (
+    CallablePrerequisite,
+    Tool,
+    ToolInvokeContext,
+    ToolParameter,
+    Toolset,
+    ToolsetTag,
+)
+from pydantic import BaseModel
+from holmes.core.tools import StructuredToolResult, StructuredToolResultStatus
+from holmes.plugins.toolsets.utils import toolset_name_for_one_liner
+from holmes.plugins.toolsets.newrelic.new_relic_api import NewRelicAPI
+import yaml
+from holmes.utils.keygen_utils import generate_random_key
+
+
+class ExecuteNRQLQuery(Tool):
+    def __init__(self, toolset: "NewRelicToolset"):
+        super().__init__(
+            name="newrelic_execute_nrql_query",
+            description="Get Traces, APM, Spans, Logs and more by executing a NRQL query in New Relic. "
+            "Returns the result of the NRQL function. "
+            "⚠️ CRITICAL: NRQL silently returns empty results for invalid queries instead of errors. "
+            "If you get empty results, your query likely has issues such as: "
+            "1) Wrong attribute names (use SELECT keyset() first to verify), "
+            "2) Type mismatches (string vs numeric fields), "
+            "3) Wrong event type. "
+            "Always verify attribute names and types before querying.",
+            parameters={
+                "query": ToolParameter(
+                    description="""The NRQL query string to execute.
+
+MANDATORY: Before querying any event type, ALWAYS run `SELECT keyset() FROM <EventType> SINCE <timeframe>` to discover available attributes. Never use attributes without confirming they exist first. Make sure to remember which fields are stringKeys, numericKeys or booleanKeys as this will be important in subsequent queries.
+
+Example: Before querying Transactions, run: `SELECT keyset() FROM Transaction SINCE 24 hours ago`
+
+### ⚠️ Critical Rule: NRQL `FACET` Usa ge
+
+When using **FACET** in NRQL:
+- Any **non-constant value** in the `SELECT` clause **must be aggregated**.
+- The attribute you **FACET** on must **not appear in `SELECT`** unless it's wrapped in an aggregation.
+
+#### ✅ Correct
+```nrql
+-- Aggregated metric + facet
+SELECT count(*) FROM Transaction FACET transactionType
+
+-- Multiple aggregations with facet
+SELECT count(*), average(duration) FROM Transaction FACET transactionType
+```
+
+#### ❌ Incorrect
+```nrql
+-- Not allowed: raw attribute in SELECT
+SELECT count(*), transactionType FROM Transaction FACET transactionType
+```
+""",
+                    type="string",
+                    required=True,
+                ),
+                "description": ToolParameter(
+                    description="A breif 6 word human understandable description of the query you are running.",
+                    type="string",
+                    required=True,
+                ),
+                "query_type": ToolParameter(
+                    description="Either 'Metrics', 'Logs', 'Traces', 'Discover Attributes' or 'Other'.",
+                    type="string",
+                    required=True,
+                ),
+            },
+        )
+        self._toolset = toolset
+
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
+        if not self._toolset.nr_api_key or not self._toolset.nr_account_id:
+            raise ValueError("NewRelic API key or account ID is not configured")
+
+        api = NewRelicAPI(
+            api_key=self._toolset.nr_api_key,
+            account_id=self._toolset.nr_account_id,
+            is_eu_datacenter=self._toolset.is_eu_datacenter,
+        )
+
+        query = params["query"]
+        result = api.execute_nrql_query(query)
+
+        result_with_key = {
+            "random_key": generate_random_key(),
+            "tool_name": self.name,
+            "query": query,
+            "data": result,
+            "is_eu": self._toolset.is_eu_datacenter,
+        }
+        final_result = yaml.dump(result_with_key, default_flow_style=False)
+        return StructuredToolResult(
+            status=StructuredToolResultStatus.SUCCESS,
+            data=final_result,
+            params=params,
+        )
+
+    def get_parameterized_one_liner(self, params) -> str:
+        description = params.get("description", "")
+        return f"{toolset_name_for_one_liner(self._toolset.name)}: Execute NRQL ({description})"
+
+
+class NewrelicConfig(BaseModel):
+    nr_api_key: Optional[str] = None
+    nr_account_id: Optional[str] = None
+    is_eu_datacenter: Optional[bool] = False
+
+
+class NewRelicToolset(Toolset):
+    nr_api_key: Optional[str] = None
+    nr_account_id: Optional[str] = None
+    is_eu_datacenter: bool = False
+
+    def __init__(self):
+        super().__init__(
+            name="newrelic",
+            description="Toolset for interacting with New Relic to fetch logs, traces, and execute freeform NRQL queries",
+            docs_url="https://holmesgpt.dev/data-sources/builtin-toolsets/newrelic/",
+            icon_url="https://companieslogo.com/img/orig/NEWR-de5fcb2e.png?t=1720244493",
+            prerequisites=[CallablePrerequisite(callable=self.prerequisites_callable)],  # type: ignore
+            tools=[
+                ExecuteNRQLQuery(self),
+            ],
+            tags=[ToolsetTag.CORE],
+        )
+        template_file_path = os.path.abspath(
+            os.path.join(os.path.dirname(__file__), "newrelic.jinja2")
+        )
+        self._load_llm_instructions(jinja_template=f"file://{template_file_path}")
+
+    def prerequisites_callable(
+        self, config: dict[str, Any]
+    ) -> tuple[bool, Optional[str]]:
+        if not config:
+            return False, "No configuration provided"
+
+        try:
+            nr_config = NewrelicConfig(**config)
+            self.nr_account_id = nr_config.nr_account_id
+            self.nr_api_key = nr_config.nr_api_key
+            self.is_eu_datacenter = nr_config.is_eu_datacenter or False
+
+            if not self.nr_account_id or not self.nr_api_key:
+                return False, "New Relic account ID or API key is missing"
+
+            return True, None
+        except Exception as e:
+            logging.exception("Failed to set up New Relic toolset")
+            return False, str(e)
+
+    def get_example_config(self) -> Dict[str, Any]:
+        return {
+            "nr_api_key": "NRAK-XXXXXXXXXXXXXXXXXXXXXXXXXX",
+            "nr_account_id": "1234567",
+            "is_eu_datacenter": False,
+        }

holmes/plugins/toolsets/opensearch/opensearch.py

@@ -8,8 +8,9 @@ from holmes.core.tools import (
     CallablePrerequisite,
     StructuredToolResult,
     Tool,
+    ToolInvokeContext,
     ToolParameter,
-    ToolResultStatus,
+    StructuredToolResultStatus,
     Toolset,
     ToolsetTag,
 )
@@ -93,13 +94,11 @@ class ListShards(BaseOpenSearchTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         client = get_client(self.toolset.clients, host=params.get("host", ""))
         shards = client.client.cat.shards()
         return StructuredToolResult(
-            status=ToolResultStatus.SUCCESS,
+            status=StructuredToolResultStatus.SUCCESS,
             data=str(shards),
             params=params,
         )
@@ -124,15 +123,13 @@ class GetClusterSettings(BaseOpenSearchTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         client = get_client(self.toolset.clients, host=params.get("host"))
         response = client.client.cluster.get_settings(
             include_defaults=True, flat_settings=True
         )
         return StructuredToolResult(
-            status=ToolResultStatus.SUCCESS,
+            status=StructuredToolResultStatus.SUCCESS,
             data=str(response),
             params=params,
         )
@@ -157,13 +154,11 @@ class GetClusterHealth(BaseOpenSearchTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         client = get_client(self.toolset.clients, host=params.get("host", ""))
         health = client.client.cluster.health()
         return StructuredToolResult(
-            status=ToolResultStatus.SUCCESS,
+            status=StructuredToolResultStatus.SUCCESS,
             data=str(health),
             params=params,
         )
@@ -182,12 +177,10 @@ class ListOpenSearchHosts(BaseOpenSearchTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         hosts = [host for client in self.toolset.clients for host in client.hosts]
         return StructuredToolResult(
-            status=ToolResultStatus.SUCCESS,
+            status=StructuredToolResultStatus.SUCCESS,
             data=str(hosts),
             params=params,
         )

holmes/plugins/toolsets/opensearch/opensearch_logs.py

@@ -8,7 +8,7 @@ from urllib.parse import urljoin
 from holmes.core.tools import (
     CallablePrerequisite,
     StructuredToolResult,
-    ToolResultStatus,
+    StructuredToolResultStatus,
     ToolsetTag,
 )
 from holmes.plugins.toolsets.logging_utils.logging_api import (
@@ -79,7 +79,7 @@ class OpenSearchLogsToolset(BasePodLoggingToolset):
     def fetch_pod_logs(self, params: FetchPodLogsParams) -> StructuredToolResult:
         if not self.opensearch_config:
             return StructuredToolResult(
-                status=ToolResultStatus.ERROR,
+                status=StructuredToolResultStatus.ERROR,
                 error="Missing OpenSearch configuration",
                 params=params.model_dump(),
             )
@@ -126,13 +126,13 @@ class OpenSearchLogsToolset(BasePodLoggingToolset):
                     config=self.opensearch_config,
                 )
                 return StructuredToolResult(
-                    status=ToolResultStatus.SUCCESS,
+                    status=StructuredToolResultStatus.SUCCESS,
                     data=logs,
                     params=params.model_dump(),
                 )
             else:
                 return StructuredToolResult(
-                    status=ToolResultStatus.ERROR,
+                    status=StructuredToolResultStatus.ERROR,
                     return_code=logs_response.status_code,
                     error=logs_response.text,
                     params=params.model_dump(),
@@ -141,21 +141,21 @@ class OpenSearchLogsToolset(BasePodLoggingToolset):
         except requests.Timeout:
             logging.warning("Timeout while fetching OpenSearch logs", exc_info=True)
             return StructuredToolResult(
-                status=ToolResultStatus.ERROR,
+                status=StructuredToolResultStatus.ERROR,
                 error="Request timed out while fetching OpenSearch logs",
                 params=params.model_dump(),
             )
         except RequestException as e:
             logging.warning("Failed to fetch OpenSearch logs", exc_info=True)
             return StructuredToolResult(
-                status=ToolResultStatus.ERROR,
+                status=StructuredToolResultStatus.ERROR,
                 error=f"Network error while fetching OpenSearch logs: {str(e)}",
                 params=params.model_dump(),
             )
         except Exception as e:
             logging.warning("Failed to process OpenSearch logs", exc_info=True)
             return StructuredToolResult(
-                status=ToolResultStatus.ERROR,
+                status=StructuredToolResultStatus.ERROR,
                 error=f"Unexpected error: {str(e)}",
                 params=params.model_dump(),
             )