holmesgpt 0.14.1a0__py3-none-any.whl → 0.14.3a0__py3-none-any.whl

holmes/plugins/toolsets/prometheus/prometheus_instructions.jinja2

@@ -1,6 +1,27 @@
 
 # Prometheus/PromQL queries
-* ALWAYS call list_prometheus_rules to get the alert definition
+
+## Efficient Metric Discovery (when needed)
+* When you need to discover metrics, use `get_metric_names` with filters - it's the fastest method
+* Combine multiple patterns with regex OR (|) to reduce API calls:
+  - `{__name__=~"node_cpu.*|node_memory.*|node_disk.*"}` - get all node resource metrics in one call
+  - `{__name__=~"container.*|pod.*|kube.*"}` - get all Kubernetes-related metrics
+  - `{namespace=~"example1|example2|example3"}` - metrics from multiple namespaces
+* Use `get_metric_metadata` after discovering names to get types/descriptions if needed
+* Use `get_label_values` to discover pods, namespaces, jobs: e.g., get_label_values(label="pod")
+* Only use `get_series` when you need full label sets (slower than other methods)
+
+## Retrying queries that return too much data
+* When a Prometheus query returns too much data (e.g., truncation error), you MUST retry with a more specific query or less data points or topk/bottomk
+* NEVER EVER EVER answer a question based on Prometheus data that was truncated as you might be missing important information and give the totally wrong answer
+* Prefer telling the user you can't answer the question because of too much data rather than answering based on incomplete data
+* You are also able to show graphs to the user (using the promql embed functionality mentioned below) so you can show users graphs and THEY can interpret the data themselves, even if you can't answer.
+* Do NOT hestitate to try alternative queries and try to reduce the amount of data returned until you get a successful query
+* Be extremely, extremely cautious when answering based on get_label_values because the existence of a label value says NOTHING about the metric value itself (is it high, low, or perhaps the label exists in Prometheus but its an older series not present right now)
+* DO NOT give answers about metrics based on what 'is typically the case' or 'common knowledge' - if you can't see the actual metric value, you MUST NEVER EVER answer about it - just tell the user your limitations due to the size of the data
+
+## Alert Investigation & Query Execution
+* When investigating a Prometheus alert, ALWAYS call list_prometheus_rules to get the alert definition
 * Use Prometheus to query metrics from the alert promql
 * Use prometheus to execute promql queries with the tools `execute_prometheus_instant_query` and `execute_prometheus_range_query`
 * To create queries, use 'start_timestamp' and 'end_timestamp' as graphs start and end times
@@ -16,7 +37,7 @@
 ** Avoid global averages like `sum(rate(<metric>_sum)) / sum(rate(<metric>_count))` because it hides data and is not generally informative
 * Timestamps MUST be in string date format. For example: '2025-03-15 10:10:08.610862+00:00'
 * Post processing will parse your response, re-run the query from the tool output and create a chart visible to the user
-* Only generate and execute a prometheus query after checking what metrics are available with the `list_available_metrics` tool
+* When unsure about available metrics, use `get_metric_names` with appropriate filters (combine multiple patterns with | for efficiency). Then use `get_metric_metadata` if you need descriptions/types
 * Check that any node, service, pod, container, app, namespace, etc. mentioned in the query exist in the kubernetes cluster before making a query. Use any appropriate kubectl tool(s) for this
 * The toolcall will return no data to you. That is expected. You MUST however ensure that the query is successful.
 
@@ -25,24 +46,19 @@
 * ALWAYS use `topk()` or `bottomk()` to limit the number of series returned
 * Standard pattern for high-cardinality queries:
   - Use `topk(5, <your_query>)` to get the top 5 series
-  - Example: `topk(5, rate(container_cpu_usage_seconds_total{namespace="default"}[5m]))`
+  - Example: `topk(5, rate(container_cpu_usage_seconds_total{namespace="example"}[5m]))`
   - This prevents context overflow and focuses on the most relevant data
 * To also capture the aggregate of remaining series as "other":
   ```
-  topk(5, rate(container_cpu_usage_seconds_total{namespace="default"}[5m]))
-  or
-  label_replace(
-    (sum(rate(container_cpu_usage_seconds_total{namespace="default"}[5m])) - sum(topk(5, rate(container_cpu_usage_seconds_total{namespace="default"}[5m])))),
-    "pod", "other", "", ""
-  )
+  topk(5, rate(container_cpu_usage_seconds_total{namespace="example"}[5m])) or label_replace((sum(rate(container_cpu_usage_seconds_total{namespace="example"}[5m])) - sum(topk(5, rate(container_cpu_usage_seconds_total{namespace="example"}[5m])))), "pod", "other", "", "")
   ```
 * Common high-cardinality scenarios requiring topk():
   - Pod-level metrics in namespaces with many pods
   - Container-level CPU/memory metrics
   - HTTP metrics with many endpoints or status codes
   - Any query returning more than 10 time series
-* For initial exploration, use instant queries with `count()` to check cardinality:
-  - Example: `count(count by (pod) (container_cpu_usage_seconds_total{namespace="default"}))`
+* For initial exploration, you may use instant queries with `count()` to check cardinality:
+  - Example: `count(count by (pod) (container_cpu_usage_seconds_total{namespace="example"}))`
   - If count > 10, use topk() in your range query
 * When doing queries, always extend the time range, to 15 min before and after the alert start time
 * ALWAYS embed the execution results into your answer

holmes/plugins/toolsets/rabbitmq/toolset_rabbitmq.py

@@ -7,6 +7,7 @@ from holmes.core.tools import (
     CallablePrerequisite,
     StructuredToolResult,
     Tool,
+    ToolInvokeContext,
     ToolParameter,
     StructuredToolResultStatus,
     Toolset,
@@ -63,9 +64,7 @@ class ListConfiguredClusters(BaseRabbitMQTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         if not self.toolset.config:
             raise ValueError("RabbitMQ is not configured.")
 
@@ -103,9 +102,7 @@ class GetRabbitMQClusterStatus(BaseRabbitMQTool):
             toolset=toolset,
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         try:
             # Fetch node details which include partition info
             cluster_config = self._get_cluster_config(

holmes/plugins/toolsets/robusta/robusta.py

@@ -7,6 +7,7 @@ from holmes.core.supabase_dal import SupabaseDal
 from holmes.core.tools import (
     StaticPrerequisite,
     Tool,
+    ToolInvokeContext,
     ToolParameter,
     Toolset,
     ToolsetTag,
@@ -45,9 +46,7 @@ class FetchRobustaFinding(Tool):
             logging.error(error)
             return {"error": error}
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         finding_id = params[PARAM_FINDING_ID]
         try:
             finding = self._fetch_finding(finding_id)
@@ -115,9 +114,7 @@ class FetchResourceRecommendation(Tool):
             )
         return None
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         try:
             recommendations = self._resource_recommendation(params)
             if recommendations:
@@ -175,9 +172,7 @@ class FetchConfigurationChanges(Tool):
             )
         return None
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         try:
             changes = self._fetch_change_history(params)
             if changes:

holmes/plugins/toolsets/runbook/runbook_fetcher.py

@@ -1,17 +1,23 @@
 import logging
+import os
 import textwrap
 from typing import Any, Dict, List, Optional
 
 from holmes.core.tools import (
     StructuredToolResult,
     Tool,
+    ToolInvokeContext,
     ToolParameter,
     StructuredToolResultStatus,
     Toolset,
     ToolsetTag,
 )
 
-from holmes.plugins.runbooks import get_runbook_by_path, DEFAULT_RUNBOOK_SEARCH_PATH
+from holmes.plugins.runbooks import (
+    get_runbook_by_path,
+    load_runbook_catalog,
+    DEFAULT_RUNBOOK_SEARCH_PATH,
+)
 from holmes.plugins.toolsets.utils import toolset_name_for_one_liner
 
 
@@ -19,30 +25,104 @@ from holmes.plugins.toolsets.utils import toolset_name_for_one_liner
 # runbooks from external sources as well.
 class RunbookFetcher(Tool):
     toolset: "RunbookToolset"
+    available_runbooks: List[str] = []
+    additional_search_paths: Optional[List[str]] = None
+
+    def __init__(
+        self,
+        toolset: "RunbookToolset",
+        additional_search_paths: Optional[List[str]] = None,
+    ):
+        catalog = load_runbook_catalog()
+        available_runbooks = []
+        if catalog:
+            available_runbooks = [entry.link for entry in catalog.catalog]
+
+        # If additional search paths are configured (e.g., for testing), also scan those for .md files
+        if additional_search_paths:
+            for search_path in additional_search_paths:
+                if not os.path.isdir(search_path):
+                    continue
+
+                for file in os.listdir(search_path):
+                    if file.endswith(".md") and file not in available_runbooks:
+                        available_runbooks.append(file)
+
+        # Build description with available runbooks
+        runbook_list = ", ".join([f'"{rb}"' for rb in available_runbooks])
 
-    def __init__(self, toolset: "RunbookToolset"):
         super().__init__(
             name="fetch_runbook",
             description="Get runbook content by runbook link. Use this to get troubleshooting steps for incidents",
             parameters={
-                # use link as a more generic term for runbook path, considering we may have external links in the future
                 "link": ToolParameter(
-                    description="The link to the runbook",
+                    description=f"The link to the runbook (non-empty string required). Must be one of: {runbook_list}",
                     type="string",
                     required=True,
                 ),
             },
-            toolset=toolset,  # type: ignore
+            toolset=toolset,  # type: ignore[call-arg]
+            available_runbooks=available_runbooks,  # type: ignore[call-arg]
+            additional_search_paths=additional_search_paths,  # type: ignore[call-arg]
         )
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
-        link: str = params["link"]
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
+        link: str = params.get("link", "")
+        # Validate link is not empty
+        if not link or not link.strip():
+            err_msg = (
+                "Runbook link cannot be empty. Please provide a valid runbook path."
+            )
+            logging.error(err_msg)
+            return StructuredToolResult(
+                status=StructuredToolResultStatus.ERROR,
+                error=err_msg,
+                params=params,
+            )
 
+        # Build list of allowed search paths
         search_paths = [DEFAULT_RUNBOOK_SEARCH_PATH]
-        if self.toolset.config and "additional_search_paths" in self.toolset.config:
-            search_paths.extend(self.toolset.config["additional_search_paths"])
+        if self.additional_search_paths:
+            search_paths.extend(self.additional_search_paths)
+
+        # Validate link is in the available runbooks list OR is a valid path within allowed directories
+        if link not in self.available_runbooks:
+            # For links not in the catalog, perform strict path validation
+            if not link.endswith(".md"):
+                err_msg = f"Invalid runbook link '{link}'. Must end with .md extension."
+                logging.error(err_msg)
+                return StructuredToolResult(
+                    status=StructuredToolResultStatus.ERROR,
+                    error=err_msg,
+                    params=params,
+                )
+
+            # Check if the link would resolve to a valid path within allowed directories
+            # This prevents path traversal attacks like ../../secret.md
+            is_valid_path = False
+            for search_path in search_paths:
+                candidate_path = os.path.join(search_path, link)
+                # Canonicalize both paths to resolve any .. or . components
+                real_search_path = os.path.realpath(search_path)
+                real_candidate_path = os.path.realpath(candidate_path)
+
+                # Check if the resolved path is within the allowed directory
+                if (
+                    real_candidate_path.startswith(real_search_path + os.sep)
+                    or real_candidate_path == real_search_path
+                ):
+                    if os.path.isfile(real_candidate_path):
+                        is_valid_path = True
+                        break
+
+            if not is_valid_path:
+                err_msg = f"Invalid runbook link '{link}'. Must be one of: {', '.join(self.available_runbooks) if self.available_runbooks else 'No runbooks available'}"
+                logging.error(err_msg)
+                return StructuredToolResult(
+                    status=StructuredToolResultStatus.ERROR,
+                    error=err_msg,
+                    params=params,
+                )
 
         runbook_path = get_runbook_by_path(link, search_paths)
 
@@ -116,7 +196,7 @@ class RunbookFetcher(Tool):
 
 class RunbookToolset(Toolset):
     def __init__(self, additional_search_paths: Optional[List[str]] = None):
-        # Store additional search paths in config
+        # Store additional search paths in config for RunbookFetcher to access
         config = {}
         if additional_search_paths:
             config["additional_search_paths"] = additional_search_paths
@@ -126,7 +206,7 @@ class RunbookToolset(Toolset):
             description="Fetch runbooks",
             icon_url="https://platform.robusta.dev/demos/runbook.svg",
             tools=[
-                RunbookFetcher(self),
+                RunbookFetcher(self, additional_search_paths),
             ],
             docs_url="https://holmesgpt.dev/data-sources/",
             tags=[

holmes/plugins/toolsets/servicenow/servicenow.py

@@ -5,6 +5,7 @@ from typing import Any, Dict, Tuple, List
 from holmes.core.tools import (
     CallablePrerequisite,
     Tool,
+    ToolInvokeContext,
     ToolParameter,
     Toolset,
     ToolsetTag,
@@ -56,7 +57,7 @@ class ServiceNowToolset(Toolset):
             self.config: Dict = ServiceNowConfig(**config).model_dump()
             self._session.headers.update(
                 {
-                    "x-sn-apikey": self.config.get("api_key"),
+                    "x-sn-apikey": self.config.get("api_key"),  # type: ignore
                 }
             )
 
@@ -115,9 +116,7 @@ class ReturnChangesInTimerange(ServiceNowBaseTool):
         start = params.get("start", "last hour")
         return f"{toolset_name_for_one_liner(self.toolset.name)}: Get Change Requests ({start})"
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         parsed_params = {}
         try:
             (start, _) = process_timestamps_to_rfc3339(
@@ -160,9 +159,7 @@ class ReturnChange(ServiceNowBaseTool):
         sys_id = params.get("sys_id", "")
         return f"{toolset_name_for_one_liner(self.toolset.name)}: Get Change Details ({sys_id})"
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         try:
             url = "https://{instance}.service-now.com/api/now/v2/table/change_request/{sys_id}".format(
                 instance=self.toolset.config.get("instance"),
@@ -194,9 +191,7 @@ class ReturnChangesWithKeyword(ServiceNowBaseTool):
         keyword = params.get("keyword", "")
         return f"{toolset_name_for_one_liner(self.toolset.name)}: Search Changes ({keyword})"
 
-    def _invoke(
-        self, params: dict, user_approved: bool = False
-    ) -> StructuredToolResult:
+    def _invoke(self, params: dict, context: ToolInvokeContext) -> StructuredToolResult:
         parsed_params = {}
         try:
             url = f"https://{self.toolset.config.get('instance')}.service-now.com/api/now/v2/table/change_request"

holmes/utils/sentry_helper.py

@@ -1,5 +1,5 @@
 import sentry_sdk
-from holmes.core.tools_utils.data_types import ToolCallResult, TruncationMetadata
+from holmes.core.models import ToolCallResult, TruncationMetadata
 
 
 def capture_tool_truncations(truncations: list[TruncationMetadata]):

holmes/utils/stream.py

@@ -14,6 +14,7 @@ class StreamEvents(str, Enum):
     TOOL_RESULT = "tool_calling_result"
     ERROR = "error"
     AI_MESSAGE = "ai_message"
+    APPROVAL_REQUIRED = "approval_required"
 
 
 class StreamMessage(BaseModel):
@@ -78,14 +79,28 @@ def stream_chat_formatter(
     try:
         for message in call_stream:
             if message.event == StreamEvents.ANSWER_END:
+                response_data = {
+                    "analysis": message.data.get("content"),
+                    "conversation_history": message.data.get("messages"),
+                    "follow_up_actions": followups,
+                    "metadata": message.data.get("metadata") or {},
+                }
+
+                yield create_sse_message(StreamEvents.ANSWER_END.value, response_data)
+            elif message.event == StreamEvents.APPROVAL_REQUIRED:
+                response_data = {
+                    "analysis": message.data.get("content"),
+                    "conversation_history": message.data.get("messages"),
+                    "follow_up_actions": followups,
+                }
+
+                response_data["requires_approval"] = True
+                response_data["pending_approvals"] = message.data.get(
+                    "pending_approvals", []
+                )
+
                 yield create_sse_message(
-                    StreamEvents.ANSWER_END.value,
-                    {
-                        "analysis": message.data.get("content"),
-                        "conversation_history": message.data.get("messages"),
-                        "follow_up_actions": followups,
-                        "metadata": message.data.get("metadata") or {},
-                    },
+                    StreamEvents.APPROVAL_REQUIRED.value, response_data
                 )
             else:
                 yield create_sse_message(message.event.value, message.data)

holmes/version.py

@@ -57,11 +57,41 @@ def get_version() -> str:
         return __version__
 
     # we are running from an unreleased dev version
+    archival_file_path = os.path.join(this_path, ".git_archival.json")
+    if os.path.exists(archival_file_path):
+        try:
+            with open(archival_file_path, "r") as f:
+                archival_data = json.load(f)
+                refs = archival_data.get("refs", "")
+                hash_short = archival_data.get("hash-short", "")
+
+                # Check if Git substitution didn't happen (placeholders are still present)
+                if "$Format:" in refs or "$Format:" in hash_short:
+                    # Placeholders not substituted, skip to next method
+                    pass
+                else:
+                    # Valid archival data found
+                    return f"dev-{refs}-{hash_short}"
+        except Exception:
+            pass
+
+    # Now try git commands for development environments
     try:
+        env = os.environ.copy()
+        # Set ceiling to prevent walking up beyond the project root
+        # We want to allow access to holmes/.git but not beyond holmes
+        project_root = os.path.dirname(this_path)  # holmes
+        env["GIT_CEILING_DIRECTORIES"] = os.path.dirname(
+            project_root
+        )  # holmes's parent
+
         # Get the latest git tag
         tag = (
             subprocess.check_output(
-                ["git", "describe", "--tags"], stderr=subprocess.STDOUT, cwd=this_path
+                ["git", "describe", "--tags"],
+                stderr=subprocess.STDOUT,
+                cwd=this_path,
+                env=env,
             )
             .decode()
             .strip()
@@ -73,6 +103,7 @@ def get_version() -> str:
                 ["git", "rev-parse", "--abbrev-ref", "HEAD"],
                 stderr=subprocess.STDOUT,
                 cwd=this_path,
+                env=env,
             )
             .decode()
             .strip()
@@ -84,6 +115,7 @@ def get_version() -> str:
                 ["git", "status", "--porcelain"],
                 stderr=subprocess.STDOUT,
                 cwd=this_path,
+                env=env,
             )
             .decode()
             .strip()
@@ -95,19 +127,7 @@ def get_version() -> str:
     except Exception:
         pass
 
-    # we are running without git history, but we still might have git archival data (e.g. if we were pip installed)
-    archival_file_path = os.path.join(this_path, ".git_archival.json")
-    if os.path.exists(archival_file_path):
-        try:
-            with open(archival_file_path, "r") as f:
-                archival_data = json.load(f)
-                return f"dev-{archival_data['refs']}-{archival_data['hash-short']}"
-        except Exception:
-            pass
-
-        return "dev-version"
-
-    return "unknown-version"
+    return "dev-unknown"
 
 
 @cache

{holmesgpt-0.14.1a0.dist-info → holmesgpt-0.14.3a0.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: holmesgpt
-Version: 0.14.1a0
+Version: 0.14.3a0
 Summary: 
 Author: Natan Yellin
 Author-email: natan@robusta.dev
@@ -23,10 +23,11 @@ Requires-Dist: certifi (>=2024.7.4,<2025.0.0)
 Requires-Dist: colorlog (>=6.8.2,<7.0.0)
 Requires-Dist: confluent-kafka (>=2.6.1,<3.0.0)
 Requires-Dist: fastapi (>=0.116,<0.117)
+Requires-Dist: httpx[socks] (<0.28)
 Requires-Dist: humanize (>=4.9.0,<5.0.0)
 Requires-Dist: jinja2 (>=3.1.2,<4.0.0)
 Requires-Dist: kubernetes (>=32.0.1,<33.0.0)
-Requires-Dist: litellm (>=1.75.4,<2.0.0)
+Requires-Dist: litellm (==1.77.1)
 Requires-Dist: markdown (>=3.6,<4.0)
 Requires-Dist: markdownify (>=1.1.0,<2.0.0)
 Requires-Dist: mcp (==v1.12.2)
@@ -245,14 +246,11 @@ Distributed under the MIT License. See [LICENSE.txt](https://github.com/robusta-
 
 ## Community
 
-Join our community meetings to discuss the HolmesGPT roadmap and share feedback:
+Join our community to discuss the HolmesGPT roadmap and share feedback:
 
-📅 **First Community Meeting:** Thursday, August 21, 2025
-- **Time:** 8:00-9:00 AM PT / 11:00 AM-12:00 PM ET / 8:30-9:30 PM IST
-- **Where:** [Google Meet](https://meet.google.com/jxc-ujyf-xwy)
-- **Agenda:** [Roadmap discussion](https://github.com/orgs/robusta-dev/projects/2), community feedback, and Q&A
-
-[📝 Meeting Notes](https://docs.google.com/document/d/1sIHCcTivyzrF5XNvos7ZT_UcxEOqgwfawsTbb9wMJe4/edit?tab=t.0) | [📋 Full Details](https://holmesgpt.dev/community/)
+📹 **First Community Meetup Recording:** [Watch on YouTube](https://youtu.be/slQRc6nlFQU)
+- **Topics:** Roadmap discussion, community feedback, and Q&A
+- **Resources:** [📝 Meeting Notes](https://docs.google.com/document/d/1sIHCcTivyzrF5XNvos7ZT_UcxEOqgwfawsTbb9wMJe4/edit?tab=t.0) | [📋 Community Page](https://holmesgpt.dev/community/)
 
 ## Support