bc-telemetry-buddy-mcp 3.0.2 → 3.1.0

package/CHANGELOG.md

@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [3.1.0] - 2026-02-26
+
+### Added
+- **Agentic Autonomous Telemetry Monitoring** (`src/agent/`) — complete implementation of the autonomous monitoring runtime described in `Instructions/4. Agentic Monitoring/Design.md`
+  - `AgentRuntime` (`runtime.ts`): ReAct loop that drives agents through LLM reasoning + tool calls
+  - `AgentContextManager` (`context.ts`): Manages agent state, instruction, run logs, and sliding-window context compaction
+  - `ActionDispatcher` (`actions.ts`): Executes external actions — Teams webhook, SMTP email, Graph API email, generic webhooks, Azure DevOps pipeline triggers
+  - `prompts.ts`: System prompt, prompt builder, structured output parser, tool scope filtering
+  - `report.ts`: Markdown run report generator (creates human-readable `.md` alongside `.json` audit trail)
+  - `providers/anthropic.ts`: Anthropic/Claude LLM provider (translates OpenAI-style tool calls ↔ Anthropic Messages API)
+- **Agent CLI commands** (`bctb-mcp agent <command>`):
+  - `agent start <instruction> --name <name>` — Create a new monitoring agent
+  - `agent run <name> --once` — Run a single monitoring pass
+  - `agent run-all --once` — Run all active agents
+  - `agent list` — List all agents with status and issue counts
+  - `agent history <name> --limit <n>` — Show run history
+  - `agent pause <name>` / `agent resume <name>` — Set agent status
+- **LLM Provider Abstraction**: `LLMProvider` interface decouples runtime from any specific SDK. Supports Azure OpenAI (default) and Anthropic/Claude.
+- **Pipeline Templates** (`templates/`):
+  - `templates/github-actions/telemetry-agent.yml` + `README.md` — GitHub Actions hourly schedule with state commit
+  - `templates/azure-devops/azure-pipelines.yml` + `README.md` — Azure DevOps pipeline equivalent
+  - `templates/agents/` — Four fully-documented example agent instruction templates:
+    - `appsource-validation` — Monitors AppSource extension validation failures
+    - `performance-monitoring` — Tracks page/report/AL performance with p95 thresholds
+    - `error-rate-monitoring` — Catch-all error rate monitor with dynamic event discovery
+    - `post-deployment-check` — Short-lived post-deployment regression monitor
+- **Config Schema** (`config-schema.json`): Added `agents` section schema with full type definitions for LLM config, defaults, and all 5 action types
+- **Config Template** (`bctb-mcp init`): Now includes agents section template with Azure OpenAI placeholder
+- **7 new test files** (`src/__tests__/agent/`): `context.test.ts`, `runtime.test.ts`, `runtime.integration.test.ts`, `actions.test.ts`, `prompts.test.ts`, `providers.test.ts`, `report.test.ts`, `cli-agent.test.ts` — 18+ new tests; all 564 MCP tests pass
+
 ## [3.0.2] - 2026-02-26
 
 ### Fixed

package/README.md

@@ -12,9 +12,10 @@ BC Telemetry Buddy MCP Server is a **standalone NPM package** that enables AI as
 **Key Features:**
 - 🚀 **Standalone Package**: Install globally with `npm install -g bc-telemetry-buddy-mcp` - no dependencies on VSCode extension
 - 🤖 **AI Assistant Ready**: Works with GitHub Copilot, Claude Desktop, Claude Code, Copilot Studio, and Cursor via the official MCP SDK over stdio
-- 🔧 **CLI Interface**: `bctb-mcp` command with init, validate, test-auth, and start subcommands
+- 🔧 **CLI Interface**: `bctb-mcp` command with init, validate, test-auth, start, and **agent** subcommands
 - 📁 **File-Based Config**: Simple `.bctb-config.json` with schema validation and environment variable substitution
 - 👥 **Multi-Profile Support**: Manage multiple customers/environments in single config file with profile switching
+- 🤖 **Agentic Monitoring**: Autonomous scheduled telemetry monitoring via CI/CD pipelines — LLM-powered reasoning, issue tracking, Teams/email alerts
 - 🔌 **Optional for VSCode**: VSCode extension works standalone; MCP only required for chat participant features
 - 🔄 **Automatic Updates**: Update notifications on startup when newer versions are available on NPM
 - 🧪 **Comprehensive Testing**: 70%+ test coverage with dedicated test suites for Claude Desktop workflows
@@ -263,6 +264,86 @@ bctb-mcp --version
 bctb-mcp --help
 ```
 
+## Agentic Monitoring
+
+BC Telemetry Buddy MCP includes a built-in **autonomous agent runtime** for scheduled telemetry monitoring. Agents use an LLM (Azure OpenAI or Anthropic) to query telemetry, reason about findings, track issues across runs, and take action — all without manual intervention.
+
+### Agent CLI Commands
+
+```bash
+# Create a new monitoring agent
+bctb-mcp agent start "<instruction>" --name <agent-name>
+
+# Run one monitoring pass
+bctb-mcp agent run <agent-name> --once
+
+# Run all active agents
+bctb-mcp agent run-all --once
+
+# List agents with status
+bctb-mcp agent list
+
+# View run history
+bctb-mcp agent history <agent-name> [--limit 10]
+
+# Pause / resume an agent
+bctb-mcp agent pause <agent-name>
+bctb-mcp agent resume <agent-name>
+```
+
+### Minimal Config for Agents
+
+Add an `agents` section to `.bctb-config.json`:
+
+```json
+{
+  "profiles": { ... },
+  "defaultProfile": "default",
+  "agents": {
+    "llm": {
+      "provider": "azure-openai",
+      "endpoint": "https://your-resource.openai.azure.com",
+      "deployment": "gpt-4o",
+      "apiVersion": "2024-10-21"
+    },
+    "actions": {
+      "teams-webhook": { "url": "${TEAMS_WEBHOOK_URL}" }
+    }
+  }
+}
+```
+
+Set `AZURE_OPENAI_KEY` environment variable (or `ANTHROPIC_API_KEY` for Anthropic).
+
+### CI/CD Pipeline Integration
+
+Copy a ready-made pipeline template from the npm package:
+
+```bash
+# GitHub Actions
+cp node_modules/bc-telemetry-buddy-mcp/templates/github-actions/telemetry-agent.yml .github/workflows/
+
+# Azure DevOps
+cp node_modules/bc-telemetry-buddy-mcp/templates/azure-devops/azure-pipelines.yml ./
+```
+
+### Pre-Built Agent Templates
+
+| Template | Use Case |
+|----------|----------|
+| `appsource-validation` | Monitor extension install/update failures |
+| `performance-monitoring` | Track p95 latencies and slow operations |
+| `error-rate-monitoring` | Catch-all error rate monitoring |
+| `post-deployment-check` | Short-lived regression detection after deployments |
+
+```bash
+# Copy a template to your workspace
+cp -r node_modules/bc-telemetry-buddy-mcp/templates/agents/performance-monitoring agents/
+bctb-mcp agent run performance-monitoring --once
+```
+
+See the [User Guide](https://github.com/waldo1001/waldo.BCTelemetryBuddy/blob/main/docs/UserGuide.md#agentic-monitoring) for full docs including action types, state management, and troubleshooting.
+
 ## Features
 
 - **Authentication**: Azure CLI, Device Code, and Client Credentials flows via MSAL

package/config-schema.json

@@ -98,6 +98,103 @@
             "items": {
                 "$ref": "#/definitions/reference"
             }
+        },
+        "agents": {
+            "type": "object",
+            "description": "Agentic Autonomous Telemetry Monitoring — configuration for scheduled monitoring agents that run via bctb-mcp agent commands or CI/CD pipelines.",
+            "properties": {
+                "llm": {
+                    "type": "object",
+                    "description": "LLM provider configuration for agent reasoning.",
+                    "properties": {
+                        "provider": {
+                            "type": "string",
+                            "enum": [
+                                "azure-openai",
+                                "anthropic"
+                            ],
+                            "description": "LLM provider. 'azure-openai' uses AZURE_OPENAI_KEY env var; 'anthropic' uses ANTHROPIC_API_KEY env var.",
+                            "default": "azure-openai"
+                        },
+                        "endpoint": {
+                            "type": "string",
+                            "description": "Azure OpenAI endpoint URL (azure-openai only). Can also be set via AZURE_OPENAI_ENDPOINT env var."
+                        },
+                        "deployment": {
+                            "type": "string",
+                            "description": "Azure OpenAI deployment name or Anthropic model (e.g. 'gpt-4o', 'claude-opus-4-5'). Can also be set via AZURE_OPENAI_DEPLOYMENT / ANTHROPIC_MODEL env var."
+                        },
+                        "model": {
+                            "type": "string",
+                            "description": "Alias for 'deployment'. Preferred for Anthropic configs (e.g. 'claude-opus-4-5')."
+                        },
+                        "apiVersion": {
+                            "type": "string",
+                            "description": "Azure OpenAI API version (azure-openai only).",
+                            "default": "2024-10-21"
+                        }
+                    },
+                    "required": [
+                        "provider"
+                    ]
+                },
+                "defaults": {
+                    "type": "object",
+                    "description": "Default runtime settings applied to all agents.",
+                    "properties": {
+                        "maxToolCalls": {
+                            "type": "number",
+                            "description": "Maximum LLM tool calls per agent run before aborting.",
+                            "default": 20
+                        },
+                        "maxTokens": {
+                            "type": "number",
+                            "description": "Maximum tokens per LLM response.",
+                            "default": 4096
+                        },
+                        "contextWindowRuns": {
+                            "type": "number",
+                            "description": "Sliding window size for recent runs kept in state.json.",
+                            "default": 5
+                        },
+                        "resolvedIssueTTLDays": {
+                            "type": "number",
+                            "description": "Days to keep resolved issues in state.json before pruning.",
+                            "default": 30
+                        },
+                        "toolScope": {
+                            "type": "string",
+                            "enum": [
+                                "read-only",
+                                "full"
+                            ],
+                            "description": "Tool access scope for agents. 'read-only' prevents save_query and other write tools.",
+                            "default": "read-only"
+                        }
+                    }
+                },
+                "actions": {
+                    "type": "object",
+                    "description": "External action configurations (Teams, email, webhooks, pipelines). Secrets should be provided as environment variables, not stored here.",
+                    "properties": {
+                        "teams-webhook": {
+                            "$ref": "#/definitions/teamsWebhookAction"
+                        },
+                        "email-smtp": {
+                            "$ref": "#/definitions/emailSmtpAction"
+                        },
+                        "email-graph": {
+                            "$ref": "#/definitions/emailGraphAction"
+                        },
+                        "generic-webhook": {
+                            "$ref": "#/definitions/genericWebhookAction"
+                        },
+                        "pipeline-trigger": {
+                            "$ref": "#/definitions/pipelineTriggerAction"
+                        }
+                    }
+                }
+            }
         }
     },
     "definitions": {
@@ -202,6 +299,160 @@
                 "type",
                 "url"
             ]
+        },
+        "teamsWebhookAction": {
+            "type": "object",
+            "description": "Microsoft Teams Incoming Webhook action.",
+            "properties": {
+                "url": {
+                    "type": "string",
+                    "description": "Teams Incoming Webhook URL. Can also be set via TEAMS_WEBHOOK_URL env var."
+                }
+            },
+            "required": [
+                "url"
+            ]
+        },
+        "emailSmtpAction": {
+            "type": "object",
+            "description": "Email via SMTP relay (SendGrid, Office 365, etc.).",
+            "properties": {
+                "host": {
+                    "type": "string",
+                    "description": "SMTP host"
+                },
+                "port": {
+                    "type": "number",
+                    "description": "SMTP port (587 for TLS, 465 for SSL)",
+                    "default": 587
+                },
+                "secure": {
+                    "type": "boolean",
+                    "description": "Use TLS (true for port 465)",
+                    "default": false
+                },
+                "auth": {
+                    "type": "object",
+                    "properties": {
+                        "user": {
+                            "type": "string",
+                            "description": "SMTP username or 'apikey' for SendGrid"
+                        },
+                        "pass": {
+                            "type": "string",
+                            "description": "SMTP password (prefer SMTP_PASSWORD env var)"
+                        }
+                    },
+                    "required": [
+                        "user"
+                    ]
+                },
+                "from": {
+                    "type": "string",
+                    "description": "From address"
+                },
+                "defaultTo": {
+                    "type": "array",
+                    "items": {
+                        "type": "string"
+                    },
+                    "description": "Default recipient email addresses"
+                }
+            },
+            "required": [
+                "host",
+                "port",
+                "auth",
+                "from",
+                "defaultTo"
+            ]
+        },
+        "emailGraphAction": {
+            "type": "object",
+            "description": "Email via Microsoft Graph API (requires Mail.Send app permission).",
+            "properties": {
+                "tenantId": {
+                    "type": "string",
+                    "description": "Azure AD tenant ID"
+                },
+                "clientId": {
+                    "type": "string",
+                    "description": "App Registration client ID with Mail.Send permission"
+                },
+                "from": {
+                    "type": "string",
+                    "description": "Sender mailbox UPN (the mailbox the app sends from)"
+                },
+                "defaultTo": {
+                    "type": "array",
+                    "items": {
+                        "type": "string"
+                    },
+                    "description": "Default recipient email addresses"
+                }
+            },
+            "required": [
+                "tenantId",
+                "clientId",
+                "from",
+                "defaultTo"
+            ]
+        },
+        "genericWebhookAction": {
+            "type": "object",
+            "description": "Generic HTTP webhook (Slack, PagerDuty, custom endpoints).",
+            "properties": {
+                "url": {
+                    "type": "string",
+                    "description": "Webhook URL"
+                },
+                "method": {
+                    "type": "string",
+                    "enum": [
+                        "POST",
+                        "PUT",
+                        "PATCH"
+                    ],
+                    "default": "POST"
+                },
+                "headers": {
+                    "type": "object",
+                    "description": "Additional HTTP headers (e.g. Authorization for Slack or PagerDuty)",
+                    "additionalProperties": {
+                        "type": "string"
+                    }
+                }
+            },
+            "required": [
+                "url"
+            ]
+        },
+        "pipelineTriggerAction": {
+            "type": "object",
+            "description": "Azure DevOps pipeline trigger action.",
+            "properties": {
+                "orgUrl": {
+                    "type": "string",
+                    "description": "Azure DevOps organization URL (e.g. https://dev.azure.com/contoso)"
+                },
+                "project": {
+                    "type": "string",
+                    "description": "Azure DevOps project name"
+                },
+                "pipelineId": {
+                    "type": "number",
+                    "description": "Pipeline definition ID"
+                },
+                "pat": {
+                    "type": "string",
+                    "description": "Personal Access Token (prefer DEVOPS_PAT env var)"
+                }
+            },
+            "required": [
+                "orgUrl",
+                "project",
+                "pipelineId"
+            ]
         }
     }
 }