agentx-python 0.6.5__tar.gz → 0.6.7__tar.gz

{agentx_python-0.6.5 → agentx_python-0.6.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentx-python
-Version: 0.6.5
+Version: 0.6.7
 Summary: Official Python SDK for AgentX (https://www.agentx.so/)
 Home-page: https://github.com/AgentX-ai/AgentX-python
 Author: Robin Wang and AgentX Team
@@ -70,6 +70,7 @@ Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs
   - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
 - [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration) — teams of agents with a designated manager
 - [Production tracing](#production-tracing) — record live agent runs from any framework
+- [Monitor](#monitor) — automatic production monitoring, patterns and signals
 - [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
 - [Links](#links)
 
@@ -245,6 +246,30 @@ See **[TRACING.md](TRACING.md)** for the complete guide — session grouping, er
 
 ---
 
+## Monitor
+
+Automatic production monitoring: check traces against detection patterns and get back triage-ready signals. A **pattern** is a first-class SDK resource with a real id, just like a `Dataset` or `EvaluationSettings`: create one once, then reference it by id at trace time.
+
+```python
+pattern = client.monitor.patterns.builder(
+    name="Promises a refund",
+    detector_kind="semantic",
+    semantic_prompt="The response promises a refund.",
+    severity="high",
+).publish()
+
+with client.tracer.trace("support-agent", monitor=True, pattern_ids=[pattern.id]) as span:
+    span.output = call_llm(query)
+```
+
+`monitor=True` checks the trace immediately, no dashboard setup required. `pattern_ids` restricts detection to exactly those patterns; omit it to run the full default sweep instead (built-in checks like empty response, trace error, and latency regression, plus every pattern enabled for the workspace).
+
+This works independently of the dashboard's per-agent monitoring toggle (Governance > Observe > Agents), which still auto-checks every trace from an agent once enabled there, with no code changes needed either way.
+
+See **[TRACING.md](TRACING.md)** for the complete Monitor guide.
+
+---
+
 ## Custom agent evaluations
 
 Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
@@ -261,8 +286,13 @@ report = (
 print(report.average_rating)       # LLM-graded score, 0–10
 print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
 print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+
+print(report.summary)              # AI-generated narrative from .analyze()
+print(report.recommendations)      # list of prioritized, actionable fixes
 ```
 
+`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
+
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, and the complete API reference.
 
 ---
@@ -274,3 +304,4 @@ See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder,
 - **PyPI** — [agentx-python](https://pypi.org/project/agentx-python/)
 - **Tracing docs** — [TRACING.md](TRACING.md)
 - **Evaluations docs** — [EVALUATIONS.md](EVALUATIONS.md)
+- **Monitor docs** — [docs.agentx.so/sdk/monitor](https://docs.agentx.so/sdk/monitor)

{agentx_python-0.6.5 → agentx_python-0.6.7}/README.md

@@ -22,6 +22,7 @@ Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs
   - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
 - [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration) — teams of agents with a designated manager
 - [Production tracing](#production-tracing) — record live agent runs from any framework
+- [Monitor](#monitor) — automatic production monitoring, patterns and signals
 - [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
 - [Links](#links)
 
@@ -197,6 +198,30 @@ See **[TRACING.md](TRACING.md)** for the complete guide — session grouping, er
 
 ---
 
+## Monitor
+
+Automatic production monitoring: check traces against detection patterns and get back triage-ready signals. A **pattern** is a first-class SDK resource with a real id, just like a `Dataset` or `EvaluationSettings`: create one once, then reference it by id at trace time.
+
+```python
+pattern = client.monitor.patterns.builder(
+    name="Promises a refund",
+    detector_kind="semantic",
+    semantic_prompt="The response promises a refund.",
+    severity="high",
+).publish()
+
+with client.tracer.trace("support-agent", monitor=True, pattern_ids=[pattern.id]) as span:
+    span.output = call_llm(query)
+```
+
+`monitor=True` checks the trace immediately, no dashboard setup required. `pattern_ids` restricts detection to exactly those patterns; omit it to run the full default sweep instead (built-in checks like empty response, trace error, and latency regression, plus every pattern enabled for the workspace).
+
+This works independently of the dashboard's per-agent monitoring toggle (Governance > Observe > Agents), which still auto-checks every trace from an agent once enabled there, with no code changes needed either way.
+
+See **[TRACING.md](TRACING.md)** for the complete Monitor guide.
+
+---
+
 ## Custom agent evaluations
 
 Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
@@ -213,8 +238,13 @@ report = (
 print(report.average_rating)       # LLM-graded score, 0–10
 print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
 print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+
+print(report.summary)              # AI-generated narrative from .analyze()
+print(report.recommendations)      # list of prioritized, actionable fixes
 ```
 
+`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
+
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, and the complete API reference.
 
 ---
@@ -226,3 +256,4 @@ See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder,
 - **PyPI** — [agentx-python](https://pypi.org/project/agentx-python/)
 - **Tracing docs** — [TRACING.md](TRACING.md)
 - **Evaluations docs** — [EVALUATIONS.md](EVALUATIONS.md)
+- **Monitor docs** — [docs.agentx.so/sdk/monitor](https://docs.agentx.so/sdk/monitor)

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/agentx.py

@@ -29,6 +29,7 @@ class AgentX:
 
         from agentx.evaluations.client import EvaluationsClient
         from agentx.evaluations.runner import EvaluationsRunner
+        from agentx.monitor.client import MonitorClient
         from agentx.tracing.ingest_client import IngestClient
         from agentx.tracing.tracer import Tracer
         from agentx.version import VERSION
@@ -41,6 +42,15 @@ class AgentX:
         )
         self.evaluations = EvaluationsRunner(_eval_client)
 
+        # Monitor: create/reuse patterns (client.monitor.patterns) that a trace can be checked
+        # against at send time via tracer.trace(..., monitor=True, pattern_ids=[...]).
+        self.monitor = MonitorClient(
+            api_key=self.api_key,
+            sdk_version=VERSION,
+            base_url=self.base_url,
+            workspace_id=self.workspace_id,
+        )
+
         _ingest_client = IngestClient(
             api_key=self.api_key,
             sdk_version=VERSION,

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/evaluations/_term.py

@@ -71,6 +71,12 @@ class Spinner:
         self._thread.start()
         return self
 
+    def update(self, message: str) -> None:
+        """Change the displayed message while the spinner keeps running."""
+        self._message = message
+        if not _IS_TTY:
+            print(f"  {message}...", flush=True)
+
     def __exit__(self, *_) -> None:
         if not _IS_TTY:
             return

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/evaluations/client.py

@@ -9,6 +9,7 @@ from typing import Any, Dict, List, Optional
 import requests
 
 from agentx.evaluations.models import (
+    AnalysisStatus,
     BatchAppendResponse,
     Dataset,
     EvaluationResult,
@@ -238,8 +239,28 @@ class EvaluationsClient:
             "POST", f"/runs/{run_id}/finalize", json={"status": "completed"}
         )
 
-    def analyze_run(self, run_id: str) -> Dict[str, Any]:
-        return self._request("POST", f"/runs/{run_id}/analyze", json={}, timeout=300)
+    def analyze_run(
+        self,
+        run_id: str,
+        mode: Optional[str] = None,
+        quality_mode: Optional[str] = None,
+        judges: Optional[List[str]] = None,
+    ) -> Dict[str, Any]:
+        # Starts the durable analysis job and returns immediately (e.g. {"jobId": ..., "status":
+        # "pending"}); poll get_analysis_status() until it reaches a terminal status, then call
+        # get_report(). mode/quality_mode/judges mirror the dashboard's AnalyzeEvaluationRequest.
+        payload: Dict[str, Any] = {}
+        if mode is not None:
+            payload["mode"] = mode
+        if quality_mode is not None:
+            payload["qualityMode"] = quality_mode
+        if judges is not None:
+            payload["judges"] = [{"model": m} for m in judges]
+        return self._request("POST", f"/runs/{run_id}/analyze", json=payload, timeout=30)
+
+    def get_analysis_status(self, run_id: str) -> AnalysisStatus:
+        data = self._request("GET", f"/runs/{run_id}/analyze-status")
+        return AnalysisStatus(**data)
 
     def get_run(self, run_id: str) -> Dict[str, Any]:
         return self._request("GET", f"/runs/{run_id}")

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/evaluations/models.py

@@ -438,11 +438,9 @@ class SovereigntyIndex(BaseModel):
         extra = "ignore"
 
 
-class Report(BaseModel):
-    run_id: str = Field(alias="runId")
-    dataset_id: str = Field(alias="datasetId")
-    status: str = "completed"
-    statistics: Optional[ReportStatistics] = None
+class AnalysisResult(BaseModel):
+    """Shared qualitative-report fields, produced by ``client.evaluations.run(...).analyze()``."""
+
     summary: Optional[str] = None
     consistency_score: Optional[float] = Field(default=None, alias="consistencyScore")
     instruction_adherence: Optional[ReportInstructionAdherence] = Field(
@@ -461,6 +459,17 @@ class Report(BaseModel):
     weaknesses: List[str] = Field(default_factory=list)
     overall_rating: Optional[str] = Field(default=None, alias="overallRating")
     recommendations: List[ReportRecommendation] = Field(default_factory=list)
+
+    class Config:
+        populate_by_name = True
+        extra = "ignore"
+
+
+class Report(AnalysisResult):
+    run_id: str = Field(alias="runId")
+    dataset_id: str = Field(alias="datasetId")
+    status: str = "completed"
+    statistics: Optional[ReportStatistics] = None
     low_scoring_cases: List[Dict[str, Any]] = Field(
         default_factory=list, alias="lowScoringCases"
     )
@@ -548,3 +557,52 @@ class Report(BaseModel):
     def average_rating(self) -> Optional[float]:
         """Convenience accessor matching cosine_similarity / jaccard_similarity."""
         return self.statistics.average_rating if self.statistics is not None else None
+
+
+class AnalysisLevelProgress(BaseModel):
+    total: int = 0
+    completed: int = 0
+    failed: int = 0
+    percentage: int = 0
+
+    class Config:
+        extra = "ignore"
+
+
+class AnalysisProgress(BaseModel):
+    overall_percentage: int = Field(default=0, alias="overallPercentage")
+    current_level: Optional[str] = Field(default=None, alias="currentLevel")
+    levels: Dict[str, AnalysisLevelProgress] = Field(default_factory=dict)
+
+    class Config:
+        populate_by_name = True
+        extra = "ignore"
+
+
+class AnalysisFailureReason(BaseModel):
+    code: str
+    message: str
+    retryable: bool = False
+
+    class Config:
+        extra = "ignore"
+
+
+class AnalysisStatus(BaseModel):
+    """Returned by ``client.evaluations.run(...).analyze()``'s polling loop
+    (``EvaluationsClient.get_analysis_status``). ``status`` is terminal once it's one of
+    "completed", "partially_failed", or "failed"."""
+
+    job_id: Optional[str] = Field(default=None, alias="jobId")
+    status: str = "not_started"
+    progress: AnalysisProgress = Field(default_factory=AnalysisProgress)
+    failure_reason: Optional[AnalysisFailureReason] = Field(default=None, alias="failureReason")
+    warnings: List[Dict[str, Any]] = Field(default_factory=list)
+
+    class Config:
+        populate_by_name = True
+        extra = "ignore"
+
+    @property
+    def is_terminal(self) -> bool:
+        return self.status in ("completed", "partially_failed", "failed")

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/evaluations/runner.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import logging
+import time
 import uuid
 from typing import Any, Callable, Dict, List, Optional, Set, Union
 
@@ -9,6 +10,7 @@ from agentx.evaluations.adapters.precomputed import PrecomputedAdapter
 from agentx.evaluations.adapters.http_endpoint import HttpEndpointAdapter
 from agentx.evaluations.client import EvaluationsClient
 from agentx.evaluations.models import (
+    AnalysisStatus,
     Dataset,
     EvaluationCase,
     EvaluationResult,
@@ -43,6 +45,15 @@ AdapterLike = Union[
     HttpEndpointAdapter,
 ]
 
+_ANALYSIS_LEVEL_LABELS = {
+    "l1_score": "scoring responses",
+    "l2_question_reduce": "reducing questions",
+    "l3_cluster_reduce": "reducing clusters",
+    "l4_final_reduce": "writing final report",
+}
+
+_DEFAULT_JUDGE_MODEL = "gpt-5.5"
+
 
 class EvaluationRunContext:
     """
@@ -228,12 +239,61 @@ class EvaluationRunContext:
     # Step 3: analyze + report
     # ------------------------------------------------------------------
 
-    def analyze(self) -> Report:
+    def analyze(
+        self,
+        mode: Optional[str] = None,
+        quality_mode: Optional[str] = None,
+        judges: Optional[List[str]] = None,
+        poll_interval: float = 5.0,
+        timeout: float = 1800.0,
+    ) -> Report:
+        """Generate the qualitative AI analysis report.
+
+        Runs the same durable, multi-stage pipeline as the dashboard's "Analyze" button: each
+        response is scored by 1-3 LLM judges (``judges``), then reduced into the final report.
+        This starts the job and polls until it finishes, which can take noticeably longer than a
+        single LLM call for larger runs.
+
+        Args:
+            mode: "auto" (default), "sync", or "batch" - how item scoring executes server-side.
+            quality_mode: "quality_first" or "balanced" - how many items get a second/third judge.
+            judges: 1-3 model ids, e.g. ``["gpt-5.5", "claude-opus-4-8"]``. Defaults to a single
+                judge, ``["gpt-5.5"]``, rather than the dashboard's 3-judge default - SDK runs are
+                typically lighter-weight, quick-start evaluations.
+            poll_interval: seconds between status checks while waiting.
+            timeout: give up waiting after this many seconds (the job keeps running server-side;
+                call ``get_report()`` later to check on it).
+        """
+        if judges is not None and not (1 <= len(judges) <= 3):
+            raise ValueError("judges must contain 1-3 model ids")
+        resolved_judges = judges if judges is not None else [_DEFAULT_JUDGE_MODEL]
+
         print()
-        with Spinner("Analyzing — AI is reviewing your results"):
+        with Spinner("Analyzing — AI is reviewing your results") as spinner:
             try:
-                self._client.analyze_run(self._run.run_id)
-                print(f"  {green('✓')}  Analysis complete")
+                self._client.analyze_run(
+                    self._run.run_id,
+                    mode=mode,
+                    quality_mode=quality_mode,
+                    judges=resolved_judges,
+                )
+                deadline = time.monotonic() + timeout
+                status = self._client.get_analysis_status(self._run.run_id)
+                while not status.is_terminal and time.monotonic() < deadline:
+                    level = _ANALYSIS_LEVEL_LABELS.get(status.progress.current_level, "")
+                    spinner.update(
+                        f"Analyzing, {level + ' ' if level else ''}{status.progress.overall_percentage}%"
+                    )
+                    time.sleep(poll_interval)
+                    status = self._client.get_analysis_status(self._run.run_id)
+
+                if not status.is_terminal:
+                    print(f"  {yellow('!')}  Still running after {int(timeout)}s, check the dashboard for status")
+                elif status.status == "failed":
+                    reason = status.failure_reason.message if status.failure_reason else "unknown error"
+                    print(f"  {red('✗')}  Analyze failed: {dim(reason)}")
+                else:
+                    print(f"  {green('✓')}  Analysis complete")
             except Exception as exc:
                 print(f"  {red('✗')}  Analyze failed: {dim(str(exc))}")
                 logger.warning("Analyze request failed: %s", exc)
@@ -280,6 +340,12 @@ class EvaluationsRunner:
         against."""
         return self._client.list_models(provider)
 
+    def get_analysis_status(self, run_id: str) -> AnalysisStatus:
+        """Check on an in-progress ``.analyze()`` job by run id, without needing
+        the ``EvaluationRunContext`` that started it (e.g. from a separate
+        script execution)."""
+        return self._client.get_analysis_status(run_id)
+
     def run(
         self,
         dataset_id: str,

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/integrations/crewai.py

@@ -121,8 +121,13 @@ class AgentXCrewObserver:
         input: Optional[Any] = None,
         metadata: Optional[Dict[str, Any]] = None,
         session_id: Optional[str] = None,
+        sync: bool = False,
     ):
-        """Return a context-manager span for manual kickoff wrapping."""
+        """Return a context-manager span for manual kickoff wrapping.
+
+        Pass ``sync=True`` to send synchronously so ``span.trace_id`` is populated once the
+        block exits — e.g. to attach the trace to an evaluation result. See ``Tracer.trace()``.
+        """
         from agentx.tracing.tracer import _TraceSpan
 
         return _TraceSpan(
@@ -132,4 +137,5 @@ class AgentXCrewObserver:
             metadata=metadata or self._metadata,
             framework="crewai",
             session_id=session_id or self._session_id,
+            sync=sync,
         )

agentx_python-0.6.7/agentx/monitor/__init__.py

@@ -0,0 +1,10 @@
+from agentx.monitor.client import MonitorClient
+from agentx.monitor.models import MonitorPattern
+from agentx.monitor.patterns import MonitorPatternBuilder, MonitorPatternClient
+
+__all__ = [
+    "MonitorClient",
+    "MonitorPattern",
+    "MonitorPatternBuilder",
+    "MonitorPatternClient",
+]

agentx_python-0.6.7/agentx/monitor/client.py

@@ -0,0 +1,134 @@
+from __future__ import annotations
+
+import logging
+import os
+import time
+from typing import Any, List, Optional
+
+import requests
+
+from agentx.monitor.models import MonitorPattern
+
+logger = logging.getLogger(__name__)
+
+from agentx.util import _DEFAULT_API_BASE as _UTIL_API_BASE
+
+SDK_NAME = "agentx-python"
+
+_RETRYABLE_STATUS = {429, 500, 502, 503, 504}
+_MAX_RETRIES = 3
+_RETRY_BACKOFF = [1.0, 2.0, 4.0]
+
+
+class AgentXMonitorError(Exception):
+    pass
+
+
+class AgentXAuthError(AgentXMonitorError):
+    pass
+
+
+class AgentXValidationError(AgentXMonitorError):
+    pass
+
+
+class MonitorClient:
+    """Low-level HTTP client for the Monitor API (``/monitor``). Accessed via
+    ``client.monitor`` on the top-level :class:`agentx.AgentX` instance; most callers
+    should use ``client.monitor.patterns.builder(...)`` instead of this directly."""
+
+    def __init__(
+        self,
+        api_key: str,
+        sdk_version: str = "unknown",
+        base_url: Optional[str] = None,
+        workspace_id: Optional[str] = None,
+    ):
+        if not api_key:
+            raise AgentXAuthError("AGENTX_API_KEY is required")
+        self._api_key = api_key
+        self._sdk_version = sdk_version
+        # Falls back to the caller's default workspace server-side when unset — mirrors
+        # EvaluationsClient. Without this, pattern creation silently lands in whatever
+        # workspace the API key's user defaults to, not the one the caller intended.
+        self._workspace_id = workspace_id
+        _api_base = (
+            base_url or os.getenv("AGENTX_API_BASE_URL", _UTIL_API_BASE)
+        ).rstrip("/")
+        if not _api_base.endswith("/monitor"):
+            _api_base = f"{_api_base}/monitor"
+        self._base_url = _api_base
+        self._session = requests.Session()
+        self._session.headers.update(
+            {
+                "x-api-key": self._api_key,
+                "Content-Type": "application/json",
+                "User-Agent": f"{SDK_NAME}/{self._sdk_version}",
+                "accept": "*/*",
+            }
+        )
+
+        from agentx.monitor.patterns import MonitorPatternClient
+
+        self.patterns = MonitorPatternClient(self)
+
+    # ------------------------------------------------------------------
+    # Low-level HTTP
+    # ------------------------------------------------------------------
+
+    def _with_workspace(self, payload: dict) -> dict:
+        if self._workspace_id and not payload.get("workspaceId"):
+            return {**payload, "workspaceId": self._workspace_id}
+        return payload
+
+    def _workspace_params(self) -> Optional[dict]:
+        return {"workspaceId": self._workspace_id} if self._workspace_id else None
+
+    def _request(self, method: str, path: str, timeout: int = 30, **kwargs) -> Any:
+        url = f"{self._base_url}{path}"
+        last_exc: Optional[Exception] = None
+        for attempt, wait in enumerate([0.0] + _RETRY_BACKOFF):
+            if wait:
+                time.sleep(wait)
+            try:
+                resp = self._session.request(method, url, timeout=timeout, **kwargs)
+            except requests.RequestException as e:
+                last_exc = e
+                logger.debug("Request error (attempt %d): %s", attempt + 1, e)
+                continue
+
+            if resp.status_code == 401:
+                raise AgentXAuthError("Invalid or missing API key")
+            if resp.status_code == 422:
+                raise AgentXValidationError(resp.text)
+            if resp.status_code in _RETRYABLE_STATUS and attempt < _MAX_RETRIES - 1:
+                logger.debug(
+                    "Retryable status %d (attempt %d)", resp.status_code, attempt + 1
+                )
+                last_exc = AgentXMonitorError(f"HTTP {resp.status_code}")
+                continue
+            if not resp.ok:
+                raise AgentXMonitorError(f"HTTP {resp.status_code}: {resp.text}")
+            try:
+                return resp.json()
+            except Exception:
+                return resp.text
+        raise AgentXMonitorError(f"Request failed after retries: {last_exc}")
+
+    # ------------------------------------------------------------------
+    # Pattern endpoints
+    # ------------------------------------------------------------------
+
+    def create_pattern(self, payload: dict) -> MonitorPattern:
+        data = self._request("POST", "/patterns", json=self._with_workspace(payload))
+        return MonitorPattern(**data["pattern"])
+
+    def list_patterns(self) -> List[MonitorPattern]:
+        data = self._request("GET", "/patterns", params=self._workspace_params())
+        return [MonitorPattern(**p) for p in data.get("patterns", [])]
+
+    def get_pattern(self, pattern_id: str) -> MonitorPattern:
+        data = self._request(
+            "GET", f"/patterns/{pattern_id}", params=self._workspace_params()
+        )
+        return MonitorPattern(**data["pattern"])

agentx_python-0.6.7/agentx/monitor/models.py

@@ -0,0 +1,37 @@
+from __future__ import annotations
+
+from typing import List, Optional
+from pydantic import BaseModel, Field
+
+
+class MonitorPattern(BaseModel):
+    """A detection rule checked against production traces. Built via
+    ``client.monitor.patterns.builder(...).publish()`` and referenced by id as a
+    ``pattern_ids`` entry in ``tracer.trace(..., monitor=True, pattern_ids=[...])``.
+
+    A "failure" pattern (the default) raises a signal to triage; a "proper" pattern logs a
+    healthy tally instead. Only one of ``include_terms``/``regex``/``semantic_prompt`` is
+    meaningful at a time, selected by ``detector_kind``.
+    """
+
+    id: str = Field(alias="_id")
+    key: str
+    name: str
+    description: Optional[str] = None
+    category: Optional[str] = None
+    detector_kind: str = Field(default="contains", alias="detectorKind")
+    match_target: List[str] = Field(default_factory=lambda: ["response"], alias="matchTarget")
+    match_mode: str = Field(default="any", alias="matchMode")
+    include_terms: List[str] = Field(default_factory=list, alias="includeTerms")
+    exclude_terms: List[str] = Field(default_factory=list, alias="excludeTerms")
+    regex: Optional[str] = None
+    semantic_prompt: Optional[str] = Field(default=None, alias="semanticPrompt")
+    severity: str = "medium"
+    polarity: str = "failure"
+    enabled: bool = True
+    sample_rate: float = Field(default=1.0, alias="sampleRate")
+    scope_mode: str = Field(default="all", alias="scopeMode")
+    agent_ids: List[str] = Field(default_factory=list, alias="agentIds")
+
+    class Config:
+        extra = "ignore"

agentx_python-0.6.7/agentx/monitor/patterns.py

@@ -0,0 +1,121 @@
+from __future__ import annotations
+
+import logging
+from typing import Any, Dict, List, Optional, TYPE_CHECKING
+
+from agentx.monitor.models import MonitorPattern
+
+if TYPE_CHECKING:
+    from agentx.monitor.client import MonitorClient
+
+logger = logging.getLogger(__name__)
+
+
+class MonitorPatternBuilder:
+    """Fluent builder for creating a custom pattern. ``detector_kind`` selects which of
+    ``include_terms``/``regex``/``semantic_prompt`` is used:
+
+    - ``"contains"`` (default): ``include_terms`` — a match if any (or all, with
+      ``match_mode="all"``) phrase appears in the target text.
+    - ``"regex"``: ``regex`` — a single regular expression.
+    - ``"semantic"``: ``semantic_prompt`` — an LLM judges whether the response violates the
+      described rubric.
+    """
+
+    def __init__(
+        self,
+        client: "MonitorClient",
+        name: str,
+        description: Optional[str] = None,
+        category: Optional[str] = None,
+        detector_kind: str = "contains",
+        match_target: Optional[List[str]] = None,
+        match_mode: str = "any",
+        include_terms: Optional[List[str]] = None,
+        exclude_terms: Optional[List[str]] = None,
+        regex: Optional[str] = None,
+        semantic_prompt: Optional[str] = None,
+        severity: str = "medium",
+        polarity: str = "failure",
+        enabled: bool = True,
+        sample_rate: float = 1.0,
+        scope_mode: str = "all",
+        agent_ids: Optional[List[str]] = None,
+    ):
+        self._client = client
+        self._payload: Dict[str, Any] = {
+            "name": name,
+            "description": description,
+            "category": category,
+            "detectorKind": detector_kind,
+            "matchTarget": match_target or ["response"],
+            "matchMode": match_mode,
+            "includeTerms": include_terms or [],
+            "excludeTerms": exclude_terms or [],
+            "regex": regex,
+            "semanticPrompt": semantic_prompt,
+            "severity": severity,
+            # A "failure" pattern (default) raises a signal to triage; a "proper" pattern logs
+            # a healthy tally instead.
+            "polarity": polarity,
+            "enabled": enabled,
+            "sampleRate": sample_rate,
+            "scopeMode": scope_mode,
+            "agentIds": agent_ids or [],
+        }
+
+    def publish(self) -> MonitorPattern:
+        logger.info("Publishing monitor pattern '%s'", self._payload["name"])
+        return self._client.create_pattern(self._payload)
+
+
+class MonitorPatternClient:
+    """Thin wrapper surfaced as ``client.monitor.patterns``."""
+
+    def __init__(self, client: "MonitorClient"):
+        self._client = client
+
+    def builder(
+        self,
+        name: str,
+        description: Optional[str] = None,
+        category: Optional[str] = None,
+        detector_kind: str = "contains",
+        match_target: Optional[List[str]] = None,
+        match_mode: str = "any",
+        include_terms: Optional[List[str]] = None,
+        exclude_terms: Optional[List[str]] = None,
+        regex: Optional[str] = None,
+        semantic_prompt: Optional[str] = None,
+        severity: str = "medium",
+        polarity: str = "failure",
+        enabled: bool = True,
+        sample_rate: float = 1.0,
+        scope_mode: str = "all",
+        agent_ids: Optional[List[str]] = None,
+    ) -> MonitorPatternBuilder:
+        return MonitorPatternBuilder(
+            self._client,
+            name=name,
+            description=description,
+            category=category,
+            detector_kind=detector_kind,
+            match_target=match_target,
+            match_mode=match_mode,
+            include_terms=include_terms,
+            exclude_terms=exclude_terms,
+            regex=regex,
+            semantic_prompt=semantic_prompt,
+            severity=severity,
+            polarity=polarity,
+            enabled=enabled,
+            sample_rate=sample_rate,
+            scope_mode=scope_mode,
+            agent_ids=agent_ids,
+        )
+
+    def get(self, pattern_id: str) -> MonitorPattern:
+        return self._client.get_pattern(pattern_id)
+
+    def list(self) -> List[MonitorPattern]:
+        return self._client.list_patterns()

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx/tracing/tracer.py

@@ -78,6 +78,8 @@ class _TraceSpan:
         model: Optional[str] = None,
         session_id: Optional[str] = None,
         sync: bool = False,
+        monitor: bool = False,
+        pattern_ids: Optional[List[str]] = None,
     ) -> None:
         self._tracer = tracer
         self.name = name
@@ -91,6 +93,11 @@ class _TraceSpan:
         # is populated by the time the `with` block exits — see Tracer.trace()'s sync param.
         self._sync = sync
         self._trace_id: Optional[str] = None
+        # Monitor: check this trace against patterns immediately on ingest, no dashboard profile
+        # required. pattern_ids (if given) fully defines what's checked — only those patterns run,
+        # the built-in checks are skipped. See Tracer.trace()'s monitor/pattern_ids params.
+        self._monitor = monitor
+        self._pattern_ids = pattern_ids
 
         # Fields the caller can set while inside the context manager
         self.output: Any = None
@@ -171,6 +178,8 @@ class _TraceSpan:
 
         self._trace_id = self._tracer._send(
             sync=self._sync,
+            monitor=self._monitor or None,
+            pattern_ids=self._pattern_ids,
             name=self.name,
             input=_safe_serialize(self.input) if self.input is not None else None,
             output=_safe_serialize(self.output) if self.output is not None else None,
@@ -615,6 +624,8 @@ class Tracer:
         model: Optional[str] = None,
         session_id: Optional[str] = None,
         sync: bool = False,
+        monitor: bool = False,
+        pattern_ids: Optional[List[str]] = None,
     ) -> _TraceSpan:
         """
         Return a :class:`_TraceSpan` that works as both a decorator and a
@@ -629,6 +640,19 @@ class Tracer:
                 resp = call_llm(...)
                 span.output = resp
             return {"output": resp, "trace_id": span.trace_id}
+
+        Pass ``monitor=True`` to check this trace against Monitor patterns immediately, with no
+        dashboard profile required. ``pattern_ids`` (ids from ``client.monitor.patterns.builder(
+        ...).publish()``) restricts detection to exactly those patterns; omit it to run the full
+        default sweep (built-in checks plus every enabled workspace pattern)::
+
+            pattern = client.monitor.patterns.builder(
+                name="Promises a refund", detector_kind="semantic",
+                semantic_prompt="The response promises a refund.",
+            ).publish()
+
+            with client.tracer.trace("support_agent_call", monitor=True, pattern_ids=[pattern.id]) as span:
+                span.output = call_llm(...)
         """
         return _TraceSpan(
             tracer=self,
@@ -639,6 +663,8 @@ class Tracer:
             model=model,
             session_id=session_id,
             sync=sync,
+            monitor=monitor,
+            pattern_ids=pattern_ids,
         )
 
     def flush(self, timeout: float = 5.0) -> None:
@@ -862,6 +888,10 @@ class Tracer:
             wire["input_tokens"] = payload["input_tokens"]
         if "output_tokens" in payload:
             wire["output_tokens"] = payload["output_tokens"]
+        if "monitor" in payload:
+            wire["monitor"] = payload["monitor"]
+        if "pattern_ids" in payload:
+            wire["pattern_ids"] = payload["pattern_ids"]
 
         pending_retrievals, self._pending_retrievals = self._pending_retrievals, []
         pending_tool_calls, self._pending_tool_calls = self._pending_tool_calls, []

agentx_python-0.6.7/agentx/version.py

@@ -0,0 +1 @@
+VERSION = "0.6.7"

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx_python.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentx-python
-Version: 0.6.5
+Version: 0.6.7
 Summary: Official Python SDK for AgentX (https://www.agentx.so/)
 Home-page: https://github.com/AgentX-ai/AgentX-python
 Author: Robin Wang and AgentX Team
@@ -70,6 +70,7 @@ Also see [SDK Developer Docs](https://developers.agentx.so), [API Reference Docs
   - [Chat (streaming and non-streaming)](#chat-streaming-and-non-streaming)
 - [Workforce (multi-agent orchestration)](#workforce-multi-agent-orchestration) — teams of agents with a designated manager
 - [Production tracing](#production-tracing) — record live agent runs from any framework
+- [Monitor](#monitor) — automatic production monitoring, patterns and signals
 - [Custom agent evaluations](#custom-agent-evaluations) — LLM-as-a-judge, cosine / Jaccard similarity
 - [Links](#links)
 
@@ -245,6 +246,30 @@ See **[TRACING.md](TRACING.md)** for the complete guide — session grouping, er
 
 ---
 
+## Monitor
+
+Automatic production monitoring: check traces against detection patterns and get back triage-ready signals. A **pattern** is a first-class SDK resource with a real id, just like a `Dataset` or `EvaluationSettings`: create one once, then reference it by id at trace time.
+
+```python
+pattern = client.monitor.patterns.builder(
+    name="Promises a refund",
+    detector_kind="semantic",
+    semantic_prompt="The response promises a refund.",
+    severity="high",
+).publish()
+
+with client.tracer.trace("support-agent", monitor=True, pattern_ids=[pattern.id]) as span:
+    span.output = call_llm(query)
+```
+
+`monitor=True` checks the trace immediately, no dashboard setup required. `pattern_ids` restricts detection to exactly those patterns; omit it to run the full default sweep instead (built-in checks like empty response, trace error, and latency regression, plus every pattern enabled for the workspace).
+
+This works independently of the dashboard's per-agent monitoring toggle (Governance > Observe > Agents), which still auto-checks every trace from an agent once enabled there, with no code changes needed either way.
+
+See **[TRACING.md](TRACING.md)** for the complete Monitor guide.
+
+---
+
 ## Custom agent evaluations
 
 Evaluate **any** AI agent — LangChain, CrewAI, AutoGen, LlamaIndex, OpenAI, Anthropic, HTTP endpoints, or plain Python — using AgentX as the scoring and reporting backend. Includes optional **cosine** and **Jaccard** similarity metrics alongside LLM-graded ratings.
@@ -261,8 +286,13 @@ report = (
 print(report.average_rating)       # LLM-graded score, 0–10
 print(report.cosine_similarity)    # embedding cosine, 0–1 (None if not enabled)
 print(report.jaccard_similarity)   # token-set overlap, 0–1 (None if not enabled)
+
+print(report.summary)              # AI-generated narrative from .analyze()
+print(report.recommendations)      # list of prioritized, actionable fixes
 ```
 
+`.analyze()` also generates a full qualitative report (strengths, weaknesses, instruction adherence, reasoning quality, and recommendations), running the same durable, multi-judge pipeline as the dashboard's "Analyze" button. `analyze(mode=..., quality_mode=..., judges=[...])` controls how items are scored and by which models. See [AI analysis report](EVALUATIONS.md#ai-analysis-report) in the full guide for the complete field and parameter reference.
+
 See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder, framework adapters, similarity metrics, and the complete API reference.
 
 ---
@@ -274,3 +304,4 @@ See **[EVALUATIONS.md](EVALUATIONS.md)** for the full guide — dataset builder,
 - **PyPI** — [agentx-python](https://pypi.org/project/agentx-python/)
 - **Tracing docs** — [TRACING.md](TRACING.md)
 - **Evaluations docs** — [EVALUATIONS.md](EVALUATIONS.md)
+- **Monitor docs** — [docs.agentx.so/sdk/monitor](https://docs.agentx.so/sdk/monitor)

{agentx_python-0.6.5 → agentx_python-0.6.7}/agentx_python.egg-info/SOURCES.txt

@@ -30,6 +30,10 @@ agentx/integrations/google_adk.py
 agentx/integrations/google_genai.py
 agentx/integrations/langchain.py
 agentx/integrations/openai_agents.py
+agentx/monitor/__init__.py
+agentx/monitor/client.py
+agentx/monitor/models.py
+agentx/monitor/patterns.py
 agentx/resources/__init__.py
 agentx/resources/agent.py
 agentx/resources/conversation.py

agentx_python-0.6.5/agentx/version.py

@@ -1 +0,0 @@
-VERSION = "0.6.5"