mongo-x-ray-ftdc 2.0.0__tar.gz

mongo_x_ray_ftdc-2.0.0/PKG-INFO

@@ -0,0 +1,135 @@
+Metadata-Version: 2.4
+Name: mongo-x-ray-ftdc
+Version: 2.0.0
+Summary: FTDC analysis plugin for x-ray
+Requires-Python: <4,>=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: mongo-x-ray>=2.0.0
+Requires-Dist: pymongoftdc>=0.1.1
+Requires-Dist: Pillow>=10.0.0
+
+# mongo-x-ray-ftdc
+
+[![CI](https://github.com/zhangyaoxing/mongo-x-ray-ftdc/actions/workflows/ci.yml/badge.svg)](https://github.com/zhangyaoxing/mongo-x-ray-ftdc/actions/workflows/ci.yml)
+
+FTDC (Full Time Diagnostic Data Capture) analysis plugin for [x-ray](https://github.com/mongodb-ps/ce-mongo-x-ray).
+
+## Install
+
+```bash
+pip install mongo-x-ray mongo-x-ray-ftdc
+```
+
+## Usage
+
+```bash
+x-ray ftdc /var/lib/mongo/diagnostic.data
+x-ray ftdc /var/lib/mongo/diagnostic.data 2026-06-17T08:00:00Z 2026-06-17T10:00:00Z
+x-ray ftdc --discover /data/
+```
+
+## Compatibility
+
+Supports MongoDB 5.0 and above on all topologies:
+
+| Replica Set | Sharded Cluster | Standalone |
+| :---------: | :-------------: | :--------: |
+| ✅ | ✅ | ✅ |
+
+## Parameters
+
+```bash
+x-ray ftdc [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] [--no-browser]
+           [--svg] [-r RATE] [--discover] ftdc_path [start_time] [end_time]
+```
+
+| Argument | Description | Default |
+| --- | --- | --- |
+| `ftdc_path` | Path to a directory containing FTDC files. | required |
+| `start_time` | Inclusive UTC start time in ISO-8601 format. | first data point |
+| `end_time` | Inclusive UTC end time in ISO-8601 format. | last data point |
+| `-s, --checkset` | Checkset to run. | `default` |
+| `-o, --output` | Output folder path. | `output/` |
+| `-f, --format` | Output format: `markdown`, `html` or `pdf` (PDF also keeps Markdown and HTML). | `html` |
+| `--no-browser` | Do not open the generated report in the browser. | `false` |
+| `--svg` | Reference SVG charts in the report instead of converting them to PNG. | `false` |
+| `-r, --rate` | FTDC sampling rate (0-1). | `1 / number of ingested files` |
+| `--discover` | Recursively search the given path for folders containing FTDC files. | `false` |
+
+## Report
+
+The baseline analysis reports the capture timespan and the effective sample rate,
+then groups metrics into Workload, Read/Write Operations and Latencies, and
+Performance sections. It includes operation rates and latencies, host memory
+and CPU utilization, WiredTiger cache utilization, queue depth for each block
+device, and free-space and utilization charts for every reported mount point.
+Each metric shows its peak, average, unit, and a chart saved under the report
+output's `charts` directory.
+
+`start_time` and `end_time` are inclusive UTC ISO-8601 timestamps. When omitted,
+the first and last data points in the archive are used.
+
+### Chart configuration
+
+Chart dimensions can be tuned in the checkset config:
+
+```json
+"BaselineAnalysisItem": {
+  "chart_width": 450,
+  "chart_height": 150
+}
+```
+
+The fallback dimensions are defined in `mongo_x_ray_ftdc/charts.py`. Vertical
+grid lines are spaced every 100 pixels and horizontal grid lines every 50 pixels.
+Workload and operation/latency charts use lines. Performance charts use bars.
+Member-state charts are always 450×50 pixel bars.
+
+## AI Analysis (Optional)
+
+FTDC reports can include AI-generated summaries for each section (Workload,
+Ops and Latencies, Performance). The analysis appears as a brief 2-3 sentence
+assessment at the end of each section, flagging potential issues or confirming
+normal operation.
+
+**Configuration** — set the following environment variables:
+
+| Variable          | Required | Default        | Description                             |
+| ----------------- | :------: | -------------- | --------------------------------------- |
+| `OPENAI_API_KEY`  |   Yes    | —              | API key for the AI service              |
+| `OPENAI_BASE_URL` |    No    | OpenAI default | Compatible API endpoint (e.g. DeepSeek) |
+| `AI_MODEL`        |    No    | `gpt-4o`       | Model name to use                       |
+
+If `OPENAI_API_KEY` is not set, AI analysis is silently skipped.
+
+**Example** `.env` file:
+
+```bash
+OPENAI_API_KEY="sk-..."
+OPENAI_BASE_URL="https://api.deepseek.com"
+AI_MODEL="deepseek-v4-pro"
+```
+
+Or export directly in the shell:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+x-ray ftdc /var/lib/mongo/diagnostic.data
+```
+
+## Analysis Items
+
+| Item | Purpose |
+| --- | --- |
+| `BaselineAnalysisItem` | Summarize the workload and performance of an FTDC capture: Workload, Ops and Latencies and Performance sections with charts, plus an optional AI-generated summary per section. |
+| `MetadataReviewItem` | Display all FTDC metadata in tabbed code blocks for review. |
+
+## Development
+
+Requires Python 3.10+, MongoDB 5.0 or later, and the [mongo-x-ray](https://github.com/mongodb-ps/ce-mongo-x-ray) core package.
+
+```bash
+make unit-test   # run the unit tests
+make lint        # ruff check + ruff format --check
+make minify      # minify templates
+```

mongo_x_ray_ftdc-2.0.0/README.md

@@ -0,0 +1,125 @@
+# mongo-x-ray-ftdc
+
+[![CI](https://github.com/zhangyaoxing/mongo-x-ray-ftdc/actions/workflows/ci.yml/badge.svg)](https://github.com/zhangyaoxing/mongo-x-ray-ftdc/actions/workflows/ci.yml)
+
+FTDC (Full Time Diagnostic Data Capture) analysis plugin for [x-ray](https://github.com/mongodb-ps/ce-mongo-x-ray).
+
+## Install
+
+```bash
+pip install mongo-x-ray mongo-x-ray-ftdc
+```
+
+## Usage
+
+```bash
+x-ray ftdc /var/lib/mongo/diagnostic.data
+x-ray ftdc /var/lib/mongo/diagnostic.data 2026-06-17T08:00:00Z 2026-06-17T10:00:00Z
+x-ray ftdc --discover /data/
+```
+
+## Compatibility
+
+Supports MongoDB 5.0 and above on all topologies:
+
+| Replica Set | Sharded Cluster | Standalone |
+| :---------: | :-------------: | :--------: |
+| ✅ | ✅ | ✅ |
+
+## Parameters
+
+```bash
+x-ray ftdc [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] [--no-browser]
+           [--svg] [-r RATE] [--discover] ftdc_path [start_time] [end_time]
+```
+
+| Argument | Description | Default |
+| --- | --- | --- |
+| `ftdc_path` | Path to a directory containing FTDC files. | required |
+| `start_time` | Inclusive UTC start time in ISO-8601 format. | first data point |
+| `end_time` | Inclusive UTC end time in ISO-8601 format. | last data point |
+| `-s, --checkset` | Checkset to run. | `default` |
+| `-o, --output` | Output folder path. | `output/` |
+| `-f, --format` | Output format: `markdown`, `html` or `pdf` (PDF also keeps Markdown and HTML). | `html` |
+| `--no-browser` | Do not open the generated report in the browser. | `false` |
+| `--svg` | Reference SVG charts in the report instead of converting them to PNG. | `false` |
+| `-r, --rate` | FTDC sampling rate (0-1). | `1 / number of ingested files` |
+| `--discover` | Recursively search the given path for folders containing FTDC files. | `false` |
+
+## Report
+
+The baseline analysis reports the capture timespan and the effective sample rate,
+then groups metrics into Workload, Read/Write Operations and Latencies, and
+Performance sections. It includes operation rates and latencies, host memory
+and CPU utilization, WiredTiger cache utilization, queue depth for each block
+device, and free-space and utilization charts for every reported mount point.
+Each metric shows its peak, average, unit, and a chart saved under the report
+output's `charts` directory.
+
+`start_time` and `end_time` are inclusive UTC ISO-8601 timestamps. When omitted,
+the first and last data points in the archive are used.
+
+### Chart configuration
+
+Chart dimensions can be tuned in the checkset config:
+
+```json
+"BaselineAnalysisItem": {
+  "chart_width": 450,
+  "chart_height": 150
+}
+```
+
+The fallback dimensions are defined in `mongo_x_ray_ftdc/charts.py`. Vertical
+grid lines are spaced every 100 pixels and horizontal grid lines every 50 pixels.
+Workload and operation/latency charts use lines. Performance charts use bars.
+Member-state charts are always 450×50 pixel bars.
+
+## AI Analysis (Optional)
+
+FTDC reports can include AI-generated summaries for each section (Workload,
+Ops and Latencies, Performance). The analysis appears as a brief 2-3 sentence
+assessment at the end of each section, flagging potential issues or confirming
+normal operation.
+
+**Configuration** — set the following environment variables:
+
+| Variable          | Required | Default        | Description                             |
+| ----------------- | :------: | -------------- | --------------------------------------- |
+| `OPENAI_API_KEY`  |   Yes    | —              | API key for the AI service              |
+| `OPENAI_BASE_URL` |    No    | OpenAI default | Compatible API endpoint (e.g. DeepSeek) |
+| `AI_MODEL`        |    No    | `gpt-4o`       | Model name to use                       |
+
+If `OPENAI_API_KEY` is not set, AI analysis is silently skipped.
+
+**Example** `.env` file:
+
+```bash
+OPENAI_API_KEY="sk-..."
+OPENAI_BASE_URL="https://api.deepseek.com"
+AI_MODEL="deepseek-v4-pro"
+```
+
+Or export directly in the shell:
+
+```bash
+export OPENAI_API_KEY="sk-..."
+x-ray ftdc /var/lib/mongo/diagnostic.data
+```
+
+## Analysis Items
+
+| Item | Purpose |
+| --- | --- |
+| `BaselineAnalysisItem` | Summarize the workload and performance of an FTDC capture: Workload, Ops and Latencies and Performance sections with charts, plus an optional AI-generated summary per section. |
+| `MetadataReviewItem` | Display all FTDC metadata in tabbed code blocks for review. |
+
+## Development
+
+Requires Python 3.10+, MongoDB 5.0 or later, and the [mongo-x-ray](https://github.com/mongodb-ps/ce-mongo-x-ray) core package.
+
+```bash
+make unit-test   # run the unit tests
+make lint        # ruff check + ruff format --check
+make minify      # minify templates
+```

mongo_x_ray_ftdc-2.0.0/pyproject.toml

@@ -0,0 +1,61 @@
+[build-system]
+requires = ["setuptools==83.0.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "mongo-x-ray-ftdc"
+version = "2.0.0"
+description = "FTDC analysis plugin for x-ray"
+readme = "README.md"
+requires-python = ">=3.10,<4"
+dependencies = [
+  "mongo-x-ray>=2.0.0",
+  "pymongoftdc>=0.1.1",
+  "Pillow>=10.0.0",
+]
+
+[project.entry-points."mongo_x_ray.plugins"]
+ftdc = "mongo_x_ray_ftdc.plugin:FtdcPlugin"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["mongo_x_ray_ftdc*"]
+
+[tool.setuptools.package-data]
+"mongo_x_ray_ftdc" = ["templates/**/*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]
+python_files = ["test_*.py"]
+python_classes = ["Test*"]
+python_functions = ["test_*"]
+markers = [
+  "slow: marks tests as slow",
+  "integration: marks tests as integration tests",
+  "unit: marks tests as unit tests",
+]
+
+
+[tool.ruff]
+line-length = 120
+target-version = "py310"
+exclude = [".venv", "build", "dist"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I"]
+ignore = ["E501"]
+
+[tool.ruff.lint.isort]
+known-first-party = ["mongo_x_ray", "mongo_x_ray_ftdc", "mongo_x_ray_gmd", "mongo_x_ray_hc", "mongo_x_ray_log", "mongo_x_ray_risk"]
+
+
+[tool.pyright]
+pythonVersion = "3.10"
+venvPath = "../ce-mongo-x-ray"
+venv = ".venv"
+typeCheckingMode = "basic"
+extraPaths = [
+  "src",
+  "../ce-mongo-x-ray/src",
+]

mongo_x_ray_ftdc-2.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

mongo_x_ray_ftdc-2.0.0/src/mongo_x_ray_ftdc/__init__.py

@@ -0,0 +1,11 @@
+"""
+Copyright (c) 2026 MongoDB Inc.
+
+DISCLAIMER: THESE CODE SAMPLES ARE PROVIDED FOR EDUCATIONAL AND ILLUSTRATIVE PURPOSES ONLY,
+TO DEMONSTRATE THE FUNCTIONALITY OF SPECIFIC MONGODB FEATURES.
+THEY ARE NOT PRODUCTION-READY AND MAY LACK THE SECURITY HARDENING, ERROR HANDLING, AND TESTING REQUIRED FOR A LIVE ENVIRONMENT.
+YOU ARE RESPONSIBLE FOR TESTING, VALIDATING, AND SECURING THIS CODE WITHIN YOUR OWN ENVIRONMENT BEFORE IMPLEMENTATION.
+THIS MATERIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OR LIABILITY.
+
+MongoDB FTDC analysis package.
+"""

mongo_x_ray_ftdc-2.0.0/src/mongo_x_ray_ftdc/ai.py

@@ -0,0 +1,146 @@
+"""
+Copyright (c) 2026 MongoDB Inc.
+
+DISCLAIMER: THESE CODE SAMPLES ARE PROVIDED FOR EDUCATIONAL AND ILLUSTRATIVE PURPOSES ONLY,
+TO DEMONSTRATE THE FUNCTIONALITY OF SPECIFIC MONGODB FEATURES.
+THEY ARE NOT PRODUCTION-READY AND MAY LACK THE SECURITY HARDENING, ERROR HANDLING, AND TESTING REQUIRED FOR A LIVE ENVIRONMENT.
+YOU ARE RESPONSIBLE FOR TESTING, VALIDATING, AND SECURING THIS CODE WITHIN YOUR OWN ENVIRONMENT BEFORE IMPLEMENTATION.
+THIS MATERIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OR LIABILITY.
+
+AI analysis helpers for FTDC reports.
+
+These functions build the FTDC metric prompts and send them through the
+shared client in :mod:`mongo_x_ray.ai_client`.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+
+from mongo_x_ray.ai_client import complete
+
+_logger = logging.getLogger(__name__)
+
+
+def _build_section_prompt(section_title: str, metrics_data: list[dict[str, object]]) -> str:
+    """Build the prompt for a single FTDC section."""
+    parts = [
+        "You are analyzing MongoDB FTDC (Full-Time Diagnostic Data Capture) "
+        + "metrics from a 24-hour monitoring period.",
+        "",
+        f"## {section_title}",
+        "",
+        "Each metric below has ~1440 data points, sampled every 60 seconds "
+        + "from the raw 1-second FTDC data. Values are provided as a JSON array.",
+        "",
+    ]
+
+    for entry in metrics_data:
+        metric = entry["metric"]
+        unit = entry.get("unit", "")
+        peak = entry.get("peak", "N/A")
+        avg = entry.get("average", "N/A")
+        values = entry.get("values", [])
+
+        parts.append(f"### {metric}")
+        parts.append(f"- Unit: {unit}")
+        parts.append(f"- Peak: {peak}")
+        parts.append(f"- Average: {avg}")
+        parts.append(f"- Values: {json.dumps(values)}")
+        parts.append("")
+
+    parts.extend(
+        [
+            "Provide a very brief summary (2-3 sentences) indicating whether "
+            + "these metrics show any potential issues that need attention. "
+            + "If everything looks normal, simply state that no obvious "
+            + "problems were detected.",
+        ]
+    )
+
+    return "\n".join(parts)
+
+
+def analyze_ftdc_section(
+    section_title: str,
+    metrics_data: list[dict[str, object]],
+) -> str | None:
+    """Send a section's FTDC metrics to the AI for analysis.
+
+    Args:
+        section_title: e.g. ``"1.1 Workload"``.
+        metrics_data: List of dicts with keys:
+            - ``metric``: metric display name
+            - ``unit``: unit string (e.g. ``"ops/s"``)
+            - ``peak``: peak value
+            - ``average``: average value
+            - ``values``: list of ~1440 downsampled float values
+
+    Returns:
+        AI analysis text, or ``None`` if the AI client is not configured.
+    """
+    prompt = _build_section_prompt(section_title, metrics_data)
+    _logger.info(
+        "Sending AI analysis request for %s (%d metrics, %d chars)",
+        section_title,
+        len(metrics_data),
+        len(prompt),
+    )
+    return complete(prompt)
+
+
+def analyze_ftdc_overview(metrics_data: list[dict[str, object]]) -> str | None:
+    """Send all metrics from all sections to AI for cross-section correlation analysis.
+
+    Args:
+        metrics_data: Combined list of all metrics across all sections.
+
+    Returns:
+        AI overview text, or ``None`` if the AI client is not configured.
+    """
+    prompt_parts = [
+        "You are analyzing MongoDB FTDC metrics from a 24-hour monitoring "
+        + "period. Below are ALL metrics from every section (Workload, "
+        + "Ops and Latencies, Performance) combined.",
+        "",
+        "## Cross-Section Overview",
+        "",
+        "For each metric, the downsampled values (~1440 points each) and summary statistics are provided.",
+        "",
+    ]
+
+    for entry in metrics_data:
+        metric = entry["metric"]
+        unit = entry.get("unit", "")
+        peak = entry.get("peak", "N/A")
+        avg = entry.get("average", "N/A")
+        values = entry.get("values", [])
+
+        prompt_parts.append(f"### {metric}")
+        prompt_parts.append(f"- Unit: {unit}")
+        prompt_parts.append(f"- Peak: {peak}")
+        prompt_parts.append(f"- Average: {avg}")
+        prompt_parts.append(f"- Values: {json.dumps(values)}")
+        prompt_parts.append("")
+
+    prompt_parts.extend(
+        [
+            "Look across ALL the above metrics and provide a brief overview "
+            + "(2-3 sentences) of any correlations or relationships between "
+            + "metrics from different sections. For example:",
+            "- Does high workload correlate with increased latency?",
+            "- Does CPU or memory pressure coincide with cache changes?",
+            "- Are there any cascading effects visible across sections?",
+            "",
+            "If no notable cross-section patterns are found, simply state that.",
+        ]
+    )
+
+    prompt = "\n".join(prompt_parts)
+    _logger.info(
+        "Sending AI overview request (%d metrics, %d chars)",
+        len(metrics_data),
+        len(prompt),
+    )
+    return complete(prompt)