failure-forensics 0.1.1__tar.gz

failure_forensics-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 jasstt
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

failure_forensics-0.1.1/MANIFEST.in

@@ -0,0 +1,5 @@
+include README.md
+include LICENSE
+recursive-include failure_forensics *.py
+recursive-include data *.json
+recursive-include data *.jsonl

failure_forensics-0.1.1/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.4
+Name: failure-forensics
+Version: 0.1.1
+Summary: Production AI pipeline monitoring — root cause detection, anomaly alerts, regression guard
+License: MIT License
+Project-URL: Homepage, https://github.com/jasstt/failure-forensics
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests
+Requires-Dist: python-dotenv
+Requires-Dist: google-generativeai
+Dynamic: license-file
+
+# Failure Forensics
+
+Production AI pipeline monitoring — root cause detection, anomaly alerts, regression guard, and Gemini-powered recommendations.
+
+## Installation
+
+```bash
+pip install failure-forensics
+```
+
+## Quick Start
+
+```python
+from failure_forensics import trace
+
+@trace(step="retrieval", version="v1")
+def my_retrieval_function(query):
+    # your code here
+    pass
+```
+
+## Features 🔬
+
+![Python](https://img.shields.io/badge/Python-3.10+-blue)
+![Tests](https://img.shields.io/badge/Tests-8%2F8_PASS-brightgreen)
+![License](https://img.shields.io/badge/License-MIT-lightgrey)
+![Alerts](https://img.shields.io/badge/Slack-Alerts_Ready-4A154B)
+
+A **self-hosted, zero-cost** LLM pipeline observability tool that gives you root cause detection, anomaly alerts, A/B reporting, and a live terminal dashboard — without sending your data to any third-party service.
+
+---
+
+## 🆚 Why Not LangSmith or Braintrust?
+
+| | **Failure Forensics** | LangSmith | Braintrust |
+|---|---|---|---|
+| Cost | **Free** | Paid tiers | Paid tiers |
+| Data privacy | **Stays on your machine** | Sent to cloud | Sent to cloud |
+| Customization | **Full control** | Limited | Limited |
+| Slack alerts | **Built-in** | Premium only | Premium only |
+| A/B reporting | **Built-in** | Basic | Basic |
+| Circuit breaker / trend | **Built-in** | ❌ | ❌ |
+
+**Failure Forensics is designed for teams who need production-grade observability without vendor lock-in.**
+
+---
+
+## ✨ What It Does
+
+Every pipeline run passes through a structured logging and analysis layer:
+
+```
+Pipeline Step  →  logger.py  →  requests.jsonl
+                                     ↓
+                    ┌────────────────┴────────────────┐
+                    │                                 │
+              forensics.py                       pattern.py
+          (root cause detection)          (time series + anomaly)
+                    │                                 │
+              versioning.py                      baseline.py
+           (v1 vs v2 comparison)            (7-day moving average)
+                    │                                 │
+               ab_report.py                      alerts.py
+            (A/B comparison table)          (Slack / console alert)
+                    └────────────────┬────────────────┘
+                                     ↓
+                              dashboard.py
+                         (ASCII terminal dashboard)
+```
+
+---
+
+## 📁 Project Structure
+
+```
+failure-forensics/
+├── src/
+│   ├── logger.py          # Logs every pipeline step to JSONL
+│   ├── forensics.py       # Root cause detection (5 categories)
+│   ├── pattern.py         # Time-series failure rate + anomaly detection
+│   ├── baseline.py        # 7-day moving average + trend (IMPROVING/STABLE/DEGRADING)
+│   ├── alerts.py          # Slack webhook + console alerts
+│   ├── versioning.py      # Per-version failure rate stats
+│   ├── ab_report.py       # A/B comparison report (table + JSON)
+│   └── dashboard.py       # ASCII bar chart terminal dashboard
+├── data/
+│   └── logs/
+│       └── requests.jsonl # All pipeline logs (gitignored)
+├── tests/
+│   └── test_forensics.py  # 8 unit tests
+├── config.py              # Thresholds, Slack URL, step limits
+├── main.py                # 5-scenario demo runner
+├── simulate.py            # Realistic test data generator (100 runs, anomaly day)
+└── requirements.txt
+```
+
+---
+
+## 🚀 Getting Started
+
+### 1. Clone & Install
+
+```bash
+git clone https://github.com/jasstt/failure-forensics.git
+cd failure-forensics
+pip install -r requirements.txt
+```
+
+### 2. (Optional) Configure Slack Alerts
+
+Edit `config.py`:
+
+```python
+SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
+```
+
+If left empty, all alerts print to the console.
+
+### 3. Run the Full Demo
+
+```bash
+python main.py
+```
+
+This runs 5 scenarios:
+1. **Simulation** — generates 100 realistic pipeline runs (2 prompt versions, anomaly day)
+2. **Root cause analysis** — detects the failing step and assigns a category
+3. **7-day pattern report** — failure rate per day + step breakdown + anomaly check
+4. **A/B report** — `prompt_v1` vs `prompt_v2` with per-step improvement table
+5. **Terminal dashboard** — live ASCII bar charts, trend, top 5 failed runs
+
+### 4. Run Unit Tests
+
+```bash
+python tests/test_forensics.py
+python tests/test_advanced.py
+```
+
+---
+
+## 🚀 Advanced Features (New in v2)
+
+| Katman | Özellik | Teknoloji |
+|--------|---------|-----------|
+| 1 | Otomatik öneri motoru | Kural tabanlı |
+| 2 | AI destekli hata analizi | Gemini 2.5 Pro |
+| 3 | Eval seti otomatik büyütme | Frequency analysis |
+| 4 | Prompt optimizasyon açıklaması | Gemini 2.5 Pro |
+| 5 | Regression guard | Baseline comparison |
+
+**Senaryo 6: Regression Guard**
+Yeni bir prompt (`v3`) deploy edilmeden önce otomatik regresyon kontrolü yapar:
+```
+REGRESSION CHECK — v3
+Baseline (v2): 11.0% failure rate
+Yeni (v3):     24.5% failure rate
+Delta: +13.5pp → REGRESSION_DETECTED ❌
+```
+
+## Test Results
+
+| Katman | Test | Sonuç |
+|--------|------|-------|
+| 1 — Recommender | Kategori → öneri mapping | ✅ PASS |
+| 2 — LLM Analyzer | Gemini fallback | ✅ PASS |
+| 3 — Eval Collector | Duplicate prevention | ✅ PASS |
+| 4 — Prompt Optimizer | A/B açıklama (v2: +10pp) | ✅ PASS |
+| 5 — Regression Guard | DETECTED + PASS senaryoları | ✅ PASS |
+
+### Key Results
+- A/B: prompt_v2, v1'e göre 10pp iyileşme
+- Regression Guard: v3 deploy'u +6pp delta ile WARNING olarak engelledi
+- Eval Collector: 5 yeni eval adayı otomatik toplandı
+- LLM Analyzer: Gemini kapalıyken kural tabanlına sorunsuz fallback
+
+---
+
+## 📊 Results
+
+| Feature | Result |
+|---------|--------|
+| Unit Tests | **8/8 PASS** ✅ |
+| Root cause categories | **5 types** (RETRIEVAL_QUALITY, RERANKER_FAILURE, LLM_HALLUCINATION, CITATION_MISS, API_ERROR) |
+| Anomaly detection | **20% delta threshold** — flags when today's rate exceeds 7-day average by >20pp |
+| A/B comparison | **v2: 11.5pp improvement** over v1 (22.5% → 11.0% failure rate) |
+| Trend analysis | **IMPROVING / STABLE / DEGRADING** based on 7-day moving average |
+| Slack integration | **Webhook ready** — fires on rate threshold, anomaly, or 3 consecutive failures |
+
+---
+
+## ⚙️ Configuration (`config.py`)
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `FAILURE_RATE_THRESHOLD` | `0.25` | Alert fires above this failure rate |
+| `ANOMALY_THRESHOLD` | `0.20` | Flag if today exceeds 7-day avg by this delta |
+| `SLACK_WEBHOOK_URL` | `""` | Empty = console output |
+| `CONSECUTIVE_FAILURE_THRESHOLD` | `3` | Alert after N consecutive step failures |
+| `STEP_THRESHOLDS` | see config | Per-step max acceptable failure rate |
+
+---
+
+## 🧪 Root Cause Categories
+
+| Category | Trigger |
+|----------|---------|
+| `RETRIEVAL_QUALITY` | Retrieval step fails — no results, low score |
+| `RERANKER_FAILURE` | Reranker can't parse LLM response or times out |
+| `LLM_HALLUCINATION` | Generation returns empty or uncited response |
+| `CITATION_MISS` | Answer produced but no source citations found |
+| `API_ERROR` | Timeout, 429 rate limit, 503 service unavailable |
+
+---
+
+## 📈 Terminal Dashboard (Sample Output)
+
+```
+═════════════════════════════════════════════════════════════
+  🔬  FAILURE FORENSICS — Terminal Dashboard
+═════════════════════════════════════════════════════════════
+
+  📅 SON 7 GÜNÜN FAILURE RATE GRAFİĞİ
+  2026-06-03  [███░░░░░░░░░░░░░░░░░░░░░░░░░░░] 13.0%
+  2026-06-07  [████████░░░░░░░░░░░░░░░░░░░░░░] 27.3% ⚠️
+  2026-06-10  [███░░░░░░░░░░░░░░░░░░░░░░░░░░░] 12.0%
+
+  🔍 ADIM BAZINDA HATA DAĞILIMI
+  retrieval     [███████░░░░░░░░░░░░░] 38.0%  (38/100 hatalı)
+  reranking     [██░░░░░░░░░░░░░░░░░░] 13.0%  (13/100 hatalı)
+  generation    [██░░░░░░░░░░░░░░░░░░] 10.0%  (10/100 hatalı)
+  citation      [█░░░░░░░░░░░░░░░░░░░]  6.0%  (6/100 hatalı)
+
+  ⚡ ANOMALİ: ✅ Normal: Bugün (12.0%) ≈ 7g ort. (16.2%)
+  📊 TREND: ➡️  STABLE — Hareketli Ort: 16.0%
+```
+
+---
+
+## 🛠 Technologies Used
+
+* **Python standard library** — `json`, `collections`, `datetime`, `threading`
+* **[requests](https://pypi.org/project/requests/)** — Slack webhook HTTP calls
+* **[python-dotenv](https://pypi.org/project/python-dotenv/)** — Environment variable management
+
+No heavy dependencies. No cloud. No API keys required.
+
+---
+
+## 🔭 Roadmap
+
+- [ ] FastAPI REST endpoint for remote log ingestion
+- [ ] HTML report export
+- [ ] PostgreSQL backend for large-scale log storage
+- [ ] Multi-pipeline support (compare RAG vs fine-tuned model)
+- [ ] Email alerts as alternative to Slack

failure_forensics-0.1.1/README.md

@@ -0,0 +1,255 @@
+# Failure Forensics
+
+Production AI pipeline monitoring — root cause detection, anomaly alerts, regression guard, and Gemini-powered recommendations.
+
+## Installation
+
+```bash
+pip install failure-forensics
+```
+
+## Quick Start
+
+```python
+from failure_forensics import trace
+
+@trace(step="retrieval", version="v1")
+def my_retrieval_function(query):
+    # your code here
+    pass
+```
+
+## Features 🔬
+
+![Python](https://img.shields.io/badge/Python-3.10+-blue)
+![Tests](https://img.shields.io/badge/Tests-8%2F8_PASS-brightgreen)
+![License](https://img.shields.io/badge/License-MIT-lightgrey)
+![Alerts](https://img.shields.io/badge/Slack-Alerts_Ready-4A154B)
+
+A **self-hosted, zero-cost** LLM pipeline observability tool that gives you root cause detection, anomaly alerts, A/B reporting, and a live terminal dashboard — without sending your data to any third-party service.
+
+---
+
+## 🆚 Why Not LangSmith or Braintrust?
+
+| | **Failure Forensics** | LangSmith | Braintrust |
+|---|---|---|---|
+| Cost | **Free** | Paid tiers | Paid tiers |
+| Data privacy | **Stays on your machine** | Sent to cloud | Sent to cloud |
+| Customization | **Full control** | Limited | Limited |
+| Slack alerts | **Built-in** | Premium only | Premium only |
+| A/B reporting | **Built-in** | Basic | Basic |
+| Circuit breaker / trend | **Built-in** | ❌ | ❌ |
+
+**Failure Forensics is designed for teams who need production-grade observability without vendor lock-in.**
+
+---
+
+## ✨ What It Does
+
+Every pipeline run passes through a structured logging and analysis layer:
+
+```
+Pipeline Step  →  logger.py  →  requests.jsonl
+                                     ↓
+                    ┌────────────────┴────────────────┐
+                    │                                 │
+              forensics.py                       pattern.py
+          (root cause detection)          (time series + anomaly)
+                    │                                 │
+              versioning.py                      baseline.py
+           (v1 vs v2 comparison)            (7-day moving average)
+                    │                                 │
+               ab_report.py                      alerts.py
+            (A/B comparison table)          (Slack / console alert)
+                    └────────────────┬────────────────┘
+                                     ↓
+                              dashboard.py
+                         (ASCII terminal dashboard)
+```
+
+---
+
+## 📁 Project Structure
+
+```
+failure-forensics/
+├── src/
+│   ├── logger.py          # Logs every pipeline step to JSONL
+│   ├── forensics.py       # Root cause detection (5 categories)
+│   ├── pattern.py         # Time-series failure rate + anomaly detection
+│   ├── baseline.py        # 7-day moving average + trend (IMPROVING/STABLE/DEGRADING)
+│   ├── alerts.py          # Slack webhook + console alerts
+│   ├── versioning.py      # Per-version failure rate stats
+│   ├── ab_report.py       # A/B comparison report (table + JSON)
+│   └── dashboard.py       # ASCII bar chart terminal dashboard
+├── data/
+│   └── logs/
+│       └── requests.jsonl # All pipeline logs (gitignored)
+├── tests/
+│   └── test_forensics.py  # 8 unit tests
+├── config.py              # Thresholds, Slack URL, step limits
+├── main.py                # 5-scenario demo runner
+├── simulate.py            # Realistic test data generator (100 runs, anomaly day)
+└── requirements.txt
+```
+
+---
+
+## 🚀 Getting Started
+
+### 1. Clone & Install
+
+```bash
+git clone https://github.com/jasstt/failure-forensics.git
+cd failure-forensics
+pip install -r requirements.txt
+```
+
+### 2. (Optional) Configure Slack Alerts
+
+Edit `config.py`:
+
+```python
+SLACK_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
+```
+
+If left empty, all alerts print to the console.
+
+### 3. Run the Full Demo
+
+```bash
+python main.py
+```
+
+This runs 5 scenarios:
+1. **Simulation** — generates 100 realistic pipeline runs (2 prompt versions, anomaly day)
+2. **Root cause analysis** — detects the failing step and assigns a category
+3. **7-day pattern report** — failure rate per day + step breakdown + anomaly check
+4. **A/B report** — `prompt_v1` vs `prompt_v2` with per-step improvement table
+5. **Terminal dashboard** — live ASCII bar charts, trend, top 5 failed runs
+
+### 4. Run Unit Tests
+
+```bash
+python tests/test_forensics.py
+python tests/test_advanced.py
+```
+
+---
+
+## 🚀 Advanced Features (New in v2)
+
+| Katman | Özellik | Teknoloji |
+|--------|---------|-----------|
+| 1 | Otomatik öneri motoru | Kural tabanlı |
+| 2 | AI destekli hata analizi | Gemini 2.5 Pro |
+| 3 | Eval seti otomatik büyütme | Frequency analysis |
+| 4 | Prompt optimizasyon açıklaması | Gemini 2.5 Pro |
+| 5 | Regression guard | Baseline comparison |
+
+**Senaryo 6: Regression Guard**
+Yeni bir prompt (`v3`) deploy edilmeden önce otomatik regresyon kontrolü yapar:
+```
+REGRESSION CHECK — v3
+Baseline (v2): 11.0% failure rate
+Yeni (v3):     24.5% failure rate
+Delta: +13.5pp → REGRESSION_DETECTED ❌
+```
+
+## Test Results
+
+| Katman | Test | Sonuç |
+|--------|------|-------|
+| 1 — Recommender | Kategori → öneri mapping | ✅ PASS |
+| 2 — LLM Analyzer | Gemini fallback | ✅ PASS |
+| 3 — Eval Collector | Duplicate prevention | ✅ PASS |
+| 4 — Prompt Optimizer | A/B açıklama (v2: +10pp) | ✅ PASS |
+| 5 — Regression Guard | DETECTED + PASS senaryoları | ✅ PASS |
+
+### Key Results
+- A/B: prompt_v2, v1'e göre 10pp iyileşme
+- Regression Guard: v3 deploy'u +6pp delta ile WARNING olarak engelledi
+- Eval Collector: 5 yeni eval adayı otomatik toplandı
+- LLM Analyzer: Gemini kapalıyken kural tabanlına sorunsuz fallback
+
+---
+
+## 📊 Results
+
+| Feature | Result |
+|---------|--------|
+| Unit Tests | **8/8 PASS** ✅ |
+| Root cause categories | **5 types** (RETRIEVAL_QUALITY, RERANKER_FAILURE, LLM_HALLUCINATION, CITATION_MISS, API_ERROR) |
+| Anomaly detection | **20% delta threshold** — flags when today's rate exceeds 7-day average by >20pp |
+| A/B comparison | **v2: 11.5pp improvement** over v1 (22.5% → 11.0% failure rate) |
+| Trend analysis | **IMPROVING / STABLE / DEGRADING** based on 7-day moving average |
+| Slack integration | **Webhook ready** — fires on rate threshold, anomaly, or 3 consecutive failures |
+
+---
+
+## ⚙️ Configuration (`config.py`)
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `FAILURE_RATE_THRESHOLD` | `0.25` | Alert fires above this failure rate |
+| `ANOMALY_THRESHOLD` | `0.20` | Flag if today exceeds 7-day avg by this delta |
+| `SLACK_WEBHOOK_URL` | `""` | Empty = console output |
+| `CONSECUTIVE_FAILURE_THRESHOLD` | `3` | Alert after N consecutive step failures |
+| `STEP_THRESHOLDS` | see config | Per-step max acceptable failure rate |
+
+---
+
+## 🧪 Root Cause Categories
+
+| Category | Trigger |
+|----------|---------|
+| `RETRIEVAL_QUALITY` | Retrieval step fails — no results, low score |
+| `RERANKER_FAILURE` | Reranker can't parse LLM response or times out |
+| `LLM_HALLUCINATION` | Generation returns empty or uncited response |
+| `CITATION_MISS` | Answer produced but no source citations found |
+| `API_ERROR` | Timeout, 429 rate limit, 503 service unavailable |
+
+---
+
+## 📈 Terminal Dashboard (Sample Output)
+
+```
+═════════════════════════════════════════════════════════════
+  🔬  FAILURE FORENSICS — Terminal Dashboard
+═════════════════════════════════════════════════════════════
+
+  📅 SON 7 GÜNÜN FAILURE RATE GRAFİĞİ
+  2026-06-03  [███░░░░░░░░░░░░░░░░░░░░░░░░░░░] 13.0%
+  2026-06-07  [████████░░░░░░░░░░░░░░░░░░░░░░] 27.3% ⚠️
+  2026-06-10  [███░░░░░░░░░░░░░░░░░░░░░░░░░░░] 12.0%
+
+  🔍 ADIM BAZINDA HATA DAĞILIMI
+  retrieval     [███████░░░░░░░░░░░░░] 38.0%  (38/100 hatalı)
+  reranking     [██░░░░░░░░░░░░░░░░░░] 13.0%  (13/100 hatalı)
+  generation    [██░░░░░░░░░░░░░░░░░░] 10.0%  (10/100 hatalı)
+  citation      [█░░░░░░░░░░░░░░░░░░░]  6.0%  (6/100 hatalı)
+
+  ⚡ ANOMALİ: ✅ Normal: Bugün (12.0%) ≈ 7g ort. (16.2%)
+  📊 TREND: ➡️  STABLE — Hareketli Ort: 16.0%
+```
+
+---
+
+## 🛠 Technologies Used
+
+* **Python standard library** — `json`, `collections`, `datetime`, `threading`
+* **[requests](https://pypi.org/project/requests/)** — Slack webhook HTTP calls
+* **[python-dotenv](https://pypi.org/project/python-dotenv/)** — Environment variable management
+
+No heavy dependencies. No cloud. No API keys required.
+
+---
+
+## 🔭 Roadmap
+
+- [ ] FastAPI REST endpoint for remote log ingestion
+- [ ] HTML report export
+- [ ] PostgreSQL backend for large-scale log storage
+- [ ] Multi-pipeline support (compare RAG vs fine-tuned model)
+- [ ] Email alerts as alternative to Slack

failure_forensics-0.1.1/data/ab_report.json

@@ -0,0 +1,133 @@
+{
+  "summary": {
+    "v1": "v1",
+    "v2": "v2",
+    "winner": "v2",
+    "v1_failure_rate": 0.25,
+    "v2_failure_rate": 0.15,
+    "improvement_pct": 10.0,
+    "v1_runs": 50,
+    "v2_runs": 50
+  },
+  "step_breakdown": {
+    "reranking": {
+      "v1": {
+        "total": 50,
+        "failed": 13,
+        "rate": 0.26
+      },
+      "v2": {
+        "total": 50,
+        "failed": 8,
+        "rate": 0.16
+      },
+      "delta_pct": 10.0,
+      "step_winner": "v2"
+    },
+    "citation": {
+      "v1": {
+        "total": 50,
+        "failed": 3,
+        "rate": 0.06
+      },
+      "v2": {
+        "total": 50,
+        "failed": 6,
+        "rate": 0.12
+      },
+      "delta_pct": -6.0,
+      "step_winner": "v1"
+    },
+    "generation": {
+      "v1": {
+        "total": 50,
+        "failed": 15,
+        "rate": 0.3
+      },
+      "v2": {
+        "total": 50,
+        "failed": 2,
+        "rate": 0.04
+      },
+      "delta_pct": 26.0,
+      "step_winner": "v2"
+    },
+    "retrieval": {
+      "v1": {
+        "total": 50,
+        "failed": 19,
+        "rate": 0.38
+      },
+      "v2": {
+        "total": 50,
+        "failed": 14,
+        "rate": 0.28
+      },
+      "delta_pct": 10.0,
+      "step_winner": "v2"
+    }
+  },
+  "raw_stats": {
+    "v1": {
+      "total_steps": 200,
+      "failed_steps": 50,
+      "failure_rate": 0.25,
+      "runs": 50,
+      "steps": {
+        "retrieval": {
+          "total": 50,
+          "failed": 19,
+          "rate": 0.38
+        },
+        "reranking": {
+          "total": 50,
+          "failed": 13,
+          "rate": 0.26
+        },
+        "generation": {
+          "total": 50,
+          "failed": 15,
+          "rate": 0.3
+        },
+        "citation": {
+          "total": 50,
+          "failed": 3,
+          "rate": 0.06
+        }
+      }
+    },
+    "v2": {
+      "total_steps": 200,
+      "failed_steps": 30,
+      "failure_rate": 0.15,
+      "runs": 50,
+      "steps": {
+        "retrieval": {
+          "total": 50,
+          "failed": 14,
+          "rate": 0.28
+        },
+        "reranking": {
+          "total": 50,
+          "failed": 8,
+          "rate": 0.16
+        },
+        "generation": {
+          "total": 50,
+          "failed": 2,
+          "rate": 0.04
+        },
+        "citation": {
+          "total": 50,
+          "failed": 6,
+          "rate": 0.12
+        }
+      }
+    }
+  },
+  "improvement_reason": {
+    "error": "Gemini not available",
+    "improvements": [],
+    "next_iteration": "Manuel inceleme gerekiyor."
+  }
+}

failure_forensics-0.1.1/data/baseline.json

@@ -0,0 +1,10 @@
+{
+  "2026-01-01": 0.1,
+  "2026-01-02": 0.15,
+  "2026-01-03": 0.2,
+  "2026-01-04": 0.25,
+  "2026-01-05": 0.3,
+  "2026-01-06": 0.35,
+  "2026-01-07": 0.4,
+  "2026-06-11": 0.2
+}

failure_forensics-0.1.1/data/eval_candidates/candidates.jsonl

@@ -0,0 +1 @@
+{"query": "Aynı sorgu iki kere patladı", "failure_count": 3, "root_cause_category": "RETRIEVAL_QUALITY", "first_seen": "2026-06-11T01:53:53.961023+00:00", "last_seen": "2026-06-11T01:53:53.963734+00:00", "auto_add_eligible": true}