syslog-postmortem 1.0.0__tar.gz

syslog_postmortem-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Serber1990
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

syslog_postmortem-1.0.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: syslog-postmortem
+Version: 1.0.0
+Summary: Generate structured postmortem drafts from journalctl, syslog, auth.log and dmesg
+Author-email: Serber1990 <serber1990@pm.me>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/serber1990/syslog-postmortem
+Project-URL: Bug Tracker, https://github.com/serber1990/syslog-postmortem/issues
+Project-URL: Source Code, https://github.com/serber1990/syslog-postmortem
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Environment :: Console
+Classifier: Topic :: System :: Logging
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: shellcolorize
+Dynamic: license-file
+
+# syslog-postmortem
+
+[![PyPI version](https://badge.fury.io/py/syslog-postmortem.svg)](https://badge.fury.io/py/syslog-postmortem)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+
+Generate a **structured postmortem draft** from system logs in seconds — no more reconstructing incident timelines by hand.
+
+Pulls from `journalctl`, `/var/log/syslog`, `/var/log/auth.log` and `/var/log/dmesg`, correlates events, detects patterns, and produces a ready-to-edit Markdown or HTML postmortem.
+
+---
+
+## ✨ What it does
+
+Given a time window and optional service list, `syslog-postmortem`:
+
+1. **Collects** entries from `journalctl` (primary) and `/var/log` files (supplement)
+2. **Deduplicates** repeated messages and counts occurrences
+3. **Detects patterns** — OOM kills, service crashes, auth bursts, disk exhaustion, cascading failures, connection errors
+4. **Builds a timeline** sorted chronologically with severity icons
+5. **Identifies contributing factors** automatically (restart loops, cascade chains, error bursts)
+6. **Generates action items** from pattern hints
+7. **Outputs** a complete Markdown postmortem ready to edit and share
+
+---
+
+## 📥 Installation
+
+```bash
+pip install syslog-postmortem
+```
+
+---
+
+## 🛠 Usage
+
+```bash
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00"
+```
+
+```bash
+# Focus on specific services
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" \
+           --services nginx,postgresql,redis \
+           --title "Database outage" \
+           --output incident-2026-05-10.md
+
+# HTML output
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" \
+           --format html --output report.html
+
+# journalctl only (skip /var/log files)
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" --no-files
+```
+
+---
+
+## 📋 Options
+
+| Option | Description |
+|--------|-------------|
+| `--from DATETIME` | Start of incident window **required** |
+| `--to DATETIME` | End of incident window **required** |
+| `--title TEXT` | Postmortem title (default: `Incident YYYY-MM-DD`) |
+| `--services LIST` | Comma-separated services to focus on |
+| `--output FILE` | Output path (default: `postmortem_YYYYMMDD_HHMM.md`) |
+| `--format` | `markdown` (default) or `html` |
+| `--no-files` | Skip `/var/log` parsing, use journalctl only |
+| `--priorities` | journalctl priority filter (default: `0..4`) |
+
+---
+
+## 📄 Output structure
+
+```markdown
+# Postmortem: Database outage
+
+| | |
+|---|---|
+| **Date** | 2026-05-10 |
+| **Window** | 2026-05-10 14:00 → 2026-05-10 16:00 |
+| **Duration** | 2h 0m |
+| **Severity** | Critical |
+
+## Summary
+Analysis of 1,243 raw log entries (89 unique events after deduplication)...
+First anomaly detected at **14:03:22** in **postgresql** (CRITICAL).
+
+## Timeline
+| Time | Service | Severity | Event |
+|------|---------|----------|-------|
+| 14:03:22 | `postgresql` | ⛔ CRITICAL | could not connect to server |
+| 14:03:45 | `nginx` | 🔴 ERROR | upstream connect error |
+
+## Contributing Factors
+- **Service instability**: `postgresql` triggered restart-loop detection 4 time(s)
+- **Cascading failure**: `nginx` errors began 23s after first `postgresql` critical event
+
+## Action Items
+- [ ] Investigate postgresql restart cause; check dependencies and configuration
+- [ ] Verify the downstream service is running and listening on the expected port
+```
+
+---
+
+## 🔍 Detected patterns
+
+| Pattern | Triggers |
+|---------|----------|
+| OOM Killer | `out of memory`, `oom_kill`, `killed process` |
+| Disk Full | `no space left on device`, `disk full` |
+| Kernel Error | `kernel: BUG`, `segfault`, `general protection` |
+| Service Failed | `failed to start`, `unit entered failed state` |
+| Process Crash | `segmentation fault`, `core dump`, `aborted` |
+| Connection Refused | `connection refused`, `upstream connect error` |
+| Timeout | `timed out`, `ETIMEDOUT`, `request timeout` |
+| Auth Failure | `Failed password`, `authentication failure`, `invalid user` |
+| SSL/TLS Error | `certificate expired`, `TLS handshake failed` |
+| Database Error | `could not connect to database`, `max connections reached` |
+| High Load | `load average` spike, `cpu throttled` |
+
+---
+
+## 📝 License
+
+MIT — see [LICENSE](LICENSE).
+
+## 🌐 Connect
+
+[![GitHub](https://img.shields.io/badge/GitHub-@serber1990-181717?style=flat-square&logo=github)](https://github.com/serber1990)

syslog_postmortem-1.0.0/README.md

@@ -0,0 +1,129 @@
+# syslog-postmortem
+
+[![PyPI version](https://badge.fury.io/py/syslog-postmortem.svg)](https://badge.fury.io/py/syslog-postmortem)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+
+Generate a **structured postmortem draft** from system logs in seconds — no more reconstructing incident timelines by hand.
+
+Pulls from `journalctl`, `/var/log/syslog`, `/var/log/auth.log` and `/var/log/dmesg`, correlates events, detects patterns, and produces a ready-to-edit Markdown or HTML postmortem.
+
+---
+
+## ✨ What it does
+
+Given a time window and optional service list, `syslog-postmortem`:
+
+1. **Collects** entries from `journalctl` (primary) and `/var/log` files (supplement)
+2. **Deduplicates** repeated messages and counts occurrences
+3. **Detects patterns** — OOM kills, service crashes, auth bursts, disk exhaustion, cascading failures, connection errors
+4. **Builds a timeline** sorted chronologically with severity icons
+5. **Identifies contributing factors** automatically (restart loops, cascade chains, error bursts)
+6. **Generates action items** from pattern hints
+7. **Outputs** a complete Markdown postmortem ready to edit and share
+
+---
+
+## 📥 Installation
+
+```bash
+pip install syslog-postmortem
+```
+
+---
+
+## 🛠 Usage
+
+```bash
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00"
+```
+
+```bash
+# Focus on specific services
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" \
+           --services nginx,postgresql,redis \
+           --title "Database outage" \
+           --output incident-2026-05-10.md
+
+# HTML output
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" \
+           --format html --output report.html
+
+# journalctl only (skip /var/log files)
+postmortem --from "2026-05-10 14:00" --to "2026-05-10 16:00" --no-files
+```
+
+---
+
+## 📋 Options
+
+| Option | Description |
+|--------|-------------|
+| `--from DATETIME` | Start of incident window **required** |
+| `--to DATETIME` | End of incident window **required** |
+| `--title TEXT` | Postmortem title (default: `Incident YYYY-MM-DD`) |
+| `--services LIST` | Comma-separated services to focus on |
+| `--output FILE` | Output path (default: `postmortem_YYYYMMDD_HHMM.md`) |
+| `--format` | `markdown` (default) or `html` |
+| `--no-files` | Skip `/var/log` parsing, use journalctl only |
+| `--priorities` | journalctl priority filter (default: `0..4`) |
+
+---
+
+## 📄 Output structure
+
+```markdown
+# Postmortem: Database outage
+
+| | |
+|---|---|
+| **Date** | 2026-05-10 |
+| **Window** | 2026-05-10 14:00 → 2026-05-10 16:00 |
+| **Duration** | 2h 0m |
+| **Severity** | Critical |
+
+## Summary
+Analysis of 1,243 raw log entries (89 unique events after deduplication)...
+First anomaly detected at **14:03:22** in **postgresql** (CRITICAL).
+
+## Timeline
+| Time | Service | Severity | Event |
+|------|---------|----------|-------|
+| 14:03:22 | `postgresql` | ⛔ CRITICAL | could not connect to server |
+| 14:03:45 | `nginx` | 🔴 ERROR | upstream connect error |
+
+## Contributing Factors
+- **Service instability**: `postgresql` triggered restart-loop detection 4 time(s)
+- **Cascading failure**: `nginx` errors began 23s after first `postgresql` critical event
+
+## Action Items
+- [ ] Investigate postgresql restart cause; check dependencies and configuration
+- [ ] Verify the downstream service is running and listening on the expected port
+```
+
+---
+
+## 🔍 Detected patterns
+
+| Pattern | Triggers |
+|---------|----------|
+| OOM Killer | `out of memory`, `oom_kill`, `killed process` |
+| Disk Full | `no space left on device`, `disk full` |
+| Kernel Error | `kernel: BUG`, `segfault`, `general protection` |
+| Service Failed | `failed to start`, `unit entered failed state` |
+| Process Crash | `segmentation fault`, `core dump`, `aborted` |
+| Connection Refused | `connection refused`, `upstream connect error` |
+| Timeout | `timed out`, `ETIMEDOUT`, `request timeout` |
+| Auth Failure | `Failed password`, `authentication failure`, `invalid user` |
+| SSL/TLS Error | `certificate expired`, `TLS handshake failed` |
+| Database Error | `could not connect to database`, `max connections reached` |
+| High Load | `load average` spike, `cpu throttled` |
+
+---
+
+## 📝 License
+
+MIT — see [LICENSE](LICENSE).
+
+## 🌐 Connect
+
+[![GitHub](https://img.shields.io/badge/GitHub-@serber1990-181717?style=flat-square&logo=github)](https://github.com/serber1990)

syslog_postmortem-1.0.0/pyproject.toml

@@ -0,0 +1,32 @@
+[build-system]
+requires = ["setuptools>=61", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "syslog-postmortem"
+version = "1.0.0"
+description = "Generate structured postmortem drafts from journalctl, syslog, auth.log and dmesg"
+readme = "README.md"
+license = "MIT"
+authors = [{ name = "Serber1990", email = "serber1990@pm.me" }]
+requires-python = ">=3.8"
+dependencies = ["shellcolorize"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: POSIX :: Linux",
+    "Environment :: Console",
+    "Topic :: System :: Logging",
+    "Topic :: System :: Systems Administration",
+]
+
+[project.urls]
+Homepage     = "https://github.com/serber1990/syslog-postmortem"
+"Bug Tracker" = "https://github.com/serber1990/syslog-postmortem/issues"
+"Source Code" = "https://github.com/serber1990/syslog-postmortem"
+
+[project.scripts]
+postmortem = "syslog_postmortem.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["syslog_postmortem*"]

syslog_postmortem-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

syslog_postmortem-1.0.0/syslog_postmortem/__init__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+__version__ = "1.0.0"
+__all__ = ["main"]

syslog_postmortem-1.0.0/syslog_postmortem/analyzer.py

@@ -0,0 +1,231 @@
+"""
+Pattern analysis, deduplication, cascade detection and contributing factor generation.
+"""
+from collections import Counter, defaultdict
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta
+from typing import List, Dict, Optional
+
+from .collector import RawEntry, priority_to_severity
+from .patterns import match_patterns, Pattern
+
+
+# ── Normalised event ──────────────────────────────────────────────────────────
+
+@dataclass
+class Event:
+    timestamp: datetime
+    service: str
+    severity: str           # CRITICAL | ERROR | WARNING | INFO
+    message: str
+    source: str
+    patterns: List[Pattern] = field(default_factory=list)
+    count: int = 1          # after deduplication
+
+
+# ── Analysis result ───────────────────────────────────────────────────────────
+
+@dataclass
+class AnalysisResult:
+    events: List[Event]
+    timeline: List[Event]                           # deduplicated, sorted
+    by_severity: Dict[str, List[Event]]
+    by_service: Dict[str, List[Event]]
+    pattern_counts: Dict[str, int]                  # pattern_name → count
+    contributing_factors: List[str]
+    action_items: List[str]
+    first_anomaly: Optional[Event]
+    peak_window: Optional[datetime]                 # minute with most errors
+    services_affected: List[str]
+    total_raw: int
+
+
+# ── Helpers ───────────────────────────────────────────────────────────────────
+
+def _deduplicate(events: List[Event], window_seconds: int = 60) -> List[Event]:
+    """
+    Merge identical messages that repeat within `window_seconds`.
+    Keeps the first occurrence with a count of how many times it appeared.
+    """
+    result: List[Event] = []
+    seen: Dict[str, Event] = {}
+
+    for ev in events:
+        key = (ev.service, ev.message[:120])
+        if key in seen:
+            last = seen[key]
+            if (ev.timestamp - last.timestamp).total_seconds() <= window_seconds:
+                last.count += 1
+                continue
+        seen[key] = ev
+        result.append(ev)
+
+    return result
+
+
+def _peak_minute(events: List[Event]) -> Optional[datetime]:
+    buckets: Counter = Counter()
+    for ev in events:
+        if ev.severity in ('CRITICAL', 'ERROR', 'WARNING'):
+            buckets[ev.timestamp.replace(second=0, microsecond=0)] += 1
+    return buckets.most_common(1)[0][0] if buckets else None
+
+
+def _detect_cascades(events: List[Event], window_seconds: int = 120) -> List[str]:
+    """
+    Detect cascading failures: service A fails, service B fails shortly after.
+    Returns human-readable descriptions.
+    """
+    cascades = []
+    criticals = [e for e in events if e.severity == 'CRITICAL']
+    errors = [e for e in events if e.severity == 'ERROR']
+
+    for trigger in criticals:
+        for follow in errors:
+            if follow.service == trigger.service:
+                continue
+            delta = (follow.timestamp - trigger.timestamp).total_seconds()
+            if 0 < delta <= window_seconds:
+                cascades.append(
+                    f"**{follow.service}** errors began "
+                    f"**{int(delta)}s** after first **{trigger.service}** critical event "
+                    f"({trigger.timestamp.strftime('%H:%M:%S')})"
+                )
+                break  # one cascade per trigger service pair
+
+    return cascades
+
+
+# ── Main analyser ─────────────────────────────────────────────────────────────
+
+def analyze(raw: List[RawEntry], services_filter: List[str] = None) -> AnalysisResult:
+    total_raw = len(raw)
+
+    # 1. Normalise RawEntry → Event and run pattern matching
+    events: List[Event] = []
+    for r in raw:
+        if services_filter and r.service.lower() not in [s.lower() for s in services_filter]:
+            continue
+        severity = priority_to_severity(r.priority)
+        matched = match_patterns(r.message)
+        # Upgrade severity based on pattern if stricter
+        sev_rank = {'CRITICAL': 0, 'ERROR': 1, 'WARNING': 2, 'INFO': 3, 'DEBUG': 4}
+        for p in matched:
+            if sev_rank.get(p.severity, 4) < sev_rank.get(severity, 4):
+                severity = p.severity
+        events.append(Event(
+            timestamp=r.timestamp,
+            service=r.service,
+            severity=severity,
+            message=r.message,
+            source=r.source,
+            patterns=matched,
+        ))
+
+    # 2. Deduplicate
+    timeline = _deduplicate(events)
+
+    # 3. Group by severity and service
+    by_severity: Dict[str, List[Event]] = defaultdict(list)
+    by_service:  Dict[str, List[Event]] = defaultdict(list)
+    for ev in timeline:
+        by_severity[ev.severity].append(ev)
+        by_service[ev.service].append(ev)
+
+    # 4. Pattern counts
+    pattern_counts: Counter = Counter()
+    for ev in timeline:
+        for p in ev.patterns:
+            pattern_counts[p.name] += ev.count
+
+    # 5. First anomaly (first CRITICAL or ERROR)
+    first_anomaly = next(
+        (e for e in timeline if e.severity in ('CRITICAL', 'ERROR')), None
+    )
+
+    # 6. Peak minute
+    peak_window = _peak_minute(timeline)
+
+    # 7. Services affected (those with at least one ERROR+)
+    services_affected = sorted({
+        e.service for e in timeline
+        if e.severity in ('CRITICAL', 'ERROR')
+    })
+
+    # 8. Contributing factors
+    factors: List[str] = []
+
+    # Restart loops
+    restart_counts = Counter(
+        e.service for e in timeline
+        if any(p.name == 'service_restart' for p in e.patterns)
+    )
+    for svc, cnt in restart_counts.most_common():
+        factors.append(
+            f"**Service instability**: `{svc}` triggered restart-loop detection **{cnt} time(s)**"
+        )
+
+    # OOM events
+    oom_count = pattern_counts.get('oom', 0)
+    if oom_count:
+        factors.append(
+            f"**Memory pressure**: OOM killer fired **{oom_count} time(s)** during the window"
+        )
+
+    # Disk full
+    if pattern_counts.get('disk_full', 0):
+        factors.append("**Disk exhaustion**: 'No space left on device' events detected")
+
+    # Auth failure burst (>5 in window)
+    auth_count = sum(
+        e.count for e in timeline
+        if any(p.name == 'auth_failure' for p in e.patterns)
+    )
+    if auth_count >= 5:
+        factors.append(
+            f"**Auth anomaly**: Burst of **{auth_count} authentication failure(s)** detected"
+            " — possible brute-force attempt"
+        )
+
+    # Cascading failures
+    factors.extend(_detect_cascades(timeline))
+
+    # High error density (>30 error events in any 5-minute window)
+    buckets: Counter = Counter()
+    for ev in timeline:
+        if ev.severity in ('CRITICAL', 'ERROR'):
+            slot = ev.timestamp.replace(second=0, microsecond=0)
+            slot = slot.replace(minute=(slot.minute // 5) * 5)
+            buckets[slot] += ev.count
+    dense = [(t, c) for t, c in buckets.items() if c >= 30]
+    for t, c in sorted(dense):
+        factors.append(
+            f"**Error burst**: **{c} errors** in the 5-minute window starting "
+            f"{t.strftime('%H:%M')}"
+        )
+
+    if not factors:
+        factors.append("No specific contributing factors auto-detected — manual analysis required")
+
+    # 9. Action items (deduplicated from pattern hints)
+    seen_hints: set = set()
+    action_items: List[str] = []
+    for ev in timeline:
+        for p in ev.patterns:
+            if p.action_hint not in seen_hints:
+                seen_hints.add(p.action_hint)
+                action_items.append(p.action_hint)
+
+    return AnalysisResult(
+        events=events,
+        timeline=timeline,
+        by_severity=dict(by_severity),
+        by_service=dict(by_service),
+        pattern_counts=dict(pattern_counts),
+        contributing_factors=factors,
+        action_items=action_items,
+        first_anomaly=first_anomaly,
+        peak_window=peak_window,
+        services_affected=services_affected,
+        total_raw=total_raw,
+    )