loglensai 0.3.0__tar.gz

loglensai-0.3.0/.github/workflows/ci.yml

@@ -0,0 +1,30 @@
+name: CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: |
+          pip install -e ".[dev]"
+
+      - name: Run tests
+        run: pytest tests/ -v
+
+      - name: Accuracy gate (Stage 4 benchmark)
+        run: |
+          printf -- '- INFO node boot ok\n- INFO heartbeat\nKERN FATAL kernel panic\nKERN ERROR disk failure timeout\n%.0s- INFO ok\n' {1..30} > /tmp/bgl_ci.log
+          loglens benchmark /tmp/bgl_ci.log --min-f1 0.30

loglensai-0.3.0/.gitignore

@@ -0,0 +1,37 @@
+# Python
+__pycache__/
+*.py[cod]
+*.pyo
+.pytest_cache/
+*.egg-info/
+dist/
+build/
+.eggs/
+
+# Virtual env
+venv/
+.venv/
+env/
+data/
+# Test fixtures (generated files — too large for git)
+tests/fixtures/large_sample.log
+tests/fixtures/large_sample_labels.json
+tests/fixtures/demo_incident.log
+tests/fixtures/sample.log
+
+# Auto-learned synonyms (runtime generated)
+.loglens_synonyms.json
+
+# Env / secrets
+.env
+*.env
+
+# IDE
+.vscode/
+.idea/
+
+# OS
+.DS_Store
+Thumbs.db
+
+benchmarks/*.log

loglensai-0.3.0/BENCHMARK.md

@@ -0,0 +1,68 @@
+# LogLens AI — Benchmark Results
+
+Anomaly detection accuracy on **real-world production logs** from the
+[Loghub](https://github.com/logpai/loghub) collection. All numbers are
+reproducible with the commands below. Algorithms in `fast` / `turbo` modes are
+written from scratch (no ML anomaly libraries); `deep` mode adds AI semantic
+embeddings.
+
+## Headline
+
+> On **500,000 lines** of real supercomputer logs (Loghub BGL), LogLens AI
+> caught **100% of anomalies (zero misses)** at up to **91.7% precision**
+> (F1 **0.957**). Applied **unchanged** to a different cluster (Thunderbird),
+> it held a **0.67% false-positive rate** — no retuning. Fully local, zero setup.
+
+## Accuracy — Loghub BGL (500,000 lines, 206,847 labeled alerts)
+
+| Mode  | Type          | Precision | Recall | F1    | Speed      | FN |
+|-------|---------------|-----------|--------|-------|------------|----|
+| fast  | from-scratch  | 0.901     | 1.000  | 0.948 | 6,691 l/s  | 0  |
+| turbo | from-scratch  | 0.901     | 1.000  | 0.948 | 7,282 l/s  | 0  |
+| deep  | AI embeddings | **0.917** | 1.000  | **0.957** | 3,351 l/s | 0 |
+
+- **Zero false negatives** across all 206,847 alerts in every mode.
+- **Deep (AI) mode** cuts false positives 22,810 -> 18,624, proving semantic
+  embeddings add real precision over the keyword/statistical baseline.
+- **Turbo** matches fast accuracy exactly at higher throughput.
+
+## Generality — Loghub Thunderbird (500,000 lines, all-normal slice)
+
+Threshold tuned on BGL, applied **unchanged** to a different system's logs.
+On all-normal data every flag is a false positive, so this measures specificity.
+
+| Mode | Threshold (BGL-tuned) | False-flag rate | Specificity | Speed     |
+|------|-----------------------|-----------------|-------------|-----------|
+| fast | 0.82                  | 0.68% (3,418)   | 99.32%      | 8,590 l/s |
+| deep | 0.8339                | 0.67% (3,343)   | 99.33%      | 1,836 l/s |
+
+The score ordering generalizes across architectures with **no retuning** —
+strong evidence the detector is not overfit to a single dataset.
+
+## Reproduce
+
+Download datasets from https://github.com/logpai/loghub (BGL, Thunderbird).
+
+```bash
+# Accuracy on BGL (fast / turbo / deep)
+python benchmark_labeled.py --file data/bgl/BGL.log --limit 500000 --sweep --threshold 0.8339 --mode fast
+python benchmark_labeled.py --file data/bgl/BGL.log --limit 500000 --sweep --threshold 0.8339 --mode turbo
+python benchmark_labeled.py --file data/bgl/BGL.log --limit 500000 --sweep --threshold 0.8339 --mode deep
+
+# Slice a manageable chunk from the 31GB Thunderbird file
+head -n 500000 data/tbird/Thunderbird.log > data/tbird/Thunderbird_500k.log
+
+# Cross-dataset generality (unchanged threshold)
+python benchmark_labeled.py --file data/tbird/Thunderbird_500k.log --limit 500000 --threshold 0.8339 --mode deep
+```
+
+## Notes on honesty
+
+- Label convention: first whitespace token `-` = normal, anything else = alert
+  (Loghub line-level convention).
+- `deep` mode runs the transformer on **unique templates only**
+  (`embed_templates()`), a legitimate optimization that keeps it fast.
+- Recommended default operating threshold: **0.8339** (F1-optimal on BGL,
+  generalizes to Thunderbird).
+- Thunderbird's first 500K lines are all-normal, so that run validates
+  false-positive control, not recall, on Thunderbird.