logometer 0.1.0__tar.gz

logometer-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 AmanSg098
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

logometer-0.1.0/PKG-INFO

@@ -0,0 +1,294 @@
+Metadata-Version: 2.4
+Name: logometer
+Version: 0.1.0
+Summary: Tail your logs. Catch anomalies. Skip the 2am grep.
+Author: AmanSg098
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/AmanSg098/logometer
+Project-URL: Issues, https://github.com/AmanSg098/logometer/issues
+Keywords: logs,logging,anomaly-detection,monitoring,cli,tail
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: POSIX
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: System :: Logging
+Classifier: Topic :: System :: Monitoring
+Classifier: Topic :: Utilities
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: pretty
+Requires-Dist: rich>=13; extra == "pretty"
+Dynamic: license-file
+
+# logometer
+
+**Tail your logs. Catch anomalies. Skip the 2am grep.**
+
+`logometer` watches a log file (or stdin), buckets lines into time windows, and flags windows that look statistically off — an error spike, or an error type that's never shown up before. It's silent when things are fine. No AI required for the core detection; point it at an LLM with `--explain` if you want a plain-English guess at *why* a window looks weird.
+
+Zero dependencies for the core tool — just Python 3.10+.
+
+<!-- TODO: record a short asciinema/GIF demo and drop it here before publishing.
+     Suggested recording: `logometer tail examples/sample.log --replay` -->
+
+## Install
+
+As a standalone command, without cloning:
+
+```bash
+pipx install git+https://github.com/AmanSg098/logometer.git
+```
+
+Or from a clone, for development:
+
+```bash
+git clone https://github.com/AmanSg098/logometer.git
+cd logometer
+pip install -e .              # core tool, no dependencies
+pip install -e ".[pretty]"    # optional: styled output via rich
+```
+
+## Quick start
+
+Try it on the included sample log (has a normal-traffic baseline plus an injected error burst):
+
+```bash
+python3 -m logometer.cli tail examples/sample.log --replay
+```
+
+or, once installed:
+
+```bash
+logometer tail examples/sample.log --replay
+```
+
+## Usage
+
+```bash
+# Tail a live log file (like tail -f, but with anomaly detection)
+logometer tail /var/log/app.log
+
+# Pipe from stdin
+tail -f app.log | logometer tail -
+
+# Replay a static file from start to finish, then exit
+logometer tail app.log --replay
+
+# Only show anomalous windows, hide the "ok" noise
+logometer tail app.log --replay --quiet
+
+# Adjust window size (seconds) and how aggressively to flag deviations
+logometer tail app.log --window 30 --sensitivity high
+
+# Mute known-noisy lines so they never reach the baseline (repeatable regex)
+logometer tail app.log --ignore 'healthcheck' --ignore 'DeprecationWarning'
+
+# Ask an LLM for a one-sentence explanation of each anomaly
+export ANTHROPIC_API_KEY=your-key-here
+logometer tail app.log --explain
+
+# JSON output, one object per line — good for piping into other tools
+logometer tail app.log --format json
+```
+
+Stop a live tail with Ctrl-C (or `SIGTERM` from a process manager); the window in progress is reported before exiting.
+
+### All options
+
+| Option | Default | What it does |
+|---|---|---|
+| `file` | — | Log file to read, or `-` for stdin |
+| `--window SECONDS` | `10` | Window size, when timestamps are parseable |
+| `--sensitivity` | `medium` | `low`, `medium` or `high` — how far above the baseline a window must be to count as a spike |
+| `--format` | `plain` | `plain` or `json` |
+| `--replay` | off | Read the file start to end and exit, instead of following it live |
+| `--quiet` | off | Only print anomalous windows |
+| `--ignore REGEX` | — | Drop matching lines before analysis; repeatable |
+| `--explain` | off | Ask an LLM for a one-sentence cause of each anomaly |
+| `--explain-provider` | `anthropic` | `anthropic`, `openai` or `openrouter` |
+| `--explain-model MODEL` | see below | Override the model used by `--explain` |
+| `--no-pretty` | off | Use plain output even if `rich` is installed |
+| `--version` | — | Print the version and exit |
+
+### Supported timestamp formats
+
+Timestamps are read from each line to decide which window it belongs to. Two formats are recognised:
+
+- **ISO 8601** — `2026-09-17T12:03:10`, `2026-09-17 12:03:10.123Z`, and Python `logging`'s default `2026-09-17 12:03:10,123`
+- **Syslog** — `Sep 17 12:03:10` (syslog has no year, so the current year is assumed)
+
+The first 20 lines decide the mode: if at least half have a recognisable timestamp, lines are bucketed by time; otherwise every 50 lines form a window.
+
+### `--explain`
+
+Requires an API key for one of three providers, set in your environment:
+
+| `--explain-provider` | Key | Default model |
+|---|---|---|
+| `anthropic` (default) | `ANTHROPIC_API_KEY` | `claude-haiku-4-5-20251001` |
+| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
+| `openrouter` | `OPENROUTER_API_KEY` | `anthropic/claude-haiku-4.5` |
+
+[OpenRouter](https://openrouter.ai) gives one key access to models from many vendors; its model ids are prefixed with the vendor (`openai/gpt-4o-mini`, `google/gemini-2.5-flash`, …). No SDK install needed — it's a plain HTTPS call. If the key is missing or the request fails, `logometer` prints a one-line warning to stderr and keeps tailing normally; it never crashes because `--explain` had a bad day.
+
+Only the anomalous window's ERROR and WARN lines are sent (capped at 30 lines). Pick a different model with `--explain-model`:
+
+```bash
+export OPENAI_API_KEY=your-key-here
+logometer tail app.log --explain --explain-provider openai --explain-model gpt-4o
+
+export OPENROUTER_API_KEY=your-key-here
+logometer tail app.log --explain --explain-provider openrouter --explain-model google/gemini-2.5-flash
+```
+
+### `--ignore`
+
+Real logs usually have one or two messages that fire constantly and mean nothing. Left alone they dominate the error count, so the rolling baseline learns them as "normal" and a genuine problem has to shout louder to get noticed. `--ignore` drops matching lines before any analysis:
+
+```bash
+# on a real macOS install log, one benign repeating message accounted for
+# 97% of all "errors" — muting it cut the noise dramatically
+logometer tail /var/log/install.log --replay --window 60 --quiet \
+  --ignore 'installation-check'          # 27 anomalies -> 5, all genuine
+```
+
+Each `--ignore` takes a regular expression and can be repeated. An invalid pattern is reported as a normal CLI error rather than a crash.
+
+### Live tailing
+
+`logometer tail app.log` (without `--replay`) follows the file the way `tail -f` does, with two things a naive tail loop gets wrong:
+
+- **Windows close on time.** A window is emitted once its duration has elapsed, even if no further lines arrive. Without this, a service that errors and then dies would never report that final burst — the most important one — because nothing follows it to trigger the flush.
+- **Rotation is handled.** If the log is rotated out from under it (`logrotate`, or truncated in place with `copytruncate`), it notices and follows the new file instead of reading a now-orphaned file forever.
+
+Output is flushed as each window is reported, so piping or redirecting works in real time:
+
+```bash
+logometer tail app.log --quiet --format json >> alerts.jsonl
+tail -f app.log | logometer tail -
+```
+
+### Pretty output
+
+If you `pip install rich` (or `pip install -e ".[pretty]"`), terminal output automatically upgrades to styled panels. Not required — plain ANSI output works everywhere. When output is piped or redirected to a file it stays plain text either way; pass `--no-pretty` to get plain output in the terminal too.
+
+## Example output
+
+From the included sample log:
+
+```
+$ logometer tail examples/sample.log --replay --no-pretty
+
+  [12:01:20 – 12:01:30]  ok        errors: 0   baseline: ~0.1
+
+  [12:01:30 – 12:01:40]  ! ANOMALY  errors: 10  warns: 1  baseline: ~0.1   score: 9.9x
+    New error signature detected: '<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>'
+
+  [12:01:40 – 12:01:50]  ! ANOMALY  errors: 6  baseline: ~0.1   score: 5.9x
+    (error rate spike — no brand-new error signature)
+
+  [12:01:50 – 12:02:00]  ok        errors: 0   baseline: ~0.1
+  ...
+-- 18 window(s) processed, 4 anomaly(ies) flagged --
+```
+
+`score` is how many standard deviations the window's error count sits above the baseline. `<x>` marks the ids, numbers and timestamps stripped out when building an error signature.
+
+With `--explain`, each anomaly gets one more line:
+
+```
+  [12:01:30 – 12:01:40]  ! ANOMALY  errors: 10  warns: 1  baseline: ~0.1   score: 9.9x
+    New error signature detected: '<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>'
+    Explanation: Database or downstream service became unresponsive, causing clients to forcibly close connections due to timeouts or hangs.
+```
+
+With `--format json`, each window is one line with these fields:
+
+```json
+{"window_index": 9, "start": "12:01:30", "end": "12:01:40", "error_count": 10, "warn_count": 1, "baseline_mean": 0.111, "error_score": 9.889, "is_anomaly": true, "new_shapes": ["<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>"], "explanation": null}
+```
+
+The end-of-run summary line is omitted in JSON mode, so every line of output is valid JSON.
+
+## How the detection works
+
+<!-- Source: docs/flow.mmd. Regenerate after editing it with:
+     npx -p @mermaid-js/mermaid-cli mmdc -i docs/flow.mmd -o docs/flow.png -b white -s 2 -c docs/mermaid-config.json -->
+![How logometer processes a log: prepare each line, judge each window, report](https://raw.githubusercontent.com/AmanSg098/logometer/main/docs/flow.png)
+
+1. **Classify.** Each line is tagged by keyword: ERROR (`error`, `err`, `fatal`, `critical`, `exception`, `traceback`, `panic`), then WARN (`warn`, `warning`), INFO (`info`, `notice`), DEBUG (`debug`, `trace`). First match wins, so a line mentioning both an error and a warning counts as ERROR.
+2. **Fingerprint.** ERROR and WARN lines are reduced to a "shape": UUIDs, hex addresses, timestamps, quoted strings and numbers are replaced with `<x>`, so the same underlying error collapses to one shape regardless of the specific id.
+3. **Window.** Lines are batched into fixed-size windows — by time if timestamps are parseable, otherwise by a fixed line count.
+4. **Baseline.** The error counts of the last 20 windows give a rolling mean and standard deviation — what "normal" error volume looks like. Scoring starts once 3 windows of history exist.
+5. **Flag.** A window is anomalous if either:
+   - its error count is at least N standard deviations above the mean, where N is 3.0 for `--sensitivity low`, 2.0 for `medium` and 1.2 for `high`; or
+   - it contains an error *or warning* shape not seen earlier in this run (checked from the second window on).
+
+Two tuning choices keep this honest:
+
+- **Noise floor.** The standard deviation is never taken as less than 1.0. On a log that's normally error-free the real deviation is 0, and a single stray error would otherwise score as infinitely anomalous.
+- **Spikes don't train the baseline.** A window flagged as a spike isn't added to the history, so a long outage doesn't slowly teach the tool that a high error rate is normal.
+
+No ML model, no training step — deliberately simple and explainable.
+
+## Known limitations
+
+Worth knowing before you point this at something you care about. These are real, tested behaviours, not hypotheticals:
+
+- **Severity is keyword matching, not parsing.** A line counts as an error if it contains a word like `error`, `fatal` or `critical`. That means `Downloading 1 products: Critical []` — an *empty* list, i.e. good news — is counted as an error. Use `--ignore` to mute these.
+- **Python tracebacks are only partly seen.** `Traceback (most recent call last):` is detected, but the final `ValueError: ...` line is not, because the match looks for `error` as a standalone word. And since that first line is identical for every exception, new-signature detection can't tell one crash type from another. Logs that print their own level (`ERROR ValueError: ...`) work properly.
+- **JSON logs break new-signature detection.** Fingerprinting strips quoted strings, which in a JSON line removes the entire message — so unrelated errors collapse into one shape. Error *counting* still works. Extracting the message first works around it: `jq -r '.level + " " + .msg' app.log | logometer tail -`.
+- **Only the error count is scored.** A flood of identical *warnings* won't trigger a spike (though a never-seen-before warning shape will be flagged), and a sudden *drop* in traffic isn't detected either — only rises above the baseline are.
+- **Window labels show time, not date.** On a log spanning several days you'll see the same `[17:14:39 – 17:15:39]` label more than once.
+- **Timezone offsets are ignored.** `2026-09-20 19:05:04+05:30` is read as local wall-clock time; a log mixing offsets is bucketed as though they were the same clock.
+- **Nothing persists between runs.** The baseline and the set of known error shapes are rebuilt from scratch each start, so expect the first few windows of any run to over-flag. (Persistence is on the roadmap.)
+- **If timestamps can't be parsed** it silently falls back to fixed 50-line windows, and `--window` stops having any effect. You can tell from the labels: `[line 1 – line 50]` instead of a time range.
+- **`--explain` only sees ERROR and WARN lines.** Lines without a level keyword — like the body of a Python traceback — aren't sent, so the model can miss the root cause and give a vaguer answer.
+- **`--explain` sends log lines to a third party.** There's no redaction — don't use it on logs containing secrets or personal data.
+
+## Running the tests
+
+```bash
+python3 -m unittest discover -s tests -v
+```
+
+No network or API key is needed — the `--explain` tests mock the HTTP call.
+
+## Project layout
+
+```
+logometer/
+  cli.py         command-line entry point, file/stdin reading, output formatting
+  classifier.py  severity tagging and error-shape fingerprinting
+  timeparse.py   timestamp extraction (ISO 8601, syslog)
+  windower.py    groups lines into time- or count-based windows
+  baseline.py    rolling mean / standard deviation of errors per window
+  detector.py    decides whether a window is anomalous
+  explain.py     optional LLM explanations (Anthropic / OpenAI / OpenRouter over plain HTTPS)
+  pretty.py      optional rich-styled output
+tests/           unit and end-to-end tests
+examples/        sample log with an injected error burst
+docs/            flow diagram (Mermaid source + rendered PNG)
+```
+
+## Roadmap
+
+- Config file for custom log-format parsing
+- Multiple file / glob support
+- Slack/Discord webhook alerts
+- Persistent anomaly history (SQLite)
+
+## Contributing
+
+Issues and PRs welcome — especially around log-format parsing (every stack logs differently) and reducing false positives.
+
+## License
+
+MIT — see [LICENSE](https://github.com/AmanSg098/logometer/blob/main/LICENSE).

logometer-0.1.0/README.md

@@ -0,0 +1,265 @@
+# logometer
+
+**Tail your logs. Catch anomalies. Skip the 2am grep.**
+
+`logometer` watches a log file (or stdin), buckets lines into time windows, and flags windows that look statistically off — an error spike, or an error type that's never shown up before. It's silent when things are fine. No AI required for the core detection; point it at an LLM with `--explain` if you want a plain-English guess at *why* a window looks weird.
+
+Zero dependencies for the core tool — just Python 3.10+.
+
+<!-- TODO: record a short asciinema/GIF demo and drop it here before publishing.
+     Suggested recording: `logometer tail examples/sample.log --replay` -->
+
+## Install
+
+As a standalone command, without cloning:
+
+```bash
+pipx install git+https://github.com/AmanSg098/logometer.git
+```
+
+Or from a clone, for development:
+
+```bash
+git clone https://github.com/AmanSg098/logometer.git
+cd logometer
+pip install -e .              # core tool, no dependencies
+pip install -e ".[pretty]"    # optional: styled output via rich
+```
+
+## Quick start
+
+Try it on the included sample log (has a normal-traffic baseline plus an injected error burst):
+
+```bash
+python3 -m logometer.cli tail examples/sample.log --replay
+```
+
+or, once installed:
+
+```bash
+logometer tail examples/sample.log --replay
+```
+
+## Usage
+
+```bash
+# Tail a live log file (like tail -f, but with anomaly detection)
+logometer tail /var/log/app.log
+
+# Pipe from stdin
+tail -f app.log | logometer tail -
+
+# Replay a static file from start to finish, then exit
+logometer tail app.log --replay
+
+# Only show anomalous windows, hide the "ok" noise
+logometer tail app.log --replay --quiet
+
+# Adjust window size (seconds) and how aggressively to flag deviations
+logometer tail app.log --window 30 --sensitivity high
+
+# Mute known-noisy lines so they never reach the baseline (repeatable regex)
+logometer tail app.log --ignore 'healthcheck' --ignore 'DeprecationWarning'
+
+# Ask an LLM for a one-sentence explanation of each anomaly
+export ANTHROPIC_API_KEY=your-key-here
+logometer tail app.log --explain
+
+# JSON output, one object per line — good for piping into other tools
+logometer tail app.log --format json
+```
+
+Stop a live tail with Ctrl-C (or `SIGTERM` from a process manager); the window in progress is reported before exiting.
+
+### All options
+
+| Option | Default | What it does |
+|---|---|---|
+| `file` | — | Log file to read, or `-` for stdin |
+| `--window SECONDS` | `10` | Window size, when timestamps are parseable |
+| `--sensitivity` | `medium` | `low`, `medium` or `high` — how far above the baseline a window must be to count as a spike |
+| `--format` | `plain` | `plain` or `json` |
+| `--replay` | off | Read the file start to end and exit, instead of following it live |
+| `--quiet` | off | Only print anomalous windows |
+| `--ignore REGEX` | — | Drop matching lines before analysis; repeatable |
+| `--explain` | off | Ask an LLM for a one-sentence cause of each anomaly |
+| `--explain-provider` | `anthropic` | `anthropic`, `openai` or `openrouter` |
+| `--explain-model MODEL` | see below | Override the model used by `--explain` |
+| `--no-pretty` | off | Use plain output even if `rich` is installed |
+| `--version` | — | Print the version and exit |
+
+### Supported timestamp formats
+
+Timestamps are read from each line to decide which window it belongs to. Two formats are recognised:
+
+- **ISO 8601** — `2026-09-17T12:03:10`, `2026-09-17 12:03:10.123Z`, and Python `logging`'s default `2026-09-17 12:03:10,123`
+- **Syslog** — `Sep 17 12:03:10` (syslog has no year, so the current year is assumed)
+
+The first 20 lines decide the mode: if at least half have a recognisable timestamp, lines are bucketed by time; otherwise every 50 lines form a window.
+
+### `--explain`
+
+Requires an API key for one of three providers, set in your environment:
+
+| `--explain-provider` | Key | Default model |
+|---|---|---|
+| `anthropic` (default) | `ANTHROPIC_API_KEY` | `claude-haiku-4-5-20251001` |
+| `openai` | `OPENAI_API_KEY` | `gpt-4o-mini` |
+| `openrouter` | `OPENROUTER_API_KEY` | `anthropic/claude-haiku-4.5` |
+
+[OpenRouter](https://openrouter.ai) gives one key access to models from many vendors; its model ids are prefixed with the vendor (`openai/gpt-4o-mini`, `google/gemini-2.5-flash`, …). No SDK install needed — it's a plain HTTPS call. If the key is missing or the request fails, `logometer` prints a one-line warning to stderr and keeps tailing normally; it never crashes because `--explain` had a bad day.
+
+Only the anomalous window's ERROR and WARN lines are sent (capped at 30 lines). Pick a different model with `--explain-model`:
+
+```bash
+export OPENAI_API_KEY=your-key-here
+logometer tail app.log --explain --explain-provider openai --explain-model gpt-4o
+
+export OPENROUTER_API_KEY=your-key-here
+logometer tail app.log --explain --explain-provider openrouter --explain-model google/gemini-2.5-flash
+```
+
+### `--ignore`
+
+Real logs usually have one or two messages that fire constantly and mean nothing. Left alone they dominate the error count, so the rolling baseline learns them as "normal" and a genuine problem has to shout louder to get noticed. `--ignore` drops matching lines before any analysis:
+
+```bash
+# on a real macOS install log, one benign repeating message accounted for
+# 97% of all "errors" — muting it cut the noise dramatically
+logometer tail /var/log/install.log --replay --window 60 --quiet \
+  --ignore 'installation-check'          # 27 anomalies -> 5, all genuine
+```
+
+Each `--ignore` takes a regular expression and can be repeated. An invalid pattern is reported as a normal CLI error rather than a crash.
+
+### Live tailing
+
+`logometer tail app.log` (without `--replay`) follows the file the way `tail -f` does, with two things a naive tail loop gets wrong:
+
+- **Windows close on time.** A window is emitted once its duration has elapsed, even if no further lines arrive. Without this, a service that errors and then dies would never report that final burst — the most important one — because nothing follows it to trigger the flush.
+- **Rotation is handled.** If the log is rotated out from under it (`logrotate`, or truncated in place with `copytruncate`), it notices and follows the new file instead of reading a now-orphaned file forever.
+
+Output is flushed as each window is reported, so piping or redirecting works in real time:
+
+```bash
+logometer tail app.log --quiet --format json >> alerts.jsonl
+tail -f app.log | logometer tail -
+```
+
+### Pretty output
+
+If you `pip install rich` (or `pip install -e ".[pretty]"`), terminal output automatically upgrades to styled panels. Not required — plain ANSI output works everywhere. When output is piped or redirected to a file it stays plain text either way; pass `--no-pretty` to get plain output in the terminal too.
+
+## Example output
+
+From the included sample log:
+
+```
+$ logometer tail examples/sample.log --replay --no-pretty
+
+  [12:01:20 – 12:01:30]  ok        errors: 0   baseline: ~0.1
+
+  [12:01:30 – 12:01:40]  ! ANOMALY  errors: 10  warns: 1  baseline: ~0.1   score: 9.9x
+    New error signature detected: '<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>'
+
+  [12:01:40 – 12:01:50]  ! ANOMALY  errors: 6  baseline: ~0.1   score: 5.9x
+    (error rate spike — no brand-new error signature)
+
+  [12:01:50 – 12:02:00]  ok        errors: 0   baseline: ~0.1
+  ...
+-- 18 window(s) processed, 4 anomaly(ies) flagged --
+```
+
+`score` is how many standard deviations the window's error count sits above the baseline. `<x>` marks the ids, numbers and timestamps stripped out when building an error signature.
+
+With `--explain`, each anomaly gets one more line:
+
+```
+  [12:01:30 – 12:01:40]  ! ANOMALY  errors: 10  warns: 1  baseline: ~0.1   score: 9.9x
+    New error signature detected: '<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>'
+    Explanation: Database or downstream service became unresponsive, causing clients to forcibly close connections due to timeouts or hangs.
+```
+
+With `--format json`, each window is one line with these fields:
+
+```json
+{"window_index": 9, "start": "12:01:30", "end": "12:01:40", "error_count": 10, "warn_count": 1, "baseline_mean": 0.111, "error_score": 9.889, "is_anomaly": true, "new_shapes": ["<x> ERROR ConnectionResetError: [Errno <x>] Connection reset by peer id=<x>"], "explanation": null}
+```
+
+The end-of-run summary line is omitted in JSON mode, so every line of output is valid JSON.
+
+## How the detection works
+
+<!-- Source: docs/flow.mmd. Regenerate after editing it with:
+     npx -p @mermaid-js/mermaid-cli mmdc -i docs/flow.mmd -o docs/flow.png -b white -s 2 -c docs/mermaid-config.json -->
+![How logometer processes a log: prepare each line, judge each window, report](https://raw.githubusercontent.com/AmanSg098/logometer/main/docs/flow.png)
+
+1. **Classify.** Each line is tagged by keyword: ERROR (`error`, `err`, `fatal`, `critical`, `exception`, `traceback`, `panic`), then WARN (`warn`, `warning`), INFO (`info`, `notice`), DEBUG (`debug`, `trace`). First match wins, so a line mentioning both an error and a warning counts as ERROR.
+2. **Fingerprint.** ERROR and WARN lines are reduced to a "shape": UUIDs, hex addresses, timestamps, quoted strings and numbers are replaced with `<x>`, so the same underlying error collapses to one shape regardless of the specific id.
+3. **Window.** Lines are batched into fixed-size windows — by time if timestamps are parseable, otherwise by a fixed line count.
+4. **Baseline.** The error counts of the last 20 windows give a rolling mean and standard deviation — what "normal" error volume looks like. Scoring starts once 3 windows of history exist.
+5. **Flag.** A window is anomalous if either:
+   - its error count is at least N standard deviations above the mean, where N is 3.0 for `--sensitivity low`, 2.0 for `medium` and 1.2 for `high`; or
+   - it contains an error *or warning* shape not seen earlier in this run (checked from the second window on).
+
+Two tuning choices keep this honest:
+
+- **Noise floor.** The standard deviation is never taken as less than 1.0. On a log that's normally error-free the real deviation is 0, and a single stray error would otherwise score as infinitely anomalous.
+- **Spikes don't train the baseline.** A window flagged as a spike isn't added to the history, so a long outage doesn't slowly teach the tool that a high error rate is normal.
+
+No ML model, no training step — deliberately simple and explainable.
+
+## Known limitations
+
+Worth knowing before you point this at something you care about. These are real, tested behaviours, not hypotheticals:
+
+- **Severity is keyword matching, not parsing.** A line counts as an error if it contains a word like `error`, `fatal` or `critical`. That means `Downloading 1 products: Critical []` — an *empty* list, i.e. good news — is counted as an error. Use `--ignore` to mute these.
+- **Python tracebacks are only partly seen.** `Traceback (most recent call last):` is detected, but the final `ValueError: ...` line is not, because the match looks for `error` as a standalone word. And since that first line is identical for every exception, new-signature detection can't tell one crash type from another. Logs that print their own level (`ERROR ValueError: ...`) work properly.
+- **JSON logs break new-signature detection.** Fingerprinting strips quoted strings, which in a JSON line removes the entire message — so unrelated errors collapse into one shape. Error *counting* still works. Extracting the message first works around it: `jq -r '.level + " " + .msg' app.log | logometer tail -`.
+- **Only the error count is scored.** A flood of identical *warnings* won't trigger a spike (though a never-seen-before warning shape will be flagged), and a sudden *drop* in traffic isn't detected either — only rises above the baseline are.
+- **Window labels show time, not date.** On a log spanning several days you'll see the same `[17:14:39 – 17:15:39]` label more than once.
+- **Timezone offsets are ignored.** `2026-09-20 19:05:04+05:30` is read as local wall-clock time; a log mixing offsets is bucketed as though they were the same clock.
+- **Nothing persists between runs.** The baseline and the set of known error shapes are rebuilt from scratch each start, so expect the first few windows of any run to over-flag. (Persistence is on the roadmap.)
+- **If timestamps can't be parsed** it silently falls back to fixed 50-line windows, and `--window` stops having any effect. You can tell from the labels: `[line 1 – line 50]` instead of a time range.
+- **`--explain` only sees ERROR and WARN lines.** Lines without a level keyword — like the body of a Python traceback — aren't sent, so the model can miss the root cause and give a vaguer answer.
+- **`--explain` sends log lines to a third party.** There's no redaction — don't use it on logs containing secrets or personal data.
+
+## Running the tests
+
+```bash
+python3 -m unittest discover -s tests -v
+```
+
+No network or API key is needed — the `--explain` tests mock the HTTP call.
+
+## Project layout
+
+```
+logometer/
+  cli.py         command-line entry point, file/stdin reading, output formatting
+  classifier.py  severity tagging and error-shape fingerprinting
+  timeparse.py   timestamp extraction (ISO 8601, syslog)
+  windower.py    groups lines into time- or count-based windows
+  baseline.py    rolling mean / standard deviation of errors per window
+  detector.py    decides whether a window is anomalous
+  explain.py     optional LLM explanations (Anthropic / OpenAI / OpenRouter over plain HTTPS)
+  pretty.py      optional rich-styled output
+tests/           unit and end-to-end tests
+examples/        sample log with an injected error burst
+docs/            flow diagram (Mermaid source + rendered PNG)
+```
+
+## Roadmap
+
+- Config file for custom log-format parsing
+- Multiple file / glob support
+- Slack/Discord webhook alerts
+- Persistent anomaly history (SQLite)
+
+## Contributing
+
+Issues and PRs welcome — especially around log-format parsing (every stack logs differently) and reducing false positives.
+
+## License
+
+MIT — see [LICENSE](https://github.com/AmanSg098/logometer/blob/main/LICENSE).

logometer-0.1.0/logometer/__init__.py

@@ -0,0 +1,3 @@
+"""logometer — tail your logs, catch anomalies, skip the 2am grep."""
+
+__version__ = "0.1.0"

logometer-0.1.0/logometer/baseline.py

@@ -0,0 +1,54 @@
+"""
+A small rolling baseline: keeps the last N observed values for a metric
+(e.g. error count per window) and reports the mean/std of that history.
+
+Deliberately simple — a moving window of raw values, not an
+exponentially-weighted or Bayesian estimator. Easy to explain, easy to
+reason about when it gets something wrong.
+"""
+from __future__ import annotations
+
+import statistics
+from collections import deque
+
+
+class RollingBaseline:
+    def __init__(self, history_size: int = 20):
+        """Keep a fixed-size deque of recent metric samples (e.g. errors per window).
+        Older values drop off automatically when history_size is exceeded."""
+        self._values: deque[float] = deque(maxlen=history_size)
+
+    def update(self, value: float) -> None:
+        """Append one observation; drop the oldest when history exceeds history_size.
+        Called after each non-spike window in AnomalyDetector.evaluate."""
+        self._values.append(value)
+
+    @property
+    def ready(self) -> bool:
+        """True once there are enough samples to compare against (>= 3).
+        Until ready, deviation_score returns 0 and spikes are not scored."""
+        return len(self._values) >= 3
+
+    @property
+    def mean(self) -> float:
+        """Arithmetic mean of stored samples, or 0.0 if history is empty.
+        Shown in CLI output as the approximate errors-per-window baseline."""
+        if not self._values:
+            return 0.0
+        return statistics.fmean(self._values)
+
+    @property
+    def stdev(self) -> float:
+        """Population standard deviation of stored samples, or 0.0 if fewer than two.
+        Used with mean inside deviation_score for spike detection."""
+        if len(self._values) < 2:
+            return 0.0
+        return statistics.pstdev(self._values)
+
+    def deviation_score(self, value: float, min_stdev: float = 1.0) -> float:
+        """Return how many std devs above the mean `value` is (0 if not ready).
+        Uses max(stdev, min_stdev) so flat histories do not over-flag single events."""
+        if not self.ready:
+            return 0.0
+        std = max(self.stdev, min_stdev)
+        return (value - self.mean) / std