mdbkit 0.5.1__tar.gz → 0.5.2__tar.gz

{mdbkit-0.5.1 → mdbkit-0.5.2}/PKG-INFO

@@ -1,1065 +1,1120 @@
-Metadata-Version: 2.4
-Name: mdbkit
-Version: 0.5.1
-Summary: Offline toolkit for MongoDB 4.4+ structured logs: log analysis, slow-query shapes, connection churn, and deterministic index advice. A spiritual successor to mtools' log tools.
-Author: Saqib Ameen Subhan
-License: MIT
-Project-URL: Homepage, https://github.com/saqibameen86/mdbkit
-Project-URL: Repository, https://github.com/saqibameen86/mdbkit
-Project-URL: Documentation, https://github.com/saqibameen86/mdbkit#readme
-Project-URL: Issues, https://github.com/saqibameen86/mdbkit/issues
-Project-URL: Changelog, https://github.com/saqibameen86/mdbkit/releases
-Keywords: mongodb,mongodb-logs,logv2,mtools,mloginfo,mongod,slow-query,slow-query-log,log-analysis,log-parser,index-advisor,dba,sre,devops,ftdc,diagnostic-data,explain-plan,query-performance,database-performance,replica-set,wiredtiger,troubleshooting,incident-response
-Classifier: Development Status :: 4 - Beta
-Classifier: Environment :: Console
-Classifier: Intended Audience :: System Administrators
-Classifier: Intended Audience :: Developers
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Topic :: Database
-Classifier: Topic :: System :: Systems Administration
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: dev
-Requires-Dist: pytest>=7; extra == "dev"
-Dynamic: license-file
-
-# mdbkit
-
-[![CI](https://github.com/saqibameen86/mdbkit/actions/workflows/ci.yml/badge.svg)](https://github.com/saqibameen86/mdbkit/actions)
-[![PyPI](https://img.shields.io/pypi/v/mdbkit.svg)](https://pypi.org/project/mdbkit/)
-[![Python](https://img.shields.io/pypi/pyversions/mdbkit.svg)](https://pypi.org/project/mdbkit/)
-[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
-
-**An offline toolkit for MongoDB structured logs** — slow-query analysis,
-deterministic index advice, incident triage, and diagnostic-data decoding.
-For MongoDB 4.4 – 8.0, from the terminal, without connecting to anything.
-
-A spiritual successor to mtools' log tools, which never learned to read the
-JSON log format introduced in 4.4.
-
-```
-namespace    op         count  cumMs  docsEx      scan     plan           shape
-shop.events  aggregate  29     3.2m   25,810,000  98889:1  COLLSCAN+SORT  {tenantId:eq, ts:gte} sort:{ts:-1}
-shop.orders  find       48     1.4m   6,000,000   2976:1   COLLSCAN+SORT  {status:eq, createdAt:gt}
-shop.users   find       31     3.7s   31          1:1      IXSCAN{email}  {email:eq}
-```
-
-Two of those need an index. One is already fine. That distinction is the
-whole point.
-
----
-
-## Try it right now — no MongoDB required
-
-`mdbkit demo` writes a realistic log containing a real incident, so you can
-evaluate the tool in about a minute without touching a cluster:
-
-```bash
-pip install mdbkit
-
-mdbkit demo --with-extras -o demo.log     # a log + indexes.json, schema.json, explain.json
-
-mdbkit loginfo demo.log                   # what is in this log?
-mdbkit queries demo.log                   # which query shapes cost the most?
-mdbkit triage demo.log --window 0         # what went wrong, and when?
-mdbkit connections demo.log               # who connected, and did anyone fail to?
-mdbkit advise demo.log --indexes indexes.json --schema schema.json
-mdbkit explain explain.json               # read a saved explain plan
-```
-
-The generated log contains a connection storm from one client, a replica set
-election, an index build, five failed logins from a service account, and an
-aggregation burning 25 million document reads to return 261 documents — then
-`advise` tells you which index fixes it.
-
-Output is deterministic: the same `--seed` always produces the same log, so a
-demo behaves identically every time. Scenarios are `incident`, `healthy` (the
-control case — useful for seeing what "nothing wrong" looks like) and `mixed`.
-
-Ready for a real server? Jump to [the workflows](#the-four-questions-it-answers).
-
----
-
-## Is it safe to run on a production server?
-
-This is the right question to ask of any tool someone hands you. The honest
-answer, and how to check it yourself.
-
-**What mdbkit never does:**
-
-| | |
-|---|---|
-| Connect to your database | Analysis commands read **files**. There is no driver, no URI, no connection. |
-| Send anything anywhere | There is no network code at all. No telemetry, no update check, no crash reporting. |
-| Change anything | It is strictly read-only. Where an action would help, it **prints the command** for you to review and run. |
-| Execute what it reads | Log lines and explain files are parsed as data with `json.loads`. Nothing is ever evaluated. |
-| Pull in dependencies | Zero runtime dependencies. Nothing in the supply chain but the Python standard library. |
-
-**Verify it yourself in 60 seconds** — this is a small, dependency-free
-codebase specifically so that you can:
-
-```bash
-# 1. No network, no shell-outs, no eval anywhere in the analysis code
-pip show -f mdbkit | head -3
-grep -rn "socket\|urllib\|requests\|http\|eval(\|exec(" $(python -c "import mdbkit,os;print(os.path.dirname(mdbkit.__file__))")
-
-# 2. Confirm it has no dependencies
-pip show mdbkit | grep Requires
-
-# 3. Watch it make no connections while it runs (Linux)
-strace -f -e trace=network mdbkit queries mongod.log 2>&1 | grep -c socket
-```
-
-The grep returns nothing for every analysis module. The only file that starts
-a process is `lab.py`, which exists to create a *throwaway test cluster* and
-is documented as an explicit exception below.
-
-**What it does read:** the log file you point it at; optionally
-`diagnostic.data` (metrics only, never documents); optionally `indexes.json`
-and `schema.json` that **you** generate with scripts mdbkit prints for you to
-inspect first. On the database host it also reads `/proc` and calls `statvfs`
-for disk and memory figures — nothing that leaves the machine.
-
-**What leaves your machine: nothing.** There is no server to send it to.
-
-**Still cautious?** That is reasonable. Run `mdbkit demo` first and see what
-the output looks like on synthetic data, or `mdbkit lab` to try it against a
-disposable local cluster before you point it at anything real. Both exist for
-exactly this reason.
-
-Full detail: [SECURITY.md](SECURITY.md).
-
----
-
-## Install
-
-**Most systems:**
-```bash
-pip install mdbkit
-```
-
-**Ubuntu 20.04 / Debian / Amazon Linux 2 (Python 3.8 hosts):**
-```bash
-sudo apt install pipx        # or: sudo dnf install pipx
-pipx install mdbkit
-pipx ensurepath && source ~/.bashrc
-```
-
-**Modern Ubuntu/Debian complaining about "externally-managed-environment":**
-```bash
-pip install mdbkit --break-system-packages
-```
-
-**Air-gapped database hosts:**
-```bash
-pip download mdbkit -d ./wheels          # on a connected machine
-# copy ./wheels across, then:
-pip install --no-index --find-links ./wheels mdbkit
-```
-
-**Upgrading:**
-```bash
-pip install --upgrade mdbkit             # or: pipx upgrade mdbkit
-mdbkit --version
-```
-
-Requires Python 3.8+. mdbkit never updates itself and never checks for
-updates — upgrades are always explicit.
-
-> mdbkit is a Python package on PyPI. It is **not** in `apt`/`dnf`/`yum`.
-
----
-
-## The four questions it answers
-
-Every command reads files or stdin and accepts several files or a glob, so
-rotated logs work as one stream: `mdbkit queries "mongod.log*"`.
-
-### 1. Why is my database slow?
-
-```bash
-mdbkit queries mongod.log                       # shapes ranked by total time
-mdbkit queries mongod.log --sort scanRatio      # worst examined:returned first
-mdbkit queries mongod.log --shape 1             # full detail on one shape
-```
-
-`cumMs` is time summed across **all** occurrences of a shape, not one query.
-`scan` is documents examined per document returned — `1:1` is healthy,
-`98889:1` is a missing index. `plan` shows what MongoDB actually chose.
-
-### 2. What index would fix it?
-
-```bash
-# Optional but much sharper: export what already exists.
-mdbkit export-script indexes > export_indexes.js
-mdbkit export-script schema  > export_schema.js
-
-mongosh --quiet --host your_db_host --port 27017 \
-  --username your_username --password your_password \
-  --authenticationDatabase admin \
-  --eval "$(cat export_indexes.js)" > indexes.json      # repeat for schema
-
-mdbkit advise mongod.log --indexes indexes.json --schema schema.json --ns shop.orders
-```
-
-Every recommendation states the evidence it reasoned from, a confidence
-level, the caveats, and how to validate it. It says *candidate*, not
-*command*, and it never tells you to drop an index.
-
-### 3. What happened at 3am?
-
-```bash
-mdbkit triage /var/log/mongodb/mongod.log       # last 60 minutes by default
-mdbkit triage mongod.log --window 0             # the whole file
-mdbkit triage mongod.log --report incident.html # something to attach to a ticket
-```
-
-Cluster health, elections, connection storms, hot collections, index builds,
-error clusters, slow-query peaks — plus disk, memory, CPU and FTDC metrics
-when run on the database host. Every finding ends with the next command to
-run.
-
-### 4. Did my change actually help?
-
-```bash
-mdbkit compare before.log --after after.log
-```
-
-```
-slow-query time DOWN 32%  (6.0m -> 4.1m across compared shapes)
-shapes: 1 improved, 0 regressed, 0 new, 0 gone, 4 unchanged
-
-IMPROVED
-  shop.orders {createdAt:gt, status:eq} sort:{createdAt:-1}
-    mean 1.7s -> 33ms (-98%)   scan 2976:1 -> 1:1  [COLLSCAN -> index, in-memory sort gone]
-```
-
-The natural follow-up to `advise`: you created the index, a day passed, and
-this tells you whether it worked.
-
-### 5. Why did it die? (and what does serverStatus say?)
-
-```bash
-mdbkit oslog /var/log/syslog                  # OOM kills, fd limits, I/O errors
-mdbkit triage mongod.log --oslog /var/log/syslog
-```
-
-A mongod log cannot record its own OOM kill — the process is gone before it
-can write anything. The system log has the answer in one line, and `triage`
-will correlate it with the unexplained restart.
-
-```bash
-mdbkit export-script serverstatus > export_serverstatus.js
-mongosh --quiet --host HOST --eval "$(cat export_serverstatus.js)" > status.json
-mdbkit serverstatus status.json
-```
-
-```
-[CRIT] Concurrency tickets: Exhausted tickets queue every new operation,
-       which looks like slowness with no slow query to blame.
-        - read: 1 of 128 free (1%)
-[CRIT] WiredTiger cache: Above 80% WiredTiger evicts in the background;
-       above 95% application threads are made to evict.
-        - 31.0 GiB of 32.0 GiB used (96.9%)
-```
-
-**Bonus — who connected?**
-
-```bash
-mdbkit connections mongod.log
-```
-
-Per-IP churn with first/last seen, plus an authenticated-users table showing
-successful and failed logins per account and when each last authenticated —
-the question that starts most access incidents.
-
----
-
-## Running it on a schedule
-
-mdbkit is deterministic, offline and read-only, which makes it well suited to
-a cron job. It deliberately **cannot send anything anywhere** — there is no
-network code and never will be. So mdbkit produces the verdict and your own
-script does the talking.
-
-The primitive that makes this work is `--exit-code`:
-
-| Exit code | Meaning |
-|---|---|
-| `0` | Nothing above INFO |
-| `1` | At least one WARN |
-| `2` | At least one CRIT |
-
-```bash
-#!/usr/bin/env bash
-# /usr/local/bin/mdbkit-watch.sh — hourly health check
-set -uo pipefail
-
-LOG=/var/log/mongodb/mongod.log
-OUT=$(mktemp)
-
-mdbkit triage "$LOG" --window 60 --oslog /var/log/syslog \
-       --only CRIT,WARN --exit-code > "$OUT" 2>&1
-STATUS=$?
-
-if [ "$STATUS" -ge 2 ]; then
-    # CRIT: wake someone up. Your channel, your call — mdbkit stays offline.
-    curl -sf -X POST -H 'Content-type: application/json' \
-         --data "{\"text\": \"MongoDB CRIT on $(hostname)\n\`\`\`$(cat "$OUT")\`\`\`\"}" \
-         "$SLACK_WEBHOOK_URL"
-elif [ "$STATUS" -eq 1 ]; then
-    mail -s "MongoDB warnings on $(hostname)" dba@example.com < "$OUT"
-fi
-
-rm -f "$OUT"
-```
-
-```cron
-# hourly triage; only speaks up when something is wrong
-0 * * * * /usr/local/bin/mdbkit-watch.sh
-
-# daily slow-query digest, kept for trend comparison
-30 6 * * * mdbkit queries /var/log/mongodb/mongod.log \
-             --report /var/log/mdbkit/$(date +\%F).html
-```
-
-Because the reports are dated, `compare` turns them into a trend:
-
-```bash
-mdbkit compare /var/log/mongodb/mongod.log.1 --after /var/log/mongodb/mongod.log
-```
-
-Two things worth knowing. `--only CRIT,WARN` keeps the mail short, and a run
-that finds nothing prints almost nothing — so a silent cron job means a
-healthy database rather than a broken script. And because every command is
-read-only and never connects to the database, running this hourly on a
-production host costs one log read and no risk.
-
----
-
-## Trying it against a real cluster
-
-`mdbkit lab` starts a **disposable local MongoDB** so you can test against a
-real server without touching anything that matters.
-
-```bash
-mdbkit lab start                    # 3-node replica set on 127.0.0.1:28110-28112
-mdbkit lab seed                     # 50k documents + a deliberately mixed workload
-mdbkit queries $(mdbkit lab logs | head -1)
-mdbkit lab destroy --yes            # remove it entirely
-```
-
-It binds to localhost only, uses ports far from 27017 so it can never be
-confused with a real deployment, and refuses to touch any directory it did
-not create. It is the one command that starts external processes — see
-[SECURITY.md](SECURITY.md).
-
-Full options and more examples: [`mdbkit lab`](#mdbkit-lab) in the reference.
-
----
-
-## Command reference
-
-Every command reads files or stdin and writes to stdout. `--help` works on any
-command (`mdbkit queries --help`). Global: `mdbkit --version`.
-
-All commands that read a log accept one or more paths, a shell glob, a
-rotated `.gz` file, or `-` for stdin. Several files are read as a single
-stream in filename order, which matches MongoDB's rotation naming:
-
-```bash
-mdbkit queries mongod.log                       # one file
-mdbkit queries mongod.log.1 mongod.log          # explicit list
-mdbkit queries "mongod.log*"                    # glob (quote it)
-mdbkit queries /var/log/mongodb/mongod.log.*.gz # compressed archives
-cat mongod.log | mdbkit queries -               # stdin
-```
-
----
-
-### `mdbkit loginfo <log>`
-
-Overall log summary: server version, host, restarts, connections accepted,
-slow-query count, warning/error counts, and a per-component line breakdown.
-
-| Option | Description |
-|---|---|
-| `--json` | Machine-readable output |
-
-```bash
-mdbkit loginfo /var/log/mongodb/mongod.log
-mdbkit loginfo mongod.log.2.gz --json
-```
-
----
-
-### `mdbkit queries <log>`
-
-Slow queries grouped by **query shape** — literal values stripped, so the same
-query with different parameters is counted once.
-
-| Option | Default | Description |
-|---|---|---|
-| `--sort FIELD` | `totalMs` | Order by `totalMs`, `count`, `mean`, `max`, `docsExamined`, or `scanRatio` |
-| `--limit N` | all | Show only the top N shapes |
-| `--min-ms N` | 0 | Ignore operations faster than N milliseconds |
-| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces (hidden by default — they are server housekeeping, not your workload) |
-| `--report FILE` | | Write a shareable `.md` or `.html` report instead (see [Shareable reports](#shareable-reports----report-file)) |
-| `--json` | | Machine-readable output |
-
-**Reading the columns:**
-
-| Column | Meaning |
-|---|---|
-| `cumMs` | Time summed across **all** occurrences of that shape — not one query |
-| `mean` / `max` | Per-occurrence average and worst case |
-| `docsEx` | Documents examined, summed across all occurrences |
-| `scan` | Documents examined per document returned. `1:1` is ideal; `3444:1` means a missing or weak index |
-| `plan` | The plan MongoDB chose: `COLLSCAN` (no index), `IXSCAN{fields}` (index used), `IDHACK` (`_id` lookup), `+SORT` (in-memory sort). `?` = the plan was not recorded on that line |
-| `shape` | Fields and operators queried, with the sort |
-
-```bash
-mdbkit queries mongod.log
-mdbkit queries mongod.log --sort scanRatio --limit 10
-mdbkit queries mongod.log --min-ms 500 --json
-mdbkit queries "mongod.log*"              # rotated logs as one stream
-mdbkit queries mongod.log --shape 1       # drill into the worst offender
-```
-
-`--shape N` expands one row of the table:
-
-```
-namespace : shop.events
-shape     : {tenantId:eq, ts:gte} sort:{ts:-1}
-
-occurrences   : 29
-total time    : 3.2m
-mean / max    : 6.6s / 8.9s
-docs examined : 25,810,000
-docs returned : 261
-scan ratio    : 98889 examined per document returned
-
-plans observed
-  COLLSCAN                                 29x
-
-flags
-  COLLSCAN — no index used for at least one execution
-  in-memory SORT — results sorted after retrieval
-
-client applications
-  ReportWorker                   15x
-  OrderService                   7x
-```
-
----
-
-### `mdbkit connections <log>`
-
-Connection churn and **who authenticated**: totals, peak concurrent count,
-per-source-IP breakdown with first/last seen, the client applications and
-drivers, and a per-user table.
-
-| Option | Description |
-|---|---|
-| `--json` | Machine-readable output |
-
-```bash
-mdbkit connections mongod.log
-```
-
-```
-source ip   accepted  ended  first seen           last seen            appName
-----------  --------  -----  -------------------  -------------------  ------------
-10.20.9.77  220       0      2026-07-01 08:49:30  2026-07-01 08:49:30  checkout-api
-10.20.4.11  4         1      2026-07-01 08:00:15  2026-07-01 09:29:30  OrderService
-
-authenticated users
-user          auth db  ok   failed  last authenticated   from
-------------  -------  ---  ------  -------------------  -----------
-svc_checkout  admin    221  0       2026-07-01 08:49:30  10.20.9.77
-etl_batch     admin    0    5       2026-07-01 08:51:54  10.20.11.40
-
-  etl_batch: 5 failed authentication(s) — last error: AuthenticationFailed
-```
-
-This answers the question that starts most access incidents: *did that
-account connect, from where, and when last?* If the log shows no
-authentication events at all, mdbkit says so — either auth is disabled, or
-the window contains no new logins because clients are reusing connections.
-
----
-
-### `mdbkit filter <log>`
-
-Streams **matching raw log lines** to stdout. Output stays valid logv2 JSON, so
-it chains with other tools (including mdbkit itself).
-
-| Option | Description |
-|---|---|
-| `--component NAME` | `COMMAND`, `NETWORK`, `REPL`, `STORAGE`, `INDEX`, `WRITE`, `QUERY`, `CONTROL`, … |
-| `--severity S` | `I` info, `W` warning, `E` error, `F` fatal |
-| `--ns NAMESPACE` | Exact namespace, e.g. `shop.orders` |
-| `--slow N` | Only operations with `durationMillis` >= N |
-| `--from TIMESTAMP` | Lower time bound (inclusive) |
-| `--to TIMESTAMP` | Upper time bound (inclusive) |
-| `--msg TEXT` | Substring match on the message field |
-| `--limit N` | Print only the **first** N matches |
-| `--last N` | Print only the **last** N matches — usually what you want during an incident |
-| `--as-explain` | Rebuild each matching slow query as a runnable `mongosh` `.explain()` command instead of printing the raw log line |
-| `--explain-script` | With `--as-explain`, wrap in `EJSON.stringify()` plus usage comments so it can be saved as a `.js` file |
-
-**Timestamp formats accepted** by `--from` / `--to`:
-
-```
-2026-07-01T08:00:00+04:00     with an explicit offset (production logs)
-2026-07-01T08:00:00Z          UTC
-2026-07-01T08:00:00           no offset — read as the log's own timezone
-2026-07-01 08:00:00           space instead of T
-2026-07-01T08:00               minute precision
-2026-07-01                     whole day
-```
-
-```bash
-mdbkit filter mongod.log --severity E --last 20    # errors (most recent 20)
-mdbkit filter mongod.log --severity F               # fatal — always investigate
-mdbkit filter mongod.log --severity W --last 50     # warnings
-mdbkit filter mongod.log --component REPL --msg election
-mdbkit filter mongod.log --slow 500 --ns shop.orders --limit 50
-mdbkit filter mongod.log --from 2026-07-01T14:30:00+04:00 --to 2026-07-01T15:00:00+04:00
-mdbkit filter mongod.log --slow 100 | mdbkit queries -
-```
-
-**From a slow query in the log to an explain plan**, without hand-writing the
-query — `--as-explain` rebuilds the command that ran:
-
-```bash
-# See the actual commands behind your slowest operations
-mdbkit filter mongod.log --ns shop.orders --slow 500 --last 3 --as-explain
-
-# Or produce a runnable script, get the plan, and analyze it
-mdbkit filter mongod.log --slow 500 --last 1 --as-explain --explain-script > q.js
-mongosh --quiet --host your_db_host --username your_username \\
-        --password your_password --authenticationDatabase admin \\
-        --eval "$(cat q.js)" > explain.json
-mdbkit explain explain.json
-```
-
-> Rebuilt commands contain the **real values** from your log (not redacted
-> shapes) — treat them as sensitive.
-
----
-
-### `mdbkit advise <log>`
-
-Deterministic **candidate** index recommendations from observed slow-query
-shapes, using the ESR guideline (Equality → Sort → Range). Rules, not AI: the
-same log always produces the same advice.
-
-| Option | Default | Description |
-|---|---|---|
-| `--indexes FILE` | | `indexes.json` from `mdbkit export-script indexes` — enables overlap checks against existing indexes |
-| `--schema FILE` | | `schema.json` from `mdbkit export-script schema` — enables field-type caveats and confidence adjustment |
-| `--ns NAMESPACE` | all | Only advise on one namespace (recommended on large logs) |
-| `--limit N` | 10 | Show only the top N recommendations (`0` = all) |
-| `--min-ms N` | 0 | Ignore operations faster than N milliseconds |
-| `--min-count N` | 1 | Only advise on shapes seen at least N times |
-| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces |
-| `--json` | | Machine-readable output |
-
-Each recommendation carries a candidate key pattern, the evidence behind it, a
-confidence level, caveats, and a validation step. mdbkit never advises dropping
-an index — at most it flags an overlap to investigate.
-
-```bash
-mdbkit advise mongod.log
-mdbkit advise mongod.log --indexes indexes.json --schema schema.json
-mdbkit advise mongod.log --ns shop.orders --limit 3
-```
-
----
-
-### `mdbkit explain <file>`
-
-Analyzes a saved `explain("executionStats")` document: the plan chain, the
-examined-vs-returned math, plain-English verdicts, and — when the plan needs
-help — a candidate index from the same advisor engine.
-
-| Option | Description |
-|---|---|
-| `--indexes FILE` | Overlap check against existing indexes |
-| `--schema FILE` | Field-type caveats |
-| `--json` | Machine-readable output |
-
-**Full example.** Get a plan for a query and analyze it:
-
-```bash
-# 1. Capture the plan (adjust host/credentials for your deployment)
-mongosh --quiet \\
-  --host your_db_host \\
-  --port 27017 \\
-  --username your_username \\
-  --password your_password \\
-  --authenticationDatabase admin \\
-  --eval 'EJSON.stringify(db.getSiblingDB("shop").orders.find({status:"open"}).sort({ts:-1}).explain("executionStats"))' \\
-  > explain.json
-
-# 2. Analyze it
-mdbkit explain explain.json
-
-# 3. Sharper, with your existing indexes and sampled schema
-mdbkit explain explain.json --indexes indexes.json --schema schema.json
-```
-
-Don't want to write the query by hand? `mdbkit filter ... --as-explain`
-rebuilds it from the log for you (see the `filter` section above).
-
-Legacy `mongo` shell and Compass output containing `NumberLong(...)`,
-`ISODate(...)` or `ObjectId(...)` is accepted — mdbkit unwraps those
-automatically, so you do not have to re-export.
-
----
-
-### `mdbkit triage <log>`
-
-**"Triage" means: quickly work out what is wrong and what to look at first.**
-Run this when something has gone wrong — or has just gone wrong — and you need
-one screen that says what happened, how bad it is, and where to look next.
-**Defaults to the last 60 minutes of log time.**
-
-| Option | Default | Description |
-|---|---|---|
-| `--window N` | 60 | Analyze the last N minutes of log time; `0` = the whole file |
-| `--dbpath PATH` | auto | Override the data directory used for the disk check |
-| `--no-sysprobe` | off | Skip local disk/memory/CPU probes — use when analyzing a log copied off the host |
-| `--ftdc PATH` | | `diagnostic.data` directory — adds CPU, memory, cache, queue and connection history from MongoDB's own recorder |
-| `--report FILE` | | Write a shareable `.md` or `.html` report instead of terminal output |
-| `--json` | | Machine-readable output |
-
-```bash
-mdbkit triage /var/log/mongodb/mongod.log
-mdbkit triage mongod.log --window 30
-mdbkit triage mongod.log --ftdc /var/lib/mongodb/diagnostic.data
-mdbkit triage mongod.log --report incident.html
-mdbkit triage mongod.log --window 0 --no-sysprobe
-```
-
----
-
-### `mdbkit ftdc {summary|timeline|export} <path>`
-
-Decodes `diagnostic.data` — **FTDC (Full-Time Diagnostic Data Capture)**, the
-metrics recorder every mongod already runs. It holds CPU, memory, WiredTiger
-cache, connection, queue and operation history for every node, with no
-monitoring agent installed and no database connection. It is compressed BSON,
-not encrypted; mdbkit decodes it offline.
-
-| Action | Description |
-|---|---|
-| `summary` | min / avg / max / last per metric, plus per-second rates for counters |
-| `timeline` | Values bucketed over time — shows *when* something spiked |
-| `export` | CSV to stdout, for a spreadsheet or your own tooling |
-
-| Option | Default | Description |
-|---|---|---|
-| `--last DURATION` | `4h` | Analyze only the most recent window — `90m`, `4h`, `2d` |
-| `--all` | off | Analyze the entire history (see the performance note below) |
-| `--metric LABEL` | all | Restrict to one metric (repeatable), e.g. `--metric conns.current` |
-| `--step SECONDS` | 60 | Timeline bucket size |
-| `--from` / `--to` | | Explicit time bounds (same formats as `filter`) |
-| `--json` | | Machine-readable output |
-
-**Performance note.** `diagnostic.data` can hold weeks of per-second samples —
-a few hundred megabytes covering thousands of chunks and several thousand
-metrics each. Decoding all of it is CPU-bound and takes minutes, so these
-commands **default to the last 4 hours** and skip older chunks before
-decompressing them. On a 250 MB directory that is the difference between about
-a second and about a minute. Use `--last`/`--from`/`--to` to move the window,
-and `--all` when you really do want the whole history.
-
-```bash
-mdbkit ftdc summary /var/lib/mongodb/diagnostic.data
-mdbkit ftdc timeline diagnostic.data --metric conns.current --step 300
-mdbkit ftdc export diagnostic.data > metrics.csv
-```
-
-Metric labels include `ops.*` (insert/query/update/delete/getmore/command),
-`conns.current`, `conns.available`, `queue.readers`, `queue.writers`,
-`cache.usedBytes`, `cache.maxBytes`, `cache.dirtyBytes`, `tickets.*`,
-`mem.residentMB`, and on Linux `sys.cpu.*` and `sys.mem.availableKB`.
-
-The data directory can be copied off the host and analyzed elsewhere — it
-contains metrics only, never document contents.
-
----
-
-### Shareable reports — `--report FILE`
-
-`triage` and `queries` can write a self-contained report instead of printing to
-the terminal — for a ticket, a handover, or a post-incident review.
-
-```bash
-mdbkit triage mongod.log --report incident.html     # styled, self-contained
-mdbkit triage mongod.log --report incident.md       # for tickets and PRs
-mdbkit queries mongod.log --limit 20 --report slow-queries.md
-```
-
-The format follows the file extension: `.html` or `.md`.
-
-Markdown output looks like this:
-
-```markdown
-# MongoDB incident triage
-
-*window 2026-07-01 08:10 -> 09:10  ·  generated 2026-07-01 09:12*
-
-## Findings
-
-- **[CRIT] Replica set instability** — 3 election/stepdown event(s) at 08:41:02, 08:58:14
-    - Starting an election, since we've seen no PRIMARY in election timeout period
-    - *next:* `Correlate with connection storms and slow checkpoints below`
-- **[WARN] Connection storm** — 2 minute(s) at >= 60 new connections/min; peak 480 at 08:41
-    - 10.2.1.7: 312 in the peak minute
-    - *next:* `mdbkit connections <log>`
-- **[OK] Errors** — No error/fatal severity lines in window.
-```
-
-The HTML version carries the same content with a dark, print-friendly
-stylesheet. It is **fully self-contained**: inline CSS, no JavaScript, no
-external assets or CDN references, so it opens on an air-gapped machine and
-sends nothing anywhere.
-
-Reports contain the same information as the terminal output — query **shapes**
-and metrics, never literal values from your documents.
-
----
-
-### `mdbkit demo`
-
-Generates a realistic MongoDB structured log so you can evaluate mdbkit — or
-run a live demo — without a cluster. Output is deterministic for a given
-seed, so a demo behaves identically every time, including on a projector.
-
-| Option | Default | Description |
-|---|---|---|
-| `--scenario` | `mixed` | `healthy`, `incident`, or `mixed` |
-| `--minutes N` | 90 | How much log time to generate |
-| `--seed N` | 7 | Same seed produces byte-identical output |
-| `-o, --out FILE` | stdout | Write to a file |
-| `--with-extras` | off | Also write `indexes.json`, `schema.json` and `explain.json` beside the log |
-
-```bash
-mdbkit demo -o demo.log                          # 90 minutes, mixed
-mdbkit demo --scenario incident --minutes 30 -o incident.log
-mdbkit demo --scenario healthy -o quiet.log      # nothing wrong: the control case
-mdbkit demo | mdbkit queries -                   # straight down a pipe
-```
-
-The `incident` scenario contains an index build, a connection storm from a
-single client, a replica set election, plan-executor errors, a slow
-WiredTiger checkpoint, and a burst of unindexed queries afterwards — the
-shape of a real bad afternoon.
-
----
-
-### `mdbkit lab`
-
-Starts a **disposable local MongoDB** for testing, reproducing a slow query,
-or rehearsing a demo. This is the only command that starts external
-processes; see [SECURITY.md](SECURITY.md) for exactly how it is bounded.
-
-Requires `mongod` on your `PATH` (and `mongosh` to initiate the replica set
-and seed data). Linux and macOS.
-
-| Action | What it does |
-|---|---|
-| `start` | Create and start a replica set, print the connection string and log paths |
-| `seed` | Insert sample data and run a workload with deliberately interesting queries |
-| `status` | Show ports, pids and whether each node is running |
-| `logs` | Print the log file paths, ready to pipe into other commands |
-| `stop` | Stop the nodes, keep the data |
-| `destroy` | Stop and delete the lab (requires `--yes`) |
-
-| Option | Default | Description |
-|---|---|---|
-| `--dir PATH` | `~/.mdbkit-lab` | Where the lab lives |
-| `--nodes N` | 3 | Replica set size |
-| `--port N` | 28110 | Base port — deliberately far from 27017 |
-| `--slowms N` | 0 | Log every operation, which is what makes the log worth reading |
-| `--standalone` | off | Single node, no replica set |
-| `--docs N` | 50000 | Documents inserted by `seed` |
-| `--yes` | | Confirm `destroy` |
-
-**The full loop:**
-
-```bash
-mdbkit lab start                    # 3-node replica set on 28110-28112
-mdbkit lab seed                     # sample data + a mixed workload
-
-mdbkit queries $(mdbkit lab logs | head -1)
-mdbkit advise  $(mdbkit lab logs | head -1)
-
-mdbkit lab destroy --yes            # remove everything
-```
-
-**`mdbkit lab logs`** prints the log file path of every node, one per line,
-so it composes with the other commands instead of you hunting for paths:
-
-```bash
-mdbkit lab logs
-# /home/you/.mdbkit-lab/node0/mongod.log
-# /home/you/.mdbkit-lab/node1/mongod.log
-# /home/you/.mdbkit-lab/node2/mongod.log
-
-mdbkit queries $(mdbkit lab logs | head -1)     # just the primary
-mdbkit triage  $(mdbkit lab logs)               # all three as one stream
-mdbkit loginfo $(mdbkit lab logs | sed -n 2p)   # a specific secondary
-```
-
-**A single node**, when you do not need replication — faster to start and it
-does not require `mongosh`:
-
-```bash
-mdbkit lab start --standalone
-mdbkit lab seed --docs 5000
-mdbkit queries $(mdbkit lab logs)
-mdbkit lab destroy --yes
-```
-
-**Several labs side by side**, for example to compare two MongoDB versions or
-keep one running while you break another:
-
-```bash
-mdbkit lab start --dir ~/lab-a --port 28110
-mdbkit lab start --dir ~/lab-b --port 28210 --standalone
-
-mdbkit lab status --dir ~/lab-a
-mdbkit lab destroy --dir ~/lab-b --yes
-```
-
-**Pause without losing data** — `stop` leaves the data directory intact so
-you can start again later; only `destroy` deletes anything:
-
-```bash
-mdbkit lab stop                     # nodes down, data kept
-mdbkit lab start                    # back up with the same data
-mdbkit lab status                   # ports, pids, running or not
-```
-
-**A complete before/after experiment**, which is what `lab` is really for:
-
-```bash
-mdbkit lab start && mdbkit lab seed
-cp $(mdbkit lab logs | head -1) before.log
-
-mongosh --port 28110 --eval \
-  'db.getSiblingDB("shop").orders.createIndex({status:1, createdAt:-1})'
-
-mdbkit lab seed                     # run the workload again with the index
-cp $(mdbkit lab logs | head -1) after.log
-
-mdbkit compare before.log --after after.log
-mdbkit lab destroy --yes
-```
-
-`seed` runs indexed point lookups alongside deliberately unindexed queries —
-an equality-plus-range-plus-sort with no supporting index, an aggregation
-that scans the collection, and updates whose predicate has no index — so the
-log immediately contains something worth analysing.
-
-**Safety.** The lab binds to `127.0.0.1` only, refuses to use or delete any
-directory it did not create, and never touches a MongoDB it did not start.
-It is a laptop and scratch-VM tool, not a deployment tool.
-
----
-
-### `mdbkit oslog [FILE...]`
-
-Scans a system log for the things that affect a database process: OOM kills,
-file-descriptor limits, segmentation faults, filesystem and I/O errors,
-read-only remounts, conntrack exhaustion, and systemd service exits.
-
-With no argument it reads `/var/log/syslog` or `/var/log/messages` if they are
-readable.
-
-| Option | Description |
-|---|---|
-| `--exit-code` | Exit 2 on CRIT, 1 on WARN, else 0 |
-| `--json` | Machine-readable output |
-
-```bash
-mdbkit oslog                                  # whichever system log exists
-mdbkit oslog /var/log/messages
-mdbkit oslog /var/log/syslog.1 /var/log/syslog
-```
-
-**On journald systems** there is no text log to read, and mdbkit does not run
-commands on your behalf. It tells you what to capture instead:
-
-```bash
-journalctl -k --since '4 hours ago' > kern.log
-journalctl -u mongod --since '4 hours ago' >> kern.log
-mdbkit oslog kern.log
-```
-
-The same file can be handed to `triage --oslog`, which correlates it with the
-mongod log — so an unexplained restart at 09:14 lines up with the OOM kill
-that caused it.
-
----
-
-### `mdbkit serverstatus FILE [--after FILE]`
-
-Digests a saved `db.adminCommand({serverStatus: 1})` dump. That command
-returns several hundred fields; this reports the handful that explain a
-struggling server.
-
-| Option | Description |
-|---|---|
-| `--after FILE` | A second dump taken later — turns cumulative counters into true rates |
-| `--report FILE` | Write a shareable `.md` or `.html` report |
-| `--exit-code` | Exit 2 on CRIT, 1 on WARN, else 0 |
-| `--json` | Machine-readable output |
-
-```bash
-mdbkit export-script serverstatus > export_serverstatus.js
-
-mongosh --quiet --host HOST --port PORT \
-  --username USER --password PASS --authenticationDatabase admin \
-  --eval "$(cat export_serverstatus.js)" > status.json
-
-mdbkit serverstatus status.json
-```
-
-What it checks: **concurrency tickets** (exhaustion queues every operation and
-looks like slowness with no slow query to blame), **WiredTiger cache** against
-the 80% background-eviction and 95% application-thread-eviction thresholds,
-**dirty cache**, **application-thread eviction**, **connection headroom**,
-**queued readers and writers**, **assertions**, **flow control**, replication
-role and process memory. Ticket layout is read from either
-`wiredTiger.concurrentTransactions` (pre-7.0) or `queues.execution` (7.0+).
-
-**Two dumps give true rates.** Almost everything in serverStatus is cumulative
-since process start, so a single dump only yields lifetime averages:
-
-```bash
-mongosh ... > before.json ; sleep 60 ; mongosh ... > after.json
-mdbkit serverstatus before.json --after after.json
-```
-
-```
-[INFO] Operation counters: Measured over 60 seconds between the two dumps.
-        - query    742,000 in 60s  (12366.7/sec)
-```
-
-That is real current load. The same counter read from one dump would have
-reported 2,127/sec — the average since the server started ten days ago.
-
----
-
-### `mdbkit compare BEFORE --after AFTER`
-
-Diffs query shapes between two logs and reports what improved, what
-regressed, and what is new. The natural follow-up to `advise`: you created an
-index, a day passed, and this answers whether it worked.
-
-| Option | Default | Description |
-|---|---|---|
-| `--after FILE...` | required | The log(s) from after the change |
-| `--ns NAMESPACE` | all | Compare only one namespace |
-| `--min-count N` | 3 | Ignore shapes seen fewer than N times, so noise in a quiet log does not read as a regression |
-| `--min-ms N` | 0 | Ignore operations faster than this |
-| `--limit N` | 15 | Shapes to print (`0` = all) |
-| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces |
-| `--report FILE` | | Write a shareable `.md` or `.html` report |
-| `--json` | | Machine-readable output |
-
-```bash
-mdbkit compare before.log --after after.log
-mdbkit compare before.log --after after.log --ns shop.orders
-mdbkit compare "old/mongod.log*" --after "new/mongod.log*" --report change.html
-```
-
-```
-slow-query time DOWN 32%  (6.0m -> 4.1m across compared shapes)
-shapes: 1 improved, 0 regressed, 0 new, 0 gone, 4 unchanged
-
-IMPROVED
-  shop.orders {createdAt:gt, status:eq} sort:{createdAt:-1}
-    mean 1.7s -> 33ms (-98%)   scan 2976:1 -> 1:1  [COLLSCAN -> index, in-memory sort gone]
-```
-
-A shape counts as improved or regressed on a plan change (COLLSCAN becoming
-an index scan, or the reverse), on an in-memory sort disappearing, or on mean
-duration moving by more than 20%.
-
----
-
-### `mdbkit export-script {schema|indexes}`
-
-Prints a small `mongosh` script to stdout. **mdbkit never connects to your
-database** — you run these yourself, so you can read exactly what they do
-first. Both are read-only and export **field names and types only, never
-document values**.
-
-```bash
-mdbkit export-script indexes > export_indexes.js
-mdbkit export-script schema  > export_schema.js
-```
-
----
-
-## Roadmap
-
-Terminal output is and will remain first-class — this tool is built for the
-Linux box the database actually runs on.
-
-**Shipped in v0.4:** `compare`, rotated-log globbing, per-shape drill-down —
-on top of v0.3's `demo` and `lab`, and v0.2's FTDC decoding, incident triage,
-query reconstruction and shareable reports.
-
-Next up, roughly in order:
-
-* **Sharded clusters.** `mongos` logs are a different shape, and the classic
-  sharded failure — a query with no shard key fanning out to every shard — is
-  visible in the log. Also chunk migrations, balancer windows and jumbo
-  chunks. Would come with `mdbkit lab --sharded` so it can be tested.
-* **Startup configuration audit.** mongod logs warnings at startup about
-  transparent huge pages, readahead, ulimits, NUMA and filesystem choice.
-  These are classic production misconfigurations and they are already in
-  your log — nothing new needs collecting.
-* **Index usage candidates.** Prefix-redundant indexes (an index on `{a: 1}`
-  when `{a: 1, b: 1}` exists) are worth *examining*, but static analysis is
-  not sufficient grounds to drop one — the planner may still be choosing it.
-  So mdbkit will flag candidates and print an `$indexStats` script to confirm
-  real usage first, never a drop recommendation.
-* **Confirming the FTDC-based checkpoint, eviction and flow-control
-  detectors** against real `diagnostic.data` — see
-  `docs/TESTING-PLAYBOOK.md`. Real logs and metrics very welcome.
-
-mdbkit is validated against real-world structured logs (tens of thousands of
-lines) in addition to its synthetic test fixtures.
-
-## Bugs, feature requests, questions
-
-Please use [GitHub Issues](../../issues) — it keeps problems and fixes public
-so the next person can find them. Real-world log lines that parse wrongly are
-the most valuable bug reports of all (redact literals first!).
-
-## Security
-
-mdbkit is offline by design: the codebase contains no network code, never
-executes or evaluates input, and treats every log line as untrusted data
-(strict JSON parsing only — shell constructors are never evaluated). See
-[SECURITY.md](SECURITY.md) for the reporting process.
-
-## Non-affiliation
-
-mdbkit is an independent community project. It is **not affiliated with,
-endorsed by, or sponsored by MongoDB, Inc.** "MongoDB" is a registered
-trademark of MongoDB, Inc., used here only to describe compatibility.
-
-## License
-
-MIT — see [LICENSE](LICENSE).
+Metadata-Version: 2.4
+Name: mdbkit
+Version: 0.5.2
+Summary: Offline toolkit for MongoDB 4.4+ structured logs: log analysis, slow-query shapes, connection churn, and deterministic index advice. A spiritual successor to mtools' log tools.
+Author: Saqib Ameen Subhan
+License: MIT
+Project-URL: Homepage, https://github.com/saqibameen86/mdbkit
+Project-URL: Repository, https://github.com/saqibameen86/mdbkit
+Project-URL: Documentation, https://github.com/saqibameen86/mdbkit#readme
+Project-URL: Issues, https://github.com/saqibameen86/mdbkit/issues
+Project-URL: Changelog, https://github.com/saqibameen86/mdbkit/releases
+Keywords: mongodb,mongodb-logs,logv2,mtools,mloginfo,mongod,slow-query,slow-query-log,log-analysis,log-parser,index-advisor,dba,sre,devops,ftdc,diagnostic-data,explain-plan,query-performance,database-performance,replica-set,wiredtiger,troubleshooting,incident-response
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: System Administrators
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Database
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# mdbkit
+
+[![CI](https://github.com/saqibameen86/mdbkit/actions/workflows/ci.yml/badge.svg)](https://github.com/saqibameen86/mdbkit/actions)
+[![PyPI](https://img.shields.io/pypi/v/mdbkit.svg)](https://pypi.org/project/mdbkit/)
+[![Python](https://img.shields.io/pypi/pyversions/mdbkit.svg)](https://pypi.org/project/mdbkit/)
+[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+
+**An offline toolkit for MongoDB structured logs** — slow-query analysis,
+deterministic index advice, incident triage, and diagnostic-data decoding.
+For MongoDB 4.4 – 8.0, from the terminal, without connecting to anything.
+
+A spiritual successor to mtools' log tools, which never learned to read the
+JSON log format introduced in 4.4.
+
+```
+namespace    op         count  cumMs  docsEx      scan     plan           shape
+shop.events  aggregate  29     3.2m   25,810,000  98889:1  COLLSCAN+SORT  {tenantId:eq, ts:gte} sort:{ts:-1}
+shop.orders  find       48     1.4m   6,000,000   2976:1   COLLSCAN+SORT  {status:eq, createdAt:gt}
+shop.users   find       31     3.7s   31          1:1      IXSCAN{email}  {email:eq}
+```
+
+Two of those need an index. One is already fine. That distinction is the
+whole point.
+
+---
+
+## Try it right now — no MongoDB required
+
+`mdbkit demo` writes a realistic log containing a real incident, so you can
+evaluate the tool in about a minute without touching a cluster:
+
+```bash
+pip install mdbkit          # macOS has no pip by default — see Install below
+
+mdbkit demo --with-extras -o demo.log     # a log + indexes.json, schema.json, explain.json
+
+mdbkit loginfo demo.log                   # what is in this log?
+mdbkit queries demo.log                   # which query shapes cost the most?
+mdbkit triage demo.log --window 0         # what went wrong, and when?
+mdbkit connections demo.log               # who connected, and did anyone fail to?
+mdbkit advise demo.log --indexes indexes.json --schema schema.json
+mdbkit explain explain.json               # read a saved explain plan
+```
+
+The generated log contains a connection storm from one client, a replica set
+election, an index build, five failed logins from a service account, and an
+aggregation burning 25 million document reads to return 261 documents — then
+`advise` tells you which index fixes it.
+
+Output is deterministic: the same `--seed` always produces the same log, so a
+demo behaves identically every time. Scenarios are `incident`, `healthy` (the
+control case — useful for seeing what "nothing wrong" looks like) and `mixed`.
+
+Ready for a real server? Jump to [the workflows](#the-four-questions-it-answers).
+
+---
+
+## Is it safe to run on a production server?
+
+This is the right question to ask of any tool someone hands you. The honest
+answer, and how to check it yourself.
+
+**What mdbkit never does:**
+
+| | |
+|---|---|
+| Connect to your database | Analysis commands read **files**. There is no driver, no URI, no connection. |
+| Send anything anywhere | There is no network code at all. No telemetry, no update check, no crash reporting. |
+| Change anything | It is strictly read-only. Where an action would help, it **prints the command** for you to review and run. |
+| Execute what it reads | Log lines and explain files are parsed as data with `json.loads`. Nothing is ever evaluated. |
+| Pull in dependencies | Zero runtime dependencies. Nothing in the supply chain but the Python standard library. |
+
+**Verify it yourself in 60 seconds** — this is a small, dependency-free
+codebase specifically so that you can:
+
+```bash
+# 1. No network, no shell-outs, no eval anywhere in the analysis code
+pip show -f mdbkit | head -3
+grep -rn "socket\|urllib\|requests\|http\|eval(\|exec(" $(python -c "import mdbkit,os;print(os.path.dirname(mdbkit.__file__))")
+
+# 2. Confirm it has no dependencies
+pip show mdbkit | grep Requires
+
+# 3. Watch it make no connections while it runs (Linux)
+strace -f -e trace=network mdbkit queries mongod.log 2>&1 | grep -c socket
+```
+
+The grep returns nothing for every analysis module. The only file that starts
+a process is `lab.py`, which exists to create a *throwaway test cluster* and
+is documented as an explicit exception below.
+
+**What it does read:** the log file you point it at; optionally
+`diagnostic.data` (metrics only, never documents); optionally `indexes.json`
+and `schema.json` that **you** generate with scripts mdbkit prints for you to
+inspect first. On the database host it also reads `/proc` and calls `statvfs`
+for disk and memory figures — nothing that leaves the machine.
+
+**What leaves your machine: nothing.** There is no server to send it to.
+
+**Still cautious?** That is reasonable. Run `mdbkit demo` first and see what
+the output looks like on synthetic data, or `mdbkit lab` to try it against a
+disposable local cluster before you point it at anything real. Both exist for
+exactly this reason.
+
+Full detail: [SECURITY.md](SECURITY.md).
+
+---
+
+## Install
+
+### macOS
+
+macOS ships no `pip` and no `pipx`, so `pip install` fails out of the box.
+Pick whichever of these matches your setup.
+
+**With Homebrew (recommended — keeps mdbkit in its own environment):**
+```bash
+brew install pipx
+pipx ensurepath          # then open a new terminal window
+pipx install mdbkit
+```
+
+**Without Homebrew, using the Python that comes with macOS:**
+```bash
+python3 --version        # accept the Command Line Tools prompt if it appears
+python3 -m pip install --user mdbkit
+```
+
+Then put the install location on your `PATH` — macOS does not do this for
+you:
+```bash
+echo 'export PATH="$HOME/Library/Python/'"$(python3 -c 'import sys;print(f"{sys.version_info.major}.{sys.version_info.minor}")')"'/bin:$PATH"' >> ~/.zshrc
+source ~/.zshrc
+mdbkit --version
+```
+
+**If you use uv:**
+```bash
+uv tool install mdbkit
+```
+
+**If pip reports `externally-managed-environment`**, add
+`--break-system-packages`, or use the pipx route above, which avoids the
+problem entirely.
+
+Do not install with `sudo`. mdbkit is a user-level CLI and needs no
+elevated privileges — not to install, and not to run.
+
+### Linux
+
+**Most distributions:**
+```bash
+pip install mdbkit
+```
+
+**Ubuntu 20.04 / Debian / Amazon Linux 2 (Python 3.8 hosts):**
+```bash
+sudo apt install pipx        # or: sudo dnf install pipx
+pipx install mdbkit
+pipx ensurepath && source ~/.bashrc
+```
+
+**Modern Ubuntu/Debian complaining about "externally-managed-environment":**
+```bash
+pip install mdbkit --break-system-packages
+```
+
+### Windows
+
+```powershell
+py -m pip install mdbkit
+py -m mdbkit --version
+```
+
+If `mdbkit` is not recognised as a command afterwards, the Scripts directory
+is not on your `PATH`; `py -m mdbkit` works regardless.
+
+### Air-gapped database hosts
+
+```bash
+pip download mdbkit -d ./wheels          # on a connected machine
+# copy ./wheels across, then:
+pip install --no-index --find-links ./wheels mdbkit
+```
+
+Zero runtime dependencies means this is a single wheel — nothing else to
+resolve.
+
+### Upgrading
+
+```bash
+pip install --upgrade mdbkit             # or: pipx upgrade mdbkit
+mdbkit --version
+```
+
+Requires Python 3.8+. mdbkit never updates itself and never checks for
+updates — upgrades are always explicit.
+
+> mdbkit is a Python package on PyPI. It is **not** in `apt`/`dnf`/`yum`.
+
+---
+
+## The four questions it answers
+
+Every command reads files or stdin and accepts several files or a glob, so
+rotated logs work as one stream: `mdbkit queries "mongod.log*"`.
+
+### 1. Why is my database slow?
+
+```bash
+mdbkit queries mongod.log                       # shapes ranked by total time
+mdbkit queries mongod.log --sort scanRatio      # worst examined:returned first
+mdbkit queries mongod.log --shape 1             # full detail on one shape
+```
+
+`cumMs` is time summed across **all** occurrences of a shape, not one query.
+`scan` is documents examined per document returned — `1:1` is healthy,
+`98889:1` is a missing index. `plan` shows what MongoDB actually chose.
+
+### 2. What index would fix it?
+
+```bash
+# Optional but much sharper: export what already exists.
+mdbkit export-script indexes > export_indexes.js
+mdbkit export-script schema  > export_schema.js
+
+mongosh --quiet --host your_db_host --port 27017 \
+  --username your_username --password your_password \
+  --authenticationDatabase admin \
+  --eval "$(cat export_indexes.js)" > indexes.json      # repeat for schema
+
+mdbkit advise mongod.log --indexes indexes.json --schema schema.json --ns shop.orders
+```
+
+Every recommendation states the evidence it reasoned from, a confidence
+level, the caveats, and how to validate it. It says *candidate*, not
+*command*, and it never tells you to drop an index.
+
+### 3. What happened at 3am?
+
+```bash
+mdbkit triage /var/log/mongodb/mongod.log       # last 60 minutes by default
+mdbkit triage mongod.log --window 0             # the whole file
+mdbkit triage mongod.log --report incident.html # something to attach to a ticket
+```
+
+Cluster health, elections, connection storms, hot collections, index builds,
+error clusters, slow-query peaks — plus disk, memory, CPU and FTDC metrics
+when run on the database host. Every finding ends with the next command to
+run.
+
+### 4. Did my change actually help?
+
+```bash
+mdbkit compare before.log --after after.log
+```
+
+```
+slow-query time DOWN 32%  (6.0m -> 4.1m across compared shapes)
+shapes: 1 improved, 0 regressed, 0 new, 0 gone, 4 unchanged
+
+IMPROVED
+  shop.orders {createdAt:gt, status:eq} sort:{createdAt:-1}
+    mean 1.7s -> 33ms (-98%)   scan 2976:1 -> 1:1  [COLLSCAN -> index, in-memory sort gone]
+```
+
+The natural follow-up to `advise`: you created the index, a day passed, and
+this tells you whether it worked.
+
+### 5. Why did it die? (and what does serverStatus say?)
+
+```bash
+mdbkit oslog /var/log/syslog                  # OOM kills, fd limits, I/O errors
+mdbkit triage mongod.log --oslog /var/log/syslog
+```
+
+A mongod log cannot record its own OOM kill — the process is gone before it
+can write anything. The system log has the answer in one line, and `triage`
+will correlate it with the unexplained restart.
+
+```bash
+mdbkit export-script serverstatus > export_serverstatus.js
+mongosh --quiet --host HOST --eval "$(cat export_serverstatus.js)" > status.json
+mdbkit serverstatus status.json
+```
+
+```
+[CRIT] Concurrency tickets: Exhausted tickets queue every new operation,
+       which looks like slowness with no slow query to blame.
+        - read: 1 of 128 free (1%)
+[CRIT] WiredTiger cache: Above 80% WiredTiger evicts in the background;
+       above 95% application threads are made to evict.
+        - 31.0 GiB of 32.0 GiB used (96.9%)
+```
+
+**Bonus — who connected?**
+
+```bash
+mdbkit connections mongod.log
+```
+
+Per-IP churn with first/last seen, plus an authenticated-users table showing
+successful and failed logins per account and when each last authenticated —
+the question that starts most access incidents.
+
+---
+
+## Running it on a schedule
+
+mdbkit is deterministic, offline and read-only, which makes it well suited to
+a cron job. It deliberately **cannot send anything anywhere** — there is no
+network code and never will be. So mdbkit produces the verdict and your own
+script does the talking.
+
+The primitive that makes this work is `--exit-code`:
+
+| Exit code | Meaning |
+|---|---|
+| `0` | Nothing above INFO |
+| `1` | At least one WARN |
+| `2` | At least one CRIT |
+
+```bash
+#!/usr/bin/env bash
+# /usr/local/bin/mdbkit-watch.sh — hourly health check
+set -uo pipefail
+
+LOG=/var/log/mongodb/mongod.log
+OUT=$(mktemp)
+
+mdbkit triage "$LOG" --window 60 --oslog /var/log/syslog \
+       --only CRIT,WARN --exit-code > "$OUT" 2>&1
+STATUS=$?
+
+if [ "$STATUS" -ge 2 ]; then
+    # CRIT: wake someone up. Your channel, your call — mdbkit stays offline.
+    curl -sf -X POST -H 'Content-type: application/json' \
+         --data "{\"text\": \"MongoDB CRIT on $(hostname)\n\`\`\`$(cat "$OUT")\`\`\`\"}" \
+         "$SLACK_WEBHOOK_URL"
+elif [ "$STATUS" -eq 1 ]; then
+    mail -s "MongoDB warnings on $(hostname)" dba@example.com < "$OUT"
+fi
+
+rm -f "$OUT"
+```
+
+```cron
+# hourly triage; only speaks up when something is wrong
+0 * * * * /usr/local/bin/mdbkit-watch.sh
+
+# daily slow-query digest, kept for trend comparison
+30 6 * * * mdbkit queries /var/log/mongodb/mongod.log \
+             --report /var/log/mdbkit/$(date +\%F).html
+```
+
+Because the reports are dated, `compare` turns them into a trend:
+
+```bash
+mdbkit compare /var/log/mongodb/mongod.log.1 --after /var/log/mongodb/mongod.log
+```
+
+Two things worth knowing. `--only CRIT,WARN` keeps the mail short, and a run
+that finds nothing prints almost nothing — so a silent cron job means a
+healthy database rather than a broken script. And because every command is
+read-only and never connects to the database, running this hourly on a
+production host costs one log read and no risk.
+
+---
+
+## Trying it against a real cluster
+
+`mdbkit lab` starts a **disposable local MongoDB** so you can test against a
+real server without touching anything that matters.
+
+```bash
+mdbkit lab start                    # 3-node replica set on 127.0.0.1:28110-28112
+mdbkit lab seed                     # 50k documents + a deliberately mixed workload
+mdbkit queries $(mdbkit lab logs | head -1)
+mdbkit lab destroy --yes            # remove it entirely
+```
+
+It binds to localhost only, uses ports far from 27017 so it can never be
+confused with a real deployment, and refuses to touch any directory it did
+not create. It is the one command that starts external processes — see
+[SECURITY.md](SECURITY.md).
+
+Full options and more examples: [`mdbkit lab`](#mdbkit-lab) in the reference.
+
+---
+
+## Command reference
+
+Every command reads files or stdin and writes to stdout. `--help` works on any
+command (`mdbkit queries --help`). Global: `mdbkit --version`.
+
+All commands that read a log accept one or more paths, a shell glob, a
+rotated `.gz` file, or `-` for stdin. Several files are read as a single
+stream in filename order, which matches MongoDB's rotation naming:
+
+```bash
+mdbkit queries mongod.log                       # one file
+mdbkit queries mongod.log.1 mongod.log          # explicit list
+mdbkit queries "mongod.log*"                    # glob (quote it)
+mdbkit queries /var/log/mongodb/mongod.log.*.gz # compressed archives
+cat mongod.log | mdbkit queries -               # stdin
+```
+
+---
+
+### `mdbkit loginfo <log>`
+
+Overall log summary: server version, host, restarts, connections accepted,
+slow-query count, warning/error counts, and a per-component line breakdown.
+
+| Option | Description |
+|---|---|
+| `--json` | Machine-readable output |
+
+```bash
+mdbkit loginfo /var/log/mongodb/mongod.log
+mdbkit loginfo mongod.log.2.gz --json
+```
+
+---
+
+### `mdbkit queries <log>`
+
+Slow queries grouped by **query shape** — literal values stripped, so the same
+query with different parameters is counted once.
+
+| Option | Default | Description |
+|---|---|---|
+| `--sort FIELD` | `totalMs` | Order by `totalMs`, `count`, `mean`, `max`, `docsExamined`, or `scanRatio` |
+| `--limit N` | all | Show only the top N shapes |
+| `--min-ms N` | 0 | Ignore operations faster than N milliseconds |
+| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces (hidden by default — they are server housekeeping, not your workload) |
+| `--report FILE` | | Write a shareable `.md` or `.html` report instead (see [Shareable reports](#shareable-reports----report-file)) |
+| `--json` | | Machine-readable output |
+
+**Reading the columns:**
+
+| Column | Meaning |
+|---|---|
+| `cumMs` | Time summed across **all** occurrences of that shape — not one query |
+| `mean` / `max` | Per-occurrence average and worst case |
+| `docsEx` | Documents examined, summed across all occurrences |
+| `scan` | Documents examined per document returned. `1:1` is ideal; `3444:1` means a missing or weak index |
+| `plan` | The plan MongoDB chose: `COLLSCAN` (no index), `IXSCAN{fields}` (index used), `IDHACK` (`_id` lookup), `+SORT` (in-memory sort). `?` = the plan was not recorded on that line |
+| `shape` | Fields and operators queried, with the sort |
+
+```bash
+mdbkit queries mongod.log
+mdbkit queries mongod.log --sort scanRatio --limit 10
+mdbkit queries mongod.log --min-ms 500 --json
+mdbkit queries "mongod.log*"              # rotated logs as one stream
+mdbkit queries mongod.log --shape 1       # drill into the worst offender
+```
+
+`--shape N` expands one row of the table:
+
+```
+namespace : shop.events
+shape     : {tenantId:eq, ts:gte} sort:{ts:-1}
+
+occurrences   : 29
+total time    : 3.2m
+mean / max    : 6.6s / 8.9s
+docs examined : 25,810,000
+docs returned : 261
+scan ratio    : 98889 examined per document returned
+
+plans observed
+  COLLSCAN                                 29x
+
+flags
+  COLLSCAN — no index used for at least one execution
+  in-memory SORT — results sorted after retrieval
+
+client applications
+  ReportWorker                   15x
+  OrderService                   7x
+```
+
+---
+
+### `mdbkit connections <log>`
+
+Connection churn and **who authenticated**: totals, peak concurrent count,
+per-source-IP breakdown with first/last seen, the client applications and
+drivers, and a per-user table.
+
+| Option | Description |
+|---|---|
+| `--json` | Machine-readable output |
+
+```bash
+mdbkit connections mongod.log
+```
+
+```
+source ip   accepted  ended  first seen           last seen            appName
+----------  --------  -----  -------------------  -------------------  ------------
+10.20.9.77  220       0      2026-07-01 08:49:30  2026-07-01 08:49:30  checkout-api
+10.20.4.11  4         1      2026-07-01 08:00:15  2026-07-01 09:29:30  OrderService
+
+authenticated users
+user          auth db  ok   failed  last authenticated   from
+------------  -------  ---  ------  -------------------  -----------
+svc_checkout  admin    221  0       2026-07-01 08:49:30  10.20.9.77
+etl_batch     admin    0    5       2026-07-01 08:51:54  10.20.11.40
+
+  etl_batch: 5 failed authentication(s) — last error: AuthenticationFailed
+```
+
+This answers the question that starts most access incidents: *did that
+account connect, from where, and when last?* If the log shows no
+authentication events at all, mdbkit says so — either auth is disabled, or
+the window contains no new logins because clients are reusing connections.
+
+---
+
+### `mdbkit filter <log>`
+
+Streams **matching raw log lines** to stdout. Output stays valid logv2 JSON, so
+it chains with other tools (including mdbkit itself).
+
+| Option | Description |
+|---|---|
+| `--component NAME` | `COMMAND`, `NETWORK`, `REPL`, `STORAGE`, `INDEX`, `WRITE`, `QUERY`, `CONTROL`, … |
+| `--severity S` | `I` info, `W` warning, `E` error, `F` fatal |
+| `--ns NAMESPACE` | Exact namespace, e.g. `shop.orders` |
+| `--slow N` | Only operations with `durationMillis` >= N |
+| `--from TIMESTAMP` | Lower time bound (inclusive) |
+| `--to TIMESTAMP` | Upper time bound (inclusive) |
+| `--msg TEXT` | Substring match on the message field |
+| `--limit N` | Print only the **first** N matches |
+| `--last N` | Print only the **last** N matches — usually what you want during an incident |
+| `--as-explain` | Rebuild each matching slow query as a runnable `mongosh` `.explain()` command instead of printing the raw log line |
+| `--explain-script` | With `--as-explain`, wrap in `EJSON.stringify()` plus usage comments so it can be saved as a `.js` file |
+
+**Timestamp formats accepted** by `--from` / `--to`:
+
+```
+2026-07-01T08:00:00+04:00     with an explicit offset (production logs)
+2026-07-01T08:00:00Z          UTC
+2026-07-01T08:00:00           no offset — read as the log's own timezone
+2026-07-01 08:00:00           space instead of T
+2026-07-01T08:00               minute precision
+2026-07-01                     whole day
+```
+
+```bash
+mdbkit filter mongod.log --severity E --last 20    # errors (most recent 20)
+mdbkit filter mongod.log --severity F               # fatal — always investigate
+mdbkit filter mongod.log --severity W --last 50     # warnings
+mdbkit filter mongod.log --component REPL --msg election
+mdbkit filter mongod.log --slow 500 --ns shop.orders --limit 50
+mdbkit filter mongod.log --from 2026-07-01T14:30:00+04:00 --to 2026-07-01T15:00:00+04:00
+mdbkit filter mongod.log --slow 100 | mdbkit queries -
+```
+
+**From a slow query in the log to an explain plan**, without hand-writing the
+query — `--as-explain` rebuilds the command that ran:
+
+```bash
+# See the actual commands behind your slowest operations
+mdbkit filter mongod.log --ns shop.orders --slow 500 --last 3 --as-explain
+
+# Or produce a runnable script, get the plan, and analyze it
+mdbkit filter mongod.log --slow 500 --last 1 --as-explain --explain-script > q.js
+mongosh --quiet --host your_db_host --username your_username \\
+        --password your_password --authenticationDatabase admin \\
+        --eval "$(cat q.js)" > explain.json
+mdbkit explain explain.json
+```
+
+> Rebuilt commands contain the **real values** from your log (not redacted
+> shapes) — treat them as sensitive.
+
+---
+
+### `mdbkit advise <log>`
+
+Deterministic **candidate** index recommendations from observed slow-query
+shapes, using the ESR guideline (Equality → Sort → Range). Rules, not AI: the
+same log always produces the same advice.
+
+| Option | Default | Description |
+|---|---|---|
+| `--indexes FILE` | | `indexes.json` from `mdbkit export-script indexes` — enables overlap checks against existing indexes |
+| `--schema FILE` | | `schema.json` from `mdbkit export-script schema` — enables field-type caveats and confidence adjustment |
+| `--ns NAMESPACE` | all | Only advise on one namespace (recommended on large logs) |
+| `--limit N` | 10 | Show only the top N recommendations (`0` = all) |
+| `--min-ms N` | 0 | Ignore operations faster than N milliseconds |
+| `--min-count N` | 1 | Only advise on shapes seen at least N times |
+| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces |
+| `--json` | | Machine-readable output |
+
+Each recommendation carries a candidate key pattern, the evidence behind it, a
+confidence level, caveats, and a validation step. mdbkit never advises dropping
+an index — at most it flags an overlap to investigate.
+
+```bash
+mdbkit advise mongod.log
+mdbkit advise mongod.log --indexes indexes.json --schema schema.json
+mdbkit advise mongod.log --ns shop.orders --limit 3
+```
+
+---
+
+### `mdbkit explain <file>`
+
+Analyzes a saved `explain("executionStats")` document: the plan chain, the
+examined-vs-returned math, plain-English verdicts, and — when the plan needs
+help — a candidate index from the same advisor engine.
+
+| Option | Description |
+|---|---|
+| `--indexes FILE` | Overlap check against existing indexes |
+| `--schema FILE` | Field-type caveats |
+| `--json` | Machine-readable output |
+
+**Full example.** Get a plan for a query and analyze it:
+
+```bash
+# 1. Capture the plan (adjust host/credentials for your deployment)
+mongosh --quiet \\
+  --host your_db_host \\
+  --port 27017 \\
+  --username your_username \\
+  --password your_password \\
+  --authenticationDatabase admin \\
+  --eval 'EJSON.stringify(db.getSiblingDB("shop").orders.find({status:"open"}).sort({ts:-1}).explain("executionStats"))' \\
+  > explain.json
+
+# 2. Analyze it
+mdbkit explain explain.json
+
+# 3. Sharper, with your existing indexes and sampled schema
+mdbkit explain explain.json --indexes indexes.json --schema schema.json
+```
+
+Don't want to write the query by hand? `mdbkit filter ... --as-explain`
+rebuilds it from the log for you (see the `filter` section above).
+
+Legacy `mongo` shell and Compass output containing `NumberLong(...)`,
+`ISODate(...)` or `ObjectId(...)` is accepted — mdbkit unwraps those
+automatically, so you do not have to re-export.
+
+---
+
+### `mdbkit triage <log>`
+
+**"Triage" means: quickly work out what is wrong and what to look at first.**
+Run this when something has gone wrong — or has just gone wrong — and you need
+one screen that says what happened, how bad it is, and where to look next.
+**Defaults to the last 60 minutes of log time.**
+
+| Option | Default | Description |
+|---|---|---|
+| `--window N` | 60 | Analyze the last N minutes of log time; `0` = the whole file |
+| `--dbpath PATH` | auto | Override the data directory used for the disk check |
+| `--no-sysprobe` | off | Skip local disk/memory/CPU probes — use when analyzing a log copied off the host |
+| `--ftdc PATH` | | `diagnostic.data` directory — adds CPU, memory, cache, queue and connection history from MongoDB's own recorder |
+| `--report FILE` | | Write a shareable `.md` or `.html` report instead of terminal output |
+| `--json` | | Machine-readable output |
+
+```bash
+mdbkit triage /var/log/mongodb/mongod.log
+mdbkit triage mongod.log --window 30
+mdbkit triage mongod.log --ftdc /var/lib/mongodb/diagnostic.data
+mdbkit triage mongod.log --report incident.html
+mdbkit triage mongod.log --window 0 --no-sysprobe
+```
+
+---
+
+### `mdbkit ftdc {summary|timeline|export} <path>`
+
+Decodes `diagnostic.data` — **FTDC (Full-Time Diagnostic Data Capture)**, the
+metrics recorder every mongod already runs. It holds CPU, memory, WiredTiger
+cache, connection, queue and operation history for every node, with no
+monitoring agent installed and no database connection. It is compressed BSON,
+not encrypted; mdbkit decodes it offline.
+
+| Action | Description |
+|---|---|
+| `summary` | min / avg / max / last per metric, plus per-second rates for counters |
+| `timeline` | Values bucketed over time — shows *when* something spiked |
+| `export` | CSV to stdout, for a spreadsheet or your own tooling |
+
+| Option | Default | Description |
+|---|---|---|
+| `--last DURATION` | `4h` | Analyze only the most recent window — `90m`, `4h`, `2d` |
+| `--all` | off | Analyze the entire history (see the performance note below) |
+| `--metric LABEL` | all | Restrict to one metric (repeatable), e.g. `--metric conns.current` |
+| `--step SECONDS` | 60 | Timeline bucket size |
+| `--from` / `--to` | | Explicit time bounds (same formats as `filter`) |
+| `--json` | | Machine-readable output |
+
+**Performance note.** `diagnostic.data` can hold weeks of per-second samples —
+a few hundred megabytes covering thousands of chunks and several thousand
+metrics each. Decoding all of it is CPU-bound and takes minutes, so these
+commands **default to the last 4 hours** and skip older chunks before
+decompressing them. On a 250 MB directory that is the difference between about
+a second and about a minute. Use `--last`/`--from`/`--to` to move the window,
+and `--all` when you really do want the whole history.
+
+```bash
+mdbkit ftdc summary /var/lib/mongodb/diagnostic.data
+mdbkit ftdc timeline diagnostic.data --metric conns.current --step 300
+mdbkit ftdc export diagnostic.data > metrics.csv
+```
+
+Metric labels include `ops.*` (insert/query/update/delete/getmore/command),
+`conns.current`, `conns.available`, `queue.readers`, `queue.writers`,
+`cache.usedBytes`, `cache.maxBytes`, `cache.dirtyBytes`, `tickets.*`,
+`mem.residentMB`, and on Linux `sys.cpu.*` and `sys.mem.availableKB`.
+
+The data directory can be copied off the host and analyzed elsewhere — it
+contains metrics only, never document contents.
+
+---
+
+### Shareable reports — `--report FILE`
+
+`triage` and `queries` can write a self-contained report instead of printing to
+the terminal — for a ticket, a handover, or a post-incident review.
+
+```bash
+mdbkit triage mongod.log --report incident.html     # styled, self-contained
+mdbkit triage mongod.log --report incident.md       # for tickets and PRs
+mdbkit queries mongod.log --limit 20 --report slow-queries.md
+```
+
+The format follows the file extension: `.html` or `.md`.
+
+Markdown output looks like this:
+
+```markdown
+# MongoDB incident triage
+
+*window 2026-07-01 08:10 -> 09:10  ·  generated 2026-07-01 09:12*
+
+## Findings
+
+- **[CRIT] Replica set instability** — 3 election/stepdown event(s) at 08:41:02, 08:58:14
+    - Starting an election, since we've seen no PRIMARY in election timeout period
+    - *next:* `Correlate with connection storms and slow checkpoints below`
+- **[WARN] Connection storm** — 2 minute(s) at >= 60 new connections/min; peak 480 at 08:41
+    - 10.2.1.7: 312 in the peak minute
+    - *next:* `mdbkit connections <log>`
+- **[OK] Errors** — No error/fatal severity lines in window.
+```
+
+The HTML version carries the same content with a dark, print-friendly
+stylesheet. It is **fully self-contained**: inline CSS, no JavaScript, no
+external assets or CDN references, so it opens on an air-gapped machine and
+sends nothing anywhere.
+
+Reports contain the same information as the terminal output — query **shapes**
+and metrics, never literal values from your documents.
+
+---
+
+### `mdbkit demo`
+
+Generates a realistic MongoDB structured log so you can evaluate mdbkit — or
+run a live demo — without a cluster. Output is deterministic for a given
+seed, so a demo behaves identically every time, including on a projector.
+
+| Option | Default | Description |
+|---|---|---|
+| `--scenario` | `mixed` | `healthy`, `incident`, or `mixed` |
+| `--minutes N` | 90 | How much log time to generate |
+| `--seed N` | 7 | Same seed produces byte-identical output |
+| `-o, --out FILE` | stdout | Write to a file |
+| `--with-extras` | off | Also write `indexes.json`, `schema.json` and `explain.json` beside the log |
+
+```bash
+mdbkit demo -o demo.log                          # 90 minutes, mixed
+mdbkit demo --scenario incident --minutes 30 -o incident.log
+mdbkit demo --scenario healthy -o quiet.log      # nothing wrong: the control case
+mdbkit demo | mdbkit queries -                   # straight down a pipe
+```
+
+The `incident` scenario contains an index build, a connection storm from a
+single client, a replica set election, plan-executor errors, a slow
+WiredTiger checkpoint, and a burst of unindexed queries afterwards — the
+shape of a real bad afternoon.
+
+---
+
+### `mdbkit lab`
+
+Starts a **disposable local MongoDB** for testing, reproducing a slow query,
+or rehearsing a demo. This is the only command that starts external
+processes; see [SECURITY.md](SECURITY.md) for exactly how it is bounded.
+
+Requires `mongod` on your `PATH` (and `mongosh` to initiate the replica set
+and seed data). Linux and macOS.
+
+| Action | What it does |
+|---|---|
+| `start` | Create and start a replica set, print the connection string and log paths |
+| `seed` | Insert sample data and run a workload with deliberately interesting queries |
+| `status` | Show ports, pids and whether each node is running |
+| `logs` | Print the log file paths, ready to pipe into other commands |
+| `stop` | Stop the nodes, keep the data |
+| `destroy` | Stop and delete the lab (requires `--yes`) |
+
+| Option | Default | Description |
+|---|---|---|
+| `--dir PATH` | `~/.mdbkit-lab` | Where the lab lives |
+| `--nodes N` | 3 | Replica set size |
+| `--port N` | 28110 | Base port — deliberately far from 27017 |
+| `--slowms N` | 0 | Log every operation, which is what makes the log worth reading |
+| `--standalone` | off | Single node, no replica set |
+| `--docs N` | 50000 | Documents inserted by `seed` |
+| `--yes` | | Confirm `destroy` |
+
+**The full loop:**
+
+```bash
+mdbkit lab start                    # 3-node replica set on 28110-28112
+mdbkit lab seed                     # sample data + a mixed workload
+
+mdbkit queries $(mdbkit lab logs | head -1)
+mdbkit advise  $(mdbkit lab logs | head -1)
+
+mdbkit lab destroy --yes            # remove everything
+```
+
+**`mdbkit lab logs`** prints the log file path of every node, one per line,
+so it composes with the other commands instead of you hunting for paths:
+
+```bash
+mdbkit lab logs
+# /home/you/.mdbkit-lab/node0/mongod.log
+# /home/you/.mdbkit-lab/node1/mongod.log
+# /home/you/.mdbkit-lab/node2/mongod.log
+
+mdbkit queries $(mdbkit lab logs | head -1)     # just the primary
+mdbkit triage  $(mdbkit lab logs)               # all three as one stream
+mdbkit loginfo $(mdbkit lab logs | sed -n 2p)   # a specific secondary
+```
+
+**A single node**, when you do not need replication — faster to start and it
+does not require `mongosh`:
+
+```bash
+mdbkit lab start --standalone
+mdbkit lab seed --docs 5000
+mdbkit queries $(mdbkit lab logs)
+mdbkit lab destroy --yes
+```
+
+**Several labs side by side**, for example to compare two MongoDB versions or
+keep one running while you break another:
+
+```bash
+mdbkit lab start --dir ~/lab-a --port 28110
+mdbkit lab start --dir ~/lab-b --port 28210 --standalone
+
+mdbkit lab status --dir ~/lab-a
+mdbkit lab destroy --dir ~/lab-b --yes
+```
+
+**Pause without losing data** — `stop` leaves the data directory intact so
+you can start again later; only `destroy` deletes anything:
+
+```bash
+mdbkit lab stop                     # nodes down, data kept
+mdbkit lab start                    # back up with the same data
+mdbkit lab status                   # ports, pids, running or not
+```
+
+**A complete before/after experiment**, which is what `lab` is really for:
+
+```bash
+mdbkit lab start && mdbkit lab seed
+cp $(mdbkit lab logs | head -1) before.log
+
+mongosh --port 28110 --eval \
+  'db.getSiblingDB("shop").orders.createIndex({status:1, createdAt:-1})'
+
+mdbkit lab seed                     # run the workload again with the index
+cp $(mdbkit lab logs | head -1) after.log
+
+mdbkit compare before.log --after after.log
+mdbkit lab destroy --yes
+```
+
+`seed` runs indexed point lookups alongside deliberately unindexed queries —
+an equality-plus-range-plus-sort with no supporting index, an aggregation
+that scans the collection, and updates whose predicate has no index — so the
+log immediately contains something worth analysing.
+
+**Safety.** The lab binds to `127.0.0.1` only, refuses to use or delete any
+directory it did not create, and never touches a MongoDB it did not start.
+It is a laptop and scratch-VM tool, not a deployment tool.
+
+---
+
+### `mdbkit oslog [FILE...]`
+
+Scans a system log for the things that affect a database process: OOM kills,
+file-descriptor limits, segmentation faults, filesystem and I/O errors,
+read-only remounts, conntrack exhaustion, and systemd service exits.
+
+With no argument it reads `/var/log/syslog` or `/var/log/messages` if they are
+readable.
+
+| Option | Description |
+|---|---|
+| `--exit-code` | Exit 2 on CRIT, 1 on WARN, else 0 |
+| `--json` | Machine-readable output |
+
+```bash
+mdbkit oslog                                  # whichever system log exists
+mdbkit oslog /var/log/messages
+mdbkit oslog /var/log/syslog.1 /var/log/syslog
+```
+
+**On journald systems** there is no text log to read, and mdbkit does not run
+commands on your behalf. It tells you what to capture instead:
+
+```bash
+journalctl -k --since '4 hours ago' > kern.log
+journalctl -u mongod --since '4 hours ago' >> kern.log
+mdbkit oslog kern.log
+```
+
+The same file can be handed to `triage --oslog`, which correlates it with the
+mongod log — so an unexplained restart at 09:14 lines up with the OOM kill
+that caused it.
+
+---
+
+### `mdbkit serverstatus FILE [--after FILE]`
+
+Digests a saved `db.adminCommand({serverStatus: 1})` dump. That command
+returns several hundred fields; this reports the handful that explain a
+struggling server.
+
+| Option | Description |
+|---|---|
+| `--after FILE` | A second dump taken later — turns cumulative counters into true rates |
+| `--report FILE` | Write a shareable `.md` or `.html` report |
+| `--exit-code` | Exit 2 on CRIT, 1 on WARN, else 0 |
+| `--json` | Machine-readable output |
+
+```bash
+mdbkit export-script serverstatus > export_serverstatus.js
+
+mongosh --quiet --host HOST --port PORT \
+  --username USER --password PASS --authenticationDatabase admin \
+  --eval "$(cat export_serverstatus.js)" > status.json
+
+mdbkit serverstatus status.json
+```
+
+What it checks: **concurrency tickets** (exhaustion queues every operation and
+looks like slowness with no slow query to blame), **WiredTiger cache** against
+the 80% background-eviction and 95% application-thread-eviction thresholds,
+**dirty cache**, **application-thread eviction**, **connection headroom**,
+**queued readers and writers**, **assertions**, **flow control**, replication
+role and process memory. Ticket layout is read from either
+`wiredTiger.concurrentTransactions` (pre-7.0) or `queues.execution` (7.0+).
+
+**Two dumps give true rates.** Almost everything in serverStatus is cumulative
+since process start, so a single dump only yields lifetime averages:
+
+```bash
+mongosh ... > before.json ; sleep 60 ; mongosh ... > after.json
+mdbkit serverstatus before.json --after after.json
+```
+
+```
+[INFO] Operation counters: Measured over 60 seconds between the two dumps.
+        - query    742,000 in 60s  (12366.7/sec)
+```
+
+That is real current load. The same counter read from one dump would have
+reported 2,127/sec — the average since the server started ten days ago.
+
+---
+
+### `mdbkit compare BEFORE --after AFTER`
+
+Diffs query shapes between two logs and reports what improved, what
+regressed, and what is new. The natural follow-up to `advise`: you created an
+index, a day passed, and this answers whether it worked.
+
+| Option | Default | Description |
+|---|---|---|
+| `--after FILE...` | required | The log(s) from after the change |
+| `--ns NAMESPACE` | all | Compare only one namespace |
+| `--min-count N` | 3 | Ignore shapes seen fewer than N times, so noise in a quiet log does not read as a regression |
+| `--min-ms N` | 0 | Ignore operations faster than this |
+| `--limit N` | 15 | Shapes to print (`0` = all) |
+| `--include-system` | off | Include internal `admin`/`config`/`local` namespaces |
+| `--report FILE` | | Write a shareable `.md` or `.html` report |
+| `--json` | | Machine-readable output |
+
+```bash
+mdbkit compare before.log --after after.log
+mdbkit compare before.log --after after.log --ns shop.orders
+mdbkit compare "old/mongod.log*" --after "new/mongod.log*" --report change.html
+```
+
+```
+slow-query time DOWN 32%  (6.0m -> 4.1m across compared shapes)
+shapes: 1 improved, 0 regressed, 0 new, 0 gone, 4 unchanged
+
+IMPROVED
+  shop.orders {createdAt:gt, status:eq} sort:{createdAt:-1}
+    mean 1.7s -> 33ms (-98%)   scan 2976:1 -> 1:1  [COLLSCAN -> index, in-memory sort gone]
+```
+
+A shape counts as improved or regressed on a plan change (COLLSCAN becoming
+an index scan, or the reverse), on an in-memory sort disappearing, or on mean
+duration moving by more than 20%.
+
+---
+
+### `mdbkit export-script {schema|indexes}`
+
+Prints a small `mongosh` script to stdout. **mdbkit never connects to your
+database** — you run these yourself, so you can read exactly what they do
+first. Both are read-only and export **field names and types only, never
+document values**.
+
+```bash
+mdbkit export-script indexes > export_indexes.js
+mdbkit export-script schema  > export_schema.js
+```
+
+---
+
+## Roadmap
+
+Terminal output is and will remain first-class — this tool is built for the
+Linux box the database actually runs on.
+
+**Shipped in v0.4:** `compare`, rotated-log globbing, per-shape drill-down —
+on top of v0.3's `demo` and `lab`, and v0.2's FTDC decoding, incident triage,
+query reconstruction and shareable reports.
+
+Next up, roughly in order:
+
+* **Sharded clusters.** `mongos` logs are a different shape, and the classic
+  sharded failure — a query with no shard key fanning out to every shard — is
+  visible in the log. Also chunk migrations, balancer windows and jumbo
+  chunks. Would come with `mdbkit lab --sharded` so it can be tested.
+* **Startup configuration audit.** mongod logs warnings at startup about
+  transparent huge pages, readahead, ulimits, NUMA and filesystem choice.
+  These are classic production misconfigurations and they are already in
+  your log — nothing new needs collecting.
+* **Index usage candidates.** Prefix-redundant indexes (an index on `{a: 1}`
+  when `{a: 1, b: 1}` exists) are worth *examining*, but static analysis is
+  not sufficient grounds to drop one — the planner may still be choosing it.
+  So mdbkit will flag candidates and print an `$indexStats` script to confirm
+  real usage first, never a drop recommendation.
+* **Confirming the FTDC-based checkpoint, eviction and flow-control
+  detectors** against real `diagnostic.data` — see
+  `docs/TESTING-PLAYBOOK.md`. Real logs and metrics very welcome.
+
+mdbkit is validated against real-world structured logs (tens of thousands of
+lines) in addition to its synthetic test fixtures.
+
+## Bugs, feature requests, questions
+
+Please use [GitHub Issues](../../issues) — it keeps problems and fixes public
+so the next person can find them. Real-world log lines that parse wrongly are
+the most valuable bug reports of all (redact literals first!).
+
+## Security
+
+mdbkit is offline by design: the codebase contains no network code, never
+executes or evaluates input, and treats every log line as untrusted data
+(strict JSON parsing only — shell constructors are never evaluated). See
+[SECURITY.md](SECURITY.md) for the reporting process.
+
+## Non-affiliation
+
+mdbkit is an independent community project. It is **not affiliated with,
+endorsed by, or sponsored by MongoDB, Inc.** "MongoDB" is a registered
+trademark of MongoDB, Inc., used here only to describe compatibility.
+
+## License
+
+MIT — see [LICENSE](LICENSE).