querydig 0.1.1 → 0.2.0

package/README.md

@@ -12,6 +12,7 @@ npx querydig            # scan the current directory
 - **Fast.** Tree-sitter (WASM) parsing in a worker pool: about 2,000 files in under 1 second.
 - **Multi-language.** JavaScript, TypeScript, Python and Go.
 - **Multi-database.** MongoDB (Mongoose, native driver, PyMongo/Motor, mongo-go-driver) and SQL (Prisma, TypeORM, Sequelize, pg/mysql2/knex raw, Django ORM, SQLAlchemy, DB-API cursors, GORM/xorm, `database/sql`).
+- **Live evidence (optional).** Point `--uri` at a MongoDB or Postgres replica to confirm full scans, missing and unused indexes, and slow queries.
 - **CI-ready.** `--fail-on`, baselines, `--diff`, SARIF for GitHub code scanning, Markdown for PR comments, HTML report.
 
 > Requires Node.js 22.12+.
@@ -91,6 +92,70 @@ npm i -D querydig
 
 Built-in noise control: `while`/`for {}` polling loops, batching loops (`skip`/`offset` in the loop), loops over collections or model classes, and array helpers like `arr.find(x => …)` or Python `str.find()` are not flagged.
 
+## Live mode: evidence from your database
+
+Static analysis finds the patterns. Live mode connects **read-only** to MongoDB or Postgres and checks them against reality: how big each collection is, which indexes exist, and what the query planner actually does.
+
+```bash
+npx querydig --uri "mongodb://reader:***@replica-host/shop"
+QUERYDIG_DB_URI="postgres://reader:***@replica-host/shop" npx querydig   # keeps the secret out of shell history
+```
+
+What changes:
+
+- **Promotes** an N+1 whose inner query is a full scan of a large table to critical ("N × full scan of 4.2M docs") and adds the missing index.
+- **Demotes** unbounded reads on tiny lookup tables to low.
+- **Adds live-only findings:**
+
+| ID | Severity | Finds |
+|---|---|---|
+| LV001 | high / critical | Queries in your code that do a COLLSCAN / Seq Scan on 10k+ rows, with the exact index to create |
+| LV002 | medium | Queries that sort in memory |
+| LV003 | high / critical | Slow queries from the MongoDB profiler or `pg_stat_statements`, traced back to the line of code when possible |
+| LV004 | low | Indexes that are never used |
+| LV005 | medium | Postgres foreign keys without an index that your code filters on |
+
+Every live finding shows its evidence:
+
+```text
+ CRITICAL  QD002 Nested query over another query's results (N² risk) [mongodb]
+  src/reports.ts:15:25
+  live: Each iteration does a full scan of orders (300k docs): N × 300k docs read. (high → critical)
+  fix: … Until then, at least add an index: db.orders.createIndex({ userId: 1 })
+```
+
+**Safety model**
+
+- Only metadata and query plans are read by default: collection stats, index lists, `explain('queryPlanner')`, and `EXPLAIN (GENERIC_PLAN)` (PostgreSQL 16+). Queries are **planned, never executed**.
+- `--explain-execute` opts into execution stats (`executionStats` / `EXPLAIN ANALYZE`, SELECTs only), which do run the query. Use it on a replica.
+- Postgres: every statement runs in a read-only transaction (`default_transaction_read_only = on`) with a 5s statement timeout. MongoDB: `secondaryPreferred` reads with `maxTimeMS: 5000`.
+- Passwords are redacted from all output. If the database can't be reached, querydig warns and still reports static findings (`--live-required` makes that an error).
+
+**Recommended read-only users**
+
+```js
+// MongoDB
+db.createUser({ user: 'querydig', pwd: '…', roles: [{ role: 'read', db: 'shop' }, { role: 'clusterMonitor', db: 'admin' }] })
+```
+
+```sql
+-- Postgres
+CREATE ROLE querydig LOGIN PASSWORD '…';
+GRANT pg_read_all_stats TO querydig;
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO querydig;
+```
+
+Slow-query detection needs the MongoDB profiler (`db.setProfilingLevel(1, { slowms: 100 })`) or the `pg_stat_statements` extension. querydig tells you when either is missing.
+
+| Option | Default | |
+|---|---|---|
+| `--uri <uri>` / `QUERYDIG_DB_URI` | | `mongodb://`, `mongodb+srv://`, `postgres://`, `postgresql://` |
+| `--db <name>` | from URI | MongoDB database when the URI has none |
+| `--slow-ms <n>` | 100 | Slow-query threshold |
+| `--explain-execute` | off | Execute explain (replicas only) |
+| `--max-explain <n>` | 200 | Cap on query shapes explained |
+| `--live-required` | off | Exit 2 if the database can't be reached |
+
 ## Suppressing findings
 
 ```ts
@@ -165,7 +230,8 @@ console.log(summary.bySeverity, summary.grade);
 
 ## Roadmap
 
-- **Live mode** (`--uri`, read-only): MongoDB profiler + `explain()` (COLLSCAN, docs-examined ratio, in-memory SORT), missing/unused indexes; Postgres `pg_stat_statements` + `EXPLAIN`; MySQL `performance_schema`. Live evidence will be merged with static findings to promote severity.
+- Live mode for MySQL (`performance_schema` + `EXPLAIN`).
+- Repo-wide index recommendations and PR comments.
 - Java/Kotlin (Hibernate/Spring Data), Ruby (ActiveRecord), PHP (Eloquent).
 - An official GitHub Action and incremental cache.
 

package/dist/analyzer.d.ts

@@ -1,6 +1,18 @@
-import type { Finding, LangId } from './types.js';
+import type { Finding, LangId, QueryShape, SortKey } from './types.js';
 export interface AnalyzeOptions {
     relPath: string;
     lang: LangId;
 }
+export interface AnalyzeResult {
+    findings: Finding[];
+    shapes: QueryShape[];
+}
+export declare function analyzeSourceFull(source: string, opts: AnalyzeOptions): Promise<AnalyzeResult>;
 export declare function analyzeSource(source: string, opts: AnalyzeOptions): Promise<Finding[]>;
+/** Table name and WHERE / ORDER BY columns from a SQL string (best effort). */
+export declare function sqlShape(sql: string): {
+    table?: string;
+    filterKeys: string[];
+    sortKeys: SortKey[];
+    hasLimit: boolean;
+};

package/dist/chunk-DZLU5MLV.js

@@ -0,0 +1,26 @@
+import {
+  analyzeSourceFull,
+  langForFile
+} from "./chunk-YKUITQ6N.js";
+
+// src/fileAnalyzer.ts
+import { readFile } from "fs/promises";
+import { join } from "path";
+var MAX_BYTES = 15e5;
+var QUICK_FILTER = /find|aggregate|query|execute|Query|Exec|objects|count|Count|update|Update|insert|Insert|delete|Delete|save|Save|create|Create|select|SELECT|First|Take|remove|upsert|distinct|Distinct/;
+async function analyzeFile(root, rel) {
+  const lang = langForFile(rel);
+  if (!lang) return { file: rel, findings: [] };
+  try {
+    const src = await readFile(join(root, rel), "utf8");
+    if (src.length > MAX_BYTES || !QUICK_FILTER.test(src)) return { file: rel, findings: [] };
+    const { findings, shapes } = await analyzeSourceFull(src, { relPath: rel, lang });
+    return { file: rel, findings, shapes };
+  } catch (e) {
+    return { file: rel, findings: [], error: e.message };
+  }
+}
+
+export {
+  analyzeFile
+};

package/dist/{chunk-WTZLJ5ZV.js → chunk-P3O2JP5S.js}

@@ -1,9 +1,11 @@
+import {
+  analyzeFile
+} from "./chunk-DZLU5MLV.js";
 import {
   RULES,
   RULE_MAP,
-  SUPPORTED_EXTENSIONS,
-  analyzeFile
-} from "./chunk-VAYEO4RI.js";
+  SUPPORTED_EXTENSIONS
+} from "./chunk-YKUITQ6N.js";
 
 // src/config.ts
 import { existsSync, readFileSync } from "fs";
@@ -103,14 +105,14 @@ function renderSarif(s) {
           }))
         }
       },
-      results: s.findings.map((f) => ({
+      results: s.findings.filter((f) => f.line > 0 && !f.file.startsWith("db://")).map((f) => ({
         ruleId: f.ruleId,
         ruleIndex: rules.findIndex((r) => r.id === f.ruleId),
         level: SARIF_LEVEL[f.severity],
         message: { text: `${f.message} Fix: ${f.fix}` },
         locations: [{ physicalLocation: { artifactLocation: { uri: f.file }, region: { startLine: f.line, startColumn: f.column } } }],
         partialFingerprints: { querydig: f.fingerprint },
-        properties: { severity: f.severity, db: f.db, confidence: f.confidence }
+        properties: { severity: f.severity, db: f.db, confidence: f.confidence, ...f.evidence ? { evidence: f.evidence } : {} }
       }))
     }]
   };
@@ -121,7 +123,7 @@ function renderMarkdown(s, limit = 50) {
   const o = [];
   o.push(`## querydig \u2014 query performance report`);
   o.push("");
-  o.push(`**Health: ${s.grade} (${s.score}/100)** \xB7 ${s.filesScanned} files \xB7 ${s.findings.length} issue(s)` + (s.suppressedByBaseline ? ` \xB7 ${s.suppressedByBaseline} hidden by baseline` : ""));
+  o.push(`**Health: ${s.grade} (${s.score}/100)** \xB7 ${s.filesScanned} files \xB7 ${s.findings.length} issue(s)` + (s.suppressedByBaseline ? ` \xB7 ${s.suppressedByBaseline} hidden by baseline` : "") + (s.live && !s.live.error ? ` \xB7 live: ${s.live.adapter} \`${s.live.database}\`, ${s.live.explained} plans` : ""));
   o.push("");
   o.push("| Severity | Count |");
   o.push("|---|---|");
@@ -134,7 +136,9 @@ function renderMarkdown(s, limit = 50) {
   o.push("| | Rule | Location | Issue |");
   o.push("|---|---|---|---|");
   for (const f of s.findings.slice(0, limit)) {
-    o.push(`| ${EMOJI[f.severity]} | \`${f.ruleId}\` ${RULE_MAP.get(f.ruleId)?.name ?? ""} | \`${f.file}:${f.line}\` | ${f.message.replace(/\|/g, "\\|")} |`);
+    const loc = f.line > 0 ? `${f.file}:${f.line}` : f.file;
+    const ev = f.evidence ? `<br>\u{1F50E} ${f.evidence.note}${f.evidence.severityChange ? ` (${f.evidence.severityChange.from} \u2192 ${f.evidence.severityChange.to})` : ""}` : "";
+    o.push(`| ${EMOJI[f.severity]} | \`${f.ruleId}\` ${RULE_MAP.get(f.ruleId)?.name ?? ""} | \`${loc}\` | ${(f.message + ev).replace(/\|/g, "\\|")} |`);
   }
   if (s.findings.length > limit) o.push(`
 \u2026and ${s.findings.length - limit} more.`);
@@ -152,7 +156,7 @@ function renderMarkdown(s, limit = 50) {
 function renderHtml(s) {
   const used = new Set(s.findings.map((f) => f.ruleId));
   const data = {
-    summary: { root: s.root, files: s.filesScanned, ms: s.durationMs, grade: s.grade, score: s.score, bySeverity: s.bySeverity, baseline: s.suppressedByBaseline, version: s.version, generated: (/* @__PURE__ */ new Date()).toISOString() },
+    summary: { root: s.root, files: s.filesScanned, ms: s.durationMs, grade: s.grade, score: s.score, bySeverity: s.bySeverity, baseline: s.suppressedByBaseline, version: s.version, generated: (/* @__PURE__ */ new Date()).toISOString(), live: s.live ?? null },
     findings: s.findings,
     rules: RULES.filter((r) => used.has(r.id))
   };
@@ -194,6 +198,8 @@ h1{font-size:22px;margin:0 0 4px;letter-spacing:-.01em}
 .loc{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12px;color:var(--muted);margin:4px 0 8px;word-break:break-all}
 pre{background:var(--code);border-radius:8px;padding:10px 12px;overflow-x:auto;margin:8px 0;font:12.5px/1.45 ui-monospace,SFMono-Regular,Menlo,monospace}
 .fix{color:var(--ok)}
+.ev{margin-top:6px;padding:6px 10px;border-radius:8px;background:color-mix(in oklab,#a855f7 12%,transparent);color:var(--ink);font-size:13px}
+.ev b{color:#a855f7;font-weight:600;margin-right:6px}
 .tag{background:var(--chip);border-radius:6px;padding:1px 7px;font-size:11px;color:var(--muted)}
 details{margin-top:6px}summary{cursor:pointer;color:var(--muted)}
 .empty{text-align:center;color:var(--muted);padding:48px 0}
@@ -220,7 +226,7 @@ const D=${json};
 const SEV=['critical','high','medium','low','info'];
 const esc=s=>String(s).replace(/[&<>"']/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
 const S=D.summary, on=new Set(['critical','high','medium','low','info']);
-document.getElementById('sub').textContent=S.root+' \xB7 '+S.files+' files \xB7 '+(S.ms/1000).toFixed(2)+'s'+(S.baseline?' \xB7 '+S.baseline+' hidden by baseline':'');
+document.getElementById('sub').textContent=S.root+' \xB7 '+S.files+' files \xB7 '+(S.ms/1000).toFixed(2)+'s'+(S.baseline?' \xB7 '+S.baseline+' hidden by baseline':'')+(S.live&&!S.live.error?' \xB7 live: '+S.live.adapter+' '+S.live.database+', '+S.live.explained+' query plans':'');
 const gc={A:'var(--ok)',B:'var(--ok)',C:'var(--med)',D:'var(--high)',F:'var(--crit)'}[S.grade];
 document.getElementById('cards').innerHTML='<div class="card grade"><div class="k">Health</div><div class="v" style="color:'+gc+'">'+S.grade+' <small style="font-size:14px;color:var(--muted)">'+S.score+'/100</small></div></div>'+
  ['critical','high','medium','low'].map(k=>'<div class="card"><div class="k">'+k+'</div><div class="v c-'+k+'">'+S.bySeverity[k]+'</div></div>').join('');
@@ -238,7 +244,7 @@ function render(){
  const L=document.getElementById('list');
  if(!fs.length){L.innerHTML='<div class="empty">'+(D.findings.length?'No findings match the filters.':'\u2714 No query performance issues found.')+'</div>';return}
  L.innerHTML=fs.slice(0,1000).map(f=>{const r=RM[f.ruleId]||{};return '<div class="f '+f.severity+'"><h3><span class="sev c-'+f.severity+'">'+f.severity+'</span><span class="rid">'+f.ruleId+'</span>'+esc(f.title)+' <span class="tag">'+f.db+'</span><span class="tag">'+f.confidence+' confidence</span></h3>'+
-  '<div class="loc">'+esc(f.file)+':'+f.line+':'+f.column+'</div><pre>'+esc(f.snippet)+'</pre><div>'+esc(f.message)+'</div><div class="fix"><b>Fix:</b> '+esc(f.fix)+'</div>'+
+  '<div class="loc">'+esc(f.file)+(f.line>0?':'+f.line+':'+f.column:'')+'</div><pre>'+esc(f.snippet)+'</pre><div>'+esc(f.message)+'</div>'+(f.evidence?'<div class="ev"><b>live</b>'+esc(f.evidence.note)+(f.evidence.severityChange?' ('+f.evidence.severityChange.from+' \u2192 '+f.evidence.severityChange.to+')':'')+'</div>':'')+'<div class="fix"><b>Fix:</b> '+esc(f.fix)+'</div>'+
   (r.why?'<details><summary>Why is this slow?</summary><p>'+esc(r.why)+'</p>'+(r.example?'<div>Before</div><pre>'+esc(r.example.bad)+'</pre><div>After</div><pre>'+esc(r.example.good)+'</pre>':'')+'</details>':'')+'</div>'}).join('')+(fs.length>1000?'<div class="empty">Showing first 1000 of '+fs.length+'</div>':'');
 }
 ['q','db'].forEach(id=>document.getElementById(id).addEventListener('input',render));rsel.addEventListener('input',render);
@@ -266,6 +272,10 @@ function renderTerminal(s, opts = {}) {
   out.push("");
   out.push(pc.bold(`querydig ${pc.dim("v" + s.version)}  \u2014  slow-query scan of ${s.root}`));
   out.push(pc.dim(`${s.filesScanned} files in ${(s.durationMs / 1e3).toFixed(2)}s`));
+  if (s.live && !s.live.error) {
+    out.push(pc.dim(`live: ${s.live.adapter} ${s.live.database} \xB7 ${s.live.tables} ${s.live.adapter === "mongodb" ? "collections" : "tables"} \xB7 ${s.live.explained} query plans \xB7 ${s.live.slowQueries} slow queries`));
+  }
+  for (const w of s.live?.warnings ?? []) out.push(pc.yellow(`  ! ${w}`));
   out.push("");
   if (!s.findings.length) {
     out.push(pc.green(pc.bold("\u2714 No query performance issues found.")));
@@ -297,9 +307,13 @@ function renderFinding(f, verbose) {
   const lines = [];
   const db = f.db === "unknown" ? "" : pc.dim(` [${f.db}]`);
   lines.push(`${SEV_COLOR[f.severity](` ${f.severity.toUpperCase()} `)} ${pc.bold(f.ruleId)} ${f.title}${db}`);
-  lines.push(`  ${pc.underline(`${f.file}:${f.line}:${f.column}`)}`);
+  lines.push(`  ${pc.underline(f.line > 0 ? `${f.file}:${f.line}:${f.column}` : f.file)}`);
   for (const l of f.snippet.split("\n").slice(0, 3)) lines.push(pc.dim("  \u2502 ") + l);
   lines.push(`  ${f.message}`);
+  if (f.evidence) {
+    const sc = f.evidence.severityChange ? ` ${pc.bold(`(${f.evidence.severityChange.from} \u2192 ${f.evidence.severityChange.to})`)}` : "";
+    lines.push(`  ${pc.magenta("live:")} ${f.evidence.note}${sc}`);
+  }
   lines.push(`  ${pc.green("fix:")} ${f.fix}`);
   if (verbose) lines.push(pc.dim(`  confidence: ${f.confidence}  fingerprint: ${f.fingerprint}`));
   lines.push("");
@@ -311,7 +325,7 @@ var SEVERITIES = ["critical", "high", "medium", "low", "info"];
 var SEVERITY_RANK = { critical: 4, high: 3, medium: 2, low: 1, info: 0 };
 
 // src/version.ts
-var VERSION = true ? "0.1.1" : "0.0.0-dev";
+var VERSION = true ? "0.2.0" : "0.0.0-dev";
 
 // src/scan.ts
 import { execFileSync } from "child_process";
@@ -406,11 +420,21 @@ async function scan(opts) {
   const threads = opts.threads ?? Math.max(1, availableParallelism() - 1);
   const results = await runPool(root, files, threads, opts.onProgress);
   let findings = [];
+  const shapes = [];
   const errors = [];
   for (const r of results) {
     if (!r) continue;
     if (r.error) errors.push({ file: r.file, error: r.error });
     findings.push(...r.findings);
+    if (r.shapes) shapes.push(...r.shapes);
+  }
+  let live;
+  if (opts.live) {
+    opts.onLiveStart?.();
+    const { runLive } = await import("./live-DSIYWIT4.js");
+    const res = await runLive(findings, shapes, opts.live);
+    findings = res.findings;
+    live = res.summary;
   }
   if (opts.rules) {
     findings = findings.flatMap((f) => {
@@ -454,7 +478,8 @@ async function scan(opts) {
     score,
     suppressedByBaseline,
     errors,
-    version: VERSION
+    version: VERSION,
+    ...live ? { live } : {}
   };
 }
 

package/dist/{chunk-VAYEO4RI.js → chunk-YKUITQ6N.js}

@@ -215,6 +215,52 @@ var RULES = [
     db: "sql",
     why: "NOT IN with a subquery is often executed as a nested loop and has surprising NULL semantics.",
     fix: "Rewrite as NOT EXISTS or a LEFT JOIN ... WHERE x IS NULL."
+  }),
+  // ---------- live database (--uri) ----------
+  r({
+    id: "LV001",
+    name: "full-scan",
+    title: "Query does a full collection / table scan",
+    severity: "high",
+    db: "any",
+    why: "The live query plan shows COLLSCAN / Seq Scan: every document or row is read to answer this query. Cost grows linearly with the table and competes with every other query for I/O and cache.",
+    fix: "Create an index that matches the filter (equality fields first), then the sort fields, then range fields."
+  }),
+  r({
+    id: "LV002",
+    name: "in-memory-sort",
+    title: "Query sorts in memory",
+    severity: "medium",
+    db: "any",
+    why: "The plan contains a blocking SORT stage: all matching rows are buffered and sorted before the first one is returned.",
+    fix: "Add the sort fields to an index after the equality fields, in the same order and direction as the sort."
+  }),
+  r({
+    id: "LV003",
+    name: "slow-query",
+    title: "Slow query recorded by the database",
+    severity: "high",
+    db: "any",
+    why: "The database profiler / pg_stat_statements recorded this query shape above the slow threshold in real traffic.",
+    fix: "Check its plan with explain(), add a matching index, or reduce the documents/rows it has to read."
+  }),
+  r({
+    id: "LV004",
+    name: "unused-index",
+    title: "Index is never used",
+    severity: "low",
+    db: "any",
+    why: "The index has zero recorded uses since statistics were last reset, but every write still has to update it and it takes RAM.",
+    fix: "Confirm across all replicas and a full business cycle, then drop it."
+  }),
+  r({
+    id: "LV005",
+    name: "missing-fk-index",
+    title: "Foreign key column has no index",
+    severity: "medium",
+    db: "sql",
+    why: "Queries that filter or join on this foreign key, and deletes on the parent table, have to scan the whole child table.",
+    fix: "CREATE INDEX CONCURRENTLY on the foreign key column."
   })
 ];
 var RULE_MAP = new Map(RULES.map((x) => [x.id, x]));
@@ -528,16 +574,30 @@ var BUILTIN_ROOTS = /* @__PURE__ */ new Set([
 var DEFAULT_TEST_LOOP_BREAKERS = /* @__PURE__ */ new Set(["while_statement", "do_statement"]);
 var same = (a, b) => !!a && !!b && a.startIndex === b.startIndex && a.endIndex === b.endIndex && a.type === b.type;
 var stripQuotes = (s) => s.replace(/^[rRbBuUfF]{0,2}(['"`]{1,3})([\s\S]*?)\1$/, "$2");
-async function analyzeSource(source, opts) {
-  if (/querydig-ignore-file/.test(source.slice(0, 2e3))) return [];
+async function analyzeSourceFull(source, opts) {
+  if (/querydig-ignore-file/.test(source.slice(0, 2e3))) return { findings: [], shapes: [] };
   const parser = await getParser(opts.lang);
   const tree = parser.parse(source);
   try {
-    return new Analyzer(source, opts, LANGS[opts.lang]).run(tree.rootNode);
+    const a = new Analyzer(source, opts, LANGS[opts.lang]);
+    const findings = a.run(tree.rootNode);
+    return { findings, shapes: a.shapes };
   } finally {
     tree.delete();
   }
 }
+async function analyzeSource(source, opts) {
+  return (await analyzeSourceFull(source, opts)).findings;
+}
+function sqlShape(sql) {
+  const unq = (x) => x.replace(/["`\[\]]/g, "").split(".").pop();
+  const table = /\b(?:from|update|into)\s+([\w."`\[\]]+)/i.exec(sql)?.[1];
+  const where = /\bwhere\b([\s\S]*?)(\border\s+by\b|\bgroup\s+by\b|\blimit\b|\boffset\b|\breturning\b|$)/i.exec(sql)?.[1] ?? "";
+  const filterKeys = [...where.matchAll(/([\w."`]+)\s*(?:=|<>|!=|<=|>=|<|>|\bin\b|\blike\b|\bilike\b|\bis\b|\bbetween\b)/gi)].map((m) => unq(m[1])).filter((k) => !/^(and|or|not|\d+)$/i.test(k));
+  const order = /\border\s+by\s+([\s\S]*?)(\blimit\b|\boffset\b|$)/i.exec(sql)?.[1] ?? "";
+  const sortKeys = order.split(",").map((p) => p.trim()).filter((p) => /^[\w."`]+/.test(p) && !/\(/.test(p)).map((p) => ({ key: unq(p.split(/\s+/)[0]), dir: /\bdesc\b/i.test(p) ? -1 : 1 }));
+  return { table: table ? unq(table) : void 0, filterKeys: [...new Set(filterKeys)], sortKeys, hasLimit: /\blimit\b|\bfetch\s+first\b/i.test(sql) };
+}
 var Analyzer = class {
   constructor(src, opts, spec) {
     this.src = src;
@@ -554,6 +614,7 @@ var Analyzer = class {
   spec;
   lines;
   findings = [];
+  shapes = [];
   queryVars = /* @__PURE__ */ new Set();
   hasMongo;
   hasSql;
@@ -846,6 +907,7 @@ var Analyzer = class {
   // ------------------------------------------------------------- rules
   evaluate(q, ctx) {
     this.trackAssignment(q);
+    this.recordShape(q, ctx);
     const target = q.collection ? `${q.collection}.${q.method}` : `${q.receiver}.${q.method}`;
     const loopInfo = ctx.loopLine ? ` (loop at line ${ctx.loopLine})` : "";
     const paginating = q.chain.some((c) => /^(skip|Skip|offset|Offset)$/.test(c.name) || /^(Limit)$/.test(c.name) && c.args.includes(","));
@@ -875,6 +937,80 @@ var Analyzer = class {
     if (q.db === "mongodb") this.mongoRules(q);
     if (q.db === "sql") this.sqlOrmRules(q);
   }
+  recordShape(q, ctx) {
+    const id = `${this.opts.relPath}:${q.node.startPosition.row + 1}:${q.node.startPosition.column + 1}`;
+    q.shapeId = id;
+    let filterKeys = this.shapeFilterKeys(q);
+    let sortKeys = this.sortKeys(q);
+    let collection = q.collection;
+    let sql;
+    if (q.kind === "raw" && q.args[0] && this.spec.strings.includes(q.args[0].type)) {
+      sql = stripQuotes(q.args[0].text.replace(/\$\{[^}]*\}/g, "?")).replace(/\s+/g, " ").trim();
+      const sh = sqlShape(sql);
+      collection = sh.table;
+      filterKeys = sh.filterKeys;
+      sortKeys = sh.sortKeys;
+    }
+    const hasLimit = q.kind === "read" || this.chainHas(q, /^(limit|Limit|take|first|First|one|get|Take|Last)$/) || /\b(limit|take|SetLimit|first)\b/.test(q.argsText) || !!sql && /\blimit\b/i.test(sql);
+    this.shapes.push({
+      id,
+      db: q.db,
+      orm: q.orm,
+      collection,
+      method: q.method,
+      kind: q.kind,
+      filterKeys,
+      sortKeys,
+      hasLimit,
+      inLoop: ctx.depth > 0,
+      sql,
+      file: this.opts.relPath,
+      line: q.node.startPosition.row + 1
+    });
+  }
+  shapeFilterKeys(q) {
+    const clean = (k) => k.split("__")[0];
+    if (q.orm === "prisma" && q.args[0]?.type === "object") {
+      const where = q.args[0].namedChildren.find((c) => c.type === "pair" && c.childForFieldName("key")?.text === "where");
+      const v = where?.childForFieldName("value");
+      return v ? this.objectKeys(v) : [];
+    }
+    if (this.spec.id === "python" && (q.orm === "django" || q.orm === "sqlalchemy")) {
+      const txt = [q.argsText, ...q.chain.filter((c) => /^(filter|filter_by|get|exclude)$/.test(c.name)).map((c) => c.args)].join(" ");
+      return [...new Set([...txt.matchAll(/(?:^|[(,\s])([a-z_][\w]*)\s*=(?!=)/gi)].map((m) => clean(m[1])))];
+    }
+    if (q.orm === "gorm" || this.spec.id === "go") {
+      const w = q.chain.filter((c) => /^(Where|And|Or)$/.test(c.name)).map((c) => c.args).join(" ");
+      if (w) return sqlShape("select 1 from t where " + stripQuotes(w.replace(/^\(|\)$/g, "").split(",")[0].trim())).filterKeys;
+    }
+    if (/^(findById|findByIdAndUpdate|findByIdAndDelete|findByIdAndRemove|findByPk|UpdateByID|findUnique)$/.test(q.method) && !this.filterKeys(q).length) return q.db === "mongodb" ? ["_id"] : ["id"];
+    return this.filterKeys(q);
+  }
+  /** Sort keys from .sort({ a: -1 }), .sort("a", -1), sort=[("a", -1)], SetSort(bson.D{{"a", -1}}), order_by("-a"). */
+  sortKeys(q) {
+    const out = [];
+    const dirOf = (v) => v && /^(-1|['"]?desc(ending)?['"]?|pymongo\.DESCENDING|DESCENDING)$/i.test(v.trim()) ? -1 : 1;
+    const links = q.chain.filter((c) => /^(sort|Sort|order_by|orderBy|Order)$/.test(c.name)).map((c) => c.args);
+    const fromArgs = /\bSetSort\(([^)]*\))|\bsort\s*=\s*(\[[^\]]*\])/.exec(q.argsText);
+    if (fromArgs) links.push(fromArgs[1] ?? fromArgs[2]);
+    const prismaOrder = /orderBy\s*:\s*(\{[^}]*\}|\[[^\]]*\])/.exec(q.argsText);
+    if (prismaOrder) links.push(prismaOrder[1]);
+    for (const txt of links) {
+      const obj = /\{([^{}]*)\}/.exec(txt);
+      if (obj && /:/.test(obj[1])) {
+        for (const part of obj[1].split(",")) {
+          const [k, v] = part.split(":").map((x) => x.trim().replace(/['"]/g, ""));
+          if (k && /^[\w.]+$/.test(k)) out.push({ key: k, dir: dirOf(v) });
+        }
+        continue;
+      }
+      for (const m of txt.matchAll(/["'](-?)([\w.]+)["']\s*(?:,\s*(-?1|pymongo\.\w+|[A-Z]+))?/g)) {
+        out.push({ key: m[2], dir: m[1] === "-" ? -1 : dirOf(m[3]) });
+      }
+    }
+    const seen = /* @__PURE__ */ new Set();
+    return out.filter((k) => !seen.has(k.key) && seen.add(k.key));
+  }
   /** `collections.map(c => c.deleteMany({}))` hits a different collection each time: not N+1. */
   targetVariesPerIteration(q, ctx) {
     const vars = ctx.loopVars ?? [];
@@ -1122,6 +1258,12 @@ var Analyzer = class {
     if (/^(delete)\b/i.test(txt) && !/\bfrom\b/i.test(txt)) return;
     if (/^insert\b/i.test(txt) && !/\binto\b/i.test(txt)) return;
     const pseudo = { node, top: node, method: "sql", kind: "raw", db: "sql", receiver: "sql", receiverText: "", args: [], argsText: txt, chain: [], confidence: "high" };
+    if (!dynamic) {
+      const sh = sqlShape(txt);
+      const id = `${this.opts.relPath}:${node.startPosition.row + 1}:${node.startPosition.column + 1}:sql`;
+      pseudo.shapeId = id;
+      this.shapes.push({ id, db: "sql", collection: sh.table, method: "sql", kind: /^select|^with/i.test(txt) ? "readMany" : "write", filterKeys: sh.filterKeys, sortKeys: sh.sortKeys, hasLimit: sh.hasLimit, inLoop: false, sql: txt, file: this.opts.relPath, line: node.startPosition.row + 1 });
+    }
     const concatenated = !!node.parent && ["binary_expression", "concatenated_string", "binary_operator"].includes(node.parent.type);
     if (/^select\s+(\w+\.)?\*\s+from/i.test(txt)) this.add("SQ002", node, pseudo, { message: `\`${txt.slice(0, 60)}\` selects every column.` });
     if (/\b(i?like)\s+['"]%/i.test(txt) || /\b(i?like)\s+(concat\(\s*)?['"]%['"]\s*(\|\||,|\+)/i.test(txt)) {
@@ -1165,7 +1307,8 @@ var Analyzer = class {
       message: o.message,
       fix: o.fix ?? rule.fix,
       confidence: o.confidence ?? q.confidence,
-      fingerprint
+      fingerprint,
+      ...q.shapeId ? { shapeId: q.shapeId } : {}
     });
   }
   applySuppressions() {
@@ -1180,29 +1323,13 @@ var Analyzer = class {
   }
 };
 
-// src/fileAnalyzer.ts
-import { readFile } from "fs/promises";
-import { join as join2 } from "path";
-var MAX_BYTES = 15e5;
-var QUICK_FILTER = /find|aggregate|query|execute|Query|Exec|objects|count|Count|update|Update|insert|Insert|delete|Delete|save|Save|create|Create|select|SELECT|First|Take|remove|upsert|distinct|Distinct/;
-async function analyzeFile(root, rel) {
-  const lang = langForFile(rel);
-  if (!lang) return { file: rel, findings: [] };
-  try {
-    const src = await readFile(join2(root, rel), "utf8");
-    if (src.length > MAX_BYTES || !QUICK_FILTER.test(src)) return { file: rel, findings: [] };
-    const findings = await analyzeSource(src, { relPath: rel, lang });
-    return { file: rel, findings };
-  } catch (e) {
-    return { file: rel, findings: [], error: e.message };
-  }
-}
-
 export {
   RULES,
   RULE_MAP,
   findRule,
+  langForFile,
   SUPPORTED_EXTENSIONS,
+  analyzeSourceFull,
   analyzeSource,
-  analyzeFile
+  sqlShape
 };

package/dist/cli.js

@@ -11,11 +11,12 @@ import {
   renderSarif,
   renderTerminal,
   scan
-} from "./chunk-WTZLJ5ZV.js";
+} from "./chunk-P3O2JP5S.js";
+import "./chunk-DZLU5MLV.js";
 import {
   RULES,
   findRule
-} from "./chunk-VAYEO4RI.js";
+} from "./chunk-YKUITQ6N.js";
 
 // src/cli.ts
 import { existsSync as existsSync2, writeFileSync as writeFileSync2 } from "fs";
@@ -44,7 +45,7 @@ function writeBaseline(path, findings) {
 // src/cli.ts
 var program = new Command();
 program.name("querydig").description("Find slow and unoptimized database queries (MongoDB, SQL) in JS/TS, Python and Go code.").version(VERSION);
-program.command("scan", { isDefault: true }).description("Scan a project for slow query patterns").argument("[path]", "project root", ".").addOption(new Option("-f, --format <format>", "output format").choices(["terminal", "json", "sarif", "md", "html"]).default("terminal")).option("-o, --output <file>", "write report to a file (default for html: querydig-report.html)").addOption(new Option("--fail-on <severity>", "exit with code 1 if any finding is at or above this severity").choices([...SEVERITIES, "none"])).addOption(new Option("--min-severity <severity>", "hide findings below this severity").choices(SEVERITIES)).option("--baseline <file>", "ignore findings recorded in this baseline file").option("--update-baseline [file]", "write all current findings to a baseline file and exit 0").option("--diff <ref>", "only scan files changed since a git ref (e.g. origin/main)").option("--include-tests", "also scan tests, fixtures, migrations and seeds", false).option("--ignore <globs...>", "extra glob patterns to ignore").option("--rules <list>", "only report these rule ids (comma separated)").option("--disable <list>", "disable these rule ids (comma separated)").option("-c, --config <file>", "path to a config file").option("--limit <n>", "max findings shown in terminal (0 = all)", "25").option("--threads <n>", "worker threads").option("-v, --verbose", "show confidence, fingerprints and parse errors", false).action(async (path, o) => {
+program.command("scan", { isDefault: true }).description("Scan a project for slow query patterns").argument("[path]", "project root", ".").addOption(new Option("-f, --format <format>", "output format").choices(["terminal", "json", "sarif", "md", "html"]).default("terminal")).option("-o, --output <file>", "write report to a file (default for html: querydig-report.html)").addOption(new Option("--fail-on <severity>", "exit with code 1 if any finding is at or above this severity").choices([...SEVERITIES, "none"])).addOption(new Option("--min-severity <severity>", "hide findings below this severity").choices(SEVERITIES)).option("--baseline <file>", "ignore findings recorded in this baseline file").option("--update-baseline [file]", "write all current findings to a baseline file and exit 0").option("--diff <ref>", "only scan files changed since a git ref (e.g. origin/main)").option("--include-tests", "also scan tests, fixtures, migrations and seeds", false).option("--ignore <globs...>", "extra glob patterns to ignore").option("--rules <list>", "only report these rule ids (comma separated)").option("--disable <list>", "disable these rule ids (comma separated)").option("-c, --config <file>", "path to a config file").option("--limit <n>", "max findings shown in terminal (0 = all)", "25").option("--threads <n>", "worker threads").option("-v, --verbose", "show confidence, fingerprints and parse errors", false).option("--uri <uri>", "read-only connection to MongoDB or Postgres for live evidence (or set QUERYDIG_DB_URI)").option("--db <name>", "MongoDB database name when the URI has none").option("--slow-ms <n>", "slow-query threshold for profiler / pg_stat_statements", "100").option("--explain-execute", "run explain with execution stats (executes queries; use a replica)", false).option("--max-explain <n>", "maximum number of query shapes to explain", "200").option("--live-required", "fail (exit 2) if the live database cannot be reached", false).action(async (path, o) => {
   const root = resolve(path);
   if (!existsSync2(root)) {
     console.error(pc.red(`querydig: path not found: ${root}`));
@@ -61,6 +62,8 @@ program.command("scan", { isDefault: true }).description("Scan a project for slo
   const updating = o.updateBaseline !== void 0;
   const baseline = !updating && baselinePath ? readBaseline(baselinePath) : void 0;
   if (baselinePath && !updating && !baseline) console.error(pc.yellow(`querydig: baseline ${baselinePath} not found \u2014 reporting all findings`));
+  const uri = o.uri ?? process.env.QUERYDIG_DB_URI;
+  const live = uri ? { uri, db: o.db, slowMs: Number(o.slowMs) || 100, explainExecute: !!o.explainExecute, maxExplain: Number(o.maxExplain) || 200, timeoutMs: 5e3 } : void 0;
   const interactive = o.format === "terminal" && process.stderr.isTTY;
   let last = 0;
   const summary = await scan({
@@ -73,6 +76,8 @@ program.command("scan", { isDefault: true }).description("Scan a project for slo
     diff: o.diff,
     baseline,
     threads: o.threads ? Number(o.threads) : void 0,
+    live,
+    onLiveStart: live && process.stderr.isTTY ? () => process.stderr.write(pc.dim("\rconnecting to the database for live evidence\u2026") + "\r") : void 0,
     onProgress: interactive ? (d, t) => {
       const now = Date.now();
       if (now - last > 80 || d === t) {
@@ -81,6 +86,11 @@ program.command("scan", { isDefault: true }).description("Scan a project for slo
       }
     } : void 0
   });
+  if (live && process.stderr.isTTY) process.stderr.write("\x1B[K");
+  if (summary.live?.error) {
+    console.error(pc.yellow(`querydig: live mode skipped: ${summary.live.error}`));
+    if (o.liveRequired) process.exit(2);
+  }
   if (updating) {
     const file = resolve(typeof o.updateBaseline === "string" ? o.updateBaseline : baselinePath ?? ".querydig-baseline.json");
     const n = writeBaseline(file, summary.findings);

package/dist/index.js

@@ -6,12 +6,13 @@ import {
   renderSarif,
   renderTerminal,
   scan
-} from "./chunk-WTZLJ5ZV.js";
+} from "./chunk-P3O2JP5S.js";
+import "./chunk-DZLU5MLV.js";
 import {
   RULES,
   analyzeSource,
   findRule
-} from "./chunk-VAYEO4RI.js";
+} from "./chunk-YKUITQ6N.js";
 export {
   RULES,
   analyzeSource,

package/dist/live/correlate.d.ts

@@ -0,0 +1,14 @@
+import type { Finding, QueryShape } from '../types.js';
+import type { LiveData } from './types.js';
+export declare const THRESHOLDS: {
+    big: number;
+    huge: number;
+    scanMin: number;
+    tiny: number;
+    criticalMs: number;
+};
+/**
+ * Merge live database evidence into static findings and add live-only findings.
+ * Pure: no I/O, so it can be unit tested with fake LiveData.
+ */
+export declare function correlate(findings: Finding[], shapes: QueryShape[], data: LiveData): Finding[];

package/dist/live/index.d.ts

@@ -0,0 +1,13 @@
+import type { Finding, LiveSummary, QueryShape } from '../types.js';
+import type { LiveAdapter, LiveData, LiveOptions } from './types.js';
+export type { LiveOptions } from './types.js';
+export { correlate } from './correlate.js';
+export { redactUri } from './match.js';
+export declare function adapterFor(opts: LiveOptions): LiveAdapter;
+/** Connect, collect evidence for the static shapes, and return it. Always closes the connection. */
+export declare function collectLive(shapes: QueryShape[], opts: LiveOptions, adapter?: LiveAdapter): Promise<LiveData>;
+/** Collect live evidence and merge it into the findings. */
+export declare function runLive(findings: Finding[], shapes: QueryShape[], opts: LiveOptions): Promise<{
+    findings: Finding[];
+    summary: LiveSummary;
+}>;