mongoose-query-find 3.3.0 → 4.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1566 +1,746 @@
1
- # mongoose-query-find
2
-
3
- [![npm version](https://img.shields.io/npm/v/mongoose-query-find)](https://www.npmjs.com/package/mongoose-query-find)
4
- [![license](https://img.shields.io/npm/l/mongoose-query-find)](https://github.com/jsdev-robin/mongoose-query-find/blob/main/LICENSE)
5
- [![mongoose peer](https://img.shields.io/badge/mongoose-%5E8%20%7C%7C%20%5E9-brightgreen)](https://mongoosejs.com)
6
-
7
- Two fluent, chainable, allowlisted query builders for Mongoose — **`QueryFind`** for simple `Model.find()` queries, and **`QueryAggregate`** for anything that needs joins, grouping, timezone-aware date filtering, or advanced pagination. Both are driven directly from URL query parameters with zero boilerplate.
8
-
9
- ---
10
-
11
- ## Table of Contents
12
-
13
- **QueryFind**
14
-
15
- - [Installation](#installation)
16
- - [Quick Start](#quick-start)
17
- - [Constructor](#constructor)
18
- - [Builder Methods](#builder-methods)
19
- - [.allowFields()](#allowfieldsfields-string)
20
- - [.filter()](#filter)
21
- - [.where()](#whereconditions)
22
- - [.globalSearch()](#globalsearchfields-string)
23
- - [.sort()](#sort)
24
- - [.limitFields()](#limitfieldsdefaultfields-string)
25
- - [.populate()](#populatepath-string--populateoptions-select-string)
26
- - [Terminal Method](#terminal-method)
27
- - [.paginate()](#paginate)
28
- - [Query Parameter Reference](#query-parameter-reference)
29
- - [Full Example (Express)](#full-example-express)
30
- - [More Examples](#more-examples)
31
- - [TypeScript Types](#typescript-types)
32
- - [Security](#security)
33
- - [Links](#links)
34
- - [License](#license)
35
-
36
- **QueryAggregate**
37
-
38
- - [Overview](#queryaggregate-overview)
39
- - [When to use QueryAggregate vs QueryFind](#when-to-use-queryaggregate-vs-queryfind)
40
- - [Installation](#installation-1)
41
- - [Quick Start](#quick-start-1)
42
- - [Constructor & Options](#constructor--options-1)
43
- - [Timezone Handling](#timezone-handling)
44
- - [Builder Methods](#builder-methods-1)
45
- - [.allowFields()](#allowfieldsfields-string-1)
46
- - [.dateFields()](#datefieldsfields-string)
47
- - [.objectIdFields()](#objectidfieldsfields-string)
48
- - [.filter()](#filter-1)
49
- - [.where()](#whereconditions-1)
50
- - [.range()](#rangefield-string-preset-rangepreset)
51
- - [Custom Range Methods](#custom-range-methods)
52
- - [.rangeCustomDay()](#rangecustomdayfield-string-date-string)
53
- - [.rangeBetween()](#rangebetweenfield-string-custom-customdaterangeinput)
54
- - [.rangeLastNDays()](#rangelastndaysfield-string-days-number)
55
- - [.rangeLastNHours()](#rangelastnhoursfield-string-hours-number)
56
- - [.rangeHours()](#rangehoursfield-string-custom-customhourrangeinput)
57
- - [.rangeCustom()](#rangecustomfield-string-custom-customdatetimerangeinput)
58
- - [.excludeSoftDeleted()](#excludesoftdeletedfield-deletedat)
59
- - [.whereIf()](#whereifcondition-conditions)
60
- - [.globalSearch()](#globalsearchfields-string-1)
61
- - [.sort()](#sort-1)
62
- - [.project()](#projectdefaultprojection-string--recordstring-0--1)
63
- - [.lookup()](#lookupopts-lookupoptions)
64
- - [.computeField()](#computefieldname-string-expression-unknown)
65
- - [.addStage()](#addstagestage-pipelinestage)
66
- - [Terminal & Utility Methods](#terminal--utility-methods)
67
- - [.paginate()](#paginatecachedtotal-number)
68
- - [.paginateCursor()](#paginatecursoropts-cursorpaginateoptions)
69
- - [.count()](#count)
70
- - [.distinct()](#distinctfield-string)
71
- - [.stats()](#statsspecs-statspec)
72
- - [.stream()](#stream)
73
- - [.explain()](#explainverbosity)
74
- - [.clone()](#clone)
75
- - [.debugPipeline()](#debugpipeline)
76
- - [Recipes](#recipes)
77
- - [Pipeline Stage Ordering](#pipeline-stage-ordering)
78
- - [Security](#security-1)
79
- - [TypeScript Types](#typescript-types-1)
80
- - [Links](#links-1)
81
- - [License](#license-1)
82
-
83
- ---
1
+ # QueryAnalytics
2
+
3
+ A purpose-built analytics/reporting layer for Mongoose (`^8 || ^9`), sitting alongside `QueryFind` and `QueryAggregate`. Where `QueryAggregate` is a general-purpose, client-input-safe pipeline builder, `QueryAnalytics` is a **server-authored dashboard toolkit**: time series, cohort retention, funnels, sessionization, percentiles, and more, each backed by a native MongoDB 7.0+ operator instead of a hand-rolled pipeline.
4
+
5
+ ## QueryAnalytics Overview
6
+
7
+ `QueryAnalytics` wraps `Model.aggregate()` with dashboard-shaped methods that return ready-to-chart data instead of raw pipeline rows:
8
+
9
+ - `timeSeries()` / `cumulativeSeries()` — bucketed metrics over time, gap-filled, with a native running total
10
+ - `compare()` — period-over-period totals and % change for preset windows (today, this week, last 30 days, ...)
11
+ - `breakdown()` / `rankedBreakdown()` / `multiBreakdown()` — top-N by one or two dimensions, with an optional native dense rank
12
+ - `funnel()` — step-by-step conversion counts
13
+ - `cohortRetention()` — a full cohort-by-cohort retention matrix
14
+ - `sessionize()` — gap-based event sessionization
15
+ - `percentiles()` / `stdDevOutliers()` — native `$percentile` / `$stdDevPop` statistics
16
+ - `metrics()` — several independent totals in a single round trip
17
+ - `heatmap()` — weekday × hour-of-day activity grid
18
+ - `movingAverage()` / `trend()` / `forecast()` — pure-JS post-processing on a series you already fetched
19
+ - `watch()` — a live change-stream feed for dashboards that update in real time
20
+ - `toCSV()` / `toNDJSON()` / `dashboard()` — export and batch helpers
21
+
22
+ It targets **MongoDB 7.0+** and uses `$setWindowFields`, `$denseRank`, `$shift`, `$stdDevPop`, and `$percentile` directly — there's no fallback path for older servers, so calling `cumulativeSeries()`, `rankedBreakdown()`, `sessionize()`, `percentiles()`, or `stdDevOutliers()` against a pre-7.0 deployment will fail at the server, not in this library.
23
+
24
+ ## When to use QueryAnalytics
25
+
26
+ | Need | Use |
27
+ | ---------------------------------------------------------------- | ---------------- |
28
+ | Filtered/sorted/paginated list from client query params | `QueryFind` |
29
+ | Joins, grouping, or a client-driven filtered data page | `QueryAggregate` |
30
+ | A chart-ready time series with gaps filled in | `QueryAnalytics` |
31
+ | Week-over-week / month-over-month KPI comparisons | `QueryAnalytics` |
32
+ | Top-N breakdowns, ranked leaderboards, or a two-dimension pivot | `QueryAnalytics` |
33
+ | Signup → activation → purchase funnel counts | `QueryAnalytics` |
34
+ | "% of signups still active N weeks later" retention tables | `QueryAnalytics` |
35
+ | Turning a raw event stream into user sessions | `QueryAnalytics` |
36
+ | p50/p90/p99 latency or order-value percentiles | `QueryAnalytics` |
37
+ | Flagging statistical outliers for an anomalies widget | `QueryAnalytics` |
38
+ | A whole dashboard's worth of metrics in one call from your route | `QueryAnalytics` |
39
+ | Pushing live updates to a dashboard instead of polling | `QueryAnalytics` |
40
+
41
+ `QueryAnalytics` is **not** a client-input parser. Unlike `QueryFind`/`QueryAggregate`, it has no `.filter()`, no `.allowFields()`, and no operator-rejection pass over untrusted input — see [Security](#security-2) below.
84
42
 
85
43
  ## Installation
86
44
 
87
- ```bash
88
- npm install mongoose-query-find
89
- ```
45
+ Ships from the same package as `QueryFind` and `QueryAggregate`:
90
46
 
91
47
  ```bash
92
- yarn add mongoose-query-find
93
- ```
94
-
95
- ```bash
96
- pnpm add mongoose-query-find
48
+ npm install mongoose-query-find
97
49
  ```
98
50
 
99
- > **Peer dependency:** requires `mongoose ^8` or `mongoose ^9` installed in your project.
100
-
101
- ---
102
-
103
51
  ## Quick Start
104
52
 
105
53
  ```ts
106
- import { queryFind } from 'mongoose-query-find';
107
- import UserModel from './models/user';
108
-
109
- const result = await queryFind(UserModel.find(), req.query)
110
- .allowFields(['name', 'email', 'role', 'createdAt'])
111
- .where({ deletedAt: null })
112
- .filter()
113
- .globalSearch(['name', 'email'])
114
- .sort()
115
- .limitFields('-password -__v')
116
- .paginate();
117
- ```
118
-
119
- `result` will look like:
120
-
121
- ```json
122
- {
123
- "data": [...],
124
- "total": 84,
125
- "page": 2,
126
- "totalPages": 9,
127
- "limit": 10,
128
- "hasNextPage": true,
129
- "hasPrevPage": true
130
- }
131
- ```
54
+ import { QueryAnalytics } from 'mongoose-query-find';
55
+ import OrderModel from './models/order';
132
56
 
133
- ---
57
+ const qa = new QueryAnalytics(OrderModel, { timezone: 'Asia/Dhaka' })
58
+ .match({ branchId, status: { $ne: 'CANCELLED' } })
59
+ .dateRange('createdAt', { from: '2026-08-01', to: '2026-08-31' });
134
60
 
135
- ## Constructor
136
-
137
- ```ts
138
- new QueryFind(query, queryString, options?);
139
- // or use the factory function (recommended):
140
- queryFind(query, queryString, options?);
61
+ const series = await qa.timeSeries({
62
+ dateField: 'createdAt',
63
+ interval: 'day',
64
+ metrics: [
65
+ { name: 'revenue', op: 'sum', field: 'total' },
66
+ { name: 'orders', op: 'count' },
67
+ ],
68
+ });
141
69
  ```
142
70
 
143
- | Parameter | Type | Description |
144
- | ------------- | ----------------------------------- | ----------------------------------------------------- |
145
- | `query` | `Query<TRawDocType[], TRawDocType>` | A Mongoose query, e.g. `Model.find()` |
146
- | `queryString` | `QueryParams` | The parsed URL query object, e.g. `req.query` |
147
- | `options` | `QueryFindOptions` _(optional)_ | Configuration options (see [Options](#options) below) |
71
+ `series` is a gap-filled array ready to hand straight to a charting library:
148
72
 
149
- ### Options
150
-
151
- | Option | Type | Default | Description |
152
- | ---------------------- | ---------------------------------------- | ----------- | ---------------------------------------------------------------------- |
153
- | `maxTimeMS` | `number` | `5000` | Max milliseconds MongoDB may spend on each query. Pass `0` to disable. |
154
- | `slowQueryThresholdMS` | `number` | `maxTimeMS` | Threshold above which `onSlowQuery` fires. |
155
- | `maxLimit` | `number` | `100` | Hard cap on `?limit=`. |
156
- | `lean` | `boolean` | `true` | Whether `.find()` calls `.lean()` for faster plain-object reads. |
157
- | `onSlowQuery` | `(info: SlowQueryInfo) => void` | — | Callback fired when a query exceeds `slowQueryThresholdMS`. |
158
- | `onSanitizeDrop` | `(path: string, value: unknown) => void` | — | Callback fired whenever sanitization drops a client-supplied value. |
159
-
160
- ---
161
-
162
- ## Builder Methods
163
-
164
- All builder methods return `this` and are fully chainable. The recommended call order is:
165
-
166
- ```
167
- allowFields → where → filter → globalSearch → sort → limitFields → populate → paginate
73
+ ```json
74
+ [
75
+ { "bucket": "2026-08-01T00:00:00.000Z", "revenue": 4820, "orders": 61 },
76
+ { "bucket": "2026-08-02T00:00:00.000Z", "revenue": 0, "orders": 0 },
77
+ { "bucket": "2026-08-03T00:00:00.000Z", "revenue": 5310, "orders": 74 }
78
+ ]
168
79
  ```
169
80
 
170
- ---
171
-
172
- ### `.allowFields(fields: string[])`
173
-
174
- Declares which fields may appear in URL filters, sort parameters, and field projections. Acts as an allowlist — any field not listed is silently stripped from client input before it reaches MongoDB.
175
-
176
- **Call this before `.filter()`, `.sort()`, and `.limitFields()`.**
81
+ ## Constructor & Options
177
82
 
178
83
  ```ts
179
- .allowFields(['name', 'email', 'role', 'createdAt'])
84
+ new QueryAnalytics<T>(model, options?);
180
85
  ```
181
86
 
182
- ---
87
+ | Parameter | Type | Description |
88
+ | --------- | ----------------------- | --------------------------- |
89
+ | `model` | `Model<T>` | A Mongoose model |
90
+ | `options` | `QueryAnalyticsOptions` | Configuration, all optional |
183
91
 
184
- ### `.filter()`
92
+ ### `QueryAnalyticsOptions`
185
93
 
186
- Parses the URL query string into a Mongoose filter. Automatically:
94
+ | Option | Type | Default | Description |
95
+ | -------------- | --------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
96
+ | `timezone` | `string` | `'Asia/Dhaka'` | IANA timezone for bucketing and range math. Invalid values are validated once at construction and silently fall back to the default. |
97
+ | `maxTimeMS` | `number` | `8000` | Max time MongoDB may spend per query. Higher than `QueryFind`/`QueryAggregate`'s default since analytics pipelines tend to be heavier. `0` disables. |
98
+ | `allowDiskUse` | `boolean` | `false` | Forwarded to `.allowDiskUse()` for large `$group`/`$setWindowFields` stages. |
99
+ | `weekStartsOn` | `0–6` | `6` (Saturday) | Used by weekly bucketing (`interval: 'week'`), `compare({ preset: 'thisWeek' })`, and `cohortRetention()`. |
100
+ | `cacheTTLMs` | `number` | `0` | TTL in ms for the built-in in-memory cache. `0` disables caching. **See note below.** |
187
101
 
188
- - Strips reserved keys (`page`, `limit`, `sort`, `fields`, `q`)
189
- - Converts comparison operator names to MongoDB `$` syntax (`gt` → `$gt`, `lte` → `$lte`, etc.)
190
- - Recursively coerces string booleans to real booleans (`"true"` → `true`, `"false"` → `false`)
191
- - Coerces date-like strings to `Date` instances for fields named `createdAt`, `updatedAt`, `deletedAt`, `date`, `birthDate`, or `expiresAt`
192
- - Recursively validates and rejects banned operators (`$where`, `$expr`, `$function`, etc.)
102
+ > **Note on `cacheTTLMs`:** the class ships a small in-memory TTL cache (`cacheGet`/`cacheSet`) and a `QueryAnalytics.clearCache()` static, but as of this release none of the public methods (`timeSeries`, `breakdown`, `metrics`, etc.) pass a cache key into it — so setting `cacheTTLMs` doesn't yet cache anything on its own. Treat it as reserved for a future release; if you need caching today, memoize at the call site (e.g. wrap `dashboard()` calls in your own short-lived cache).
193
103
 
194
- **Supported operators:** `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`
104
+ ## Base Filter Methods
195
105
 
196
- ```
197
- GET /users?age[gte]=18&isActive=true&role=admin
198
- ```
106
+ ### `.match(conditions)`
199
107
 
200
- ```ts
201
- .filter()
202
- // → { age: { $gte: 18 }, isActive: true, role: 'admin' }
203
- ```
204
-
205
- ---
206
-
207
- ### `.where(conditions)`
208
-
209
- Applies mandatory server-side conditions that the URL **cannot override**. Use this for multi-tenancy, soft-delete exclusion, and any security-critical constraints.
108
+ Merges hard, developer-authored conditions into the base `$match`. Call as many times as needed — later calls merge over earlier ones on the same key via `Object.assign`.
210
109
 
211
110
  ```ts
212
- .where({ orgId: req.user.orgId, deletedAt: null })
111
+ qa.match({ branchId, status: { $ne: 'CANCELLED' } });
213
112
  ```
214
113
 
215
- > `.where()` always wins — conditions are merged after `.filter()` and will overwrite any conflicting URL params.
216
-
217
- ---
114
+ Because `QueryAnalytics` has no `.filter()`, `match()` is the _only_ way conditions enter the pipeline besides `.dateRange()` and `.addStage()` — there's no allowlist because there's no untrusted input path. Don't pass raw `req.query` into it directly; validate/whitelist any client-influenced value yourself first (see [Security](#security-2)).
218
115
 
219
- ### `.globalSearch(fields: string[])`
116
+ ### `.dateRange(field: string, range: { from: string; to: string })`
220
117
 
221
- Adds a case-insensitive `$or` regex search across the specified fields when the `?q=` query parameter is present. If `q` is absent or exceeds 200 characters, this method is a no-op.
222
-
223
- Search terms are regex-escaped to prevent ReDoS attacks.
224
-
225
- ```
226
- GET /users?q=john
227
- ```
118
+ Hard, inclusive local-date range condition on `field`, converted to a UTC `$gte`/`$lt` window using the instance's timezone.
228
119
 
229
120
  ```ts
230
- .globalSearch(['name', 'email'])
231
- // → { $or: [{ name: /john/i }, { email: /john/i }] }
121
+ qa.dateRange('createdAt', { from: '2026-08-01', to: '2026-08-31' });
232
122
  ```
233
123
 
234
- If a `$or` clause already exists in the filter (e.g. from `.filter()`), both are safely merged under `$and`.
235
-
236
- ---
237
-
238
- ### `.sort()`
124
+ Throws `QueryAnalyticsError` if `from`/`to` aren't `YYYY-MM-DD`.
239
125
 
240
- Applies sort order from the `?sort=` query parameter. Prefix a field with `-` for descending order. Multiple fields are comma-separated, capped at 5 fields.
241
-
242
- ```
243
- GET /users?sort=-createdAt,name
244
- ```
245
-
246
- ```ts
247
- .sort()
248
- // → sorts by createdAt DESC, then name ASC
249
- ```
250
-
251
- Defaults to `{ createdAt: -1 }` when the `sort` param is absent. Fields not in the allowlist are silently skipped.
252
-
253
- ---
254
-
255
- ### `.limitFields(defaultFields?: string)`
256
-
257
- Controls which fields are returned (projection). Uses the `?fields=` query param when present, otherwise falls back to `defaultFields`.
258
-
259
- Fields requested via `?fields=` are filtered against the allowlist — clients cannot project sensitive fields like `password` or `resetToken`.
260
-
261
- **Priority order:**
262
-
263
- 1. `?fields=` query param — always wins when present (allowlist-filtered)
264
- 2. `defaultFields` argument — used as fallback when no query param
265
- 3. No projection when both are absent (all fields returned)
266
-
267
- ```
268
- GET /users?fields=name,email,role
269
- ```
270
-
271
- ```ts
272
- .limitFields('-password -__v')
273
- // With ?fields=name,email,role → selects only name, email, role
274
- // Without ?fields → excludes password and __v
275
- ```
276
-
277
- ---
278
-
279
- ### `.populate(path: string | PopulateOptions, select?: string)`
280
-
281
- Registers a populate directive. Chainable — each call appends to the internal list. All registered populates are applied inside `paginate()`.
126
+ ### `.addStage(stage: PipelineStage)`
282
127
 
283
- Accepts the same arguments as Mongoose's own `.populate()`:
128
+ Escape hatch for extra developer-authored stages, spliced in right after the base `$match`, before any analytics-specific stages. Blocks `$out`/`$merge` recursively, at any nesting depth.
284
129
 
285
130
  ```ts
286
- // Plain path string
287
- .populate('author')
288
-
289
- // Path + select string
290
- .populate('author', 'name email')
291
-
292
- // Full PopulateOptions object
293
- .populate({ path: 'comments', select: 'text createdAt', match: { visible: true } })
294
-
295
- // Multiple calls — fully chainable
296
- .populate('author')
297
- .populate({ path: 'comments', select: 'text createdAt' })
131
+ qa.addStage({ $addFields: { margin: { $subtract: ['$price', '$cost'] } } });
298
132
  ```
299
133
 
300
- ---
301
-
302
- ## Terminal Method
303
-
304
- ### `.paginate()`
134
+ ### `.clone()`
305
135
 
306
- Executes the query and returns a `Promise<PaginatedResult<T>>`.
136
+ Copies the current filter state (`match()`/`dateRange()` conditions and `addStage()` stages) into a new, independent `QueryAnalytics` instance sharing the same options. Handy for branching one filtered base into several analytics calls without rebuilding the filter each time — see [Recipes](#recipes-1).
307
137
 
308
- - `countDocuments` and `find` run **in parallel** via `Promise.all` (saves one network round-trip).
309
- - Uses `estimatedDocumentCount` as a fast-path when no filter is applied (O(1) vs O(n)).
310
- - `.lean()` is applied for ~3–5× faster reads on plain object responses.
311
- - `maxTimeMS` is applied to both count and find to prevent runaway collection scans.
312
- - If the requested `page` exceeds `totalPages` (e.g. after a deletion), page `1` is returned automatically.
138
+ ## Time Series
313
139
 
314
- ```
315
- GET /users?page=2&limit=20
316
- ```
140
+ ### `.timeSeries(opts)`
317
141
 
318
- **Returns:**
142
+ Buckets documents by `dateField` into timezone-aligned `interval` buckets (`'hour' | 'day' | 'week' | 'month'`) via `$dateTrunc`, computes `metrics` per bucket, and — by default — fills empty buckets with zero values so a chart never shows a broken line for a quiet day.
319
143
 
320
144
  ```ts
321
- {
322
- data: T[];
323
- total: number; // Total matching documents across all pages
324
- page: number; // Current page (auto-corrects to 1 if out of range)
325
- totalPages: number;
326
- limit: number;
327
- hasNextPage: boolean;
328
- hasPrevPage: boolean;
145
+ interface StatSpec {
146
+ name: string;
147
+ op: 'sum' | 'avg' | 'min' | 'max' | 'count' | 'distinctCount';
148
+ field?: string; // required for every op except 'count'
329
149
  }
330
- ```
331
-
332
- **Defaults:** `page=1`, `limit=10`, max `limit=100`.
333
-
334
- ---
335
-
336
- ## Query Parameter Reference
337
-
338
- | Parameter | Example | Description |
339
- | ----------- | ------------------------- | ------------------------------------------------------ |
340
- | `page` | `?page=3` | Page number (default: `1`, min: `1`) |
341
- | `limit` | `?limit=25` | Documents per page (default: `10`, max: `100`) |
342
- | `sort` | `?sort=-createdAt,name` | Sort fields; prefix `-` for descending (max: 5 fields) |
343
- | `fields` | `?fields=name,email` | Comma-separated fields to include in the response |
344
- | `q` | `?q=john` | Global search term (max: 200 chars) |
345
- | _(any key)_ | `?role=admin&age[gte]=18` | Field-level filters processed by `.filter()` |
346
-
347
- ---
348
-
349
- ## Full Example (Express)
350
-
351
- ```ts
352
- import { Request, Response } from 'express';
353
- import { queryFind } from 'mongoose-query-find';
354
- import UserModel from '../models/user';
355
-
356
- export const getUsers = async (req: Request, res: Response) => {
357
- const result = await queryFind(UserModel.find(), req.query)
358
- .allowFields(['name', 'email', 'username', 'role', 'isActive', 'createdAt'])
359
- .where({ orgId: req.user.orgId, deletedAt: null })
360
- .filter()
361
- .globalSearch(['name', 'email', 'username'])
362
- .sort()
363
- .limitFields('-password -__v')
364
- .populate('role', 'name permissions')
365
- .paginate();
366
-
367
- res.json({ status: 'success', ...result });
368
- };
369
- ```
370
-
371
- **Example requests:**
372
-
373
- ```bash
374
- # Page 2, 15 per page, only active admins sorted by name
375
- GET /users?page=2&limit=15&role=admin&isActive=true&sort=name
376
-
377
- # Search "alice" across name, email, and username
378
- GET /users?q=alice
379
-
380
- # Users older than 25, return only name and email
381
- GET /users?age[gt]=25&fields=name,email
382
-
383
- # Filter by exact date
384
- GET /users?createdAt=2024-06-01
385
-
386
- # Filter by date range
387
- GET /users?createdAt[gte]=2024-01-01&createdAt[lte]=2024-12-31
388
-
389
- # Combined: search + filter + sort + pagination
390
- GET /users?q=john&role=editor&sort=-createdAt&page=1&limit=5
391
- ```
392
-
393
- ---
394
-
395
- ## More Examples
396
-
397
- **E-commerce product listing — nested price filters + category populate:**
398
150
 
399
- ```ts
400
- export const getProducts = async (req: Request, res: Response) => {
401
- const result = await queryFind(ProductModel.find(), req.query)
402
- .allowFields(['name', 'price', 'category', 'inStock', 'brand', 'createdAt'])
403
- .where({ deletedAt: null, published: true })
404
- .filter()
405
- .globalSearch(['name', 'brand'])
406
- .sort()
407
- .limitFields('name price inStock brand')
408
- .populate('category', 'name slug')
409
- .paginate();
410
-
411
- res.json(result);
412
- };
413
- ```
414
-
415
- ```bash
416
- # Price between $20 and $100, in stock, sorted cheapest first
417
- GET /products?price[gte]=20&price[lte]=100&inStock=true&sort=price
418
-
419
- # Search "nike" across name and brand
420
- GET /products?q=nike&sort=-createdAt
421
- ```
422
-
423
- **Blog posts — multiple populates + logical operators:**
424
-
425
- ```ts
426
- export const getPosts = async (req: Request, res: Response) => {
427
- const result = await queryFind(PostModel.find(), req.query)
428
- .allowFields(['title', 'status', 'authorId', 'tags', 'publishedAt'])
429
- .where({ deletedAt: null })
430
- .filter()
431
- .globalSearch(['title'])
432
- .sort()
433
- .limitFields('title status publishedAt')
434
- .populate('authorId', 'name avatar')
435
- .populate({
436
- path: 'comments',
437
- select: 'text createdAt',
438
- match: { visible: true },
439
- })
440
- .paginate();
441
-
442
- res.json(result);
443
- };
151
+ await qa.timeSeries({
152
+ dateField: 'createdAt',
153
+ interval: 'day',
154
+ metrics: [
155
+ { name: 'revenue', op: 'sum', field: 'total' },
156
+ { name: 'orders', op: 'count' },
157
+ { name: 'customers', op: 'distinctCount', field: 'customerId' },
158
+ ],
159
+ fillGaps: true, // default
160
+ });
444
161
  ```
445
162
 
446
- ```bash
447
- # Published OR featured posts, using the safe top-level $or
448
- GET /posts?$or[0][status]=published&$or[1][featured]=true
163
+ `interval: 'week'` respects the constructor's `weekStartsOn`. Set `fillGaps: false` to get back only buckets that actually had data.
449
164
 
450
- # Posts tagged "typescript" published this year
451
- GET /posts?tags=typescript&publishedAt[gte]=2026-01-01
452
- ```
165
+ ### `.cumulativeSeries(opts)`
453
166
 
454
- **Admin panel — strict field allowlist, sensitive fields never leak:**
167
+ Same bucketing as `timeSeries()`, plus a native running-total column per metric (`${name}_cumulative`), computed server-side via `$setWindowFields` — no client-side accumulation pass.
455
168
 
456
169
  ```ts
457
- export const adminListUsers = async (req: Request, res: Response) => {
458
- const result = await queryFind(UserModel.find(), req.query, { maxLimit: 250 })
459
- .allowFields(['name', 'email', 'role', 'isActive', 'lastLoginAt'])
460
- .where({ orgId: req.user.orgId })
461
- .filter()
462
- .sort()
463
- .limitFields('-password -passwordResetToken -__v') // client's ?fields= can never override this
464
- .paginate();
465
-
466
- res.json(result);
467
- };
170
+ await qa.cumulativeSeries({
171
+ dateField: 'createdAt',
172
+ interval: 'day',
173
+ metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
174
+ });
175
+ // [{ bucket: '...', revenue: 420, revenue_cumulative: 420 }, { bucket: '...', revenue: 310, revenue_cumulative: 730 }, ...]
468
176
  ```
469
177
 
470
- ```bash
471
- # Even if a client tries ?fields=password, it's stripped — not in allowFields()
472
- GET /admin/users?fields=password,email&role=admin
473
- ```
178
+ `distinctCount` metrics are skipped for the cumulative column — a running distinct-count isn't a well-defined windowed sum. Unlike `timeSeries()`, this method has no `fillGaps` option.
474
179
 
475
- **Cached count pattern — skip re-counting on subsequent pages:**
180
+ ## Comparisons & Breakdowns
476
181
 
477
- ```ts
478
- export const getNotifications = async (req: Request, res: Response) => {
479
- const builder = queryFind(NotificationModel.find(), req.query)
480
- .allowFields(['userId', 'read', 'createdAt'])
481
- .where({ userId: req.user.id })
482
- .filter()
483
- .sort();
484
-
485
- // First request from the client includes no cachedTotal
486
- const cachedTotal = req.query.cachedTotal
487
- ? Number(req.query.cachedTotal)
488
- : undefined;
489
- const result = await builder.paginate(cachedTotal);
490
-
491
- res.json(result); // client stores result.total and sends it back as ?cachedTotal= on the next page
492
- };
493
- ```
182
+ ### `.compare(opts)`
494
183
 
495
- ---
496
-
497
- ## TypeScript Types
498
-
499
- All types are exported:
184
+ Computes `metrics` for a preset current period and the equivalent prior period, plus absolute and percentage change.
500
185
 
501
186
  ```ts
502
- import {
503
- queryFind,
504
- QueryFind,
505
- QueryParams,
506
- PaginatedResult,
507
- QueryFindOptions,
508
- QueryFindError,
509
- QueryFindValidationError,
510
- SlowQueryInfo,
511
- } from 'mongoose-query-find';
512
- ```
513
-
514
- ```ts
515
- interface QueryParams {
516
- page?: string;
517
- limit?: string;
518
- sort?: string;
519
- fields?: string;
520
- q?: string;
521
- [key: string]: unknown;
522
- }
523
-
524
- interface PaginatedResult<T> {
525
- data: T[];
526
- total: number;
527
- page: number;
528
- totalPages: number;
529
- limit: number;
530
- hasNextPage: boolean;
531
- hasPrevPage: boolean;
187
+ interface CompareResult {
188
+ current: Record<string, number>;
189
+ previous: Record<string, number>;
190
+ change: Record<string, number>; // absolute delta
191
+ changePct: Record<string, number | null>; // null when previous was 0
532
192
  }
533
193
 
534
- interface QueryFindOptions {
535
- maxTimeMS?: number; // default: 5000
536
- slowQueryThresholdMS?: number; // default: maxTimeMS
537
- maxLimit?: number; // default: 100
538
- lean?: boolean; // default: true
539
- onSlowQuery?: (info: SlowQueryInfo) => void;
540
- onSanitizeDrop?: (path: string, value: unknown) => void;
541
- }
542
-
543
- interface SlowQueryInfo {
544
- elapsedMs: number;
545
- filter: Record<string, unknown>;
546
- sort: Record<string, 1 | -1>;
547
- page: number;
548
- limit: number;
549
- }
550
- ```
551
-
552
- ---
553
-
554
- ## Security
555
-
556
- | Protection | Detail |
557
- | --------------------------- | ---------------------------------------------------------------------------------------- |
558
- | **NoSQL injection** | Allowlist enforced on filters, sort, and projection via `.allowFields()` |
559
- | **Banned operators** | `$where`, `$expr`, `$function`, and others are rejected recursively at any nesting depth |
560
- | **ReDoS** | Search terms are regex-escaped before compilation |
561
- | **Oversized search** | `?q=` capped at 200 characters |
562
- | **Deep nesting DoS** | Filter object nesting capped at depth 5 |
563
- | **Page size DoS** | `limit` hard-capped at 100 |
564
- | **Sort abuse** | Sort fields hard-capped at 5 |
565
- | **Runaway queries** | `maxTimeMS` applied to both count and find (default: 5s) |
566
- | **Sensitive field leakage** | `?fields=` projection stripped against allowlist |
567
-
568
- ---
569
-
570
- ## Links
571
-
572
- - [npm](https://www.npmjs.com/package/mongoose-query-find)
573
- - [GitHub](https://github.com/jsdev-robin/mongoose-query-find)
574
- - [Issues](https://github.com/jsdev-robin/mongoose-query-find/issues)
575
-
576
- ---
577
-
578
- ## License
579
-
580
- ISC © [jsdev.robin@gmail.com](mailto:jsdev.robin@gmail.com)
581
-
582
- <br>
583
- <br>
584
-
585
- ---
586
-
587
- ---
588
-
589
- # QueryAggregate
590
-
591
- A fluent, allowlisted **aggregation-pipeline** builder for Mongoose (`^8 || ^9`), built on `Model.aggregate()`. It mirrors the same safety model as `QueryFind` — allowlisted filter fields, sanitized/rejected operators, capped pagination — but adds `$lookup` joins, `$group`, timezone-aware date filtering, custom date/hour ranges, keyset pagination, and one-round-trip dashboard stats.
592
-
593
- ## QueryAggregate Overview
594
-
595
- `QueryFind` is built on `Model.find()` and can't express joins or grouping. `QueryAggregate` solves that while keeping the same guardrails, plus a lot more:
596
-
597
- - Allowlisted filter/sort/projection fields
598
- - Recursive sanitization of client input (drops class instances, functions, over-deep nesting)
599
- - Banned-operator rejection (`$where`, `$expr`, `$function`, `$accumulator`, `$map`, `$reduce`, `$filter`)
600
- - Capped pagination (`limit`, `page`)
601
- - Timezone-aware date coercion, `$lookup` joins, arbitrary developer-authored pipeline stages via `.addStage()`
602
- - Single-round-trip pagination via `$facet`
603
- - **New:** six custom date-range methods (arbitrary day, arbitrary span, rolling N-days/N-hours, hour-of-day windows, absolute datetime windows)
604
- - **New:** `count()`, `distinct()`, and `stats()` for cheap metrics without fetching a data page
605
- - **New:** `stream()` for memory-safe large exports, and `paginateCursor()` for keyset pagination on huge collections
606
- - **New:** `explain()`, `debugPipeline()`, and `clone()` for debugging and query reuse
607
-
608
- ## When to use QueryAggregate vs QueryFind
609
-
610
- | Need | Use |
611
- | -------------------------------------------------------------- | ---------------- |
612
- | Simple filter/sort/paginate on one collection | `QueryFind` |
613
- | Joining another collection (`$lookup`) | `QueryAggregate` |
614
- | Grouping / rollups / computed fields (`$group`, `$addFields`) | `QueryAggregate` |
615
- | Client filters need to respect the caller's local timezone | `QueryAggregate` |
616
- | A custom day / date span / rolling window / hour-of-day filter | `QueryAggregate` |
617
- | Dashboard totals (sum/avg/count) without fetching a page | `QueryAggregate` |
618
- | Very large collections where `$skip` pagination gets slow | `QueryAggregate` |
619
- | Streaming a large export without buffering it in memory | `QueryAggregate` |
620
- | You need a custom developer-defined pipeline stage | `QueryAggregate` |
621
-
622
- ## Installation
623
-
624
- Ships from the same package as `QueryFind`:
625
-
626
- ```bash
627
- npm install mongoose-query-find
628
- ```
629
-
630
- ## Quick Start
631
-
632
- ```ts
633
- import { QueryAggregate } from 'mongoose-query-find';
634
- import CategoryModel from './models/category';
635
-
636
- const timezone = (req.headers['x-timezone'] as string) ?? 'Asia/Dhaka';
637
-
638
- const result = await new QueryAggregate(CategoryModel, req.query, { timezone })
639
- .allowFields(['name', 'status', 'createdAt'])
640
- .dateFields(['createdAt'])
641
- .where({ branchId: req.self.linkedTo, status: { $ne: 'ARCHIVED' } })
642
- .filter()
643
- .globalSearch(['name'])
644
- .sort()
645
- .project('name status createdAt updatedAt')
646
- .paginate();
647
- ```
648
-
649
- `result` shape is identical to `QueryFind`'s `PaginatedResult<T>`:
650
-
651
- ```json
652
- {
653
- "data": [...],
654
- "total": 84,
655
- "page": 2,
656
- "totalPages": 9,
657
- "limit": 10,
658
- "hasNextPage": true,
659
- "hasPrevPage": true
660
- }
661
- ```
662
-
663
- ## Constructor & Options
664
-
665
- ```ts
666
- new QueryAggregate<T>(model, queryString, options?);
667
- ```
668
-
669
- | Parameter | Type | Description |
670
- | ------------- | ----------------------- | ----------------------------------------------- |
671
- | `model` | `Model<T>` | A Mongoose model |
672
- | `queryString` | `QueryParams` | Parsed URL query object, e.g. `req.query` |
673
- | `options` | `QueryAggregateOptions` | Configuration options (see below), all optional |
674
-
675
- ### `QueryAggregateOptions`
676
-
677
- | Option | Type | Default | Description |
678
- | ---------------------- | ---------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
679
- | `timezone` | `string` | `'Asia/Dhaka'` | IANA timezone for interpreting client-supplied local dates. Invalid values fall back to the default (validated once at construction, never throws mid-query). |
680
- | `maxTimeMS` | `number` | `5000` | Max time MongoDB may spend per query. `0` disables. |
681
- | `slowQueryThresholdMS` | `number` | `maxTimeMS` | Threshold above which `onSlowQuery` fires. |
682
- | `maxLimit` | `number` | `100` | Hard cap on `?limit=`. |
683
- | `allowDiskUse` | `boolean` | `false` | Forwarded to `.allowDiskUse()` — enable for large sort/group pipelines. |
684
- | `weekStartsOn` | `0–6` | `6` (Saturday) | Used by `.range('thisWeek')`. |
685
- | `onSlowQuery` | `(info: SlowQueryInfo) => void` | — | Callback fired when a query exceeds `slowQueryThresholdMS`. |
686
- | `onSanitizeDrop` | `(path: string, value: unknown) => void` | — | Callback fired whenever sanitization drops a value (bad nesting, non-plain value). |
687
-
688
- ## Timezone Handling
689
-
690
- MongoDB always stores/compares dates in UTC. Clients think and filter in local time — `?createdAt=2026-09-01` means "Sept 1st in the caller's timezone," not UTC midnight.
691
-
692
- - `fromZonedTime(localString, tz)` → exact UTC instant for a wall-clock time in `tz`. Used to turn a client-supplied local date into the correct `$gte`/`$lt` UTC range in `$match`.
693
- - `toZonedTime(utcDate, tz)` → wall-clock `Date` for `tz`. Used to compute "what day is it right now, in the caller's timezone" for `.range()` presets and the rolling-N-days custom range.
694
-
695
- The timezone is a **per-request** value, typically sourced from a header:
696
-
697
- ```ts
698
- const timezone = (req.headers['x-timezone'] as string) ?? 'Asia/Dhaka';
194
+ const result = await qa.compare({
195
+ dateField: 'createdAt',
196
+ metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
197
+ preset: 'last7Days',
198
+ });
699
199
  ```
700
200
 
701
- An invalid/unknown IANA timezone string (bad header, typo) never throws — it's validated once at construction (`resolveTimezone`) and silently falls back to the default.
702
-
703
- Plain `YYYY-MM-DD` filter values are expanded into a `$gte`/`$lt` range spanning that local day; values with an explicit offset (`Z`, `+06:00`) are treated as absolute instants and passed through as-is. Every custom range method below applies the same timezone-aware logic.
704
-
705
- ## Builder Methods
706
-
707
- All builder methods return `this` and are fully chainable. Recommended call order:
201
+ **Presets:** `'today' | 'thisWeek' | 'thisMonth' | 'last7Days' | 'last30Days'`. `'thisWeek'`/`'thisMonth'` compare against a prior period of the same length (e.g. 12 days into this month vs. the last 12 days of last month), and `'thisWeek'` respects `weekStartsOn`.
708
202
 
709
- ```
710
- allowFields → dateFields/objectIdFields → where → range/rangeXxx → filter → globalSearch → sort → project → lookup/addStage/computeField → paginate/paginateCursor/count/distinct/stats/stream
711
- ```
712
-
713
- ---
203
+ > Any `.dateRange()` you've already applied on the instance is **ignored** here — the preset defines both windows on its own, independent of the instance's filter state. `.match()` conditions still apply to both windows.
714
204
 
715
- ### `.allowFields(fields: string[])`
205
+ ### `.breakdown(field, metrics, opts)`
716
206
 
717
- Same role as in `QueryFind`: declares which fields the client may filter, sort, and project. Case-insensitive. Call before `.filter()`, `.sort()`, `.project()`.
207
+ Groups by `field`, computes `metrics` per group, sorted and capped.
718
208
 
719
209
  ```ts
720
- .allowFields(['name', 'status', 'createdAt'])
210
+ await qa.breakdown(
211
+ 'category',
212
+ [{ name: 'revenue', op: 'sum', field: 'total' }],
213
+ {
214
+ limit: 10, // default 10
215
+ sortBy: 'revenue', // default: metrics[0].name
216
+ sortDir: -1, // default -1
217
+ },
218
+ );
219
+ // [{ key: 'Electronics', revenue: 18400 }, { key: 'Home', revenue: 9210 }, ...]
721
220
  ```
722
221
 
723
- ---
724
-
725
- ### `.dateFields(fields: string[])`
222
+ ### `.rankedBreakdown(field, metrics, opts)`
726
223
 
727
- Explicit list of fields treated as dates for timezone-aware coercion. When omitted, falls back to a name heuristic matching `createdAt`, `updatedAt`, `deletedAt`, `archivedAt`, `expiresAt`, `date`, `birthDate` (only when that word is the _last_ path segment, e.g. `user.updatedAt` matches, `updatedAtSomething` doesn't).
224
+ Same as `breakdown()`, but adds a native 1-based dense-rank column via `$setWindowFields`/`$denseRank` — no client-side re-sorting needed for a leaderboard UI.
728
225
 
729
226
  ```ts
730
- .dateFields(['createdAt', 'updatedAt'])
227
+ await qa.rankedBreakdown('salesRepId', [
228
+ { name: 'revenue', op: 'sum', field: 'total' },
229
+ ]);
230
+ // [{ key: 'rep_1', rank: 1, revenue: 18400 }, { key: 'rep_2', rank: 2, revenue: 18400 }, { key: 'rep_3', rank: 4, revenue: 9210 }, ...]
731
231
  ```
732
232
 
733
- ---
233
+ Ties share a rank and the next distinct value skips ahead (standard dense-rank semantics), so two reps tied for first both show `rank: 1` and the next lowest gets `rank: 2`.
734
234
 
735
- ### `.objectIdFields(fields: string[])`
235
+ ### `.multiBreakdown(fields, metrics, opts)`
736
236
 
737
- Explicit list of fields treated as ObjectId references, so a client-supplied 24-char hex string (`?categoryId=...`) is coerced to `Types.ObjectId` instead of being compared as a plain string (which would never match, since it's a different BSON type). When omitted, falls back to a name heuristic matching `_id` or any field ending in `Id` (e.g. `categoryId`, `basicInfo.brandId`).
237
+ Pivot table: groups by two dimensions at once, one row per `(dimA, dimB)` pair.
738
238
 
739
239
  ```ts
740
- .objectIdFields(['branchId', 'categoryId'])
741
- ```
742
-
743
- ---
744
-
745
- ### `.filter()`
746
-
747
- Parses `req.query` into the aggregation `$match` stage's client-derived portion. Runs, in order:
748
-
749
- 1. Strips reserved keys (`page`, `sort`, `limit`, `fields`, `q`)
750
- 2. `sanitize()` — recursively drops non-plain values (class instances, functions), capped at nesting depth 5
751
- 3. `coerceAll()` — timezone-aware date coercion + ObjectId coercion + boolean coercion + numeric coercion + operator-name coercion (`gt` → `$gt`, etc.)
752
- 4. `enforceAllowlist()` — strips any non-operator key not in `.allowFields()`
753
- 5. `rejectDisallowedOperators()` — throws `QueryAggregateValidationError` on `$where`, `$expr`, `$function`, `$accumulator`, `$map`, `$reduce`, `$filter`, or any unexpected top-level `$` operator
754
-
755
- ```
756
- GET /categories?createdAt=2026-09-01&status=ACTIVE
240
+ await qa.multiBreakdown(
241
+ ['branchId', 'status'],
242
+ [{ name: 'orders', op: 'count' }],
243
+ {
244
+ limit: 200, // optional; no cap by default
245
+ },
246
+ );
247
+ // [{ key: 'branch_a', key2: 'PAID', orders: 40 }, { key: 'branch_a', key2: 'REFUNDED', orders: 3 }, ...]
757
248
  ```
758
249
 
759
- ```ts
760
- .filter()
761
- // → { createdAt: { $gte: <UTC midnight in tz>, $lt: <next UTC midnight in tz> }, status: 'ACTIVE' }
762
- ```
250
+ Rows are sorted by `key` then `key2` ascending. There's no default `limit` — pass one for large dimension combinations.
763
251
 
764
- ---
252
+ ## Funnels & Cohorts
765
253
 
766
- ### `.where(conditions)`
254
+ ### `.funnel(steps)`
767
255
 
768
- Mandatory server-side conditions the client **cannot override** — kept in a separate internal bucket (`_serverConditions`) and merged last via `$and`, so URL params can never win against it.
256
+ Counts how many base documents match each step's condition, independently, in parallel.
769
257
 
770
258
  ```ts
771
- .where({ branchId: req.self.linkedTo, status: { $ne: 'ARCHIVED' } })
259
+ const result = await qa.funnel([
260
+ { name: 'Visited', match: {} },
261
+ { name: 'Signed up', match: { signedUpAt: { $ne: null } } },
262
+ { name: 'Purchased', match: { firstOrderAt: { $ne: null } } },
263
+ ]);
772
264
  ```
773
265
 
774
- ---
775
-
776
- ### `.range(field: string, preset: RangePreset)`
777
-
778
- Adds a hard, server-defined UTC date range on `field`, computed in this instance's timezone. Not client-overridable (goes into `_serverConditions`, same as `.where()`).
779
-
780
- **Presets:** `'today'`, `'yesterday'`, `'last7Days'`, `'last30Days'`, `'thisWeek'`, `'thisMonth'`
781
-
782
266
  ```ts
783
- .range('createdAt', 'last7Days')
267
+ interface FunnelResult {
268
+ name: string;
269
+ count: number;
270
+ conversionFromPrevious: number | null; // %
271
+ conversionFromStart: number | null; // %
272
+ }
784
273
  ```
785
274
 
786
- `.range('thisWeek')` respects the `weekStartsOn` constructor option (defaults to Saturday).
787
-
788
- ---
789
-
790
- ## Custom Range Methods
275
+ Each step's `match` is ANDed with the instance's base filter — steps are **independent counts**, not automatically cumulative. If step 3 should imply step 2, include that condition in step 3's `match` yourself. `conversionFromPrevious`/`conversionFromStart` are `null` for the first step, and `null` (instead of `Infinity`/`NaN`) whenever the denominator is 0.
791
276
 
792
- Beyond the fixed `RangePreset` values above, six additional methods cover any custom day, span, rolling window, or hour-based filter — without touching `.range()`'s own logic. All of them behave exactly like `.range()`: they write a hard, non-client-overridable condition into `_serverConditions`, in this instance's timezone, and throw `QueryAggregateValidationError` on bad input.
277
+ ### `.cohortRetention(opts)`
793
278
 
794
- > ⚠️ Only call **one** range method per field. If you call two on the same field (e.g. `.range()` then `.rangeCustom()` on `createdAt`), the later call simply overwrites the earlier one.
795
-
796
- ### `.rangeCustomDay(field: string, date: string)`
797
-
798
- A single arbitrary local calendar date (`YYYY-MM-DD`) — for "show me everything from this specific day" rather than a relative preset.
279
+ Buckets entities into cohorts by the first occurrence of `cohortDateField`, then measures how many of each cohort had activity (`activityDateField`) in each subsequent period.
799
280
 
800
281
  ```ts
801
- .rangeCustomDay('createdAt', '2026-03-14')
802
- ```
803
-
804
- ### `.rangeBetween(field: string, custom: CustomDateRangeInput)`
805
-
806
- An arbitrary inclusive local-date span — the classic "from / to" date-picker filter.
807
-
808
- ```ts
809
- .rangeBetween('createdAt', { from: '2026-01-01', to: '2026-01-31' })
810
- ```
811
-
812
- ### `.rangeLastNDays(field: string, days: number)`
813
-
814
- A rolling window of the last N whole calendar days, inclusive of today — a parametrized version of the built-in `last7Days`/`last30Days` presets.
282
+ interface CohortRetentionOptions {
283
+ cohortDateField: string; // e.g. signup date
284
+ activityDateField: string; // e.g. any return-visit date (can equal cohortDateField)
285
+ idField: string; // e.g. userId
286
+ cohortInterval: 'hour' | 'day' | 'week' | 'month';
287
+ periods: number; // how many periods forward to measure
288
+ }
815
289
 
816
- ```ts
817
- .rangeLastNDays('createdAt', 14) // last 14 calendar days
290
+ const table = await qa.cohortRetention({
291
+ cohortDateField: 'signupAt',
292
+ activityDateField: 'lastActiveAt',
293
+ idField: 'userId',
294
+ cohortInterval: 'week',
295
+ periods: 8,
296
+ });
818
297
  ```
819
298
 
820
- ### `.rangeLastNHours(field: string, hours: number)`
821
-
822
- A rolling window of the last N hours, ending at the exact moment the query runs. Unlike the day-based methods, this uses raw UTC instants with no calendar-day snapping — ideal for "recent activity" widgets.
823
-
824
299
  ```ts
825
- .rangeLastNHours('createdAt', 6) // activity in the last 6 hours
300
+ interface CohortRow {
301
+ cohort: string; // ISO bucket start
302
+ cohortSize: number;
303
+ retention: (number | null)[]; // index 0 = period 0
304
+ retentionPct: (number | null)[];
305
+ }
826
306
  ```
827
307
 
828
- ### `.rangeHours(field: string, custom: CustomHourRangeInput)`
308
+ Two aggregate calls run first (cohort assignment, then distinct activity buckets), and the retention matrix itself is assembled in JS afterward. Notes:
829
309
 
830
- An explicit hour-of-day window on a specific local date — e.g. filtering orders placed during business hours.
831
-
832
- ```ts
833
- .rangeHours('createdAt', { date: '2026-03-14', fromHour: 9, toHour: 17 })
834
- // toHour is exclusive; pass 24 for "through end of day"
835
- ```
310
+ - `retention[0]` (period 0) falls back to the full `cohortSize` when there's no explicit activity row recorded at offset 0 — useful when the cohort event and the first activity event are the same document, but worth checking if your two date fields can genuinely diverge at period 0.
311
+ - For `cohortInterval: 'month'`, offsets are computed against an approximate 30-day month for period-bucketing math; the cohort buckets themselves are exact (via `$dateTrunc`), only the offset-from-cohort calculation is approximate.
836
312
 
837
- ### `.rangeCustom(field: string, custom: CustomDateTimeRangeInput)`
313
+ ### `.sessionize(opts)`
838
314
 
839
- The most flexible option — an explicit absolute datetime window. Each of `from`/`to` accepts:
840
-
841
- - `"YYYY-MM-DD"` → local midnight in this instance's timezone
842
- - `"YYYY-MM-DDTHH:mm:ss[.sss]"` → local wall-clock time in this instance's timezone
843
- - an offset-bearing ISO string (`...Z`, `...+06:00`) → treated as an absolute instant
315
+ Groups per-user events into sessions using native `$setWindowFields` + `$shift` gap detection — no self-lookup or JS grouping pass.
844
316
 
845
317
  ```ts
846
- .rangeCustom('createdAt', {
847
- from: '2026-03-14T09:30:00', // local wall-clock
848
- to: '2026-03-14T18:45:00',
849
- })
318
+ interface SessionizeOptions {
319
+ userField: string;
320
+ dateField: string;
321
+ gapMinutes: number; // idle gap after which a new session starts
322
+ }
850
323
 
851
- // or mixed with an explicit UTC instant:
852
- .rangeCustom('createdAt', { from: '2026-03-14T00:00:00Z', to: '2026-03-15T00:00:00Z' })
324
+ const sessions = await qa.sessionize({
325
+ userField: 'userId',
326
+ dateField: 'eventAt',
327
+ gapMinutes: 30,
328
+ });
853
329
  ```
854
330
 
855
- ---
856
-
857
- ### `.excludeSoftDeleted(field = 'deletedAt')`
858
-
859
- Convenience server-side condition excluding soft-deleted documents. Matches both `null` and a genuinely missing field, per MongoDB's equality semantics. Not client-overridable.
860
-
861
331
  ```ts
862
- .excludeSoftDeleted() // defaults to 'deletedAt'
863
- .excludeSoftDeleted('removedAt') // or a custom field name
332
+ interface SessionSummary {
333
+ userId: unknown;
334
+ sessionStart: string;
335
+ sessionEnd: string;
336
+ eventCount: number;
337
+ durationSeconds: number;
338
+ }
864
339
  ```
865
340
 
866
- ---
341
+ ## Statistics
867
342
 
868
- ### `.whereIf(condition, conditions)`
343
+ ### `.percentiles(field, ps)`
869
344
 
870
- Applies `.where(conditions)` only when `condition` is truthy — avoids scattering `if` statements around a chained builder call.
345
+ Percentiles (0–100) of `field` over the current filter, via native `$percentile` (`method: 'approximate'`).
871
346
 
872
347
  ```ts
873
- .whereIf(req.query.vip === 'true', { tier: 'vip' })
874
- .whereIf(req.self.role !== 'admin', { branchId: req.self.linkedTo })
348
+ await qa.percentiles('total', [50, 90, 99]);
349
+ // { p50: 42, p90: 118, p99: 340 }
875
350
  ```
876
351
 
877
- ---
352
+ Values are `null` when there's no matching data.
878
353
 
879
- ### `.globalSearch(fields: string[])`
354
+ ### `.stdDevOutliers(field, opts)`
880
355
 
881
- Identical behavior to `QueryFind`: case-insensitive `$or` regex search across the given fields when `?q=` is present (max 200 chars). Regex-escaped to prevent ReDoS. Fields are filtered against the allowlist first.
356
+ Flags documents whose `field` value is more than `threshold` standard deviations from the mean, using native `$stdDevPop`.
882
357
 
883
358
  ```ts
884
- .globalSearch(['name'])
359
+ await qa.stdDevOutliers('total', { threshold: 3, limit: 50 }); // defaults shown
885
360
  ```
886
361
 
887
- ---
888
-
889
- ### `.sort()`
890
-
891
- Parses `?sort=` (e.g. `-createdAt,name`), capped at 5 fields, dropping any field not in the allowlist. Falls back to `{ createdAt: -1 }` when absent.
892
-
893
362
  ```ts
894
- .sort()
363
+ interface OutlierRow {
364
+ _id: unknown;
365
+ value: number;
366
+ zScore: number;
367
+ [key: string]: unknown; // full source document is spread in
368
+ }
895
369
  ```
896
370
 
897
- > `.paginateCursor()` (below) seeks on this same sort configuration, so call `.sort()` before it if you need a field other than the default `createdAt`.
898
-
899
- ---
371
+ Two round trips: one to compute `mean`/`stdDev` over the full filtered set, one to fetch flagged rows. Returns `[]` if there's no data or the standard deviation is 0 (constant field).
900
372
 
901
- ### `.project(defaultProjection?: string | Record<string, 0 | 1>)`
373
+ > **Caveat:** rows are sorted by `zScore` descending before `limit` is applied, so with a tight `limit` on a field that has outliers on both tails, strongly _positive_ outliers can crowd out strongly _negative_ ones. Raise `limit` (or drop the `$limit` and sort by `Math.abs(zScore)` yourself) if you need both tails represented fairly.
902
374
 
903
- Controls the `$project` stage. Accepts a space-separated string (`"name status -_id"`) or a raw `{ field: 0 | 1 }` object.
375
+ ### `.metrics(specs)`
904
376
 
905
- - `?fields=` (when present) is parsed and filtered through the allowlist.
906
- - If **every** requested field is blocked, falls back to `defaultProjection` rather than silently returning everything.
907
- - `_id` and `__v` are always allowed through even if not in `allowFields()`.
377
+ Several independent `StatSpec` metrics in one round trip — cheaper than N separate calls for a KPI-strip widget.
908
378
 
909
379
  ```ts
910
- .project('name status createdAt updatedAt')
380
+ await qa.metrics([
381
+ { name: 'revenue', op: 'sum', field: 'total' },
382
+ { name: 'avgOrder', op: 'avg', field: 'total' },
383
+ { name: 'orders', op: 'count' },
384
+ ]);
385
+ // { revenue: 48213, avgOrder: 62.1, orders: 776 }
911
386
  ```
912
387
 
913
- > ⚠️ If a preceding `.addStage()` (e.g. `$group`) reshapes documents, `.project()`'s allowlist still only knows about the fields passed to `.allowFields()` — new computed fields may be silently dropped. Use `.addStage({ $project: {...} })` instead when you need full control post-reshape.
914
-
915
- ---
388
+ ## Post-Processing Helpers (static)
916
389
 
917
- ### `.lookup(opts: LookupOptions)`
390
+ These run entirely in JS against a series you've already fetched — no database call.
918
391
 
919
- Adds a `$lookup` (join) stage, optionally followed by `$unwind`.
392
+ ### `QueryAnalytics.movingAverage()`
920
393
 
921
- | Field | Type | Description |
922
- | -------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------- |
923
- | `from` | `string` | Collection name (not the model name) to join against. |
924
- | `as` | `string` | Output array field name. |
925
- | `localField` | `string?` | Local join field. |
926
- | `foreignField` | `string?` | Foreign join field. |
927
- | `pipeline` | `PipelineStage[]?` | Sub-pipeline form of `$lookup`, for complex/correlated joins. Cannot contain `$out`/`$merge`. |
928
- | `single` | `boolean?` | When `true`, `$unwind`s the joined array (`preserveNullAndEmptyArrays: true`) so `as` becomes a single object instead of an array. |
394
+ Adds a trailing moving-average field to an existing time series.
929
395
 
930
396
  ```ts
931
- .lookup({
932
- from: 'branches',
933
- as: 'branch',
934
- localField: 'branchId',
935
- foreignField: '_id',
936
- single: true,
937
- })
397
+ QueryAnalytics.movingAverage(series, 'revenue', 7); // adds `revenue_ma`
938
398
  ```
939
399
 
940
- ---
400
+ The window shrinks near the start of the series rather than looking backward past index 0 (so the first point's average is just itself, the second is the average of the first two, and so on). Throws `QueryAnalyticsError` if `windowSize < 1`.
941
401
 
942
- ### `.computeField(name: string, expression: unknown)`
402
+ ### `QueryAnalytics.trend()`
943
403
 
944
- Convenience wrapper over `.addStage()` for a single computed `$addFields` entry.
404
+ Simple up/down/flat classification, comparing the average of the last `n` points to the `n` before that.
945
405
 
946
406
  ```ts
947
- .computeField('margin', { $subtract: ['$price', '$cost'] })
948
- // equivalent to: .addStage({ $addFields: { margin: { $subtract: ['$price', '$cost'] } } })
407
+ QueryAnalytics.trend(series, 'revenue', 3, 2); // n=3, flatThresholdPct=2 (defaults)
408
+ // 'up' | 'down' | 'flat' | 'insufficient_data'
949
409
  ```
950
410
 
951
- ---
411
+ Returns `'insufficient_data'` when the series has fewer than `2 * n` points.
952
412
 
953
- ### `.addStage(stage: PipelineStage)`
413
+ ### `QueryAnalytics.forecast()`
954
414
 
955
- Escape hatch for any other developer-authored stage: `$addFields`, `$group`, a post-join `$match`, a custom `$sort`, etc.
956
-
957
- - Blocks **only** `$out` / `$merge` (destructive stages).
958
- - **Does not** run the client-input sanitizer/allowlist/operator-rejection — that only applies to `.filter()`/`.where()`, which parse untrusted `req.query`/`req.body` values.
959
- - **Never pass raw client input directly into `.addStage()`.** It's for stages _you_ author server-side. If you need a client-supplied value inside a custom stage, validate/whitelist it yourself first.
415
+ A simple linear-regression projection for `periodsAhead` future buckets, based on the series' existing trend line.
960
416
 
961
417
  ```ts
962
- .addStage({
963
- $group: {
964
- _id: '$status',
965
- count: { $sum: 1 },
966
- totalAmount: { $sum: '$amount' },
967
- },
968
- })
418
+ QueryAnalytics.forecast(series, 'revenue', 7);
419
+ // [{ index: 30, predicted: 512.4 }, { index: 31, predicted: 518.9 }, ...]
969
420
  ```
970
421
 
971
- ## Terminal & Utility Methods
972
-
973
- ### `.paginate(cachedTotal?: number)`
422
+ Good enough for a dashed projection line on a chart — not a substitute for a real forecasting model. Returns `[]` for series shorter than 2 points.
974
423
 
975
- Executes the pipeline and returns `Promise<PaginatedResult<T>>`.
424
+ ## `.heatmap(dateField, valueField?)`
976
425
 
977
- - Uses `$facet` to fetch the data page and total count in a **single round trip**.
978
- - Pass a prior page-1 `total` as `cachedTotal` on later pages to **skip the count stage entirely**.
979
- - `maxTimeMS` / `allowDiskUse` from constructor options are applied to the aggregation.
980
- - If the requested `page` exceeds `totalPages`, page `1` is returned automatically (with the real `total`/`totalPages` still reported).
981
- - `onSlowQuery` (if configured) fires with `{ elapsedMs, pipeline, sort, page, limit }` when the query exceeds `slowQueryThresholdMS`.
982
-
983
- > **Note:** `$facet` buffers both branches in memory per query. For very large collections with expensive pre-facet stages (heavy `$lookup`/`$group` before pagination), pass `cachedTotal` on subsequent pages, use `.paginateCursor()` instead, or run `.count()`/data as two separate calls.
426
+ Counts (or sums `valueField`, if given) by weekday (0=Sun..6=Sat, local timezone) × hour-of-day (0–23, local timezone).
984
427
 
985
428
  ```ts
986
- const page1 = await builder.paginate();
987
- const page2 = await builder.paginate(page1.total); // skips re-counting
988
- ```
989
-
990
- ---
429
+ await qa.heatmap('createdAt');
430
+ // [{ weekday: 0, hour: 0, value: 4 }, { weekday: 0, hour: 1, value: 1 }, ...]
991
431
 
992
- ### `.paginateCursor(opts?: CursorPaginateOptions)`
993
-
994
- Keyset ("seek") pagination — scales far better than `.paginate()`'s `$skip` on deep pages of large collections, at the cost of not supporting "jump to page N". Seeks on whatever field `.sort()` resolved to (falls back to `createdAt` if `.sort()` wasn't called), using `_id` as a stable tiebreaker.
995
-
996
- ```ts
997
- interface CursorPaginateOptions {
998
- cursor?: string; // opaque cursor from a previous page's nextCursor
999
- limit?: number; // overrides ?limit=, still capped by maxLimit
1000
- }
1001
-
1002
- interface CursorPaginatedResult<T> {
1003
- data: T[];
1004
- limit: number;
1005
- nextCursor: string | null; // pass this back in as `cursor` for the next page
1006
- hasNextPage: boolean;
1007
- }
432
+ await qa.heatmap('createdAt', 'total'); // sum revenue instead of counting docs
1008
433
  ```
1009
434
 
1010
- ```ts
1011
- const p1 = await builder.sort().paginateCursor({ limit: 20 });
1012
- const p2 = await builder.paginateCursor({ cursor: p1.nextCursor!, limit: 20 });
1013
- // hasNextPage: false / nextCursor: null means you've reached the end
1014
- ```
1015
-
1016
- Great fit for infinite-scroll feeds and large-export pagination; not a drop-in replacement where the UI needs page numbers.
435
+ Returns a flat array (one row per `weekday`/`hour` combination that occurred), not a pre-filled 7×24 matrix — fill in the zero cells client-side if your chart needs a dense grid.
1017
436
 
1018
- ---
437
+ ## Live Updates
1019
438
 
1020
- ### `.count()`
439
+ ### `.watch(onChange, opts)`
1021
440
 
1022
- Returns just the matching document count — cheaper than `.paginate()` when you don't need the data page. Ignores `.project()`/`.sort()`.
441
+ Subscribes to a MongoDB change stream on the model's collection and invokes `onChange` for every matching insert/update/delete/replace — handy for pushing live dashboard updates over a websocket instead of polling.
1023
442
 
1024
443
  ```ts
1025
- const total = await builder.count();
1026
- ```
1027
-
1028
- ---
1029
-
1030
- ### `.distinct(field: string)`
444
+ const stream = qa.watch(
445
+ (change) => io.emit('order:change', change),
446
+ { operationTypes: ['insert', 'update'] }, // optional; all types by default
447
+ );
1031
448
 
1032
- Returns the distinct values of `field` under the current `.filter()`/`.where()`/range conditions.
1033
-
1034
- ```ts
1035
- const statuses = await builder.distinct('status');
1036
- // e.g. ['PENDING', 'PAID', 'CANCELLED']
449
+ // later
450
+ stream.close();
1037
451
  ```
1038
452
 
1039
- ---
453
+ Returns the underlying `ChangeStream` — the caller owns its lifecycle and must `.close()` it. `fullDocument: 'updateLookup'` is set so update events include the full document.
1040
454
 
1041
- ### `.stats(specs: StatSpec[])`
455
+ Two things worth knowing:
1042
456
 
1043
- Computes one or more aggregate metrics — `sum`, `avg`, `min`, `max`, `count` — in a **single round trip**. Ideal for dashboard totals. Ignores `.project()`/`.sort()`/pagination.
457
+ - Only conditions set via `.match()`/`.dateRange()` are translated onto the stream (as `fullDocument.<field>` equality conditions) — stages added via `.addStage()` aren't part of the change-stream pipeline.
458
+ - Change streams require a MongoDB replica set or Atlas cluster; they don't work against a standalone `mongod`.
1044
459
 
1045
- ```ts
1046
- interface StatSpec {
1047
- name: string;
1048
- op: 'sum' | 'avg' | 'min' | 'max' | 'count';
1049
- field?: string; // required for every op except 'count'
1050
- }
1051
- ```
460
+ ## Export & Batch Helpers (static)
1052
461
 
1053
- ```ts
1054
- const totals = await builder.stats([
1055
- { name: 'revenue', op: 'sum', field: 'total' },
1056
- { name: 'avgOrder', op: 'avg', field: 'total' },
1057
- { name: 'orders', op: 'count' },
1058
- ]);
1059
- // { revenue: 48213, avgOrder: 62.1, orders: 776 }
1060
- ```
1061
-
1062
- ---
462
+ ### `QueryAnalytics.toCSV()` / `toNDJSON()`
1063
463
 
1064
- ### `.stream()`
1065
-
1066
- Returns a Mongoose `AggregationCursor` — async-iterable, no buffering of the full result set in memory. Use for large exports (CSV generation, bulk processing).
464
+ Flatten an array of plain-object rows (e.g. the output of `timeSeries()`/`breakdown()`) into CSV or newline-delimited JSON.
1067
465
 
1068
466
  ```ts
1069
- for await (const doc of builder.stream()) {
1070
- csvWriter.write(doc);
1071
- }
467
+ QueryAnalytics.toCSV(rows); // header row inferred from the union of all row keys; values with , " or \n are quoted
468
+ QueryAnalytics.toNDJSON(rows); // one JSON.stringify(row) per line — friendlier for streaming large exports
1072
469
  ```
1073
470
 
1074
- Ignores `page`/`limit`; add your own `$limit` via `.addStage()` if you want a hard cap.
1075
-
1076
- ---
471
+ ### `QueryAnalytics.dashboard()`
1077
472
 
1078
- ### `.explain(verbosity?)`
1079
-
1080
- Runs the built pipeline through Mongoose's `.explain()` — for debugging slow queries and verifying index usage. `verbosity` is `'queryPlanner'` (default), `'executionStats'`, or `'allPlansExecution'`.
473
+ Runs several named analytics calls in parallel and returns them keyed by name — one round trip from the caller's perspective (though each named task is still its own aggregate call under the hood).
1081
474
 
1082
475
  ```ts
1083
- const plan = await builder.explain('executionStats');
1084
- ```
1085
-
1086
- > Development/ops tool only — never call this in a production request path.
1087
-
1088
- ---
1089
-
1090
- ### `.clone()`
1091
-
1092
- Deep-copies the current builder state (filters, sort, projection, stages) into a new, independent `QueryAggregate` instance. Handy for branching one shared base query into e.g. both `.paginate()` and `.stats()` without rebuilding the filters twice.
1093
-
1094
- ```ts
1095
- const base = builder.filter().sort().project('name total createdAt');
1096
-
1097
- const [page, totals] = await Promise.all([
1098
- base.clone().paginate(),
1099
- base.clone().stats([{ name: 'revenue', op: 'sum', field: 'total' }]),
1100
- ]);
476
+ const data = await QueryAnalytics.dashboard({
477
+ revenue: () =>
478
+ qa.timeSeries({
479
+ dateField: 'createdAt',
480
+ interval: 'day',
481
+ metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
482
+ }),
483
+ topCategories: () =>
484
+ qa.breakdown('category', [{ name: 'revenue', op: 'sum', field: 'total' }]),
485
+ totals: () => qa.metrics([{ name: 'orders', op: 'count' }]),
486
+ });
487
+ // { revenue: [...], topCategories: [...], totals: { orders: 776 } }
1101
488
  ```
1102
489
 
1103
- ---
1104
-
1105
- ### `.debugPipeline()`
490
+ ### `QueryAnalytics.clearCache()`
1106
491
 
1107
- Returns the fully-built pipeline (`PipelineStage[]`) without executing it — for logging, snapshot tests, or sanity-checking what will actually run.
1108
-
1109
- ```ts
1110
- console.log(builder.debugPipeline());
1111
- ```
492
+ Clears the shared in-memory cache. **Process-wide** — not scoped to one instance or one model. (See the caching note under [Constructor & Options](#constructor--options-2): as of this release, nothing populates the cache yet via the public API.)
1112
493
 
1113
494
  ## Recipes
1114
495
 
1115
- **Join + post-join filter + group + paginate:**
496
+ **Branch dashboard in one round trip:**
1116
497
 
1117
498
  ```ts
1118
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1119
- .allowFields(['status', 'createdAt'])
1120
- .dateFields(['createdAt'])
1121
- .filter()
1122
- .lookup({
1123
- from: 'customers',
1124
- as: 'customer',
1125
- localField: 'customerId',
1126
- foreignField: '_id',
1127
- single: true,
1128
- })
1129
- .addStage({ $match: { 'customer.vip': true } }) // server-defined, not client-controlled
1130
- .addStage({
1131
- $group: {
1132
- _id: '$customer._id',
1133
- customerName: { $first: '$customer.name' },
1134
- orderCount: { $sum: 1 },
1135
- },
1136
- })
1137
- .sort()
1138
- .paginate();
1139
- ```
1140
-
1141
- **Relative date range preset (server-enforced, not client-overridable):**
1142
-
1143
- ```ts
1144
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1145
- .allowFields(['status'])
1146
- .where({ branchId: req.self.linkedTo })
1147
- .range('createdAt', 'last30Days')
1148
- .filter()
1149
- .sort()
1150
- .paginate();
1151
- ```
1152
-
1153
- **Custom hour-of-day window (business hours report):**
1154
-
1155
- ```ts
1156
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1157
- .allowFields(['status'])
1158
- .where({ branchId: req.self.linkedTo })
1159
- .rangeHours('createdAt', {
1160
- date: req.query.day as string,
1161
- fromHour: 9,
1162
- toHour: 21,
1163
- })
1164
- .filter()
1165
- .sort()
1166
- .paginate();
1167
- ```
1168
-
1169
- **Dashboard: page + totals in parallel, off one shared base:**
1170
-
1171
- ```ts
1172
- const base = new QueryAggregate(OrderModel, req.query, { timezone })
1173
- .allowFields(['status', 'branchId', 'createdAt', 'total'])
1174
- .dateFields(['createdAt'])
1175
- .where({ branchId: req.self.linkedTo })
1176
- .excludeSoftDeleted()
1177
- .filter()
1178
- .sort()
1179
- .project('status createdAt total');
1180
-
1181
- const [page, totals] = await Promise.all([
1182
- base.clone().paginate(),
1183
- base.clone().stats([
1184
- { name: 'revenue', op: 'sum', field: 'total' },
1185
- { name: 'orders', op: 'count' },
1186
- ]),
1187
- ]);
1188
- ```
1189
-
1190
- **Infinite scroll with keyset pagination:**
499
+ const qa = new QueryAnalytics(OrderModel, { timezone })
500
+ .match({ branchId: req.self.linkedTo, status: { $ne: 'CANCELLED' } })
501
+ .dateRange('createdAt', {
502
+ from: req.query.from as string,
503
+ to: req.query.to as string,
504
+ });
1191
505
 
1192
- ```ts
1193
- const builder = new QueryAggregate(PostModel, req.query, { timezone })
1194
- .allowFields(['authorId', 'createdAt'])
1195
- .where({ published: true })
1196
- .filter()
1197
- .sort();
1198
-
1199
- const page = await builder.paginateCursor({
1200
- cursor: req.query.cursor as string | undefined,
1201
- limit: 20,
506
+ const dashboard = await QueryAnalytics.dashboard({
507
+ series: () =>
508
+ qa.timeSeries({
509
+ dateField: 'createdAt',
510
+ interval: 'day',
511
+ metrics: [
512
+ { name: 'revenue', op: 'sum', field: 'total' },
513
+ { name: 'orders', op: 'count' },
514
+ ],
515
+ }),
516
+ topProducts: () =>
517
+ qa.breakdown(
518
+ 'productId',
519
+ [{ name: 'revenue', op: 'sum', field: 'total' }],
520
+ { limit: 5 },
521
+ ),
522
+ kpis: () =>
523
+ qa.metrics([
524
+ { name: 'revenue', op: 'sum', field: 'total' },
525
+ { name: 'orders', op: 'count' },
526
+ ]),
1202
527
  });
1203
- ```
1204
-
1205
- **Streaming a large CSV export:**
1206
528
 
1207
- ```ts
1208
- const builder = new QueryAggregate(OrderModel, req.query, { timezone })
1209
- .allowFields(['status', 'createdAt'])
1210
- .where({ branchId: req.self.linkedTo })
1211
- .range('createdAt', 'thisMonth')
1212
- .filter()
1213
- .project('status createdAt total');
1214
-
1215
- res.setHeader('Content-Type', 'text/csv');
1216
- for await (const row of builder.stream()) {
1217
- res.write(`${row.status},${row.createdAt.toISOString()},${row.total}\n`);
1218
- }
1219
- res.end();
529
+ res.json(dashboard);
1220
530
  ```
1221
531
 
1222
- **ObjectId filter fields (ref lookups by hex ID from the URL):**
1223
-
1224
- ```
1225
- GET /orders?branchId=64f1a2b3c4d5e6f7a8b9c0d1&categoryId=64f1a2b3c4d5e6f7a8b9c0d2
1226
- ```
532
+ **Week-over-week KPI strip:**
1227
533
 
1228
534
  ```ts
1229
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1230
- .allowFields(['branchId', 'categoryId', 'status'])
1231
- .objectIdFields(['branchId', 'categoryId']) // coerces hex strings -> Types.ObjectId
1232
- .filter()
1233
- .sort()
1234
- .paginate();
1235
- ```
1236
-
1237
- **Role-based server conditions with `.whereIf()` + `.excludeSoftDeleted()`:**
1238
-
1239
- ```ts
1240
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1241
- .allowFields(['status', 'branchId', 'createdAt'])
1242
- .excludeSoftDeleted() // deletedAt: null, always applied
1243
- .whereIf(req.self.role !== 'admin', { branchId: req.self.linkedTo }) // branch staff see only their branch
1244
- .whereIf(req.query.vip === 'true', { tier: 'vip' })
1245
- .filter()
1246
- .sort()
1247
- .paginate();
1248
- ```
1249
-
1250
- **Populating a filter UI — distinct values + count, no data fetch:**
535
+ const wow = await new QueryAnalytics(OrderModel, { timezone })
536
+ .match({ branchId: req.self.linkedTo })
537
+ .compare({
538
+ dateField: 'createdAt',
539
+ preset: 'thisWeek',
540
+ metrics: [
541
+ { name: 'revenue', op: 'sum', field: 'total' },
542
+ { name: 'orders', op: 'count' },
543
+ ],
544
+ });
1251
545
 
1252
- ```ts
1253
- const builder = new QueryAggregate(OrderModel, req.query, { timezone })
1254
- .allowFields(['status', 'branchId'])
1255
- .where({ branchId: req.self.linkedTo })
1256
- .filter();
1257
-
1258
- const [statuses, total] = await Promise.all([
1259
- builder.clone().distinct('status'), // populate a <select> of possible statuses
1260
- builder.clone().count(), // "X results" label
1261
- ]);
546
+ res.json(wow); // { current, previous, change, changePct }
1262
547
  ```
1263
548
 
1264
- **Debug route — inspect the built pipeline / query plan without exposing it to normal users:**
549
+ **Signup retention curve, branched off one base filter:**
1265
550
 
1266
551
  ```ts
1267
- router.get('/orders/_debug', requireAdmin, async (req, res) => {
1268
- const builder = new QueryAggregate(OrderModel, req.query, { timezone })
1269
- .allowFields(['status', 'createdAt'])
1270
- .where({ branchId: req.self.linkedTo })
1271
- .range('createdAt', 'last7Days')
1272
- .filter()
1273
- .sort();
1274
-
1275
- res.json({
1276
- pipeline: builder.debugPipeline(),
1277
- explain: await builder.explain('executionStats'),
1278
- });
552
+ const base = new QueryAnalytics(UserModel, { timezone }).match({
553
+ orgId: req.user.orgId,
1279
554
  });
1280
- ```
1281
-
1282
- **Cursor pagination on a non-default sort field:**
1283
555
 
1284
- ```ts
1285
- const result = await new QueryAggregate(ProductModel, req.query, { timezone })
1286
- .allowFields(['price', 'name'])
1287
- .where({ inStock: true })
1288
- .filter()
1289
- .sort() // resolves ?sort=price into { price: 1 } — paginateCursor() will seek on "price"
1290
- .paginateCursor({
1291
- cursor: req.query.cursor as string | undefined,
1292
- limit: 24,
1293
- });
1294
- ```
1295
-
1296
- **Correlated sub-pipeline `$lookup` (needs `let`):**
1297
-
1298
- The built-in `LookupOptions` type doesn't expose `let` for correlated sub-pipelines. For that case, drop to `.addStage()` directly:
1299
-
1300
- ```ts
1301
- .addStage({
1302
- $lookup: {
1303
- from: 'orderItems',
1304
- let: { orderId: '$_id' },
1305
- pipeline: [
1306
- { $match: { $expr: { $eq: ['$orderId', '$$orderId'] } } },
1307
- { $project: { name: 1, qty: 1 } },
1308
- ],
1309
- as: 'items',
1310
- },
1311
- })
556
+ const [retention, funnel] = await Promise.all([
557
+ base.clone().cohortRetention({
558
+ cohortDateField: 'signupAt',
559
+ activityDateField: 'lastActiveAt',
560
+ idField: '_id',
561
+ cohortInterval: 'week',
562
+ periods: 8,
563
+ }),
564
+ base.clone().funnel([
565
+ { name: 'Signed up', match: {} },
566
+ { name: 'Activated', match: { activatedAt: { $ne: null } } },
567
+ { name: 'Subscribed', match: { subscribedAt: { $ne: null } } },
568
+ ]),
569
+ ]);
1312
570
  ```
1313
571
 
1314
- **Sales analytics dashboard — join, group, and rolling window in one pipeline:**
572
+ **Anomalies widget with a projected trend line:**
1315
573
 
1316
574
  ```ts
1317
- const timezone = (req.headers['x-timezone'] as string) ?? 'Asia/Dhaka';
1318
-
1319
- const result = await new QueryAggregate(OrderModel, req.query, { timezone })
1320
- .allowFields(['branchId', 'status'])
1321
- .where({ status: 'PAID' })
1322
- .rangeLastNDays('createdAt', 30) // rolling 30-day window, client can't override
1323
- .filter()
1324
- .lookup({
1325
- from: 'branches',
1326
- as: 'branch',
1327
- localField: 'branchId',
1328
- foreignField: '_id',
1329
- single: true,
1330
- })
1331
- .addStage({
1332
- $group: {
1333
- _id: '$branchId',
1334
- branchName: { $first: '$branch.name' },
1335
- revenue: { $sum: '$total' },
1336
- orders: { $sum: 1 },
1337
- avgOrderValue: { $avg: '$total' },
1338
- },
1339
- })
1340
- .addStage({ $sort: { revenue: -1 } })
1341
- .paginate();
1342
- ```
1343
-
1344
- **Notification feed — lookup + unwind a single related document per row:**
575
+ const qa = new QueryAnalytics(OrderModel, { timezone }).match({ branchId });
1345
576
 
1346
- ```ts
1347
- const result = await new QueryAggregate(NotificationModel, req.query, {
1348
- timezone,
1349
- })
1350
- .allowFields(['read', 'type', 'createdAt'])
1351
- .dateFields(['createdAt'])
1352
- .where({ userId: req.user.id })
1353
- .filter()
1354
- .lookup({
1355
- from: 'users',
1356
- as: 'actor',
1357
- localField: 'actorId',
1358
- foreignField: '_id',
1359
- single: true, // unwinds so `actor` is a single object, not an array
1360
- })
1361
- .sort()
1362
- .project('type read createdAt actor.name actor.avatar')
1363
- .paginate();
1364
- ```
577
+ const series = await qa.timeSeries({
578
+ dateField: 'createdAt',
579
+ interval: 'day',
580
+ metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
581
+ });
1365
582
 
1366
- **Audit log search — global search + a bounded absolute datetime window:**
583
+ const withMA = QueryAnalytics.movingAverage(series, 'revenue', 7);
584
+ const trend = QueryAnalytics.trend(series, 'revenue');
585
+ const projection = QueryAnalytics.forecast(series, 'revenue', 7);
586
+ const outliers = await qa.stdDevOutliers('total', { threshold: 3 });
1367
587
 
1368
- ```ts
1369
- const result = await new QueryAggregate(AuditLogModel, req.query, { timezone })
1370
- .allowFields(['action', 'actorId', 'createdAt'])
1371
- .dateFields(['createdAt'])
1372
- .where({ orgId: req.user.orgId })
1373
- .rangeCustom('createdAt', {
1374
- from: (req.query.from as string) ?? '2026-01-01T00:00:00',
1375
- to: (req.query.to as string) ?? new Date().toISOString(),
1376
- })
1377
- .filter()
1378
- .globalSearch(['action'])
1379
- .sort()
1380
- .project('action actorId createdAt')
1381
- .paginate();
588
+ res.json({ series: withMA, trend, projection, outliers });
1382
589
  ```
1383
590
 
1384
- **Multi-branch inventory report — objectIdFields + computeField + stats side by side:**
591
+ **Live order feed alongside a polled summary:**
1385
592
 
1386
593
  ```ts
1387
- const base = new QueryAggregate(ProductModel, req.query, { timezone })
1388
- .allowFields(['branchId', 'category', 'stock', 'price'])
1389
- .objectIdFields(['branchId'])
1390
- .where({ deletedAt: null })
1391
- .filter()
1392
- .computeField('stockValue', { $multiply: ['$stock', '$price'] })
1393
- .sort()
1394
- .project('branchId category stock price stockValue');
1395
-
1396
- const [page, totals] = await Promise.all([
1397
- base.clone().paginate(),
1398
- base.clone().stats([
1399
- { name: 'totalStockValue', op: 'sum', field: 'stockValue' },
1400
- { name: 'lowStockAvg', op: 'avg', field: 'stock' },
1401
- { name: 'skuCount', op: 'count' },
1402
- ]),
1403
- ]);
594
+ const qa = new QueryAnalytics(OrderModel, { timezone }).match({ branchId });
1404
595
 
1405
- res.json({ ...page, totals });
1406
- ```
1407
-
1408
- ## Pipeline Stage Ordering
1409
-
1410
- The final pipeline is always assembled in this fixed order:
596
+ const stream = qa.watch(
597
+ (change) => io.to(`branch:${branchId}`).emit('order:change', change),
598
+ {
599
+ operationTypes: ['insert'],
600
+ },
601
+ );
1411
602
 
603
+ req.on('close', () => stream.close());
1412
604
  ```
1413
- $match (from .where() + .filter() + .range()/rangeXxx() + .globalSearch(), merged via $and)
1414
- ↓
1415
- [.lookup() / .addStage() / .computeField() calls, in the order you called them]
1416
- ↓
1417
- $project (from .project(), if set)
1418
- ↓
1419
- $sort (from .sort())
1420
- ↓
1421
- [terminal stage: $skip/$limit, $facet, keyset $match + $limit, or $count/$group — added internally by the method you call]
1422
- ```
1423
-
1424
- Implications:
1425
-
1426
- - Anything added via `.addStage()`/`.computeField()` runs **after** the base filter match and **before** projection/sort.
1427
- - If you `$group`/reshape documents, do it before `.project()` — or skip `.project()` and use `.addStage({ $project: {...} })` for full control over the new shape.
1428
- - If you add your own `$sort` via `.addStage()` _and_ also call `.sort()`, both stages are included (the builder's `.sort()` stage is always appended last in the base pipeline) — harmless but redundant. Prefer one or the other.
1429
- - `.count()`, `.distinct()`, and `.stats()` intentionally skip `$project`/`$sort` — they only need `$match` + your `.lookup()`/`.addStage()` stages before their own terminal `$count`/`$group`.
1430
- - `.paginateCursor()` builds its own `$match`/`$sort`/`$limit` tail (the keyset condition), separate from `.paginate()`'s `$facet` tail.
1431
605
 
1432
606
  ## Security
1433
607
 
1434
- | Protection | Detail |
1435
- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1436
- | **NoSQL injection** | Allowlist enforced on `.filter()`/`.sort()`/`.project()` via `.allowFields()` |
1437
- | **Banned operators** | `$where`, `$expr`, `$function`, `$accumulator`, `$map`, `$reduce`, `$filter` rejected recursively, any depth, in client input |
1438
- | **Destructive stages** | `$out` / `$merge` rejected in `.addStage()` / `.lookup()` sub-pipelines |
1439
- | **ReDoS** | `.globalSearch()` terms are regex-escaped before compilation |
1440
- | **Oversized search** | `?q=` capped at 200 characters |
1441
- | **Deep nesting DoS** | Client filter object nesting capped at depth 5 (`sanitize()`) |
1442
- | **Page size DoS** | `limit` hard-capped at `maxLimit` (default 100), including in `.paginateCursor()` |
1443
- | **Sort abuse** | Sort fields hard-capped at 5 |
1444
- | **Runaway queries** | `maxTimeMS` applied to the aggregation; `onSlowQuery` for observability |
1445
- | **Bad timezone input** | Invalid IANA zone validated once at construction, falls back to default, never throws mid-query |
1446
- | **Bad custom-range input** | `rangeCustomDay`/`rangeBetween`/`rangeLastNDays`/`rangeLastNHours`/`rangeHours`/`rangeCustom` all throw `QueryAggregateValidationError` on malformed dates, non-positive counts, or invalid hour bounds — never silently produce a wrong range |
1447
- | **Server condition override** | `.where()` / `.range()` / every custom range method are merged last via `$and` — URL params can never override them |
1448
- | **Tampered cursors** | `.paginateCursor()` cursors are opaque base64url-encoded JSON; a malformed cursor throws `QueryAggregateValidationError` rather than silently misbehaving |
1449
- | **Untrusted `.addStage()` input** | Not sanitized/allowlisted by design — documented as a trusted, developer-authored-only escape hatch |
1450
-
1451
- > **Important:** `.addStage()` and the `pipeline` option in `.lookup()` are for **server-defined stages only**. They skip the sanitizer/allowlist/operator-rejection that protects `.filter()`/`.where()`. Never interpolate raw `req.query`/`req.body` values into an `.addStage()` call without validating them yourself first.
608
+ | Protection | Detail |
609
+ | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
610
+ | **No client-input path** | `QueryAnalytics` has no `.filter()`/`.allowFields()` — every condition comes from `.match()`, `.dateRange()`, or `.addStage()`, all developer-authored. Don't pass raw `req.query`/`req.body` into any of them without validating it yourself first. |
611
+ | **Destructive stages** | `$out`/`$merge` are rejected recursively, at any nesting depth, in `.addStage()`. |
612
+ | **Strict date parsing** | `.dateRange()` requires `YYYY-MM-DD` and throws `QueryAnalyticsError` on anything else — no silent misinterpretation of ambiguous date strings. |
613
+ | **Runaway queries** | `maxTimeMS` (default `8000`) is applied to every aggregation. |
614
+ | **Bad timezone input** | An invalid IANA timezone is validated once at construction and silently falls back to the default, rather than throwing mid-query. |
615
+ | **MongoDB version requirement** | `cumulativeSeries()`, `rankedBreakdown()`, `sessionize()`, `percentiles()`, and `stdDevOutliers()` require MongoDB 7.0+; there's no feature-detection or fallback, so calling them against an older server surfaces as a server-side aggregation error. |
616
+ | **Change-stream requirements** | `.watch()` requires a replica set or Atlas cluster and only forwards `.match()`/`.dateRange()` conditions onto the stream, not `.addStage()` stages. |
617
+
618
+ > **Important:** because there's no allowlist here, field names passed to `breakdown()`, `multiBreakdown()`, `heatmap()`, etc. should come from your own code or a whitelist you maintain — not directly from a query string — the same way you'd treat any other server-authored aggregation field.
1452
619
 
1453
620
  ## TypeScript Types
1454
621
 
1455
622
  ```ts
1456
623
  import {
1457
- QueryAggregate,
1458
- QueryAggregateOptions,
1459
- QueryAggregateError,
1460
- QueryAggregateValidationError,
1461
- LookupOptions,
1462
- RangePreset,
1463
- SlowQueryInfo,
1464
- QueryParams,
1465
- PaginatedResult,
1466
- CustomDayInput,
1467
- CustomDateRangeInput,
1468
- CustomHourRangeInput,
1469
- CustomDateTimeRangeInput,
1470
- CursorPaginateOptions,
1471
- CursorPaginatedResult,
624
+ QueryAnalytics,
625
+ QueryAnalyticsOptions,
626
+ QueryAnalyticsError,
627
+ Interval,
628
+ ComparePreset,
1472
629
  StatSpec,
630
+ TimeSeriesPoint,
631
+ CompareResult,
632
+ BreakdownRow,
633
+ RankedBreakdownRow,
634
+ FunnelStep,
635
+ FunnelResult,
636
+ CohortRetentionOptions,
637
+ CohortRow,
638
+ SessionizeOptions,
639
+ SessionSummary,
640
+ OutlierRow,
1473
641
  } from 'mongoose-query-find';
1474
642
  ```
1475
643
 
1476
644
  ```ts
1477
- interface QueryAggregateOptions {
645
+ type Interval = 'hour' | 'day' | 'week' | 'month';
646
+
647
+ type ComparePreset =
648
+ | 'today'
649
+ | 'thisWeek'
650
+ | 'thisMonth'
651
+ | 'last7Days'
652
+ | 'last30Days';
653
+
654
+ interface StatSpec {
655
+ name: string;
656
+ op: 'sum' | 'avg' | 'min' | 'max' | 'count' | 'distinctCount';
657
+ field?: string; // required for every op except 'count'
658
+ }
659
+
660
+ interface QueryAnalyticsOptions {
1478
661
  timezone?: string;
1479
662
  maxTimeMS?: number;
1480
- slowQueryThresholdMS?: number;
1481
- maxLimit?: number;
1482
663
  allowDiskUse?: boolean;
1483
664
  weekStartsOn?: 0 | 1 | 2 | 3 | 4 | 5 | 6;
1484
- onSlowQuery?: (info: SlowQueryInfo) => void;
1485
- onSanitizeDrop?: (path: string, value: unknown) => void;
665
+ cacheTTLMs?: number;
1486
666
  }
1487
667
 
1488
- interface LookupOptions {
1489
- from: string;
1490
- as: string;
1491
- localField?: string;
1492
- foreignField?: string;
1493
- pipeline?: PipelineStage[];
1494
- single?: boolean;
668
+ interface TimeSeriesPoint {
669
+ bucket: string; // ISO string, local-boundary-aligned
670
+ [metric: string]: number | string;
1495
671
  }
1496
672
 
1497
- type RangePreset =
1498
- | 'today'
1499
- | 'yesterday'
1500
- | 'last7Days'
1501
- | 'last30Days'
1502
- | 'thisWeek'
1503
- | 'thisMonth';
1504
-
1505
- interface SlowQueryInfo {
1506
- elapsedMs: number;
1507
- pipeline: PipelineStage[];
1508
- sort: Record<string, 1 | -1>;
1509
- page: number;
1510
- limit: number;
673
+ interface CompareResult {
674
+ current: Record<string, number>;
675
+ previous: Record<string, number>;
676
+ change: Record<string, number>;
677
+ changePct: Record<string, number | null>;
1511
678
  }
1512
679
 
1513
- // ── Custom range types ──────────────────────────────────────────────────────
680
+ interface BreakdownRow {
681
+ key: unknown;
682
+ [metric: string]: unknown;
683
+ }
1514
684
 
1515
- type CustomDayInput = string; // "YYYY-MM-DD"
685
+ interface RankedBreakdownRow extends BreakdownRow {
686
+ rank: number;
687
+ }
1516
688
 
1517
- interface CustomDateRangeInput {
1518
- from: string; // "YYYY-MM-DD"
1519
- to: string; // "YYYY-MM-DD"
689
+ interface FunnelStep {
690
+ name: string;
691
+ match: Record<string, unknown>;
1520
692
  }
1521
693
 
1522
- interface CustomHourRangeInput {
1523
- date: string; // "YYYY-MM-DD"
1524
- fromHour: number; // 0–24
1525
- toHour: number; // 0–24, exclusive, must be > fromHour
694
+ interface FunnelResult {
695
+ name: string;
696
+ count: number;
697
+ conversionFromPrevious: number | null;
698
+ conversionFromStart: number | null;
1526
699
  }
1527
700
 
1528
- interface CustomDateTimeRangeInput {
1529
- from: string; // "YYYY-MM-DD" | "YYYY-MM-DDTHH:mm:ss[.sss]" | offset-bearing ISO string
1530
- to: string;
701
+ interface CohortRetentionOptions {
702
+ cohortDateField: string;
703
+ activityDateField: string;
704
+ idField: string;
705
+ cohortInterval: Interval;
706
+ periods: number;
1531
707
  }
1532
708
 
1533
- // ── Advanced-feature types ──────────────────────────────────────────────────
709
+ interface CohortRow {
710
+ cohort: string;
711
+ cohortSize: number;
712
+ retention: (number | null)[];
713
+ retentionPct: (number | null)[];
714
+ }
1534
715
 
1535
- interface CursorPaginateOptions {
1536
- cursor?: string;
1537
- limit?: number;
716
+ interface SessionizeOptions {
717
+ userField: string;
718
+ dateField: string;
719
+ gapMinutes: number;
1538
720
  }
1539
721
 
1540
- interface CursorPaginatedResult<T> {
1541
- data: T[];
1542
- limit: number;
1543
- nextCursor: string | null;
1544
- hasNextPage: boolean;
722
+ interface SessionSummary {
723
+ userId: unknown;
724
+ sessionStart: string;
725
+ sessionEnd: string;
726
+ eventCount: number;
727
+ durationSeconds: number;
1545
728
  }
1546
729
 
1547
- interface StatSpec {
1548
- name: string;
1549
- op: 'sum' | 'avg' | 'min' | 'max' | 'count';
1550
- field?: string;
730
+ interface OutlierRow {
731
+ _id: unknown;
732
+ value: number;
733
+ zScore: number;
734
+ [key: string]: unknown;
1551
735
  }
1552
736
  ```
1553
737
 
1554
- ---
1555
-
1556
738
  ## Links
1557
739
 
1558
740
  - [npm](https://www.npmjs.com/package/mongoose-query-find)
1559
741
  - [GitHub](https://github.com/jsdev-robin/mongoose-query-find)
1560
742
  - [Issues](https://github.com/jsdev-robin/mongoose-query-find/issues)
1561
743
 
1562
- ---
1563
-
1564
744
  ## License
1565
745
 
1566
746
  ISC © [jsdev.robin@gmail.com](mailto:jsdev.robin@gmail.com)