mongoose-query-find 4.0.0 → 4.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +754 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1609,3 +1609,757 @@ interface StatSpec {
|
|
|
1609
1609
|
## License
|
|
1610
1610
|
|
|
1611
1611
|
ISC © [jsdev.robin@gmail.com](mailto:jsdev.robin@gmail.com)
|
|
1612
|
+
|
|
1613
|
+
<br>
|
|
1614
|
+
<br>
|
|
1615
|
+
|
|
1616
|
+
---
|
|
1617
|
+
|
|
1618
|
+
---
|
|
1619
|
+
|
|
1620
|
+
# QueryAnalytics
|
|
1621
|
+
|
|
1622
|
+
A purpose-built analytics/reporting layer for Mongoose (`^8 || ^9`), sitting alongside `QueryFind` and `QueryAggregate`. Where `QueryAggregate` is a general-purpose, client-input-safe pipeline builder, `QueryAnalytics` is a **server-authored dashboard toolkit**: time series, cohort retention, funnels, sessionization, percentiles, and more, each backed by a native MongoDB 7.0+ operator instead of a hand-rolled pipeline.
|
|
1623
|
+
|
|
1624
|
+
## QueryAnalytics Overview
|
|
1625
|
+
|
|
1626
|
+
`QueryAnalytics` wraps `Model.aggregate()` with dashboard-shaped methods that return ready-to-chart data instead of raw pipeline rows:
|
|
1627
|
+
|
|
1628
|
+
- `timeSeries()` / `cumulativeSeries()` — bucketed metrics over time, gap-filled, with a native running total
|
|
1629
|
+
- `compare()` — period-over-period totals and % change for preset windows (today, this week, last 30 days, ...)
|
|
1630
|
+
- `breakdown()` / `rankedBreakdown()` / `multiBreakdown()` — top-N by one or two dimensions, with an optional native dense rank
|
|
1631
|
+
- `funnel()` — step-by-step conversion counts
|
|
1632
|
+
- `cohortRetention()` — a full cohort-by-cohort retention matrix
|
|
1633
|
+
- `sessionize()` — gap-based event sessionization
|
|
1634
|
+
- `percentiles()` / `stdDevOutliers()` — native `$percentile` / `$stdDevPop` statistics
|
|
1635
|
+
- `metrics()` — several independent totals in a single round trip
|
|
1636
|
+
- `heatmap()` — weekday × hour-of-day activity grid
|
|
1637
|
+
- `movingAverage()` / `trend()` / `forecast()` — pure-JS post-processing on a series you already fetched
|
|
1638
|
+
- `watch()` — a live change-stream feed for dashboards that update in real time
|
|
1639
|
+
- `toCSV()` / `toNDJSON()` / `dashboard()` — export and batch helpers
|
|
1640
|
+
|
|
1641
|
+
It targets **MongoDB 7.0+** and uses `$setWindowFields`, `$denseRank`, `$shift`, `$stdDevPop`, and `$percentile` directly — there's no fallback path for older servers, so calling `cumulativeSeries()`, `rankedBreakdown()`, `sessionize()`, `percentiles()`, or `stdDevOutliers()` against a pre-7.0 deployment will fail at the server, not in this library.
|
|
1642
|
+
|
|
1643
|
+
## When to use QueryAnalytics
|
|
1644
|
+
|
|
1645
|
+
| Need | Use |
|
|
1646
|
+
| ---------------------------------------------------------------- | ---------------- |
|
|
1647
|
+
| Filtered/sorted/paginated list from client query params | `QueryFind` |
|
|
1648
|
+
| Joins, grouping, or a client-driven filtered data page | `QueryAggregate` |
|
|
1649
|
+
| A chart-ready time series with gaps filled in | `QueryAnalytics` |
|
|
1650
|
+
| Week-over-week / month-over-month KPI comparisons | `QueryAnalytics` |
|
|
1651
|
+
| Top-N breakdowns, ranked leaderboards, or a two-dimension pivot | `QueryAnalytics` |
|
|
1652
|
+
| Signup → activation → purchase funnel counts | `QueryAnalytics` |
|
|
1653
|
+
| "% of signups still active N weeks later" retention tables | `QueryAnalytics` |
|
|
1654
|
+
| Turning a raw event stream into user sessions | `QueryAnalytics` |
|
|
1655
|
+
| p50/p90/p99 latency or order-value percentiles | `QueryAnalytics` |
|
|
1656
|
+
| Flagging statistical outliers for an anomalies widget | `QueryAnalytics` |
|
|
1657
|
+
| A whole dashboard's worth of metrics in one call from your route | `QueryAnalytics` |
|
|
1658
|
+
| Pushing live updates to a dashboard instead of polling | `QueryAnalytics` |
|
|
1659
|
+
|
|
1660
|
+
`QueryAnalytics` is **not** a client-input parser. Unlike `QueryFind`/`QueryAggregate`, it has no `.filter()`, no `.allowFields()`, and no operator-rejection pass over untrusted input — see [Security](#security-2) below.
|
|
1661
|
+
|
|
1662
|
+
## Installation
|
|
1663
|
+
|
|
1664
|
+
Ships from the same package as `QueryFind` and `QueryAggregate`:
|
|
1665
|
+
|
|
1666
|
+
```bash
|
|
1667
|
+
npm install mongoose-query-find
|
|
1668
|
+
```
|
|
1669
|
+
|
|
1670
|
+
## Quick Start
|
|
1671
|
+
|
|
1672
|
+
```ts
|
|
1673
|
+
import { QueryAnalytics } from 'mongoose-query-find';
|
|
1674
|
+
import OrderModel from './models/order';
|
|
1675
|
+
|
|
1676
|
+
const qa = new QueryAnalytics(OrderModel, { timezone: 'Asia/Dhaka' })
|
|
1677
|
+
.match({ branchId, status: { $ne: 'CANCELLED' } })
|
|
1678
|
+
.dateRange('createdAt', { from: '2026-08-01', to: '2026-08-31' });
|
|
1679
|
+
|
|
1680
|
+
const series = await qa.timeSeries({
|
|
1681
|
+
dateField: 'createdAt',
|
|
1682
|
+
interval: 'day',
|
|
1683
|
+
metrics: [
|
|
1684
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
1685
|
+
{ name: 'orders', op: 'count' },
|
|
1686
|
+
],
|
|
1687
|
+
});
|
|
1688
|
+
```
|
|
1689
|
+
|
|
1690
|
+
`series` is a gap-filled array ready to hand straight to a charting library:
|
|
1691
|
+
|
|
1692
|
+
```json
|
|
1693
|
+
[
|
|
1694
|
+
{ "bucket": "2026-08-01T00:00:00.000Z", "revenue": 4820, "orders": 61 },
|
|
1695
|
+
{ "bucket": "2026-08-02T00:00:00.000Z", "revenue": 0, "orders": 0 },
|
|
1696
|
+
{ "bucket": "2026-08-03T00:00:00.000Z", "revenue": 5310, "orders": 74 }
|
|
1697
|
+
]
|
|
1698
|
+
```
|
|
1699
|
+
|
|
1700
|
+
## Constructor & Options
|
|
1701
|
+
|
|
1702
|
+
```ts
|
|
1703
|
+
new QueryAnalytics<T>(model, options?);
|
|
1704
|
+
```
|
|
1705
|
+
|
|
1706
|
+
| Parameter | Type | Description |
|
|
1707
|
+
| --------- | ----------------------- | --------------------------- |
|
|
1708
|
+
| `model` | `Model<T>` | A Mongoose model |
|
|
1709
|
+
| `options` | `QueryAnalyticsOptions` | Configuration, all optional |
|
|
1710
|
+
|
|
1711
|
+
### `QueryAnalyticsOptions`
|
|
1712
|
+
|
|
1713
|
+
| Option | Type | Default | Description |
|
|
1714
|
+
| -------------- | --------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1715
|
+
| `timezone` | `string` | `'Asia/Dhaka'` | IANA timezone for bucketing and range math. Invalid values are validated once at construction and silently fall back to the default. |
|
|
1716
|
+
| `maxTimeMS` | `number` | `8000` | Max time MongoDB may spend per query. Higher than `QueryFind`/`QueryAggregate`'s default since analytics pipelines tend to be heavier. `0` disables. |
|
|
1717
|
+
| `allowDiskUse` | `boolean` | `false` | Forwarded to `.allowDiskUse()` for large `$group`/`$setWindowFields` stages. |
|
|
1718
|
+
| `weekStartsOn` | `0–6` | `6` (Saturday) | Used by weekly bucketing (`interval: 'week'`), `compare({ preset: 'thisWeek' })`, and `cohortRetention()`. |
|
|
1719
|
+
| `cacheTTLMs` | `number` | `0` | TTL in ms for the built-in in-memory cache. `0` disables caching. **See note below.** |
|
|
1720
|
+
|
|
1721
|
+
> **Note on `cacheTTLMs`:** the class ships a small in-memory TTL cache (`cacheGet`/`cacheSet`) and a `QueryAnalytics.clearCache()` static, but as of this release none of the public methods (`timeSeries`, `breakdown`, `metrics`, etc.) pass a cache key into it — so setting `cacheTTLMs` doesn't yet cache anything on its own. Treat it as reserved for a future release; if you need caching today, memoize at the call site (e.g. wrap `dashboard()` calls in your own short-lived cache).
|
|
1722
|
+
|
|
1723
|
+
## Base Filter Methods
|
|
1724
|
+
|
|
1725
|
+
### `.match(conditions)`
|
|
1726
|
+
|
|
1727
|
+
Merges hard, developer-authored conditions into the base `$match`. Call as many times as needed — later calls merge over earlier ones on the same key via `Object.assign`.
|
|
1728
|
+
|
|
1729
|
+
```ts
|
|
1730
|
+
qa.match({ branchId, status: { $ne: 'CANCELLED' } });
|
|
1731
|
+
```
|
|
1732
|
+
|
|
1733
|
+
Because `QueryAnalytics` has no `.filter()`, `match()` is the _only_ way conditions enter the pipeline besides `.dateRange()` and `.addStage()` — there's no allowlist because there's no untrusted input path. Don't pass raw `req.query` into it directly; validate/whitelist any client-influenced value yourself first (see [Security](#security-2)).
|
|
1734
|
+
|
|
1735
|
+
### `.dateRange(field: string, range: { from: string; to: string })`
|
|
1736
|
+
|
|
1737
|
+
Hard, inclusive local-date range condition on `field`, converted to a UTC `$gte`/`$lt` window using the instance's timezone.
|
|
1738
|
+
|
|
1739
|
+
```ts
|
|
1740
|
+
qa.dateRange('createdAt', { from: '2026-08-01', to: '2026-08-31' });
|
|
1741
|
+
```
|
|
1742
|
+
|
|
1743
|
+
Throws `QueryAnalyticsError` if `from`/`to` aren't `YYYY-MM-DD`.
|
|
1744
|
+
|
|
1745
|
+
### `.addStage(stage: PipelineStage)`
|
|
1746
|
+
|
|
1747
|
+
Escape hatch for extra developer-authored stages, spliced in right after the base `$match`, before any analytics-specific stages. Blocks `$out`/`$merge` recursively, at any nesting depth.
|
|
1748
|
+
|
|
1749
|
+
```ts
|
|
1750
|
+
qa.addStage({ $addFields: { margin: { $subtract: ['$price', '$cost'] } } });
|
|
1751
|
+
```
|
|
1752
|
+
|
|
1753
|
+
### `.clone()`
|
|
1754
|
+
|
|
1755
|
+
Copies the current filter state (`match()`/`dateRange()` conditions and `addStage()` stages) into a new, independent `QueryAnalytics` instance sharing the same options. Handy for branching one filtered base into several analytics calls without rebuilding the filter each time — see [Recipes](#recipes-1).
|
|
1756
|
+
|
|
1757
|
+
## Time Series
|
|
1758
|
+
|
|
1759
|
+
### `.timeSeries(opts)`
|
|
1760
|
+
|
|
1761
|
+
Buckets documents by `dateField` into timezone-aligned `interval` buckets (`'hour' | 'day' | 'week' | 'month'`) via `$dateTrunc`, computes `metrics` per bucket, and — by default — fills empty buckets with zero values so a chart never shows a broken line for a quiet day.
|
|
1762
|
+
|
|
1763
|
+
```ts
|
|
1764
|
+
interface StatSpec {
|
|
1765
|
+
name: string;
|
|
1766
|
+
op: 'sum' | 'avg' | 'min' | 'max' | 'count' | 'distinctCount';
|
|
1767
|
+
field?: string; // required for every op except 'count'
|
|
1768
|
+
}
|
|
1769
|
+
|
|
1770
|
+
await qa.timeSeries({
|
|
1771
|
+
dateField: 'createdAt',
|
|
1772
|
+
interval: 'day',
|
|
1773
|
+
metrics: [
|
|
1774
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
1775
|
+
{ name: 'orders', op: 'count' },
|
|
1776
|
+
{ name: 'customers', op: 'distinctCount', field: 'customerId' },
|
|
1777
|
+
],
|
|
1778
|
+
fillGaps: true, // default
|
|
1779
|
+
});
|
|
1780
|
+
```
|
|
1781
|
+
|
|
1782
|
+
`interval: 'week'` respects the constructor's `weekStartsOn`. Set `fillGaps: false` to get back only buckets that actually had data.
|
|
1783
|
+
|
|
1784
|
+
### `.cumulativeSeries(opts)`
|
|
1785
|
+
|
|
1786
|
+
Same bucketing as `timeSeries()`, plus a native running-total column per metric (`${name}_cumulative`), computed server-side via `$setWindowFields` — no client-side accumulation pass.
|
|
1787
|
+
|
|
1788
|
+
```ts
|
|
1789
|
+
await qa.cumulativeSeries({
|
|
1790
|
+
dateField: 'createdAt',
|
|
1791
|
+
interval: 'day',
|
|
1792
|
+
metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
1793
|
+
});
|
|
1794
|
+
// [{ bucket: '...', revenue: 420, revenue_cumulative: 420 }, { bucket: '...', revenue: 310, revenue_cumulative: 730 }, ...]
|
|
1795
|
+
```
|
|
1796
|
+
|
|
1797
|
+
`distinctCount` metrics are skipped for the cumulative column — a running distinct-count isn't a well-defined windowed sum. Unlike `timeSeries()`, this method has no `fillGaps` option.
|
|
1798
|
+
|
|
1799
|
+
## Comparisons & Breakdowns
|
|
1800
|
+
|
|
1801
|
+
### `.compare(opts)`
|
|
1802
|
+
|
|
1803
|
+
Computes `metrics` for a preset current period and the equivalent prior period, plus absolute and percentage change.
|
|
1804
|
+
|
|
1805
|
+
```ts
|
|
1806
|
+
interface CompareResult {
|
|
1807
|
+
current: Record<string, number>;
|
|
1808
|
+
previous: Record<string, number>;
|
|
1809
|
+
change: Record<string, number>; // absolute delta
|
|
1810
|
+
changePct: Record<string, number | null>; // null when previous was 0
|
|
1811
|
+
}
|
|
1812
|
+
|
|
1813
|
+
const result = await qa.compare({
|
|
1814
|
+
dateField: 'createdAt',
|
|
1815
|
+
metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
1816
|
+
preset: 'last7Days',
|
|
1817
|
+
});
|
|
1818
|
+
```
|
|
1819
|
+
|
|
1820
|
+
**Presets:** `'today' | 'thisWeek' | 'thisMonth' | 'last7Days' | 'last30Days'`. `'thisWeek'`/`'thisMonth'` compare against a prior period of the same length (e.g. 12 days into this month vs. the last 12 days of last month), and `'thisWeek'` respects `weekStartsOn`.
|
|
1821
|
+
|
|
1822
|
+
> Any `.dateRange()` you've already applied on the instance is **ignored** here — the preset defines both windows on its own, independent of the instance's filter state. `.match()` conditions still apply to both windows.
|
|
1823
|
+
|
|
1824
|
+
### `.breakdown(field, metrics, opts)`
|
|
1825
|
+
|
|
1826
|
+
Groups by `field`, computes `metrics` per group, sorted and capped.
|
|
1827
|
+
|
|
1828
|
+
```ts
|
|
1829
|
+
await qa.breakdown(
|
|
1830
|
+
'category',
|
|
1831
|
+
[{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
1832
|
+
{
|
|
1833
|
+
limit: 10, // default 10
|
|
1834
|
+
sortBy: 'revenue', // default: metrics[0].name
|
|
1835
|
+
sortDir: -1, // default -1
|
|
1836
|
+
},
|
|
1837
|
+
);
|
|
1838
|
+
// [{ key: 'Electronics', revenue: 18400 }, { key: 'Home', revenue: 9210 }, ...]
|
|
1839
|
+
```
|
|
1840
|
+
|
|
1841
|
+
### `.rankedBreakdown(field, metrics, opts)`
|
|
1842
|
+
|
|
1843
|
+
Same as `breakdown()`, but adds a native 1-based dense-rank column via `$setWindowFields`/`$denseRank` — no client-side re-sorting needed for a leaderboard UI.
|
|
1844
|
+
|
|
1845
|
+
```ts
|
|
1846
|
+
await qa.rankedBreakdown('salesRepId', [
|
|
1847
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
1848
|
+
]);
|
|
1849
|
+
// [{ key: 'rep_1', rank: 1, revenue: 18400 }, { key: 'rep_2', rank: 2, revenue: 18400 }, { key: 'rep_3', rank: 4, revenue: 9210 }, ...]
|
|
1850
|
+
```
|
|
1851
|
+
|
|
1852
|
+
Ties share a rank and the next distinct value skips ahead (standard dense-rank semantics), so two reps tied for first both show `rank: 1` and the next lowest gets `rank: 2`.
|
|
1853
|
+
|
|
1854
|
+
### `.multiBreakdown(fields, metrics, opts)`
|
|
1855
|
+
|
|
1856
|
+
Pivot table: groups by two dimensions at once, one row per `(dimA, dimB)` pair.
|
|
1857
|
+
|
|
1858
|
+
```ts
|
|
1859
|
+
await qa.multiBreakdown(
|
|
1860
|
+
['branchId', 'status'],
|
|
1861
|
+
[{ name: 'orders', op: 'count' }],
|
|
1862
|
+
{
|
|
1863
|
+
limit: 200, // optional; no cap by default
|
|
1864
|
+
},
|
|
1865
|
+
);
|
|
1866
|
+
// [{ key: 'branch_a', key2: 'PAID', orders: 40 }, { key: 'branch_a', key2: 'REFUNDED', orders: 3 }, ...]
|
|
1867
|
+
```
|
|
1868
|
+
|
|
1869
|
+
Rows are sorted by `key` then `key2` ascending. There's no default `limit` — pass one for large dimension combinations.
|
|
1870
|
+
|
|
1871
|
+
## Funnels & Cohorts
|
|
1872
|
+
|
|
1873
|
+
### `.funnel(steps)`
|
|
1874
|
+
|
|
1875
|
+
Counts how many base documents match each step's condition, independently, in parallel.
|
|
1876
|
+
|
|
1877
|
+
```ts
|
|
1878
|
+
const result = await qa.funnel([
|
|
1879
|
+
{ name: 'Visited', match: {} },
|
|
1880
|
+
{ name: 'Signed up', match: { signedUpAt: { $ne: null } } },
|
|
1881
|
+
{ name: 'Purchased', match: { firstOrderAt: { $ne: null } } },
|
|
1882
|
+
]);
|
|
1883
|
+
```
|
|
1884
|
+
|
|
1885
|
+
```ts
|
|
1886
|
+
interface FunnelResult {
|
|
1887
|
+
name: string;
|
|
1888
|
+
count: number;
|
|
1889
|
+
conversionFromPrevious: number | null; // %
|
|
1890
|
+
conversionFromStart: number | null; // %
|
|
1891
|
+
}
|
|
1892
|
+
```
|
|
1893
|
+
|
|
1894
|
+
Each step's `match` is ANDed with the instance's base filter — steps are **independent counts**, not automatically cumulative. If step 3 should imply step 2, include that condition in step 3's `match` yourself. `conversionFromPrevious`/`conversionFromStart` are `null` for the first step, and `null` (instead of `Infinity`/`NaN`) whenever the denominator is 0.
|
|
1895
|
+
|
|
1896
|
+
### `.cohortRetention(opts)`
|
|
1897
|
+
|
|
1898
|
+
Buckets entities into cohorts by the first occurrence of `cohortDateField`, then measures how many of each cohort had activity (`activityDateField`) in each subsequent period.
|
|
1899
|
+
|
|
1900
|
+
```ts
|
|
1901
|
+
interface CohortRetentionOptions {
|
|
1902
|
+
cohortDateField: string; // e.g. signup date
|
|
1903
|
+
activityDateField: string; // e.g. any return-visit date (can equal cohortDateField)
|
|
1904
|
+
idField: string; // e.g. userId
|
|
1905
|
+
cohortInterval: 'hour' | 'day' | 'week' | 'month';
|
|
1906
|
+
periods: number; // how many periods forward to measure
|
|
1907
|
+
}
|
|
1908
|
+
|
|
1909
|
+
const table = await qa.cohortRetention({
|
|
1910
|
+
cohortDateField: 'signupAt',
|
|
1911
|
+
activityDateField: 'lastActiveAt',
|
|
1912
|
+
idField: 'userId',
|
|
1913
|
+
cohortInterval: 'week',
|
|
1914
|
+
periods: 8,
|
|
1915
|
+
});
|
|
1916
|
+
```
|
|
1917
|
+
|
|
1918
|
+
```ts
|
|
1919
|
+
interface CohortRow {
|
|
1920
|
+
cohort: string; // ISO bucket start
|
|
1921
|
+
cohortSize: number;
|
|
1922
|
+
retention: (number | null)[]; // index 0 = period 0
|
|
1923
|
+
retentionPct: (number | null)[];
|
|
1924
|
+
}
|
|
1925
|
+
```
|
|
1926
|
+
|
|
1927
|
+
Two aggregate calls run first (cohort assignment, then distinct activity buckets), and the retention matrix itself is assembled in JS afterward. Notes:
|
|
1928
|
+
|
|
1929
|
+
- `retention[0]` (period 0) falls back to the full `cohortSize` when there's no explicit activity row recorded at offset 0 — useful when the cohort event and the first activity event are the same document, but worth checking if your two date fields can genuinely diverge at period 0.
|
|
1930
|
+
- For `cohortInterval: 'month'`, offsets are computed against an approximate 30-day month for period-bucketing math; the cohort buckets themselves are exact (via `$dateTrunc`), only the offset-from-cohort calculation is approximate.
|
|
1931
|
+
|
|
1932
|
+
### `.sessionize(opts)`
|
|
1933
|
+
|
|
1934
|
+
Groups per-user events into sessions using native `$setWindowFields` + `$shift` gap detection — no self-lookup or JS grouping pass.
|
|
1935
|
+
|
|
1936
|
+
```ts
|
|
1937
|
+
interface SessionizeOptions {
|
|
1938
|
+
userField: string;
|
|
1939
|
+
dateField: string;
|
|
1940
|
+
gapMinutes: number; // idle gap after which a new session starts
|
|
1941
|
+
}
|
|
1942
|
+
|
|
1943
|
+
const sessions = await qa.sessionize({
|
|
1944
|
+
userField: 'userId',
|
|
1945
|
+
dateField: 'eventAt',
|
|
1946
|
+
gapMinutes: 30,
|
|
1947
|
+
});
|
|
1948
|
+
```
|
|
1949
|
+
|
|
1950
|
+
```ts
|
|
1951
|
+
interface SessionSummary {
|
|
1952
|
+
userId: unknown;
|
|
1953
|
+
sessionStart: string;
|
|
1954
|
+
sessionEnd: string;
|
|
1955
|
+
eventCount: number;
|
|
1956
|
+
durationSeconds: number;
|
|
1957
|
+
}
|
|
1958
|
+
```
|
|
1959
|
+
|
|
1960
|
+
## Statistics
|
|
1961
|
+
|
|
1962
|
+
### `.percentiles(field, ps)`
|
|
1963
|
+
|
|
1964
|
+
Percentiles (0–100) of `field` over the current filter, via native `$percentile` (`method: 'approximate'`).
|
|
1965
|
+
|
|
1966
|
+
```ts
|
|
1967
|
+
await qa.percentiles('total', [50, 90, 99]);
|
|
1968
|
+
// { p50: 42, p90: 118, p99: 340 }
|
|
1969
|
+
```
|
|
1970
|
+
|
|
1971
|
+
Values are `null` when there's no matching data.
|
|
1972
|
+
|
|
1973
|
+
### `.stdDevOutliers(field, opts)`
|
|
1974
|
+
|
|
1975
|
+
Flags documents whose `field` value is more than `threshold` standard deviations from the mean, using native `$stdDevPop`.
|
|
1976
|
+
|
|
1977
|
+
```ts
|
|
1978
|
+
await qa.stdDevOutliers('total', { threshold: 3, limit: 50 }); // defaults shown
|
|
1979
|
+
```
|
|
1980
|
+
|
|
1981
|
+
```ts
|
|
1982
|
+
interface OutlierRow {
|
|
1983
|
+
_id: unknown;
|
|
1984
|
+
value: number;
|
|
1985
|
+
zScore: number;
|
|
1986
|
+
[key: string]: unknown; // full source document is spread in
|
|
1987
|
+
}
|
|
1988
|
+
```
|
|
1989
|
+
|
|
1990
|
+
Two round trips: one to compute `mean`/`stdDev` over the full filtered set, one to fetch flagged rows. Returns `[]` if there's no data or the standard deviation is 0 (constant field).
|
|
1991
|
+
|
|
1992
|
+
> **Caveat:** rows are sorted by `zScore` descending before `limit` is applied, so with a tight `limit` on a field that has outliers on both tails, strongly _positive_ outliers can crowd out strongly _negative_ ones. Raise `limit` (or drop the `$limit` and sort by `Math.abs(zScore)` yourself) if you need both tails represented fairly.
|
|
1993
|
+
|
|
1994
|
+
### `.metrics(specs)`
|
|
1995
|
+
|
|
1996
|
+
Several independent `StatSpec` metrics in one round trip — cheaper than N separate calls for a KPI-strip widget.
|
|
1997
|
+
|
|
1998
|
+
```ts
|
|
1999
|
+
await qa.metrics([
|
|
2000
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
2001
|
+
{ name: 'avgOrder', op: 'avg', field: 'total' },
|
|
2002
|
+
{ name: 'orders', op: 'count' },
|
|
2003
|
+
]);
|
|
2004
|
+
// { revenue: 48213, avgOrder: 62.1, orders: 776 }
|
|
2005
|
+
```
|
|
2006
|
+
|
|
2007
|
+
## Post-Processing Helpers (static)
|
|
2008
|
+
|
|
2009
|
+
These run entirely in JS against a series you've already fetched — no database call.
|
|
2010
|
+
|
|
2011
|
+
### `QueryAnalytics.movingAverage()`
|
|
2012
|
+
|
|
2013
|
+
Adds a trailing moving-average field to an existing time series.
|
|
2014
|
+
|
|
2015
|
+
```ts
|
|
2016
|
+
QueryAnalytics.movingAverage(series, 'revenue', 7); // adds `revenue_ma`
|
|
2017
|
+
```
|
|
2018
|
+
|
|
2019
|
+
The window shrinks near the start of the series rather than looking backward past index 0 (so the first point's average is just itself, the second is the average of the first two, and so on). Throws `QueryAnalyticsError` if `windowSize < 1`.
|
|
2020
|
+
|
|
2021
|
+
### `QueryAnalytics.trend()`
|
|
2022
|
+
|
|
2023
|
+
Simple up/down/flat classification, comparing the average of the last `n` points to the `n` before that.
|
|
2024
|
+
|
|
2025
|
+
```ts
|
|
2026
|
+
QueryAnalytics.trend(series, 'revenue', 3, 2); // n=3, flatThresholdPct=2 (defaults)
|
|
2027
|
+
// 'up' | 'down' | 'flat' | 'insufficient_data'
|
|
2028
|
+
```
|
|
2029
|
+
|
|
2030
|
+
Returns `'insufficient_data'` when the series has fewer than `2 * n` points.
|
|
2031
|
+
|
|
2032
|
+
### `QueryAnalytics.forecast()`
|
|
2033
|
+
|
|
2034
|
+
A simple linear-regression projection for `periodsAhead` future buckets, based on the series' existing trend line.
|
|
2035
|
+
|
|
2036
|
+
```ts
|
|
2037
|
+
QueryAnalytics.forecast(series, 'revenue', 7);
|
|
2038
|
+
// [{ index: 30, predicted: 512.4 }, { index: 31, predicted: 518.9 }, ...]
|
|
2039
|
+
```
|
|
2040
|
+
|
|
2041
|
+
Good enough for a dashed projection line on a chart — not a substitute for a real forecasting model. Returns `[]` for series shorter than 2 points.
|
|
2042
|
+
|
|
2043
|
+
## `.heatmap(dateField, valueField?)`
|
|
2044
|
+
|
|
2045
|
+
Counts (or sums `valueField`, if given) by weekday (0=Sun..6=Sat, local timezone) × hour-of-day (0–23, local timezone).
|
|
2046
|
+
|
|
2047
|
+
```ts
|
|
2048
|
+
await qa.heatmap('createdAt');
|
|
2049
|
+
// [{ weekday: 0, hour: 0, value: 4 }, { weekday: 0, hour: 1, value: 1 }, ...]
|
|
2050
|
+
|
|
2051
|
+
await qa.heatmap('createdAt', 'total'); // sum revenue instead of counting docs
|
|
2052
|
+
```
|
|
2053
|
+
|
|
2054
|
+
Returns a flat array (one row per `weekday`/`hour` combination that occurred), not a pre-filled 7×24 matrix — fill in the zero cells client-side if your chart needs a dense grid.
|
|
2055
|
+
|
|
2056
|
+
## Live Updates
|
|
2057
|
+
|
|
2058
|
+
### `.watch(onChange, opts)`
|
|
2059
|
+
|
|
2060
|
+
Subscribes to a MongoDB change stream on the model's collection and invokes `onChange` for every matching insert/update/delete/replace — handy for pushing live dashboard updates over a websocket instead of polling.
|
|
2061
|
+
|
|
2062
|
+
```ts
|
|
2063
|
+
const stream = qa.watch(
|
|
2064
|
+
(change) => io.emit('order:change', change),
|
|
2065
|
+
{ operationTypes: ['insert', 'update'] }, // optional; all types by default
|
|
2066
|
+
);
|
|
2067
|
+
|
|
2068
|
+
// later
|
|
2069
|
+
stream.close();
|
|
2070
|
+
```
|
|
2071
|
+
|
|
2072
|
+
Returns the underlying `ChangeStream` — the caller owns its lifecycle and must `.close()` it. `fullDocument: 'updateLookup'` is set so update events include the full document.
|
|
2073
|
+
|
|
2074
|
+
Two things worth knowing:
|
|
2075
|
+
|
|
2076
|
+
- Only conditions set via `.match()`/`.dateRange()` are translated onto the stream (as `fullDocument.<field>` equality conditions) — stages added via `.addStage()` aren't part of the change-stream pipeline.
|
|
2077
|
+
- Change streams require a MongoDB replica set or Atlas cluster; they don't work against a standalone `mongod`.
|
|
2078
|
+
|
|
2079
|
+
## Export & Batch Helpers (static)
|
|
2080
|
+
|
|
2081
|
+
### `QueryAnalytics.toCSV()` / `toNDJSON()`
|
|
2082
|
+
|
|
2083
|
+
Flatten an array of plain-object rows (e.g. the output of `timeSeries()`/`breakdown()`) into CSV or newline-delimited JSON.
|
|
2084
|
+
|
|
2085
|
+
```ts
|
|
2086
|
+
QueryAnalytics.toCSV(rows); // header row inferred from the union of all row keys; values with , " or \n are quoted
|
|
2087
|
+
QueryAnalytics.toNDJSON(rows); // one JSON.stringify(row) per line — friendlier for streaming large exports
|
|
2088
|
+
```
|
|
2089
|
+
|
|
2090
|
+
### `QueryAnalytics.dashboard()`
|
|
2091
|
+
|
|
2092
|
+
Runs several named analytics calls in parallel and returns them keyed by name — one round trip from the caller's perspective (though each named task is still its own aggregate call under the hood).
|
|
2093
|
+
|
|
2094
|
+
```ts
|
|
2095
|
+
const data = await QueryAnalytics.dashboard({
|
|
2096
|
+
revenue: () =>
|
|
2097
|
+
qa.timeSeries({
|
|
2098
|
+
dateField: 'createdAt',
|
|
2099
|
+
interval: 'day',
|
|
2100
|
+
metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
2101
|
+
}),
|
|
2102
|
+
topCategories: () =>
|
|
2103
|
+
qa.breakdown('category', [{ name: 'revenue', op: 'sum', field: 'total' }]),
|
|
2104
|
+
totals: () => qa.metrics([{ name: 'orders', op: 'count' }]),
|
|
2105
|
+
});
|
|
2106
|
+
// { revenue: [...], topCategories: [...], totals: { orders: 776 } }
|
|
2107
|
+
```
|
|
2108
|
+
|
|
2109
|
+
### `QueryAnalytics.clearCache()`
|
|
2110
|
+
|
|
2111
|
+
Clears the shared in-memory cache. **Process-wide** — not scoped to one instance or one model. (See the caching note under [Constructor & Options](#constructor--options-2): as of this release, nothing populates the cache yet via the public API.)
|
|
2112
|
+
|
|
2113
|
+
## Recipes
|
|
2114
|
+
|
|
2115
|
+
**Branch dashboard in one round trip:**
|
|
2116
|
+
|
|
2117
|
+
```ts
|
|
2118
|
+
const qa = new QueryAnalytics(OrderModel, { timezone })
|
|
2119
|
+
.match({ branchId: req.self.linkedTo, status: { $ne: 'CANCELLED' } })
|
|
2120
|
+
.dateRange('createdAt', {
|
|
2121
|
+
from: req.query.from as string,
|
|
2122
|
+
to: req.query.to as string,
|
|
2123
|
+
});
|
|
2124
|
+
|
|
2125
|
+
const dashboard = await QueryAnalytics.dashboard({
|
|
2126
|
+
series: () =>
|
|
2127
|
+
qa.timeSeries({
|
|
2128
|
+
dateField: 'createdAt',
|
|
2129
|
+
interval: 'day',
|
|
2130
|
+
metrics: [
|
|
2131
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
2132
|
+
{ name: 'orders', op: 'count' },
|
|
2133
|
+
],
|
|
2134
|
+
}),
|
|
2135
|
+
topProducts: () =>
|
|
2136
|
+
qa.breakdown(
|
|
2137
|
+
'productId',
|
|
2138
|
+
[{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
2139
|
+
{ limit: 5 },
|
|
2140
|
+
),
|
|
2141
|
+
kpis: () =>
|
|
2142
|
+
qa.metrics([
|
|
2143
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
2144
|
+
{ name: 'orders', op: 'count' },
|
|
2145
|
+
]),
|
|
2146
|
+
});
|
|
2147
|
+
|
|
2148
|
+
res.json(dashboard);
|
|
2149
|
+
```
|
|
2150
|
+
|
|
2151
|
+
**Week-over-week KPI strip:**
|
|
2152
|
+
|
|
2153
|
+
```ts
|
|
2154
|
+
const wow = await new QueryAnalytics(OrderModel, { timezone })
|
|
2155
|
+
.match({ branchId: req.self.linkedTo })
|
|
2156
|
+
.compare({
|
|
2157
|
+
dateField: 'createdAt',
|
|
2158
|
+
preset: 'thisWeek',
|
|
2159
|
+
metrics: [
|
|
2160
|
+
{ name: 'revenue', op: 'sum', field: 'total' },
|
|
2161
|
+
{ name: 'orders', op: 'count' },
|
|
2162
|
+
],
|
|
2163
|
+
});
|
|
2164
|
+
|
|
2165
|
+
res.json(wow); // { current, previous, change, changePct }
|
|
2166
|
+
```
|
|
2167
|
+
|
|
2168
|
+
**Signup retention curve, branched off one base filter:**
|
|
2169
|
+
|
|
2170
|
+
```ts
|
|
2171
|
+
const base = new QueryAnalytics(UserModel, { timezone }).match({
|
|
2172
|
+
orgId: req.user.orgId,
|
|
2173
|
+
});
|
|
2174
|
+
|
|
2175
|
+
const [retention, funnel] = await Promise.all([
|
|
2176
|
+
base.clone().cohortRetention({
|
|
2177
|
+
cohortDateField: 'signupAt',
|
|
2178
|
+
activityDateField: 'lastActiveAt',
|
|
2179
|
+
idField: '_id',
|
|
2180
|
+
cohortInterval: 'week',
|
|
2181
|
+
periods: 8,
|
|
2182
|
+
}),
|
|
2183
|
+
base.clone().funnel([
|
|
2184
|
+
{ name: 'Signed up', match: {} },
|
|
2185
|
+
{ name: 'Activated', match: { activatedAt: { $ne: null } } },
|
|
2186
|
+
{ name: 'Subscribed', match: { subscribedAt: { $ne: null } } },
|
|
2187
|
+
]),
|
|
2188
|
+
]);
|
|
2189
|
+
```
|
|
2190
|
+
|
|
2191
|
+
**Anomalies widget with a projected trend line:**
|
|
2192
|
+
|
|
2193
|
+
```ts
|
|
2194
|
+
const qa = new QueryAnalytics(OrderModel, { timezone }).match({ branchId });
|
|
2195
|
+
|
|
2196
|
+
const series = await qa.timeSeries({
|
|
2197
|
+
dateField: 'createdAt',
|
|
2198
|
+
interval: 'day',
|
|
2199
|
+
metrics: [{ name: 'revenue', op: 'sum', field: 'total' }],
|
|
2200
|
+
});
|
|
2201
|
+
|
|
2202
|
+
const withMA = QueryAnalytics.movingAverage(series, 'revenue', 7);
|
|
2203
|
+
const trend = QueryAnalytics.trend(series, 'revenue');
|
|
2204
|
+
const projection = QueryAnalytics.forecast(series, 'revenue', 7);
|
|
2205
|
+
const outliers = await qa.stdDevOutliers('total', { threshold: 3 });
|
|
2206
|
+
|
|
2207
|
+
res.json({ series: withMA, trend, projection, outliers });
|
|
2208
|
+
```
|
|
2209
|
+
|
|
2210
|
+
**Live order feed alongside a polled summary:**
|
|
2211
|
+
|
|
2212
|
+
```ts
|
|
2213
|
+
const qa = new QueryAnalytics(OrderModel, { timezone }).match({ branchId });
|
|
2214
|
+
|
|
2215
|
+
const stream = qa.watch(
|
|
2216
|
+
(change) => io.to(`branch:${branchId}`).emit('order:change', change),
|
|
2217
|
+
{
|
|
2218
|
+
operationTypes: ['insert'],
|
|
2219
|
+
},
|
|
2220
|
+
);
|
|
2221
|
+
|
|
2222
|
+
req.on('close', () => stream.close());
|
|
2223
|
+
```
|
|
2224
|
+
|
|
2225
|
+
## Security
|
|
2226
|
+
|
|
2227
|
+
| Protection | Detail |
|
|
2228
|
+
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
2229
|
+
| **No client-input path** | `QueryAnalytics` has no `.filter()`/`.allowFields()` — every condition comes from `.match()`, `.dateRange()`, or `.addStage()`, all developer-authored. Don't pass raw `req.query`/`req.body` into any of them without validating it yourself first. |
|
|
2230
|
+
| **Destructive stages** | `$out`/`$merge` are rejected recursively, at any nesting depth, in `.addStage()`. |
|
|
2231
|
+
| **Strict date parsing** | `.dateRange()` requires `YYYY-MM-DD` and throws `QueryAnalyticsError` on anything else — no silent misinterpretation of ambiguous date strings. |
|
|
2232
|
+
| **Runaway queries** | `maxTimeMS` (default `8000`) is applied to every aggregation. |
|
|
2233
|
+
| **Bad timezone input** | An invalid IANA timezone is validated once at construction and silently falls back to the default, rather than throwing mid-query. |
|
|
2234
|
+
| **MongoDB version requirement** | `cumulativeSeries()`, `rankedBreakdown()`, `sessionize()`, `percentiles()`, and `stdDevOutliers()` require MongoDB 7.0+; there's no feature-detection or fallback, so calling them against an older server surfaces as a server-side aggregation error. |
|
|
2235
|
+
| **Change-stream requirements** | `.watch()` requires a replica set or Atlas cluster and only forwards `.match()`/`.dateRange()` conditions onto the stream, not `.addStage()` stages. |
|
|
2236
|
+
|
|
2237
|
+
> **Important:** because there's no allowlist here, field names passed to `breakdown()`, `multiBreakdown()`, `heatmap()`, etc. should come from your own code or a whitelist you maintain — not directly from a query string — the same way you'd treat any other server-authored aggregation field.
|
|
2238
|
+
|
|
2239
|
+
## TypeScript Types
|
|
2240
|
+
|
|
2241
|
+
```ts
|
|
2242
|
+
import {
|
|
2243
|
+
QueryAnalytics,
|
|
2244
|
+
QueryAnalyticsOptions,
|
|
2245
|
+
QueryAnalyticsError,
|
|
2246
|
+
Interval,
|
|
2247
|
+
ComparePreset,
|
|
2248
|
+
StatSpec,
|
|
2249
|
+
TimeSeriesPoint,
|
|
2250
|
+
CompareResult,
|
|
2251
|
+
BreakdownRow,
|
|
2252
|
+
RankedBreakdownRow,
|
|
2253
|
+
FunnelStep,
|
|
2254
|
+
FunnelResult,
|
|
2255
|
+
CohortRetentionOptions,
|
|
2256
|
+
CohortRow,
|
|
2257
|
+
SessionizeOptions,
|
|
2258
|
+
SessionSummary,
|
|
2259
|
+
OutlierRow,
|
|
2260
|
+
} from 'mongoose-query-find';
|
|
2261
|
+
```
|
|
2262
|
+
|
|
2263
|
+
```ts
|
|
2264
|
+
type Interval = 'hour' | 'day' | 'week' | 'month';
|
|
2265
|
+
|
|
2266
|
+
type ComparePreset =
|
|
2267
|
+
| 'today'
|
|
2268
|
+
| 'thisWeek'
|
|
2269
|
+
| 'thisMonth'
|
|
2270
|
+
| 'last7Days'
|
|
2271
|
+
| 'last30Days';
|
|
2272
|
+
|
|
2273
|
+
interface StatSpec {
|
|
2274
|
+
name: string;
|
|
2275
|
+
op: 'sum' | 'avg' | 'min' | 'max' | 'count' | 'distinctCount';
|
|
2276
|
+
field?: string; // required for every op except 'count'
|
|
2277
|
+
}
|
|
2278
|
+
|
|
2279
|
+
interface QueryAnalyticsOptions {
|
|
2280
|
+
timezone?: string;
|
|
2281
|
+
maxTimeMS?: number;
|
|
2282
|
+
allowDiskUse?: boolean;
|
|
2283
|
+
weekStartsOn?: 0 | 1 | 2 | 3 | 4 | 5 | 6;
|
|
2284
|
+
cacheTTLMs?: number;
|
|
2285
|
+
}
|
|
2286
|
+
|
|
2287
|
+
interface TimeSeriesPoint {
|
|
2288
|
+
bucket: string; // ISO string, local-boundary-aligned
|
|
2289
|
+
[metric: string]: number | string;
|
|
2290
|
+
}
|
|
2291
|
+
|
|
2292
|
+
interface CompareResult {
|
|
2293
|
+
current: Record<string, number>;
|
|
2294
|
+
previous: Record<string, number>;
|
|
2295
|
+
change: Record<string, number>;
|
|
2296
|
+
changePct: Record<string, number | null>;
|
|
2297
|
+
}
|
|
2298
|
+
|
|
2299
|
+
interface BreakdownRow {
|
|
2300
|
+
key: unknown;
|
|
2301
|
+
[metric: string]: unknown;
|
|
2302
|
+
}
|
|
2303
|
+
|
|
2304
|
+
interface RankedBreakdownRow extends BreakdownRow {
|
|
2305
|
+
rank: number;
|
|
2306
|
+
}
|
|
2307
|
+
|
|
2308
|
+
interface FunnelStep {
|
|
2309
|
+
name: string;
|
|
2310
|
+
match: Record<string, unknown>;
|
|
2311
|
+
}
|
|
2312
|
+
|
|
2313
|
+
interface FunnelResult {
|
|
2314
|
+
name: string;
|
|
2315
|
+
count: number;
|
|
2316
|
+
conversionFromPrevious: number | null;
|
|
2317
|
+
conversionFromStart: number | null;
|
|
2318
|
+
}
|
|
2319
|
+
|
|
2320
|
+
interface CohortRetentionOptions {
|
|
2321
|
+
cohortDateField: string;
|
|
2322
|
+
activityDateField: string;
|
|
2323
|
+
idField: string;
|
|
2324
|
+
cohortInterval: Interval;
|
|
2325
|
+
periods: number;
|
|
2326
|
+
}
|
|
2327
|
+
|
|
2328
|
+
interface CohortRow {
|
|
2329
|
+
cohort: string;
|
|
2330
|
+
cohortSize: number;
|
|
2331
|
+
retention: (number | null)[];
|
|
2332
|
+
retentionPct: (number | null)[];
|
|
2333
|
+
}
|
|
2334
|
+
|
|
2335
|
+
interface SessionizeOptions {
|
|
2336
|
+
userField: string;
|
|
2337
|
+
dateField: string;
|
|
2338
|
+
gapMinutes: number;
|
|
2339
|
+
}
|
|
2340
|
+
|
|
2341
|
+
interface SessionSummary {
|
|
2342
|
+
userId: unknown;
|
|
2343
|
+
sessionStart: string;
|
|
2344
|
+
sessionEnd: string;
|
|
2345
|
+
eventCount: number;
|
|
2346
|
+
durationSeconds: number;
|
|
2347
|
+
}
|
|
2348
|
+
|
|
2349
|
+
interface OutlierRow {
|
|
2350
|
+
_id: unknown;
|
|
2351
|
+
value: number;
|
|
2352
|
+
zScore: number;
|
|
2353
|
+
[key: string]: unknown;
|
|
2354
|
+
}
|
|
2355
|
+
```
|
|
2356
|
+
|
|
2357
|
+
## Links
|
|
2358
|
+
|
|
2359
|
+
- [npm](https://www.npmjs.com/package/mongoose-query-find)
|
|
2360
|
+
- [GitHub](https://github.com/jsdev-robin/mongoose-query-find)
|
|
2361
|
+
- [Issues](https://github.com/jsdev-robin/mongoose-query-find/issues)
|
|
2362
|
+
|
|
2363
|
+
## License
|
|
2364
|
+
|
|
2365
|
+
ISC © [jsdev.robin@gmail.com](mailto:jsdev.robin@gmail.com)
|