mongo-query-normalizer 0.2.2 → 0.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +109 -443
- package/README.zh-CN.md +109 -432
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,525 +1,191 @@
|
|
|
1
|
-
#
|
|
1
|
+
# mongo-query-normalizer
|
|
2
2
|
|
|
3
|
-
**
|
|
4
|
-
|
|
5
|
-
An **observable, level-based** normalizer for MongoDB query objects. It stabilizes query **shape** at the conservative default, and adds **`predicate`** and **`scope`** levels with **documented, test-backed contracts** (see [SPEC.md](SPEC.md) and [docs/normalization-matrix.md](docs/normalization-matrix.md) / [中文](docs/normalization-matrix.zh-CN.md)). It returns **predictable** output plus **metadata**—not a MongoDB planner optimizer.
|
|
6
|
-
|
|
7
|
-
> **Default posture:** **`shape`** is the smallest, structural-only pass and the recommended default for the widest production use. **`predicate`** and **`scope`** apply additional conservative rewrites under explicit contracts; adopt them when you need those transforms and accept their modeled-operator scope (opaque operators stay preserved).
|
|
8
|
-
>
|
|
9
|
-
> **As of `v0.2.0`:** predicate rewrites are intentionally narrowed to an explicitly validated surface (`eq.eq`, `eq.ne`, `eq.in`, `eq.range`, `range.range`). High-risk combinations (for example null-vs-missing, array-sensitive semantics, `$exists`/`$nin`, object-vs-dotted-path mixes, opaque mixes) remain conservative by design.
|
|
10
|
-
|
|
11
|
-
> **Note:** `predicate.safetyPolicy.allowArraySensitiveRewrite` is **deprecated**. It no longer enables unsatisfiable deductions for `$eq`/`$in` non-membership (the normalizer must not emit `IMPOSSIBLE_SELECTOR` solely from `eq ∉ in` without schema).
|
|
3
|
+
> A safe MongoDB query normalizer — **correctness over cleverness**
|
|
12
4
|
|
|
13
5
|
---
|
|
14
6
|
|
|
15
|
-
##
|
|
16
|
-
|
|
17
|
-
- Query **shape** diverges across builders and hand-written filters.
|
|
18
|
-
- Outputs can be **hard to compare**, log, or diff without a stable pass.
|
|
19
|
-
- You need a **low-risk normalization layer** that defaults to conservative behavior.
|
|
20
|
-
|
|
21
|
-
This library does **not** promise to make queries faster or to pick optimal indexes.
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
## Features
|
|
26
|
-
|
|
27
|
-
- **Level-based** normalization (`shape` → `predicate` → `scope`)
|
|
28
|
-
- **Conservative default**: `shape` only out of the box (lowest-risk structural pass)
|
|
29
|
-
- **Observable** `meta`: changed flags, applied/skipped rules, warnings, hashes, optional stats
|
|
30
|
-
- **Stable / idempotent** output when rules apply (same options)
|
|
31
|
-
- **Opaque fallback** for unsupported operators (passthrough, not semantically rewritten)
|
|
32
|
-
|
|
33
|
-
---
|
|
7
|
+
## ✨ What it does
|
|
34
8
|
|
|
35
|
-
|
|
9
|
+
**Turn messy Mongo queries into clean, stable, and predictable ones — safely.**
|
|
36
10
|
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
```ts
|
|
46
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
47
|
-
|
|
48
|
-
const result = normalizeQuery({
|
|
49
|
-
$and: [{ status: "open" }, { $and: [{ priority: { $gte: 1 } }] }],
|
|
50
|
-
});
|
|
11
|
+
```js
|
|
12
|
+
// before
|
|
13
|
+
{
|
|
14
|
+
$and: [
|
|
15
|
+
{ status: "open" },
|
|
16
|
+
{ status: { $in: ["open", "closed"] } }
|
|
17
|
+
]
|
|
18
|
+
}
|
|
51
19
|
|
|
52
|
-
|
|
53
|
-
|
|
20
|
+
// after
|
|
21
|
+
{ status: "open" }
|
|
54
22
|
```
|
|
55
23
|
|
|
56
24
|
---
|
|
57
25
|
|
|
58
|
-
##
|
|
59
|
-
|
|
60
|
-
### 1) Minimal usage (recommended default)
|
|
61
|
-
|
|
62
|
-
```ts
|
|
63
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
64
|
-
|
|
65
|
-
const { query: normalizedQuery, meta } = normalizeQuery(inputQuery);
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
- Without `options`, default behavior is `level: "shape"`.
|
|
69
|
-
- Best for low-risk structural stabilization: logging, cache-key normalization, query diff alignment.
|
|
70
|
-
|
|
71
|
-
### 2) Pick a level explicitly
|
|
72
|
-
|
|
73
|
-
```ts
|
|
74
|
-
normalizeQuery(inputQuery, { level: "shape" }); // structural only (default)
|
|
75
|
-
normalizeQuery(inputQuery, { level: "predicate" }); // modeled predicate cleanup
|
|
76
|
-
normalizeQuery(inputQuery, { level: "scope" }); // scope propagation / conservative pruning
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
- `shape`: safest structural normalization.
|
|
80
|
-
- `predicate`: dedupe / merge comparable predicates, and contradiction collapse only for **provably safe** modeled cases (no schema assumption; multikey/array fields stay conservative).
|
|
81
|
-
- `scope`: adds inherited-constraint propagation and conservative branch decisions on top of `predicate`.
|
|
82
|
-
|
|
83
|
-
### 3) Full `options` example
|
|
84
|
-
|
|
85
|
-
```ts
|
|
86
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
87
|
-
|
|
88
|
-
const result = normalizeQuery(inputQuery, {
|
|
89
|
-
level: "scope",
|
|
90
|
-
rules: {
|
|
91
|
-
// shape-related
|
|
92
|
-
flattenLogical: true,
|
|
93
|
-
removeEmptyLogical: true,
|
|
94
|
-
collapseSingleChildLogical: true,
|
|
95
|
-
dedupeLogicalChildren: true,
|
|
96
|
-
// predicate-related
|
|
97
|
-
dedupeSameFieldPredicates: true,
|
|
98
|
-
mergeComparablePredicates: true,
|
|
99
|
-
collapseContradictions: true,
|
|
100
|
-
// ordering-related
|
|
101
|
-
sortLogicalChildren: true,
|
|
102
|
-
sortFieldPredicates: true,
|
|
103
|
-
// scope observe-only rule (no structural hoist)
|
|
104
|
-
detectCommonPredicatesInOr: true,
|
|
105
|
-
},
|
|
106
|
-
safety: {
|
|
107
|
-
maxNormalizeDepth: 32,
|
|
108
|
-
maxNodeGrowthRatio: 1.5,
|
|
109
|
-
},
|
|
110
|
-
observe: {
|
|
111
|
-
collectWarnings: true,
|
|
112
|
-
collectMetrics: false,
|
|
113
|
-
collectPredicateTraces: false,
|
|
114
|
-
collectScopeTraces: false,
|
|
115
|
-
},
|
|
116
|
-
predicate: {
|
|
117
|
-
safetyPolicy: {
|
|
118
|
-
// override only fields you care about
|
|
119
|
-
},
|
|
120
|
-
},
|
|
121
|
-
scope: {
|
|
122
|
-
safetyPolicy: {
|
|
123
|
-
// override only fields you care about
|
|
124
|
-
},
|
|
125
|
-
},
|
|
126
|
-
});
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### 4) Inspect resolved runtime options
|
|
26
|
+
## ⚠️ Why this matters
|
|
130
27
|
|
|
131
|
-
|
|
132
|
-
import { resolveNormalizeOptions } from "mongo-query-normalizer";
|
|
28
|
+
If you build dynamic queries, you will eventually get:
|
|
133
29
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
30
|
+
* duplicated conditions
|
|
31
|
+
* inconsistent query shapes
|
|
32
|
+
* hard-to-debug filters
|
|
33
|
+
* subtle semantic bugs
|
|
138
34
|
|
|
139
|
-
|
|
140
|
-
```
|
|
35
|
+
Most tools try to “optimize” queries.
|
|
141
36
|
|
|
142
|
-
|
|
143
|
-
- Useful for logging a startup-time normalization config snapshot.
|
|
37
|
+
👉 This library does something different:
|
|
144
38
|
|
|
145
|
-
|
|
39
|
+
> **It only applies transformations that are provably safe.**
|
|
146
40
|
|
|
147
|
-
|
|
148
|
-
const { query: normalizedQuery, meta } = normalizeQuery(inputQuery, options);
|
|
41
|
+
---
|
|
149
42
|
|
|
150
|
-
|
|
151
|
-
logger.warn({ reason: meta.bailoutReason }, "normalization bailed out");
|
|
152
|
-
}
|
|
43
|
+
## 🛡️ Safe by design
|
|
153
44
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
},
|
|
162
|
-
"query normalized"
|
|
163
|
-
);
|
|
45
|
+
```js
|
|
46
|
+
// NOT simplified (correctly)
|
|
47
|
+
{
|
|
48
|
+
$and: [
|
|
49
|
+
{ uids: "1" },
|
|
50
|
+
{ uids: "2" }
|
|
51
|
+
]
|
|
164
52
|
}
|
|
165
53
|
```
|
|
166
54
|
|
|
167
|
-
|
|
168
|
-
- `meta`: observability data (changed flag, rule traces, warnings, hashes, optional stats/traces).
|
|
169
|
-
|
|
170
|
-
### 6) Typical integration patterns
|
|
55
|
+
Why?
|
|
171
56
|
|
|
172
|
-
|
|
173
|
-
// A. Normalize centrally in data-access layer
|
|
174
|
-
export function normalizeForFind(rawFilter) {
|
|
175
|
-
return normalizeQuery(rawFilter, { level: "shape" }).query;
|
|
176
|
-
}
|
|
57
|
+
Because MongoDB arrays can match both:
|
|
177
58
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
return normalizeQuery(rawFilter, { level: "predicate" }).query;
|
|
181
|
-
}
|
|
59
|
+
```js
|
|
60
|
+
{ uids: ["1", "2"] }
|
|
182
61
|
```
|
|
183
62
|
|
|
184
|
-
- Prefer `shape` for online request paths.
|
|
185
|
-
- Enable `predicate` / `scope` when there is clear benefit plus test coverage.
|
|
186
|
-
|
|
187
|
-
### 7) Errors and boundaries
|
|
188
|
-
|
|
189
|
-
- Invalid `level` throws an error (for example, typos).
|
|
190
|
-
- Unsupported or unknown operators are generally preserved as opaque; semantic merge behavior is not guaranteed for them.
|
|
191
|
-
- The library target is stability and observability, not query planning optimization.
|
|
192
|
-
|
|
193
|
-
---
|
|
194
|
-
|
|
195
|
-
## Default behavior
|
|
196
|
-
|
|
197
|
-
- **Default `level` is `"shape"`** (see `resolveNormalizeOptions()`).
|
|
198
|
-
- By default there is **no** predicate merge at `shape`. At **`scope`**, core work is inherited-constraint propagation and conservative branch decisions; **`detectCommonPredicatesInOr`** is an **optional, observe-only** rule (warnings / traces)—never a structural hoist.
|
|
199
|
-
- The goal is **stability and observability**, not “smart optimization.”
|
|
200
|
-
|
|
201
|
-
---
|
|
202
|
-
|
|
203
|
-
## Choosing a level
|
|
204
|
-
|
|
205
|
-
- Use **`shape`** when you only need structural stabilization (flatten, dedupe children, ordering, etc.).
|
|
206
|
-
- Use **`predicate`** when you need same-field dedupe, modeled comparable merges, and contradiction collapse on **modeled** operators; opaque subtrees stay preserved.
|
|
207
|
-
- Use **`scope`** when you need inherited-constraint propagation, conservative pruning, and narrow coverage elimination as described in the spec and matrix. **`detectCommonPredicatesInOr`** (when enabled) is **observe-only** and does not rewrite structure.
|
|
208
|
-
|
|
209
|
-
Authoritative behavior boundaries are in **[SPEC.md](SPEC.md)**, **[docs/normalization-matrix.md](docs/normalization-matrix.md)**, and contract tests under **`test/contracts/`**—not informal README prose alone.
|
|
210
|
-
|
|
211
|
-
---
|
|
212
|
-
|
|
213
|
-
## Levels
|
|
214
|
-
|
|
215
|
-
### `shape` (default)
|
|
216
|
-
|
|
217
|
-
**Recommended default** for the lowest-risk path. Safe structural normalization only, for example:
|
|
218
|
-
|
|
219
|
-
- flatten compound (`$and` / `$or`) nodes
|
|
220
|
-
- remove empty compound nodes
|
|
221
|
-
- collapse single-child compound nodes
|
|
222
|
-
- dedupe compound children
|
|
223
|
-
- canonical ordering
|
|
224
|
-
|
|
225
|
-
### `predicate`
|
|
226
|
-
|
|
227
|
-
On top of `shape`, conservative **predicate** cleanup on **modeled** operators:
|
|
228
|
-
|
|
229
|
-
- dedupe same-field predicates
|
|
230
|
-
- merge comparable predicates where modeled
|
|
231
|
-
- collapse clear contradictions to an unsatisfiable filter
|
|
232
|
-
- merge **direct** `$and` children that share the same field name before further predicate work (so contradictions like `{ $and: [{ a: 1 }, { a: 2 }] }` can be detected)
|
|
233
|
-
|
|
234
|
-
### `scope`
|
|
235
|
-
|
|
236
|
-
On top of `predicate`:
|
|
237
|
-
|
|
238
|
-
- **Inherited constraint propagation** (phase-1 allowlist) and **conservative branch pruning**; **coverage elimination** only in narrow, tested cases when policy allows
|
|
239
|
-
- Optional **`detectCommonPredicatesInOr`**: observe-only (warnings / traces); **no** structural rewrite
|
|
240
|
-
|
|
241
|
-
---
|
|
242
|
-
|
|
243
|
-
## `meta` fields
|
|
244
|
-
|
|
245
|
-
| Field | Meaning |
|
|
246
|
-
|--------|---------|
|
|
247
|
-
| `changed` | Structural/predicate output differs from input (hash-based) |
|
|
248
|
-
| `level` | Resolved normalization level |
|
|
249
|
-
| `appliedRules` / `skippedRules` | Rule tracing |
|
|
250
|
-
| `warnings` | Non-fatal issues when `observe.collectWarnings` is enabled (rule notices, detection text, etc.) |
|
|
251
|
-
| `bailedOut` | Safety stop; output reverts to pre-pass parse for that call |
|
|
252
|
-
| `bailoutReason` | Why bailout happened, if any |
|
|
253
|
-
| `beforeHash` / `afterHash` | Stable hashes for diffing |
|
|
254
|
-
| `stats` | Optional before/after tree metrics (`observe.collectMetrics`) |
|
|
255
|
-
| `predicateTraces` | When `observe.collectPredicateTraces`: per-field planner / skip / contradiction signals |
|
|
256
|
-
| `scopeTrace` | When `observe.collectScopeTraces`: constraint extraction rejections + scope decision events |
|
|
257
|
-
|
|
258
|
-
---
|
|
259
|
-
|
|
260
|
-
## Unsupported / opaque behavior
|
|
261
|
-
|
|
262
|
-
Structures such as **`$nor`**, **`$regex`**, **`$not`**, **`$elemMatch`**, **`$expr`**, geo/text queries, and **unknown** operators are generally treated as **opaque**: they pass through or are preserved without full semantic rewriting. They are **not** guaranteed to participate in merge or contradiction logic.
|
|
263
|
-
|
|
264
63
|
---
|
|
265
64
|
|
|
266
|
-
##
|
|
267
|
-
|
|
268
|
-
The **public contract** is:
|
|
65
|
+
## ❌ What this is NOT
|
|
269
66
|
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
67
|
+
* Not a query optimizer
|
|
68
|
+
* Not an index advisor
|
|
69
|
+
* Not a performance tool
|
|
273
70
|
|
|
274
|
-
|
|
71
|
+
It will **never guess**:
|
|
275
72
|
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
73
|
+
* field cardinality
|
|
74
|
+
* schema constraints
|
|
75
|
+
* data distribution
|
|
279
76
|
|
|
280
|
-
|
|
281
|
-
2. **`predicate`** / **`scope`** may change structure while aiming for **semantic equivalence** on **modeled** operators.
|
|
282
|
-
3. **Opaque** nodes are not rewritten semantically.
|
|
283
|
-
4. Output should be **idempotent** under the same options when no bailout occurs.
|
|
284
|
-
5. This library is **not** the MongoDB query planner or an optimizer.
|
|
77
|
+
If unsure → **skip**
|
|
285
78
|
|
|
286
79
|
---
|
|
287
80
|
|
|
288
|
-
##
|
|
289
|
-
|
|
290
|
-
**Online main path** — use default (`shape`); this remains the most production-safe baseline in `v0.2.0`:
|
|
81
|
+
## 🚀 Quick start
|
|
291
82
|
|
|
292
83
|
```ts
|
|
293
|
-
normalizeQuery
|
|
294
|
-
```
|
|
295
|
-
|
|
296
|
-
**Predicate or scope** — pass `level` explicitly; review [SPEC.md](SPEC.md) and contract tests for supported vs preserved patterns:
|
|
84
|
+
import { normalizeQuery } from "mongo-query-normalizer";
|
|
297
85
|
|
|
298
|
-
|
|
299
|
-
normalizeQuery(query, { level: "predicate" });
|
|
86
|
+
const { query } = normalizeQuery(inputQuery);
|
|
300
87
|
```
|
|
301
88
|
|
|
302
89
|
---
|
|
303
90
|
|
|
304
|
-
##
|
|
91
|
+
## 🧠 Where it fits
|
|
305
92
|
|
|
306
|
-
```
|
|
307
|
-
|
|
308
|
-
|
|
93
|
+
```text
|
|
94
|
+
Query Builder / ORM
|
|
95
|
+
↓
|
|
96
|
+
normalizeQuery ← (this library)
|
|
97
|
+
↓
|
|
98
|
+
MongoDB
|
|
309
99
|
```
|
|
310
100
|
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
---
|
|
314
|
-
|
|
315
|
-
## Testing
|
|
316
|
-
|
|
317
|
-
### Test layout
|
|
318
|
-
|
|
319
|
-
This repository organizes tests by **API surface**, **normalization level**, and **cross-level contracts**, while preserving deeper semantic and regression suites.
|
|
320
|
-
|
|
321
|
-
### Directory responsibilities
|
|
322
|
-
|
|
323
|
-
#### `test/api/`
|
|
324
|
-
|
|
325
|
-
Tests the public API and configuration surface.
|
|
326
|
-
|
|
327
|
-
Put tests here when they verify:
|
|
328
|
-
|
|
329
|
-
* `normalizeQuery` return shape and top-level behavior
|
|
330
|
-
* `resolveNormalizeOptions`
|
|
331
|
-
* package exports
|
|
332
|
-
|
|
333
|
-
Do **not** put level-specific normalization behavior here.
|
|
334
|
-
|
|
335
|
-
---
|
|
336
|
-
|
|
337
|
-
#### `test/levels/`
|
|
338
|
-
|
|
339
|
-
Tests the behavior boundary of each `NormalizeLevel`.
|
|
340
|
-
|
|
341
|
-
Current levels:
|
|
342
|
-
|
|
343
|
-
* `shape`
|
|
344
|
-
* `predicate`
|
|
345
|
-
* `scope`
|
|
346
|
-
|
|
347
|
-
Each level test file should focus on four things:
|
|
348
|
-
|
|
349
|
-
1. positive capabilities of that level
|
|
350
|
-
2. behavior explicitly not enabled at that level
|
|
351
|
-
3. contrast with the adjacent level(s)
|
|
352
|
-
4. a small number of representative contracts for that level
|
|
353
|
-
|
|
354
|
-
Prefer asserting:
|
|
355
|
-
|
|
356
|
-
* normalized query structure
|
|
357
|
-
* observable cross-level differences
|
|
358
|
-
* stable public metadata
|
|
359
|
-
|
|
360
|
-
Avoid overfitting to:
|
|
361
|
-
|
|
362
|
-
* exact warning text
|
|
363
|
-
* exact internal rule IDs
|
|
364
|
-
* fixed child ordering unless ordering itself is part of the contract
|
|
365
|
-
|
|
366
|
-
---
|
|
367
|
-
|
|
368
|
-
#### `test/contracts/`
|
|
369
|
-
|
|
370
|
-
Tests contracts that should hold across levels, or default behavior that is separate from any single level.
|
|
371
|
-
|
|
372
|
-
Put tests here when they verify:
|
|
373
|
-
|
|
374
|
-
* default level behavior
|
|
375
|
-
* idempotency across all levels
|
|
376
|
-
* output invariants across all levels
|
|
377
|
-
* opaque subtree preservation across all levels
|
|
378
|
-
* formal **`predicate` / `scope`** contracts (supported merges, opaque preservation, scope policy guards, rule toggles)—see `test/contracts/predicate-scope-stable-contract.test.js`
|
|
379
|
-
|
|
380
|
-
Use `test/helpers/level-contract-runner.js` for all-level suites.
|
|
101
|
+
You don’t replace your builder.
|
|
102
|
+
You **sanitize its output**.
|
|
381
103
|
|
|
382
104
|
---
|
|
383
105
|
|
|
384
|
-
|
|
106
|
+
## 🧩 When to use
|
|
385
107
|
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
108
|
+
* dynamic filters / search APIs
|
|
109
|
+
* BI / reporting systems
|
|
110
|
+
* user-generated queries
|
|
111
|
+
* multi-team codebases with inconsistent query styles
|
|
112
|
+
* logging / caching / diffing queries
|
|
390
113
|
|
|
391
114
|
---
|
|
392
115
|
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
Tests property-based and metamorphic behavior.
|
|
396
|
-
|
|
397
|
-
Use this directory for:
|
|
116
|
+
## ⚙️ Levels
|
|
398
117
|
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
118
|
+
| Level | What it does | Safety |
|
|
119
|
+
| ----------- | ------------------------------ | --------- |
|
|
120
|
+
| `shape` | structural normalization | 🟢 safest |
|
|
121
|
+
| `predicate` | safe predicate simplification | 🟡 |
|
|
122
|
+
| `scope` | limited constraint propagation | 🟡 |
|
|
402
123
|
|
|
403
|
-
|
|
124
|
+
Default is `shape`.
|
|
404
125
|
|
|
405
126
|
---
|
|
406
127
|
|
|
407
|
-
|
|
128
|
+
## 📦 Output
|
|
408
129
|
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
#### `test/performance/`
|
|
416
|
-
|
|
417
|
-
Tests performance guards or complexity-sensitive behavior.
|
|
418
|
-
|
|
419
|
-
These tests should stay focused on performance-related expectations, not general normalization structure.
|
|
130
|
+
```ts
|
|
131
|
+
{
|
|
132
|
+
query, // normalized query
|
|
133
|
+
meta // debug / trace info
|
|
134
|
+
}
|
|
135
|
+
```
|
|
420
136
|
|
|
421
137
|
---
|
|
422
138
|
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
#### `test/helpers/level-runner.js`
|
|
426
|
-
|
|
427
|
-
Shared helper for running a query at a specific level.
|
|
428
|
-
|
|
429
|
-
#### `test/helpers/level-cases.js`
|
|
430
|
-
|
|
431
|
-
Shared fixed inputs used across level tests.
|
|
432
|
-
Prefer adding reusable representative cases here instead of duplicating inline fixtures.
|
|
139
|
+
## 🎯 Design philosophy
|
|
433
140
|
|
|
434
|
-
|
|
141
|
+
> If a rewrite might be wrong, don’t do it.
|
|
435
142
|
|
|
436
|
-
|
|
143
|
+
* no schema assumptions
|
|
144
|
+
* no array guessing
|
|
145
|
+
* no unsafe merges
|
|
146
|
+
* deterministic output
|
|
147
|
+
* idempotent results
|
|
437
148
|
|
|
438
149
|
---
|
|
439
150
|
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
#### When adding a new normalization rule
|
|
443
|
-
|
|
444
|
-
Ask first:
|
|
445
|
-
|
|
446
|
-
* Is this a public API behavior?
|
|
447
|
-
|
|
448
|
-
* Add to `test/api/`
|
|
449
|
-
* Is this enabled only at a specific level?
|
|
450
|
-
|
|
451
|
-
* Add to `test/levels/`
|
|
452
|
-
* Should this hold for all levels?
|
|
151
|
+
## 🔍 Example
|
|
453
152
|
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
---
|
|
463
|
-
|
|
464
|
-
#### When adding a new level
|
|
465
|
-
|
|
466
|
-
At minimum, update all of the following:
|
|
153
|
+
```ts
|
|
154
|
+
const result = normalizeQuery({
|
|
155
|
+
$and: [
|
|
156
|
+
{ status: "open" },
|
|
157
|
+
{ status: { $in: ["open", "closed"] } }
|
|
158
|
+
]
|
|
159
|
+
});
|
|
467
160
|
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
4. add at least one contrast case against the adjacent level
|
|
161
|
+
console.log(result.query);
|
|
162
|
+
// { status: "open" }
|
|
163
|
+
```
|
|
472
164
|
|
|
473
165
|
---
|
|
474
166
|
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
Prefer:
|
|
478
|
-
|
|
479
|
-
* example-based tests for level boundaries
|
|
480
|
-
* query-shape assertions
|
|
481
|
-
* contrast tests between adjacent levels
|
|
482
|
-
* shared fixtures for representative cases
|
|
167
|
+
## 📚 Docs
|
|
483
168
|
|
|
484
|
-
|
|
169
|
+
* [`SPEC.md`](SPEC.md) — behavior spec
|
|
170
|
+
* [`docs/normalization-matrix.md`](docs/normalization-matrix.md) — rule coverage by operator and level
|
|
171
|
+
* [`docs/CANONICAL_FORM.md`](docs/CANONICAL_FORM.md) — canonical output shape and idempotency
|
|
172
|
+
* [`CHANGELOG.md`](CHANGELOG.md) — release notes
|
|
173
|
+
* [`test/REGRESSION.md`](test/REGRESSION.md) — reproducing property / semantic test failures
|
|
485
174
|
|
|
486
|
-
|
|
487
|
-
* repeating the same fixture with only superficial assertion changes
|
|
488
|
-
* putting default-level behavior inside a specific level test
|
|
489
|
-
* mixing exports/API tests with normalization behavior tests
|
|
175
|
+
**中文:** [`README.zh-CN.md`](README.zh-CN.md) · [`SPEC.zh-CN.md`](SPEC.zh-CN.md) · [`docs/normalization-matrix.zh-CN.md`](docs/normalization-matrix.zh-CN.md) · [`CHANGELOG.zh-CN.md`](CHANGELOG.zh-CN.md)
|
|
490
176
|
|
|
491
177
|
---
|
|
492
178
|
|
|
493
|
-
|
|
179
|
+
## 🧪 Testing
|
|
494
180
|
|
|
495
|
-
*
|
|
496
|
-
*
|
|
497
|
-
*
|
|
498
|
-
* `semantic/property/regression/performance` answer: **whether the system remains correct, robust, and efficient**
|
|
181
|
+
* semantic equivalence tests (real MongoDB)
|
|
182
|
+
* property-based testing
|
|
183
|
+
* regression suites
|
|
499
184
|
|
|
500
185
|
---
|
|
501
186
|
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
Randomized semantic tests use **`mongodb-memory-server`** + **`fast-check`** to compare **real** `find` results (same `sort` / `skip` / `limit`, projection `{ _id: 1 }`) before and after `normalizeQuery` on a **fixed document schema** and a **restricted operator set** (see `test/helpers/arbitraries.js`). They assert matching **`_id` order**, **idempotency** of the returned `query`, and (for opaque operators) **non-crash / stable second pass** only. **`FC_SEED` / `FC_RUNS` defaults are centralized in `test/helpers/fc-config.js`** (also re-exported from `arbitraries.js`).
|
|
505
|
-
|
|
506
|
-
To **avoid downloading** a MongoDB binary, set one of **`MONGODB_BINARY`**, **`MONGOD_BINARY`**, or **`MONGOMS_SYSTEM_BINARY`** to your local `mongod` path before running semantic tests (see `test/helpers/mongo-fixture.js`).
|
|
507
|
-
|
|
508
|
-
* **`npm run test`** — build, then `test:unit`, then `test:semantic`.
|
|
509
|
-
* **`npm run test:api`** — `test/api/**/*.test.js` only.
|
|
510
|
-
* **`npm run test:levels`** — `test/levels/**/*.test.js` and `test/contracts/*.test.js`.
|
|
511
|
-
* **`npm run test:unit`** — all `test/**/*.test.js` except `test/semantic/**`, `test/regression/**`, and `test/property/**` (includes `test/api/**`, `test/levels/**`, `test/contracts/**`, `test/performance/**`, and other unit tests).
|
|
512
|
-
* **`npm run test:semantic`** — semantic + regression + property folders (defaults when env unset: see `fc-config.js`).
|
|
513
|
-
* **`npm run test:semantic:quick`** — lower **`FC_RUNS`** (script sets `45`) + **`FC_SEED=42`**, still runs `test/regression/**` and `test/property/**`.
|
|
514
|
-
* **`npm run test:semantic:ci`** — CI-oriented env (`FC_RUNS=200`, `FC_SEED=42` in script).
|
|
515
|
-
|
|
516
|
-
Override property-test parameters: **`FC_SEED`**, **`FC_RUNS`**, optional **`FC_QUICK=1`** (see `fc-config.js`). How to reproduce failures and when to add a fixed regression case: **`test/REGRESSION.md`**.
|
|
517
|
-
|
|
518
|
-
Full-text, geo, heavy **`$expr`**, **`$where`**, aggregation, collation, etc. stay **out** of the main semantic equivalence generator; opaque contracts live in **`test/contracts/opaque-operators.all-levels.test.js`**.
|
|
519
|
-
|
|
520
|
-
---
|
|
187
|
+
## ⭐ Philosophy
|
|
521
188
|
|
|
522
|
-
|
|
189
|
+
Most query tools try to be smart.
|
|
523
190
|
|
|
524
|
-
|
|
525
|
-
- [docs/CANONICAL_FORM.md](docs/CANONICAL_FORM.md) — idempotency and canonical shape notes.
|
|
191
|
+
This one tries to be **correct**.
|
package/README.zh-CN.md
CHANGED
|
@@ -1,515 +1,192 @@
|
|
|
1
|
-
#
|
|
1
|
+
# mongo-query-normalizer
|
|
2
2
|
|
|
3
3
|
[English](README.md) | **中文**
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
> **默认策略:** **`shape`** 仅做结构规范化,适合作为**覆盖面最广**的默认路径。 **`predicate`**、**`scope`** 在 **SPEC**、**normalization-matrix** 与 **契约测试** 中有明确边界;仅在需要对应能力且接受「已建模算子」范围时启用;**opaque** 算子保持透传。
|
|
8
|
-
>
|
|
9
|
-
> **`v0.2.0` 起:** `predicate` 改写面有意收敛到显式验证能力(`eq.eq`、`eq.ne`、`eq.in`、`eq.range`、`range.range`)。高风险组合(如 `null`/缺失语义、数组敏感语义、`$exists`/`$nin`、整对象与点路径混用、opaque 混用)按设计保持保守处理。
|
|
10
|
-
|
|
11
|
-
> **说明:** `predicate.safetyPolicy.allowArraySensitiveRewrite` 已**废弃**。它不再用于启用 `$eq`/`$in` 的“未命中即判死”(无 schema 时不得仅基于 `eq ∉ in` 就输出 `IMPOSSIBLE_SELECTOR`)。
|
|
12
|
-
|
|
13
|
-
---
|
|
14
|
-
|
|
15
|
-
## 为什么需要它
|
|
16
|
-
|
|
17
|
-
- 查询 **结构** 在不同写法下容易发散。
|
|
18
|
-
- 没有稳定层时,**对比、日志、回放** 成本高。
|
|
19
|
-
- 需要一层 **低风险** 的 query normalization,默认行为要保守。
|
|
20
|
-
|
|
21
|
-
本库**不以**「自动让查询更快」或「替代 planner」作为卖点。
|
|
22
|
-
|
|
23
|
-
---
|
|
24
|
-
|
|
25
|
-
## 核心特性
|
|
26
|
-
|
|
27
|
-
- **按 level 分层**:`shape` → `predicate` → `scope`
|
|
28
|
-
- **默认保守**:开箱仅 `shape`(风险最小的结构层)
|
|
29
|
-
- **可观测的 `meta`**:变更、规则、告警、哈希、可选统计
|
|
30
|
-
- **稳定 / 幂等**(相同 options、未熔断时)
|
|
31
|
-
- **不透明(opaque)回退**:不支持的算子以透传为主,不做完整语义改写
|
|
32
|
-
|
|
33
|
-
---
|
|
34
|
-
|
|
35
|
-
## 安装
|
|
36
|
-
|
|
37
|
-
```bash
|
|
38
|
-
npm install mongo-query-normalizer
|
|
39
|
-
```
|
|
5
|
+
> 安全的 MongoDB 查询规范化器 —— **正确优先于「聪明」**
|
|
40
6
|
|
|
41
7
|
---
|
|
42
8
|
|
|
43
|
-
##
|
|
9
|
+
## ✨ 它能做什么
|
|
44
10
|
|
|
45
|
-
|
|
46
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
11
|
+
**把杂乱的 Mongo 查询,安全地变成干净、稳定、可预期的形态。**
|
|
47
12
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
13
|
+
```js
|
|
14
|
+
// 之前
|
|
15
|
+
{
|
|
16
|
+
$and: [
|
|
17
|
+
{ status: "open" },
|
|
18
|
+
{ status: { $in: ["open", "closed"] } }
|
|
19
|
+
]
|
|
20
|
+
}
|
|
51
21
|
|
|
52
|
-
|
|
53
|
-
|
|
22
|
+
// 之后
|
|
23
|
+
{ status: "open" }
|
|
54
24
|
```
|
|
55
25
|
|
|
56
26
|
---
|
|
57
27
|
|
|
58
|
-
##
|
|
59
|
-
|
|
60
|
-
### 1) 最小可用(推荐默认)
|
|
61
|
-
|
|
62
|
-
```ts
|
|
63
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
64
|
-
|
|
65
|
-
const { query: normalizedQuery, meta } = normalizeQuery(inputQuery);
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
- 不传 `options` 时,默认 `level: "shape"`。
|
|
69
|
-
- 适合日志归一化、缓存 key 稳定化、查询 diff 对齐等“低风险结构规范化”场景。
|
|
70
|
-
|
|
71
|
-
### 2) 显式选择 level
|
|
72
|
-
|
|
73
|
-
```ts
|
|
74
|
-
normalizeQuery(inputQuery, { level: "shape" }); // 仅结构层(默认)
|
|
75
|
-
normalizeQuery(inputQuery, { level: "predicate" }); // 启用已建模谓词整理
|
|
76
|
-
normalizeQuery(inputQuery, { level: "scope" }); // 启用 scope 传播/保守剪枝能力
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
- `shape`:结构稳定优先,风险最低。
|
|
80
|
-
- `predicate`:在已建模算子范围内做去重、可比合并;矛盾折叠仅针对**可证明安全**的情形(默认不做 schema 假设,数组/多键字段保持保守)。
|
|
81
|
-
- `scope`:在 `predicate` 之上增加继承约束传播与保守分支决策。
|
|
82
|
-
|
|
83
|
-
### 3) `options` 全量示例
|
|
84
|
-
|
|
85
|
-
```ts
|
|
86
|
-
import { normalizeQuery } from "mongo-query-normalizer";
|
|
87
|
-
|
|
88
|
-
const result = normalizeQuery(inputQuery, {
|
|
89
|
-
level: "scope",
|
|
90
|
-
rules: {
|
|
91
|
-
// shape 相关
|
|
92
|
-
flattenLogical: true,
|
|
93
|
-
removeEmptyLogical: true,
|
|
94
|
-
collapseSingleChildLogical: true,
|
|
95
|
-
dedupeLogicalChildren: true,
|
|
96
|
-
// predicate 相关
|
|
97
|
-
dedupeSameFieldPredicates: true,
|
|
98
|
-
mergeComparablePredicates: true,
|
|
99
|
-
collapseContradictions: true,
|
|
100
|
-
// 排序相关
|
|
101
|
-
sortLogicalChildren: true,
|
|
102
|
-
sortFieldPredicates: true,
|
|
103
|
-
// scope 观测规则(仅观测,不上提改写)
|
|
104
|
-
detectCommonPredicatesInOr: true,
|
|
105
|
-
},
|
|
106
|
-
safety: {
|
|
107
|
-
maxNormalizeDepth: 32,
|
|
108
|
-
maxNodeGrowthRatio: 1.5,
|
|
109
|
-
},
|
|
110
|
-
observe: {
|
|
111
|
-
collectWarnings: true,
|
|
112
|
-
collectMetrics: false,
|
|
113
|
-
collectPredicateTraces: false,
|
|
114
|
-
collectScopeTraces: false,
|
|
115
|
-
},
|
|
116
|
-
predicate: {
|
|
117
|
-
safetyPolicy: {
|
|
118
|
-
// 仅覆盖你关心的字段;其余使用默认值
|
|
119
|
-
},
|
|
120
|
-
},
|
|
121
|
-
scope: {
|
|
122
|
-
safetyPolicy: {
|
|
123
|
-
// 仅覆盖你关心的字段;其余使用默认值
|
|
124
|
-
},
|
|
125
|
-
},
|
|
126
|
-
});
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### 4) 用 `resolveNormalizeOptions` 查看最终生效配置
|
|
28
|
+
## ⚠️ 为什么重要
|
|
130
29
|
|
|
131
|
-
|
|
132
|
-
import { resolveNormalizeOptions } from "mongo-query-normalizer";
|
|
30
|
+
如果你在做动态查询,迟早会遇到:
|
|
133
31
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
32
|
+
* 重复条件
|
|
33
|
+
* 查询结构不一致
|
|
34
|
+
* 难以调试的过滤器
|
|
35
|
+
* 隐蔽的语义问题
|
|
138
36
|
|
|
139
|
-
|
|
140
|
-
```
|
|
37
|
+
多数工具会试图「优化」查询。
|
|
141
38
|
|
|
142
|
-
|
|
143
|
-
- 适合在服务启动时打印一次“规范化配置快照”。
|
|
39
|
+
👉 本库做法不同:
|
|
144
40
|
|
|
145
|
-
|
|
41
|
+
> **只应用可证明安全的变换。**
|
|
146
42
|
|
|
147
|
-
|
|
148
|
-
const { query: normalizedQuery, meta } = normalizeQuery(inputQuery, options);
|
|
43
|
+
---
|
|
149
44
|
|
|
150
|
-
|
|
151
|
-
logger.warn({ reason: meta.bailoutReason }, "normalization bailed out");
|
|
152
|
-
}
|
|
45
|
+
## 🛡️ 设计上就安全
|
|
153
46
|
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
},
|
|
162
|
-
"query normalized"
|
|
163
|
-
);
|
|
47
|
+
```js
|
|
48
|
+
// 不会简化(这是对的)
|
|
49
|
+
{
|
|
50
|
+
$and: [
|
|
51
|
+
{ uids: "1" },
|
|
52
|
+
{ uids: "2" }
|
|
53
|
+
]
|
|
164
54
|
}
|
|
165
55
|
```
|
|
166
56
|
|
|
167
|
-
|
|
168
|
-
- `meta`:观测信息(是否变化、规则轨迹、告警、哈希、可选统计与 trace)。
|
|
57
|
+
原因?
|
|
169
58
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
```ts
|
|
173
|
-
// A. 在数据访问层统一规范化
|
|
174
|
-
export function normalizeForFind(rawFilter) {
|
|
175
|
-
return normalizeQuery(rawFilter, { level: "shape" }).query;
|
|
176
|
-
}
|
|
59
|
+
因为 MongoDB 数组可以同时满足两者:
|
|
177
60
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
return normalizeQuery(rawFilter, { level: "predicate" }).query;
|
|
181
|
-
}
|
|
61
|
+
```js
|
|
62
|
+
{ uids: ["1", "2"] }
|
|
182
63
|
```
|
|
183
64
|
|
|
184
|
-
- 在线主路径优先 `shape`。
|
|
185
|
-
- `predicate` / `scope` 建议在有明确收益与测试兜底时再启用。
|
|
186
|
-
|
|
187
|
-
### 7) 错误与边界
|
|
188
|
-
|
|
189
|
-
- `level` 非法会抛错(例如拼写错误)。
|
|
190
|
-
- 不支持或未知算子通常按 opaque 保留,不保证参与语义合并。
|
|
191
|
-
- 本库目标是“稳定与可观测”,不是查询优化器。
|
|
192
|
-
|
|
193
|
-
---
|
|
194
|
-
|
|
195
|
-
## 默认行为说明
|
|
196
|
-
|
|
197
|
-
- **默认 `level` 为 `shape`**(见 `resolveNormalizeOptions()`)。
|
|
198
|
-
- `shape` 默认**不做**谓词级合并。**`scope`** 主路径是继承约束传播与保守分支决策;**`detectCommonPredicatesInOr`** 为**可选、仅观测**规则(告警/轨迹),**从不**做结构上提。
|
|
199
|
-
- 默认目标是 **稳定与可观测**,不是「智能优化」。
|
|
200
|
-
|
|
201
|
-
---
|
|
202
|
-
|
|
203
|
-
## 如何选择 level
|
|
204
|
-
|
|
205
|
-
- 仅需结构稳定时,用 **`shape`**。
|
|
206
|
-
- 需要同字段去重、可建模比较合并、矛盾折叠时,用 **`predicate`**(仅针对已建模算子)。
|
|
207
|
-
- 需要继承约束传播、保守剪枝与狭窄覆盖消除时,用 **`scope`**(详见 [SPEC.zh-CN.md](SPEC.zh-CN.md) 与 [docs/normalization-matrix.zh-CN.md](docs/normalization-matrix.zh-CN.md))。**`detectCommonPredicatesInOr`**(开启时)仅观测,不改写结构。
|
|
208
|
-
|
|
209
|
-
**行为边界**以 **SPEC**、**normalization-matrix** 与 **`test/contracts/`** 为准,而非仅靠 README 叙述。
|
|
210
|
-
|
|
211
|
-
---
|
|
212
|
-
|
|
213
|
-
## Level 说明
|
|
214
|
-
|
|
215
|
-
### `shape`(默认)
|
|
216
|
-
|
|
217
|
-
**推荐默认路径**(风险最小):只做安全结构规范化,例如:
|
|
218
|
-
|
|
219
|
-
- 展平复合(`$and` / `$or`)节点
|
|
220
|
-
- 移除空复合节点
|
|
221
|
-
- 折叠单子复合节点
|
|
222
|
-
- 复合子节点去重
|
|
223
|
-
- canonical ordering
|
|
224
|
-
|
|
225
|
-
### `predicate`
|
|
226
|
-
|
|
227
|
-
在 `shape` 之上对**已建模**算子做**保守**谓词整理:
|
|
228
|
-
|
|
229
|
-
- 同字段谓词去重
|
|
230
|
-
- 可建模的比较类谓词合并
|
|
231
|
-
- 明确矛盾收敛为不可满足过滤器
|
|
232
|
-
- 在 `normalizePredicate` 中,**`$and` 下同名 field 的直接子 `FieldNode` 会先合并**,以便检出诸如 `{ $and: [{ a: 1 }, { a: 2 }] }` 的矛盾
|
|
233
|
-
|
|
234
|
-
### `scope`
|
|
235
|
-
|
|
236
|
-
在 `predicate` 之上:
|
|
237
|
-
|
|
238
|
-
- **继承约束传播**(phase-1 白名单)、**保守分支剪枝**;**覆盖消除**仅在狭窄、已测试场景且策略允许时进行
|
|
239
|
-
- 可选 **`detectCommonPredicatesInOr`**:仅观测(告警/轨迹);**不改写**查询结构
|
|
240
|
-
|
|
241
|
-
---
|
|
242
|
-
|
|
243
|
-
## `meta` 说明
|
|
244
|
-
|
|
245
|
-
| 字段 | 含义 |
|
|
246
|
-
|------|------|
|
|
247
|
-
| `changed` | 输出相对输入是否变化(基于哈希) |
|
|
248
|
-
| `level` | 实际使用的规范化层级 |
|
|
249
|
-
| `appliedRules` / `skippedRules` | 规则应用轨迹 |
|
|
250
|
-
| `warnings` | `observe.collectWarnings` 为真时的非致命告警(规则说明、检测文案等) |
|
|
251
|
-
| `bailedOut` | 是否触发安全熔断 |
|
|
252
|
-
| `bailoutReason` | 熔断原因 |
|
|
253
|
-
| `beforeHash` / `afterHash` | 前后稳定哈希 |
|
|
254
|
-
| `stats` | 可选的前后树统计(`observe.collectMetrics`) |
|
|
255
|
-
| `predicateTraces` | `observe.collectPredicateTraces` 为真时:每字段 planner / 跳过 / 矛盾等轨迹 |
|
|
256
|
-
| `scopeTrace` | `observe.collectScopeTraces` 为真时:约束抽取拒绝原因与 scope 决策事件 |
|
|
257
|
-
|
|
258
|
-
---
|
|
259
|
-
|
|
260
|
-
## 不支持 / opaque 行为
|
|
261
|
-
|
|
262
|
-
以下结构通常**只透传或不参与完整语义改写**,例如:
|
|
263
|
-
|
|
264
|
-
`$nor`、`$regex`、`$not`、`$elemMatch`、`$expr`、geo / text、未知算子等。
|
|
265
|
-
|
|
266
65
|
---
|
|
267
66
|
|
|
268
|
-
##
|
|
269
|
-
|
|
270
|
-
**对外承诺**仅包括:
|
|
67
|
+
## ❌ 这不是什么
|
|
271
68
|
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
69
|
+
* 不是查询优化器
|
|
70
|
+
* 不是索引顾问
|
|
71
|
+
* 不是性能工具
|
|
275
72
|
|
|
276
|
-
|
|
73
|
+
**绝不会猜测**:
|
|
277
74
|
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
75
|
+
* 字段基数
|
|
76
|
+
* schema 约束
|
|
77
|
+
* 数据分布
|
|
281
78
|
|
|
282
|
-
|
|
283
|
-
2. **`predicate` / `scope`** 可能改变查询结构,但在已建模算子上追求 **语义等价**。
|
|
284
|
-
3. **opaque** 节点不会被语义重写。
|
|
285
|
-
4. 在未熔断时,输出应对相同 options 保持 **幂等**。
|
|
286
|
-
5. 本库 **不是** MongoDB 的 planner optimizer。
|
|
79
|
+
不确定 → **跳过**
|
|
287
80
|
|
|
288
81
|
---
|
|
289
82
|
|
|
290
|
-
##
|
|
291
|
-
|
|
292
|
-
**在线主路径** —— 使用默认(`shape`);在 `v0.2.0` 中仍是最稳妥的生产基线:
|
|
83
|
+
## 🚀 快速开始
|
|
293
84
|
|
|
294
85
|
```ts
|
|
295
|
-
normalizeQuery
|
|
296
|
-
```
|
|
297
|
-
|
|
298
|
-
**Predicate 或 Scope** —— 显式传 `level`;请结合 [SPEC.zh-CN.md](SPEC.zh-CN.md) 与契约测试理解“可改写”与“保留”边界:
|
|
86
|
+
import { normalizeQuery } from "mongo-query-normalizer";
|
|
299
87
|
|
|
300
|
-
|
|
301
|
-
normalizeQuery(query, { level: "predicate" });
|
|
88
|
+
const { query } = normalizeQuery(inputQuery);
|
|
302
89
|
```
|
|
303
90
|
|
|
304
91
|
---
|
|
305
92
|
|
|
306
|
-
##
|
|
93
|
+
## 🧠 在架构中的位置
|
|
307
94
|
|
|
308
|
-
```
|
|
309
|
-
|
|
310
|
-
|
|
95
|
+
```text
|
|
96
|
+
Query Builder / ORM
|
|
97
|
+
↓
|
|
98
|
+
normalizeQuery ← (本库)
|
|
99
|
+
↓
|
|
100
|
+
MongoDB
|
|
311
101
|
```
|
|
312
102
|
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
---
|
|
316
|
-
|
|
317
|
-
## 测试
|
|
318
|
-
|
|
319
|
-
### 测试布局
|
|
320
|
-
|
|
321
|
-
本仓库按 **对外 API**、**规范化 level** 与 **跨 level 契约** 组织测试,并保留更深的语义与回归套件。
|
|
322
|
-
|
|
323
|
-
### 目录职责
|
|
324
|
-
|
|
325
|
-
#### `test/api/`
|
|
326
|
-
|
|
327
|
-
覆盖对外 API 与配置面。
|
|
328
|
-
|
|
329
|
-
适合放在此处的验证包括:
|
|
330
|
-
|
|
331
|
-
* `normalizeQuery` 的返回形态与顶层行为
|
|
332
|
-
* `resolveNormalizeOptions`
|
|
333
|
-
* 包导出
|
|
334
|
-
|
|
335
|
-
**不要**把「某一 level 专属的规范化行为」放在这里。
|
|
336
|
-
|
|
337
|
-
---
|
|
338
|
-
|
|
339
|
-
#### `test/levels/`
|
|
340
|
-
|
|
341
|
-
覆盖每个 `NormalizeLevel` 的行为边界。
|
|
342
|
-
|
|
343
|
-
当前 level:
|
|
344
|
-
|
|
345
|
-
* `shape`
|
|
346
|
-
* `predicate`
|
|
347
|
-
* `scope`
|
|
348
|
-
|
|
349
|
-
每个 level 的测试文件宜聚焦四件事:
|
|
350
|
-
|
|
351
|
-
1. 该 level 的**正向能力**
|
|
352
|
-
2. 该 level **明确未启用**的行为
|
|
353
|
-
3. 与**相邻 level** 的对比
|
|
354
|
-
4. 少量**代表性契约**
|
|
355
|
-
|
|
356
|
-
断言上优先:
|
|
357
|
-
|
|
358
|
-
* 规范化后的 **query 结构**
|
|
359
|
-
* **跨 level 可观察的差异**
|
|
360
|
-
* **稳定的对外 meta**(如 `meta.level` 等)
|
|
361
|
-
|
|
362
|
-
尽量避免过度绑定:
|
|
363
|
-
|
|
364
|
-
* warning **逐字全文**
|
|
365
|
-
* 内部 **规则 ID 字符串**
|
|
366
|
-
* **子句顺序**(除非顺序本身就是契约的一部分)
|
|
367
|
-
|
|
368
|
-
---
|
|
369
|
-
|
|
370
|
-
#### `test/contracts/`
|
|
371
|
-
|
|
372
|
-
覆盖「应对所有 level 成立」的契约,或与单一 level 无关的默认行为。
|
|
373
|
-
|
|
374
|
-
适合放在此处的内容包括:
|
|
375
|
-
|
|
376
|
-
* 默认 level 行为
|
|
377
|
-
* 各 level 下的幂等
|
|
378
|
-
* 各 level 下的输出不变式
|
|
379
|
-
* 各 level 下的 opaque 子树保留
|
|
380
|
-
* **`predicate` / `scope` 的正式契约**(支持合并、opaque 保留、scope 策略护栏、规则开关)——见 `test/contracts/predicate-scope-stable-contract.test.js`
|
|
381
|
-
|
|
382
|
-
全 level 套件请配合 `test/helpers/level-contract-runner.js` 使用。
|
|
383
|
-
|
|
384
|
-
---
|
|
385
|
-
|
|
386
|
-
#### `test/semantic/`
|
|
387
|
-
|
|
388
|
-
对照真实执行行为做**语义等价**验证,确保规范化不改变含义。
|
|
389
|
-
|
|
390
|
-
该目录有意与 `levels/`、`contracts/` 分开。
|
|
391
|
-
|
|
392
|
-
---
|
|
393
|
-
|
|
394
|
-
#### `test/property/`
|
|
395
|
-
|
|
396
|
-
基于属性的随机测试与变形(metamorphic)行为。
|
|
397
|
-
|
|
398
|
-
适用于:
|
|
399
|
-
|
|
400
|
-
* 随机语义检查
|
|
401
|
-
* 变形不变式
|
|
402
|
-
* 较宽输入空间上的校验
|
|
403
|
-
|
|
404
|
-
**不要**把它当作表达「level 边界」的主战场。
|
|
103
|
+
你不是要换掉构建器。
|
|
104
|
+
你是要**净化它的输出**。
|
|
405
105
|
|
|
406
106
|
---
|
|
407
107
|
|
|
408
|
-
|
|
108
|
+
## 🧩 适用场景
|
|
409
109
|
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
110
|
+
* 动态筛选 / 搜索 API
|
|
111
|
+
* BI / 报表系统
|
|
112
|
+
* 用户生成的查询
|
|
113
|
+
* 多团队、查询写法不一致的代码库
|
|
114
|
+
* 日志 / 缓存 / 对查询做 diff
|
|
413
115
|
|
|
414
116
|
---
|
|
415
117
|
|
|
416
|
-
|
|
118
|
+
## ⚙️ Levels
|
|
417
119
|
|
|
418
|
-
|
|
120
|
+
| Level | 作用 | 安全级别 |
|
|
121
|
+
| ----------- | -------------- | ---------- |
|
|
122
|
+
| `shape` | 结构规范化 | 🟢 最稳妥 |
|
|
123
|
+
| `predicate` | 安全的谓词简化 | 🟡 |
|
|
124
|
+
| `scope` | 有限的约束传播 | 🟡 |
|
|
419
125
|
|
|
420
|
-
|
|
126
|
+
默认为 `shape`。
|
|
421
127
|
|
|
422
128
|
---
|
|
423
129
|
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
#### `test/helpers/level-runner.js`
|
|
427
|
-
|
|
428
|
-
在指定 level 下执行 `normalizeQuery` 的共享封装。
|
|
429
|
-
|
|
430
|
-
#### `test/helpers/level-cases.js`
|
|
431
|
-
|
|
432
|
-
跨 level 测试共用的固定输入;优先把可复用的代表用例加在这里,避免在多个文件里复制同一段 fixture。
|
|
130
|
+
## 📦 输出
|
|
433
131
|
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
132
|
+
```ts
|
|
133
|
+
{
|
|
134
|
+
query, // 规范化后的查询
|
|
135
|
+
meta // 调试 / 轨迹信息
|
|
136
|
+
}
|
|
137
|
+
```
|
|
437
138
|
|
|
438
139
|
---
|
|
439
140
|
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
#### 新增一条规范化规则时
|
|
141
|
+
## 🎯 设计理念
|
|
443
142
|
|
|
444
|
-
|
|
143
|
+
> 若某次改写可能出错,就不要做。
|
|
445
144
|
|
|
446
|
-
*
|
|
447
|
-
*
|
|
448
|
-
*
|
|
449
|
-
*
|
|
450
|
-
*
|
|
145
|
+
* 不做 schema 假设
|
|
146
|
+
* 不猜数组语义
|
|
147
|
+
* 不做不安全合并
|
|
148
|
+
* 输出确定
|
|
149
|
+
* 结果幂等
|
|
451
150
|
|
|
452
151
|
---
|
|
453
152
|
|
|
454
|
-
|
|
153
|
+
## 🔍 示例
|
|
455
154
|
|
|
456
|
-
|
|
155
|
+
```ts
|
|
156
|
+
const result = normalizeQuery({
|
|
157
|
+
$and: [
|
|
158
|
+
{ status: "open" },
|
|
159
|
+
{ status: { $in: ["open", "closed"] } }
|
|
160
|
+
]
|
|
161
|
+
});
|
|
457
162
|
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
4. 至少补一条与相邻 level 的**对照**用例
|
|
163
|
+
console.log(result.query);
|
|
164
|
+
// { status: "open" }
|
|
165
|
+
```
|
|
462
166
|
|
|
463
167
|
---
|
|
464
168
|
|
|
465
|
-
|
|
169
|
+
## 📚 文档
|
|
466
170
|
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
*
|
|
470
|
-
*
|
|
471
|
-
*
|
|
472
|
-
*
|
|
473
|
-
|
|
474
|
-
忌:
|
|
475
|
-
|
|
476
|
-
* 把 level 测试绑死在易变的实现细节上
|
|
477
|
-
* 同一 fixture 只改断言表面、重复堆砌
|
|
478
|
-
* 把「默认 level」契约塞进某个具体 level 文件
|
|
479
|
-
* 把导出/API 测试与规范化行为测试混在同一文件语义里
|
|
171
|
+
* [`SPEC.zh-CN.md`](SPEC.zh-CN.md) — 行为规格([English](SPEC.md))
|
|
172
|
+
* [`docs/normalization-matrix.zh-CN.md`](docs/normalization-matrix.zh-CN.md) — 规则覆盖([English](docs/normalization-matrix.md))
|
|
173
|
+
* [`docs/CANONICAL_FORM.md`](docs/CANONICAL_FORM.md) — 规范形态与幂等性(目前仅英文)
|
|
174
|
+
* [`CHANGELOG.zh-CN.md`](CHANGELOG.zh-CN.md) — 更新日志([English](CHANGELOG.md))
|
|
175
|
+
* [`test/REGRESSION.md`](test/REGRESSION.md) — 复现 property / 语义测试失败(目前仅英文)
|
|
176
|
+
* [`README.md`](README.md) — English README
|
|
480
177
|
|
|
481
178
|
---
|
|
482
179
|
|
|
483
|
-
|
|
180
|
+
## 🧪 测试
|
|
484
181
|
|
|
485
|
-
*
|
|
486
|
-
*
|
|
487
|
-
*
|
|
488
|
-
* `semantic` / `property` / `regression` / `performance`:**正确、稳健、效率是否仍成立**
|
|
182
|
+
* 语义等价测试(真实 MongoDB)
|
|
183
|
+
* 基于属性的测试
|
|
184
|
+
* 回归套件
|
|
489
185
|
|
|
490
186
|
---
|
|
491
187
|
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
随机语义测试使用 **`mongodb-memory-server`** 与 **`fast-check`**,在固定文档 schema 与受限算子集合下,对比 normalize 前后真实 `find` 结果(相同 `sort` / `skip` / `limit`,投影 `{ _id: 1 }`),并断言 **`_id` 顺序一致**、返回 **`query` 幂等**;对 opaque 算子仅要求**不崩溃、第二次 normalize 稳定**。生成器见 `test/helpers/arbitraries.js`;**`FC_SEED` / `FC_RUNS` 默认值统一由 `test/helpers/fc-config.js` 管理**(也由 `arbitraries.js` 再导出)。
|
|
495
|
-
|
|
496
|
-
为**避免在线下载** MongoDB 二进制,可在运行语义测试前设置 **`MONGODB_BINARY`**、**`MONGOD_BINARY`** 或 **`MONGOMS_SYSTEM_BINARY`** 指向本机 `mongod`(见 `test/helpers/mongo-fixture.js`)。
|
|
497
|
-
|
|
498
|
-
* **`npm run test`**:先 build,再 `test:unit`,再 `test:semantic`。
|
|
499
|
-
* **`npm run test:api`**:仅 `test/api/**/*.test.js`。
|
|
500
|
-
* **`npm run test:levels`**:`test/levels/**/*.test.js` 与 `test/contracts/*.test.js`。
|
|
501
|
-
* **`npm run test:unit`**:除 `test/semantic/**`、`test/regression/**`、`test/property/**` 外的 `test/**/*.test.js`(含 `test/api/**`、`test/levels/**`、`test/contracts/**`、`test/performance/**` 等单元侧用例)。
|
|
502
|
-
* **`npm run test:semantic`**:语义 + 回归 + property(环境变量未设时的默认见 `fc-config.js`)。
|
|
503
|
-
* **`npm run test:semantic:quick`**:降低 **`FC_RUNS`(脚本内为 45)** 并设 **`FC_SEED=42`**,仍包含 `test/regression/**` 与 `test/property/**`。
|
|
504
|
-
* **`npm run test:semantic:ci`**:面向 CI(脚本内 `FC_RUNS=200`、`FC_SEED=42`)。
|
|
505
|
-
|
|
506
|
-
可通过 **`FC_SEED`**、**`FC_RUNS`**、可选 **`FC_QUICK=1`** 覆盖 property 参数(见 `fc-config.js`)。**property 失败如何复现、何时沉淀成固定用例**:见 [`test/REGRESSION.md`](test/REGRESSION.md)。
|
|
507
|
-
|
|
508
|
-
主随机语义等价**不包含**全文、地理、复杂 `$expr`、`$where`、聚合、collation 等;opaque 算子契约见 **`test/contracts/opaque-operators.all-levels.test.js`**。
|
|
509
|
-
|
|
510
|
-
---
|
|
188
|
+
## ⭐ 理念
|
|
511
189
|
|
|
512
|
-
|
|
190
|
+
多数查询工具追求「聪明」。
|
|
513
191
|
|
|
514
|
-
|
|
515
|
-
- [docs/CANONICAL_FORM.md](docs/CANONICAL_FORM.md)
|
|
192
|
+
本库追求**正确**。
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "mongo-query-normalizer",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.3",
|
|
4
4
|
"description": "Observable, level-based normalizer for MongoDB query objects. Defaults to conservative shape stabilization; optional predicate and scope levels with documented contracts. Predictable output and metadata—not planner optimization.",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"types": "dist/index.d.ts",
|