@osqd/jql 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +220 -0
- package/LICENSE +102 -0
- package/README.md +84 -0
- package/bin/jql.mjs +5 -0
- package/conformance/cases.json +290 -0
- package/dist/array.d.ts +11 -0
- package/dist/async.d.ts +44 -0
- package/dist/canonical.d.ts +18 -0
- package/dist/cjs/array.d.ts +11 -0
- package/dist/cjs/async.d.ts +44 -0
- package/dist/cjs/canonical.d.ts +18 -0
- package/dist/cjs/cli.d.ts +15 -0
- package/dist/cjs/collections.d.ts +34 -0
- package/dist/cjs/core.d.ts +117 -0
- package/dist/cjs/errors.d.ts +17 -0
- package/dist/cjs/explain.d.ts +30 -0
- package/dist/cjs/global.d.ts +90 -0
- package/dist/cjs/group.d.ts +47 -0
- package/dist/cjs/index.d.ts +25 -0
- package/dist/cjs/internal/closest.d.ts +9 -0
- package/dist/cjs/internal/duration.d.ts +15 -0
- package/dist/cjs/internal/equal.d.ts +34 -0
- package/dist/cjs/internal/glob.d.ts +30 -0
- package/dist/cjs/internal/order.d.ts +24 -0
- package/dist/cjs/internal/path.d.ts +109 -0
- package/dist/cjs/internal/record.d.ts +18 -0
- package/dist/cjs/internal/values.d.ts +41 -0
- package/dist/cjs/limits.d.ts +64 -0
- package/dist/cjs/operators.d.ts +81 -0
- package/dist/cjs/package.json +3 -0
- package/dist/cjs/plan.d.ts +79 -0
- package/dist/cjs/search.d.ts +42 -0
- package/dist/cjs/targets/mongo.d.ts +55 -0
- package/dist/cjs/text/index.d.ts +12 -0
- package/dist/cjs/text/parse.d.ts +91 -0
- package/dist/cjs/text/suggest.d.ts +16 -0
- package/dist/cjs/text/write.d.ts +34 -0
- package/dist/cjs/types.d.ts +236 -0
- package/dist/cjs/vocabulary.d.ts +106 -0
- package/dist/cli.d.ts +15 -0
- package/dist/cli.js +2729 -0
- package/dist/cli.js.map +1 -0
- package/dist/collections.d.ts +34 -0
- package/dist/core.d.ts +117 -0
- package/dist/errors.d.ts +17 -0
- package/dist/explain.d.ts +30 -0
- package/dist/global.cjs +1953 -0
- package/dist/global.cjs.map +1 -0
- package/dist/global.d.ts +90 -0
- package/dist/global.js +1950 -0
- package/dist/global.js.map +1 -0
- package/dist/group.d.ts +47 -0
- package/dist/index.cjs +2529 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.ts +25 -0
- package/dist/index.js +2495 -0
- package/dist/index.js.map +1 -0
- package/dist/internal/closest.d.ts +9 -0
- package/dist/internal/duration.d.ts +15 -0
- package/dist/internal/equal.d.ts +34 -0
- package/dist/internal/glob.d.ts +30 -0
- package/dist/internal/order.d.ts +24 -0
- package/dist/internal/path.d.ts +109 -0
- package/dist/internal/record.d.ts +18 -0
- package/dist/internal/values.d.ts +41 -0
- package/dist/limits.d.ts +64 -0
- package/dist/mongo.cjs +357 -0
- package/dist/mongo.cjs.map +1 -0
- package/dist/mongo.js +354 -0
- package/dist/mongo.js.map +1 -0
- package/dist/operators.d.ts +81 -0
- package/dist/plan.d.ts +79 -0
- package/dist/search.d.ts +42 -0
- package/dist/targets/mongo.d.ts +55 -0
- package/dist/text/index.d.ts +12 -0
- package/dist/text/parse.d.ts +91 -0
- package/dist/text/suggest.d.ts +16 -0
- package/dist/text/write.d.ts +34 -0
- package/dist/text.cjs +674 -0
- package/dist/text.cjs.map +1 -0
- package/dist/text.js +667 -0
- package/dist/text.js.map +1 -0
- package/dist/types.d.ts +236 -0
- package/dist/vocabulary.d.ts +106 -0
- package/docs/course/01-first-query.md +217 -0
- package/docs/course/02-operators.md +285 -0
- package/docs/course/03-arrays-and-paths.md +239 -0
- package/docs/course/04-combining.md +221 -0
- package/docs/course/05-dates.md +214 -0
- package/docs/course/06-typed-queries.md +240 -0
- package/docs/course/07-requests.md +261 -0
- package/docs/course/08-grouping.md +210 -0
- package/docs/course/09-explaining.md +171 -0
- package/docs/course/10-vocabulary.md +276 -0
- package/docs/course/11-the-search-box.md +349 -0
- package/docs/course/12-untrusted.md +257 -0
- package/docs/course/13-saved-filters.md +199 -0
- package/docs/course/14-streams-and-cli.md +276 -0
- package/docs/course/15-pushdown.md +240 -0
- package/docs/course/16-extending.md +199 -0
- package/docs/course/index.md +185 -0
- package/docs/design/decisions.md +198 -0
- package/docs/design/performance.md +102 -0
- package/docs/guides/adopting.md +81 -0
- package/docs/guides/pushdown.md +147 -0
- package/docs/guides/typescript.md +115 -0
- package/docs/guides/untrusted-input.md +86 -0
- package/docs/index.md +102 -0
- package/docs/reference/api.md +266 -0
- package/docs/reference/cli.md +103 -0
- package/docs/reference/index.md +12 -0
- package/docs/reference/specification.md +549 -0
- package/docs/reference/text-syntax.md +152 -0
- package/docs/start/quick-start.md +84 -0
- package/package.json +136 -0
|
@@ -0,0 +1,549 @@
|
|
|
1
|
+
# The JQL specification
|
|
2
|
+
|
|
3
|
+
JSON Query Language, version 1.1: what a query means, independent of any implementation.
|
|
4
|
+
|
|
5
|
+
← [Documentation](../index.md) · [Reference](index.md)
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
This page is the standard. The TypeScript library in this repository is one implementation
|
|
10
|
+
of it; a service written in another language that reads the same saved filters is another,
|
|
11
|
+
and it must give the same answers. Where this page and the library disagree, the library has
|
|
12
|
+
a bug.
|
|
13
|
+
|
|
14
|
+
The executable form of this page is [`conformance/cases.json`](../../conformance/cases.json):
|
|
15
|
+
plain JSON documents, plain JSON queries, and the positions that must match. **An
|
|
16
|
+
implementation conforms when it passes every case there.** Nothing in the suite depends on
|
|
17
|
+
JavaScript.
|
|
18
|
+
|
|
19
|
+
The words *must*, *must not* and *may* mean what they mean in an RFC.
|
|
20
|
+
|
|
21
|
+
## 1. Documents
|
|
22
|
+
|
|
23
|
+
A query runs against one **item** at a time. An item is any JSON value: usually an object,
|
|
24
|
+
sometimes a string or a number (a list of tags, a column of scores).
|
|
25
|
+
|
|
26
|
+
An implementation may extend the data model with values its language has and JSON does not,
|
|
27
|
+
provided each maps onto this page:
|
|
28
|
+
|
|
29
|
+
| Value | Treated as |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| a date object | a date, for `$date` comparisons and `$type: "date"`; an object nowhere else |
|
|
32
|
+
| a map / dictionary type | an object whose keys are the map's keys |
|
|
33
|
+
| a big integer | a number — for equality, ordering, `$mod` and `$type: "number"`; `$type: "bigint"` names it as well |
|
|
34
|
+
|
|
35
|
+
Only a document's **own** fields are visible. Inherited properties, prototype members and the
|
|
36
|
+
methods of a map type are never reached by a path.
|
|
37
|
+
|
|
38
|
+
## 2. Queries
|
|
39
|
+
|
|
40
|
+
A query is a JSON **object**. Anything else — an array, a string, `null` — must be refused.
|
|
41
|
+
|
|
42
|
+
Each key is either a **field path** (§3) or an **operator** beginning with `$`. Every key must
|
|
43
|
+
hold: a query is the conjunction of its keys. The empty query `{}` matches every item.
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{ "status": "open", "total": { "$gte": 100 } }
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
A key whose value is absent (`undefined` in languages that have it) must be refused. JSON
|
|
50
|
+
cannot carry such a key, and an implementation that dropped it would turn a condition on
|
|
51
|
+
nothing into a query that matches everything.
|
|
52
|
+
|
|
53
|
+
## 3. Paths
|
|
54
|
+
|
|
55
|
+
A field path is a string of **segments** separated by `.`. A path with an empty segment
|
|
56
|
+
(`"a..b"`, `""`, `"a."`) must be refused: it reaches nothing.
|
|
57
|
+
|
|
58
|
+
A path is resolved against an item, producing a list of **reached values**:
|
|
59
|
+
|
|
60
|
+
1. Start with the item.
|
|
61
|
+
2. For each segment:
|
|
62
|
+
- on an **object**, take the field of that name. A missing field reaches *missing*.
|
|
63
|
+
- on an **array**, if the segment is a non-negative integer written without leading
|
|
64
|
+
zeros, take the element at that position (missing if out of range). Otherwise apply the
|
|
65
|
+
same segment to **every element** — the array is seen through. An empty array reaches
|
|
66
|
+
*missing*.
|
|
67
|
+
- on anything else (a string, a number, `null`, *missing*), reach *missing*.
|
|
68
|
+
3. The values left at the end are the reached values.
|
|
69
|
+
|
|
70
|
+
*Missing* is a value a condition can test (§5.3). So `{ "a.b": null }` matches an item where
|
|
71
|
+
`a` is absent, where `a` has no `b`, and where `a.b` is `null`.
|
|
72
|
+
|
|
73
|
+
The first segment is a segment like any other: an item that is **itself** an array is seen
|
|
74
|
+
through, so `{ "sku": "pen" }` matches `[{ "sku": "pen" }]`, and an array's own `length` is
|
|
75
|
+
not a field — `$size` asks that question.
|
|
76
|
+
|
|
77
|
+
So is an array met **inside** an array. Applying a segment to every element, and then to the
|
|
78
|
+
elements of an element that is itself an array, follows from the rule above and is worth
|
|
79
|
+
saying out loud: `{ "a.b": 1 }` matches `{ "a": [[{ "b": 1 }]]}`. A store that resolves only
|
|
80
|
+
one array level will answer such a document differently, which is a thing to know before
|
|
81
|
+
handing it half a query (§14).
|
|
82
|
+
|
|
83
|
+
An implementation may stop descending into a document past a depth of its choosing (at least
|
|
84
|
+
64), treating anything deeper as missing. A document is data from outside; a walk over it that
|
|
85
|
+
could not stop would be a way to end the process.
|
|
86
|
+
|
|
87
|
+
It may also cap the **number of segments** in a path, and must refuse a longer one rather
|
|
88
|
+
than shorten it. A path is the one part of a query whose size the operator count does not
|
|
89
|
+
bound — `{"a.a.a…": 1}` is one field however long it is — and a path longer than the depth
|
|
90
|
+
the implementation descends to can reach nothing in any case. Whatever the cap is, it must be
|
|
91
|
+
the same for a path in a query, in a sort, and in a projection: a path that matches and is
|
|
92
|
+
then left out of the result is one name meaning two things. The reference implementation
|
|
93
|
+
allows 512.
|
|
94
|
+
|
|
95
|
+
```json
|
|
96
|
+
{ "items": [ { "sku": "pen" }, { "sku": "ink" } ] }
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
| Path | Reaches |
|
|
100
|
+
| --- | --- |
|
|
101
|
+
| `items.sku` | `"pen"`, `"ink"` |
|
|
102
|
+
| `items.1.sku` | `"ink"` |
|
|
103
|
+
| `items.5.sku` | *missing* |
|
|
104
|
+
| `items` | the array itself |
|
|
105
|
+
|
|
106
|
+
## 4. Conditions on a field
|
|
107
|
+
|
|
108
|
+
The value under a field path is one of:
|
|
109
|
+
|
|
110
|
+
- **a literal**: any JSON value that is not an object with `$` keys. It means `{ "$eq": literal }`.
|
|
111
|
+
- **a condition**: an object whose keys all begin with `$`. Every operator in it must hold.
|
|
112
|
+
- **a date literal** `{ "$date": … }` (§7), which is a literal.
|
|
113
|
+
|
|
114
|
+
An object that mixes `$` keys with other keys must be refused: it is neither a condition nor a
|
|
115
|
+
value, and guessing which was meant would answer a different question.
|
|
116
|
+
|
|
117
|
+
### 4.1 How a condition meets an array
|
|
118
|
+
|
|
119
|
+
Each operator is evaluated against **every reached value** and, where the table in §5 says
|
|
120
|
+
*elementwise*, also against **every element** of a reached value that is an array. A positive
|
|
121
|
+
operator holds when it holds for **any** of them.
|
|
122
|
+
|
|
123
|
+
Every operator in a condition is evaluated **separately**. On an array, `{ "$gt": 5, "$lt": 10 }`
|
|
124
|
+
holds for `[1, 20]`: some element is above 5 and some element is below 10. To require one
|
|
125
|
+
element to satisfy several operators, use `$elemMatch`.
|
|
126
|
+
|
|
127
|
+
The negating operators — `$ne`, `$nin`, `$exists: false`, `$not` — hold when their positive
|
|
128
|
+
counterpart holds for **none** of the reached values. So they match an item where the field
|
|
129
|
+
is missing.
|
|
130
|
+
|
|
131
|
+
## 5. Field operators
|
|
132
|
+
|
|
133
|
+
### 5.1 Equality and membership
|
|
134
|
+
|
|
135
|
+
| Operator | Operand | Holds when | Elementwise |
|
|
136
|
+
| --- | --- | --- | --- |
|
|
137
|
+
| `$eq` | a literal | a reached value equals it (§5.1.1) | yes |
|
|
138
|
+
| `$ne` | a literal | `$eq` holds for none | — |
|
|
139
|
+
| `$in` | a list of literals | a reached value equals any of them. An empty list matches nothing | yes |
|
|
140
|
+
| `$nin` | a list of literals | `$in` holds for none. An empty list matches everything | — |
|
|
141
|
+
|
|
142
|
+
#### 5.1.1 Equality
|
|
143
|
+
|
|
144
|
+
- Values of different JSON types are never equal. `1` is not `"1"`, `true` is not `1`.
|
|
145
|
+
- Numbers compare numerically; `1` and `1.0` are equal. In a language with a separate big
|
|
146
|
+
integer type, a big integer is a number here too (§1): `10` equals a stored `10n`. The
|
|
147
|
+
alternative is a value that is at once `$gte: 10`, `$lte: 10` and `$ne: 10`. It matters
|
|
148
|
+
only for documents held in memory, since JSON has no big integer and a query cannot carry
|
|
149
|
+
one; a query may only ever name such a value with an ordinary number.
|
|
150
|
+
- Strings compare by code point, exactly, unless `$options` has `i` (§5.5).
|
|
151
|
+
- Arrays are equal when they have equal elements **in the same order**.
|
|
152
|
+
- Objects are equal when they have the same keys with equal values, **in any order**. A key
|
|
153
|
+
whose value is absent counts as not present.
|
|
154
|
+
- `null` as a literal matches `null` and *missing*.
|
|
155
|
+
- A `$date` literal equals any value that reads as the same instant (§7).
|
|
156
|
+
|
|
157
|
+
Because `$eq` is elementwise, `{ "tags": "a" }` matches `["a", "b"]`, and because the whole
|
|
158
|
+
value is tried first, `{ "tags": ["a", "b"] }` matches that array exactly.
|
|
159
|
+
|
|
160
|
+
### 5.2 Ordering
|
|
161
|
+
|
|
162
|
+
| Operator | Holds when a reached value (or element) is |
|
|
163
|
+
| --- | --- |
|
|
164
|
+
| `$gt` | greater than the operand |
|
|
165
|
+
| `$gte` | greater than or equal to it |
|
|
166
|
+
| `$lt` | less than it |
|
|
167
|
+
| `$lte` | less than or equal to it |
|
|
168
|
+
|
|
169
|
+
The operand must be a number, a string or a `$date` literal; anything else must be refused.
|
|
170
|
+
**Nothing is coerced:** a number bound compares only with numbers, a string bound only with
|
|
171
|
+
strings (by code unit), a `$date` bound with anything that reads as a date. `{ "$gt": 5 }` never
|
|
172
|
+
matches `"10"`. An ordering operator never matches *missing* or `null`.
|
|
173
|
+
|
|
174
|
+
### 5.3 Existence and type
|
|
175
|
+
|
|
176
|
+
| Operator | Operand | Holds when |
|
|
177
|
+
| --- | --- | --- |
|
|
178
|
+
| `$exists` | `true` / `false` | some reached value is not *missing* / every reached value is *missing*. `null` exists |
|
|
179
|
+
| `$type` | a type name, or a non-empty list of them | a reached value (or element) is of one of those types |
|
|
180
|
+
|
|
181
|
+
The type names are `string`, `number`, `integer` (a number with no fractional part), `bigint`,
|
|
182
|
+
`boolean`, `null`, `array`, `object` and `date`. An unknown name must be refused. `object`
|
|
183
|
+
does not include arrays or dates. A big integer is a `number` and an `integer` as well as a
|
|
184
|
+
`bigint`, because it is one for ordering (§1) and a value that `$gt` treats as a number while
|
|
185
|
+
`$type` does not is a distinction nobody could explain.
|
|
186
|
+
|
|
187
|
+
### 5.4 Strings
|
|
188
|
+
|
|
189
|
+
| Operator | Operand | Holds when a reached string (or string element) |
|
|
190
|
+
| --- | --- | --- |
|
|
191
|
+
| `$contains` | a string, or a list meaning any of them | contains it |
|
|
192
|
+
| `$startsWith` | the same | starts with it |
|
|
193
|
+
| `$endsWith` | the same | ends with it |
|
|
194
|
+
| `$word` | the same | contains it as a whole component (below) |
|
|
195
|
+
| `$glob` | a glob, or a list of them | matches it as a whole (below) |
|
|
196
|
+
| `$regex` | an ECMAScript pattern, as a string | matches it |
|
|
197
|
+
|
|
198
|
+
A string operator never matches a value that is not a string: numbers are not converted.
|
|
199
|
+
An empty list matches nothing.
|
|
200
|
+
|
|
201
|
+
**`$word`** matches when the operand occurs in the string, and at each end of the occurrence
|
|
202
|
+
either the string ends or the characters on both sides of the boundary are not both letters or
|
|
203
|
+
digits. So `1.2.3` matches `1.2.3.4` and `1.2.3.0/24` but not `11.2.3.4`, and `1.2.3.4` does not
|
|
204
|
+
match `1.2.3.45`. An operand that itself ends in a separator, such as `203.0.113.`, chooses its
|
|
205
|
+
own boundary on that side. An empty operand matches nothing. "Letter" and "digit" are the
|
|
206
|
+
Unicode `L` and `N` categories.
|
|
207
|
+
|
|
208
|
+
**`$glob`** matches the **whole** value, not part of it. `*` stands for any run of characters
|
|
209
|
+
including none, `?` for exactly one character, and `\` escapes either of them or itself. There
|
|
210
|
+
are no character classes: `[a-z]` is those five characters. A glob needs no
|
|
211
|
+
regular-expression engine and cannot backtrack exponentially, so an implementation must keep
|
|
212
|
+
it available when patterns are turned off.
|
|
213
|
+
|
|
214
|
+
**`$regex`** uses ECMAScript regular-expression syntax. An implementation on another platform
|
|
215
|
+
must either use an ECMAScript-compatible engine or refuse patterns it cannot run faithfully;
|
|
216
|
+
it must not run them under a different dialect. A pattern that does not compile must be
|
|
217
|
+
refused.
|
|
218
|
+
|
|
219
|
+
### 5.5 `$options`
|
|
220
|
+
|
|
221
|
+
`$options` is a string of flags that changes the other operators in **the same condition**:
|
|
222
|
+
|
|
223
|
+
| Flag | Effect |
|
|
224
|
+
| --- | --- |
|
|
225
|
+
| `i` | `$eq`, `$ne`, `$in`, `$nin`, `$all`, the string operators and `$regex` ignore case |
|
|
226
|
+
| `m`, `s`, `u` | passed to `$regex` |
|
|
227
|
+
|
|
228
|
+
Any other flag must be refused — `g` and `y` in particular, which make an ECMAScript pattern
|
|
229
|
+
stateful. A condition whose `$options` has nothing to apply to must be refused, and `m`, `s` or
|
|
230
|
+
`u` without a `$regex` must be refused.
|
|
231
|
+
|
|
232
|
+
**Ignoring case** means comparing both sides after Unicode default lower-case mapping, with no
|
|
233
|
+
locale. It applies to **every string the operator compares**, including the strings inside a
|
|
234
|
+
list or an object being compared — `{ "$eq": ["A"], "$options": "i" }` matches `["a"]` — but
|
|
235
|
+
never to the *keys* of an object, which are field names rather than values. `$regex` with `i`
|
|
236
|
+
uses the regular-expression engine's own case folding.
|
|
237
|
+
|
|
238
|
+
### 5.6 Arrays and length
|
|
239
|
+
|
|
240
|
+
| Operator | Operand | Holds when a reached value |
|
|
241
|
+
| --- | --- | --- |
|
|
242
|
+
| `$size` | a non-negative integer, or a condition on one | is an array of that length |
|
|
243
|
+
| `$length` | the same | is an array of that many elements, or a string of that many UTF-16 code units |
|
|
244
|
+
| `$all` | a list of literals | contains each literal as an element (or equals it). An empty list matches nothing |
|
|
245
|
+
| `$elemMatch` | a query | is an array with at least one element that matches the whole query |
|
|
246
|
+
|
|
247
|
+
`$size` and `$length` are the only operators that are **not** tried against the elements of
|
|
248
|
+
an array: `{ "tags": { "$length": 1 } }` would otherwise mean both "one tag" and "a tag one
|
|
249
|
+
character long", and nothing in the query would say which. A string's length is counted in
|
|
250
|
+
UTF-16 code units, which is what JSON's own escape syntax is defined in, so a character
|
|
251
|
+
outside the basic plane counts as two.
|
|
252
|
+
|
|
253
|
+
`$elemMatch` takes a full query, so its operand may name fields of object elements
|
|
254
|
+
(`{ "name": "en", "level": { "$gte": 3 } }`) or be a condition on primitive elements
|
|
255
|
+
(`{ "$gte": 3, "$lt": 5 }`, see §6.2).
|
|
256
|
+
|
|
257
|
+
### 5.7 Arithmetic
|
|
258
|
+
|
|
259
|
+
| Operator | Operand | Holds when |
|
|
260
|
+
| --- | --- | --- |
|
|
261
|
+
| `$mod` | `[divisor, remainder]` | a reached number (or element) divided by `divisor` leaves `remainder`, with the sign of the dividend |
|
|
262
|
+
|
|
263
|
+
A divisor of zero must be refused. "Number" here means what it means in §5.3, so a big
|
|
264
|
+
integer is one; a divisor with a fraction cannot divide one, and no big integer matches it.
|
|
265
|
+
|
|
266
|
+
### 5.8 `$not` on a field
|
|
267
|
+
|
|
268
|
+
`$not` takes a **condition** and holds when that condition does not hold. `{ "score": { "$not":
|
|
269
|
+
{ "$gt": 50 } } }` matches a missing score. A literal operand must be refused; `$ne` is the way
|
|
270
|
+
to negate a value.
|
|
271
|
+
|
|
272
|
+
### 5.9 Comparing with another field
|
|
273
|
+
|
|
274
|
+
`{ "$field": "path" }` in place of a value compares the field with **another field of the same
|
|
275
|
+
item**:
|
|
276
|
+
|
|
277
|
+
```json
|
|
278
|
+
{ "bytesOut": { "$gt": { "$field": "bytesIn" } } }
|
|
279
|
+
{ "paidAt": { "$field": "refundedAt" } }
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
- It is accepted by `$eq`, `$ne` and the four ordering operators, and as a value on its own,
|
|
283
|
+
which means `$eq`. Anywhere else — inside `$in`, `$all`, a literal object — it must be
|
|
284
|
+
refused: a reference that was read as data would compare against the characters
|
|
285
|
+
`{"$field":…}` and match nothing, silently.
|
|
286
|
+
- Both sides are reached by the rules of §3 and §4.1, so either may fan out over an array;
|
|
287
|
+
the comparison holds when **some** pair of reached values satisfies it, and `$ne` when no
|
|
288
|
+
pair does.
|
|
289
|
+
- **A reference that reaches nothing matches nothing.** A comparison with an absent field is
|
|
290
|
+
not a comparison, and treating it as one would widen a filter exactly where the data is
|
|
291
|
+
incomplete.
|
|
292
|
+
- Nothing is read as a date, and nothing is coerced. A `$date` in a query is the query saying
|
|
293
|
+
what it means; a reference says only where to look, so `"5"` and `5` are still different.
|
|
294
|
+
|
|
295
|
+
It carries no expressions: a reference names a field and nothing else. There is deliberately
|
|
296
|
+
no arithmetic, no function, no stored code and no query operator that evaluates one. Every
|
|
297
|
+
operand in this language is data, which is what lets a query be stored, sent and checked
|
|
298
|
+
before it runs.
|
|
299
|
+
|
|
300
|
+
## 6. Query operators
|
|
301
|
+
|
|
302
|
+
### 6.1 Combining queries
|
|
303
|
+
|
|
304
|
+
| Operator | Operand | Holds when | Empty operand |
|
|
305
|
+
| --- | --- | --- | --- |
|
|
306
|
+
| `$and` | a list of queries | every one holds | matches everything |
|
|
307
|
+
| `$or` | a list of queries | at least one holds | matches nothing |
|
|
308
|
+
| `$nor` | a list of queries | none holds | matches everything |
|
|
309
|
+
| `$not` | a query | it does not hold | — |
|
|
310
|
+
| `$comment` | a string | always; it is carried and ignored | — |
|
|
311
|
+
|
|
312
|
+
A query operator inside a field condition (`{ "a": { "$or": … } }`) must be refused.
|
|
313
|
+
|
|
314
|
+
### 6.2 Conditions on the item itself
|
|
315
|
+
|
|
316
|
+
A field operator at the top level of a query is a condition on the **item itself**. It is what
|
|
317
|
+
lets a query filter a list of primitives:
|
|
318
|
+
|
|
319
|
+
```json
|
|
320
|
+
{ "$gte": 10, "$lt": 20 }
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
matches the numbers from 10 up to 20. Field paths and field operators may appear side by side;
|
|
324
|
+
each still has to hold.
|
|
325
|
+
|
|
326
|
+
### 6.3 `$text`
|
|
327
|
+
|
|
328
|
+
`$text` searches free text. Its operand is a phrase, or an object:
|
|
329
|
+
|
|
330
|
+
```json
|
|
331
|
+
{ "$text": "new york" }
|
|
332
|
+
{ "$text": { "$search": "Tor", "$fields": ["nick", "name"], "$caseSensitive": true } }
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
- The phrase is matched as a **substring**, ignoring case unless `$caseSensitive` is `true`.
|
|
336
|
+
Words are not split: `"new york"` does not match an item holding `york new`.
|
|
337
|
+
- Without `$fields`, every string anywhere in the item is searched, through objects and
|
|
338
|
+
arrays. Keys are not searched.
|
|
339
|
+
- Numbers are searched too, in their shortest round-trip decimal form, when the phrase
|
|
340
|
+
contains a digit.
|
|
341
|
+
- With `$fields`, only the values those paths reach are searched (and everything inside them).
|
|
342
|
+
- An empty phrase matches everything.
|
|
343
|
+
- `$fields` is checked whichever phrase stands beside it: a list that is not a non-empty list
|
|
344
|
+
of field names must be refused even when the phrase is empty. Otherwise the same saved
|
|
345
|
+
query is valid or not depending on how much of it somebody has typed.
|
|
346
|
+
- Any other key in the object must be refused.
|
|
347
|
+
|
|
348
|
+
An implementation must stop descending at a depth of its choosing (at least 8), so a cyclic
|
|
349
|
+
item cannot hang a search.
|
|
350
|
+
|
|
351
|
+
## 7. Dates
|
|
352
|
+
|
|
353
|
+
JSON has no date type, so a date is written `{ "$date": value }`, where `value` is one of:
|
|
354
|
+
|
|
355
|
+
| Value | Means |
|
|
356
|
+
| --- | --- |
|
|
357
|
+
| an ISO 8601 string | that instant |
|
|
358
|
+
| a number | that many milliseconds since the Unix epoch |
|
|
359
|
+
| `"now"` | the instant the query is compiled |
|
|
360
|
+
| `{ "$ago": "<duration>" }` | that long before it |
|
|
361
|
+
| `{ "$ahead": "<duration>" }` | that long after it |
|
|
362
|
+
|
|
363
|
+
A duration is one or more counts with a unit, in any order: `"90s"`, `"1h30m"`, `"7d"`. The
|
|
364
|
+
units are `ms`, `s`, `m`, `h`, `d` and `w`. Months and years are **not** units, because
|
|
365
|
+
neither has a fixed length and a window that changed size with the calendar is one nobody can
|
|
366
|
+
reason about. A `$date` that does not parse must be refused.
|
|
367
|
+
|
|
368
|
+
**Relative dates resolve once**, when the query is compiled, against an instant the caller may
|
|
369
|
+
supply. Resolving per comparison would let the window move while a scan was in progress, so
|
|
370
|
+
two items a second apart could be judged against different hours; and an implementation whose
|
|
371
|
+
"now" cannot be supplied cannot be tested, which is why the conformance suite gives an instant
|
|
372
|
+
with every relative case. A compiled query therefore keeps the instant it was compiled at:
|
|
373
|
+
compile it again to move the window.
|
|
374
|
+
|
|
375
|
+
When a query compares with a `$date` — as an ordering bound or as an equality literal — the
|
|
376
|
+
value in the item is **read as a date for that comparison**: a native date object, an ISO 8601
|
|
377
|
+
string or an epoch-milliseconds number. A value that does not read as a date does not match.
|
|
378
|
+
No value is ever read as a date unless the query said `$date`.
|
|
379
|
+
|
|
380
|
+
**Only ISO 8601**, with either `T` or a space before the time. What a host language's own date
|
|
381
|
+
parser accepts beyond that — `"12/31/2020"`, `"Mar 5 2021"`, a bare `"5"` — is its own
|
|
382
|
+
business, and a document that matched in one implementation and not another would make the
|
|
383
|
+
same query mean two things.
|
|
384
|
+
|
|
385
|
+
A date-only ISO string (`"2026-09-01"`) is midnight UTC, and so is a timestamp written
|
|
386
|
+
without an offset: `"2026-09-01T12:00:00"` means 12:00 **UTC**, not 12:00 wherever the process
|
|
387
|
+
happens to be running. JavaScript reads those two forms differently — the first as UTC and the
|
|
388
|
+
second as local time — which would make the same query and the same document answer one way in
|
|
389
|
+
Tokyo and another in London. A machine's location is not part of a query.
|
|
390
|
+
|
|
391
|
+
## 8. Requests
|
|
392
|
+
|
|
393
|
+
A request asks a whole question of a collection. It is an object with these keys, all optional;
|
|
394
|
+
any other key must be refused:
|
|
395
|
+
|
|
396
|
+
| Key | Meaning |
|
|
397
|
+
| --- | --- |
|
|
398
|
+
| `where` | a query; absent matches everything |
|
|
399
|
+
| `sort` | an object of field paths to directions (`1`, `-1`, `"asc"`, `"desc"`), in order of precedence |
|
|
400
|
+
| `skip` | how many matching items to pass over first; a non-negative integer, default 0 |
|
|
401
|
+
| `limit` | the most items to return; a non-negative integer, absent for all |
|
|
402
|
+
| `fields` | a list of paths to keep in each result; absent returns whole items |
|
|
403
|
+
| `omit` | a list of paths to drop from each result, applied after `fields` |
|
|
404
|
+
|
|
405
|
+
```json
|
|
406
|
+
{ "where": { "status": "open" }, "sort": { "total": -1, "id": 1 }, "limit": 20, "fields": ["id", "total"] }
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
The conformance suite covers requests as well as matching: its `requests` section gives each
|
|
410
|
+
request and the positions it must return, or — where the request projects — the objects
|
|
411
|
+
themselves.
|
|
412
|
+
|
|
413
|
+
### 8.1 Sort order
|
|
414
|
+
|
|
415
|
+
Values of different types sort in this fixed order, least first:
|
|
416
|
+
|
|
417
|
+
*missing* and `null` < numbers < strings < objects < booleans < dates
|
|
418
|
+
|
|
419
|
+
Numbers compare numerically, strings by UTF-16 code unit (not by locale), booleans `false`
|
|
420
|
+
before `true`, dates by instant. Objects compare equal to one another. A value that is a
|
|
421
|
+
number but not a numeric one — `NaN`, which JSON cannot carry but a document in memory can —
|
|
422
|
+
sorts with *missing* and `null`, because it has no place among the numbers: every comparison
|
|
423
|
+
with it is false.
|
|
424
|
+
|
|
425
|
+
A path reaching several values sorts by the **least** of them ascending and the **greatest**
|
|
426
|
+
descending, looking into arrays.
|
|
427
|
+
|
|
428
|
+
**The sort is stable:** items equal on every key keep the order they were given in. That is
|
|
429
|
+
what makes `skip` and `limit` page through a sorted result without repeating or dropping an
|
|
430
|
+
item.
|
|
431
|
+
|
|
432
|
+
### 8.2 Projection and redaction
|
|
433
|
+
|
|
434
|
+
`fields` keeps the named paths and drops everything else, in the item's own shape:
|
|
435
|
+
`"address.city"` returns `{ "address": { "city": … } }`. An array on the way is kept as an
|
|
436
|
+
array of its projected object elements; elements that are not objects are dropped.
|
|
437
|
+
|
|
438
|
+
`omit` removes the named paths from each result, through arrays as a path does, and applies
|
|
439
|
+
after `fields` — so `fields: ["request"]` with `omit: ["request.headers.cookie"]` keeps the
|
|
440
|
+
request without that header. A result with something dropped is a **copy**: a request is a
|
|
441
|
+
question, and a question must not edit what it is asked of.
|
|
442
|
+
|
|
443
|
+
A field is data whatever it is called. An implementation must carry a field named
|
|
444
|
+
`__proto__`, `constructor` or anything else its language treats specially as an ordinary
|
|
445
|
+
property of the result — writing it the way that language writes a field it does not trust. A missing path is left out. Naming a path and one of its
|
|
446
|
+
parents keeps the parent whole.
|
|
447
|
+
|
|
448
|
+
## 9. Refusals
|
|
449
|
+
|
|
450
|
+
An implementation must refuse, rather than evaluate, every query this page says must be
|
|
451
|
+
refused. A refusal must say what was wrong and where: the location is written the way the
|
|
452
|
+
value would be reached in code, with `.` between keys and `[n]` for list positions —
|
|
453
|
+
`$or[1].age.$gt`.
|
|
454
|
+
|
|
455
|
+
It must also refuse an unknown operator, at the top level or in a condition. Refusing is the
|
|
456
|
+
point: an operator that is ignored makes a query that looks restrictive and is not.
|
|
457
|
+
|
|
458
|
+
## 10. Limits
|
|
459
|
+
|
|
460
|
+
An implementation should also let a caller say **which operators** a query may use, and
|
|
461
|
+
refuse the rest by name. An endpoint that has no use for a whole-document text search or for
|
|
462
|
+
`$elemMatch` should be able to say so once, rather than discovering later which of them
|
|
463
|
+
somebody found.
|
|
464
|
+
|
|
465
|
+
An implementation may cap the size of a query it accepts — its depth, its number of operators,
|
|
466
|
+
the length of a pattern — and must refuse a query past its caps rather than truncate it. It
|
|
467
|
+
should document its caps. The reference implementation's defaults are a depth of 32, 10 000
|
|
468
|
+
fields and operators, 1 024 characters per pattern and 512 segments per path (§3), with a
|
|
469
|
+
stricter set for untrusted input that also turns `$regex` off.
|
|
470
|
+
|
|
471
|
+
## 11. Versions
|
|
472
|
+
|
|
473
|
+
This is version 1.1 of the language: 1.0 plus `$glob`, `$length`, `{ "$field": … }`, relative
|
|
474
|
+
dates and `omit`. Every query valid under 1.0 means the same thing under 1.1. Adding an
|
|
475
|
+
operator is a minor version, and a query valid under an earlier minor version means the same
|
|
476
|
+
thing under a later one. Changing what an existing query matches is a major version — as is
|
|
477
|
+
*narrowing* a case the specification left open, which is why the rule for a timestamp with no
|
|
478
|
+
offset (§7) had to be settled before 1.1 was published rather than after. The conformance
|
|
479
|
+
suite carries the version it tests.
|
|
480
|
+
|
|
481
|
+
## 12. Added operators
|
|
482
|
+
|
|
483
|
+
An implementation may let a project add operators of its own — an address-in-network test, a
|
|
484
|
+
domain-specific score — and a query using one is **not portable JQL**. To keep that visible
|
|
485
|
+
and to leave the language room to grow, an added operator's name must begin `$x` followed by a
|
|
486
|
+
capital: `$xCidr`. An engine that was not given the operator must refuse the query rather than
|
|
487
|
+
ignore the clause, and should say that the name is an addition it was not given.
|
|
488
|
+
|
|
489
|
+
No name beginning `$x` will ever be part of this specification.
|
|
490
|
+
|
|
491
|
+
## 13. Canonical form (optional)
|
|
492
|
+
|
|
493
|
+
Two queries can ask the same question and be written differently, which makes a list of saved
|
|
494
|
+
filters hold duplicates nobody can see. An implementation may offer a canonical form, and if
|
|
495
|
+
it does, these are the rules, so that two implementations agree on which filters are the same
|
|
496
|
+
one:
|
|
497
|
+
|
|
498
|
+
- Every field's value is written as a condition: `{ "a": 1 }` becomes `{ "a": { "$eq": 1 } }`.
|
|
499
|
+
- Object keys are sorted.
|
|
500
|
+
- `$comment` is dropped.
|
|
501
|
+
- `$and` within `$and`, and `$or` within `$or`, are flattened; a one-part `$and` or `$or` is
|
|
502
|
+
replaced by that part **only when it is the whole query** (after `$comment` is dropped),
|
|
503
|
+
since `{ "a": 1, "$or": [x] }` is not `x`. `$nor` is left as it is.
|
|
504
|
+
- A part of an `$and` is **lifted into the query beside it** when every key it holds is still
|
|
505
|
+
free, so `{ "$and": [x, y] }` and `{ …x, …y }` — the same question, one written by a search
|
|
506
|
+
box and one by a person — have one form. A part whose key is already taken stays in the
|
|
507
|
+
`$and`: two conditions on one field cannot become one key, and merging them would change
|
|
508
|
+
which operators an `$options` beside them reaches.
|
|
509
|
+
- `$in`, `$nin`, `$all`, `$fields` and a string operator's list are sets: sorted by their
|
|
510
|
+
written form, with repeats removed. A one-name `$type` list becomes the name.
|
|
511
|
+
- **A value being compared with is not rewritten**, beyond sorting the keys of an object
|
|
512
|
+
(which equality ignores anyway). It is data, not a query: rewritten as one, `{ "a": { "x":
|
|
513
|
+
1 } }` would become a filter for a document holding `{ "x": { "$eq": 1 } }`.
|
|
514
|
+
- `$comment` is dropped before any of this, so it does not decide whether an operator is the
|
|
515
|
+
whole query.
|
|
516
|
+
- `$options` flags are sorted; `$text` is written in its object form, with `$caseSensitive`
|
|
517
|
+
present only when true.
|
|
518
|
+
- `{ "$date": … }` and `{ "$field": … }` are left as they are.
|
|
519
|
+
|
|
520
|
+
The canonical form is a query, and matches exactly what the query it was made from matches.
|
|
521
|
+
Rewriting a canonical query changes nothing. Equal forms mean the same query; **different
|
|
522
|
+
forms do not mean different answers** — `{ "$gt": 3 }` and `{ "$gte": 4 }` agree on every
|
|
523
|
+
integer, and no rewriting will notice.
|
|
524
|
+
|
|
525
|
+
## 14. Splitting a query
|
|
526
|
+
|
|
527
|
+
A query may be split between two engines — a store that can answer part of it, and an engine
|
|
528
|
+
that answers the rest — and the split is safe only along a **conjunction**:
|
|
529
|
+
|
|
530
|
+
- the keys of a query all have to hold, and so do the parts of an `$and`, so each one may be
|
|
531
|
+
answered by either side;
|
|
532
|
+
- nothing else may be divided: a branch of an `$or`, or one operator of a condition, cannot
|
|
533
|
+
be separated from the rest without changing what is asked.
|
|
534
|
+
|
|
535
|
+
For any such split,
|
|
536
|
+
|
|
537
|
+
> `pushed` ∧ `remaining` ≡ the query
|
|
538
|
+
|
|
539
|
+
and neither part is ever **narrower** than the query, so an engine that answers only the
|
|
540
|
+
pushed part returns too many items rather than too few.
|
|
541
|
+
|
|
542
|
+
A relative date must be resolved before it is handed to another engine: `{ "$ago": "1h" }`
|
|
543
|
+
means nothing to a store, and both halves of a split must be judged against one instant.
|
|
544
|
+
|
|
545
|
+
## Related
|
|
546
|
+
|
|
547
|
+
- [Text syntax](text-syntax.md) — the search-box form, which compiles into this
|
|
548
|
+
- [Library API](api.md) — the TypeScript implementation
|
|
549
|
+
- [Design decisions](../design/decisions.md) — why the language is shaped this way
|