sqllang 0.0.0-stage → 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +282 -2
- package/dist/cjs/index.js +13652 -0
- package/dist/cjs/package.json +3 -0
- package/dist/esm/index.js +13612 -0
- package/dist/esm/index.min.js +6 -0
- package/dist/esm/package.json +3 -0
- package/dist/types/index.d.ts +221 -0
- package/package.json +58 -4
- package/src/aggregates.js +129 -0
- package/src/compile.js +579 -0
- package/src/errors.js +16 -0
- package/src/execute.js +1362 -0
- package/src/functions.js +691 -0
- package/src/grammar.gram +581 -0
- package/src/guard.js +79 -0
- package/src/index.js +78 -0
- package/src/parser.js +5481 -0
- package/src/values.js +196 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Jesse Daniel Mitchell
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,283 @@
|
|
|
1
|
-
#
|
|
1
|
+
# sqllang
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://github.com/TotalTechGeek/sqllang/actions/workflows/ci.yml)
|
|
4
|
+
|
|
5
|
+
**SQL over your own data.** Query arrays of rows, columnar tables and nested
|
|
6
|
+
documents with a familiar SQL dialect — no database, no schema, no loading step.
|
|
7
|
+
|
|
8
|
+
```js
|
|
9
|
+
import { query } from 'sqllang'
|
|
10
|
+
|
|
11
|
+
query(`
|
|
12
|
+
SELECT c.city, COUNT(*) AS orders, SUM(o.total) AS revenue
|
|
13
|
+
FROM orders o
|
|
14
|
+
JOIN customers c ON c.name = o.customer
|
|
15
|
+
WHERE o.meta.src = 'web' -- meta is a JSON string; paths parse it
|
|
16
|
+
GROUP BY c.city
|
|
17
|
+
ORDER BY revenue DESC
|
|
18
|
+
`, { orders, customers })
|
|
19
|
+
// [ { city: 'Lisbon', orders: 2, revenue: 210 }, ... ]
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
It is the SQL sibling of [sheetlang](../formulas): the grammar is a
|
|
23
|
+
[grammatik](../jl-grammar-claude) `.gram` file, every expression compiles to
|
|
24
|
+
JSON Logic, and [json-logic-engine](https://github.com/TotalTechGeek/json-logic-engine)
|
|
25
|
+
runs it. A compiled query is a plain JSON plan you can store and run later.
|
|
26
|
+
|
|
27
|
+
## Install
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
npm install sqllang
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
```js
|
|
34
|
+
import { query } from 'sqllang' // ESM
|
|
35
|
+
const { query } = require('sqllang') // CommonJS
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
TypeScript types are included. The only runtime dependency is `json-logic-engine`.
|
|
39
|
+
Node 18+ or any modern bundler; about 36 KB minified and gzipped, plus the engine.
|
|
40
|
+
|
|
41
|
+
## Data
|
|
42
|
+
|
|
43
|
+
`data` is an object of tables. A table can be:
|
|
44
|
+
|
|
45
|
+
| shape | example |
|
|
46
|
+
| --- | --- |
|
|
47
|
+
| rows | `[{ id: 1, name: 'a' }, { id: 2, name: 'b' }]` |
|
|
48
|
+
| columnar | `{ id: [1, 2], name: ['a', 'b'] }` (every value an array, same length) |
|
|
49
|
+
| header row | `[['id', 'name'], [1, 'a'], [2, 'b']]` |
|
|
50
|
+
| a document | `{ id: 1, items: [...] }` — one row |
|
|
51
|
+
| JSON text | any of the above as a string |
|
|
52
|
+
|
|
53
|
+
A bare array is the table `data`: `query('SELECT * FROM data', rows)`.
|
|
54
|
+
|
|
55
|
+
A table name is a path, so documents nest: `FROM report.sections[0].rows`.
|
|
56
|
+
Keys containing dots work too — `FROM analytics.events` finds the key
|
|
57
|
+
`'analytics.events'` before trying `analytics` → `events`.
|
|
58
|
+
|
|
59
|
+
### Nested values and JSON-in-a-column
|
|
60
|
+
|
|
61
|
+
Wide tables exported from a database usually carry nested data as JSON strings.
|
|
62
|
+
Paths walk straight into them:
|
|
63
|
+
|
|
64
|
+
```sql
|
|
65
|
+
SELECT properties.metadata.dom.id, -- dotted path, parses the JSON text
|
|
66
|
+
properties->>'label', -- Postgres arrows work too
|
|
67
|
+
context.screen.width,
|
|
68
|
+
tags[0], items[*].sku, -- index, or every element
|
|
69
|
+
"column with spaces"
|
|
70
|
+
FROM activity
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
### Lateral joins over nested arrays
|
|
74
|
+
|
|
75
|
+
A FROM item that is a path into an earlier alias ranges over that array:
|
|
76
|
+
|
|
77
|
+
```sql
|
|
78
|
+
SELECT o.id, i.sku, i.qty
|
|
79
|
+
FROM orders o, o.items i -- one row per item
|
|
80
|
+
WHERE i.qty > 1
|
|
81
|
+
|
|
82
|
+
SELECT a.id, anc.id AS ancestor
|
|
83
|
+
FROM activity a, a.properties.metadata.dom.ancestors anc
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Also `UNNEST(expr)`, `GENERATE_SERIES(from, to[, step])` and `JSON_EACH(obj)`
|
|
87
|
+
(rows of `{ key, value }`). An element that is a plain value is the column `value`.
|
|
88
|
+
|
|
89
|
+
### Loading a SQL dump
|
|
90
|
+
|
|
91
|
+
A script can `INSERT` and then query, so a database dump runs as is:
|
|
92
|
+
|
|
93
|
+
```js
|
|
94
|
+
query(readFileSync('dump.sql', 'utf8') + '; SELECT event, COUNT(*) FROM analytics.events GROUP BY 1', {})
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
INSERT never mutates your data; `execute()` returns the tables it built.
|
|
98
|
+
|
|
99
|
+
## The dialect
|
|
100
|
+
|
|
101
|
+
Mostly Postgres-flavoured, with a few SQLite conveniences.
|
|
102
|
+
|
|
103
|
+
- `SELECT [DISTINCT] … FROM … WHERE … GROUP BY … HAVING … ORDER BY … LIMIT … OFFSET …`
|
|
104
|
+
- joins: `[INNER] | LEFT | RIGHT | FULL [OUTER] | CROSS JOIN`, `ON` or `USING (…)`, comma joins.
|
|
105
|
+
Equality conditions run as hash joins.
|
|
106
|
+
- `WITH` (CTEs), subqueries in FROM, scalar / `EXISTS` / `IN (SELECT …)` subqueries, correlated ones too
|
|
107
|
+
- `UNION [ALL]`, `EXCEPT [ALL]`, `INTERSECT [ALL]`, `VALUES (…), (…)`
|
|
108
|
+
- `CASE`, `CAST(x AS type)` and `x::type`, `EXTRACT(part FROM ts)`, `||`, `LIKE`, `ILIKE`,
|
|
109
|
+
`BETWEEN`, `IN`, `IS [NOT] NULL | TRUE | FALSE | DISTINCT FROM`
|
|
110
|
+
- aggregates with `DISTINCT`, `FILTER (WHERE …)` and `ORDER BY` inside:
|
|
111
|
+
`STRING_AGG(name, ', ' ORDER BY name)`
|
|
112
|
+
- window functions: `ROW_NUMBER`, `RANK`, `DENSE_RANK`, `PERCENT_RANK`, `CUME_DIST`, `NTILE`,
|
|
113
|
+
`LAG`, `LEAD`, `FIRST_VALUE`, `LAST_VALUE`, `NTH_VALUE`, and any aggregate `OVER (PARTITION BY … ORDER BY …)`
|
|
114
|
+
(default frame: running to the current row's peers when ordered, the whole partition otherwise)
|
|
115
|
+
- parameters: `?`, `$1`, `:name`
|
|
116
|
+
- comments: `-- …` and `/* … */`; identifiers `"quoted"` or `` `backticked` ``
|
|
117
|
+
|
|
118
|
+
`ORDER BY` and `GROUP BY` accept select-list aliases and positions; `HAVING` accepts aliases.
|
|
119
|
+
|
|
120
|
+
### Semantics worth knowing
|
|
121
|
+
|
|
122
|
+
- **NULL** is SQL NULL: comparisons with it are NULL, `AND`/`OR` are three-valued, and only
|
|
123
|
+
TRUE passes `WHERE`. A missing key is NULL.
|
|
124
|
+
- **Loose typing**, because data is: `1 = '1'` is true and numeric strings sort as numbers
|
|
125
|
+
against numbers. `/` is always real division; division by zero is NULL.
|
|
126
|
+
- **NULLs sort last** ascending and first descending (Postgres); say `NULLS FIRST|LAST` to choose.
|
|
127
|
+
- **Identifiers are case-sensitive** (they are JSON keys). Keywords are not. Table names fall back
|
|
128
|
+
to a case-insensitive match.
|
|
129
|
+
- **Timestamps** are text in, ISO text out. Parsing accepts ISO 8601 and Postgres' output format
|
|
130
|
+
(`2026-06-30 16:33:13.070000 +00:00`). Date functions work in UTC.
|
|
131
|
+
- `SELECT *` over rows of different shapes returns the union of their keys, NULL-filled.
|
|
132
|
+
|
|
133
|
+
### Functions
|
|
134
|
+
|
|
135
|
+
Strings: `LOWER UPPER LENGTH TRIM LTRIM RTRIM SUBSTR LEFT RIGHT REPLACE REVERSE REPEAT LPAD RPAD
|
|
136
|
+
STRPOS SPLIT_PART SPLIT CONCAT CONCAT_WS STARTS_WITH ENDS_WITH REGEXP_LIKE REGEXP_REPLACE REGEXP_EXTRACT`
|
|
137
|
+
|
|
138
|
+
Nulls & logic: `COALESCE IFNULL NVL NULLIF IF IIF GREATEST LEAST`
|
|
139
|
+
|
|
140
|
+
Math: `ABS ROUND FLOOR CEIL TRUNC SIGN SQRT EXP LN LOG LOG10 POWER RANDOM PI`
|
|
141
|
+
|
|
142
|
+
Dates: `NOW CURRENT_DATE DATE TIMESTAMP TO_TIMESTAMP DATE_TRUNC DATE_PART DATE_DIFF DATE_ADD EPOCH_MS STRFTIME`
|
|
143
|
+
|
|
144
|
+
JSON & arrays: `JSON_EXTRACT JSON_VALUE JSON_PARSE TO_JSON JSON_KEYS JSON_TYPE JSON_OBJECT JSON_ARRAY
|
|
145
|
+
ARRAY ARRAY_LENGTH CARDINALITY ARRAY_CONTAINS CONTAINS ARRAY_JOIN ARRAY_DISTINCT ELEMENT_AT`
|
|
146
|
+
|
|
147
|
+
Aggregates: `COUNT SUM TOTAL AVG MIN MAX MEDIAN MODE STDDEV STDDEV_POP VARIANCE VAR_POP
|
|
148
|
+
ARRAY_AGG STRING_AGG GROUP_CONCAT BOOL_AND BOOL_OR COUNT_IF ANY_VALUE FIRST LAST OBJECT_AGG`
|
|
149
|
+
|
|
150
|
+
Add your own:
|
|
151
|
+
|
|
152
|
+
```js
|
|
153
|
+
import { createEngine, query } from 'sqllang'
|
|
154
|
+
const engine = createEngine({ functions: { SLUG: ([s]) => s.toLowerCase().replace(/\W+/g, '-') } })
|
|
155
|
+
query('SELECT SLUG(title) FROM posts', { posts }, { engine })
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
## Running queries you didn't write
|
|
159
|
+
|
|
160
|
+
Some protections are always on:
|
|
161
|
+
|
|
162
|
+
- **No prototype tricks.** Columns, tables and parameters are own keys only, so
|
|
163
|
+
`constructor` is just a missing column. Objects built from data or aliases
|
|
164
|
+
(`AS "__proto__"`, `JSON_OBJECT`, `OBJECT_AGG`, a `__proto__` key in JSON) get
|
|
165
|
+
an ordinary field, never a new prototype.
|
|
166
|
+
- **LIKE cannot hang.** It is matched directly, not through a RegExp, so a
|
|
167
|
+
pattern like `'%a%a%a%a%a%b'` is linear-ish instead of exponential.
|
|
168
|
+
- **Deep nesting is an error**, not a stack overflow.
|
|
169
|
+
- Queries only call the listed functions. They cannot reach JavaScript.
|
|
170
|
+
|
|
171
|
+
Resource limits are opt-in, because only you know what is reasonable:
|
|
172
|
+
|
|
173
|
+
```js
|
|
174
|
+
import { query, SAFE_LIMITS } from 'sqllang'
|
|
175
|
+
|
|
176
|
+
query(userSql, data, { limits: SAFE_LIMITS })
|
|
177
|
+
// { timeoutMs: 1000, maxRows: 1e6, maxStringLength: 1e6, maxQueryLength: 20000, regex: false }
|
|
178
|
+
|
|
179
|
+
query(userSql, data, { limits: { ...SAFE_LIMITS, timeoutMs: 200 } })
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
| limit | stops |
|
|
183
|
+
| --- | --- |
|
|
184
|
+
| `timeoutMs` | slow queries — checked inside joins, filters, grouping, windows and subqueries |
|
|
185
|
+
| `maxRows` | any single step producing too many rows: a cross join, a `GENERATE_SERIES(1, 1e12)` (refused before it is built), the result |
|
|
186
|
+
| `maxStringLength` | `REPEAT`, `LPAD`, `\|\|`, `CONCAT`, `STRING_AGG`, `REPLACE` building huge strings |
|
|
187
|
+
| `maxQueryLength` | oversized SQL text, checked before parsing |
|
|
188
|
+
| `regex: false` | `REGEXP_LIKE` / `REGEXP_REPLACE` / `REGEXP_EXTRACT`. JavaScript cannot interrupt a running regex, so a hostile pattern can hang the process regardless of `timeoutMs`. Turn this off for untrusted SQL. |
|
|
189
|
+
|
|
190
|
+
Exceeding a limit throws `SqlLimitError` (a `SqlError`) with `.limit` naming which one.
|
|
191
|
+
|
|
192
|
+
What the limits don't cover: memory is bounded only indirectly (by `maxRows` and
|
|
193
|
+
`maxStringLength`), a single sort of `maxRows` rows can't be interrupted, and the
|
|
194
|
+
timeout is cooperative, checked every 1024 units of work. For hard isolation of hostile
|
|
195
|
+
input, also run the query in a worker you can terminate.
|
|
196
|
+
|
|
197
|
+
## API
|
|
198
|
+
|
|
199
|
+
| | |
|
|
200
|
+
| --- | --- |
|
|
201
|
+
| `query(sql, data, options?)` | Rows of the last statement, as objects. `rows.columns` lists the columns in order. |
|
|
202
|
+
| `execute(sql, data, options?)` | `{ results, tables }` — every statement's result, and the tables INSERT built. |
|
|
203
|
+
| `prepare(sql, options?)` | Compile once: `{ plan, query(data, params), execute(data, params) }`. |
|
|
204
|
+
| `compile(sql)` | The JSON plan. |
|
|
205
|
+
| `executePlan(plan, data, options?)` | Run a stored plan. |
|
|
206
|
+
| `createEngine({ functions })` | An engine with extra scalar functions. |
|
|
207
|
+
|
|
208
|
+
Options: `params` (array or object), `engine`, `limits` (see above), `interpreted: true` (never generates code —
|
|
209
|
+
for a strict Content-Security-Policy), `tables` (extra tables by name).
|
|
210
|
+
|
|
211
|
+
## What a plan looks like
|
|
212
|
+
|
|
213
|
+
```js
|
|
214
|
+
compile('SELECT id FROM orders WHERE total > 100').statements[0].body
|
|
215
|
+
// {
|
|
216
|
+
// kind: 'select',
|
|
217
|
+
// steps: [{ alias: 'orders', source: { kind: 'table', path: ['orders'] }, join: 'FROM', ... }],
|
|
218
|
+
// where: { GT: [{ val: ['orders', 'total'] }, 100] },
|
|
219
|
+
// columns: [{ kind: 'expr', name: 'id', expr: { val: ['orders', 'id'] } }],
|
|
220
|
+
// ...
|
|
221
|
+
// }
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
Each row is evaluated against `{ <alias>: row, ... }`, so `o.total` is
|
|
225
|
+
`{"val": ["o", "total"]}`. Paths deeper than a column, or into JSON text, are
|
|
226
|
+
`{"PATH": [...]}`.
|
|
227
|
+
|
|
228
|
+
## Performance
|
|
229
|
+
|
|
230
|
+
200k rows, Node 26, Apple silicon, prepared queries, median of 7 runs, against
|
|
231
|
+
alasql 4.22.1 on the same data (it compiles SQL straight to JavaScript):
|
|
232
|
+
|
|
233
|
+
| query | sqllang | alasql |
|
|
234
|
+
| --- | --- | --- |
|
|
235
|
+
| filter + project | 5.9 ms | 5.2 ms |
|
|
236
|
+
| GROUP BY two columns | 20.7 ms | 24.3 ms |
|
|
237
|
+
| hash join + GROUP BY | 36.0 ms | 58.5 ms |
|
|
238
|
+
| nested path + GROUP BY | 12.4 ms | 17.7 ms |
|
|
239
|
+
| ORDER BY … LIMIT 10 | 18.2 ms | 118.2 ms |
|
|
240
|
+
| full ORDER BY | 71.5 ms | 164.7 ms |
|
|
241
|
+
| DISTINCT | 31.0 ms | 52.2 ms |
|
|
242
|
+
| IN (subquery) | 15.5 ms | 26.3 ms |
|
|
243
|
+
| ROW_NUMBER() OVER (PARTITION BY … ORDER BY …) | 63.5 ms | 32.5 ms * |
|
|
244
|
+
|
|
245
|
+
\* alasql ignores PARTITION BY and ORDER BY inside OVER and numbers the rows
|
|
246
|
+
1…n, so it skips the sort that dominates this query; its answer is wrong.
|
|
247
|
+
|
|
248
|
+
What makes it fast, roughly in order of effect:
|
|
249
|
+
|
|
250
|
+
- Operators carry json-logic-engine `compile` hooks, so `.build()` turns
|
|
251
|
+
`{"AND": [{"EQ": …}, {"GT": …}]}` into direct calls instead of method lookups
|
|
252
|
+
with an argument array per call.
|
|
253
|
+
- A single-table query streams rows through one reused context object; with
|
|
254
|
+
GROUP BY, aggregates are fed per row (COUNT/SUM/AVG/MIN/MAX keep running totals)
|
|
255
|
+
and a group keeps only its first row.
|
|
256
|
+
- Groups, DISTINCT, set operations and window partitions use nested Maps keyed by
|
|
257
|
+
the raw values — no key strings built per row.
|
|
258
|
+
- ORDER BY … LIMIT k keeps the best k as rows stream past, and projects a row only
|
|
259
|
+
once it makes the cut.
|
|
260
|
+
- Single-column hash joins and uncorrelated `IN (SELECT …)` look up raw values, and
|
|
261
|
+
consult a numeric side-table only when a cross-type match (`1 = '1'`) is possible.
|
|
262
|
+
- One comparator per ORDER BY, with numbers and strings compared inline, and plain
|
|
263
|
+
subtraction for a single numeric key.
|
|
264
|
+
|
|
265
|
+
`test/fastpaths.test.js` holds each of these to the general path's answers.
|
|
266
|
+
|
|
267
|
+
## Development
|
|
268
|
+
|
|
269
|
+
```
|
|
270
|
+
bun run test # regenerates src/parser.js from src/grammar.gram, then runs the tests
|
|
271
|
+
bun run build # dist/esm, dist/cjs, dist/types (+ a minified ESM build)
|
|
272
|
+
bun run test:types # the declarations, under tsc --strict
|
|
273
|
+
bun run test:dist # the built bundles, loaded through package.json "exports"
|
|
274
|
+
bun run test:csp # runs with code generation disabled, as under a strict CSP
|
|
275
|
+
bun run test:all # all of the above; also what prepublishOnly runs
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Dependencies are managed with [bun](https://bun.sh) (`bun install`); the tests
|
|
279
|
+
themselves run on Node, which is what the package supports.
|
|
280
|
+
|
|
281
|
+
`grammatik` is a build-time dependency only: it generates `src/parser.js`, and
|
|
282
|
+
the small runtime that file needs is bundled into `dist`. The published package
|
|
283
|
+
depends on `json-logic-engine` alone. Types are hand-written in `types/index.d.ts`.
|