sqllang 0.0.0-stage → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Jesse Daniel Mitchell
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,283 @@
1
- # Temporary Holding Version
1
+ # sqllang
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ [![CI](https://github.com/TotalTechGeek/sqllang/actions/workflows/ci.yml/badge.svg)](https://github.com/TotalTechGeek/sqllang/actions/workflows/ci.yml)
4
+
5
+ **SQL over your own data.** Query arrays of rows, columnar tables and nested
6
+ documents with a familiar SQL dialect — no database, no schema, no loading step.
7
+
8
+ ```js
9
+ import { query } from 'sqllang'
10
+
11
+ query(`
12
+ SELECT c.city, COUNT(*) AS orders, SUM(o.total) AS revenue
13
+ FROM orders o
14
+ JOIN customers c ON c.name = o.customer
15
+ WHERE o.meta.src = 'web' -- meta is a JSON string; paths parse it
16
+ GROUP BY c.city
17
+ ORDER BY revenue DESC
18
+ `, { orders, customers })
19
+ // [ { city: 'Lisbon', orders: 2, revenue: 210 }, ... ]
20
+ ```
21
+
22
+ It is the SQL sibling of [sheetlang](../formulas): the grammar is a
23
+ [grammatik](../jl-grammar-claude) `.gram` file, every expression compiles to
24
+ JSON Logic, and [json-logic-engine](https://github.com/TotalTechGeek/json-logic-engine)
25
+ runs it. A compiled query is a plain JSON plan you can store and run later.
26
+
27
+ ## Install
28
+
29
+ ```bash
30
+ npm install sqllang
31
+ ```
32
+
33
+ ```js
34
+ import { query } from 'sqllang' // ESM
35
+ const { query } = require('sqllang') // CommonJS
36
+ ```
37
+
38
+ TypeScript types are included. The only runtime dependency is `json-logic-engine`.
39
+ Node 18+ or any modern bundler; about 36 KB minified and gzipped, plus the engine.
40
+
41
+ ## Data
42
+
43
+ `data` is an object of tables. A table can be:
44
+
45
+ | shape | example |
46
+ | --- | --- |
47
+ | rows | `[{ id: 1, name: 'a' }, { id: 2, name: 'b' }]` |
48
+ | columnar | `{ id: [1, 2], name: ['a', 'b'] }` (every value an array, same length) |
49
+ | header row | `[['id', 'name'], [1, 'a'], [2, 'b']]` |
50
+ | a document | `{ id: 1, items: [...] }` — one row |
51
+ | JSON text | any of the above as a string |
52
+
53
+ A bare array is the table `data`: `query('SELECT * FROM data', rows)`.
54
+
55
+ A table name is a path, so documents nest: `FROM report.sections[0].rows`.
56
+ Keys containing dots work too — `FROM analytics.events` finds the key
57
+ `'analytics.events'` before trying `analytics` → `events`.
58
+
59
+ ### Nested values and JSON-in-a-column
60
+
61
+ Wide tables exported from a database usually carry nested data as JSON strings.
62
+ Paths walk straight into them:
63
+
64
+ ```sql
65
+ SELECT properties.metadata.dom.id, -- dotted path, parses the JSON text
66
+ properties->>'label', -- Postgres arrows work too
67
+ context.screen.width,
68
+ tags[0], items[*].sku, -- index, or every element
69
+ "column with spaces"
70
+ FROM activity
71
+ ```
72
+
73
+ ### Lateral joins over nested arrays
74
+
75
+ A FROM item that is a path into an earlier alias ranges over that array:
76
+
77
+ ```sql
78
+ SELECT o.id, i.sku, i.qty
79
+ FROM orders o, o.items i -- one row per item
80
+ WHERE i.qty > 1
81
+
82
+ SELECT a.id, anc.id AS ancestor
83
+ FROM activity a, a.properties.metadata.dom.ancestors anc
84
+ ```
85
+
86
+ Also `UNNEST(expr)`, `GENERATE_SERIES(from, to[, step])` and `JSON_EACH(obj)`
87
+ (rows of `{ key, value }`). An element that is a plain value is the column `value`.
88
+
89
+ ### Loading a SQL dump
90
+
91
+ A script can `INSERT` and then query, so a database dump runs as is:
92
+
93
+ ```js
94
+ query(readFileSync('dump.sql', 'utf8') + '; SELECT event, COUNT(*) FROM analytics.events GROUP BY 1', {})
95
+ ```
96
+
97
+ INSERT never mutates your data; `execute()` returns the tables it built.
98
+
99
+ ## The dialect
100
+
101
+ Mostly Postgres-flavoured, with a few SQLite conveniences.
102
+
103
+ - `SELECT [DISTINCT] … FROM … WHERE … GROUP BY … HAVING … ORDER BY … LIMIT … OFFSET …`
104
+ - joins: `[INNER] | LEFT | RIGHT | FULL [OUTER] | CROSS JOIN`, `ON` or `USING (…)`, comma joins.
105
+ Equality conditions run as hash joins.
106
+ - `WITH` (CTEs), subqueries in FROM, scalar / `EXISTS` / `IN (SELECT …)` subqueries, correlated ones too
107
+ - `UNION [ALL]`, `EXCEPT [ALL]`, `INTERSECT [ALL]`, `VALUES (…), (…)`
108
+ - `CASE`, `CAST(x AS type)` and `x::type`, `EXTRACT(part FROM ts)`, `||`, `LIKE`, `ILIKE`,
109
+ `BETWEEN`, `IN`, `IS [NOT] NULL | TRUE | FALSE | DISTINCT FROM`
110
+ - aggregates with `DISTINCT`, `FILTER (WHERE …)` and `ORDER BY` inside:
111
+ `STRING_AGG(name, ', ' ORDER BY name)`
112
+ - window functions: `ROW_NUMBER`, `RANK`, `DENSE_RANK`, `PERCENT_RANK`, `CUME_DIST`, `NTILE`,
113
+ `LAG`, `LEAD`, `FIRST_VALUE`, `LAST_VALUE`, `NTH_VALUE`, and any aggregate `OVER (PARTITION BY … ORDER BY …)`
114
+ (default frame: running to the current row's peers when ordered, the whole partition otherwise)
115
+ - parameters: `?`, `$1`, `:name`
116
+ - comments: `-- …` and `/* … */`; identifiers `"quoted"` or `` `backticked` ``
117
+
118
+ `ORDER BY` and `GROUP BY` accept select-list aliases and positions; `HAVING` accepts aliases.
119
+
120
+ ### Semantics worth knowing
121
+
122
+ - **NULL** is SQL NULL: comparisons with it are NULL, `AND`/`OR` are three-valued, and only
123
+ TRUE passes `WHERE`. A missing key is NULL.
124
+ - **Loose typing**, because data is: `1 = '1'` is true and numeric strings sort as numbers
125
+ against numbers. `/` is always real division; division by zero is NULL.
126
+ - **NULLs sort last** ascending and first descending (Postgres); say `NULLS FIRST|LAST` to choose.
127
+ - **Identifiers are case-sensitive** (they are JSON keys). Keywords are not. Table names fall back
128
+ to a case-insensitive match.
129
+ - **Timestamps** are text in, ISO text out. Parsing accepts ISO 8601 and Postgres' output format
130
+ (`2026-06-30 16:33:13.070000 +00:00`). Date functions work in UTC.
131
+ - `SELECT *` over rows of different shapes returns the union of their keys, NULL-filled.
132
+
133
+ ### Functions
134
+
135
+ Strings: `LOWER UPPER LENGTH TRIM LTRIM RTRIM SUBSTR LEFT RIGHT REPLACE REVERSE REPEAT LPAD RPAD
136
+ STRPOS SPLIT_PART SPLIT CONCAT CONCAT_WS STARTS_WITH ENDS_WITH REGEXP_LIKE REGEXP_REPLACE REGEXP_EXTRACT`
137
+
138
+ Nulls & logic: `COALESCE IFNULL NVL NULLIF IF IIF GREATEST LEAST`
139
+
140
+ Math: `ABS ROUND FLOOR CEIL TRUNC SIGN SQRT EXP LN LOG LOG10 POWER RANDOM PI`
141
+
142
+ Dates: `NOW CURRENT_DATE DATE TIMESTAMP TO_TIMESTAMP DATE_TRUNC DATE_PART DATE_DIFF DATE_ADD EPOCH_MS STRFTIME`
143
+
144
+ JSON & arrays: `JSON_EXTRACT JSON_VALUE JSON_PARSE TO_JSON JSON_KEYS JSON_TYPE JSON_OBJECT JSON_ARRAY
145
+ ARRAY ARRAY_LENGTH CARDINALITY ARRAY_CONTAINS CONTAINS ARRAY_JOIN ARRAY_DISTINCT ELEMENT_AT`
146
+
147
+ Aggregates: `COUNT SUM TOTAL AVG MIN MAX MEDIAN MODE STDDEV STDDEV_POP VARIANCE VAR_POP
148
+ ARRAY_AGG STRING_AGG GROUP_CONCAT BOOL_AND BOOL_OR COUNT_IF ANY_VALUE FIRST LAST OBJECT_AGG`
149
+
150
+ Add your own:
151
+
152
+ ```js
153
+ import { createEngine, query } from 'sqllang'
154
+ const engine = createEngine({ functions: { SLUG: ([s]) => s.toLowerCase().replace(/\W+/g, '-') } })
155
+ query('SELECT SLUG(title) FROM posts', { posts }, { engine })
156
+ ```
157
+
158
+ ## Running queries you didn't write
159
+
160
+ Some protections are always on:
161
+
162
+ - **No prototype tricks.** Columns, tables and parameters are own keys only, so
163
+ `constructor` is just a missing column. Objects built from data or aliases
164
+ (`AS "__proto__"`, `JSON_OBJECT`, `OBJECT_AGG`, a `__proto__` key in JSON) get
165
+ an ordinary field, never a new prototype.
166
+ - **LIKE cannot hang.** It is matched directly, not through a RegExp, so a
167
+ pattern like `'%a%a%a%a%a%b'` is linear-ish instead of exponential.
168
+ - **Deep nesting is an error**, not a stack overflow.
169
+ - Queries only call the listed functions. They cannot reach JavaScript.
170
+
171
+ Resource limits are opt-in, because only you know what is reasonable:
172
+
173
+ ```js
174
+ import { query, SAFE_LIMITS } from 'sqllang'
175
+
176
+ query(userSql, data, { limits: SAFE_LIMITS })
177
+ // { timeoutMs: 1000, maxRows: 1e6, maxStringLength: 1e6, maxQueryLength: 20000, regex: false }
178
+
179
+ query(userSql, data, { limits: { ...SAFE_LIMITS, timeoutMs: 200 } })
180
+ ```
181
+
182
+ | limit | stops |
183
+ | --- | --- |
184
+ | `timeoutMs` | slow queries — checked inside joins, filters, grouping, windows and subqueries |
185
+ | `maxRows` | any single step producing too many rows: a cross join, a `GENERATE_SERIES(1, 1e12)` (refused before it is built), the result |
186
+ | `maxStringLength` | `REPEAT`, `LPAD`, `\|\|`, `CONCAT`, `STRING_AGG`, `REPLACE` building huge strings |
187
+ | `maxQueryLength` | oversized SQL text, checked before parsing |
188
+ | `regex: false` | `REGEXP_LIKE` / `REGEXP_REPLACE` / `REGEXP_EXTRACT`. JavaScript cannot interrupt a running regex, so a hostile pattern can hang the process regardless of `timeoutMs`. Turn this off for untrusted SQL. |
189
+
190
+ Exceeding a limit throws `SqlLimitError` (a `SqlError`) with `.limit` naming which one.
191
+
192
+ What the limits don't cover: memory is bounded only indirectly (by `maxRows` and
193
+ `maxStringLength`), a single sort of `maxRows` rows can't be interrupted, and the
194
+ timeout is cooperative, checked every 1024 units of work. For hard isolation of hostile
195
+ input, also run the query in a worker you can terminate.
196
+
197
+ ## API
198
+
199
+ | | |
200
+ | --- | --- |
201
+ | `query(sql, data, options?)` | Rows of the last statement, as objects. `rows.columns` lists the columns in order. |
202
+ | `execute(sql, data, options?)` | `{ results, tables }` — every statement's result, and the tables INSERT built. |
203
+ | `prepare(sql, options?)` | Compile once: `{ plan, query(data, params), execute(data, params) }`. |
204
+ | `compile(sql)` | The JSON plan. |
205
+ | `executePlan(plan, data, options?)` | Run a stored plan. |
206
+ | `createEngine({ functions })` | An engine with extra scalar functions. |
207
+
208
+ Options: `params` (array or object), `engine`, `limits` (see above), `interpreted: true` (never generates code —
209
+ for a strict Content-Security-Policy), `tables` (extra tables by name).
210
+
211
+ ## What a plan looks like
212
+
213
+ ```js
214
+ compile('SELECT id FROM orders WHERE total > 100').statements[0].body
215
+ // {
216
+ // kind: 'select',
217
+ // steps: [{ alias: 'orders', source: { kind: 'table', path: ['orders'] }, join: 'FROM', ... }],
218
+ // where: { GT: [{ val: ['orders', 'total'] }, 100] },
219
+ // columns: [{ kind: 'expr', name: 'id', expr: { val: ['orders', 'id'] } }],
220
+ // ...
221
+ // }
222
+ ```
223
+
224
+ Each row is evaluated against `{ <alias>: row, ... }`, so `o.total` is
225
+ `{"val": ["o", "total"]}`. Paths deeper than a column, or into JSON text, are
226
+ `{"PATH": [...]}`.
227
+
228
+ ## Performance
229
+
230
+ 200k rows, Node 26, Apple silicon, prepared queries, median of 7 runs, against
231
+ alasql 4.22.1 on the same data (it compiles SQL straight to JavaScript):
232
+
233
+ | query | sqllang | alasql |
234
+ | --- | --- | --- |
235
+ | filter + project | 5.9 ms | 5.2 ms |
236
+ | GROUP BY two columns | 20.7 ms | 24.3 ms |
237
+ | hash join + GROUP BY | 36.0 ms | 58.5 ms |
238
+ | nested path + GROUP BY | 12.4 ms | 17.7 ms |
239
+ | ORDER BY … LIMIT 10 | 18.2 ms | 118.2 ms |
240
+ | full ORDER BY | 71.5 ms | 164.7 ms |
241
+ | DISTINCT | 31.0 ms | 52.2 ms |
242
+ | IN (subquery) | 15.5 ms | 26.3 ms |
243
+ | ROW_NUMBER() OVER (PARTITION BY … ORDER BY …) | 63.5 ms | 32.5 ms * |
244
+
245
+ \* alasql ignores PARTITION BY and ORDER BY inside OVER and numbers the rows
246
+ 1…n, so it skips the sort that dominates this query; its answer is wrong.
247
+
248
+ What makes it fast, roughly in order of effect:
249
+
250
+ - Operators carry json-logic-engine `compile` hooks, so `.build()` turns
251
+ `{"AND": [{"EQ": …}, {"GT": …}]}` into direct calls instead of method lookups
252
+ with an argument array per call.
253
+ - A single-table query streams rows through one reused context object; with
254
+ GROUP BY, aggregates are fed per row (COUNT/SUM/AVG/MIN/MAX keep running totals)
255
+ and a group keeps only its first row.
256
+ - Groups, DISTINCT, set operations and window partitions use nested Maps keyed by
257
+ the raw values — no key strings built per row.
258
+ - ORDER BY … LIMIT k keeps the best k as rows stream past, and projects a row only
259
+ once it makes the cut.
260
+ - Single-column hash joins and uncorrelated `IN (SELECT …)` look up raw values, and
261
+ consult a numeric side-table only when a cross-type match (`1 = '1'`) is possible.
262
+ - One comparator per ORDER BY, with numbers and strings compared inline, and plain
263
+ subtraction for a single numeric key.
264
+
265
+ `test/fastpaths.test.js` holds each of these to the general path's answers.
266
+
267
+ ## Development
268
+
269
+ ```
270
+ bun run test # regenerates src/parser.js from src/grammar.gram, then runs the tests
271
+ bun run build # dist/esm, dist/cjs, dist/types (+ a minified ESM build)
272
+ bun run test:types # the declarations, under tsc --strict
273
+ bun run test:dist # the built bundles, loaded through package.json "exports"
274
+ bun run test:csp # runs with code generation disabled, as under a strict CSP
275
+ bun run test:all # all of the above; also what prepublishOnly runs
276
+ ```
277
+
278
+ Dependencies are managed with [bun](https://bun.sh) (`bun install`); the tests
279
+ themselves run on Node, which is what the package supports.
280
+
281
+ `grammatik` is a build-time dependency only: it generates `src/parser.js`, and
282
+ the small runtime that file needs is bundled into `dist`. The published package
283
+ depends on `json-logic-engine` alone. Types are hand-written in `types/index.d.ts`.