holos-wasm-node 0.17.0 → 0.19.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +78 -1
- package/holos_wasm.d.ts +59 -0
- package/holos_wasm.js +107 -0
- package/holos_wasm_bg.wasm +0 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -46,10 +46,28 @@ store.dump('nquads'); // everything the store holds
|
|
|
46
46
|
| What you get back | From |
|
|
47
47
|
|---|---|
|
|
48
48
|
| `true` / `false` | ASK |
|
|
49
|
-
| terms in rdf-js shape — `{termType, value}`, plus `language` and `datatype` on a literal | SELECT |
|
|
49
|
+
| terms in rdf-js shape — `{termType, value}`, plus `language`, `direction` and `datatype` on a literal; and `.variables` on the array | SELECT |
|
|
50
50
|
| N-Triples strings, one per triple, no trailing separator | CONSTRUCT, DESCRIBE |
|
|
51
51
|
| a document in the format you asked for | `queryRdf` |
|
|
52
52
|
| `{inserted, deleted, graphsCreated, graphsDropped}` | `update` |
|
|
53
|
+
| `{pending, result}`, the host fetching what is pending | `queryFederated` |
|
|
54
|
+
| the plan as JSON, with statistics from the run | `explain` |
|
|
55
|
+
|
|
56
|
+
A SELECT result carries **`.variables`**, the projected variables in order, as a property on the
|
|
57
|
+
array — so `result.length` and `for (const row of result)` keep working. The rows cannot carry
|
|
58
|
+
this: an unbound variable is absent from a row, so one unbound in *every* row is invisible, and
|
|
59
|
+
a result with no rows has no columns at all — while `head.vars` in the SPARQL results JSON
|
|
60
|
+
format must list every projected variable regardless. Without it a caller has to parse the
|
|
61
|
+
projection out of the query text.
|
|
62
|
+
|
|
63
|
+
A literal's **`direction`** is its RDF 1.2 base direction, `"ltr"`, `"rtl"` or `""` — the same
|
|
64
|
+
"always present, empty when absent" convention as `language`, because a field that is sometimes
|
|
65
|
+
missing makes every reader check before it can compare.
|
|
66
|
+
|
|
67
|
+
**`explain` evaluates.** The results are drained and discarded before the plan is written,
|
|
68
|
+
because the statistics are gathered as rows flow through the operators — a plan serialised
|
|
69
|
+
before that reports zeroes everywhere. So it costs a full run and tells you what the run did,
|
|
70
|
+
not what the planner intended.
|
|
53
71
|
|
|
54
72
|
**SELECT returns terms, not strings, since 0.17.0.** They used to be N-Triples strings,
|
|
55
73
|
which made every consumer parse term syntax to reach a value, and no RDF library in JS
|
|
@@ -85,11 +103,70 @@ facilities.
|
|
|
85
103
|
| **No file paths** | `std::fs` compiles for wasm32 and fails at runtime. RDF arrives as a string the host has already read, so a load costs memory proportional to the document — a browser tab is not where a 60 GB dump goes. |
|
|
86
104
|
| **No spilling `ORDER BY`** | The external merge sort needs somewhere to put run files. A sort is bounded by memory here, and a large one exhausts the module rather than reaching for disk. |
|
|
87
105
|
| **No timeout, no memory ceiling** | Both are enforced by a watchdog thread sampling a clock, and this target has neither. `QueryOptions::guard` returns `None`, and this crate exposes no way to ask — so nobody can set a limit and believe it is being honoured. |
|
|
106
|
+
| **No network** | There is no HTTP client in here at all. `SERVICE` works, but only because the *host* fetches and hands the answer in — see below. |
|
|
88
107
|
| **One thread** | A load parses and interns in one serial loop rather than overlapping them. Same answer, same order, less throughput. |
|
|
89
108
|
|
|
90
109
|
Bringing the timeout back means a cooperative deadline checked in the row loop, with the
|
|
91
110
|
clock routed through the host. That is the one reduction above that is worth revisiting.
|
|
92
111
|
|
|
112
|
+
## `SERVICE`, without a network client
|
|
113
|
+
|
|
114
|
+
`store.query()` cannot answer a `SERVICE` clause. `store.queryFederated()` can, by asking the
|
|
115
|
+
host to do the fetching:
|
|
116
|
+
|
|
117
|
+
```js
|
|
118
|
+
for (let round = 0; round < 8; round += 1) {
|
|
119
|
+
const pass = store.queryFederated(query, undefined);
|
|
120
|
+
if (!pass.pending.length) return pass.result;
|
|
121
|
+
for (const { endpoint, query: ask } of pass.pending) {
|
|
122
|
+
// The host decides what may be called. Nothing in the wasm can reach the network.
|
|
123
|
+
const res = await fetch(endpoint, {
|
|
124
|
+
method: 'POST',
|
|
125
|
+
headers: { 'Content-Type': 'application/sparql-query',
|
|
126
|
+
Accept: 'application/sparql-results+json' },
|
|
127
|
+
body: ask,
|
|
128
|
+
});
|
|
129
|
+
store.cacheService(endpoint, ask, await res.text());
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
throw new Error('SERVICE did not settle');
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**A pass with anything `pending` has wrong results, and says so by having them.** The
|
|
136
|
+
unanswered `SERVICE` contributed no rows, so the join above it lost rows too. Discard `result`
|
|
137
|
+
whenever `pending` is non-empty rather than showing it.
|
|
138
|
+
|
|
139
|
+
### Why it is shaped like this
|
|
140
|
+
|
|
141
|
+
`spareval::DefaultServiceHandler::handle` is synchronous and `fetch` is not, and you cannot
|
|
142
|
+
await a Promise from synchronous wasm. Three ways round that, and they commit you to different
|
|
143
|
+
things:
|
|
144
|
+
|
|
145
|
+
| | Cost |
|
|
146
|
+
|---|---|
|
|
147
|
+
| synchronous `XMLHttpRequest` | deprecated, warns in every browser, blocks the tab, may stop working |
|
|
148
|
+
| a worker blocking on `Atomics.wait` | needs COOP/COEP cross-origin-isolation headers wherever the app is hosted, and breaks quietly if they are lost |
|
|
149
|
+
| **asking twice** | the query is evaluated more than once |
|
|
150
|
+
|
|
151
|
+
The third is what this does. It buys no deprecated API, no hosting requirement, and an
|
|
152
|
+
allow-list that lives in the host — which is the part that matters. `holos-engine`'s own
|
|
153
|
+
`service` module refuses remote `SERVICE` because on a *server* it is an SSRF primitive: a
|
|
154
|
+
query from a stranger makes the server fetch an address of the stranger's choosing. That
|
|
155
|
+
objection does not transfer here, and not because a browser is safer — because **this crate
|
|
156
|
+
cannot make a request at all**. There is nothing to point anywhere. The fetch is the host's, in
|
|
157
|
+
the user's own browser, with the user's own network position and CORS between them.
|
|
158
|
+
|
|
159
|
+
### More than two rounds is normal
|
|
160
|
+
|
|
161
|
+
A `SERVICE` whose pattern carries bindings from an earlier join only takes its final shape once
|
|
162
|
+
that join has rows, and the first pass gives it none — so an answer can reveal a request that
|
|
163
|
+
could not have been seen before. The same semi-naive iteration the rules engine uses. The cap
|
|
164
|
+
belongs to the host, because only the host knows how long it will wait.
|
|
165
|
+
|
|
166
|
+
The cache key is `(endpoint, query)` and the query is byte-exact: a reformatted query is a
|
|
167
|
+
different question and will be asked again. That is deliberate — matching a near-miss would
|
|
168
|
+
serve one endpoint's answer to another's question.
|
|
169
|
+
|
|
93
170
|
## Four things had to be cfg-ed for this target
|
|
94
171
|
|
|
95
172
|
The compile-time surface was almost nothing — one getrandom error. The work was runtime:
|
package/holos_wasm.d.ts
CHANGED
|
@@ -7,6 +7,15 @@
|
|
|
7
7
|
export class Store {
|
|
8
8
|
free(): void;
|
|
9
9
|
[Symbol.dispose](): void;
|
|
10
|
+
/**
|
|
11
|
+
* Give the store an endpoint's answer to one of `queryFederated`'s pending queries.
|
|
12
|
+
*
|
|
13
|
+
* `results` is SPARQL Results JSON, which is what an endpoint returns for
|
|
14
|
+
* `Accept: application/sparql-results+json`. `query` must be the string the pending entry
|
|
15
|
+
* carried, byte for byte: it is the cache key, so a reformatted query is a different
|
|
16
|
+
* question and will be asked again.
|
|
17
|
+
*/
|
|
18
|
+
cacheService(endpoint: string, query: string, results: string): void;
|
|
10
19
|
/**
|
|
11
20
|
* Everything the store holds, serialised in `format`.
|
|
12
21
|
*
|
|
@@ -21,6 +30,23 @@ export class Store {
|
|
|
21
30
|
* and every JS RDF library can parse what comes back.
|
|
22
31
|
*/
|
|
23
32
|
dump(format: string): string;
|
|
33
|
+
/**
|
|
34
|
+
* The query plan, as JSON, with the statistics from actually running it.
|
|
35
|
+
*
|
|
36
|
+
* HOLOS has had this since the CLI's `--explain`; it was simply not reachable from here,
|
|
37
|
+
* so a browser could run a query far faster than a server and still not say how.
|
|
38
|
+
*
|
|
39
|
+
* **It evaluates.** The results are drained and discarded, because an explanation
|
|
40
|
+
* serialised before the results are consumed reports zeroes for every operator — the
|
|
41
|
+
* statistics are gathered as rows flow through it. So this costs a full run and answers
|
|
42
|
+
* what that run did, rather than what the planner intended.
|
|
43
|
+
*
|
|
44
|
+
* # Errors
|
|
45
|
+
*
|
|
46
|
+
* As `query`, plus the case where the engine returns no explanation at all, which would
|
|
47
|
+
* mean the request for one was dropped on the way through.
|
|
48
|
+
*/
|
|
49
|
+
explain(query: string, base?: string | null): string;
|
|
24
50
|
/**
|
|
25
51
|
* Load Turtle, which is the common case.
|
|
26
52
|
*/
|
|
@@ -36,6 +62,39 @@ export class Store {
|
|
|
36
62
|
* A new, empty store.
|
|
37
63
|
*/
|
|
38
64
|
constructor();
|
|
65
|
+
/**
|
|
66
|
+
* Run a query that may contain `SERVICE`, one pass at a time.
|
|
67
|
+
*
|
|
68
|
+
* Returns `{pending: [{endpoint, query}, …]}` when the engine asked for an endpoint this
|
|
69
|
+
* store has no answer for, and `{result: …}` -- shaped exactly as `query` returns -- when
|
|
70
|
+
* it did not. The host fetches each pending query, hands it back with `cacheService`, and
|
|
71
|
+
* calls this again:
|
|
72
|
+
*
|
|
73
|
+
* ```js
|
|
74
|
+
* for (let round = 0; round < 8; round += 1) {
|
|
75
|
+
* const pass = store.queryFederated(q, undefined);
|
|
76
|
+
* if (!pass.pending.length) return pass.result;
|
|
77
|
+
* for (const { endpoint, query } of pass.pending) {
|
|
78
|
+
* // The host decides what may be called. Nothing in the wasm can reach the network.
|
|
79
|
+
* store.cacheService(endpoint, query, await fetchSparqlJson(endpoint, query));
|
|
80
|
+
* }
|
|
81
|
+
* }
|
|
82
|
+
* throw new Error('SERVICE did not settle');
|
|
83
|
+
* ```
|
|
84
|
+
*
|
|
85
|
+
* **A pass with anything pending has wrong results, and says so by having them.** The
|
|
86
|
+
* unanswered `SERVICE` contributed no rows, so the join above it lost rows too. Discard
|
|
87
|
+
* `result` whenever `pending` is non-empty rather than showing it.
|
|
88
|
+
*
|
|
89
|
+
* More than two rounds are possible and not a fault: a `SERVICE` whose pattern carries
|
|
90
|
+
* bindings from an earlier join only takes its final shape once that join has rows, so an
|
|
91
|
+
* answer can reveal a request that could not have been seen before. The cap belongs to the
|
|
92
|
+
* host because only the host knows how long it is prepared to wait.
|
|
93
|
+
*
|
|
94
|
+
* A query with no `SERVICE` never consults the handler, so `pending` is empty and this is
|
|
95
|
+
* `query` with one more allocation.
|
|
96
|
+
*/
|
|
97
|
+
queryFederated(query: string, base?: string | null): any;
|
|
39
98
|
/**
|
|
40
99
|
* Run a CONSTRUCT or DESCRIBE and return the graph serialised in `format`.
|
|
41
100
|
*
|
package/holos_wasm.js
CHANGED
|
@@ -14,6 +14,26 @@ class Store {
|
|
|
14
14
|
const ptr = this.__destroy_into_raw();
|
|
15
15
|
wasm.__wbg_store_free(ptr, 0);
|
|
16
16
|
}
|
|
17
|
+
/**
|
|
18
|
+
* Give the store an endpoint's answer to one of `queryFederated`'s pending queries.
|
|
19
|
+
*
|
|
20
|
+
* `results` is SPARQL Results JSON, which is what an endpoint returns for
|
|
21
|
+
* `Accept: application/sparql-results+json`. `query` must be the string the pending entry
|
|
22
|
+
* carried, byte for byte: it is the cache key, so a reformatted query is a different
|
|
23
|
+
* question and will be asked again.
|
|
24
|
+
* @param {string} endpoint
|
|
25
|
+
* @param {string} query
|
|
26
|
+
* @param {string} results
|
|
27
|
+
*/
|
|
28
|
+
cacheService(endpoint, query, results) {
|
|
29
|
+
const ptr0 = passStringToWasm0(endpoint, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
30
|
+
const len0 = WASM_VECTOR_LEN;
|
|
31
|
+
const ptr1 = passStringToWasm0(query, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
32
|
+
const len1 = WASM_VECTOR_LEN;
|
|
33
|
+
const ptr2 = passStringToWasm0(results, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
34
|
+
const len2 = WASM_VECTOR_LEN;
|
|
35
|
+
wasm.store_cacheService(this.__wbg_ptr, ptr0, len0, ptr1, len1, ptr2, len2);
|
|
36
|
+
}
|
|
17
37
|
/**
|
|
18
38
|
* Distinct terms in the dictionary.
|
|
19
39
|
*
|
|
@@ -60,6 +80,47 @@ class Store {
|
|
|
60
80
|
wasm.__wbindgen_free(deferred3_0, deferred3_1, 1);
|
|
61
81
|
}
|
|
62
82
|
}
|
|
83
|
+
/**
|
|
84
|
+
* The query plan, as JSON, with the statistics from actually running it.
|
|
85
|
+
*
|
|
86
|
+
* HOLOS has had this since the CLI's `--explain`; it was simply not reachable from here,
|
|
87
|
+
* so a browser could run a query far faster than a server and still not say how.
|
|
88
|
+
*
|
|
89
|
+
* **It evaluates.** The results are drained and discarded, because an explanation
|
|
90
|
+
* serialised before the results are consumed reports zeroes for every operator — the
|
|
91
|
+
* statistics are gathered as rows flow through it. So this costs a full run and answers
|
|
92
|
+
* what that run did, rather than what the planner intended.
|
|
93
|
+
*
|
|
94
|
+
* # Errors
|
|
95
|
+
*
|
|
96
|
+
* As `query`, plus the case where the engine returns no explanation at all, which would
|
|
97
|
+
* mean the request for one was dropped on the way through.
|
|
98
|
+
* @param {string} query
|
|
99
|
+
* @param {string | null} [base]
|
|
100
|
+
* @returns {string}
|
|
101
|
+
*/
|
|
102
|
+
explain(query, base) {
|
|
103
|
+
let deferred4_0;
|
|
104
|
+
let deferred4_1;
|
|
105
|
+
try {
|
|
106
|
+
const ptr0 = passStringToWasm0(query, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
107
|
+
const len0 = WASM_VECTOR_LEN;
|
|
108
|
+
var ptr1 = isLikeNone(base) ? 0 : passStringToWasm0(base, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
109
|
+
var len1 = WASM_VECTOR_LEN;
|
|
110
|
+
const ret = wasm.store_explain(this.__wbg_ptr, ptr0, len0, ptr1, len1);
|
|
111
|
+
var ptr3 = ret[0];
|
|
112
|
+
var len3 = ret[1];
|
|
113
|
+
if (ret[3]) {
|
|
114
|
+
ptr3 = 0; len3 = 0;
|
|
115
|
+
throw takeFromExternrefTable0(ret[2]);
|
|
116
|
+
}
|
|
117
|
+
deferred4_0 = ptr3;
|
|
118
|
+
deferred4_1 = len3;
|
|
119
|
+
return getStringFromWasm0(ptr3, len3);
|
|
120
|
+
} finally {
|
|
121
|
+
wasm.__wbindgen_free(deferred4_0, deferred4_1, 1);
|
|
122
|
+
}
|
|
123
|
+
}
|
|
63
124
|
/**
|
|
64
125
|
* Load Turtle, which is the common case.
|
|
65
126
|
* @param {string} text
|
|
@@ -109,6 +170,52 @@ class Store {
|
|
|
109
170
|
StoreFinalization.register(this, this.__wbg_ptr, this);
|
|
110
171
|
return this;
|
|
111
172
|
}
|
|
173
|
+
/**
|
|
174
|
+
* Run a query that may contain `SERVICE`, one pass at a time.
|
|
175
|
+
*
|
|
176
|
+
* Returns `{pending: [{endpoint, query}, …]}` when the engine asked for an endpoint this
|
|
177
|
+
* store has no answer for, and `{result: …}` -- shaped exactly as `query` returns -- when
|
|
178
|
+
* it did not. The host fetches each pending query, hands it back with `cacheService`, and
|
|
179
|
+
* calls this again:
|
|
180
|
+
*
|
|
181
|
+
* ```js
|
|
182
|
+
* for (let round = 0; round < 8; round += 1) {
|
|
183
|
+
* const pass = store.queryFederated(q, undefined);
|
|
184
|
+
* if (!pass.pending.length) return pass.result;
|
|
185
|
+
* for (const { endpoint, query } of pass.pending) {
|
|
186
|
+
* // The host decides what may be called. Nothing in the wasm can reach the network.
|
|
187
|
+
* store.cacheService(endpoint, query, await fetchSparqlJson(endpoint, query));
|
|
188
|
+
* }
|
|
189
|
+
* }
|
|
190
|
+
* throw new Error('SERVICE did not settle');
|
|
191
|
+
* ```
|
|
192
|
+
*
|
|
193
|
+
* **A pass with anything pending has wrong results, and says so by having them.** The
|
|
194
|
+
* unanswered `SERVICE` contributed no rows, so the join above it lost rows too. Discard
|
|
195
|
+
* `result` whenever `pending` is non-empty rather than showing it.
|
|
196
|
+
*
|
|
197
|
+
* More than two rounds are possible and not a fault: a `SERVICE` whose pattern carries
|
|
198
|
+
* bindings from an earlier join only takes its final shape once that join has rows, so an
|
|
199
|
+
* answer can reveal a request that could not have been seen before. The cap belongs to the
|
|
200
|
+
* host because only the host knows how long it is prepared to wait.
|
|
201
|
+
*
|
|
202
|
+
* A query with no `SERVICE` never consults the handler, so `pending` is empty and this is
|
|
203
|
+
* `query` with one more allocation.
|
|
204
|
+
* @param {string} query
|
|
205
|
+
* @param {string | null} [base]
|
|
206
|
+
* @returns {any}
|
|
207
|
+
*/
|
|
208
|
+
queryFederated(query, base) {
|
|
209
|
+
const ptr0 = passStringToWasm0(query, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
210
|
+
const len0 = WASM_VECTOR_LEN;
|
|
211
|
+
var ptr1 = isLikeNone(base) ? 0 : passStringToWasm0(base, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
|
|
212
|
+
var len1 = WASM_VECTOR_LEN;
|
|
213
|
+
const ret = wasm.store_queryFederated(this.__wbg_ptr, ptr0, len0, ptr1, len1);
|
|
214
|
+
if (ret[2]) {
|
|
215
|
+
throw takeFromExternrefTable0(ret[1]);
|
|
216
|
+
}
|
|
217
|
+
return takeFromExternrefTable0(ret[0]);
|
|
218
|
+
}
|
|
112
219
|
/**
|
|
113
220
|
* Run a CONSTRUCT or DESCRIBE and return the graph serialised in `format`.
|
|
114
221
|
*
|
package/holos_wasm_bg.wasm
CHANGED
|
Binary file
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "holos-wasm-node",
|
|
3
3
|
"description": "WebAssembly bindings for the HOLOS RDF 1.2 store and SPARQL 1.2 engine (CommonJS, for Node and require())",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.19.0",
|
|
5
5
|
"license": "MIT OR Apache-2.0",
|
|
6
6
|
"repository": {
|
|
7
7
|
"type": "git",
|