@rhydb/rhydb-wasm 0.14.1 → 0.14.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -32
- package/dist/rhydb_wasm.js +1 -1
- package/dist/rhydb_wasm.wasm +0 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,16 +1,16 @@
|
|
|
1
|
-
#
|
|
1
|
+
# RhyDB WASM
|
|
2
2
|
|
|
3
|
-
This directory builds a browser-usable version of
|
|
3
|
+
This directory builds a browser-usable version of RhyDB using WebAssembly (WASM). It uses Emscripten to compile the C++ sources to a `.wasm` binary and generates a small JavaScript loader that lets browser code call selected C++ functions.
|
|
4
4
|
|
|
5
|
-
`CMakeLists.txt` collects
|
|
5
|
+
`CMakeLists.txt` collects RhyDB core sources, excludes the CLI, HTTP API, tests, and benchmark code, and links the dependencies needed for preprocessing and SaneQL query execution. `src/rhydb_wasm.cpp` is the boundary between JavaScript and C++.
|
|
6
6
|
|
|
7
7
|
## What It Exposes
|
|
8
8
|
|
|
9
9
|
The generated JavaScript module exports these embind functions:
|
|
10
10
|
|
|
11
|
-
- `preprocess(preprocessingConfigPath)`: reads a
|
|
12
|
-
- `save(handle, outputDirectory)`: writes the in-memory database as
|
|
13
|
-
- `load(stateDirectory)`: loads a saved
|
|
11
|
+
- `preprocess(preprocessingConfigPath)`: reads a RhyDB preprocessing config from Emscripten's virtual filesystem and returns a database handle.
|
|
12
|
+
- `save(handle, outputDirectory)`: writes the in-memory database as RhyDB's normal processed-state files into the virtual filesystem.
|
|
13
|
+
- `load(stateDirectory)`: loads a saved RhyDB processed state from the virtual filesystem and returns a database handle.
|
|
14
14
|
- `query(handle, saneqlQuery)`: runs SaneQL and returns the result as NDJSON.
|
|
15
15
|
- `info(handle)`: returns database information as JSON.
|
|
16
16
|
- `dispose(handle)`: releases a database handle.
|
|
@@ -19,15 +19,15 @@ The module also exposes Emscripten's `FS` runtime API. Browser code uses `FS` to
|
|
|
19
19
|
|
|
20
20
|
The WASM build supports raw and `.zst` preprocessing inputs. The native `.xz` input path is not included in the WASM target.
|
|
21
21
|
|
|
22
|
-
Browser queries use a smaller materialization cutoff than native
|
|
22
|
+
Browser queries use a smaller materialization cutoff than native RhyDB to keep intermediate batches within browser memory limits, especially when projecting full nucleotide sequences.
|
|
23
23
|
|
|
24
24
|
## Artifacts
|
|
25
25
|
|
|
26
|
-
Building the WASM target creates these files relative to the
|
|
26
|
+
Building the WASM target creates these files relative to the RhyDB repository root:
|
|
27
27
|
|
|
28
28
|
- `wasm/dist/rhydb_wasm.js`: ES module loader generated by Emscripten. Import this from browser code.
|
|
29
29
|
- `wasm/dist/rhydb_wasm.d.ts`: Autogenerated type declarations.
|
|
30
|
-
- `wasm/dist/rhydb_wasm.wasm`: compiled
|
|
30
|
+
- `wasm/dist/rhydb_wasm.wasm`: compiled RhyDB C++ code and linked dependencies.
|
|
31
31
|
|
|
32
32
|
All files must be served by the web app. The JavaScript loader fetches the `.wasm` file at runtime.
|
|
33
33
|
|
|
@@ -42,21 +42,21 @@ npm install @rhydb/rhydb-wasm
|
|
|
42
42
|
The package is an ES module. Import the loader as a default import:
|
|
43
43
|
|
|
44
44
|
```js
|
|
45
|
-
import
|
|
45
|
+
import createRhydbModule from "@rhydb/rhydb-wasm";
|
|
46
46
|
|
|
47
|
-
const
|
|
47
|
+
const rhydb = await createRhydbModule();
|
|
48
48
|
```
|
|
49
49
|
|
|
50
|
-
The `.wasm` file ships alongside `rhydb_wasm.js` and must be resolvable by your bundler so the loader can fetch it at runtime. The package version matches the
|
|
50
|
+
The `.wasm` file ships alongside `rhydb_wasm.js` and must be resolvable by your bundler so the loader can fetch it at runtime. The package version matches the RhyDB release version.
|
|
51
51
|
|
|
52
|
-
The current build uses pthreads, so the page serving the module must be cross-origin isolated. Pthreads are the POSIX thread API that Emscripten maps to Web Workers, and
|
|
52
|
+
The current build uses pthreads, so the page serving the module must be cross-origin isolated. Pthreads are the POSIX thread API that Emscripten maps to Web Workers, and RhyDB uses them because its dependencies and query execution are built around threaded native code. Serve it over HTTP with these headers:
|
|
53
53
|
|
|
54
54
|
```text
|
|
55
55
|
Cross-Origin-Opener-Policy: same-origin
|
|
56
56
|
Cross-Origin-Embedder-Policy: require-corp
|
|
57
57
|
```
|
|
58
58
|
|
|
59
|
-
For a local example server, run this from the
|
|
59
|
+
For a local example server, run this from the RhyDB repository root:
|
|
60
60
|
|
|
61
61
|
```sh
|
|
62
62
|
python3 - <<'PY'
|
|
@@ -89,7 +89,7 @@ Prerequisites:
|
|
|
89
89
|
- uv
|
|
90
90
|
- a native C++ compiler for Conan build tools
|
|
91
91
|
|
|
92
|
-
From the
|
|
92
|
+
From the RhyDB repository root, build the WASM artifacts:
|
|
93
93
|
|
|
94
94
|
```sh
|
|
95
95
|
make wasm
|
|
@@ -120,7 +120,7 @@ Import and initialize the module:
|
|
|
120
120
|
```js
|
|
121
121
|
import createRhydbModule from "./rhydb_wasm.js";
|
|
122
122
|
|
|
123
|
-
const
|
|
123
|
+
const rhydb = await createRhydbModule();
|
|
124
124
|
```
|
|
125
125
|
|
|
126
126
|
This assumes `rhydb_wasm.js` and `rhydb_wasm.wasm` are served from the same URL directory. If a web app serves them from different locations, pass Emscripten's `locateFile` option to tell the loader where to fetch `rhydb_wasm.wasm`.
|
|
@@ -132,26 +132,26 @@ function mkdirp(path) {
|
|
|
132
132
|
let current = "";
|
|
133
133
|
for (const part of path.split("/").filter(Boolean)) {
|
|
134
134
|
current += `/${part}`;
|
|
135
|
-
if (!
|
|
136
|
-
|
|
135
|
+
if (!rhydb.FS.analyzePath(current).exists) {
|
|
136
|
+
rhydb.FS.mkdir(current);
|
|
137
137
|
}
|
|
138
138
|
}
|
|
139
139
|
}
|
|
140
140
|
|
|
141
141
|
mkdirp("/input");
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
142
|
+
rhydb.FS.writeFile("/input/preprocessing_config.yaml", preprocessingConfigBytes);
|
|
143
|
+
rhydb.FS.writeFile("/input/database_config.yaml", databaseConfigBytes);
|
|
144
|
+
rhydb.FS.writeFile("/input/reference_genomes.json", referenceGenomeBytes);
|
|
145
|
+
rhydb.FS.writeFile("/input/sequences.ndjson.zst", ndjsonBytes);
|
|
146
146
|
|
|
147
|
-
|
|
148
|
-
const handle =
|
|
147
|
+
rhydb.FS.chdir("/input");
|
|
148
|
+
const handle = rhydb.preprocess("preprocessing_config.yaml");
|
|
149
149
|
```
|
|
150
150
|
|
|
151
151
|
Query an in-memory database:
|
|
152
152
|
|
|
153
153
|
```js
|
|
154
|
-
const ndjson =
|
|
154
|
+
const ndjson = rhydb.query(handle, "default.groupBy({count:=count()})");
|
|
155
155
|
console.log(ndjson);
|
|
156
156
|
```
|
|
157
157
|
|
|
@@ -159,9 +159,9 @@ Save a processed state and read the files back:
|
|
|
159
159
|
|
|
160
160
|
```js
|
|
161
161
|
mkdirp("/state");
|
|
162
|
-
|
|
162
|
+
rhydb.save(handle, "/state");
|
|
163
163
|
|
|
164
|
-
for (const entry of
|
|
164
|
+
for (const entry of rhydb.FS.readdir("/state")) {
|
|
165
165
|
if (entry !== "." && entry !== "..") {
|
|
166
166
|
console.log(entry);
|
|
167
167
|
}
|
|
@@ -172,16 +172,16 @@ Load an existing processed state:
|
|
|
172
172
|
|
|
173
173
|
```js
|
|
174
174
|
mkdirp("/loaded-state");
|
|
175
|
-
// Write the saved
|
|
176
|
-
const loadedHandle =
|
|
177
|
-
console.log(
|
|
175
|
+
// Write the saved RhyDB state files under /loaded-state first.
|
|
176
|
+
const loadedHandle = rhydb.load("/loaded-state");
|
|
177
|
+
console.log(rhydb.info(loadedHandle));
|
|
178
178
|
```
|
|
179
179
|
|
|
180
180
|
Release memory when a database is no longer needed:
|
|
181
181
|
|
|
182
182
|
```js
|
|
183
|
-
|
|
184
|
-
|
|
183
|
+
rhydb.dispose(handle);
|
|
184
|
+
rhydb.dispose(loadedHandle);
|
|
185
185
|
```
|
|
186
186
|
|
|
187
187
|
For a complete browser example that uploads input files, downloads a saved state, uploads that state again, and runs a query, see `example/`.
|