@qe-mcp/ena 0.1.17 → 0.1.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +99 -1
- package/assets.js +54 -0
- package/manifest.json +1 -1
- package/package.json +9 -6
- package/server.js +348 -108
- package/xlsx.js +56 -5
package/README.md
CHANGED
|
@@ -157,6 +157,104 @@ manual install under [Get started](#get-started). Don't edit
|
|
|
157
157
|
be pasted. A plain-text copy of this page is at
|
|
158
158
|
[llms.txt](https://qe-libs.org/mcp/qe-mcp-ena/llms.txt).
|
|
159
159
|
|
|
160
|
+
### Hosting it (remote MCP)
|
|
161
|
+
|
|
162
|
+
Everything above runs the server on the user's own machine, which is what
|
|
163
|
+
keeps their data there. For web clients that can only reach remote MCP
|
|
164
|
+
endpoints (Claude.ai custom connectors, ChatGPT connectors, Gemini custom
|
|
165
|
+
apps), the same package can serve MCP over streamable HTTP instead:
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
npx --yes @qe-mcp/ena --http --port 8000 --host 0.0.0.0
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
The endpoint is `/mcp`; `GET /healthz` answers for probes. `--port` and
|
|
172
|
+
`--host` fall back to `$PORT` and `$HOST`; `--max-body-mb` caps request size
|
|
173
|
+
(default 25).
|
|
174
|
+
|
|
175
|
+
A hosted server has no access to anyone's files, so this mode is a different
|
|
176
|
+
privacy promise, and it says so to the model. In `--http` mode:
|
|
177
|
+
|
|
178
|
+
- Data arrives inline as `data_csv`; `data_path` is refused. The server
|
|
179
|
+
parses it in memory and discards it when the call returns: no rows and no
|
|
180
|
+
CSV text are kept, in memory or on disk (locally, parsed rows are cached for
|
|
181
|
+
speed). What stays is the fitted model behind a `model_id` — points,
|
|
182
|
+
networks, unit labels and group values — for `$ENA_MCP_MODEL_TTL_MIN`
|
|
183
|
+
minutes after its last use (default 60), then it is forgotten.
|
|
184
|
+
`ena_simplify` refits, so hosted it takes `data_csv` again and checks that
|
|
185
|
+
it is the data the model was fitted on.
|
|
186
|
+
- `ena_profile`, `ena_tables`, `ena_lines` and `ena_export` are not offered
|
|
187
|
+
(they read or write files), and the prompts are not listed (they take a
|
|
188
|
+
path). Plots come back as the inline PNG plus the SVG as text; no files are
|
|
189
|
+
written.
|
|
190
|
+
- The server's instructions tell the model to say that the data leaves the
|
|
191
|
+
user's machine and must be de-identified, and to get their OK first.
|
|
192
|
+
- `model_id`s carry a random suffix so one conversation cannot plot another's
|
|
193
|
+
model.
|
|
194
|
+
|
|
195
|
+
Each request is answered statelessly, so any replica can take any request,
|
|
196
|
+
but fitted models live in the process that fitted them, until they expire:
|
|
197
|
+
run one replica, or use sticky sessions, for `model_id`s to stay valid
|
|
198
|
+
across calls.
|
|
199
|
+
Put TLS and whatever authentication or rate limiting you need in front of it;
|
|
200
|
+
the server itself does neither. `node test-http.mjs` starts it on a free port
|
|
201
|
+
and checks all of the above; with `ENA_MCP_TEST_URL=http://host:port` it
|
|
202
|
+
checks a server that is already running instead.
|
|
203
|
+
|
|
204
|
+
#### Kubernetes
|
|
205
|
+
|
|
206
|
+
The `Dockerfile` builds the server as `node server.js --http` on port 8000
|
|
207
|
+
from the files the package ships plus production dependencies (no token
|
|
208
|
+
needed: the `@qe-libs` packages resolve from a public registry endpoint).
|
|
209
|
+
CI builds it for every main commit that touches those files and re-tags it
|
|
210
|
+
with each `vX.Y.Z` release tag, at
|
|
211
|
+
`registry.gitlab.com/epistemic-analytics/qe-packages/server-ena`.
|
|
212
|
+
|
|
213
|
+
`inst/k8s/` is a Helm chart in the shape the other qe apps use: one replica
|
|
214
|
+
with readiness and liveness on `/healthz`, and an nginx Ingress for
|
|
215
|
+
`hostUrl` with a body limit above the server's, buffering off, and a
|
|
216
|
+
per-client-IP rate limit (`rateLimit` in `values.yaml`: requests per second,
|
|
217
|
+
burst, concurrent connections; over it nginx answers 503). The limiter keys
|
|
218
|
+
on the address nginx sees, so it only tells clients apart if the controller
|
|
219
|
+
takes the real client address from the load balancer; and all claude.ai
|
|
220
|
+
users share Anthropic's egress addresses, so the defaults are generous.
|
|
221
|
+
`ENA_MCP_MODEL_TTL_MIN` and `ENA_MCP_MAX_BODY_MB` come from the chart
|
|
222
|
+
values. `inst/k8s/argo/production.yaml` is the Argo CD Application to copy
|
|
223
|
+
into the argo-deployments repository; its image updater follows the
|
|
224
|
+
semver image tags, so a release rolls out by itself.
|
|
225
|
+
|
|
226
|
+
Locally:
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
docker build -t qe-mcp-ena . && docker run --rm -p 8000:8000 qe-mcp-ena
|
|
230
|
+
ENA_MCP_TEST_URL=http://127.0.0.1:8000 node test-http.mjs
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
#### Cloudflare Workers
|
|
234
|
+
|
|
235
|
+
The same server runs as a Worker, without Node. `worker/build.mjs` bundles it
|
|
236
|
+
with esbuild into `worker/dist/`: the files the server reads from disk
|
|
237
|
+
(fonts, the qeviz bundle, the skill text) are embedded, and the WASM modules
|
|
238
|
+
are imported precompiled, as Workers require. Then wrangler runs or deploys
|
|
239
|
+
it (Node 22 or newer for wrangler):
|
|
240
|
+
|
|
241
|
+
```bash
|
|
242
|
+
node worker/build.mjs
|
|
243
|
+
cd worker && npx wrangler dev # http://127.0.0.1:8787/mcp
|
|
244
|
+
cd worker && npx wrangler deploy
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
Workers forbid `eval` and `new Function`, and Emscripten's Embind glue uses
|
|
248
|
+
them unless the module was linked with `-sDYNAMIC_EXECUTION=0`:
|
|
249
|
+
`@qe-libs/libqe-wasm` 0.2.4 and `@qe-libs/rena-wasm` 0.1.17 are the first
|
|
250
|
+
builds linked that way. To test a locally built module instead,
|
|
251
|
+
`ENA_WORKER_LIBQE_DIR` and `ENA_WORKER_LIBENA_DIR` name directories holding
|
|
252
|
+
`libqe.js`/`libqe.wasm` and `libena.js`/`libena.wasm`. Fits need the Workers Paid plan (the free plan's
|
|
253
|
+
10 ms CPU limit is too tight; a fit on 3,800 rows with its PNG takes a few
|
|
254
|
+
hundred milliseconds). The model cache lives in the isolate that served the
|
|
255
|
+
request, so `model_id`s stay valid only while the same isolate keeps
|
|
256
|
+
answering; a shared store (KV or a Durable Object) would make them durable.
|
|
257
|
+
|
|
160
258
|
## Prompts
|
|
161
259
|
|
|
162
260
|
Describe what you want; the assistant picks the tools. These use the columns
|
|
@@ -299,7 +397,7 @@ rule such as `Read(./data/**)` in `.claude/settings.json`, plus the hook below.
|
|
|
299
397
|
**Results come only from the server.** Every number and plot should come from
|
|
300
398
|
this server, never from code the assistant writes; the server and every tool
|
|
301
399
|
error say so. Each result carries `provenance` (e.g.
|
|
302
|
-
`@qe-mcp/ena 0.1.
|
|
400
|
+
`@qe-mcp/ena 0.1.18 (rena-wasm 0.1.14)`), the interactive plot names it
|
|
303
401
|
in its footer and the SVG in its `<desc>`. A plot or table without it didn't
|
|
304
402
|
come from this server.
|
|
305
403
|
|
package/assets.js
ADDED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* assets.js — the files the server reads from its own package.
|
|
3
|
+
*
|
|
4
|
+
* Node reads them from disk next to this file. A Worker bundle (see
|
|
5
|
+
* worker/) has no disk: it calls setAssets() with the same contents embedded
|
|
6
|
+
* at build time, before server.js is imported. Every accessor is lazy so the
|
|
7
|
+
* Node reads only happen when they are actually used.
|
|
8
|
+
*/
|
|
9
|
+
|
|
10
|
+
import { readFileSync } from 'node:fs';
|
|
11
|
+
import { createRequire } from 'node:module';
|
|
12
|
+
import { fileURLToPath } from 'node:url';
|
|
13
|
+
|
|
14
|
+
// The bundled plot font (Inter, OFL), as paths for the native resvg and as
|
|
15
|
+
// bytes for the WASM one.
|
|
16
|
+
const FONT_FILES = ['Inter-Regular.ttf', 'Inter-Bold.ttf'];
|
|
17
|
+
|
|
18
|
+
const nodeAssets = {
|
|
19
|
+
pkgJson: () => JSON.parse(readFileSync(new URL('./package.json', import.meta.url), 'utf8')),
|
|
20
|
+
renaVersion() {
|
|
21
|
+
try { return JSON.parse(readFileSync(createRequire(import.meta.url).resolve('@qe-libs/rena-wasm/package.json'), 'utf8')).version; }
|
|
22
|
+
catch { return 'unknown'; }
|
|
23
|
+
},
|
|
24
|
+
// The qeviz UMD bundle, inlined into interactive pages.
|
|
25
|
+
qevizJS: () => readFileSync(createRequire(import.meta.url).resolve('@qe-libs/qeviz'), 'utf8'),
|
|
26
|
+
skillMarkdown() {
|
|
27
|
+
try { return readFileSync(new URL('./skill/ena-analysis/SKILL.md', import.meta.url), 'utf8'); }
|
|
28
|
+
catch { return null; }
|
|
29
|
+
},
|
|
30
|
+
// What @resvg/resvg-wasm's initWasm() accepts: bytes here, a compiled
|
|
31
|
+
// WebAssembly.Module in a Worker.
|
|
32
|
+
resvgWasm: () => readFileSync(createRequire(import.meta.url).resolve('@resvg/resvg-wasm/index_bg.wasm')),
|
|
33
|
+
fontPaths: () => FONT_FILES.map(f => fileURLToPath(new URL(`./fonts/${f}`, import.meta.url))),
|
|
34
|
+
fontBuffers: () => FONT_FILES.map(f => readFileSync(new URL(`./fonts/${f}`, import.meta.url))),
|
|
35
|
+
};
|
|
36
|
+
|
|
37
|
+
let impl = nodeAssets;
|
|
38
|
+
|
|
39
|
+
/** Replace the asset source (a Worker bundle does this before importing server.js). */
|
|
40
|
+
export function setAssets(assets) { impl = { ...nodeAssets, ...assets }; }
|
|
41
|
+
|
|
42
|
+
export const pkgJson = () => impl.pkgJson();
|
|
43
|
+
export const renaVersion = () => impl.renaVersion();
|
|
44
|
+
export const skillMarkdown = () => impl.skillMarkdown();
|
|
45
|
+
export const resvgWasm = () => impl.resvgWasm();
|
|
46
|
+
export const fontPaths = () => impl.fontPaths();
|
|
47
|
+
export const fontBuffers = () => impl.fontBuffers();
|
|
48
|
+
|
|
49
|
+
let _qevizJS = null;
|
|
50
|
+
export function qevizJS() {
|
|
51
|
+
// Drop the source-map comment: the .map file is not next to the inlined copy.
|
|
52
|
+
if (!_qevizJS) _qevizJS = String(impl.qevizJS()).replace(/\n\/\/# sourceMappingURL=\S*\s*$/, '\n');
|
|
53
|
+
return _qevizJS;
|
|
54
|
+
}
|
package/manifest.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
"manifest_version": "0.2",
|
|
3
3
|
"name": "qe-mcp-ena",
|
|
4
4
|
"display_name": "ENA — Epistemic Network Analysis",
|
|
5
|
-
"version": "0.1.
|
|
5
|
+
"version": "0.1.19",
|
|
6
6
|
"description": "Epistemic Network Analysis on tabular coded data. Runs entirely on your machine — data never leaves your computer.",
|
|
7
7
|
"long_description": "Run Epistemic Network Analysis (ENA), ordered network analysis (ONA) and transmodal analysis (TMA) on your coded data from a conversation with Claude. No R or Python to install.\n\n**Your data stays on your machine**\n\n- The analysis runs locally. Only results — networks, points, statistics and plots — go back to the conversation.\n- Give Claude a file path rather than pasting data; the server reads the file itself.\n- It makes no network connections, starts no other programs, and writes files only when you ask for an export.\n\n**About the install warning**\n\nClaude Desktop shows \"access to everything on your computer\" for every extension outside Anthropic's reviewed directory. It means you are trusting the source, as with any program you install. The source is published on npm as @qe-mcp/ena.\n\nDocumentation and example prompts: https://qe-libs.org/mcp/qe-mcp-ena/",
|
|
8
8
|
"author": {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qe-mcp/ena",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.19",
|
|
4
4
|
"description": "Epistemic Network Analysis for Claude and other assistants — runs on your machine",
|
|
5
5
|
"repository": {
|
|
6
6
|
"type": "git",
|
|
@@ -13,6 +13,7 @@
|
|
|
13
13
|
},
|
|
14
14
|
"files": [
|
|
15
15
|
"server.js",
|
|
16
|
+
"assets.js",
|
|
16
17
|
"webena.js",
|
|
17
18
|
"xlsx.js",
|
|
18
19
|
"lines.js",
|
|
@@ -26,10 +27,9 @@
|
|
|
26
27
|
},
|
|
27
28
|
"dependencies": {
|
|
28
29
|
"@modelcontextprotocol/sdk": "^1.4.0",
|
|
29
|
-
"@qe-libs/qeviz": "0.
|
|
30
|
-
"@qe-libs/rena-wasm": "^0.1.
|
|
31
|
-
"@resvg/resvg-wasm": "^2.6.2"
|
|
32
|
-
"jsdom": "^25.0.1"
|
|
30
|
+
"@qe-libs/qeviz": "0.7.0",
|
|
31
|
+
"@qe-libs/rena-wasm": "^0.1.17",
|
|
32
|
+
"@resvg/resvg-wasm": "^2.6.2"
|
|
33
33
|
},
|
|
34
34
|
"optionalDependencies": {
|
|
35
35
|
"@resvg/resvg-js": "^2.6.2"
|
|
@@ -40,5 +40,8 @@
|
|
|
40
40
|
"epistemic-network-analysis",
|
|
41
41
|
"wasm"
|
|
42
42
|
],
|
|
43
|
-
"license": "GPL-3.0-only"
|
|
43
|
+
"license": "GPL-3.0-only",
|
|
44
|
+
"devDependencies": {
|
|
45
|
+
"wrangler": "^4.156.0"
|
|
46
|
+
}
|
|
44
47
|
}
|
package/server.js
CHANGED
|
@@ -29,6 +29,41 @@ import { connectionWindows, viewerHtml } from './lines.js';
|
|
|
29
29
|
import { defaultTensor } from '@qe-libs/rena-wasm/src/tensor.js';
|
|
30
30
|
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
|
|
31
31
|
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
|
32
|
+
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamableHttp.js';
|
|
33
|
+
import { createServer as createHttpServer } from 'http';
|
|
34
|
+
import { randomBytes } from 'crypto';
|
|
35
|
+
|
|
36
|
+
// ── Mode ──────────────────────────────────────────────────────────────────────
|
|
37
|
+
// Default: stdio, launched by the client on the user's machine (the bundle,
|
|
38
|
+
// npx, claude mcp add). `--http` serves MCP over streamable HTTP instead, for
|
|
39
|
+
// hosting the server where web clients (Claude.ai, ChatGPT, Gemini) can reach
|
|
40
|
+
// it. In that mode there is no user disk: data arrives inline, nothing is
|
|
41
|
+
// written to files, and the file-based tools are not offered.
|
|
42
|
+
const ARGV = process.argv.slice(2);
|
|
43
|
+
const argOf = (flag, dflt) => { const i = ARGV.indexOf(flag); return i >= 0 && ARGV[i + 1] != null ? ARGV[i + 1] : dflt; };
|
|
44
|
+
// Embedded in a Worker (globalThis.ENA_MCP_EMBEDDED) is always the hosted mode.
|
|
45
|
+
const REMOTE = ARGV.includes('--http') || !!globalThis.ENA_MCP_EMBEDDED;
|
|
46
|
+
const HTTP_PORT = Number(argOf('--port', process.env.PORT ?? 8000));
|
|
47
|
+
const HTTP_HOST = argOf('--host', process.env.HOST ?? '127.0.0.1');
|
|
48
|
+
const HTTP_PATH = '/mcp';
|
|
49
|
+
// Largest request body accepted in HTTP mode (the CSV travels inside it).
|
|
50
|
+
const HTTP_MAX_BODY = Math.round(Number(argOf('--max-body-mb', process.env.ENA_MCP_MAX_BODY_MB ?? 25)) * 1024 * 1024);
|
|
51
|
+
if (ARGV.includes('--help') || ARGV.includes('-h')) {
|
|
52
|
+
process.stdout.write([
|
|
53
|
+
'Usage: qe-mcp-ena [--http [--port N] [--host H] [--max-body-mb N]]',
|
|
54
|
+
'',
|
|
55
|
+
' (no flags) MCP over stdio — what Claude Desktop, Claude Code and other local clients run.',
|
|
56
|
+
' --http MCP over streamable HTTP at /mcp, for hosting. Data must be passed inline',
|
|
57
|
+
' (data_csv) and is discarded after each call; only fitted models are kept, for',
|
|
58
|
+
' $ENA_MCP_MODEL_TTL_MIN minutes (default 60); nothing is written to disk; ena_profile,',
|
|
59
|
+
' ena_tables, ena_lines and ena_export are not offered. Health check: GET /healthz.',
|
|
60
|
+
' --port N Port for --http (default: $PORT or 8000).',
|
|
61
|
+
' --host H Interface for --http (default: $HOST or 127.0.0.1; use 0.0.0.0 in a container).',
|
|
62
|
+
' --max-body-mb N Largest request body in MB (default: $ENA_MCP_MAX_BODY_MB or 25).',
|
|
63
|
+
'',
|
|
64
|
+
].join('\n'));
|
|
65
|
+
process.exit(0);
|
|
66
|
+
}
|
|
32
67
|
import {
|
|
33
68
|
CallToolRequestSchema,
|
|
34
69
|
ListToolsRequestSchema,
|
|
@@ -37,18 +72,12 @@ import {
|
|
|
37
72
|
} from '@modelcontextprotocol/sdk/types.js';
|
|
38
73
|
import loadENA, { detectParams, isBinary } from '@qe-libs/rena-wasm';
|
|
39
74
|
|
|
40
|
-
//
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
// The bundle is inlined into pages and jsdom, so drop its source-map comment
|
|
47
|
-
// (the .map file is not next to the inlined copy).
|
|
48
|
-
if (!_qevizJS) _qevizJS = readFileSync(QEVIZ_BUNDLE, 'utf8')
|
|
49
|
-
.replace(/\n\/\/# sourceMappingURL=\S*\s*$/, '\n');
|
|
50
|
-
return _qevizJS;
|
|
51
|
-
}
|
|
75
|
+
// qeviz: renderSVG() draws the static plot here, with no DOM; the UMD bundle
|
|
76
|
+
// (self-contained, no external deps) is inlined into the interactive page.
|
|
77
|
+
import { renderSVG as qevizRenderSVG } from '@qe-libs/qeviz';
|
|
78
|
+
// Package files (qeviz bundle, fonts, skill text, versions): read from disk
|
|
79
|
+
// in Node, embedded at build time in a Worker (see assets.js).
|
|
80
|
+
import { qevizJS, pkgJson, renaVersion, skillMarkdown, resvgWasm, fontBuffers, fontPaths } from './assets.js';
|
|
52
81
|
|
|
53
82
|
// ── CSV parser ────────────────────────────────────────────────────────────────
|
|
54
83
|
|
|
@@ -101,7 +130,12 @@ function dataFileText(dataPath) {
|
|
|
101
130
|
return readFileSync(dataPath, 'utf8');
|
|
102
131
|
}
|
|
103
132
|
|
|
133
|
+
const REMOTE_NO_PATH =
|
|
134
|
+
'data_path is not available on a hosted server: it has no access to the user\'s files. ' +
|
|
135
|
+
'Pass the data inline as data_csv, after confirming with the user that it is de-identified.';
|
|
136
|
+
|
|
104
137
|
function loadData(dataPath, dataCsv) {
|
|
138
|
+
if (dataPath && REMOTE) throw new Error(REMOTE_NO_PATH);
|
|
105
139
|
if (dataPath) return parseCSV(dataFileText(dataPath));
|
|
106
140
|
if (dataCsv) return parseCSV(dataCsv);
|
|
107
141
|
throw new Error('Provide either data_path (path to a .csv or .xlsx file) or data_csv (raw CSV string).');
|
|
@@ -348,7 +382,7 @@ function buildGmrParams(rows, unitCols, unitLabels, p) {
|
|
|
348
382
|
// rotationSet (= R's rotation.set): centred on the source's centerVec, rotated
|
|
349
383
|
// by its rotation matrix, with its node positions.
|
|
350
384
|
function rotationSetFrom(sourceId, codes, ordered) {
|
|
351
|
-
const src =
|
|
385
|
+
const src = modelGet(sourceId);
|
|
352
386
|
if (!src)
|
|
353
387
|
throw new Error(`rotation_from: unknown model_id '${sourceId}'. Fit the source model first ` +
|
|
354
388
|
`(the ${MAX_MODELS} most recently used models are kept).`);
|
|
@@ -444,7 +478,7 @@ function detectColumns(rows) {
|
|
|
444
478
|
// <qe-graph>), attrs (<qe-graph> attributes), edgePx, title }.
|
|
445
479
|
|
|
446
480
|
// Group values and titles come from the data and from tool arguments; escape
|
|
447
|
-
// them before they go into
|
|
481
|
+
// them before they go into the interactive page's markup.
|
|
448
482
|
const escHtml = (v) => String(v).replace(/[&<>"']/g, ch =>
|
|
449
483
|
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[ch]);
|
|
450
484
|
|
|
@@ -645,35 +679,44 @@ function vizSlug(result, args = {}) {
|
|
|
645
679
|
.join('_');
|
|
646
680
|
}
|
|
647
681
|
|
|
648
|
-
// ──
|
|
649
|
-
|
|
650
|
-
import { JSDOM } from 'jsdom';
|
|
682
|
+
// ── static SVG renderer ───────────────────────────────────────────────────────
|
|
651
683
|
|
|
652
684
|
// PNG output (the inline image block and the png_plot file) prefers the native
|
|
653
|
-
// @resvg/resvg-js, an OPTIONAL dependency: fastest
|
|
654
|
-
//
|
|
655
|
-
//
|
|
656
|
-
//
|
|
685
|
+
// @resvg/resvg-js, an OPTIONAL dependency: fastest. Where it is absent — the
|
|
686
|
+
// cross-platform .mcpb bundle, or a platform without a prebuilt binary —
|
|
687
|
+
// @resvg/resvg-wasm renders the same image. Both draw plot labels with the
|
|
688
|
+
// bundled Inter (fonts/, OFL) and never scan system fonts: resvg's default
|
|
689
|
+
// loadSystemFonts walks the whole system font set on EVERY render, which on
|
|
690
|
+
// macOS costs ~1.5 s per PNG against ~25 ms with the two bundled files.
|
|
657
691
|
// Loaded lazily; if neither loads, SVG and interactive HTML are still produced.
|
|
692
|
+
//
|
|
693
|
+
// The two builds take the fonts differently. The WASM build takes the bytes
|
|
694
|
+
// (fontBuffers). The native build takes paths (fontFiles): given fontBuffers,
|
|
695
|
+
// resvg-js 2.6 scans the system fonts anyway, loadSystemFonts: false or not,
|
|
696
|
+
// and the 1.5 s is back.
|
|
697
|
+
const FONT_FAMILY = { defaultFontFamily: 'Inter', sansSerifFamily: 'Inter', loadSystemFonts: false };
|
|
698
|
+
// Wrap a Resvg class so every render uses the bundled fonts and asPng() yields
|
|
699
|
+
// a Buffer (the WASM build returns a Uint8Array).
|
|
700
|
+
function withBundledFonts(Resvg, fonts) {
|
|
701
|
+
let font;
|
|
702
|
+
return function FontedResvg(svg, opts = {}) {
|
|
703
|
+
font ??= { ...FONT_FAMILY, ...fonts() };
|
|
704
|
+
const r = new Resvg(svg, { ...opts, font: { ...font, ...opts.font } });
|
|
705
|
+
return { render: () => { const img = r.render(); return { asPng: () => Buffer.from(img.asPng()) }; } };
|
|
706
|
+
};
|
|
707
|
+
}
|
|
658
708
|
let _Resvg; // undefined = not yet attempted; null = unavailable
|
|
659
709
|
async function getResvg() {
|
|
660
710
|
if (_Resvg !== undefined) return _Resvg;
|
|
661
711
|
try {
|
|
662
|
-
|
|
712
|
+
const { Resvg } = await import('@resvg/resvg-js');
|
|
713
|
+
_Resvg = withBundledFonts(Resvg, () => ({ fontFiles: fontPaths() }));
|
|
663
714
|
return _Resvg;
|
|
664
715
|
} catch { /* fall back to WASM */ }
|
|
665
716
|
try {
|
|
666
717
|
const { initWasm, Resvg } = await import('@resvg/resvg-wasm');
|
|
667
|
-
await initWasm(
|
|
668
|
-
|
|
669
|
-
.map(f => readFileSync(new URL(`./fonts/${f}`, import.meta.url)));
|
|
670
|
-
const font = { fontBuffers, defaultFontFamily: 'Inter', sansSerifFamily: 'Inter' };
|
|
671
|
-
// Same shape as the native class — new Resvg(svg, opts).render().asPng()
|
|
672
|
-
// — but asPng() gives a Uint8Array here, so hand back a Buffer as native does.
|
|
673
|
-
_Resvg = function WasmResvg(svg, opts = {}) {
|
|
674
|
-
const r = new Resvg(svg, { ...opts, font });
|
|
675
|
-
return { render: () => { const img = r.render(); return { asPng: () => Buffer.from(img.asPng()) }; } };
|
|
676
|
-
};
|
|
718
|
+
await initWasm(await resvgWasm());
|
|
719
|
+
_Resvg = withBundledFonts(Resvg, () => ({ fontBuffers: fontBuffers() }));
|
|
677
720
|
} catch (e) {
|
|
678
721
|
_Resvg = null;
|
|
679
722
|
process.stderr.write(
|
|
@@ -686,34 +729,27 @@ async function getResvg() {
|
|
|
686
729
|
|
|
687
730
|
const SVG_SIZE = 600;
|
|
688
731
|
|
|
732
|
+
// qeviz's headless renderSVG() draws the same <qe-graph> the interactive page
|
|
733
|
+
// shows (same attributes, same layer children) as a standalone SVG string,
|
|
734
|
+
// without a DOM: no browser, no timers. Anything qeviz logs while drawing goes
|
|
735
|
+
// to stderr, since stdout is the stdio transport.
|
|
689
736
|
async function renderSVG(plot) {
|
|
690
|
-
const dom = new JSDOM(
|
|
691
|
-
`<!DOCTYPE html><body><qe-visual id="vis"><qe-graph id="g" width="${SVG_SIZE}" height="${SVG_SIZE}"` +
|
|
692
|
-
`${attrString({ ...plot.attrs, 'edge-width': edgeWidth(plot.edgePx, SVG_SIZE) })}>${layerMarkup(plot.layers)}</qe-graph></qe-visual></body>`,
|
|
693
|
-
{ runScripts: 'dangerously', pretendToBeVisual: true }
|
|
694
|
-
);
|
|
695
|
-
const { window } = dom;
|
|
696
|
-
|
|
697
|
-
window.Element.prototype.getBoundingClientRect = () =>
|
|
698
|
-
({ x:0, y:0, width:SVG_SIZE, height:SVG_SIZE, top:0, left:0, right:SVG_SIZE, bottom:SVG_SIZE });
|
|
699
|
-
window.requestAnimationFrame = cb => setTimeout(cb, 0);
|
|
700
|
-
|
|
701
737
|
const toStderr = (...a) => process.stderr.write(a.map(String).join(' ') + '\n');
|
|
702
|
-
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
|
|
708
|
-
|
|
709
|
-
|
|
710
|
-
|
|
711
|
-
|
|
712
|
-
|
|
738
|
+
const saved = { log: console.log, info: console.info, warn: console.warn, debug: console.debug };
|
|
739
|
+
Object.assign(console, { log: toStderr, info: toStderr, warn: toStderr, debug: () => {} });
|
|
740
|
+
let svgHtml;
|
|
741
|
+
try {
|
|
742
|
+
svgHtml = qevizRenderSVG(plot.data, {
|
|
743
|
+
width: SVG_SIZE,
|
|
744
|
+
height: SVG_SIZE,
|
|
745
|
+
graph: { ...plot.attrs, 'edge-width': edgeWidth(plot.edgePx, SVG_SIZE) },
|
|
746
|
+
layers: plot.layers,
|
|
747
|
+
});
|
|
748
|
+
} finally {
|
|
749
|
+
Object.assign(console, saved);
|
|
750
|
+
}
|
|
751
|
+
if (!svgHtml) return null;
|
|
713
752
|
|
|
714
|
-
// XML serialization: outerHTML (HTML rules) leaves '<' unescaped inside
|
|
715
|
-
// attribute values, which resvg rejects when a label contains one.
|
|
716
|
-
let svgHtml = new window.XMLSerializer().serializeToString(svgEl);
|
|
717
753
|
// resvg requires the SVG namespace declaration to parse the document
|
|
718
754
|
if (!svgHtml.includes('xmlns=')) {
|
|
719
755
|
svgHtml = svgHtml.replace('<svg', '<svg xmlns="http://www.w3.org/2000/svg"');
|
|
@@ -818,6 +854,15 @@ let modelSeq = 0;
|
|
|
818
854
|
const MAX_ACCUM = 8;
|
|
819
855
|
const MAX_MODELS = 50;
|
|
820
856
|
|
|
857
|
+
// Hosted, the process serves strangers, so it keeps no data: parsed rows and
|
|
858
|
+
// accumulations are not cached (every call re-parses the CSV it was sent, tens
|
|
859
|
+
// of ms), the CSV text is dropped from a model's stored parameters, and a
|
|
860
|
+
// model — points, networks, unit and group labels — is forgotten MODEL_TTL_MS
|
|
861
|
+
// after its last use instead of waiting to be displaced.
|
|
862
|
+
const KEEP_ROWS = !REMOTE;
|
|
863
|
+
let MODEL_TTL_MS = REMOTE ? Number(process.env.ENA_MCP_MODEL_TTL_MIN ?? 60) * 60_000 : 0;
|
|
864
|
+
export function setModelTtl(minutes) { MODEL_TTL_MS = Number(minutes) * 60_000; }
|
|
865
|
+
|
|
821
866
|
function cacheGet(map, key) {
|
|
822
867
|
if (!map.has(key)) return undefined;
|
|
823
868
|
const v = map.get(key);
|
|
@@ -830,6 +875,27 @@ function cacheSet(map, key, value, max) {
|
|
|
830
875
|
while (map.size > max) map.delete(map.keys().next().value);
|
|
831
876
|
}
|
|
832
877
|
|
|
878
|
+
function expireModels(now = Date.now()) {
|
|
879
|
+
if (!MODEL_TTL_MS) return;
|
|
880
|
+
for (const [id, e] of modelCache)
|
|
881
|
+
if (now - e._used > MODEL_TTL_MS) modelCache.delete(id);
|
|
882
|
+
}
|
|
883
|
+
function modelGet(model_id) {
|
|
884
|
+
expireModels();
|
|
885
|
+
const e = cacheGet(modelCache, model_id);
|
|
886
|
+
if (e) e._used = Date.now();
|
|
887
|
+
return e;
|
|
888
|
+
}
|
|
889
|
+
function modelSet(model_id, entry) {
|
|
890
|
+
expireModels();
|
|
891
|
+
if (!KEEP_ROWS && entry.params) {
|
|
892
|
+
entry._data = dataStamp(entry.params); // to check resent data against
|
|
893
|
+
entry.params = { ...entry.params, data_csv: undefined };
|
|
894
|
+
}
|
|
895
|
+
entry._used = Date.now();
|
|
896
|
+
cacheSet(modelCache, model_id, entry, MAX_MODELS);
|
|
897
|
+
}
|
|
898
|
+
|
|
833
899
|
// Identifies the input data. For a file, include its mtime and size so an
|
|
834
900
|
// edited CSV is re-read instead of serving a model fitted on the old contents.
|
|
835
901
|
function dataStamp(p) {
|
|
@@ -981,7 +1047,11 @@ function optsLines(p, nUnits) {
|
|
|
981
1047
|
];
|
|
982
1048
|
}
|
|
983
1049
|
|
|
984
|
-
|
|
1050
|
+
// Hosted, the one process serves every client, so a model_id must not be
|
|
1051
|
+
// guessable: m1 from one conversation is not for another to plot.
|
|
1052
|
+
function nextModelId() {
|
|
1053
|
+
return REMOTE ? `m${++modelSeq}-${randomBytes(6).toString('hex')}` : `m${++modelSeq}`;
|
|
1054
|
+
}
|
|
985
1055
|
|
|
986
1056
|
// ── Web tool defaults ─────────────────────────────────────────────────────────
|
|
987
1057
|
|
|
@@ -992,13 +1062,13 @@ function loadRows(p) {
|
|
|
992
1062
|
const key = dataStamp(p);
|
|
993
1063
|
let rows;
|
|
994
1064
|
if (rowsCache.has(key)) rows = cacheGet(rowsCache, key);
|
|
995
|
-
else { rows = loadData(p.data_path, p.data_csv); cacheSet(rowsCache, key, rows, MAX_ROWS); }
|
|
1065
|
+
else { rows = loadData(p.data_path, p.data_csv); if (KEEP_ROWS) cacheSet(rowsCache, key, rows, MAX_ROWS); }
|
|
996
1066
|
if (!toArray(p.exclude_conversations).length && !p.order_by) return rows;
|
|
997
1067
|
|
|
998
1068
|
const vKey = JSON.stringify([key, toArray(p.conversations), toArray(p.exclude_conversations), p.order_by ?? null]);
|
|
999
1069
|
if (rowsCache.has(vKey)) return cacheGet(rowsCache, vKey);
|
|
1000
1070
|
const view = prepareRows(rows, p);
|
|
1001
|
-
cacheSet(rowsCache, vKey, view, MAX_ROWS);
|
|
1071
|
+
if (KEEP_ROWS) cacheSet(rowsCache, vKey, view, MAX_ROWS);
|
|
1002
1072
|
return view;
|
|
1003
1073
|
}
|
|
1004
1074
|
|
|
@@ -1402,7 +1472,7 @@ async function toolTmaFit(ena, params) {
|
|
|
1402
1472
|
result,
|
|
1403
1473
|
};
|
|
1404
1474
|
entry.view = defaultView(params, entry.web);
|
|
1405
|
-
|
|
1475
|
+
modelSet(model_id, entry);
|
|
1406
1476
|
|
|
1407
1477
|
return { ...result, _plot: web.buildPlot(entry, entry.view) };
|
|
1408
1478
|
}
|
|
@@ -1587,7 +1657,7 @@ function toolInspect({ data_path, data_csv }) {
|
|
|
1587
1657
|
nextStep = {
|
|
1588
1658
|
tool: 'ena_fit',
|
|
1589
1659
|
parameters: {
|
|
1590
|
-
data_path:
|
|
1660
|
+
...(REMOTE ? { data_csv: '<the same CSV text>' } : { data_path: data_path ?? '<save CSV to a path first>' }),
|
|
1591
1661
|
units: bestUnit,
|
|
1592
1662
|
conversations: bestConvo,
|
|
1593
1663
|
codes: codeCols,
|
|
@@ -1627,6 +1697,8 @@ function toolInspect({ data_path, data_csv }) {
|
|
|
1627
1697
|
|
|
1628
1698
|
const privacyNote = isPrivacySafe
|
|
1629
1699
|
? 'PRIVACY SAFE — data was read from a local file path. No raw row data was sent to Claude; only this analysis result was transmitted.'
|
|
1700
|
+
: REMOTE
|
|
1701
|
+
? 'HOSTED SERVER — the data was sent inline, processed in memory and not stored. It left the user\'s machine, so it must be de-identified; confirm that with the user if you have not.'
|
|
1630
1702
|
: 'RAW DATA TRANSMITTED — data_csv was used, so row data passed through this conversation. For sensitive datasets, ask the user to save the file and provide data_path instead.';
|
|
1631
1703
|
|
|
1632
1704
|
return {
|
|
@@ -1663,9 +1735,9 @@ async function toolFit(ena, params) {
|
|
|
1663
1735
|
// Return cached model if params are identical
|
|
1664
1736
|
const fKey = fitCacheKey(params);
|
|
1665
1737
|
const cachedId = fitCache.get(fKey);
|
|
1666
|
-
|
|
1667
|
-
|
|
1668
|
-
const
|
|
1738
|
+
const cached = cachedId && modelGet(cachedId); // the model may have been evicted or expired
|
|
1739
|
+
if (cached) {
|
|
1740
|
+
const model_id = cachedId, entry = cached;
|
|
1669
1741
|
return {
|
|
1670
1742
|
model_id,
|
|
1671
1743
|
...entry.result,
|
|
@@ -1685,7 +1757,7 @@ async function toolFit(ena, params) {
|
|
|
1685
1757
|
codes, units: unitCols, conversations: convoCols,
|
|
1686
1758
|
window: windowValue(window_size), binary, ...accumOpts(params),
|
|
1687
1759
|
});
|
|
1688
|
-
cacheSet(accumCache, aKey, { rows, acc }, MAX_ACCUM);
|
|
1760
|
+
if (KEEP_ROWS) cacheSet(accumCache, aKey, { rows, acc }, MAX_ACCUM);
|
|
1689
1761
|
}
|
|
1690
1762
|
|
|
1691
1763
|
const { unitLabels, nConnections, connectionNames, nUnits } = acc;
|
|
@@ -1785,7 +1857,7 @@ async function toolFit(ena, params) {
|
|
|
1785
1857
|
result,
|
|
1786
1858
|
};
|
|
1787
1859
|
entry.view = defaultView(params, entry.web);
|
|
1788
|
-
|
|
1860
|
+
modelSet(model_id, entry);
|
|
1789
1861
|
cacheSet(fitCache, fKey, model_id, MAX_MODELS);
|
|
1790
1862
|
|
|
1791
1863
|
return { ...result, _plot: web.buildPlot(entry, entry.view) };
|
|
@@ -1906,9 +1978,9 @@ async function toolCompareGroups(ena, params) {
|
|
|
1906
1978
|
// Return cached model if params are identical
|
|
1907
1979
|
const fKey = fitCacheKey(params, 'compare');
|
|
1908
1980
|
const cachedId = fitCache.get(fKey);
|
|
1909
|
-
|
|
1910
|
-
|
|
1911
|
-
const
|
|
1981
|
+
const cached = cachedId && modelGet(cachedId); // the model may have been evicted or expired
|
|
1982
|
+
if (cached) {
|
|
1983
|
+
const model_id = cachedId, entry = cached;
|
|
1912
1984
|
return {
|
|
1913
1985
|
model_id,
|
|
1914
1986
|
...entry.result,
|
|
@@ -1932,7 +2004,7 @@ async function toolCompareGroups(ena, params) {
|
|
|
1932
2004
|
codes, units: unitCols, conversations: convoCols,
|
|
1933
2005
|
window: windowValue(window_size), binary, ...accumOpts(params),
|
|
1934
2006
|
});
|
|
1935
|
-
cacheSet(accumCache, aKey, { rows, acc }, MAX_ACCUM);
|
|
2007
|
+
if (KEEP_ROWS) cacheSet(accumCache, aKey, { rows, acc }, MAX_ACCUM);
|
|
1936
2008
|
}
|
|
1937
2009
|
|
|
1938
2010
|
const { unitLabels, connectionNames, nUnits, nConnections } = acc;
|
|
@@ -2064,7 +2136,7 @@ async function toolCompareGroups(ena, params) {
|
|
|
2064
2136
|
result,
|
|
2065
2137
|
};
|
|
2066
2138
|
entry.view = defaultView(params, entry.web);
|
|
2067
|
-
|
|
2139
|
+
modelSet(model_id, entry);
|
|
2068
2140
|
cacheSet(fitCache, fKey, model_id, MAX_MODELS);
|
|
2069
2141
|
|
|
2070
2142
|
return { ...result, _plot: web.buildPlot(entry, entry.view) };
|
|
@@ -2073,8 +2145,8 @@ async function toolCompareGroups(ena, params) {
|
|
|
2073
2145
|
// ena_plot's display options → buildPlot's view.style (the web tool's Plot
|
|
2074
2146
|
// Settings). Validated here so a bad colour or group fails with a message
|
|
2075
2147
|
// instead of a blank plot.
|
|
2076
|
-
//
|
|
2077
|
-
//
|
|
2148
|
+
// Colour names are turned into hex here so the SVG and the interactive page
|
|
2149
|
+
// agree whatever resolves them: the CSS named colours.
|
|
2078
2150
|
const CSS_NAMED = Object.fromEntries((
|
|
2079
2151
|
'aliceblue:f0f8ff antiquewhite:faebd7 aqua:00ffff aquamarine:7fffd4 azure:f0ffff beige:f5f5dc ' +
|
|
2080
2152
|
'bisque:ffe4c4 black:000000 blanchedalmond:ffebcd blue:0000ff blueviolet:8a2be2 brown:a52a2a ' +
|
|
@@ -2143,7 +2215,7 @@ async function toolPlot({ model_id, type = 'all', group_value, group1, group2, u
|
|
|
2143
2215
|
x_dim = 1, y_dim = 2, flip_x = false, flip_y = false,
|
|
2144
2216
|
min_edge_weight, hide_unconnected_codes, text_size, colors,
|
|
2145
2217
|
confidence_intervals, outlier_intervals }) {
|
|
2146
|
-
const entry =
|
|
2218
|
+
const entry = modelGet(model_id);
|
|
2147
2219
|
if (!entry)
|
|
2148
2220
|
throw new Error(`Unknown model_id '${model_id}'. Call ena_fit, ena_compare_groups or tma_fit first ` +
|
|
2149
2221
|
`(the ${MAX_MODELS} most recently used models are kept; refitting is cheap).`);
|
|
@@ -2317,7 +2389,7 @@ function csvCell(v) {
|
|
|
2317
2389
|
const csvText = (header, rows) => [header, ...rows].map(r => r.map(csvCell).join(',')).join('\n') + '\n';
|
|
2318
2390
|
|
|
2319
2391
|
async function toolTables(ena, { model_id, out_path }) {
|
|
2320
|
-
const entry =
|
|
2392
|
+
const entry = modelGet(model_id);
|
|
2321
2393
|
if (!entry)
|
|
2322
2394
|
throw new Error(`Unknown model_id '${model_id}'. Fit the model first (ena_fit, ena_compare_groups or tma_fit).`);
|
|
2323
2395
|
const { model, codes, unitLabels } = entry;
|
|
@@ -2372,12 +2444,20 @@ async function toolTables(ena, { model_id, out_path }) {
|
|
|
2372
2444
|
// then, as the web tool does, refit without the removed codes.
|
|
2373
2445
|
const PRIA_MIN = 0.8, PRIA_MAX = 0.99;
|
|
2374
2446
|
|
|
2375
|
-
async function toolSimplify(ena, { model_id, threshold = 0.95, remove_num = 3 }) {
|
|
2376
|
-
const entry =
|
|
2447
|
+
async function toolSimplify(ena, { model_id, threshold = 0.95, remove_num = 3, data_csv }) {
|
|
2448
|
+
const entry = modelGet(model_id);
|
|
2377
2449
|
if (!entry)
|
|
2378
2450
|
throw new Error(`Unknown model_id '${model_id}'. Fit the model first (ena_fit, ena_compare_groups or tma_fit).`);
|
|
2379
2451
|
if (!entry.fitArgs) throw new Error(`Model ${model_id} can't be simplified. Refit it with this server version.`);
|
|
2380
|
-
|
|
2452
|
+
let p = entry.params ?? {};
|
|
2453
|
+
if (!KEEP_ROWS) {
|
|
2454
|
+
// The hosted server kept nothing but the model: PRIA refits, so the
|
|
2455
|
+
// same data must come again, and it must be the same data.
|
|
2456
|
+
if (!data_csv) throw new Error('Hosted server: the data is not stored. Pass data_csv again (the CSV the model was fitted on).');
|
|
2457
|
+
if (dataStamp({ data_csv }) !== entry._data)
|
|
2458
|
+
throw new Error(`data_csv is not the data model ${model_id} was fitted on. Pass the same CSV, or fit a new model.`);
|
|
2459
|
+
p = { ...p, data_csv };
|
|
2460
|
+
}
|
|
2381
2461
|
const codes = entry.codes;
|
|
2382
2462
|
if (codes.length < 4)
|
|
2383
2463
|
throw new Error(`Model ${model_id} has ${codes.length} codes; PRIA needs more than 3 (it never leaves fewer than 3).`);
|
|
@@ -2424,7 +2504,7 @@ async function toolSimplify(ena, { model_id, threshold = 0.95, remove_num = 3 })
|
|
|
2424
2504
|
const DEFAULT_MAX_WINDOWS = 500;
|
|
2425
2505
|
|
|
2426
2506
|
async function toolLines(ena, { model_id, connection, units, group, columns, max_windows, out_path }) {
|
|
2427
|
-
const entry =
|
|
2507
|
+
const entry = modelGet(model_id);
|
|
2428
2508
|
if (!entry)
|
|
2429
2509
|
throw new Error(`Unknown model_id '${model_id}'. Fit the model first (ena_fit, ena_compare_groups or tma_fit).`);
|
|
2430
2510
|
const { model, codes, unitLabels } = entry;
|
|
@@ -2559,7 +2639,7 @@ async function toolLines(ena, { model_id, connection, units, group, columns, max
|
|
|
2559
2639
|
}
|
|
2560
2640
|
|
|
2561
2641
|
async function toolExport(ena, { model_id, set_name, out_path }) {
|
|
2562
|
-
const entry =
|
|
2642
|
+
const entry = modelGet(model_id);
|
|
2563
2643
|
if (!entry)
|
|
2564
2644
|
throw new Error(`Unknown model_id '${model_id}'. Fit the model first (ena_fit, ena_compare_groups or tma_fit).`);
|
|
2565
2645
|
const blockers = exportBlockers(entry);
|
|
@@ -2756,8 +2836,8 @@ const ROTATION_PROPS = {
|
|
|
2756
2836
|
|
|
2757
2837
|
const GUIDE = (() => {
|
|
2758
2838
|
let md;
|
|
2759
|
-
|
|
2760
|
-
|
|
2839
|
+
md = skillMarkdown();
|
|
2840
|
+
if (md == null) return null;
|
|
2761
2841
|
md = md.replace(/^---\n[\s\S]*?\n---\n+/, ''); // frontmatter
|
|
2762
2842
|
const sections = [];
|
|
2763
2843
|
for (const part of md.split(/^(?=## )/m).slice(1)) {
|
|
@@ -2772,12 +2852,17 @@ const GUIDE_TOPICS = GUIDE?.sections.map(s => s.topic) ?? [];
|
|
|
2772
2852
|
function toolGuide({ topic } = {}) {
|
|
2773
2853
|
if (!GUIDE) throw new Error('The analysis guide is missing from this installation (skill/ena-analysis/SKILL.md).');
|
|
2774
2854
|
const footer = `\n\n---\nTopics: ${GUIDE_TOPICS.join(', ')}`;
|
|
2775
|
-
|
|
2855
|
+
const header = REMOTE
|
|
2856
|
+
? '> **Hosted server.** This server runs remotely, not on the user\'s machine. Where the guide says to pass ' +
|
|
2857
|
+
'data_path, pass the data inline as data_csv instead, after the user confirms it is de-identified. ' +
|
|
2858
|
+
'ena_profile, ena_tables, ena_lines and ena_export are not available here; plots come back inline.\n\n'
|
|
2859
|
+
: '';
|
|
2860
|
+
if (!topic) return header + GUIDE.text + footer;
|
|
2776
2861
|
const t = String(topic).toLowerCase().trim();
|
|
2777
2862
|
const hit = GUIDE.sections.find(s => s.topic === t)
|
|
2778
2863
|
?? GUIDE.sections.find(s => s.topic.includes(t.replace(/[^a-z0-9]+/g, '-')));
|
|
2779
2864
|
if (!hit) throw new Error(`Unknown topic. Topics: ${GUIDE_TOPICS.join(', ')}`);
|
|
2780
|
-
return hit.text + footer;
|
|
2865
|
+
return header + hit.text + footer;
|
|
2781
2866
|
}
|
|
2782
2867
|
|
|
2783
2868
|
const TOOLS = [
|
|
@@ -3069,6 +3154,7 @@ const TOOLS = [
|
|
|
3069
3154
|
model_id: { type: 'string', description: 'model_id from ena_fit, ena_compare_groups or tma_fit' },
|
|
3070
3155
|
threshold: { type: 'number', minimum: 0.8, maximum: 0.99, description: 'How close the simplified model must stay to the full one (default 0.95; the web tool allows 0.80–0.99).' },
|
|
3071
3156
|
remove_num: { type: 'integer', minimum: 1, description: 'Most codes to remove (default 3).' },
|
|
3157
|
+
data_csv: { type: 'string', description: 'Hosted server only: the CSV the model was fitted on, again (the server does not store it). Locally, omit.', _hostedOnly: true },
|
|
3072
3158
|
},
|
|
3073
3159
|
required: ['model_id'],
|
|
3074
3160
|
},
|
|
@@ -3135,13 +3221,49 @@ const TOOLS = [
|
|
|
3135
3221
|
},
|
|
3136
3222
|
];
|
|
3137
3223
|
|
|
3224
|
+
// Hosted: no user files to read or write, so the file tools go, data_path goes
|
|
3225
|
+
// from every schema, and the privacy wording changes from "keep it local" to
|
|
3226
|
+
// "send it de-identified".
|
|
3227
|
+
const REMOTE_HIDDEN_TOOLS = new Set(['ena_profile', 'ena_tables', 'ena_lines', 'ena_export']);
|
|
3228
|
+
const REMOTE_PRIVACY = 'PRIVACY (hosted server): the data is sent inline and processed in memory; it is discarded when the call returns, and only the fitted model is kept, for a limited time. ' +
|
|
3229
|
+
'Before sending it, confirm with the user that it is de-identified. ';
|
|
3230
|
+
function remoteTools(tools) {
|
|
3231
|
+
return tools.filter(t => !REMOTE_HIDDEN_TOOLS.has(t.name)).map(t => {
|
|
3232
|
+
const { data_path, ...properties } = t.inputSchema.properties ?? {};
|
|
3233
|
+
let description = t.description
|
|
3234
|
+
.replace(/The ENA network plot is written to disk[^]*?Use these paths to show the plot; /,
|
|
3235
|
+
'The ENA network plot comes back as an inline image. Use it to show the plot; ')
|
|
3236
|
+
.replace(/The ENA network plot is written to disk[^]*?is installed\./,
|
|
3237
|
+
'The ENA network plot comes back as an inline image (and the SVG as text).')
|
|
3238
|
+
.replace(/The interactive HTML has its own controls[^.]*\. /, '')
|
|
3239
|
+
.replace(/PRIVACY: .*?\.(?: (?=[A-Z])|$)/g, REMOTE_PRIVACY)
|
|
3240
|
+
.replace(/ ?The WASM model runs entirely on the user's machine[^.]*\./, '')
|
|
3241
|
+
.replace(/If the user has not yet indicated whether their data is sensitive, ask before using data_csv\. ?/, '')
|
|
3242
|
+
.replace(/If they indicate sensitivity, use ena_profile instead \(no raw data transmitted\)\. ?/, '')
|
|
3243
|
+
.trim();
|
|
3244
|
+
const required = (t.inputSchema.required ?? []).filter(r => r !== 'data_path');
|
|
3245
|
+
if (properties.data_csv && !description.includes(REMOTE_PRIVACY.trim()))
|
|
3246
|
+
description = `${description} ${REMOTE_PRIVACY.trim()}`;
|
|
3247
|
+
if (properties.data_csv && !properties.data_csv._hostedOnly) properties.data_csv = {
|
|
3248
|
+
type: 'string',
|
|
3249
|
+
description: 'The data as CSV text. The server has no access to the user\'s files; it processes this in memory and does not store it. Send only de-identified data.',
|
|
3250
|
+
};
|
|
3251
|
+
if (properties.data_csv?._hostedOnly) { const { _hostedOnly, ...d } = properties.data_csv; properties.data_csv = d; }
|
|
3252
|
+
return { ...t, description, inputSchema: { ...t.inputSchema, properties, ...(required.length ? { required } : {}) } };
|
|
3253
|
+
});
|
|
3254
|
+
}
|
|
3255
|
+
function localTools(tools) {
|
|
3256
|
+
return tools.map(t => {
|
|
3257
|
+
const properties = Object.fromEntries(Object.entries(t.inputSchema.properties ?? {}).filter(([, v]) => !v._hostedOnly));
|
|
3258
|
+
return { ...t, inputSchema: { ...t.inputSchema, properties } };
|
|
3259
|
+
});
|
|
3260
|
+
}
|
|
3261
|
+
const OFFERED_TOOLS = REMOTE ? remoteTools(TOOLS) : localTools(TOOLS);
|
|
3262
|
+
|
|
3138
3263
|
// ── Server setup ──────────────────────────────────────────────────────────────
|
|
3139
3264
|
|
|
3140
|
-
const PKG =
|
|
3141
|
-
const RENA_VERSION = (
|
|
3142
|
-
try { return JSON.parse(readFileSync(_require.resolve('@qe-libs/rena-wasm/package.json'), 'utf8')).version; }
|
|
3143
|
-
catch { return 'unknown'; }
|
|
3144
|
-
})();
|
|
3265
|
+
const PKG = pkgJson();
|
|
3266
|
+
const RENA_VERSION = renaVersion();
|
|
3145
3267
|
// Stamped on every result and plot, so results from this server can be told
|
|
3146
3268
|
// apart from anything computed in the conversation.
|
|
3147
3269
|
const PROVENANCE = `${PKG.name} ${PKG.version} (rena-wasm ${RENA_VERSION})`;
|
|
@@ -3175,7 +3297,24 @@ function withholdDataValues(msg, args) {
|
|
|
3175
3297
|
|
|
3176
3298
|
// Sent to the client at connection; MCP clients (Claude, Codex, Gemini CLI,
|
|
3177
3299
|
// Cursor, ...) pass it to the model, so the privacy rule reaches every one.
|
|
3178
|
-
const
|
|
3300
|
+
const serverInstructions = () => REMOTE ? [
|
|
3301
|
+
'ENA / ONA / TMA analysis on a hosted server. The server does not run on the user\'s machine and cannot read their files.',
|
|
3302
|
+
'',
|
|
3303
|
+
'Data privacy — follow in every conversation:',
|
|
3304
|
+
'- Data reaches these tools inline, as data_csv. It is parsed in memory for the call and discarded when the call returns: the server keeps no rows and no CSV text, in memory or on disk.',
|
|
3305
|
+
`- What a model_id refers to — the fitted model: points, networks, unit labels and group values, never the rows — stays in memory for ${Math.round(MODEL_TTL_MS / 60_000)} minutes after its last use, then is forgotten. ena_simplify refits, so it takes data_csv again.`,
|
|
3306
|
+
'- Before the first analysis, tell the user that the data leaves their machine and must be de-identified: no names, e-mail addresses or identifying free text. Ask them to confirm before sending it.',
|
|
3307
|
+
'- Results include unit labels (the values of the unit columns, often participant IDs) and group values. If those are identifying, tell the user before fitting so they can pseudonymize the data first.',
|
|
3308
|
+
'- The guide may mention data_path, ena_profile, ena_tables, ena_lines or ena_export; those are for the locally installed server and are not available here.',
|
|
3309
|
+
'',
|
|
3310
|
+
'Results come only from these tools — follow in every conversation:',
|
|
3311
|
+
'- Every ENA, ONA or TMA result shown to the user (accumulated networks, points, rotations, statistics, plots) must come from these tools. Do not compute any part of an ENA analysis yourself — in code, a notebook, R, Python or by hand — and do not draw ENA-style network plots of your own.',
|
|
3312
|
+
'- If a tool fails, read its error, fix the call and retry, or tell the user what failed and stop. Never fill the gap with a substitute that looks like ENA, and never present estimated or illustrative numbers as results.',
|
|
3313
|
+
'- If the user wants something these tools cannot do, say so, and suggest the locally installed server (https://qe-libs.org/mcp/qe-mcp-ena/), rENA in R or the ENA Web Tool, rather than improvising it.',
|
|
3314
|
+
`- Results carry provenance ("${PKG.name} <version>"); anything without it did not come from this server.`,
|
|
3315
|
+
'',
|
|
3316
|
+
'Before the first analysis in a conversation, call ena_guide (no arguments) for the analysis workflow: triage, window, rotations, masking, ONA/TMA and reading the results. ena_guide with a topic returns one section.',
|
|
3317
|
+
].join('\n') : [
|
|
3179
3318
|
'ENA / ONA / TMA analysis that runs locally on the user\'s machine.',
|
|
3180
3319
|
'',
|
|
3181
3320
|
'Data privacy — follow in every conversation:',
|
|
@@ -3231,10 +3370,19 @@ const PROMPTS = [
|
|
|
3231
3370
|
},
|
|
3232
3371
|
];
|
|
3233
3372
|
|
|
3234
|
-
|
|
3235
|
-
|
|
3236
|
-
|
|
3237
|
-
|
|
3373
|
+
// Handlers are collected, then attached to each Server instance: stdio has one
|
|
3374
|
+
// for the process, HTTP mode builds one per request (stateless, so any replica
|
|
3375
|
+
// can answer any request). The caches above are per process either way.
|
|
3376
|
+
const HANDLERS = [];
|
|
3377
|
+
const on = (schema, handler) => HANDLERS.push([schema, handler]);
|
|
3378
|
+
function buildServer() {
|
|
3379
|
+
const server = new Server(
|
|
3380
|
+
{ name: 'ena', version: PKG.version },
|
|
3381
|
+
{ capabilities: { tools: {}, prompts: {} }, instructions: serverInstructions() }
|
|
3382
|
+
);
|
|
3383
|
+
for (const [schema, handler] of HANDLERS) server.setRequestHandler(schema, handler);
|
|
3384
|
+
return server;
|
|
3385
|
+
}
|
|
3238
3386
|
|
|
3239
3387
|
// ── rena-wasm API adapter ─────────────────────────────────────────────────────
|
|
3240
3388
|
// rena-wasm ≥ the R-mirroring rewrite returns an R-style ENA set object
|
|
@@ -3322,15 +3470,16 @@ function getENA() {
|
|
|
3322
3470
|
return enaPromise;
|
|
3323
3471
|
}
|
|
3324
3472
|
|
|
3325
|
-
|
|
3473
|
+
on(ListToolsRequestSchema, async () => ({ tools: OFFERED_TOOLS }));
|
|
3326
3474
|
|
|
3327
|
-
|
|
3328
|
-
|
|
3475
|
+
// The prompts take a data_path, so hosted there are none.
|
|
3476
|
+
on(ListPromptsRequestSchema, async () => ({
|
|
3477
|
+
prompts: REMOTE ? [] : PROMPTS.map(({ text, ...p }) => p),
|
|
3329
3478
|
}));
|
|
3330
3479
|
|
|
3331
|
-
|
|
3480
|
+
on(GetPromptRequestSchema, async (request) => {
|
|
3332
3481
|
const { name, arguments: a = {} } = request.params;
|
|
3333
|
-
const p = PROMPTS.find(x => x.name === name);
|
|
3482
|
+
const p = REMOTE ? null : PROMPTS.find(x => x.name === name);
|
|
3334
3483
|
if (!p) throw new Error(`Unknown prompt: ${name}`);
|
|
3335
3484
|
for (const arg of p.arguments)
|
|
3336
3485
|
if (arg.required && !a[arg.name]) throw new Error(`Missing argument: ${arg.name}`);
|
|
@@ -3340,9 +3489,12 @@ server.setRequestHandler(GetPromptRequestSchema, async (request) => {
|
|
|
3340
3489
|
};
|
|
3341
3490
|
});
|
|
3342
3491
|
|
|
3343
|
-
|
|
3492
|
+
on(CallToolRequestSchema, async (request) => {
|
|
3344
3493
|
const { name, arguments: args } = request.params;
|
|
3345
3494
|
try {
|
|
3495
|
+
if (REMOTE && REMOTE_HIDDEN_TOOLS.has(name))
|
|
3496
|
+
throw new Error(`${name} is not available on a hosted server: it writes or reads files on the user's machine. ` +
|
|
3497
|
+
'It is available in the locally installed server (https://qe-libs.org/mcp/qe-mcp-ena/).');
|
|
3346
3498
|
// rena-wasm windows look backward only (= rENA's window.size.back); a
|
|
3347
3499
|
// forward window would otherwise be silently ignored.
|
|
3348
3500
|
if (Number(args?.window_forward) > 0)
|
|
@@ -3388,7 +3540,10 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
3388
3540
|
svg = await renderSVG(_plot);
|
|
3389
3541
|
if (svg) {
|
|
3390
3542
|
// SVG file — always available (pure JS/WASM), vector, embeddable.
|
|
3391
|
-
|
|
3543
|
+
// Hosted there is no disk the user can reach: the SVG rides
|
|
3544
|
+
// along as a text block instead, after the JSON.
|
|
3545
|
+
if (REMOTE) content.push({ type: 'text', text: svg });
|
|
3546
|
+
else try { artifacts.svg_plot = writeVizFile(svg, slug, 'svg'); }
|
|
3392
3547
|
catch (e) { process.stderr.write(`[ena-mcp] SVG write failed: ${e.message}\n`); }
|
|
3393
3548
|
|
|
3394
3549
|
// PNG (inline image block + png_plot file) only when the optional
|
|
@@ -3401,11 +3556,11 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
3401
3556
|
data: png.toString('base64'),
|
|
3402
3557
|
mimeType: 'image/png',
|
|
3403
3558
|
});
|
|
3404
|
-
try { artifacts.png_plot = writeVizFile(png, slug, 'png'); }
|
|
3559
|
+
if (!REMOTE) try { artifacts.png_plot = writeVizFile(png, slug, 'png'); }
|
|
3405
3560
|
catch (e) { process.stderr.write(`[ena-mcp] PNG write failed: ${e.message}\n`); }
|
|
3406
3561
|
}
|
|
3407
3562
|
} else {
|
|
3408
|
-
renderError = 'renderSVG returned null (
|
|
3563
|
+
renderError = 'renderSVG returned null (qeviz produced no SVG)';
|
|
3409
3564
|
}
|
|
3410
3565
|
} catch (e) {
|
|
3411
3566
|
renderError = e.message;
|
|
@@ -3418,12 +3573,13 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
3418
3573
|
}
|
|
3419
3574
|
|
|
3420
3575
|
// Self-contained interactive HTML for clients that don't render inline images.
|
|
3421
|
-
try {
|
|
3576
|
+
if (!REMOTE) try {
|
|
3422
3577
|
const html = generateHTML(_plot, _plot.title ?? cleanResult.label ?? name);
|
|
3423
3578
|
artifacts.interactive_plot = writeVizFile(html, slug, 'html');
|
|
3424
3579
|
} catch (e) {
|
|
3425
3580
|
process.stderr.write(`[ena-mcp] HTML write failed: ${e.message}\n`);
|
|
3426
3581
|
}
|
|
3582
|
+
if (REMOTE) artifacts.plot_note = 'Hosted server: the plot is the inline image above (PNG) and the SVG text block below; no files were written.';
|
|
3427
3583
|
|
|
3428
3584
|
// Merge all artifact paths into the text block in one pass.
|
|
3429
3585
|
const textBlock = content.find(c => c.type === 'text');
|
|
@@ -3436,6 +3592,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
3436
3592
|
|
|
3437
3593
|
return { content };
|
|
3438
3594
|
} catch (err) {
|
|
3595
|
+
if (process.env.ENA_MCP_DEBUG) process.stderr.write((err.stack ?? '') + '\n');
|
|
3439
3596
|
return {
|
|
3440
3597
|
content: [{ type: 'text', text: `Error: ${withholdDataValues(err.message, args)}\n\n${NO_SUBSTITUTE}` }],
|
|
3441
3598
|
isError: true,
|
|
@@ -3443,7 +3600,90 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
3443
3600
|
}
|
|
3444
3601
|
});
|
|
3445
3602
|
|
|
3446
|
-
// ──
|
|
3603
|
+
// ── Web-standard entry (Workers and other fetch runtimes) ─────────────────────
|
|
3604
|
+
// Same stateless scheme as --http below, over Request/Response. A Worker
|
|
3605
|
+
// bundle imports this and never reaches the start block.
|
|
3606
|
+
|
|
3607
|
+
export { buildServer };
|
|
3608
|
+
|
|
3609
|
+
export async function handleWebRequest(request, { maxBody = HTTP_MAX_BODY } = {}) {
|
|
3610
|
+
const url = new URL(request.url);
|
|
3611
|
+
const json = (status, body) => Response.json(body, { status });
|
|
3612
|
+
if (url.pathname === '/healthz') return json(200, { ok: true, server: PKG.name, version: PKG.version });
|
|
3613
|
+
if (url.pathname !== HTTP_PATH) return json(404, { error: `MCP endpoint is ${HTTP_PATH}` });
|
|
3614
|
+
let parsedBody;
|
|
3615
|
+
if (request.method === 'POST') {
|
|
3616
|
+
const tooLarge = () => json(413, { error: `request body too large (limit ${Math.round(maxBody / 1048576)} MB)` });
|
|
3617
|
+
if (Number(request.headers.get('content-length')) > maxBody) return tooLarge();
|
|
3618
|
+
const raw = await request.text();
|
|
3619
|
+
if (raw.length > maxBody) return tooLarge();
|
|
3620
|
+
try { parsedBody = JSON.parse(raw); }
|
|
3621
|
+
catch { return json(400, { jsonrpc: '2.0', error: { code: -32700, message: 'Parse error' }, id: null }); }
|
|
3622
|
+
}
|
|
3623
|
+
const { WebStandardStreamableHTTPServerTransport } = await import('@modelcontextprotocol/sdk/server/webStandardStreamableHttp.js');
|
|
3624
|
+
const transport = new WebStandardStreamableHTTPServerTransport({ sessionIdGenerator: undefined });
|
|
3625
|
+
await buildServer().connect(transport);
|
|
3626
|
+
return transport.handleRequest(request, { parsedBody });
|
|
3627
|
+
}
|
|
3447
3628
|
|
|
3448
|
-
|
|
3449
|
-
|
|
3629
|
+
// ── Start ─────────────────────────────────────────────────────────────────────
|
|
3630
|
+
// Skipped when embedded (a Worker sets globalThis.ENA_MCP_EMBEDDED before
|
|
3631
|
+
// importing this module and calls handleWebRequest itself).
|
|
3632
|
+
|
|
3633
|
+
if (globalThis.ENA_MCP_EMBEDDED) {
|
|
3634
|
+
// nothing to start
|
|
3635
|
+
} else if (!REMOTE) {
|
|
3636
|
+
buildServer().connect(new StdioServerTransport()).catch((e) => {
|
|
3637
|
+
process.stderr.write(`[ena-mcp] stdio: ${e.message}\n`);
|
|
3638
|
+
process.exit(1);
|
|
3639
|
+
});
|
|
3640
|
+
} else {
|
|
3641
|
+
// Streamable HTTP, stateless: every POST gets a fresh Server + transport
|
|
3642
|
+
// and is answered in full, so no session state lives in the process and
|
|
3643
|
+
// any replica can take any request. Model caches are per process (see
|
|
3644
|
+
// README: one replica, or sticky sessions, keeps model_ids valid).
|
|
3645
|
+
// Reads the body up to the cap; past it the rest is drained unread so the
|
|
3646
|
+
// client gets the 413 instead of a dropped connection.
|
|
3647
|
+
const readBody = (req) => new Promise((resolve, reject) => {
|
|
3648
|
+
const chunks = []; let size = 0, tooLarge = Number(req.headers['content-length']) > HTTP_MAX_BODY;
|
|
3649
|
+
req.on('data', (c) => {
|
|
3650
|
+
size += c.length;
|
|
3651
|
+
if (tooLarge || size > HTTP_MAX_BODY) { tooLarge = true; chunks.length = 0; }
|
|
3652
|
+
else chunks.push(c);
|
|
3653
|
+
});
|
|
3654
|
+
req.on('end', () => tooLarge
|
|
3655
|
+
? reject(Object.assign(new Error(`request body too large (limit ${Math.round(HTTP_MAX_BODY / 1048576)} MB)`), { status: 413 }))
|
|
3656
|
+
: resolve(Buffer.concat(chunks).toString('utf8')));
|
|
3657
|
+
req.on('error', reject);
|
|
3658
|
+
});
|
|
3659
|
+
const json = (res, status, body) => {
|
|
3660
|
+
res.writeHead(status, { 'content-type': 'application/json' });
|
|
3661
|
+
res.end(JSON.stringify(body));
|
|
3662
|
+
};
|
|
3663
|
+
const http = createHttpServer(async (req, res) => {
|
|
3664
|
+
const url = new URL(req.url, `http://${req.headers.host ?? 'localhost'}`);
|
|
3665
|
+
if (url.pathname === '/healthz') return json(res, 200, { ok: true, server: PKG.name, version: PKG.version });
|
|
3666
|
+
if (url.pathname !== HTTP_PATH) return json(res, 404, { error: `MCP endpoint is ${HTTP_PATH}` });
|
|
3667
|
+
try {
|
|
3668
|
+
let parsedBody;
|
|
3669
|
+
if (req.method === 'POST') {
|
|
3670
|
+
const raw = await readBody(req);
|
|
3671
|
+
try { parsedBody = JSON.parse(raw); }
|
|
3672
|
+
catch { return json(res, 400, { jsonrpc: '2.0', error: { code: -32700, message: 'Parse error' }, id: null }); }
|
|
3673
|
+
}
|
|
3674
|
+
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined });
|
|
3675
|
+
res.on('close', () => transport.close());
|
|
3676
|
+
const server = buildServer();
|
|
3677
|
+
await server.connect(transport);
|
|
3678
|
+
await transport.handleRequest(req, res, parsedBody);
|
|
3679
|
+
} catch (e) {
|
|
3680
|
+
process.stderr.write(`[ena-mcp] http: ${e.message}\n`);
|
|
3681
|
+
if (!res.headersSent) json(res, e.status ?? 500, { error: e.status === 413 ? e.message : 'internal error' });
|
|
3682
|
+
else res.end();
|
|
3683
|
+
}
|
|
3684
|
+
});
|
|
3685
|
+
http.listen(HTTP_PORT, HTTP_HOST, () => {
|
|
3686
|
+
process.stderr.write(`[ena-mcp] ${PKG.name} ${PKG.version} serving MCP over HTTP at http://${HTTP_HOST}:${HTTP_PORT}${HTTP_PATH} (health: /healthz)\n`);
|
|
3687
|
+
});
|
|
3688
|
+
setInterval(expireModels, 60_000).unref(); // idle models leave on time, not at the next request
|
|
3689
|
+
}
|
package/xlsx.js
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* xlsx.js — read the first worksheet of an .xlsx file as CSV text, as the ENA
|
|
3
3
|
* Web Tool does for an uploaded .xlsx (rENA-webtool js/XlsxReader.js, ported
|
|
4
|
-
* to Node: zlib instead of JSZip,
|
|
4
|
+
* to Node: zlib instead of JSZip, a small XML reader instead of the browser's
|
|
5
|
+
* DOMParser).
|
|
5
6
|
*
|
|
6
7
|
* Parity with the web tool, and so with R's openxlsx::read.xlsx(sheet = 1):
|
|
7
8
|
* - first sheet in tab order only; its first row is the header row
|
|
@@ -11,7 +12,6 @@
|
|
|
11
12
|
*/
|
|
12
13
|
|
|
13
14
|
import { inflateRawSync } from 'zlib';
|
|
14
|
-
import { JSDOM } from 'jsdom';
|
|
15
15
|
|
|
16
16
|
// ── ZIP: the entries of an .xlsx (a ZIP of XML parts) ────────────────────────
|
|
17
17
|
function unzip(buf) {
|
|
@@ -42,10 +42,61 @@ function unzip(buf) {
|
|
|
42
42
|
return { file: (name) => files.get(name)?.() ?? null };
|
|
43
43
|
}
|
|
44
44
|
|
|
45
|
-
|
|
45
|
+
// ── XML: the little of it that SpreadsheetML needs ───────────────────────────
|
|
46
|
+
// Elements with getElementsByTagName (descendants, local name), getAttribute
|
|
47
|
+
// (prefixed or not) and textContent — the subset the readers below use. No
|
|
48
|
+
// namespaces, DTDs or entities beyond the five predefined and numeric ones.
|
|
49
|
+
const XML_ENTITIES = { lt: '<', gt: '>', amp: '&', quot: '"', apos: "'" };
|
|
50
|
+
const unescapeXml = (s) => s.indexOf('&') < 0 ? s : s.replace(/&(#x[0-9a-fA-F]+|#\d+|\w+);/g, (m, e) =>
|
|
51
|
+
e[0] === '#' ? String.fromCodePoint(e[1] === 'x' ? parseInt(e.slice(2), 16) : parseInt(e.slice(1), 10))
|
|
52
|
+
: (XML_ENTITIES[e] ?? m));
|
|
53
|
+
|
|
54
|
+
class XmlElement {
|
|
55
|
+
constructor(name, attrs) { this.name = name; this.attrs = attrs; this.children = []; }
|
|
56
|
+
get localName() { const i = this.name.indexOf(':'); return i < 0 ? this.name : this.name.slice(i + 1); }
|
|
57
|
+
getAttribute(name) {
|
|
58
|
+
if (name in this.attrs) return this.attrs[name];
|
|
59
|
+
const i = name.indexOf(':'); // "r:id" also matches an unprefixed "id"
|
|
60
|
+
return i >= 0 && name.slice(i + 1) in this.attrs ? this.attrs[name.slice(i + 1)] : null;
|
|
61
|
+
}
|
|
62
|
+
getAttributeNS(ns, local) {
|
|
63
|
+
for (const k in this.attrs) if (k === local || k.endsWith(':' + local)) return this.attrs[k];
|
|
64
|
+
return null;
|
|
65
|
+
}
|
|
66
|
+
get textContent() {
|
|
67
|
+
let out = '';
|
|
68
|
+
for (const c of this.children) out += typeof c === 'string' ? c : c.textContent;
|
|
69
|
+
return out;
|
|
70
|
+
}
|
|
71
|
+
getElementsByTagName(local) {
|
|
72
|
+
const out = [];
|
|
73
|
+
const walk = (el) => { for (const c of el.children) if (typeof c !== 'string') { if (c.localName === local) out.push(c); walk(c); } };
|
|
74
|
+
walk(this);
|
|
75
|
+
return out;
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
const ATTR_RE = /([^\s=\/>]+)\s*=\s*(?:"([^"]*)"|'([^']*)')/g;
|
|
46
80
|
function parseXml(buf) {
|
|
47
|
-
|
|
48
|
-
|
|
81
|
+
const xml = buf.toString('utf8');
|
|
82
|
+
const root = new XmlElement('#document', {});
|
|
83
|
+
const stack = [root];
|
|
84
|
+
const TAG_RE = /<(\/)?([^\s\/>!?]+)([^>]*?)(\/)?>|<!\[CDATA\[([\s\S]*?)\]\]>|<!--[\s\S]*?-->|<[!?][^>]*>/g;
|
|
85
|
+
let last = 0, m;
|
|
86
|
+
while ((m = TAG_RE.exec(xml))) {
|
|
87
|
+
const text = xml.slice(last, m.index);
|
|
88
|
+
if (text) stack[stack.length - 1].children.push(unescapeXml(text));
|
|
89
|
+
last = TAG_RE.lastIndex;
|
|
90
|
+
if (m[5] !== undefined) { stack[stack.length - 1].children.push(m[5]); continue; } // CDATA
|
|
91
|
+
if (m[2] === undefined) continue; // comment, PI, doctype
|
|
92
|
+
if (m[1]) { if (stack.length > 1) stack.pop(); continue; } // closing tag
|
|
93
|
+
const attrs = {};
|
|
94
|
+
for (const a of m[3].matchAll(ATTR_RE)) attrs[a[1]] = unescapeXml(a[2] ?? a[3] ?? '');
|
|
95
|
+
const el = new XmlElement(m[2], attrs);
|
|
96
|
+
stack[stack.length - 1].children.push(el);
|
|
97
|
+
if (!m[4]) stack.push(el);
|
|
98
|
+
}
|
|
99
|
+
return root;
|
|
49
100
|
}
|
|
50
101
|
|
|
51
102
|
// "A1" / "BC12" → 0-based column. Column letters are base-26 (bijective).
|