vitallens 0.0.1 → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +247 -110
- package/dist/types/index.browser.d.ts +3 -0
- package/dist/vitallens.browser.js +703 -6551
- package/dist/vitallens.cjs.js +8 -15621
- package/dist/vitallens.esm.js +8 -15622
- package/package.json +25 -8
- package/scripts/check-node-version.cjs +23 -0
package/README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# vitallens.js
|
|
2
2
|
|
|
3
|
-
[](https://github.com/Rouast-Labs/vitallens.js/actions/workflows/ci.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/vitallens)
|
|
4
5
|
[](https://www.rouast.com/api/)
|
|
5
6
|
[](https://docs.rouast.com/)
|
|
6
7
|
[](https://doi.org/10.48550/arXiv.2312.06892)
|
|
@@ -8,7 +9,7 @@
|
|
|
8
9
|
Estimate vital signs such as heart rate and respiratory rate from video in JavaScript.
|
|
9
10
|
|
|
10
11
|
`vitallens.js` is a JavaScript client for the [**VitalLens API**](https://www.rouast.com/api/), which leverages the same inference engine as our [free iOS app VitalLens](https://apps.apple.com/us/app/vitallens/id6472757649).
|
|
11
|
-
Furthermore, it includes fast implementations of several other heart rate estimation methods from video such as `
|
|
12
|
+
Furthermore, it includes fast implementations of several other heart rate estimation methods from video such as `g`, `chrom`, and `pos`.
|
|
12
13
|
|
|
13
14
|
This library works both in browser environments and in Node.js, and comes with a set of examples for file-based processing and real-time webcam streaming.
|
|
14
15
|
|
|
@@ -24,16 +25,18 @@ Using a different language or platform? We also have a [Python client](https://g
|
|
|
24
25
|
|
|
25
26
|
- **Multiple Estimation Methods:**
|
|
26
27
|
Choose the method that fits your needs:
|
|
27
|
-
- **`vitallens
|
|
28
|
-
- **`g`**, **`chrom`**, **`pos
|
|
29
|
-
- While `vitallens` requires an API Key, `g`, `chrom`, and `pos` do not. [Register on our website to get a free API Key.](https://www.rouast.com/api/)
|
|
28
|
+
- **`vitallens`**: Provides *heart rate*, *respiratory rate*, *pulse waveform*, and *respiratory waveform* estimates with associated confidences. *(Requires an API key - [get one for free on our website](https://www.rouast.com/api/))*
|
|
29
|
+
- **`g`**, **`chrom`**, **`pos`**: Offer faster (but less accurate) *heart rate* and *pulse waveform* estimates. *(No API key required.)*
|
|
30
30
|
|
|
31
31
|
- **Fast Face Detection & ROI Support:**
|
|
32
32
|
Perform rapid face detection when required—or optionally, pass a global region of interest (ROI) to skip detection for even faster processing.
|
|
33
33
|
|
|
34
34
|
- **Event-Driven API:**
|
|
35
35
|
Register event listeners to receive real-time updates on estimated vitals.
|
|
36
|
-
|
|
36
|
+
|
|
37
|
+
- **Pre-Built Web Component Widgets:**
|
|
38
|
+
In addition to the core API, vitallens.js provides ready-to-use web components. Use the unified widget (supporting both file and webcam modes) or choose the specialized file-only or webcam-only widget for streamlined integration.
|
|
39
|
+
|
|
37
40
|
- **TypeScript-Ready:**
|
|
38
41
|
Written in TypeScript with complete type definitions for enhanced developer experience.
|
|
39
42
|
|
|
@@ -45,161 +48,293 @@ Please review our [Terms of Service](https://www.rouast.com/api/terms) and [Priv
|
|
|
45
48
|
|
|
46
49
|
## Installation
|
|
47
50
|
|
|
48
|
-
|
|
51
|
+
### Node.js
|
|
52
|
+
|
|
53
|
+
Install `vitallens.js` via npm or yarn:
|
|
49
54
|
|
|
50
55
|
```bash
|
|
51
56
|
npm install vitallens
|
|
57
|
+
# or
|
|
58
|
+
yarn add vitallens
|
|
52
59
|
```
|
|
53
60
|
|
|
54
|
-
|
|
61
|
+
Then use it as follows:
|
|
55
62
|
|
|
56
|
-
```
|
|
57
|
-
|
|
63
|
+
```js
|
|
64
|
+
import { VitalLens } from 'vitallens';
|
|
65
|
+
const vl = new VitalLens({ method: 'vitallens', apiKey: 'YOUR_API_KEY' });
|
|
66
|
+
const result = await vl.processVideoFile(myVideoFile);
|
|
67
|
+
console.log(result);
|
|
58
68
|
```
|
|
59
69
|
|
|
60
|
-
|
|
70
|
+
### Browser
|
|
61
71
|
|
|
62
|
-
|
|
72
|
+
For browser usage, you can either bundle `vitallens.js` with your project or load it directly from a CDN. In addition to the core API, `vitallens.js` also provides pre-built web component widgets. We offer three variants:
|
|
63
73
|
|
|
64
|
-
|
|
74
|
+
- **Unified Widget:** Supports both file and webcam modes with mode toggles.
|
|
75
|
+
- **File-Only Widget:** For processing video files only.
|
|
76
|
+
- **Webcam-Only Widget:** For live webcam streaming only.
|
|
65
77
|
|
|
66
|
-
|
|
78
|
+
For example, using **jsDelivr**:
|
|
67
79
|
|
|
68
80
|
```html
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
81
|
+
<!-- Latest version -->
|
|
82
|
+
<script src="https://cdn.jsdelivr.net/npm/vitallens/dist/vitallens.browser.js"></script>
|
|
83
|
+
|
|
84
|
+
<!-- Or pin a specific version -->
|
|
85
|
+
<script src="https://cdn.jsdelivr.net/npm/vitallens@0.0.1/dist/vitallens.browser.js"></script>
|
|
86
|
+
|
|
87
|
+
<!-- Use with core API -->
|
|
88
|
+
<script>
|
|
89
|
+
// vitallens.js is exposed as a global, for example as window.VitalLens.
|
|
90
|
+
const vl = new VitalLens({ method: 'vitallens', apiKey: 'YOUR_API_KEY' });
|
|
91
|
+
// Suppose myMediaStream and myVideoElement are defined:
|
|
92
|
+
vl.addVideoStream(myMediaStream, myVideoElement);
|
|
93
|
+
vl.addEventListener('vitals', (data) => console.log(data));
|
|
94
|
+
vl.startVideoStream();
|
|
72
95
|
</script>
|
|
73
|
-
```
|
|
74
96
|
|
|
75
|
-
|
|
97
|
+
<!-- Or use our widget -->
|
|
98
|
+
<vitallens-widget api-key="YOUR_API_KEY"></vitallens-widget>
|
|
76
99
|
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
### Processing a Video File (Node.js Example)
|
|
100
|
+
<!-- Or, to use a specialized widget: -->
|
|
101
|
+
<!-- File-only widget -->
|
|
102
|
+
<vitallens-file-widget api-key="YOUR_API_KEY"></vitallens-file-widget>
|
|
83
103
|
|
|
84
|
-
|
|
85
|
-
|
|
104
|
+
<!-- Webcam-only widget -->
|
|
105
|
+
<vitallens-webcam-widget api-key="YOUR_API_KEY"></vitallens-webcam-widget>
|
|
106
|
+
```
|
|
86
107
|
|
|
87
|
-
|
|
88
|
-
method: 'vitallens', // Choose from 'vitallens', 'g', 'chrom', or 'pos'
|
|
89
|
-
apiKey: 'YOUR_API_KEY', // Required when using the 'vitallens' method
|
|
90
|
-
};
|
|
108
|
+
Alternatively, you can use **unpkg**:
|
|
91
109
|
|
|
92
|
-
|
|
110
|
+
```html
|
|
111
|
+
<script src="https://unpkg.com/vitallens/dist/vitallens.browser.js"></script>
|
|
112
|
+
```
|
|
93
113
|
|
|
94
|
-
|
|
95
|
-
try {
|
|
96
|
-
const result = await vitallens.processFile(filePath);
|
|
97
|
-
console.log('Processing complete!', result);
|
|
98
|
-
} catch (error) {
|
|
99
|
-
console.error('Error processing video:', error);
|
|
100
|
-
}
|
|
101
|
-
}
|
|
114
|
+
Or **Skypack** if you prefer ES modules:
|
|
102
115
|
|
|
103
|
-
|
|
116
|
+
```html
|
|
117
|
+
<script type="module">
|
|
118
|
+
import { VitalLens } from 'https://cdn.skypack.dev/vitallens';
|
|
119
|
+
// Continue as above…
|
|
120
|
+
</script>
|
|
104
121
|
```
|
|
105
122
|
|
|
106
|
-
|
|
123
|
+
## Configuration Options
|
|
107
124
|
|
|
108
|
-
|
|
125
|
+
When creating a new `VitalLens` instance, you can configure various options:
|
|
109
126
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
127
|
+
| Parameter | Description | Default |
|
|
128
|
+
| -------------- | ------------------------------------------------------------------------------------------ | -------------- |
|
|
129
|
+
| `method` | Inference method: `'vitallens'`, `'g'`, `'chrom'`, or `'pos'`. | `'vitallens'` |
|
|
130
|
+
| `apiKey` | API key for the VitalLens API (required for method `'vitallens'`). | `null` |
|
|
131
|
+
| `globalRoi` | Optional region of interest for face detection (object with `{ x0, y0, x1, y1 }`). | `undefined` |
|
|
132
|
+
| `waveformMode` | Optional setting how waveform is returned: `'incremental'`, `'windowed'`, or `'complete'`. | *(see below)* |
|
|
133
|
+
|
|
134
|
+
The default value for `waveformMode` is `windowed` if a stream is being analyzed, and `complete` if a file is being processed. Additional options (e.g., face detection settings, buffering) are available. See [docs](https://docs.rouast.com/) for details.
|
|
135
|
+
|
|
136
|
+
## Understanding the Estimation Results
|
|
137
|
+
|
|
138
|
+
When you process a video or a MediaStream with `vitallens.js`, the library returns vital sign estimates in a structured object. **vitallens.js is designed to process only a single face** — so you always receive a single result object with the following structure:
|
|
139
|
+
|
|
140
|
+
```typescript
|
|
141
|
+
export interface VitalLensResult {
|
|
142
|
+
face: {
|
|
143
|
+
// Detected face coordinates for each frame, formatted as [x0, y0, x1, y1].
|
|
144
|
+
coordinates: Array<[number, number, number, number]>;
|
|
145
|
+
// Confidence values for the face per frame.
|
|
146
|
+
confidence: number[];
|
|
147
|
+
// An explanatory note regarding the face detection.
|
|
148
|
+
note: string;
|
|
149
|
+
};
|
|
150
|
+
vital_signs: {
|
|
151
|
+
// Estimated global heart rate.
|
|
152
|
+
heart_rate: {
|
|
153
|
+
// Estimated heart rate value.
|
|
154
|
+
value: number;
|
|
155
|
+
// Unit of the heart rate value.
|
|
156
|
+
unit: string;
|
|
157
|
+
// Overall confidence of the heart rate estimation.
|
|
158
|
+
confidence: number;
|
|
159
|
+
// An explanatory note regarding the estimation.
|
|
160
|
+
note: string;
|
|
125
161
|
};
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
162
|
+
// Estimated global respiratory rate.
|
|
163
|
+
respiratory_rate?: {
|
|
164
|
+
// Estimated respiratory rate value (in breaths per minute).
|
|
165
|
+
value: number;
|
|
166
|
+
// Unit of the respiratory rate value.
|
|
167
|
+
unit: string;
|
|
168
|
+
// Overall confidence of the respiratory rate estimation.
|
|
169
|
+
confidence: number;
|
|
170
|
+
// An explanatory note regarding the estimation.
|
|
171
|
+
note: string;
|
|
172
|
+
};
|
|
173
|
+
// Photoplethysmogram (PPG) waveform estimation.
|
|
174
|
+
ppg_waveform: {
|
|
175
|
+
// Estimated PPG waveform data (one value per processed frame).
|
|
176
|
+
data: number[];
|
|
177
|
+
// Unit of the waveform data.
|
|
178
|
+
unit: string;
|
|
179
|
+
// Confidence values for the waveform estimation per frame.
|
|
180
|
+
confidence: number[];
|
|
181
|
+
// An explanatory note regarding the waveform estimation.
|
|
182
|
+
note: string;
|
|
183
|
+
};
|
|
184
|
+
// Respiratory waveform estimation.
|
|
185
|
+
respiratory_waveform?: {
|
|
186
|
+
// Estimated respiratory waveform data (one value per processed frame).
|
|
187
|
+
data: number[];
|
|
188
|
+
// Unit of the waveform data.
|
|
189
|
+
unit: string;
|
|
190
|
+
// Confidence values for the waveform estimation per frame.
|
|
191
|
+
confidence: number[];
|
|
192
|
+
// An explanatory note regarding the waveform estimation.
|
|
193
|
+
note: string;
|
|
194
|
+
};
|
|
195
|
+
};
|
|
196
|
+
// A list of timestamps (one per processed frame).
|
|
197
|
+
time: number[];
|
|
198
|
+
// The frames per second (fps) of the input video.
|
|
199
|
+
fps: number;
|
|
200
|
+
// The effective fps used for inference.
|
|
201
|
+
estFps: number;
|
|
202
|
+
// A message providing additional information about the estimation.
|
|
203
|
+
message: string;
|
|
204
|
+
}
|
|
157
205
|
```
|
|
158
206
|
|
|
159
|
-
|
|
207
|
+
## Examples
|
|
160
208
|
|
|
161
|
-
|
|
209
|
+
Before running any of the examples, make sure to build the project by executing:
|
|
162
210
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
| `apiKey` | API key for the VitalLens API (required for method `'vitallens'`). | `null` |
|
|
167
|
-
| `globalRoi` | Optional region of interest for face detection (object with `{ x0, y0, x1, y1 }`). | `undefined` |
|
|
168
|
-
| *Others* | Additional options (e.g., face detection settings, buffering) are available. See [docs](https://docs.rouast.com/) for details. | |
|
|
211
|
+
```bash
|
|
212
|
+
npm run build
|
|
213
|
+
```
|
|
169
214
|
|
|
170
|
-
|
|
215
|
+
Also, note that each example requires an API key. Replace `YOUR_API_KEY` with your actual API key when running the examples.
|
|
171
216
|
|
|
172
|
-
|
|
217
|
+
- **Browser Unified Widget:**
|
|
218
|
+
[examples/browser/widget.html](examples/browser/widget.html)
|
|
219
|
+
To run this example, execute:
|
|
220
|
+
```bash
|
|
221
|
+
API_KEY=YOUR_API_KEY npm run start:browser
|
|
222
|
+
```
|
|
173
223
|
|
|
174
|
-
- **Browser File Input:**
|
|
175
|
-
[examples/browser/
|
|
224
|
+
- **Browser File Input Widget:**
|
|
225
|
+
[examples/browser/file_widget.html](examples/browser/file_widget.html)
|
|
176
226
|
To run this example, execute:
|
|
177
227
|
```bash
|
|
178
|
-
npm run start:browser-file
|
|
228
|
+
API_KEY=YOUR_API_KEY npm run start:browser-file
|
|
179
229
|
```
|
|
180
230
|
|
|
181
|
-
- **
|
|
182
|
-
[examples/browser/
|
|
231
|
+
- **Browser File Input Minimal:**
|
|
232
|
+
[examples/browser/file_minimal.html](examples/browser/file_minimal.html)
|
|
233
|
+
To run this example, execute:
|
|
234
|
+
```bash
|
|
235
|
+
API_KEY=YOUR_API_KEY npm run start:browser-file-minimal
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
- **Browser Webcam Input Widget:**
|
|
239
|
+
[examples/browser/webcam_widget.html](examples/browser/webcam_widget.html)
|
|
183
240
|
To run this example, execute:
|
|
184
241
|
```bash
|
|
185
|
-
npm run start:browser-webcam
|
|
242
|
+
API_KEY=YOUR_API_KEY npm run start:browser-webcam
|
|
186
243
|
```
|
|
187
244
|
|
|
188
|
-
- **
|
|
189
|
-
[examples/browser/
|
|
245
|
+
- **Browser Webcam Input Minimal:**
|
|
246
|
+
[examples/browser/webcam_minimal.html](examples/browser/webcam_minimal.html)
|
|
190
247
|
To run this example, execute:
|
|
191
248
|
```bash
|
|
192
|
-
npm run start:browser-webcam
|
|
249
|
+
API_KEY=YOUR_API_KEY npm run start:browser-webcam-minimal
|
|
193
250
|
```
|
|
194
251
|
|
|
195
252
|
- **Node File Processing:**
|
|
196
253
|
[examples/node/file.js](examples/node/file.js)
|
|
197
254
|
To run this example, execute:
|
|
198
255
|
```bash
|
|
199
|
-
npm run start:node-file
|
|
256
|
+
API_KEY=YOUR_API_KEY npm run start:node-file
|
|
200
257
|
```
|
|
201
258
|
|
|
202
|
-
Try opening the HTML examples in your browser or running the Node script to see vitallens.js in action.
|
|
259
|
+
Try opening the HTML examples in your browser or running the Node script to see `vitallens.js` in action.
|
|
260
|
+
|
|
261
|
+
## Securing your API Key
|
|
262
|
+
|
|
263
|
+
For security reasons, we recommend that you do not expose your API key directly in client-side code. There are two primary approaches to secure your API key:
|
|
264
|
+
|
|
265
|
+
### 1. Run Everything on your Server
|
|
266
|
+
|
|
267
|
+
If you are building a server-side application using Node.js, your API key remains securely on your server. Simply call the API directly from your backend code without exposing your credentials.
|
|
268
|
+
|
|
269
|
+
### 2. Use a Proxy Server for Client-Side Code
|
|
270
|
+
|
|
271
|
+
If you need to use `vitallens.js` in a browser, you can set up a proxy server. The proxy server receives requests from the client, attaches your API key (stored securely on the server), and forwards the request to the VitalLens API. This way, the API key is never exposed to the client.
|
|
272
|
+
|
|
273
|
+
Our client library supports this by accepting a `proxyUrl` option. For example:
|
|
274
|
+
|
|
275
|
+
```js
|
|
276
|
+
import { VitalLens } from 'vitallens';
|
|
277
|
+
const vl = new VitalLens({
|
|
278
|
+
method: 'vitallens',
|
|
279
|
+
proxyUrl: 'https://your-proxy-server.com/api' // URL to your deployed proxy server
|
|
280
|
+
});
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Or when using one of our widgets:
|
|
284
|
+
|
|
285
|
+
```html
|
|
286
|
+
<vitallens-widget proxy-url="https://your-proxy-server.com/api"></vitallens-widget>
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
### Sample Proxy Server Implementation
|
|
290
|
+
|
|
291
|
+
Below is a simple Node.js/Express proxy server implementation that you can use as a starting point:
|
|
292
|
+
|
|
293
|
+
```js
|
|
294
|
+
const express = require('express');
|
|
295
|
+
const bodyParser = require('body-parser');
|
|
296
|
+
const cors = require('cors');
|
|
297
|
+
|
|
298
|
+
const app = express();
|
|
299
|
+
const PORT = process.env.PORT || 3000;
|
|
300
|
+
|
|
301
|
+
// Securely store your API key in an environment variable
|
|
302
|
+
const API_KEY = process.env.VITALLENS_API_KEY;
|
|
303
|
+
const VITALLENS_ENDPOINT = 'https://api.rouast.com/vitallens-v2';
|
|
304
|
+
|
|
305
|
+
app.use(bodyParser.json({ limit: '10mb' }));
|
|
306
|
+
|
|
307
|
+
// Enable CORS for your allowed domain.
|
|
308
|
+
app.use(cors({
|
|
309
|
+
origin: 'http://example.com', // Your allowed domain
|
|
310
|
+
methods: ['GET', 'POST', 'OPTIONS'],
|
|
311
|
+
allowedHeaders: ['Content-Type', 'Authorization']
|
|
312
|
+
}));
|
|
313
|
+
|
|
314
|
+
app.post('/', async (req, res) => {
|
|
315
|
+
try {
|
|
316
|
+
const response = await fetch(VITALLENS_ENDPOINT, {
|
|
317
|
+
method: 'POST',
|
|
318
|
+
headers: {
|
|
319
|
+
'Content-Type': 'application/json',
|
|
320
|
+
'x-api-key': API_KEY,
|
|
321
|
+
},
|
|
322
|
+
body: JSON.stringify(req.body),
|
|
323
|
+
});
|
|
324
|
+
const data = await response.text();
|
|
325
|
+
res.status(response.status).send(data);
|
|
326
|
+
} catch (error) {
|
|
327
|
+
console.error('Proxy error:', error);
|
|
328
|
+
res.status(500).send('Internal server error');
|
|
329
|
+
}
|
|
330
|
+
});
|
|
331
|
+
|
|
332
|
+
app.listen(PORT, () => {
|
|
333
|
+
console.log(`Proxy server listening on port ${PORT}`);
|
|
334
|
+
});
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
You can deploy this proxy server on any Node.js hosting platform (such as Heroku, Vercel, or your own server) and then set the URL as the `proxyUrl` in your VitalLens client configuration.
|
|
203
338
|
|
|
204
339
|
## Development
|
|
205
340
|
|
|
@@ -226,6 +361,8 @@ For environment-specific tests, you can use:
|
|
|
226
361
|
```bash
|
|
227
362
|
npm run test:browser
|
|
228
363
|
npm run test:node
|
|
364
|
+
npm run test:browser-integration
|
|
365
|
+
npm run test:node-integration
|
|
229
366
|
```
|
|
230
367
|
|
|
231
368
|
### Linting
|
|
@@ -1,3 +1,6 @@
|
|
|
1
|
+
import './web-components/VitalLensWidgetUnified';
|
|
2
|
+
import './web-components/VitalLensWidgetFile';
|
|
3
|
+
import './web-components/VitalLensWidgetWebcam';
|
|
1
4
|
export { VitalLens } from './core/VitalLens.browser';
|
|
2
5
|
export type { VitalLensOptions, VitalLensResult } from './types/core';
|
|
3
6
|
export { Frame } from './processing/Frame';
|