@ar-js-org/artoolkit5-ts 0.1.0 β 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +190 -65
- package/LICENSE +28 -28
- package/README.md +349 -268
- package/dist/artoolkit5-ts.js +123 -42
- package/dist/artoolkit5-ts.umd.cjs +1 -1
- package/dist/config.d.ts +88 -0
- package/dist/detector.d.ts +22 -0
- package/dist/domain.d.ts +111 -7
- package/dist/index.d.ts +4 -2
- package/dist/tracking.d.ts +24 -0
- package/package.json +69 -68
package/README.md
CHANGED
|
@@ -1,268 +1,349 @@
|
|
|
1
|
-
# artoolkit5-ts π―
|
|
2
|
-
|
|
3
|
-
[](https://www.npmjs.com/package/@ar-js-org/artoolkit5-ts)
|
|
4
|
-
[](LICENSE)
|
|
5
|
-
[](tsconfig.json)
|
|
6
|
-
[](test)
|
|
7
|
-
[](https://github.com/AR-js-org/artoolkit5-wasm)
|
|
8
|
-
[](#-roadmap)
|
|
9
|
-
[](https://github.com/AR-js-org/AR.js-next)
|
|
10
|
-
|
|
11
|
-
TypeScript marker tracking for the browser, built on a WebAssembly build of ARToolkit5 (WebARKitLib).
|
|
12
|
-
|
|
13
|
-
**This is the replacement for [`artoolkit5-js`](https://github.com/AR-js-org/artoolkit5-js)**, which is a fork of [`andypotato/artoolkit5-js`](https://github.com/andypotato/artoolkit5-js) β itself an ES6 module port of artoolkit5.
|
|
14
|
-
|
|
15
|
-
`artoolkit5-ts` is a rewrite rather than another fork in that line. It is written in TypeScript against a maintained WebAssembly build, and it drops the monolithic `ARController` class those ports carried forward in favour of plain data and free functions. Nothing is hidden behind a class, so nothing has to be constructed before it can be used, tested, or tree-shaken.
|
|
16
|
-
|
|
17
|
-
> β οΈ **Status: alpha.** Pattern markers work end to end, but the API is not stable yet β expect breaking changes before 1.0. See [Roadmap](#-roadmap).
|
|
18
|
-
|
|
19
|
-
## 𧩠Where this fits
|
|
20
|
-
|
|
21
|
-
`artoolkit5-ts` is the detection engine layer of the AR.js-next ecosystem:
|
|
22
|
-
|
|
23
|
-
```
|
|
24
|
-
AR.js-next ECS core, event bus, frame pump
|
|
25
|
-
arjs-plugin-artoolkit ECS plugin: Web Worker, ImageBitmap, marker events
|
|
26
|
-
artoolkit5-ts β this library
|
|
27
|
-
artoolkit5-wasm Emscripten / C++ bindings
|
|
28
|
-
artoolkit5-constants ARToolkit5 constants, extracted from the headers
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
It is renderer-agnostic and DOM-free. It gives you marker poses as matrices; what you draw with them is your business β Three.js, Babylon.js, raw WebGL, or nothing at all.
|
|
32
|
-
|
|
33
|
-
## π¦ Installation
|
|
34
|
-
|
|
35
|
-
```bash
|
|
36
|
-
npm install @ar-js-org/artoolkit5-ts
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
[`@ar-js-org/artoolkit5-wasm`](https://www.npmjs.com/package/@ar-js-org/artoolkit5-wasm) (`^0.
|
|
40
|
-
|
|
41
|
-
`three` is only needed to run the examples, not the library.
|
|
42
|
-
|
|
43
|
-
## π Quick start
|
|
44
|
-
|
|
45
|
-
```typescript
|
|
46
|
-
import {
|
|
47
|
-
createARToolKitState,
|
|
48
|
-
loadPatternMarker,
|
|
49
|
-
trackMarker,
|
|
50
|
-
processFrame,
|
|
51
|
-
} from '@ar-js-org/artoolkit5-ts';
|
|
52
|
-
|
|
53
|
-
// 1. Initialise once β loads the WASM module and camera calibration
|
|
54
|
-
const state = await createARToolKitState(640, 480, './data/camera_para.dat');
|
|
55
|
-
|
|
56
|
-
// 2. Register the markers you care about.
|
|
57
|
-
// The ID is assigned by the engine β never hardcode it.
|
|
58
|
-
const markerId = await loadPatternMarker(state, './data/patt.hiro');
|
|
59
|
-
trackMarker(state, markerId, 1.0);
|
|
60
|
-
|
|
61
|
-
// 3. Per frame: pass RGBA pixels in, get poses out.
|
|
62
|
-
// Draw your video to a canvas and read it back; the library never
|
|
63
|
-
// touches the DOM, so obtaining the pixels is your side of the line.
|
|
64
|
-
const pixels = ctx.getImageData(0, 0, 640, 480).data;
|
|
65
|
-
const { detected, lost } = processFrame(state, pixels);
|
|
66
|
-
|
|
67
|
-
for (const marker of detected) {
|
|
68
|
-
// marker.matrixGL is a 4x4 column-major right-handed matrix,
|
|
69
|
-
// ready to hand to WebGL or Three.js.
|
|
70
|
-
// With Three.js, set mesh.matrixAutoUpdate = false once beforehand,
|
|
71
|
-
// or it recomputes the matrix from position/quaternion/scale and
|
|
72
|
-
// discards the pose you just wrote.
|
|
73
|
-
mesh.matrix.fromArray(marker.matrixGL);
|
|
74
|
-
}
|
|
75
|
-
|
|
76
|
-
// `lost` holds markers that were visible last frame and are not now β
|
|
77
|
-
// reported once, on the frame they disappear
|
|
78
|
-
for (const
|
|
79
|
-
hideObjectFor(id);
|
|
80
|
-
}
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
```bash
|
|
86
|
-
npm run dev
|
|
87
|
-
```
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
## π§ Why functions instead of a controller class
|
|
92
|
-
|
|
93
|
-
The `ARController` that `artoolkit5-js` inherited from `jsartoolkit5` was a God Object: it owned the WASM module, the canvas, the video element, marker state and the render loop. That made it impossible to tree-shake, awkward to run in a Worker, and hard to test without a browser.
|
|
94
|
-
|
|
95
|
-
Here, `ARToolKitState` is a plain data container with no methods, and every operation takes it as its first argument:
|
|
96
|
-
|
|
97
|
-
- **Tree-shakeable** β you bundle only the functions you import
|
|
98
|
-
- **Testable** β functions take input and return output; no mocking a class hierarchy
|
|
99
|
-
- **Worker-friendly** β no DOM anywhere in `src/`, so state can live off the main thread
|
|
100
|
-
- **Framework-agnostic** β nothing assumes React, Vue, or any renderer
|
|
101
|
-
|
|
102
|
-
The trade-off is deliberate: this library will not open your camera, create a canvas, or run a render loop for you. Those belong to your application.
|
|
103
|
-
|
|
104
|
-
## π API
|
|
105
|
-
|
|
106
|
-
### `createARToolKitState(width, height, cameraUrl, wasmUrl?)`
|
|
107
|
-
|
|
108
|
-
Initialises the WASM module and camera parameters. Returns `Promise<ARToolKitState>`.
|
|
109
|
-
|
|
110
|
-
| Parameter | Type | Description |
|
|
111
|
-
|---|---|---|
|
|
112
|
-
| `width` | `number` | Frame width; must match the frames you pass to `processFrame` |
|
|
113
|
-
| `height` | `number` | Frame height |
|
|
114
|
-
| `cameraUrl` | `string` | URL of an ARToolKit `camera_para.dat` calibration file |
|
|
115
|
-
| `wasmUrl` | `string?` | Explicit URL for `artoolkit5.wasm`. Required when your bundler rewrites asset paths, as Vite does |
|
|
116
|
-
|
|
117
|
-
### `loadPatternMarker(state, markerUrl)`
|
|
118
|
-
|
|
119
|
-
Downloads a `.patt` file, writes it to the WASM virtual filesystem and registers it. Returns `Promise<number>` β the engine-assigned marker ID.
|
|
120
|
-
|
|
121
|
-
Loading a marker does not start tracking it; pass the ID to `trackMarker`.
|
|
122
|
-
|
|
123
|
-
### `trackMarker(state, pattId, markerWidth?)`
|
|
124
|
-
|
|
125
|
-
Registers a marker for tracking and allocates its reusable pose buffers.
|
|
126
|
-
|
|
127
|
-
`markerWidth` defaults to `1.0`. Whatever unit you choose here is the unit all returned translations are expressed in β use millimetres if you want millimetres.
|
|
128
|
-
|
|
129
|
-
### `
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
1
|
+
# artoolkit5-ts π―
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/@ar-js-org/artoolkit5-ts)
|
|
4
|
+
[](LICENSE)
|
|
5
|
+
[](tsconfig.json)
|
|
6
|
+
[](test)
|
|
7
|
+
[](https://github.com/AR-js-org/artoolkit5-wasm)
|
|
8
|
+
[](#-roadmap)
|
|
9
|
+
[](https://github.com/AR-js-org/AR.js-next)
|
|
10
|
+
|
|
11
|
+
TypeScript marker tracking for the browser, built on a WebAssembly build of ARToolkit5 (WebARKitLib).
|
|
12
|
+
|
|
13
|
+
**This is the replacement for [`artoolkit5-js`](https://github.com/AR-js-org/artoolkit5-js)**, which is a fork of [`andypotato/artoolkit5-js`](https://github.com/andypotato/artoolkit5-js) β itself an ES6 module port of artoolkit5.
|
|
14
|
+
|
|
15
|
+
`artoolkit5-ts` is a rewrite rather than another fork in that line. It is written in TypeScript against a maintained WebAssembly build, and it drops the monolithic `ARController` class those ports carried forward in favour of plain data and free functions. Nothing is hidden behind a class, so nothing has to be constructed before it can be used, tested, or tree-shaken.
|
|
16
|
+
|
|
17
|
+
> β οΈ **Status: alpha.** Pattern markers work end to end, but the API is not stable yet β expect breaking changes before 1.0. See [Roadmap](#-roadmap).
|
|
18
|
+
|
|
19
|
+
## 𧩠Where this fits
|
|
20
|
+
|
|
21
|
+
`artoolkit5-ts` is the detection engine layer of the AR.js-next ecosystem:
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
AR.js-next ECS core, event bus, frame pump
|
|
25
|
+
arjs-plugin-artoolkit ECS plugin: Web Worker, ImageBitmap, marker events
|
|
26
|
+
artoolkit5-ts β this library
|
|
27
|
+
artoolkit5-wasm Emscripten / C++ bindings
|
|
28
|
+
artoolkit5-constants ARToolkit5 constants, extracted from the headers
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
It is renderer-agnostic and DOM-free. It gives you marker poses as matrices; what you draw with them is your business β Three.js, Babylon.js, raw WebGL, or nothing at all.
|
|
32
|
+
|
|
33
|
+
## π¦ Installation
|
|
34
|
+
|
|
35
|
+
```bash
|
|
36
|
+
npm install @ar-js-org/artoolkit5-ts
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
[`@ar-js-org/artoolkit5-wasm`](https://www.npmjs.com/package/@ar-js-org/artoolkit5-wasm) (`^0.3.0`) provides the WebAssembly engine. It installs automatically as a dependency, and is left external rather than bundled so the `.wasm` binary is fetched once and cached instead of being copied into every bundle that depends on it.
|
|
40
|
+
|
|
41
|
+
`three` is only needed to run the examples, not the library.
|
|
42
|
+
|
|
43
|
+
## π Quick start
|
|
44
|
+
|
|
45
|
+
```typescript
|
|
46
|
+
import {
|
|
47
|
+
createARToolKitState,
|
|
48
|
+
loadPatternMarker,
|
|
49
|
+
trackMarker,
|
|
50
|
+
processFrame,
|
|
51
|
+
} from '@ar-js-org/artoolkit5-ts';
|
|
52
|
+
|
|
53
|
+
// 1. Initialise once β loads the WASM module and camera calibration
|
|
54
|
+
const state = await createARToolKitState(640, 480, './data/camera_para.dat');
|
|
55
|
+
|
|
56
|
+
// 2. Register the markers you care about.
|
|
57
|
+
// The ID is assigned by the engine β never hardcode it.
|
|
58
|
+
const markerId = await loadPatternMarker(state, './data/patt.hiro');
|
|
59
|
+
trackMarker(state, markerId, 1.0);
|
|
60
|
+
|
|
61
|
+
// 3. Per frame: pass RGBA pixels in, get poses out.
|
|
62
|
+
// Draw your video to a canvas and read it back; the library never
|
|
63
|
+
// touches the DOM, so obtaining the pixels is your side of the line.
|
|
64
|
+
const pixels = ctx.getImageData(0, 0, 640, 480).data;
|
|
65
|
+
const { detected, lost } = processFrame(state, pixels);
|
|
66
|
+
|
|
67
|
+
for (const marker of detected) {
|
|
68
|
+
// marker.matrixGL is a 4x4 column-major right-handed matrix,
|
|
69
|
+
// ready to hand to WebGL or Three.js.
|
|
70
|
+
// With Three.js, set mesh.matrixAutoUpdate = false once beforehand,
|
|
71
|
+
// or it recomputes the matrix from position/quaternion/scale and
|
|
72
|
+
// discards the pose you just wrote.
|
|
73
|
+
mesh.matrix.fromArray(marker.matrixGL);
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
// `lost` holds markers that were visible last frame and are not now β
|
|
77
|
+
// reported once, on the frame they disappear
|
|
78
|
+
for (const marker of lost) {
|
|
79
|
+
hideObjectFor(marker.type, marker.id);
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Two complete working examples β webcam capture, marker tracking and a Three.js cube overlay:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
npm run dev
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
[`examples/webcam`](examples/webcam) tracks a pattern marker; you will need the [Hiro marker](https://commons.wikimedia.org/wiki/File:Hiro_marker_wikipedia.png) printed or on a second screen. [`examples/barcode`](examples/barcode) tracks a matrix code marker instead β the marker image it needs ships in `examples/barcode/data/`.
|
|
90
|
+
|
|
91
|
+
## π§ Why functions instead of a controller class
|
|
92
|
+
|
|
93
|
+
The `ARController` that `artoolkit5-js` inherited from `jsartoolkit5` was a God Object: it owned the WASM module, the canvas, the video element, marker state and the render loop. That made it impossible to tree-shake, awkward to run in a Worker, and hard to test without a browser.
|
|
94
|
+
|
|
95
|
+
Here, `ARToolKitState` is a plain data container with no methods, and every operation takes it as its first argument:
|
|
96
|
+
|
|
97
|
+
- **Tree-shakeable** β you bundle only the functions you import
|
|
98
|
+
- **Testable** β functions take input and return output; no mocking a class hierarchy
|
|
99
|
+
- **Worker-friendly** β no DOM anywhere in `src/`, so state can live off the main thread
|
|
100
|
+
- **Framework-agnostic** β nothing assumes React, Vue, or any renderer
|
|
101
|
+
|
|
102
|
+
The trade-off is deliberate: this library will not open your camera, create a canvas, or run a render loop for you. Those belong to your application.
|
|
103
|
+
|
|
104
|
+
## π API
|
|
105
|
+
|
|
106
|
+
### `createARToolKitState(width, height, cameraUrl, wasmUrl?)`
|
|
107
|
+
|
|
108
|
+
Initialises the WASM module and camera parameters. Returns `Promise<ARToolKitState>`.
|
|
109
|
+
|
|
110
|
+
| Parameter | Type | Description |
|
|
111
|
+
|---|---|---|
|
|
112
|
+
| `width` | `number` | Frame width; must match the frames you pass to `processFrame` |
|
|
113
|
+
| `height` | `number` | Frame height |
|
|
114
|
+
| `cameraUrl` | `string` | URL of an ARToolKit `camera_para.dat` calibration file |
|
|
115
|
+
| `wasmUrl` | `string?` | Explicit URL for `artoolkit5.wasm`. Required when your bundler rewrites asset paths, as Vite does |
|
|
116
|
+
|
|
117
|
+
### `loadPatternMarker(state, markerUrl)`
|
|
118
|
+
|
|
119
|
+
Downloads a `.patt` file, writes it to the WASM virtual filesystem and registers it. Returns `Promise<number>` β the engine-assigned marker ID.
|
|
120
|
+
|
|
121
|
+
Loading a marker does not start tracking it; pass the ID to `trackMarker`.
|
|
122
|
+
|
|
123
|
+
### `trackMarker(state, pattId, markerWidth?)`
|
|
124
|
+
|
|
125
|
+
Registers a marker for tracking and allocates its reusable pose buffers.
|
|
126
|
+
|
|
127
|
+
`markerWidth` defaults to `1.0`. Whatever unit you choose here is the unit all returned translations are expressed in β use millimetres if you want millimetres.
|
|
128
|
+
|
|
129
|
+
### `trackBarcodeMarker(state, barcodeId, markerWidth?)`
|
|
130
|
+
|
|
131
|
+
Registers a barcode (matrix code) marker for tracking. Unlike a pattern marker, there is nothing to load first: the ID is encoded directly in the marker's geometry, so `barcodeId` is a value you choose when generating the marker, not one the engine assigns β pass it straight to this function.
|
|
132
|
+
|
|
133
|
+
Detecting a barcode marker also requires `configureDetector` to have set a matrix-capable `detectionMode` (`'matrix'`, `'color_and_matrix'`, or `'mono_and_matrix'`) and a `matrixCodeType` matching the marker.
|
|
134
|
+
|
|
135
|
+
Pattern and barcode markers have **independent ID spaces**, and are kept in separate registries. Pattern IDs are assigned by the engine starting at 0; barcode IDs are encoded in the marker's own geometry and chosen by whoever printed it. So `7` in one family is unrelated to `7` in the other, and both can be tracked at once:
|
|
136
|
+
|
|
137
|
+
```typescript
|
|
138
|
+
// Pattern IDs come from the engine β never hardcode them
|
|
139
|
+
const patternId = await loadPatternMarker(state, './data/patt.hiro');
|
|
140
|
+
trackMarker(state, patternId); // -> state.patternMarkers
|
|
141
|
+
|
|
142
|
+
// Barcode IDs are yours: encoded in the marker you printed
|
|
143
|
+
trackBarcodeMarker(state, 0); // -> state.barcodeMarkers
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
If `patternId` also happens to be `0` β and it usually is, since the engine
|
|
147
|
+
assigns from zero β both are tracked independently. Detections and losses
|
|
148
|
+
carry `type`, so you can always tell which family a result came from.
|
|
149
|
+
|
|
150
|
+
The engine reports each family through its own field (`idPatt` / `idMatrix`), so a detection is only ever matched against the registry it belongs to.
|
|
151
|
+
|
|
152
|
+
### `configureDetector(state, opts)`
|
|
153
|
+
|
|
154
|
+
Tunes the underlying detector. Only the keys you pass are changed β call it again later with a single option to adjust just that one, mid-session.
|
|
155
|
+
|
|
156
|
+
```typescript
|
|
157
|
+
configureDetector(state, {
|
|
158
|
+
detectionMode: 'matrix', // 'color' | 'mono' | 'matrix' | 'color_and_matrix' | 'mono_and_matrix'
|
|
159
|
+
matrixCodeType: '4x4_BCH_13_9_3',
|
|
160
|
+
thresholdMode: 'auto_otsu', // 'manual' | 'auto_median' | 'auto_otsu' | 'auto_bracketing'
|
|
161
|
+
threshold: 100, // 0β255, only meaningful when thresholdMode is 'manual'
|
|
162
|
+
labelingMode: 'black_region', // 'white_region' | 'black_region' β the engine default
|
|
163
|
+
imageProcMode: 'frame', // 'frame' | 'field'
|
|
164
|
+
patternRatio: 0.5, // > 0 and < 1, exclusive
|
|
165
|
+
nearPlane: 1,
|
|
166
|
+
farPlane: 1000,
|
|
167
|
+
minConfidence: { pattern: 0, barcode: 0 }, // 0β1 per family; see below
|
|
168
|
+
// before choosing a value
|
|
169
|
+
});
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
An invalid string value or an out-of-range `threshold`/`patternRatio` throws `ARToolKitError` naming the option and, for string options, listing what it does accept β the engine itself would otherwise silently ignore the bad value and keep its previous setting, which is a much harder bug to notice.
|
|
173
|
+
|
|
174
|
+
`'auto_adaptive'` threshold mode is not offered: the WebARKitLib build this library ships compiles that mode's implementation out, so passing it would silently degrade to `'manual'` while claiming to work.
|
|
175
|
+
|
|
176
|
+
#### `minConfidence` β rejecting weak matches
|
|
177
|
+
|
|
178
|
+
Every other option here is handed to the engine. `minConfidence` is the exception: ARToolKit's own confidence cutoff is a compile-time constant with no setter, so this threshold is applied by `processFrame` instead. It can only ever be *stricter* than the engine's built-in 0.5.
|
|
179
|
+
|
|
180
|
+
The two families take separate thresholds because their confidences are not comparable.
|
|
181
|
+
|
|
182
|
+
**There is no safe default value, and this library does not ship one.** Measured on a real camera with a Hiro pattern marker and 3x3 matrix markers:
|
|
183
|
+
|
|
184
|
+
| | genuine match | false match |
|
|
185
|
+
|---|---|---|
|
|
186
|
+
| **pattern** (template matching) | 0.506 β 0.923 | 0.526 β 0.554 *(read off a barcode square)* |
|
|
187
|
+
| **barcode** (matrix code) | 0.500 β 1.000 | 0.633 β 0.867 *(read off a pattern square)* |
|
|
188
|
+
|
|
189
|
+
Both ranges **overlap**, in both directions. A genuine pattern match scored `0.506`, below a false one at `0.554`. A genuine barcode scored `0.500` at an awkward angle while a phantom barcode β the engine decoding a Hiro marker's interior as a 3x3 grid β reached `0.867`. The same barcode marker, in the same detection mode minutes apart, ranged from `0.500` to `0.967` purely on viewing angle and focus.
|
|
190
|
+
|
|
191
|
+
So confidence is a continuous quality score, not a verdict, for **both** families. Matrix codes are *not* digital in this respect: a clean decode does not imply `1.0`.
|
|
192
|
+
|
|
193
|
+
What that means in practice:
|
|
194
|
+
|
|
195
|
+
- Both thresholds default to `0` β nothing is filtered beyond the engine's own 0.5 cutoff.
|
|
196
|
+
- Any threshold you set trades missed real markers against admitted phantoms. There is no value that avoids both.
|
|
197
|
+
- Measure **your** markers, in **your** lighting, at the angles you expect. Log `marker.confidence` for a while before choosing a number.
|
|
198
|
+
- A threshold is most defensible when you control the conditions β fixed mounting, known print quality, consistent lighting β and least defensible in an uncontrolled environment.
|
|
199
|
+
|
|
200
|
+
### `processFrame(state, videoFrame)`
|
|
201
|
+
|
|
202
|
+
Detects registered markers in one frame. Returns a `FrameResult`:
|
|
203
|
+
|
|
204
|
+
```typescript
|
|
205
|
+
interface FrameResult {
|
|
206
|
+
detected: MarkerPose[]; // visible in this frame
|
|
207
|
+
lost: LostMarker[]; // { id, type } visible last frame, gone in this one
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
Each `lost` entry carries `type` as well as `id`, because the two families have independent ID spaces β a pattern `7` and a barcode `7` may both be registered, and an ID alone could not say which disappeared.
|
|
212
|
+
|
|
213
|
+
`lost` is reported **exactly once**, on the frame a marker disappears β it does not repeat while the marker stays absent. Tracking already computes this transition internally, so exposing it saves every consumer from diffing successive results to recover it.
|
|
214
|
+
|
|
215
|
+
`videoFrame` is a `Uint8ClampedArray` of RGBA pixels matching the width and height the state was created with β typically `ctx.getImageData(...).data`.
|
|
216
|
+
|
|
217
|
+
This runs on every animation frame and allocates no typed arrays: poses are written into buffers owned by the marker's tracking state, and **those buffers are reused next frame**. Copy the values if you need to retain them.
|
|
218
|
+
|
|
219
|
+
### `disposeARToolKitState(state)`
|
|
220
|
+
|
|
221
|
+
Releases the WASM resources the state holds. Call it when tracking stops β otherwise a page that starts and stops AR leaks the C++ instance and its heap allocations every time.
|
|
222
|
+
|
|
223
|
+
```typescript
|
|
224
|
+
const state = await createARToolKitState(640, 480, cameraUrl);
|
|
225
|
+
// β¦ track markers β¦
|
|
226
|
+
disposeARToolKitState(state);
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Safe to call more than once. Afterwards every other operation on that state throws `ARToolKitError` rather than reaching freed memory, so a use-after-dispose gives you a clear message instead of a crash inside the WASM module.
|
|
230
|
+
|
|
231
|
+
### `ARToolKitError`
|
|
232
|
+
|
|
233
|
+
Thrown for misuse of this API β currently, using a state after disposing it. Distinct from a plain `Error` so you can tell an API mistake apart from a failure inside the WASM module or your own code.
|
|
234
|
+
|
|
235
|
+
### `getCameraProjectionMatrix(state)`
|
|
236
|
+
|
|
237
|
+
Returns the 4Γ4 projection matrix ARToolKit computed from your `camera_para.dat`, as a `Float64Array`. Use it in place of a generic perspective camera: it carries the measured focal length and principal point of the actual lens, so rendered geometry lines up with the video rather than merely sitting near it. (Radial distortion is not part of this matrix β no projection matrix can express it. ARToolKit corrects for it separately, when un-distorting detected marker corners.)
|
|
238
|
+
|
|
239
|
+
### `transMatToGLMat(transMat, out?)` / `arglCameraViewRHf(glMatrix, out?, scale?)`
|
|
240
|
+
|
|
241
|
+
Matrix helpers, exported because they are occasionally useful directly. `processFrame` already applies both.
|
|
242
|
+
|
|
243
|
+
ARToolKit produces a 3Γ4 row-major pose; WebGL wants a 4Γ4 column-major matrix in a right-handed system. `transMatToGLMat` expands the matrix, `arglCameraViewRHf` negates the Y and Z axes. Without the second step, poses render behind the camera.
|
|
244
|
+
|
|
245
|
+
Both take an optional output buffer β supply one in hot paths to avoid allocating.
|
|
246
|
+
|
|
247
|
+
### Types
|
|
248
|
+
|
|
249
|
+
`ARToolKitState`, `MarkerPose`, `FrameResult`, `LostMarker`, `TrackedMarkerState`, `MarkerType`, plus `ARToolKitModule`, `ARToolKitCore` and `MarkerInfo` describing the WASM boundary. `DetectorOptions` and its option types (`DetectionMode`, `MatrixCodeType`, `ThresholdMode`, `LabelingMode`, `ImageProcMode`) describe `configureDetector`'s input.
|
|
250
|
+
|
|
251
|
+
```typescript
|
|
252
|
+
interface MarkerPose {
|
|
253
|
+
id: number;
|
|
254
|
+
type: 'pattern' | 'barcode';
|
|
255
|
+
confidence: number; // 0β1, from this marker's own family
|
|
256
|
+
matrix: Float64Array; // 3x4, row-major, as ARToolKit produces it
|
|
257
|
+
matrixGL: Float32Array; // 4x4, column-major, right-handed, WebGL-ready
|
|
258
|
+
}
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
`confidence` is read from the field belonging to the marker's family β `cfPatt` or `cfMatrix` β so it is comparable within a family but not across them. See [`minConfidence`](#minconfidence--rejecting-weak-matches) for measured ranges.
|
|
262
|
+
|
|
263
|
+
`type` says which family a detection came from. The engine reports the two through separate fields β `idPatt` for pattern markers, `idMatrix` for barcode markers β and each is matched only against its own registry, so `type` follows from which registry answered rather than from any value the engine supplies. This is also why the families have independent ID spaces: the same integer in each is two unrelated markers.
|
|
264
|
+
|
|
265
|
+
## πΌοΈ Feeding frames from an ImageBitmap
|
|
266
|
+
|
|
267
|
+
`processFrame` takes raw pixels, so `src/` never touches a canvas API. If your frames arrive as `ImageBitmap` β as they do in AR.js-next β convert them yourself, reusing one canvas rather than creating one per frame:
|
|
268
|
+
|
|
269
|
+
```typescript
|
|
270
|
+
// The same dimensions the state was created with. Reading back any other
|
|
271
|
+
// size gives processFrame a buffer it will misinterpret.
|
|
272
|
+
const WIDTH = 640;
|
|
273
|
+
const HEIGHT = 480;
|
|
274
|
+
|
|
275
|
+
const canvas = new OffscreenCanvas(WIDTH, HEIGHT);
|
|
276
|
+
const ctx = canvas.getContext('2d', { willReadFrequently: true })!;
|
|
277
|
+
|
|
278
|
+
function toPixels(bitmap: ImageBitmap): Uint8ClampedArray {
|
|
279
|
+
ctx.drawImage(bitmap, 0, 0, WIDTH, HEIGHT);
|
|
280
|
+
return ctx.getImageData(0, 0, WIDTH, HEIGHT).data;
|
|
281
|
+
}
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
A helper that does this is on the roadmap; until then it is a few lines you own.
|
|
285
|
+
|
|
286
|
+
## β οΈ Limitations
|
|
287
|
+
|
|
288
|
+
- **Combined detection requires `@ar-js-org/artoolkit5-wasm` >= 0.3.0.** `'color_and_matrix'` and `'mono_and_matrix'` rely on the per-mode marker fields (`idPatt`/`idMatrix`), which earlier versions of the binding did not expose β against `0.2.0` or older those modes silently detect nothing, or report the wrong marker. The dependency range already requires `^0.3.0`; this matters only if you override it.
|
|
289
|
+
- **Worker support is untested.** Nothing in `src/` touches the DOM, which is necessary but not proof β WASM instantiation in worker scope has not been verified.
|
|
290
|
+
- **NFT markers are out of scope** for this project β see [Roadmap](#-roadmap).
|
|
291
|
+
|
|
292
|
+
## πΊοΈ Roadmap
|
|
293
|
+
|
|
294
|
+
Detailed design lives in [`docs/DESIGN-v0.1.md`](docs/DESIGN-v0.1.md) and, for the detector and barcode work, [`docs/DESIGN-detector-and-barcode.md`](docs/DESIGN-detector-and-barcode.md); work is tracked in [issues](https://github.com/AR-js-org/artoolkit5-ts/issues).
|
|
295
|
+
|
|
296
|
+
**v0.1** (done) β lifecycle, packaging, marker-lost reporting from `processFrame`, a test suite and CI.
|
|
297
|
+
|
|
298
|
+
**v0.2** (done) β `configureDetector`, barcode markers, independent ID registries for the two families, combined pattern+barcode detection verified against a real camera, and per-family match confidence.
|
|
299
|
+
|
|
300
|
+
**Next** β a verified Worker example, an `ImageBitmap` conversion helper, and multi-marker sets.
|
|
301
|
+
|
|
302
|
+
**Out of scope** β NFT tracking. This project and `artoolkit5-wasm` cover pattern and barcode markers; NFT belongs to other projects in the ecosystem.
|
|
303
|
+
|
|
304
|
+
## π οΈ Development
|
|
305
|
+
|
|
306
|
+
```bash
|
|
307
|
+
npm run dev # Vite dev server, opens the webcam example
|
|
308
|
+
npm run build # library build (ES + UMD) plus type declarations
|
|
309
|
+
npm run preview # preview the production build
|
|
310
|
+
npm test # run the test suite once
|
|
311
|
+
npm run test:watch # re-run tests on change
|
|
312
|
+
npm run typecheck # tsc --noEmit
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
### Tests
|
|
316
|
+
|
|
317
|
+
[Vitest](https://vitest.dev) covers the matrix maths, the marker visibility state machine and the dispose lifecycle. The suite runs in well under a second because the WASM boundary is faked: `test/mock-core.ts` stands in for the Emscripten module and the bound C++ instance, so neither a browser nor a compiled binary is needed.
|
|
318
|
+
|
|
319
|
+
The suite aims at the code that fails *quietly* rather than at a line-count target β a transposed matrix still renders, just in the wrong place, and a marker-lost event that fires twice looks fine until something downstream double-handles it.
|
|
320
|
+
|
|
321
|
+
It is validated by mutation: deliberately breaking the collection order, the `Float32Array` return type, or the continuous-tracking condition each makes exactly one test fail. If you add tests, check they can actually fail.
|
|
322
|
+
|
|
323
|
+
### Contributing
|
|
324
|
+
|
|
325
|
+
Branch from `dev`; `main` holds release-ready code only. Commits follow [Conventional Commits](https://www.conventionalcommits.org/). Fuller guidance is in [`AGENTS.md`](AGENTS.md).
|
|
326
|
+
|
|
327
|
+
### Releasing
|
|
328
|
+
|
|
329
|
+
Releases are cut by the **Release** workflow, run manually from the Actions tab. Its only required input is the version to publish, without a leading `v` β for example `0.1.0`.
|
|
330
|
+
|
|
331
|
+
Everything after that is automatic: it runs typecheck, tests and build, sets the version, promotes the changelog, derives release notes from the commits, commits, tags `vX.Y.Z`, creates the GitHub Release and publishes to npm with [provenance](https://docs.npmjs.com/generating-provenance-statements) β so the package carries a verifiable link back to the commit and workflow run that built it.
|
|
332
|
+
|
|
333
|
+
**Run it with `dry_run` first.** That performs every check and prints the notes and the tarball contents without tagging, committing or publishing. It is the only way to rehearse: npm never allows a published version to be replaced.
|
|
334
|
+
|
|
335
|
+
Before running for real, the workflow refuses to start unless:
|
|
336
|
+
|
|
337
|
+
- the version is valid semver, not already tagged, and not already on npm
|
|
338
|
+
- the branch is `main`
|
|
339
|
+
- the repository is public β npm will not generate provenance from a private repository
|
|
340
|
+
|
|
341
|
+
Preparing a release means writing the changelog. Add entries to `## [Unreleased]` as you go; the workflow renames that heading to the released version and opens a fresh one. Anything between `<!-- promote:strip -->` markers is dropped during promotion, so notes meant only for editors do not survive into a released section. `npm run release-notes` prints the Conventional Commits since the last tag if you want to see what has accumulated.
|
|
342
|
+
|
|
343
|
+
It is a single workflow rather than a "create release" and a "publish" pair because a Release created with the default `GITHUB_TOKEN` does not trigger other workflows β GitHub blocks that to prevent recursion, so the second one would silently never fire.
|
|
344
|
+
|
|
345
|
+
## π Licence
|
|
346
|
+
|
|
347
|
+
MIT β see [LICENSE](LICENSE).
|
|
348
|
+
|
|
349
|
+
This library wraps a WebAssembly build of **ARToolkit5 (WebARKitLib), which is licensed under the LGPL v3.0**. The MIT licence covers this TypeScript code, not the engine underneath: redistributing a build that includes the ARToolkit5 (WebARKitLib) WebAssembly binary carries that licence's obligations as well.
|