@hachej/boring-ui-kit 0.4.4 → 0.4.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -0
- package/dist/artifact-viewer.d.ts +18 -0
- package/dist/artifact-viewer.js +125 -0
- package/dist/html-preview.d.ts +1 -1
- package/dist/html-preview.js +1 -47
- package/dist/passive-html.d.ts +2 -0
- package/dist/passive-html.js +47 -0
- package/dist/voice-capture.d.ts +43 -0
- package/dist/voice-capture.js +186 -0
- package/package.json +10 -2
- package/r/agent-target.json +4 -4
- package/r/app-shell.json +9 -9
- package/r/board-viewer.json +5 -5
- package/r/button.json +1 -1
- package/r/diagram-viewer.json +1 -1
- package/r/dock.json +1 -1
- package/r/documents.json +10 -10
- package/r/feedback-preview.json +1 -1
- package/r/feedback.json +5 -5
- package/r/file-tree.json +5 -5
- package/r/highlights.json +3 -3
- package/r/html-viewer.json +3 -3
- package/r/maquette-viewer.json +2 -2
- package/r/markdown-editor.json +3 -3
- package/r/mockup-viewer.json +1 -1
- package/r/pi-ambient.json +6 -6
- package/r/pi-app.json +16 -15
- package/r/pi-chat.json +37 -15
- package/r/pi-workspace.json +4 -4
- package/r/provider-setup.json +2 -2
- package/r/record-viewer.json +1 -1
- package/r/registry.json +148 -97
- package/r/requests-board.json +1 -1
- package/r/shared-chat.json +4 -4
- package/r/task-list-viewer.json +2 -2
- package/r/utils.json +6 -0
- package/r/version-history.json +3 -3
- package/r/viewers.json +6 -6
- package/r/windows.json +28 -0
package/README.md
CHANGED
|
@@ -12,6 +12,8 @@ Lost or malformed acknowledgements stay unknown. `actions.reconcile()` uses reta
|
|
|
12
12
|
|
|
13
13
|
Historical scaffold tests install/typecheck packed headless UI and UI+files in clean consumers without Pi or an agent package. Candidate runtime installation qualification is recorded separately in the [implementation checkpoint](../../docs/implementation/PARTIAL.md). Import interfaces with `import type`.
|
|
14
14
|
|
|
15
|
+
For read-only content without a resource client or controller, select the lightweight [`artifact-viewer`](#read-only-artifactviewer) entry. It needs only React and Marked, not files, Pi, Tiptap, canvas or pdfjs.
|
|
16
|
+
|
|
15
17
|
See [scaffold guide](../../docs/contracts/SCAFFOLD.md) and [audit](../../docs/contracts/ABSTRACTION-REVIEW.md).
|
|
16
18
|
|
|
17
19
|
## Exact proposals and commands
|
|
@@ -68,6 +70,14 @@ The default schema includes headings, lists, task lists, quotes, code and text m
|
|
|
68
70
|
|
|
69
71
|
`test/packages/ui-markdown-editor.test.mjs`, `test/packages/ui-markdown-navigation.test.mjs` and `test/packages/ui-markdown-mounted.test.mjs` and `test/packages/ui-markdown-proposals.test.mjs` drive the real React/Tiptap renderer, DOM selections and SQLite provider. `npm run test:editor-consumer` requires `npm_config_cache` pointing to a writable cache with the pinned archives; it installs actual package tarballs, checks declarations with `skipLibCheck: false`, drives the controls and bundles for browsers without Pi or server filesystem imports. These checks do not establish browser layout, caret, IME or accessibility qualification. Chromium journeys remain pending.
|
|
70
72
|
|
|
73
|
+
## Optional microphone capture
|
|
74
|
+
|
|
75
|
+
`createVoiceCapture` from `@hachej/boring-ui-kit/voice-capture` records browser audio without React, files or Pi. Call `start()` from a person's gesture. It feature-detects microphone access and returns a recording or a typed refusal. `stop()` returns the Blob, recorded MIME type, page-clock sample times and `truncated`; `cancel()` discards audio and releases only this capture's microphone tracks, including a permission request that finishes late.
|
|
76
|
+
|
|
77
|
+
`maxMs` caps capture at five minutes by default. Reaching the cap stops the microphone; callers still choose when to consume the recording through `stop()`. Transcription and persistence remain host capabilities. The existing `@hachej/boring-feedback/page` capture API re-exports this implementation unchanged. The headless root imports neither entry.
|
|
78
|
+
|
|
79
|
+
The [registry voice-message guide](../../registry/README.md#voice-messages-in-pi-chat) describes PiChat's optional hold-to-send composition. Public capture behavior runs in `test/packages/ui.test.mjs`, compatibility in `test/packages/feedback-voice.test.mjs`, and dependency-free installation in `test/scaffold/isolated-ui.test.mjs`.
|
|
80
|
+
|
|
71
81
|
## Document marks
|
|
72
82
|
|
|
73
83
|
`@hachej/boring-ui-kit/markdown-quote` exports `captureMarkdownQuote` and
|
|
@@ -202,6 +212,24 @@ Mounted `tools.propose` accepts the same complete edits as the headless command
|
|
|
202
212
|
|
|
203
213
|
`test/compatibility/canvas-mounted-native.test.mjs` composes selection and proposals with real native ToolTasks. `npm run canvas:journey:commands` requires `CHROMIUM` and permitted loopback sockets; it checks real desktop/phone viewport geometry with fictional assets. Production fonts, licensing, egress and CSS isolation remain separate qualifications.
|
|
204
214
|
|
|
215
|
+
## Read-only ArtifactViewer
|
|
216
|
+
|
|
217
|
+
```tsx
|
|
218
|
+
import { ArtifactViewer } from '@hachej/boring-ui-kit/artifact-viewer';
|
|
219
|
+
|
|
220
|
+
<ArtifactViewer name="fictional-guide.md" mimeType="text/markdown" content="# Fictional guide" />;
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
`ArtifactViewerProps` accepts `name`, `mimeType`, optional `content: string | Uint8Array`, `url`, `className` and `renderPdf`. Install the existing optional React and Marked peers when selecting this entry. It has no ResourceClient, controller, editing, syntax highlighter or agent integration. The host supplies authorized content and owns URL access policy and layout; mounting grants no file or execution authority.
|
|
224
|
+
|
|
225
|
+
Normalized MIME selects Markdown, HTML, images and PDF, with filename fallback. Other text/code is an escaped `<pre>` without highlighting. Byte text requires valid UTF-8 without NUL; leading U+FEFF is preserved. Binary audio/video does not become a text preview. Missing or unreadable content displays “Preview unavailable.” Replace byte arrays when changing content rather than mutating them in place.
|
|
226
|
+
|
|
227
|
+
Markdown uses a private Marked parser, then the same passive HTML allowlist as `HtmlViewer`. Both HTML types render in an empty-sandbox, no-referrer iframe with the fixed restrictive CSP. Links become spans; scripts, attributes, styles, images and external resources are omitted, including Markdown images. The [HTML preview limits](#optional-html-source-and-preview) apply; Markdown input is also limited to 262,144 UTF-16 units before parsing. There is no raw HTML fallback. URL-only Markdown, HTML and text are unavailable: the entry never fetches or embeds them unsanitized.
|
|
228
|
+
|
|
229
|
+
Images use `<img>` and PDFs use the browser's native sandboxed iframe. `content` takes precedence over `url`; content creates an owned Blob URL, revoked on replacement or unmount. Host URLs are borrowed and never revoked. Relative, HTTP(S), blob and matching image/PDF data URLs are supported; other URL schemes are refused. Supply `renderPdf: (props: ArtifactPdfProps) => ReactNode` to use a trusted host renderer instead, receiving the current `{ name, url }`. The entry does not load pdfjs. Browser-native PDF support and sandbox restrictions vary by browser; it provides no page controls or PDF text extraction.
|
|
230
|
+
|
|
231
|
+
`test/packages/ui-artifact-viewer.test.mjs` renders fictional fixtures through the built public entry and checks its browser import graph. `npm run test:artifact-viewer-consumer` installs the kit tarball with only React/Marked and test tooling, checks strict declarations, repeats those DOM tests and bundles without files, controllers, Tiptap, canvas, pdfjs or agent code. DOM structure tests do not qualify browser CSP enforcement, parser inertness, egress or native PDF rendering.
|
|
232
|
+
|
|
205
233
|
## Optional HTML source and preview
|
|
206
234
|
|
|
207
235
|
Both `MarkdownEditor` and `HtmlViewer` accept optional `components` bindings for host buttons, inputs, textareas and the selection bubble. Installed registry recipes supply the consumer's unchanged shadcn components. Bindings forward refs and native input events. Omitting them retains unstyled native HTML for direct package consumers; it is not installed-block design qualification. See the [registry theming contract](../../registry/README.md#styling).
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
import type { ReactNode } from 'react';
|
|
2
|
+
export interface ArtifactPdfProps {
|
|
3
|
+
readonly name: string;
|
|
4
|
+
readonly url: string;
|
|
5
|
+
}
|
|
6
|
+
export interface ArtifactViewerProps {
|
|
7
|
+
readonly name: string;
|
|
8
|
+
readonly mimeType: string;
|
|
9
|
+
/** Content takes precedence over a URL. Byte content must be replaced, not mutated in place. */
|
|
10
|
+
readonly content?: string | Uint8Array;
|
|
11
|
+
/** Borrowed host URL; used only for images and PDF, never fetched for text previews. */
|
|
12
|
+
readonly url?: string;
|
|
13
|
+
readonly className?: string;
|
|
14
|
+
/** Trusted host presentation, for example a separately installed pdfjs renderer. */
|
|
15
|
+
readonly renderPdf?: (props: ArtifactPdfProps) => ReactNode;
|
|
16
|
+
}
|
|
17
|
+
export declare function ArtifactViewer(props: ArtifactViewerProps): import("react").JSX.Element;
|
|
18
|
+
//# sourceMappingURL=artifact-viewer.d.ts.map
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
'use client';
|
|
2
|
+
import { jsx as _jsx, Fragment as _Fragment, jsxs as _jsxs } from "react/jsx-runtime";
|
|
3
|
+
import { useLayoutEffect, useRef, useState } from 'react';
|
|
4
|
+
import { Marked } from 'marked';
|
|
5
|
+
import { passiveHtml } from './passive-html.js';
|
|
6
|
+
const markdown = new Marked();
|
|
7
|
+
const imageTypes = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', gif: 'image/gif', webp: 'image/webp', svg: 'image/svg+xml', bmp: 'image/bmp', avif: 'image/avif' };
|
|
8
|
+
function artifactType(name, mimeType) {
|
|
9
|
+
const mime = mimeType.split(';')[0].trim().toLowerCase();
|
|
10
|
+
if (mime === 'text/markdown' || mime === 'text/x-markdown')
|
|
11
|
+
return { kind: 'markdown', mimeType: mime };
|
|
12
|
+
if (mime === 'text/html')
|
|
13
|
+
return { kind: 'html', mimeType: mime };
|
|
14
|
+
if (mime === 'application/pdf')
|
|
15
|
+
return { kind: 'pdf', mimeType: mime };
|
|
16
|
+
if (mime.startsWith('image/'))
|
|
17
|
+
return { kind: 'image', mimeType: mime };
|
|
18
|
+
if (/^(?:audio|video)\//.test(mime))
|
|
19
|
+
return undefined;
|
|
20
|
+
const basename = name.slice(name.lastIndexOf('/') + 1), dot = basename.lastIndexOf('.');
|
|
21
|
+
const extension = dot === -1 ? '' : basename.slice(dot + 1).toLowerCase();
|
|
22
|
+
if (extension === 'md' || extension === 'markdown')
|
|
23
|
+
return { kind: 'markdown', mimeType: 'text/markdown' };
|
|
24
|
+
if (extension === 'html' || extension === 'htm')
|
|
25
|
+
return { kind: 'html', mimeType: 'text/html' };
|
|
26
|
+
if (extension === 'pdf')
|
|
27
|
+
return { kind: 'pdf', mimeType: 'application/pdf' };
|
|
28
|
+
const imageType = Object.hasOwn(imageTypes, extension) ? imageTypes[extension] : undefined;
|
|
29
|
+
if (imageType)
|
|
30
|
+
return { kind: 'image', mimeType: imageType };
|
|
31
|
+
if (/^(?:mp4|webm|mov|m4v|ogg|ogv|mp3|wav)$/.test(extension))
|
|
32
|
+
return undefined;
|
|
33
|
+
return { kind: 'text', mimeType: mime };
|
|
34
|
+
}
|
|
35
|
+
function decodeText(content) {
|
|
36
|
+
if (content === undefined)
|
|
37
|
+
return undefined;
|
|
38
|
+
if (typeof content === 'string')
|
|
39
|
+
return content.includes('\0') ? undefined : content;
|
|
40
|
+
if (content.includes(0))
|
|
41
|
+
return undefined;
|
|
42
|
+
try {
|
|
43
|
+
return new TextDecoder('utf-8', { fatal: true, ignoreBOM: true }).decode(content);
|
|
44
|
+
}
|
|
45
|
+
catch {
|
|
46
|
+
return undefined;
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
function Unavailable() {
|
|
50
|
+
return _jsx("p", { role: "status", children: "Preview unavailable." });
|
|
51
|
+
}
|
|
52
|
+
function PassiveArtifact({ name, text, kind }) {
|
|
53
|
+
const container = useRef(null);
|
|
54
|
+
const [ready, setReady] = useState(false);
|
|
55
|
+
useLayoutEffect(() => {
|
|
56
|
+
const host = container.current;
|
|
57
|
+
if (!host)
|
|
58
|
+
return;
|
|
59
|
+
host.replaceChildren();
|
|
60
|
+
try {
|
|
61
|
+
if (text.length > 262_144)
|
|
62
|
+
throw new RangeError('Artifact preview source exceeds its limit');
|
|
63
|
+
const source = passiveHtml(kind === 'markdown' ? markdown.parse(text, { async: false }) : text, host.ownerDocument);
|
|
64
|
+
const frame = host.ownerDocument.createElement('iframe');
|
|
65
|
+
frame.setAttribute('sandbox', '');
|
|
66
|
+
frame.referrerPolicy = 'no-referrer';
|
|
67
|
+
frame.title = name;
|
|
68
|
+
frame.srcdoc = source;
|
|
69
|
+
host.replaceChildren(frame);
|
|
70
|
+
setReady(true);
|
|
71
|
+
}
|
|
72
|
+
catch {
|
|
73
|
+
host.replaceChildren();
|
|
74
|
+
setReady(false);
|
|
75
|
+
}
|
|
76
|
+
return () => { host.replaceChildren(); };
|
|
77
|
+
}, [name, text, kind]);
|
|
78
|
+
return _jsxs(_Fragment, { children: [!ready && _jsx(Unavailable, {}), _jsx("div", { ref: container })] });
|
|
79
|
+
}
|
|
80
|
+
function mediaUrl(url, kind) {
|
|
81
|
+
if (!url)
|
|
82
|
+
return undefined;
|
|
83
|
+
try {
|
|
84
|
+
const protocol = new URL(url, 'https://artifact.invalid/').protocol;
|
|
85
|
+
if (['https:', 'http:', 'blob:'].includes(protocol))
|
|
86
|
+
return url;
|
|
87
|
+
if (protocol === 'data:' && (kind === 'image' ? /^data:image\//i : /^data:application\/pdf(?:;|,)/i).test(url))
|
|
88
|
+
return url;
|
|
89
|
+
}
|
|
90
|
+
catch { /* A malformed host URL has no preview. */ }
|
|
91
|
+
return undefined;
|
|
92
|
+
}
|
|
93
|
+
function MediaArtifact({ name, kind, mimeType, content, url, renderPdf }) {
|
|
94
|
+
const [made, setMade] = useState();
|
|
95
|
+
useLayoutEffect(() => {
|
|
96
|
+
if (content === undefined) {
|
|
97
|
+
setMade(undefined);
|
|
98
|
+
return;
|
|
99
|
+
}
|
|
100
|
+
try {
|
|
101
|
+
const blob = new Blob([typeof content === 'string' ? content : new Uint8Array(content)], { type: mimeType });
|
|
102
|
+
const objectUrl = URL.createObjectURL(blob);
|
|
103
|
+
setMade({ content, mimeType, url: objectUrl });
|
|
104
|
+
return () => { URL.revokeObjectURL(objectUrl); };
|
|
105
|
+
}
|
|
106
|
+
catch {
|
|
107
|
+
setMade(undefined);
|
|
108
|
+
}
|
|
109
|
+
}, [content, mimeType]);
|
|
110
|
+
const selectedUrl = content === undefined ? mediaUrl(url, kind) : made?.content === content && made.mimeType === mimeType ? made.url : undefined;
|
|
111
|
+
if (!selectedUrl)
|
|
112
|
+
return _jsx(Unavailable, {});
|
|
113
|
+
if (kind === 'image')
|
|
114
|
+
return _jsx("img", { src: selectedUrl, alt: name, referrerPolicy: "no-referrer" });
|
|
115
|
+
return renderPdf ? renderPdf({ name, url: selectedUrl }) : _jsx("iframe", { src: selectedUrl, title: name, sandbox: "", referrerPolicy: "no-referrer" });
|
|
116
|
+
}
|
|
117
|
+
export function ArtifactViewer(props) {
|
|
118
|
+
const type = artifactType(props.name, props.mimeType);
|
|
119
|
+
const text = type && type.kind !== 'image' && type.kind !== 'pdf' ? decodeText(props.content) : undefined;
|
|
120
|
+
return _jsx("section", { "data-boring": "artifact-viewer", className: props.className, "aria-label": props.name, children: !type ? _jsx(Unavailable, {}) : type.kind === 'image' || type.kind === 'pdf'
|
|
121
|
+
? _jsx(MediaArtifact, { ...props, kind: type.kind, mimeType: type.mimeType })
|
|
122
|
+
: text === undefined ? _jsx(Unavailable, {}) : type.kind === 'text'
|
|
123
|
+
? _jsx("pre", { children: text })
|
|
124
|
+
: _jsx(PassiveArtifact, { name: props.name, text: text, kind: type.kind }) });
|
|
125
|
+
}
|
package/dist/html-preview.d.ts
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export
|
|
1
|
+
export { passiveHtml } from './passive-html.js';
|
|
2
2
|
//# sourceMappingURL=html-preview.d.ts.map
|
package/dist/html-preview.js
CHANGED
|
@@ -1,47 +1 @@
|
|
|
1
|
-
|
|
2
|
-
const htmlNamespace = 'http://www.w3.org/1999/xhtml';
|
|
3
|
-
const prefix = '<!doctype html><html><head><meta http-equiv="Content-Security-Policy" content="default-src \'none\'; base-uri \'none\'; form-action \'none\'"><meta charset="utf-8"><meta name="referrer" content="no-referrer"><meta name="format-detection" content="telephone=no,email=no,address=no"><title>HTML preview</title></head><body>';
|
|
4
|
-
export function passiveHtml(source, document) {
|
|
5
|
-
if (source.length > 262_144)
|
|
6
|
-
throw new RangeError('HTML preview source exceeds its limit');
|
|
7
|
-
const template = document.createElement('template');
|
|
8
|
-
template.innerHTML = source;
|
|
9
|
-
let nodes = 0, length = prefix.length;
|
|
10
|
-
const output = [prefix];
|
|
11
|
-
function emit(text) {
|
|
12
|
-
length += text.length;
|
|
13
|
-
if (length > 1_048_576)
|
|
14
|
-
throw new RangeError('HTML preview output exceeds its limit');
|
|
15
|
-
output.push(text);
|
|
16
|
-
}
|
|
17
|
-
function visit(node, depth) {
|
|
18
|
-
if (++nodes > 20_000 || depth > 128)
|
|
19
|
-
throw new RangeError('HTML preview structure exceeds its limit');
|
|
20
|
-
if (node.nodeType === 3) {
|
|
21
|
-
emit((node.textContent ?? '').replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>'));
|
|
22
|
-
return;
|
|
23
|
-
}
|
|
24
|
-
if (node.nodeType !== 1 || !('namespaceURI' in node) || node.namespaceURI !== htmlNamespace || !('localName' in node) || typeof node.localName !== 'string')
|
|
25
|
-
return;
|
|
26
|
-
if (node.localName === 'html' || node.localName === 'head' || node.localName === 'body') {
|
|
27
|
-
for (let child = node.firstChild; child; child = child.nextSibling)
|
|
28
|
-
visit(child, depth + 1);
|
|
29
|
-
return;
|
|
30
|
-
}
|
|
31
|
-
const tag = node.localName === 'a' ? 'span' : node.localName;
|
|
32
|
-
if (!formatting.has(tag))
|
|
33
|
-
return;
|
|
34
|
-
emit('<' + tag + '>');
|
|
35
|
-
if (tag === 'br' || tag === 'hr')
|
|
36
|
-
return;
|
|
37
|
-
if (tag === 'pre')
|
|
38
|
-
emit('\n');
|
|
39
|
-
for (let child = node.firstChild; child; child = child.nextSibling)
|
|
40
|
-
visit(child, depth + 1);
|
|
41
|
-
emit('</' + tag + '>');
|
|
42
|
-
}
|
|
43
|
-
for (let child = template.content.firstChild; child; child = child.nextSibling)
|
|
44
|
-
visit(child, 0);
|
|
45
|
-
emit('</body></html>');
|
|
46
|
-
return output.join('');
|
|
47
|
-
}
|
|
1
|
+
export { passiveHtml } from './passive-html.js';
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
const formatting = new Set(['p', 'div', 'span', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'blockquote', 'pre', 'code', 'strong', 'em', 'b', 'i', 'u', 's', 'small', 'sub', 'sup', 'kbd', 'samp', 'var', 'mark', 'ul', 'ol', 'li', 'dl', 'dt', 'dd', 'table', 'caption', 'thead', 'tbody', 'tfoot', 'tr', 'th', 'td', 'section', 'article', 'header', 'footer', 'main', 'aside', 'figure', 'figcaption', 'br', 'hr']);
|
|
2
|
+
const htmlNamespace = 'http://www.w3.org/1999/xhtml';
|
|
3
|
+
const prefix = '<!doctype html><html><head><meta http-equiv="Content-Security-Policy" content="default-src \'none\'; base-uri \'none\'; form-action \'none\'"><meta charset="utf-8"><meta name="referrer" content="no-referrer"><meta name="format-detection" content="telephone=no,email=no,address=no"><title>HTML preview</title></head><body>';
|
|
4
|
+
export function passiveHtml(source, document) {
|
|
5
|
+
if (source.length > 262_144)
|
|
6
|
+
throw new RangeError('HTML preview source exceeds its limit');
|
|
7
|
+
const template = document.createElement('template');
|
|
8
|
+
template.innerHTML = source;
|
|
9
|
+
let nodes = 0, length = prefix.length;
|
|
10
|
+
const output = [prefix];
|
|
11
|
+
function emit(text) {
|
|
12
|
+
length += text.length;
|
|
13
|
+
if (length > 1_048_576)
|
|
14
|
+
throw new RangeError('HTML preview output exceeds its limit');
|
|
15
|
+
output.push(text);
|
|
16
|
+
}
|
|
17
|
+
function visit(node, depth) {
|
|
18
|
+
if (++nodes > 20_000 || depth > 128)
|
|
19
|
+
throw new RangeError('HTML preview structure exceeds its limit');
|
|
20
|
+
if (node.nodeType === 3) {
|
|
21
|
+
emit((node.textContent ?? '').replaceAll('&', '&').replaceAll('<', '<').replaceAll('>', '>'));
|
|
22
|
+
return;
|
|
23
|
+
}
|
|
24
|
+
if (node.nodeType !== 1 || !('namespaceURI' in node) || node.namespaceURI !== htmlNamespace || !('localName' in node) || typeof node.localName !== 'string')
|
|
25
|
+
return;
|
|
26
|
+
if (node.localName === 'html' || node.localName === 'head' || node.localName === 'body') {
|
|
27
|
+
for (let child = node.firstChild; child; child = child.nextSibling)
|
|
28
|
+
visit(child, depth + 1);
|
|
29
|
+
return;
|
|
30
|
+
}
|
|
31
|
+
const tag = node.localName === 'a' ? 'span' : node.localName;
|
|
32
|
+
if (!formatting.has(tag))
|
|
33
|
+
return;
|
|
34
|
+
emit('<' + tag + '>');
|
|
35
|
+
if (tag === 'br' || tag === 'hr')
|
|
36
|
+
return;
|
|
37
|
+
if (tag === 'pre')
|
|
38
|
+
emit('\n');
|
|
39
|
+
for (let child = node.firstChild; child; child = child.nextSibling)
|
|
40
|
+
visit(child, depth + 1);
|
|
41
|
+
emit('</' + tag + '>');
|
|
42
|
+
}
|
|
43
|
+
for (let child = template.content.firstChild; child; child = child.nextSibling)
|
|
44
|
+
visit(child, 0);
|
|
45
|
+
emit('</body></html>');
|
|
46
|
+
return output.join('');
|
|
47
|
+
}
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
/** Why the microphone could not be used, for a short human message: `denied` (the person or the browser refused the permission),
|
|
2
|
+
* `no-microphone`, `busy` (in use or could not be opened), `insecure` (http on a name other than localhost: no media devices at all),
|
|
3
|
+
* `unsupported` (no MediaRecorder, or it cannot record this microphone), `failed` (anything else). */
|
|
4
|
+
export type VoiceRefusal = 'denied' | 'no-microphone' | 'busy' | 'insecure' | 'unsupported' | 'failed';
|
|
5
|
+
export type VoiceStart = {
|
|
6
|
+
readonly kind: 'recording';
|
|
7
|
+
} | {
|
|
8
|
+
readonly kind: 'refused';
|
|
9
|
+
readonly reason: string;
|
|
10
|
+
readonly code?: VoiceRefusal;
|
|
11
|
+
};
|
|
12
|
+
export interface VoiceRecording {
|
|
13
|
+
readonly audio: Blob;
|
|
14
|
+
readonly mimeType: string;
|
|
15
|
+
/** Page time (`performance.now()` ms) of the first recorded sample. */
|
|
16
|
+
readonly startedAt: number;
|
|
17
|
+
readonly endedAt: number;
|
|
18
|
+
/** The recording reached `maxMs` and stopped on its own. */
|
|
19
|
+
readonly truncated: boolean;
|
|
20
|
+
}
|
|
21
|
+
export interface VoiceCapture {
|
|
22
|
+
readonly start: () => Promise<VoiceStart>;
|
|
23
|
+
/** Ends the recording and returns it; null when nothing was recorded (never started, cancelled, or empty). */
|
|
24
|
+
readonly stop: () => Promise<VoiceRecording | null>;
|
|
25
|
+
/** Ends the recording, releases the microphone and discards the audio. */
|
|
26
|
+
readonly cancel: () => void;
|
|
27
|
+
}
|
|
28
|
+
export interface VoiceCaptureOptions {
|
|
29
|
+
readonly getUserMedia?: (constraints: MediaStreamConstraints) => Promise<MediaStream>;
|
|
30
|
+
/** Longest recording in milliseconds (default 5 minutes); the recording stops there and `stop()` reports `truncated`. */
|
|
31
|
+
readonly maxMs?: number;
|
|
32
|
+
/** The page clock (default `performance.now()`). */
|
|
33
|
+
readonly now?: () => number;
|
|
34
|
+
}
|
|
35
|
+
export declare const DEFAULT_MAX_VOICE_MS: number;
|
|
36
|
+
/** Container types tried in order; the first the browser records wins. */
|
|
37
|
+
export declare const VOICE_MIME_TYPES: readonly ["audio/webm;codecs=opus", "audio/webm", "audio/mp4", "audio/ogg;codecs=opus"];
|
|
38
|
+
/** The browser's microphone, or null where the page cannot have one (insecure origin, no media devices). */
|
|
39
|
+
export declare function browserGetUserMedia(): ((constraints: MediaStreamConstraints) => Promise<MediaStream>) | null;
|
|
40
|
+
/** Short human messages, one per refusal; each says what to do next. */
|
|
41
|
+
export declare const VOICE_REFUSALS: Readonly<Record<VoiceRefusal, string>>;
|
|
42
|
+
export declare function createVoiceCapture(options?: VoiceCaptureOptions): VoiceCapture;
|
|
43
|
+
//# sourceMappingURL=voice-capture.d.ts.map
|
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
// Shared browser microphone capture. Chat keeps the audio; feedback also uses the page-clock time of the first recorded sample
|
|
2
|
+
// to align speech with the pointer.
|
|
3
|
+
//
|
|
4
|
+
// Microphone access (getUserMedia) exists only in secure contexts (https or localhost) and needs the person's permission. The
|
|
5
|
+
// capture never opens it on load: only `start()`, called from the person's click, does, through an injectable `getUserMedia`
|
|
6
|
+
// whose default feature-detects `navigator.mediaDevices.getUserMedia` and refuses with a reason when it is missing. A host that
|
|
7
|
+
// owns microphone policy passes its own. `npm run check` (scripts/check-platform-apis.mjs) allowlists this file on that condition.
|
|
8
|
+
//
|
|
9
|
+
// The clock. MediaRecorder hands audio over late and in chunks, so neither its `start` event nor a chunk's arrival says when the
|
|
10
|
+
// first recorded sample was heard. `npm run feedback:voice-sync` (tools/feedback-voice-sync.mjs) measured seven candidate
|
|
11
|
+
// strategies in Chromium against an independent reader of the same microphone; the page time read immediately before
|
|
12
|
+
// `MediaRecorder.start()` was the most accurate (2–4 ms mean, under 9 ms max over three batches, with the fake device's dropped
|
|
13
|
+
// time removed), while the `start` event is about 60 ms late (up to 125 ms) and stop-minus-duration 30–40 ms late. So
|
|
14
|
+
// `startedAt` is that time.
|
|
15
|
+
// `endedAt` is `startedAt` plus the decoded duration when the page can decode the recording, else the `stop` event's time.
|
|
16
|
+
// Measured in Chromium only; a microphone that warms up after getUserMedia resolves (no audio yet when recording starts) would
|
|
17
|
+
// make `startedAt` early by the warm-up.
|
|
18
|
+
export const DEFAULT_MAX_VOICE_MS = 5 * 60_000;
|
|
19
|
+
/** Container types tried in order; the first the browser records wins. */
|
|
20
|
+
export const VOICE_MIME_TYPES = ['audio/webm;codecs=opus', 'audio/webm', 'audio/mp4', 'audio/ogg;codecs=opus'];
|
|
21
|
+
/** The browser's microphone, or null where the page cannot have one (insecure origin, no media devices). */
|
|
22
|
+
export function browserGetUserMedia() {
|
|
23
|
+
if (typeof globalThis.navigator?.mediaDevices?.getUserMedia !== 'function')
|
|
24
|
+
return null;
|
|
25
|
+
const devices = globalThis.navigator.mediaDevices;
|
|
26
|
+
return constraints => devices.getUserMedia(constraints);
|
|
27
|
+
}
|
|
28
|
+
/** Short human messages, one per refusal; each says what to do next. */
|
|
29
|
+
export const VOICE_REFUSALS = {
|
|
30
|
+
denied: 'Microphone blocked. Allow it for this site in the browser, then press Record again.',
|
|
31
|
+
'no-microphone': 'No microphone found. Connect one, then press Record again.',
|
|
32
|
+
busy: 'The microphone is busy or could not be opened. Close other apps using it and try again.',
|
|
33
|
+
insecure: 'Recording needs a secure page (https or localhost).',
|
|
34
|
+
unsupported: 'This browser cannot record audio here.',
|
|
35
|
+
failed: 'The microphone could not be started.',
|
|
36
|
+
};
|
|
37
|
+
const refused = (code, detail) => ({ kind: 'refused', code, reason: detail ? `${VOICE_REFUSALS[code]} (${detail})` : VOICE_REFUSALS[code] });
|
|
38
|
+
function refusalOf(error) {
|
|
39
|
+
const name = error?.name;
|
|
40
|
+
if (name === 'NotAllowedError' || name === 'PermissionDeniedError' || name === 'SecurityError')
|
|
41
|
+
return refused('denied');
|
|
42
|
+
if (name === 'NotFoundError' || name === 'DevicesNotFoundError' || name === 'OverconstrainedError')
|
|
43
|
+
return refused('no-microphone');
|
|
44
|
+
if (name === 'NotReadableError' || name === 'TrackStartError' || name === 'AbortError')
|
|
45
|
+
return refused('busy');
|
|
46
|
+
return refused('failed', error instanceof Error ? error.message : String(error));
|
|
47
|
+
}
|
|
48
|
+
/** Media devices missing on a plain-http page: the usual reason is an insecure origin (http on a name that is not local; a local
|
|
49
|
+
* http page has them). Wording only; the capability itself is feature-detected (`browserGetUserMedia`). */
|
|
50
|
+
const plainHttpPage = () => globalThis.location?.protocol === 'http:';
|
|
51
|
+
const release = (stream) => { for (const track of stream?.getTracks() ?? [])
|
|
52
|
+
track.stop(); };
|
|
53
|
+
/** Duration in milliseconds of recorded audio, when the page can decode it (OfflineAudioContext or AudioContext). */
|
|
54
|
+
async function decodedMs(audio) {
|
|
55
|
+
try {
|
|
56
|
+
const Offline = globalThis.OfflineAudioContext;
|
|
57
|
+
if (typeof Offline !== 'function' || audio.size === 0)
|
|
58
|
+
return null;
|
|
59
|
+
const context = new Offline(1, 1, 48_000);
|
|
60
|
+
const buffer = await context.decodeAudioData(await audio.arrayBuffer());
|
|
61
|
+
return Number.isFinite(buffer.duration) ? buffer.duration * 1000 : null;
|
|
62
|
+
}
|
|
63
|
+
catch {
|
|
64
|
+
return null;
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
export function createVoiceCapture(options = {}) {
|
|
68
|
+
const now = options.now ?? (() => performance.now());
|
|
69
|
+
const maxMs = options.maxMs ?? DEFAULT_MAX_VOICE_MS;
|
|
70
|
+
let session = null;
|
|
71
|
+
let opening = false;
|
|
72
|
+
let cancelledWhileOpening = false;
|
|
73
|
+
const finish = (current, truncated) => {
|
|
74
|
+
if (current.timer !== null) {
|
|
75
|
+
clearTimeout(current.timer);
|
|
76
|
+
current.timer = null;
|
|
77
|
+
}
|
|
78
|
+
if (truncated)
|
|
79
|
+
current.truncated = true;
|
|
80
|
+
if (current.recorder.state !== 'inactive') {
|
|
81
|
+
try {
|
|
82
|
+
current.recorder.stop();
|
|
83
|
+
}
|
|
84
|
+
catch { /* already stopping */ }
|
|
85
|
+
}
|
|
86
|
+
};
|
|
87
|
+
return {
|
|
88
|
+
async start() {
|
|
89
|
+
if (session || opening)
|
|
90
|
+
return { kind: 'refused', code: 'failed', reason: 'Already recording.' };
|
|
91
|
+
const Recorder = globalThis.MediaRecorder;
|
|
92
|
+
const open = options.getUserMedia ?? browserGetUserMedia();
|
|
93
|
+
// No media devices on an insecure origin: say that, not "no microphone" (the person can fix the address, not the hardware).
|
|
94
|
+
if (!open)
|
|
95
|
+
return refused(plainHttpPage() ? 'insecure' : 'unsupported');
|
|
96
|
+
if (typeof Recorder !== 'function')
|
|
97
|
+
return refused('unsupported', 'no MediaRecorder');
|
|
98
|
+
// getUserMedia is called synchronously from here, inside the person's click (no await before it), so browsers that tie the
|
|
99
|
+
// permission prompt to a user gesture show it.
|
|
100
|
+
opening = true;
|
|
101
|
+
cancelledWhileOpening = false;
|
|
102
|
+
let stream;
|
|
103
|
+
try {
|
|
104
|
+
stream = await open({ audio: true });
|
|
105
|
+
}
|
|
106
|
+
catch (error) {
|
|
107
|
+
opening = false;
|
|
108
|
+
return refusalOf(error);
|
|
109
|
+
}
|
|
110
|
+
if (cancelledWhileOpening) {
|
|
111
|
+
opening = false;
|
|
112
|
+
release(stream);
|
|
113
|
+
return { kind: 'refused', code: 'failed', reason: 'Cancelled before the microphone opened.' };
|
|
114
|
+
}
|
|
115
|
+
const supported = (type) => { try {
|
|
116
|
+
return typeof Recorder.isTypeSupported !== 'function' || Recorder.isTypeSupported(type);
|
|
117
|
+
}
|
|
118
|
+
catch {
|
|
119
|
+
return false;
|
|
120
|
+
} };
|
|
121
|
+
const preferred = VOICE_MIME_TYPES.find(supported);
|
|
122
|
+
let recorder;
|
|
123
|
+
try {
|
|
124
|
+
recorder = preferred ? new Recorder(stream, { mimeType: preferred }) : new Recorder(stream);
|
|
125
|
+
}
|
|
126
|
+
catch (error) {
|
|
127
|
+
opening = false;
|
|
128
|
+
release(stream);
|
|
129
|
+
return refused('unsupported', error instanceof Error ? error.message : String(error));
|
|
130
|
+
}
|
|
131
|
+
let settle;
|
|
132
|
+
const current = {
|
|
133
|
+
stream, recorder, mimeType: preferred ?? '', chunks: [], startedAt: null, stoppedAt: null, truncated: false, timer: null,
|
|
134
|
+
stopped: new Promise(resolve => { settle = resolve; }), cancelled: false,
|
|
135
|
+
};
|
|
136
|
+
recorder.addEventListener('dataavailable', event => { if (event.data && event.data.size > 0)
|
|
137
|
+
current.chunks.push(event.data); });
|
|
138
|
+
recorder.addEventListener('stop', () => { current.stoppedAt = now(); release(stream); settle(); });
|
|
139
|
+
recorder.addEventListener('error', () => { current.stoppedAt ??= now(); release(stream); settle(); });
|
|
140
|
+
try {
|
|
141
|
+
current.startedAt = now();
|
|
142
|
+
recorder.start(1000);
|
|
143
|
+
}
|
|
144
|
+
catch (error) {
|
|
145
|
+
opening = false;
|
|
146
|
+
release(stream);
|
|
147
|
+
return refused('failed', error instanceof Error ? error.message : String(error));
|
|
148
|
+
}
|
|
149
|
+
if (Number.isFinite(maxMs) && maxMs > 0)
|
|
150
|
+
current.timer = setTimeout(() => finish(current, true), maxMs);
|
|
151
|
+
session = current;
|
|
152
|
+
opening = false;
|
|
153
|
+
return { kind: 'recording' };
|
|
154
|
+
},
|
|
155
|
+
async stop() {
|
|
156
|
+
const current = session;
|
|
157
|
+
if (!current)
|
|
158
|
+
return null;
|
|
159
|
+
finish(current, false);
|
|
160
|
+
await current.stopped;
|
|
161
|
+
session = null;
|
|
162
|
+
if (current.cancelled)
|
|
163
|
+
return null;
|
|
164
|
+
const mimeType = current.recorder.mimeType || current.mimeType || current.chunks[0]?.type || 'audio/webm';
|
|
165
|
+
const audio = new Blob(current.chunks, { type: mimeType });
|
|
166
|
+
if (audio.size === 0)
|
|
167
|
+
return null;
|
|
168
|
+
const startedAt = current.startedAt ?? current.stoppedAt ?? now();
|
|
169
|
+
const duration = await decodedMs(audio);
|
|
170
|
+
const endedAt = duration === null ? Math.max(startedAt, current.stoppedAt ?? now()) : startedAt + duration;
|
|
171
|
+
return { audio, mimeType, startedAt, endedAt, truncated: current.truncated };
|
|
172
|
+
},
|
|
173
|
+
cancel() {
|
|
174
|
+
const current = session;
|
|
175
|
+
if (opening)
|
|
176
|
+
cancelledWhileOpening = true;
|
|
177
|
+
if (!current)
|
|
178
|
+
return;
|
|
179
|
+
session = null;
|
|
180
|
+
current.cancelled = true;
|
|
181
|
+
current.chunks.length = 0;
|
|
182
|
+
finish(current, false);
|
|
183
|
+
release(current.stream);
|
|
184
|
+
},
|
|
185
|
+
};
|
|
186
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@hachej/boring-ui-kit",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.6",
|
|
4
4
|
"license": "MIT",
|
|
5
5
|
"private": false,
|
|
6
6
|
"type": "module",
|
|
@@ -55,6 +55,10 @@
|
|
|
55
55
|
"types": "./dist/native-chat.d.ts",
|
|
56
56
|
"import": "./dist/native-chat.js"
|
|
57
57
|
},
|
|
58
|
+
"./voice-capture": {
|
|
59
|
+
"types": "./dist/voice-capture.d.ts",
|
|
60
|
+
"import": "./dist/voice-capture.js"
|
|
61
|
+
},
|
|
58
62
|
"./shared-chat": {
|
|
59
63
|
"types": "./dist/shared-chat/index.d.ts",
|
|
60
64
|
"import": "./dist/shared-chat/index.js"
|
|
@@ -79,6 +83,10 @@
|
|
|
79
83
|
"types": "./dist/html-viewer.d.ts",
|
|
80
84
|
"import": "./dist/html-viewer.js"
|
|
81
85
|
},
|
|
86
|
+
"./artifact-viewer": {
|
|
87
|
+
"types": "./dist/artifact-viewer.d.ts",
|
|
88
|
+
"import": "./dist/artifact-viewer.js"
|
|
89
|
+
},
|
|
82
90
|
"./experience": {
|
|
83
91
|
"types": "./dist/experience.d.ts",
|
|
84
92
|
"import": "./dist/experience.js"
|
|
@@ -126,7 +134,7 @@
|
|
|
126
134
|
}
|
|
127
135
|
},
|
|
128
136
|
"peerDependencies": {
|
|
129
|
-
"@hachej/boring-files": "^0.4.
|
|
137
|
+
"@hachej/boring-files": "^0.4.6",
|
|
130
138
|
"@earendil-works/pi-durable": "~1.2.0",
|
|
131
139
|
"react": "^19.0.0",
|
|
132
140
|
"react-dom": "^19.0.0",
|
package/r/agent-target.json
CHANGED
|
@@ -4,15 +4,15 @@
|
|
|
4
4
|
"title": "Point and dictate on app data",
|
|
5
5
|
"description": "Tap explicitly marked JSON app context, dictate once and show the same app agent’s native reply without opening chat. Host-controlled presentation; no feedback report or second controller.",
|
|
6
6
|
"dependencies": [
|
|
7
|
-
"@hachej/boring-ui-kit@^0.4.
|
|
8
|
-
"@hachej/boring-feedback@^0.4.
|
|
7
|
+
"@hachej/boring-ui-kit@^0.4.6",
|
|
8
|
+
"@hachej/boring-feedback@^0.4.6",
|
|
9
9
|
"@earendil-works/pi-durable@~1.2.0",
|
|
10
10
|
"react@^19.0.0",
|
|
11
11
|
"react-dom@^19.0.0"
|
|
12
12
|
],
|
|
13
13
|
"registryDependencies": [
|
|
14
|
-
"https://cdn.jsdelivr.net/npm/@hachej/boring-ui-kit@0.4.
|
|
15
|
-
"https://cdn.jsdelivr.net/npm/@hachej/boring-ui-kit@0.4.
|
|
14
|
+
"https://cdn.jsdelivr.net/npm/@hachej/boring-ui-kit@0.4.6/r/app-shell.json",
|
|
15
|
+
"https://cdn.jsdelivr.net/npm/@hachej/boring-ui-kit@0.4.6/r/pi-chat.json",
|
|
16
16
|
"button",
|
|
17
17
|
"card",
|
|
18
18
|
"message",
|