@supersuit/artifacts 0.42.0 → 0.44.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +82 -0
- package/README.md +99 -1
- package/lib/artifacts/index.d.ts +1 -0
- package/lib/artifacts/index.js +1 -0
- package/lib/artifacts/recipients-store.d.ts +4 -0
- package/lib/artifacts/recipients-store.js +1 -1
- package/lib/artifacts/render.js +27 -9
- package/lib/artifacts/voice-note.d.ts +55 -0
- package/lib/artifacts/voice-note.js +161 -0
- package/lib/artifacts/widgets.js +7 -2
- package/lib/cli/transcript-fence.d.ts +2 -0
- package/lib/cli/transcript-fence.js +73 -0
- package/lib/reader/artifact-reader.d.ts +2 -8
- package/lib/reader/artifact-reader.js +2 -19
- package/lib/reader/index.d.ts +1 -0
- package/lib/reader/index.js +1 -0
- package/lib/reader/karaoke.d.ts +13 -0
- package/lib/reader/karaoke.js +33 -0
- package/lib/reader/player-bar.js +2 -1
- package/lib/routes/recipients-routes.js +1 -1
- package/lib/routes/state-routes.js +21 -3
- package/lib/widgets/voice-note.d.ts +10 -0
- package/lib/widgets/voice-note.js +125 -0
- package/package.json +4 -1
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,88 @@
|
|
|
3
3
|
`@supersuit/artifacts`. One entry per version, newest first. Each entry says what changed, how a
|
|
4
4
|
host can tell whether it is affected (DETECTOR), what a host does about it (REMEDY), and the tests.
|
|
5
5
|
|
|
6
|
+
## 0.44.0
|
|
7
|
+
|
|
8
|
+
**Voice notes with their own words, lit as they are spoken.** An audio line followed directly by a
|
|
9
|
+
` ```transcript ` fence of timed words (`<start> <end> <word>` per line, a blank line for a new
|
|
10
|
+
paragraph) now draws one card in the page's brand: the caption, a play button and a seek line
|
|
11
|
+
instead of the native bar, and the transcript beneath, where the word being spoken carries the
|
|
12
|
+
narration's highlight, words already said read normally and words to come are muted. Tapping a word
|
|
13
|
+
plays from there. Additive: a page with no transcript fence renders byte for byte as 0.43.0.
|
|
14
|
+
|
|
15
|
+
- Publish refuses, with the line in the file: a malformed transcript line, an end before its start,
|
|
16
|
+
a start before the word above, an empty fence, a name after `transcript`, and a transcript with no
|
|
17
|
+
audio line directly above it.
|
|
18
|
+
- The card is `data-nospeak` (the narration already skips images and fences), keeps
|
|
19
|
+
`data-artifact-media` on its `<audio>` for play tracking and the audio line's comment anchor, and
|
|
20
|
+
plays on iPhone with `preload="none"`, `playsinline` and no autoplay. Starting a note pauses any
|
|
21
|
+
other audio on the page.
|
|
22
|
+
- The karaoke primitives moved to `src/reader/karaoke.ts` and are shared by the narration and the
|
|
23
|
+
voice note: `litWordIndex` (same behaviour, still exported from `./reader`), new `begunCount`, and
|
|
24
|
+
one highlight rule.
|
|
25
|
+
- New bin `supersuit-transcript-fence <audio>`: runs a local transcriber already on the machine
|
|
26
|
+
(`mlx_whisper`, else `whisper`) with word timestamps and prints the fence. The package carries no
|
|
27
|
+
transcriber. New exports from `./artifacts`: `parseTranscriptFence`, `scanVoiceNotes`,
|
|
28
|
+
`transcriptFenceFromWords`, `wordsFromWhisperJson`, `isAudioSrc`, `TRANSCRIPT_LANG`,
|
|
29
|
+
`MAX_TRANSCRIPT_WORDS`.
|
|
30
|
+
|
|
31
|
+
DETECTOR: a page whose body already holds a ` ```transcript ` fence (it rendered as a code block)
|
|
32
|
+
now publishes only when that fence parses and sits under an audio line: `rg -n '^```transcript' <pages>`.
|
|
33
|
+
A host whose Tailwind build does not scan this package's `lib/` would draw the card unstyled; the
|
|
34
|
+
existing widgets need the same scan.
|
|
35
|
+
|
|
36
|
+
REMEDY: none for most hosts; bump. A page that used `transcript` as a plain code label renames the
|
|
37
|
+
fence (```text).
|
|
38
|
+
|
|
39
|
+
Tests: `src/artifacts/voice-note.test.ts` (grammar, publish refusals with line numbers, comment
|
|
40
|
+
anchors, fence writer, whisper JSON), `src/widgets/voice-note.test.tsx` (the card's markup, iPhone
|
|
41
|
+
attributes, an audio line without a transcript byte-identical to 0.43.0, lighting, tap to seek,
|
|
42
|
+
seek before load).
|
|
43
|
+
|
|
44
|
+
## 0.43.0
|
|
45
|
+
|
|
46
|
+
**A comment's id per reader in `/views`, the comments feed for one page, and a limit on new browsers
|
|
47
|
+
through a group link.** A consumer that keeps a local copy of a page's comments could not tell exactly
|
|
48
|
+
which ones it lacked: `/views` listed each reader's comments as `{ at, text }` with no id, and
|
|
49
|
+
`GET /api/comments` is the whole host's feed. Everything here is additive; a caller that ignores the
|
|
50
|
+
new fields and the new parameter sees 0.42.0.
|
|
51
|
+
|
|
52
|
+
- `GET /api/artifacts/<id>/views`: each comment gains `id` (the entry id the feed calls `entryId`) and
|
|
53
|
+
`parent` (the entry id it replies to, or null). Under `comment_name: required` every browser on a
|
|
54
|
+
group link is still listed under the link, each with its own id. Publisher replies stay out.
|
|
55
|
+
- `GET /api/comments?artifact=<page id>`: only that page's rows, read from that page's entries rather
|
|
56
|
+
than a scan of the host. `since`, `open`, `cursor` and `more` work over that page's comments. A value
|
|
57
|
+
that is not a page id is a 400; a well-formed id with no page is an empty feed. Without the parameter
|
|
58
|
+
the feed is unchanged.
|
|
59
|
+
- On a `comment_name: required` page, the first stored note of each new browser through one group
|
|
60
|
+
link (a reader key with no entry on the page yet, with or without an `artifact_anon` cookie) is
|
|
61
|
+
counted against the anonymous write limit, keyed by the link, and answers
|
|
62
|
+
`429 { error: "too many new readers through this link; try again in a minute" }` past 30 a minute.
|
|
63
|
+
Counted after validation, so a refused request spends nothing. A browser's later notes, a named link
|
|
64
|
+
and a page without the line are not counted.
|
|
65
|
+
- `summarizeViews` (a public export) takes comments with `id` and `parent` OPTIONAL: a caller passing
|
|
66
|
+
`{ readerKey, at, text }` compiles and runs as before, and gets `id: null, parent: null` on each item.
|
|
67
|
+
- README: the `id` and `parent` shapes, the `artifact` parameter, and the note that removing
|
|
68
|
+
`comment_name: required` later turns a group link back into one shared reader and hides every
|
|
69
|
+
per-browser note from its writer.
|
|
70
|
+
|
|
71
|
+
DETECTOR: a consumer that reads `comments[]` from `/views` and rebuilds each item field by field, or
|
|
72
|
+
that rejects unknown keys, meets two new ones: `rg -n "\.comments" <your consumer>`. A consumer that
|
|
73
|
+
paged the host-wide feed to find one page's comments can ask for that page alone. A group link handed
|
|
74
|
+
to a crowd that arrives all at once (more than 30 new browsers writing inside a minute) now sees 429 for
|
|
75
|
+
the rest of that minute. A caller already sending a stray `artifact=` parameter on the feed, ignored
|
|
76
|
+
until now, gets a filtered feed, or a 400 if it is not a page id.
|
|
77
|
+
|
|
78
|
+
REMEDY: read `id` and `parent` where present and tolerate them where not; add `&artifact=<id>` to a
|
|
79
|
+
per-page poll. Nothing is needed for the limit: the same people write again the next minute.
|
|
80
|
+
|
|
81
|
+
Tests: `src/routes/comment-ids.test.tsx` (ids equal the feed's `entryId`; a reply's `parent`; two
|
|
82
|
+
browsers on one group link under the link with distinct ids; no publisher reply; `artifact=` returns
|
|
83
|
+
one page's rows and composes with `since` and `open=1`; paging 300 comments of one page among 450 in
|
|
84
|
+
pages of 200 never returns another page's row; the unfiltered feed unchanged; a malformed id is 400
|
|
85
|
+
and an unknown page is empty; the limit trips for cookieless writes through one link, not for a
|
|
86
|
+
returning browser, a named link or a page without the line). Every test builds its own stores.
|
|
87
|
+
|
|
6
88
|
## 0.42.0
|
|
7
89
|
|
|
8
90
|
**One page can say who sees its version number and past versions: `history: everyone | admins` in
|
package/README.md
CHANGED
|
@@ -620,6 +620,62 @@ visibility: shared # optional: private | shared
|
|
|
620
620
|
can leave or read notes; the banner, watermark and print refusal are unchanged. On a public
|
|
621
621
|
page with `writers: signed-in`, the control offers sign-in through `signInOrigin`.
|
|
622
622
|
|
|
623
|
+
## Voice notes with their words (`transcript`)
|
|
624
|
+
|
|
625
|
+
An audio line in a page (``, or `.mp3`) plays as a clip with the native
|
|
626
|
+
player. Put a `transcript` fence **directly under it** and it becomes a voice note: one card in the
|
|
627
|
+
page's brand with the caption, a play button and a seek line, and the speaker's own words beneath,
|
|
628
|
+
lit one at a time as they are spoken. Words already said read normally, the word being said carries
|
|
629
|
+
the narration's highlight, and the words still to come are a little muted. Tapping a word plays
|
|
630
|
+
from there.
|
|
631
|
+
|
|
632
|
+
````markdown
|
|
633
|
+

|
|
634
|
+
|
|
635
|
+
```transcript
|
|
636
|
+
0.00 0.62 Hello!
|
|
637
|
+
0.80 1.10 I
|
|
638
|
+
1.10 1.30 hope
|
|
639
|
+
|
|
640
|
+
2.00 2.40 Love,
|
|
641
|
+
2.40 2.90 Robin
|
|
642
|
+
```
|
|
643
|
+
````
|
|
644
|
+
|
|
645
|
+
- **One word per line: its start and end in seconds, then the word as it should read.** A phrase
|
|
646
|
+
timed together is one line; any script works. **A blank line starts a new paragraph.**
|
|
647
|
+
- **The audio stays plain image syntax**, so the publisher uploads a local file as it does any clip
|
|
648
|
+
(including under `media: private`) and needs no change. The words travel inside the page.
|
|
649
|
+
- **Refused at publish, naming the line in the file:** a line that is not `<start> <end> <word>`, a
|
|
650
|
+
word that ends before it starts, a word that starts before the word above it, an empty fence, a
|
|
651
|
+
name after `transcript`, and a `transcript` fence with no audio line directly above it (a second
|
|
652
|
+
fence under one note included).
|
|
653
|
+
- **The page's narration skips it** (the card is `data-nospeak`, and the narration text never reads
|
|
654
|
+
images or fences): a reader hears the speaker, never a robot reading the speaker.
|
|
655
|
+
- **On a phone:** nothing is fetched until play (`preload="none"`), no autoplay, `playsinline`. A
|
|
656
|
+
tapped word before the clip has loaded seeks once its metadata arrives. Starting a note pauses any
|
|
657
|
+
other audio on the page, the narration included.
|
|
658
|
+
- **Play tracking is unchanged:** the `<audio>` keeps `data-artifact-media`, so a play counts like
|
|
659
|
+
any clip. The note keeps the audio line's comment anchor.
|
|
660
|
+
- An audio line **without** a transcript fence renders exactly as before.
|
|
661
|
+
|
|
662
|
+
**Making the fence** (never by hand): `supersuit-transcript-fence` ships with the package and runs
|
|
663
|
+
a transcriber already on your machine, locally, with word timestamps: `mlx_whisper` (Apple Silicon,
|
|
664
|
+
`pip install mlx-whisper`), else openai-whisper's `whisper`. The package itself carries no
|
|
665
|
+
transcriber.
|
|
666
|
+
|
|
667
|
+
```bash
|
|
668
|
+
npx -p @supersuit/artifacts supersuit-transcript-fence hello.m4a > fence.md
|
|
669
|
+
# --language zh the spoken language, when auto-detect guesses wrong
|
|
670
|
+
# --gap 1.5 seconds of silence that start a new paragraph (default 1.5)
|
|
671
|
+
# --model <name> the whisper model (default mlx-community/whisper-large-v3-turbo)
|
|
672
|
+
# --body the lines only, without the ``` fence
|
|
673
|
+
```
|
|
674
|
+
|
|
675
|
+
Paste it under the audio line, then correct any word (a name, a phrase in another language)
|
|
676
|
+
and keep its two times. `transcriptFenceFromWords` and `wordsFromWhisperJson` (from
|
|
677
|
+
`@supersuit/artifacts/artifacts`) do the same from code.
|
|
678
|
+
|
|
623
679
|
## Comments and personal notes
|
|
624
680
|
|
|
625
681
|
Every page takes comments pinned to any part of it. A reader selects text (a **Comment** chip
|
|
@@ -851,6 +907,19 @@ Authorization: Bearer <publish key>
|
|
|
851
907
|
quote, region, body, transcript, audioUrl, at, resolved, link } ] }
|
|
852
908
|
```
|
|
853
909
|
|
|
910
|
+
**One page's feed (0.43.0).** `&artifact=<page id>` returns only that page's rows. `since`, `open`,
|
|
911
|
+
`cursor` and `more` behave as above, over that page's comments alone, so a caller paging one page
|
|
912
|
+
reaches its end without reading another page's rows. The feed is read from that page's own entries,
|
|
913
|
+
not from a scan of the host. A value that is not a page id is a `400` ("artifact must be a page id,
|
|
914
|
+
like abcd2345"); a well-formed id with no page, or no comments, answers an empty feed
|
|
915
|
+
(`{ now, comments: [], cursor: null, more: false }`), as an empty host does. Without the parameter
|
|
916
|
+
the feed is what it was.
|
|
917
|
+
|
|
918
|
+
```
|
|
919
|
+
GET /api/comments?since=2026-09-28T12:00:00Z&artifact=abcd2345&open=1
|
|
920
|
+
Authorization: Bearer <publish key>
|
|
921
|
+
```
|
|
922
|
+
|
|
854
923
|
Oldest first, at most 200, strictly after `since` (an ISO 8601 time with a zone; anything else is
|
|
855
924
|
a 400). `name` is null for an anonymous reader; `reader` is `{ rid, label }` for a comment left through a per-reader link, else null; `by` is the name its writer typed, else null; `publisher` is true for a reply written through the reply route; `quote` is the text the comment is pinned to, or
|
|
856
925
|
null for a box (`region: true`); `parent` is the comment a reply answers; `audioUrl` is a signed
|
|
@@ -911,7 +980,7 @@ Routes (spread into your route files by `createArtifactRoutes`): `RECIPIENTS`, `
|
|
|
911
980
|
|---|---|---|
|
|
912
981
|
| `GET /api/artifacts/<id>/recipients` | publish key | Every reader, with the working link for each |
|
|
913
982
|
| `POST /api/artifacts/<id>/recipients` | publish key | Body `{ add?: [{ label, note?, group?, closes? }], revoke?: [rid] or '*', expires?: ISO time or null, closes?: { rids: [rid] or '*', at: ISO time or null } }`. `add` answers each new reader's link; `expires: null` is Keep; `closes` sets, moves or clears a reader's feedback window |
|
|
914
|
-
| `GET /api/artifacts/<id>/views` | publish key | One line per reader: opened, came back, active seconds, plays, devices, comments. A page with no readers yet answers 200 with an empty list; a missing page answers 404 |
|
|
983
|
+
| `GET /api/artifacts/<id>/views` | publish key | One line per reader: opened, came back, active seconds, plays, devices, comments. Each comment is `{ id, parent, at, text }` (see Comment ids in /views). A page with no readers yet answers 200 with an empty list; a missing page answers 404 |
|
|
915
984
|
| `POST /api/artifacts/<id>/seen` | the page itself (public) | The reader's own signal, below |
|
|
916
985
|
|
|
917
986
|
Config, both on `createArtifactRoutes`: `recipients` (a store from `createRecipientsStore(db, '<collection path>')`, or `memoryRecipientsStore()` in tests) and `recipientSecret`, a function returning the signing secret, read on every request. A secret shorter than 32 characters is treated as unset: per-reader pages stay shut and the routes answer 404 or 501. Signing with an empty secret throws.
|
|
@@ -978,6 +1047,35 @@ the one shared reader (`r:<rid>`) and are shown to no browser afterwards. A name
|
|
|
978
1047
|
reader either way. The feedback window, the feed's `reader: { rid, label }` and the `/views` line
|
|
979
1048
|
stay per link: every browser on a group link shares them.
|
|
980
1049
|
|
|
1050
|
+
### Comment ids in `/views` (0.43.0)
|
|
1051
|
+
|
|
1052
|
+
Each comment in a reader's `comments` array carries the entry id the comments feed calls
|
|
1053
|
+
`entryId`, and the id of the comment it replies to:
|
|
1054
|
+
|
|
1055
|
+
```json
|
|
1056
|
+
{ "rid": "abcdefghijkl", "label": "Anyone with the link", "group": true, "comments": [
|
|
1057
|
+
{ "id": "kq3vN8xTz2LwPd5mRa", "parent": null, "at": "2026-10-11T10:01:00.000Z", "text": "Which river?" },
|
|
1058
|
+
{ "id": "Jm7cHy4bQe9sUf1xWk", "parent": "kq3vN8xTz2LwPd5mRa", "at": "2026-10-11T10:04:00.000Z", "text": "The north one." } ] }
|
|
1059
|
+
```
|
|
1060
|
+
|
|
1061
|
+
`at` and `text` are as before. Under `comment_name: required` the notes of every browser on a
|
|
1062
|
+
group link are listed under that link's row, each with its own `id`. The publisher's replies are not
|
|
1063
|
+
listed. A caller that keeps a local copy of a page's comments compares these ids with
|
|
1064
|
+
`GET /api/comments?artifact=<id>` to see exactly which ones it lacks.
|
|
1065
|
+
|
|
1066
|
+
**Limit on new browsers.** On a page with `comment_name: required`, the first stored note of each new
|
|
1067
|
+
browser through one group link is counted against that link, per minute, on the anonymous write limit
|
|
1068
|
+
(30). A browser is new when its reader key has no entry on the page yet, whether or not it arrived with
|
|
1069
|
+
an `artifact_anon` cookie. Past the limit the state route answers
|
|
1070
|
+
`429 { error: "too many new readers through this link; try again in a minute" }`. Only a stored note is
|
|
1071
|
+
counted: a request refused for a missing name, a closed window or a malformed body spends nothing. A
|
|
1072
|
+
browser's later notes, a named link and a page without the line are never counted. "Note" here means
|
|
1073
|
+
any stored entry in any slot, and a browser that removes its only entry and writes again is new again.
|
|
1074
|
+
|
|
1075
|
+
**Removing the line later.** Taking `comment_name: required` off a page turns its group link back into
|
|
1076
|
+
one shared reader, and every note left per browser stops being shown to its writer: the notes stay with
|
|
1077
|
+
the publisher, under their per-browser keys.
|
|
1078
|
+
|
|
981
1079
|
## Broadcasts (`template: broadcast`)
|
|
982
1080
|
|
|
983
1081
|
A broadcast is its own page kind, not a document with extra flags. It is one message sent to several people, each through their own link, and it renders as one card: the sender's words at the top, then the media in the order the words put it (a video with sound and controls, photos, a voice note's player, link cards), then that person's own note. None of the document's furniture: no table of contents, no version line, no read-along, and no page menu for a reader.
|
package/lib/artifacts/index.d.ts
CHANGED
|
@@ -27,3 +27,4 @@ export type { PageSettings, SettingsFact, AccessLevel } from '../reader/settings
|
|
|
27
27
|
export { ridLink, parseRidParam, verifyRid, isUnfurlAgent, usableSecret, RECIPIENT_SECRET_MIN, ridOfReaderKey } from './recipients.js';
|
|
28
28
|
export { createRecipientsStore, memoryRecipientsStore, summarizeViews, type RecipientsStore, type Recipient, type RecipientsRecord, type ViewDoc, type RecipientSummary } from './recipients-store.js';
|
|
29
29
|
export { migrateMemosToPrivate, MEMO_KEY, type MemoMigration } from './migrate-memos.js';
|
|
30
|
+
export { parseTranscriptFence, scanVoiceNotes, transcriptFenceFromWords, wordsFromWhisperJson, isAudioSrc, TRANSCRIPT_LANG, MAX_TRANSCRIPT_WORDS, type TranscriptParagraphs, type TimedWord, type VoiceNoteBlock } from './voice-note.js';
|
package/lib/artifacts/index.js
CHANGED
|
@@ -27,3 +27,4 @@ export { settleOnPublish, applySettingsPatch, withComments, declaredOf, password
|
|
|
27
27
|
export { ridLink, parseRidParam, verifyRid, isUnfurlAgent, usableSecret, RECIPIENT_SECRET_MIN, ridOfReaderKey } from './recipients.js';
|
|
28
28
|
export { createRecipientsStore, memoryRecipientsStore, summarizeViews } from './recipients-store.js';
|
|
29
29
|
export { migrateMemosToPrivate, MEMO_KEY } from './migrate-memos.js';
|
|
30
|
+
export { parseTranscriptFence, scanVoiceNotes, transcriptFenceFromWords, wordsFromWhisperJson, isAudioSrc, TRANSCRIPT_LANG, MAX_TRANSCRIPT_WORDS } from './voice-note.js';
|
|
@@ -38,6 +38,8 @@ export type RecipientSummary = {
|
|
|
38
38
|
plays: number;
|
|
39
39
|
devices: number;
|
|
40
40
|
comments: {
|
|
41
|
+
id: string | null;
|
|
42
|
+
parent: string | null;
|
|
41
43
|
at: string;
|
|
42
44
|
text: string;
|
|
43
45
|
}[];
|
|
@@ -78,6 +80,8 @@ export declare function createRecipientsStore(db: Firestore, base: string): Reci
|
|
|
78
80
|
/** One line per reader (never the owner), in the order they were added. */
|
|
79
81
|
export declare function summarizeViews(rec: RecipientsRecord, views: ViewDoc[], comments: {
|
|
80
82
|
readerKey: string;
|
|
83
|
+
id?: string | null;
|
|
84
|
+
parent?: string | null;
|
|
81
85
|
at: string;
|
|
82
86
|
text: string;
|
|
83
87
|
}[]): RecipientSummary[];
|
|
@@ -179,7 +179,7 @@ export function summarizeViews(rec, views, comments) {
|
|
|
179
179
|
activeSeconds: mine.reduce((n, v) => n + v.activeSeconds, 0),
|
|
180
180
|
plays: mine.reduce((n, v) => n + Object.values(v.plays).reduce((a, b) => a + b, 0), 0),
|
|
181
181
|
devices: new Set(mine.map((v) => v.device)).size,
|
|
182
|
-
comments: comments.filter((c) => ridOfReaderKey(c.readerKey) === r.rid).map((c) => ({ at: c.at, text: c.text })),
|
|
182
|
+
comments: comments.filter((c) => ridOfReaderKey(c.readerKey) === r.rid).map((c) => ({ id: c.id ?? null, parent: c.parent ?? null, at: c.at, text: c.text })),
|
|
183
183
|
};
|
|
184
184
|
});
|
|
185
185
|
}
|
package/lib/artifacts/render.js
CHANGED
|
@@ -6,6 +6,7 @@ import { jsx as _jsx, jsxs as _jsxs, Fragment as _Fragment } from "react/jsx-run
|
|
|
6
6
|
// ```links one "url | note" per line, rendered as link cards
|
|
7
7
|
// ```checklist items a reader ticks, kept in their browser (widgets/checklist.tsx)
|
|
8
8
|
// > [!note] / > [!warning] a callout instead of a blockquote
|
|
9
|
+
// ```transcript right under an audio line: a voice note whose words light up as spoken (voice-note.ts)
|
|
9
10
|
// A checklist's item text and descriptions are markdown drawn by the same safe renderer: raw
|
|
10
11
|
// HTML escaped, links only where a reader can follow them, no images.
|
|
11
12
|
import { Children, isValidElement } from 'react';
|
|
@@ -22,6 +23,8 @@ import { VideoAutoplay } from '../reader/video-autoplay.js';
|
|
|
22
23
|
import { CodeCopy } from '../reader/code-copy.js';
|
|
23
24
|
import { DefinitionLayer } from '../reader/definition-layer.js';
|
|
24
25
|
import { remarkDefinitions } from './definitions.js';
|
|
26
|
+
import { AUDIO_SRC, remarkVoiceNotes, TRANSCRIPT_LANG } from './voice-note.js';
|
|
27
|
+
import { VoiceNote } from '../widgets/voice-note.js';
|
|
25
28
|
// Every colour is a theme variable (brand/theme.ts), so a page follows its light, dark or system mode.
|
|
26
29
|
const GOLD = 'var(--a-accent)';
|
|
27
30
|
function hostOf(url) {
|
|
@@ -78,7 +81,7 @@ const VIDEO = /\.mp4(?:[?#].*)?$/i;
|
|
|
78
81
|
/** `` is a clip the reader plays: an annotated conversation, a voice note, a
|
|
79
82
|
* quote heard rather than read. Native controls, and `preload="none"` so a page of clips fetches
|
|
80
83
|
* nothing until one is pressed. The alt is its caption, shown above the player. */
|
|
81
|
-
const AUDIO =
|
|
84
|
+
const AUDIO = AUDIO_SRC;
|
|
82
85
|
/** The stored file's name, for play tracking: the last path segment before any query. */
|
|
83
86
|
const mediaName = (src) => {
|
|
84
87
|
const last = src.split(/[?#]/)[0].split('/').pop() ?? '';
|
|
@@ -161,8 +164,17 @@ function ChecklistFence({ line, ctx }) {
|
|
|
161
164
|
return null;
|
|
162
165
|
return (_jsx(Checklist, { items: c.items.map((it) => ({ id: it.id, title: checklistMarkdown(it.text, true), description: it.description ? checklistMarkdown(it.description, false) : null })), artifactId: ctx.artifactId, send: ctx.send && ctx.artifactId && !ctx.sync ? c.send : undefined, sync: ctx.sync && !!ctx.artifactId, owner: ctx.owner, accent: ctx.accent }));
|
|
163
166
|
}
|
|
164
|
-
function baseComponents(block, figure, checklists, embeds) {
|
|
167
|
+
function baseComponents(block, figure, checklists, embeds, voiceNotes) {
|
|
165
168
|
return {
|
|
169
|
+
// A voice note (remarkVoiceNotes): the audio line and its transcript, drawn as one keepsake.
|
|
170
|
+
div: ({ node, ...props }) => {
|
|
171
|
+
const at = props['data-artifact-voicenote-at'];
|
|
172
|
+
const v = at === undefined ? undefined : voiceNotes?.[Number(at)];
|
|
173
|
+
// Any other div (a section body, a definition) is drawn as it would be with no override.
|
|
174
|
+
if (!v)
|
|
175
|
+
return _jsx("div", { ...props });
|
|
176
|
+
return _jsx(VoiceNote, { src: artifactUrlTransform(v.src), label: v.label, media: mediaName(v.src), paragraphs: v.paragraphs, block: block(node) });
|
|
177
|
+
},
|
|
166
178
|
p: ({ node, children }) => {
|
|
167
179
|
// An image alone in its paragraph that the pack draws itself replaces the paragraph, so the
|
|
168
180
|
// pack's figure may hold block content (a <p> cannot) and still carry the comment anchor.
|
|
@@ -201,6 +213,9 @@ function baseComponents(block, figure, checklists, embeds) {
|
|
|
201
213
|
// answers) it draws nothing: its settings are not prose and never shown as code.
|
|
202
214
|
if (lang === 'notes')
|
|
203
215
|
return null;
|
|
216
|
+
// A transcript not under an audio line draws nothing (publish refuses one; an old page may hold it).
|
|
217
|
+
if (lang === TRANSCRIPT_LANG)
|
|
218
|
+
return null;
|
|
204
219
|
if (lang === 'simulator' || lang === 'messages')
|
|
205
220
|
return _jsx(EmbedFence, { kind: lang, source: textOf(children), ctx: embeds?.[lang] });
|
|
206
221
|
if (!className) {
|
|
@@ -212,7 +227,7 @@ function baseComponents(block, figure, checklists, embeds) {
|
|
|
212
227
|
// A links fence renders its own block; do not wrap it in <pre>.
|
|
213
228
|
const inner = Array.isArray(children) ? children[0] : children;
|
|
214
229
|
const cls = inner?.props?.className ?? '';
|
|
215
|
-
if (/language-(links|notes|checklist|simulator|messages)/.test(cls))
|
|
230
|
+
if (/language-(links|notes|checklist|simulator|messages|transcript)/.test(cls))
|
|
216
231
|
return _jsx(_Fragment, { children: children });
|
|
217
232
|
return (_jsx("pre", { ...block(node), "data-artifact-code": true,
|
|
218
233
|
// whitespace-pre-wrap, because a fence here usually holds PROSE meant to be pasted
|
|
@@ -245,20 +260,20 @@ const HEADING_CLASS = {
|
|
|
245
260
|
};
|
|
246
261
|
/** Components for any page: every heading gets its slug as an id, the one headingsOf derived,
|
|
247
262
|
* so a contents link and a note both land on it. */
|
|
248
|
-
function pageComponents(headings, block, figure, checklists, embeds) {
|
|
263
|
+
function pageComponents(headings, block, figure, checklists, embeds, voiceNotes) {
|
|
249
264
|
const byLine = new Map(headings.map((h) => [h.line, h]));
|
|
250
265
|
const heading = (depth) => function Heading({ node, children }) {
|
|
251
266
|
const Tag = `h${depth}`;
|
|
252
267
|
const h = byLine.get(node?.position?.start.line ?? -1);
|
|
253
268
|
return _jsx(Tag, { id: h?.slug, ...block(node), className: HEADING_CLASS[depth], children: children });
|
|
254
269
|
};
|
|
255
|
-
return { ...baseComponents(block, figure, checklists, embeds), h1: heading(1), h2: heading(2), h3: heading(3), h4: heading(4), h5: heading(5), h6: heading(6) };
|
|
270
|
+
return { ...baseComponents(block, figure, checklists, embeds, voiceNotes), h1: heading(1), h2: heading(2), h3: heading(3), h4: heading(4), h5: heading(5), h6: heading(6) };
|
|
256
271
|
}
|
|
257
272
|
/** Components for a page with a notes block: every heading gets its slug as an id, a note
|
|
258
273
|
* control, and its notes after it. The heading is found by its source line, so the slug is the
|
|
259
274
|
* one headingsOf derived, the same one a note stores. */
|
|
260
|
-
function notesComponents(headings, block, figure, checklists, embeds) {
|
|
261
|
-
const components = baseComponents(block, figure, checklists, embeds);
|
|
275
|
+
function notesComponents(headings, block, figure, checklists, embeds, voiceNotes) {
|
|
276
|
+
const components = baseComponents(block, figure, checklists, embeds, voiceNotes);
|
|
262
277
|
const byLine = new Map(headings.map((h) => [h.line, h]));
|
|
263
278
|
const heading = (depth) => function NotedHeading({ node, children }) {
|
|
264
279
|
const Tag = `h${depth}`;
|
|
@@ -348,14 +363,17 @@ export function ArtifactMarkdown({ markdown, notes, checklist, definitions, Sect
|
|
|
348
363
|
};
|
|
349
364
|
const defined = definitions?.length ? definitions : null;
|
|
350
365
|
const outline = Section ? sectionOutline(markdown, headings) : [];
|
|
351
|
-
|
|
366
|
+
// Voice notes first, so a section groups the note node the pair became.
|
|
367
|
+
const voiceNotes = [];
|
|
368
|
+
const voiced = /^ {0,3}(`{3,}|~{3,})\s*transcript\b/m.test(markdown);
|
|
369
|
+
const plugins = [remarkGfm, ...(voiced ? [remarkVoiceNotes(voiceNotes)] : []), ...(defined ? [remarkDefinitions(defined)] : []), ...(outline.length ? [remarkSections(outline[0].depth)] : [])];
|
|
352
370
|
const lists = /^ {0,3}(`{3,}|~{3,})\s*checklist\b/m.test(markdown) ? checklistsOf(markdown) : null;
|
|
353
371
|
const checklists = lists
|
|
354
372
|
// `send` from the host means it keeps answers and the page declared the `checklist` slot; on a
|
|
355
373
|
// page whose checklists sync, that same slot is the shared set, so it turns sync on instead.
|
|
356
374
|
? { byLine: lists, artifactId: checklist?.artifactId, send: !!checklist?.send, sync: !!checklist?.send && checklistSyncs(lists.values()), owner: checklist?.owner, accent: checklist?.accent }
|
|
357
375
|
: undefined;
|
|
358
|
-
const components = on ? notesComponents(headings, block, figure, checklists, embeds) : pageComponents(headings, block, figure, checklists, embeds);
|
|
376
|
+
const components = on ? notesComponents(headings, block, figure, checklists, embeds, voiceNotes) : pageComponents(headings, block, figure, checklists, embeds, voiceNotes);
|
|
359
377
|
if (outline.length)
|
|
360
378
|
components.section = sectionComponent(Section, outline, block);
|
|
361
379
|
const body = (_jsx(ReactMarkdown, { remarkPlugins: plugins, components: components, urlTransform: artifactUrlTransform, children: markdown }));
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
import type { Root, RootContent, Image } from 'mdast';
|
|
2
|
+
import type { WordTiming } from '../reader/karaoke.js';
|
|
3
|
+
export declare const TRANSCRIPT_LANG = "transcript";
|
|
4
|
+
/** `` / `.m4a` is a clip the reader plays. */
|
|
5
|
+
export declare const AUDIO_SRC: RegExp;
|
|
6
|
+
export declare const isAudioSrc: (src: unknown) => src is string;
|
|
7
|
+
/** A page this long is a book, not a voice note. */
|
|
8
|
+
export declare const MAX_TRANSCRIPT_WORDS = 20000;
|
|
9
|
+
export type TranscriptParagraphs = WordTiming[][];
|
|
10
|
+
/** Parses a transcript fence's body. `line` is the line the fence opens on in the file, so every
|
|
11
|
+
* error names the line the author sees in their editor. */
|
|
12
|
+
export declare function parseTranscriptFence(value: string, line: number): {
|
|
13
|
+
ok: true;
|
|
14
|
+
paragraphs: TranscriptParagraphs;
|
|
15
|
+
} | {
|
|
16
|
+
ok: false;
|
|
17
|
+
error: string;
|
|
18
|
+
};
|
|
19
|
+
/** The audio image a paragraph holds and nothing else (whitespace aside), or null. */
|
|
20
|
+
export declare function soleAudio(n: RootContent | undefined): Image | null;
|
|
21
|
+
/** The publish-time check: every transcript fence parses, takes no name, and sits directly under
|
|
22
|
+
* an audio line. `offset` is how many lines of the file sit above the body. */
|
|
23
|
+
export declare function scanVoiceNotes(tree: Root, offset: number): {
|
|
24
|
+
ok: true;
|
|
25
|
+
count: number;
|
|
26
|
+
} | {
|
|
27
|
+
ok: false;
|
|
28
|
+
error: string;
|
|
29
|
+
};
|
|
30
|
+
/** What the renderer draws for one voice note. */
|
|
31
|
+
export type VoiceNoteBlock = {
|
|
32
|
+
src: string;
|
|
33
|
+
label: string;
|
|
34
|
+
paragraphs: TranscriptParagraphs;
|
|
35
|
+
};
|
|
36
|
+
/** Replaces each audio line + transcript pair with one node the renderer draws as a voice note,
|
|
37
|
+
* keeping the audio line's position so its comment anchor (blocksOf) is unchanged. A pair that
|
|
38
|
+
* does not parse is left alone: the audio plays as before and the fence draws nothing. `found`
|
|
39
|
+
* collects the notes; the node carries its index. */
|
|
40
|
+
export declare function remarkVoiceNotes(found: VoiceNoteBlock[]): () => (tree: Root) => void;
|
|
41
|
+
/** A word as a transcriber reports it (mlx_whisper / openai-whisper `word_timestamps`). */
|
|
42
|
+
export type TimedWord = {
|
|
43
|
+
word: string;
|
|
44
|
+
start: number;
|
|
45
|
+
end: number;
|
|
46
|
+
};
|
|
47
|
+
/** The body of a transcript fence for a list of timed words: one word per line, and a blank line
|
|
48
|
+
* wherever the speaker paused at least `paragraphGap` seconds. Times never run backwards, so the
|
|
49
|
+
* result always passes the publish check. Correct a word's spelling afterwards; keep its times. */
|
|
50
|
+
export declare function transcriptFenceFromWords(words: TimedWord[], opts?: {
|
|
51
|
+
paragraphGap?: number;
|
|
52
|
+
}): string;
|
|
53
|
+
/** The timed words in a whisper JSON result (`segments[].words[]`, written by `mlx_whisper` or
|
|
54
|
+
* `whisper` with word timestamps on). Anything else in the file is ignored. */
|
|
55
|
+
export declare function wordsFromWhisperJson(json: unknown): TimedWord[];
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
export const TRANSCRIPT_LANG = 'transcript';
|
|
2
|
+
/** `` / `.m4a` is a clip the reader plays. */
|
|
3
|
+
export const AUDIO_SRC = /\.(mp3|m4a)(?:[?#].*)?$/i;
|
|
4
|
+
export const isAudioSrc = (src) => typeof src === 'string' && AUDIO_SRC.test(src);
|
|
5
|
+
/** A page this long is a book, not a voice note. */
|
|
6
|
+
export const MAX_TRANSCRIPT_WORDS = 20000;
|
|
7
|
+
const ROW = /^(\d+(?:\.\d+)?)\s+(\d+(?:\.\d+)?)\s+(\S.*)$/;
|
|
8
|
+
const GRAMMAR = 'a transcript line is "<start> <end> <word>" in seconds, like "1.10 1.30 hope"';
|
|
9
|
+
/** Parses a transcript fence's body. `line` is the line the fence opens on in the file, so every
|
|
10
|
+
* error names the line the author sees in their editor. */
|
|
11
|
+
export function parseTranscriptFence(value, line) {
|
|
12
|
+
const paragraphs = [];
|
|
13
|
+
let current = [];
|
|
14
|
+
let prev = null;
|
|
15
|
+
let count = 0;
|
|
16
|
+
const rows = value.split('\n');
|
|
17
|
+
for (let i = 0; i < rows.length; i++) {
|
|
18
|
+
const raw = rows[i].trim();
|
|
19
|
+
const at = line + 1 + i;
|
|
20
|
+
if (!raw) {
|
|
21
|
+
if (current.length)
|
|
22
|
+
paragraphs.push(current);
|
|
23
|
+
current = [];
|
|
24
|
+
continue;
|
|
25
|
+
}
|
|
26
|
+
const m = ROW.exec(raw);
|
|
27
|
+
if (!m)
|
|
28
|
+
return { ok: false, error: `line ${at}: ${GRAMMAR}` };
|
|
29
|
+
const s = Number(m[1]);
|
|
30
|
+
const e = Number(m[2]);
|
|
31
|
+
if (e < s)
|
|
32
|
+
return { ok: false, error: `line ${at}: this word ends before it starts (${m[1]} to ${m[2]})` };
|
|
33
|
+
if (prev && s < prev.s)
|
|
34
|
+
return { ok: false, error: `line ${at}: this word starts at ${m[1]}, before the word above it (${prev.s}); words go in the order they are spoken` };
|
|
35
|
+
if (++count > MAX_TRANSCRIPT_WORDS)
|
|
36
|
+
return { ok: false, error: `line ${at}: a transcript takes at most ${MAX_TRANSCRIPT_WORDS} words` };
|
|
37
|
+
prev = { w: m[3].trim(), s, e };
|
|
38
|
+
current.push(prev);
|
|
39
|
+
}
|
|
40
|
+
if (current.length)
|
|
41
|
+
paragraphs.push(current);
|
|
42
|
+
if (!count)
|
|
43
|
+
return { ok: false, error: `line ${line}: this transcript has no words; ${GRAMMAR}` };
|
|
44
|
+
return { ok: true, paragraphs };
|
|
45
|
+
}
|
|
46
|
+
/** The audio image a paragraph holds and nothing else (whitespace aside), or null. */
|
|
47
|
+
export function soleAudio(n) {
|
|
48
|
+
if (!n || n.type !== 'paragraph')
|
|
49
|
+
return null;
|
|
50
|
+
const kids = n.children.filter((c) => !(c.type === 'text' && !c.value.trim()));
|
|
51
|
+
return kids.length === 1 && kids[0].type === 'image' && isAudioSrc(kids[0].url) ? kids[0] : null;
|
|
52
|
+
}
|
|
53
|
+
const isParent = (n) => !!n && typeof n === 'object' && Array.isArray(n.children);
|
|
54
|
+
/** Visits every transcript fence with the sibling right above it, at any depth. */
|
|
55
|
+
function eachTranscript(tree, visit) {
|
|
56
|
+
const walk = (p) => {
|
|
57
|
+
p.children.forEach((n, i) => {
|
|
58
|
+
if (n.type === 'code' && n.lang === TRANSCRIPT_LANG)
|
|
59
|
+
visit(n, p.children[i - 1]);
|
|
60
|
+
else if (isParent(n) && n.type !== 'paragraph' && n.type !== 'heading')
|
|
61
|
+
walk(n);
|
|
62
|
+
});
|
|
63
|
+
};
|
|
64
|
+
walk(tree);
|
|
65
|
+
}
|
|
66
|
+
/** The publish-time check: every transcript fence parses, takes no name, and sits directly under
|
|
67
|
+
* an audio line. `offset` is how many lines of the file sit above the body. */
|
|
68
|
+
export function scanVoiceNotes(tree, offset) {
|
|
69
|
+
let error = null;
|
|
70
|
+
let count = 0;
|
|
71
|
+
eachTranscript(tree, (fence, above) => {
|
|
72
|
+
if (error)
|
|
73
|
+
return;
|
|
74
|
+
const line = (fence.position?.start.line ?? 1) + offset;
|
|
75
|
+
if (fence.meta?.trim()) {
|
|
76
|
+
error = `line ${line}: the transcript block takes no name`;
|
|
77
|
+
return;
|
|
78
|
+
}
|
|
79
|
+
if (!soleAudio(above)) {
|
|
80
|
+
error = `line ${line}: a transcript belongs directly under its voice note: put an audio line like  right above it`;
|
|
81
|
+
return;
|
|
82
|
+
}
|
|
83
|
+
const r = parseTranscriptFence(fence.value, line);
|
|
84
|
+
if (!r.ok) {
|
|
85
|
+
error = r.error;
|
|
86
|
+
return;
|
|
87
|
+
}
|
|
88
|
+
count++;
|
|
89
|
+
});
|
|
90
|
+
return error ? { ok: false, error } : { ok: true, count };
|
|
91
|
+
}
|
|
92
|
+
/** Replaces each audio line + transcript pair with one node the renderer draws as a voice note,
|
|
93
|
+
* keeping the audio line's position so its comment anchor (blocksOf) is unchanged. A pair that
|
|
94
|
+
* does not parse is left alone: the audio plays as before and the fence draws nothing. `found`
|
|
95
|
+
* collects the notes; the node carries its index. */
|
|
96
|
+
export function remarkVoiceNotes(found) {
|
|
97
|
+
return () => (tree) => {
|
|
98
|
+
const walk = (p) => {
|
|
99
|
+
const out = [];
|
|
100
|
+
for (const n of p.children) {
|
|
101
|
+
const above = out[out.length - 1];
|
|
102
|
+
const audio = n.type === 'code' && n.lang === TRANSCRIPT_LANG && !n.meta?.trim() ? soleAudio(above) : null;
|
|
103
|
+
const r = audio ? parseTranscriptFence(n.value, 0) : null;
|
|
104
|
+
if (audio && r?.ok) {
|
|
105
|
+
found.push({ src: audio.url, label: audio.alt ?? '', paragraphs: r.paragraphs });
|
|
106
|
+
out[out.length - 1] = {
|
|
107
|
+
type: 'artifactVoiceNote',
|
|
108
|
+
position: above.position,
|
|
109
|
+
data: { hName: 'div', hProperties: { dataArtifactVoicenoteAt: String(found.length - 1) } },
|
|
110
|
+
children: [],
|
|
111
|
+
};
|
|
112
|
+
continue;
|
|
113
|
+
}
|
|
114
|
+
if (isParent(n) && n.type !== 'paragraph' && n.type !== 'heading')
|
|
115
|
+
walk(n);
|
|
116
|
+
out.push(n);
|
|
117
|
+
}
|
|
118
|
+
p.children = out;
|
|
119
|
+
};
|
|
120
|
+
walk(tree);
|
|
121
|
+
};
|
|
122
|
+
}
|
|
123
|
+
/** The body of a transcript fence for a list of timed words: one word per line, and a blank line
|
|
124
|
+
* wherever the speaker paused at least `paragraphGap` seconds. Times never run backwards, so the
|
|
125
|
+
* result always passes the publish check. Correct a word's spelling afterwards; keep its times. */
|
|
126
|
+
export function transcriptFenceFromWords(words, opts = {}) {
|
|
127
|
+
const gap = opts.paragraphGap ?? 1.5;
|
|
128
|
+
const lines = [];
|
|
129
|
+
let prev = null;
|
|
130
|
+
for (const w of words) {
|
|
131
|
+
const text = w.word.replace(/\s+/g, ' ').trim();
|
|
132
|
+
if (!text)
|
|
133
|
+
continue;
|
|
134
|
+
const start = Math.max(0, prev ? Math.max(w.start, prev.start) : w.start);
|
|
135
|
+
const end = Math.max(start, w.end);
|
|
136
|
+
if (prev && start - prev.end >= gap)
|
|
137
|
+
lines.push('');
|
|
138
|
+
lines.push(`${start.toFixed(2)} ${end.toFixed(2)} ${text}`);
|
|
139
|
+
prev = { start, end };
|
|
140
|
+
}
|
|
141
|
+
return lines.length ? lines.join('\n') + '\n' : '';
|
|
142
|
+
}
|
|
143
|
+
/** The timed words in a whisper JSON result (`segments[].words[]`, written by `mlx_whisper` or
|
|
144
|
+
* `whisper` with word timestamps on). Anything else in the file is ignored. */
|
|
145
|
+
export function wordsFromWhisperJson(json) {
|
|
146
|
+
const segments = json?.segments;
|
|
147
|
+
if (!Array.isArray(segments))
|
|
148
|
+
return [];
|
|
149
|
+
const out = [];
|
|
150
|
+
for (const seg of segments) {
|
|
151
|
+
const words = seg?.words;
|
|
152
|
+
if (!Array.isArray(words))
|
|
153
|
+
continue;
|
|
154
|
+
for (const w of words) {
|
|
155
|
+
const { word, start, end } = (w ?? {});
|
|
156
|
+
if (typeof word === 'string' && typeof start === 'number' && typeof end === 'number')
|
|
157
|
+
out.push({ word, start, end });
|
|
158
|
+
}
|
|
159
|
+
}
|
|
160
|
+
return out;
|
|
161
|
+
}
|
package/lib/artifacts/widgets.js
CHANGED
|
@@ -17,6 +17,7 @@ import remarkGfm from 'remark-gfm';
|
|
|
17
17
|
import { blockId } from './anchor.js';
|
|
18
18
|
import { MAX_NOTE_CHARS, NOTES_SLOT } from './notes-place.js';
|
|
19
19
|
import { isEmbedKind, parseEmbedFence } from './simulator.js';
|
|
20
|
+
import { scanVoiceNotes, TRANSCRIPT_LANG } from './voice-note.js';
|
|
20
21
|
import { CHECKLIST_SLOT, assignChecklistIds, parseChecklistFence } from './checklist.js';
|
|
21
22
|
export { MAX_NOTE_CHARS, NOTES_SLOT, placeNotes } from './notes-place.js';
|
|
22
23
|
/** Widgets the design names and this version does not draw yet. A fence using one is refused at
|
|
@@ -70,7 +71,8 @@ export function headingsOf(markdown) {
|
|
|
70
71
|
* (the front matter), so every error names the line the author sees in their editor. */
|
|
71
72
|
export function scanWidgets(body, offset) {
|
|
72
73
|
const fences = [];
|
|
73
|
-
|
|
74
|
+
const tree = parse(body);
|
|
75
|
+
walk(tree.children, (n) => { if (n.type === 'code')
|
|
74
76
|
fences.push(n); });
|
|
75
77
|
let notes = null;
|
|
76
78
|
for (const f of fences) {
|
|
@@ -108,6 +110,9 @@ export function scanWidgets(body, offset) {
|
|
|
108
110
|
if (!r.ok)
|
|
109
111
|
return r;
|
|
110
112
|
}
|
|
113
|
+
const voiced = scanVoiceNotes(tree, offset);
|
|
114
|
+
if (!voiced.ok)
|
|
115
|
+
return voiced;
|
|
111
116
|
const lists = scanChecklists(fences, offset);
|
|
112
117
|
if (!lists.ok)
|
|
113
118
|
return lists;
|
|
@@ -287,7 +292,7 @@ export function blocksOf(markdown) {
|
|
|
287
292
|
content = plain(n);
|
|
288
293
|
}
|
|
289
294
|
else if (n.type === 'code') {
|
|
290
|
-
if (n.lang === 'links' || n.lang === NOTES_SLOT || n.lang === 'checklist')
|
|
295
|
+
if (n.lang === 'links' || n.lang === NOTES_SLOT || n.lang === 'checklist' || n.lang === TRANSCRIPT_LANG)
|
|
291
296
|
continue;
|
|
292
297
|
kind = 'code';
|
|
293
298
|
content = n.value;
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Turns a recording into the ```transcript fence a voice note needs, so nobody times words by
|
|
3
|
+
// hand. The package carries no transcriber: this runs one already on the machine, locally,
|
|
4
|
+
// `mlx_whisper` (Apple Silicon: `pip install mlx-whisper`) or else openai-whisper's `whisper`,
|
|
5
|
+
// with word timestamps on, and prints the fence to paste under the audio line.
|
|
6
|
+
//
|
|
7
|
+
// npx -p @supersuit/artifacts supersuit-transcript-fence note.m4a [--language en] [--model M] [--gap 1.5] [--body]
|
|
8
|
+
//
|
|
9
|
+
// Correct any word afterwards (a name, a phrase in another language); keep its two times.
|
|
10
|
+
import { spawnSync } from 'node:child_process';
|
|
11
|
+
import { mkdtempSync, readFileSync, readdirSync, rmSync, existsSync } from 'node:fs';
|
|
12
|
+
import { tmpdir } from 'node:os';
|
|
13
|
+
import { join, resolve } from 'node:path';
|
|
14
|
+
import { transcriptFenceFromWords, wordsFromWhisperJson } from '../artifacts/voice-note.js';
|
|
15
|
+
const USAGE = 'usage: supersuit-transcript-fence <audio file> [--language en] [--model <whisper model>] [--gap <seconds>] [--body]';
|
|
16
|
+
function arg(args, name) {
|
|
17
|
+
const i = args.indexOf(name);
|
|
18
|
+
return i >= 0 ? args[i + 1] : undefined;
|
|
19
|
+
}
|
|
20
|
+
function has(cmd) {
|
|
21
|
+
return spawnSync('sh', ['-c', `command -v ${cmd}`], { stdio: 'ignore' }).status === 0;
|
|
22
|
+
}
|
|
23
|
+
function main(argv) {
|
|
24
|
+
const args = argv.slice(2);
|
|
25
|
+
const file = args.find((a, i) => !a.startsWith('--') && !['--language', '--model', '--gap'].includes(args[i - 1]));
|
|
26
|
+
if (!file || args.includes('--help')) {
|
|
27
|
+
process.stderr.write(USAGE + '\n');
|
|
28
|
+
return file ? 0 : 2;
|
|
29
|
+
}
|
|
30
|
+
const path = resolve(file);
|
|
31
|
+
if (!existsSync(path)) {
|
|
32
|
+
process.stderr.write(`no such file: ${path}\n`);
|
|
33
|
+
return 2;
|
|
34
|
+
}
|
|
35
|
+
const language = arg(args, '--language');
|
|
36
|
+
const gap = Number(arg(args, '--gap') ?? 1.5);
|
|
37
|
+
const out = mkdtempSync(join(tmpdir(), 'transcript-fence-'));
|
|
38
|
+
try {
|
|
39
|
+
let r;
|
|
40
|
+
if (has('mlx_whisper')) {
|
|
41
|
+
const model = arg(args, '--model') ?? 'mlx-community/whisper-large-v3-turbo';
|
|
42
|
+
r = spawnSync('mlx_whisper', [path, '--word-timestamps', 'True', '-f', 'json', '-o', out, '--model', model, ...(language ? ['--language', language] : [])], { stdio: ['ignore', 'ignore', 'inherit'] });
|
|
43
|
+
}
|
|
44
|
+
else if (has('whisper')) {
|
|
45
|
+
const model = arg(args, '--model') ?? 'turbo';
|
|
46
|
+
r = spawnSync('whisper', [path, '--word_timestamps', 'True', '--output_format', 'json', '--output_dir', out, '--model', model, ...(language ? ['--language', language] : [])], { stdio: ['ignore', 'ignore', 'inherit'] });
|
|
47
|
+
}
|
|
48
|
+
else {
|
|
49
|
+
process.stderr.write('no local transcriber found: install mlx-whisper (pip install mlx-whisper, Apple Silicon) or openai-whisper (pip install openai-whisper)\n');
|
|
50
|
+
return 3;
|
|
51
|
+
}
|
|
52
|
+
if (r.status !== 0) {
|
|
53
|
+
process.stderr.write(`the transcriber exited ${r.status}\n`);
|
|
54
|
+
return 1;
|
|
55
|
+
}
|
|
56
|
+
const json = readdirSync(out).find((f) => f.endsWith('.json'));
|
|
57
|
+
if (!json) {
|
|
58
|
+
process.stderr.write('the transcriber wrote no JSON\n');
|
|
59
|
+
return 1;
|
|
60
|
+
}
|
|
61
|
+
const body = transcriptFenceFromWords(wordsFromWhisperJson(JSON.parse(readFileSync(join(out, json), 'utf8'))), { paragraphGap: gap });
|
|
62
|
+
if (!body) {
|
|
63
|
+
process.stderr.write('the transcriber heard no words\n');
|
|
64
|
+
return 1;
|
|
65
|
+
}
|
|
66
|
+
process.stdout.write(args.includes('--body') ? body : '```transcript\n' + body + '```\n');
|
|
67
|
+
return 0;
|
|
68
|
+
}
|
|
69
|
+
finally {
|
|
70
|
+
rmSync(out, { recursive: true, force: true });
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
process.exitCode = main(process.argv);
|
|
@@ -1,8 +1,5 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
s: number;
|
|
4
|
-
e: number;
|
|
5
|
-
};
|
|
1
|
+
import { litWordIndex, type WordTiming } from './karaoke.js';
|
|
2
|
+
export { litWordIndex, type WordTiming };
|
|
6
3
|
/** The elements that hold a run of text; a word whose block differs from the previous word's
|
|
7
4
|
* starts a new sentence in the browser read-aloud. */
|
|
8
5
|
export declare const BLOCK = "p, li, td, th, h1, h2, h3, h4, h5, h6, blockquote, aside, dt, dd, figcaption, div";
|
|
@@ -12,9 +9,6 @@ export declare function wrapWords(root: HTMLElement): HTMLSpanElement[];
|
|
|
12
9
|
/** The karaoke word a click lands on, or null. A defined term is a control that opens its
|
|
13
10
|
* definition, so a tap on it must never seek or start the narration. */
|
|
14
11
|
export declare function seekableWord(target: Element | null): HTMLSpanElement | null;
|
|
15
|
-
/** The word to light at `now`, or -1. Nothing lights until narration has started: the first
|
|
16
|
-
* word begins at 0, so an untouched page would otherwise sit with its first word highlighted. */
|
|
17
|
-
export declare function litWordIndex(words: WordTiming[], now: number, started: boolean): number;
|
|
18
12
|
/** Whether a key press is the reader scrolling the page themselves. Space scrolls only outside a
|
|
19
13
|
* field that takes text. */
|
|
20
14
|
export declare function isUserScrollKey(key: string, target: EventTarget | null): boolean;
|
|
@@ -10,6 +10,8 @@ import { jsx as _jsx, jsxs as _jsxs, Fragment as _Fragment } from "react/jsx-run
|
|
|
10
10
|
// where they are and shows a button that brings them back to the voice.
|
|
11
11
|
import { useEffect, useLayoutEffect, useRef, useState } from 'react';
|
|
12
12
|
import { nextRate, PlayerBar } from './player-bar.js';
|
|
13
|
+
import { litWordIndex } from './karaoke.js';
|
|
14
|
+
export { litWordIndex };
|
|
13
15
|
const SKIP = '[data-nospeak], [data-artifact-links], pre, [data-footnotes], sup, img, style, script';
|
|
14
16
|
/** The elements that hold a run of text; a word whose block differs from the previous word's
|
|
15
17
|
* starts a new sentence in the browser read-aloud. */
|
|
@@ -109,25 +111,6 @@ function align(spans, words) {
|
|
|
109
111
|
}
|
|
110
112
|
return out;
|
|
111
113
|
}
|
|
112
|
-
/** The word to light at `now`, or -1. Nothing lights until narration has started: the first
|
|
113
|
-
* word begins at 0, so an untouched page would otherwise sit with its first word highlighted. */
|
|
114
|
-
export function litWordIndex(words, now, started) {
|
|
115
|
-
if (!started)
|
|
116
|
-
return -1;
|
|
117
|
-
let lo = 0;
|
|
118
|
-
let hi = words.length - 1;
|
|
119
|
-
let idx = -1;
|
|
120
|
-
while (lo <= hi) {
|
|
121
|
-
const mid = (lo + hi) >> 1;
|
|
122
|
-
if (words[mid].s <= now) {
|
|
123
|
-
idx = mid;
|
|
124
|
-
lo = mid + 1;
|
|
125
|
-
}
|
|
126
|
-
else
|
|
127
|
-
hi = mid - 1;
|
|
128
|
-
}
|
|
129
|
-
return idx >= 0 && now <= words[idx].e + 0.25 ? idx : -1;
|
|
130
|
-
}
|
|
131
114
|
function fmt(t) {
|
|
132
115
|
const m = Math.floor(t / 60);
|
|
133
116
|
const s = Math.floor(t % 60);
|
package/lib/reader/index.d.ts
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
export { ArtifactReader, wrapWords, seekableWord, litWordIndex, type WordTiming } from './artifact-reader.js';
|
|
2
|
+
export { begunCount } from './karaoke.js';
|
|
2
3
|
export { PlayerBar } from './player-bar.js';
|
|
3
4
|
export { QuietView } from './quiet-view.js';
|
|
4
5
|
export { CommentLayer } from './comment-layer.js';
|
package/lib/reader/index.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
// The `./reader` entry: the reader and the bar it uses. Each component module carries its
|
|
2
2
|
// own 'use client'; this barrel is plain re-exports so it can be imported from either side.
|
|
3
3
|
export { ArtifactReader, wrapWords, seekableWord, litWordIndex } from './artifact-reader.js';
|
|
4
|
+
export { begunCount } from './karaoke.js';
|
|
4
5
|
export { PlayerBar } from './player-bar.js';
|
|
5
6
|
export { QuietView } from './quiet-view.js';
|
|
6
7
|
export { CommentLayer } from './comment-layer.js';
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
export type WordTiming = {
|
|
2
|
+
w: string;
|
|
3
|
+
s: number;
|
|
4
|
+
e: number;
|
|
5
|
+
};
|
|
6
|
+
/** The word to light at `now`, or -1. Nothing lights until narration has started: the first
|
|
7
|
+
* word begins at 0, so an untouched page would otherwise sit with its first word highlighted. */
|
|
8
|
+
export declare function litWordIndex(words: WordTiming[], now: number, started: boolean): number;
|
|
9
|
+
/** How many words have begun by `now`: the rest are still to come. A pause between words keeps
|
|
10
|
+
* the count, so the words already said never fall back to muted in a silence. */
|
|
11
|
+
export declare function begunCount(words: WordTiming[], now: number): number;
|
|
12
|
+
/** The highlight on the word being spoken, in the page's accent. */
|
|
13
|
+
export declare const litDeclarations: (accent: string) => string;
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
// The karaoke primitives the page's narration and a voice note share: which word is being spoken
|
|
2
|
+
// at a time, how many words have begun, and the one highlight both draw. Pure and server-safe.
|
|
3
|
+
/** The last word that has begun by `now`, or -1. Words are sorted by start. */
|
|
4
|
+
function lastBegun(words, now) {
|
|
5
|
+
let lo = 0;
|
|
6
|
+
let hi = words.length - 1;
|
|
7
|
+
let idx = -1;
|
|
8
|
+
while (lo <= hi) {
|
|
9
|
+
const mid = (lo + hi) >> 1;
|
|
10
|
+
if (words[mid].s <= now) {
|
|
11
|
+
idx = mid;
|
|
12
|
+
lo = mid + 1;
|
|
13
|
+
}
|
|
14
|
+
else
|
|
15
|
+
hi = mid - 1;
|
|
16
|
+
}
|
|
17
|
+
return idx;
|
|
18
|
+
}
|
|
19
|
+
/** The word to light at `now`, or -1. Nothing lights until narration has started: the first
|
|
20
|
+
* word begins at 0, so an untouched page would otherwise sit with its first word highlighted. */
|
|
21
|
+
export function litWordIndex(words, now, started) {
|
|
22
|
+
if (!started)
|
|
23
|
+
return -1;
|
|
24
|
+
const idx = lastBegun(words, now);
|
|
25
|
+
return idx >= 0 && now <= words[idx].e + 0.25 ? idx : -1;
|
|
26
|
+
}
|
|
27
|
+
/** How many words have begun by `now`: the rest are still to come. A pause between words keeps
|
|
28
|
+
* the count, so the words already said never fall back to muted in a silence. */
|
|
29
|
+
export function begunCount(words, now) {
|
|
30
|
+
return lastBegun(words, now) + 1;
|
|
31
|
+
}
|
|
32
|
+
/** The highlight on the word being spoken, in the page's accent. */
|
|
33
|
+
export const litDeclarations = (accent) => `background: color-mix(in srgb, ${accent} 35%, transparent); color: var(--a-strong, #fff);`;
|
package/lib/reader/player-bar.js
CHANGED
|
@@ -8,11 +8,12 @@ import { jsx as _jsx, jsxs as _jsxs, Fragment as _Fragment } from "react/jsx-run
|
|
|
8
8
|
// button, the seek line, the progress and the speed. The speed button has a FIXED width wide enough
|
|
9
9
|
// for its longest label ("1.25x"), so stepping 1x -> 1.25x -> 2x never resizes it and never shifts
|
|
10
10
|
// the seek line beside it (Gary: "the bottom bar speed dial shouldn't cause reflow when clicked").
|
|
11
|
+
import { litDeclarations } from './karaoke.js';
|
|
11
12
|
export function PlayerBar({ playing, onToggle, accent, ground, value, max, step, onSeek, progress, rate, onRate, }) {
|
|
12
13
|
return (_jsxs(_Fragment, { children: [_jsx("style", { children: `
|
|
13
14
|
.artifact-word { border-radius: 3px; transition: background-color 120ms; }
|
|
14
15
|
.artifact-word:hover { background: var(--a-surface-strong, rgba(255,255,255,0.08)); cursor: pointer; }
|
|
15
|
-
.artifact-word-lit {
|
|
16
|
+
.artifact-word-lit { ${litDeclarations(accent)} }
|
|
16
17
|
` }), _jsx("div", { "data-artifact-player": true, className: "fixed inset-x-0 bottom-0 z-40 border-t border-[color:var(--a-line)] backdrop-blur", style: { background: `color-mix(in srgb, ${ground} 90%, transparent)` }, children: _jsxs("div", { className: "mx-auto flex max-w-2xl items-center gap-4 px-6 py-3", children: [_jsx("button", { type: "button", onClick: onToggle, "aria-label": playing ? 'Pause narration' : 'Play narration', className: "flex h-11 w-11 shrink-0 items-center justify-center rounded-full", style: { background: accent, color: ground }, children: playing ? (_jsxs("svg", { width: "18", height: "18", viewBox: "0 0 24 24", fill: "currentColor", children: [_jsx("rect", { x: "6", y: "5", width: "4", height: "14" }), _jsx("rect", { x: "14", y: "5", width: "4", height: "14" })] })) : (_jsx("svg", { width: "18", height: "18", viewBox: "0 0 24 24", fill: "currentColor", children: _jsx("path", { d: "M8 5v14l11-7z" }) })) }), _jsx("div", { className: "min-w-0 flex-1", children: _jsx("input", { type: "range", min: 0, max: max || 0, step: step, value: Math.min(value, max || 0), onChange: (e) => onSeek(Number(e.target.value)), "aria-label": "Seek", className: "block w-full", style: { accentColor: accent } }) }), _jsx("div", { className: "w-16 shrink-0 text-right text-xs tabular-nums text-[color:var(--a-muted)]", children: progress }), _jsxs("button", { type: "button", onClick: onRate, "data-artifact-rate": true, className: "w-14 shrink-0 whitespace-nowrap rounded border border-[color:var(--a-line-strong)] py-1 text-center text-xs tabular-nums text-[color:var(--a-body)]", "aria-label": `Playback speed, ${rate}x`, children: [rate, "x"] })] }) }), _jsx("div", { className: "h-20", "aria-hidden": true })] }));
|
|
17
18
|
}
|
|
18
19
|
/** The speeds the speed button steps through, in order, wrapping back to the first. */
|
|
@@ -136,7 +136,7 @@ export function createRecipientsRoutes(given) {
|
|
|
136
136
|
// A page nobody has been added to yet has no readers, which is an answer and not a missing page.
|
|
137
137
|
const rec = (await ctx.recipients.get(id)) ?? { recipients: {}, expireAt: null, keep: false };
|
|
138
138
|
const entries = ctx.state ? await ctx.state.entries(id) : [];
|
|
139
|
-
const comments = entries.filter((e) => e.slot === COMMENTS_SLOT && e.readerKey.startsWith('r:')).map((e) => ({ readerKey: e.readerKey, at: e.at, text: String(e.value?.body ?? '') }));
|
|
139
|
+
const comments = entries.filter((e) => e.slot === COMMENTS_SLOT && e.readerKey.startsWith('r:')).map((e) => ({ readerKey: e.readerKey, id: e.id, parent: typeof e.value?.parent === 'string' ? e.value.parent : null, at: e.at, text: String(e.value?.body ?? '') }));
|
|
140
140
|
return json({ id, expireAt: rec.expireAt, keep: rec.keep, recipients: summarizeViews(rec, await ctx.recipients.views(id), comments) });
|
|
141
141
|
}
|
|
142
142
|
/** Reader text goes with the page: every comment and answer on it, whoever wrote it. */
|
|
@@ -5,7 +5,8 @@
|
|
|
5
5
|
// GET /api/artifacts/<id>/responses publish key; every answer with who (?format=csv)
|
|
6
6
|
// DELETE /api/artifacts/<id>/responses?reader=<key> publish key; one reader's answers
|
|
7
7
|
// GET /api/comments?since=<ISO> publish key; every shared comment on any page after a time
|
|
8
|
-
// (&open=1: leave out threads that are resolved
|
|
8
|
+
// (&open=1: leave out threads that are resolved;
|
|
9
|
+
// &artifact=<id>: only that page's comments)
|
|
9
10
|
// POST /api/artifacts/<id>/resolve publish key, or the owner or an admin signed in; resolve or
|
|
10
11
|
// reopen one comment thread { entry, resolved?, note?, by? }
|
|
11
12
|
//
|
|
@@ -368,6 +369,16 @@ export function createStateRoutes(ctx) {
|
|
|
368
369
|
if (!replacing)
|
|
369
370
|
return json({ error: 'this page is not taking more answers here' }, 409);
|
|
370
371
|
}
|
|
372
|
+
// A group link under `comment_name: required` makes each browser its own reader, so one link
|
|
373
|
+
// could start any number of them. The FIRST STORED write of a reader key the page has not seen
|
|
374
|
+
// is counted against the link, whether it came with a cookie or not (any 24 characters pass as
|
|
375
|
+
// one), and only here, after validation, so a refused request spends nothing. A key that
|
|
376
|
+
// already has an entry is never counted again.
|
|
377
|
+
if (recipient?.perBrowser && !(await ctx.state.entries(id)).some((e) => e.readerKey === writer.key)) {
|
|
378
|
+
const n = await ctx.state.countAnonWrite(id, `gl.${recipient.rid}`, Math.floor(Date.now() / 60000));
|
|
379
|
+
if (n > ANON_WRITES_PER_MINUTE)
|
|
380
|
+
return json({ error: 'too many new readers through this link; try again in a minute' }, 429);
|
|
381
|
+
}
|
|
371
382
|
const r = op === 'set'
|
|
372
383
|
? await ctx.state.set({ artifactId: id, slot, writer, value })
|
|
373
384
|
: await ctx.state.append({ artifactId: id, slot, writer, value });
|
|
@@ -424,10 +435,17 @@ export function createStateRoutes(ctx) {
|
|
|
424
435
|
const openOnly = openParam === '1' || openParam === 'true';
|
|
425
436
|
if (!ctx.state)
|
|
426
437
|
return json({ error: 'this host keeps no answers' }, 501);
|
|
427
|
-
|
|
438
|
+
// One page's feed (0.43.0): read that page's entries alone, so a caller paging a single page
|
|
439
|
+
// never reads, or is handed, another page's comments.
|
|
440
|
+
const only = req.nextUrl.searchParams.get('artifact');
|
|
441
|
+
if (only !== null && !ARTIFACT_ID_RE.test(only))
|
|
442
|
+
return json({ error: 'artifact must be a page id, like abcd2345' }, 400);
|
|
443
|
+
if (only === null && !ctx.state.slotSince)
|
|
428
444
|
return json({ error: 'this host cannot list comments by time' }, 501);
|
|
429
445
|
const now = new Date().toISOString();
|
|
430
|
-
const entries =
|
|
446
|
+
const entries = only !== null
|
|
447
|
+
? (await ctx.state.entries(only)).filter((e) => e.slot === COMMENTS_SLOT && e.at > since).sort((x, y) => x.at.localeCompare(y.at) || x.id.localeCompare(y.id)).slice(0, FEED_MAX)
|
|
448
|
+
: await ctx.state.slotSince(COMMENTS_SLOT, since, FEED_MAX);
|
|
431
449
|
const pages = new Map();
|
|
432
450
|
const pageOf = (id) => { if (!pages.has(id))
|
|
433
451
|
pages.set(id, ctx.store.get(id)); return pages.get(id); };
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
import { type WordTiming } from '../reader/karaoke.js';
|
|
2
|
+
export declare function VoiceNote({ src, label, media, paragraphs, block }: {
|
|
3
|
+
src: string;
|
|
4
|
+
label: string;
|
|
5
|
+
/** The stored file's name, for play tracking. */
|
|
6
|
+
media: string;
|
|
7
|
+
paragraphs: WordTiming[][];
|
|
8
|
+
/** The comment anchor the audio line carried. */
|
|
9
|
+
block?: Record<string, string>;
|
|
10
|
+
}): import("react").JSX.Element;
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
'use client';
|
|
2
|
+
import { jsx as _jsx, jsxs as _jsxs } from "react/jsx-runtime";
|
|
3
|
+
// A voice note in the reader's browser: a recording tucked into the page like a note in a letter.
|
|
4
|
+
// The caption, a play button and a seek line drawn in the page's brand (never the native bar), and
|
|
5
|
+
// the speaker's own words beneath, lit one at a time as they are spoken: words already said read
|
|
6
|
+
// normally, the word being said carries the same highlight as the page's narration (karaoke.ts),
|
|
7
|
+
// and the words to come are a little muted. Tapping a word plays from there.
|
|
8
|
+
//
|
|
9
|
+
// The whole note is data-nospeak, so the page's own narration skips it: a reader hears the person,
|
|
10
|
+
// never a robot reading the person. The words are server-rendered, so the transcript reads as text
|
|
11
|
+
// before anything loads; only the lighting needs the browser.
|
|
12
|
+
//
|
|
13
|
+
// iPhone Safari: no autoplay, `preload="none"` (nothing fetched until play), `playsinline`. A seek
|
|
14
|
+
// asked for before the clip has loaded waits for its metadata, since Safari drops it otherwise.
|
|
15
|
+
// `data-artifact-media` stays on the <audio>, so play tracking counts it like any clip.
|
|
16
|
+
import { useEffect, useMemo, useRef, useState } from 'react';
|
|
17
|
+
import { begunCount, litDeclarations, litWordIndex } from '../reader/karaoke.js';
|
|
18
|
+
const CSS = '[data-artifact-voicenote] .vn-word{border-radius:3px;cursor:pointer;transition:background-color 120ms,opacity 200ms;-webkit-tap-highlight-color:transparent}' +
|
|
19
|
+
'[data-artifact-voicenote] .vn-word:hover{background:var(--a-surface-strong,rgba(127,127,127,0.12))}' +
|
|
20
|
+
'[data-artifact-voicenote] .vn-word.vn-ahead{opacity:0.5}' +
|
|
21
|
+
`[data-artifact-voicenote] .vn-word.vn-word-lit{${litDeclarations('var(--a-accent)')}opacity:1}` +
|
|
22
|
+
'[data-artifact-voicenote] input[type=range]{accent-color:var(--a-accent)}' +
|
|
23
|
+
'[data-artifact-voicenote] button:focus-visible,[data-artifact-voicenote] input:focus-visible{outline:2px solid var(--a-accent);outline-offset:2px}';
|
|
24
|
+
function fmt(t) {
|
|
25
|
+
const s = Math.max(0, Math.floor(t));
|
|
26
|
+
return `${Math.floor(s / 60)}:${String(s % 60).padStart(2, '0')}`;
|
|
27
|
+
}
|
|
28
|
+
export function VoiceNote({ src, label, media, paragraphs, block }) {
|
|
29
|
+
const audioRef = useRef(null);
|
|
30
|
+
const rootRef = useRef(null);
|
|
31
|
+
const words = useMemo(() => paragraphs.flat(), [paragraphs]);
|
|
32
|
+
const spans = useRef([]);
|
|
33
|
+
const shown = useRef({ lit: -1, begun: -1 });
|
|
34
|
+
const [playing, setPlaying] = useState(false);
|
|
35
|
+
const [started, setStarted] = useState(false);
|
|
36
|
+
const startedRef = useRef(false);
|
|
37
|
+
const [t, setT] = useState(0);
|
|
38
|
+
const [duration, setDuration] = useState(0);
|
|
39
|
+
const total = duration || words[words.length - 1]?.e || 0;
|
|
40
|
+
/** Light the word at the audio's current time; touch only the spans whose state changed. */
|
|
41
|
+
const sync = () => {
|
|
42
|
+
const a = audioRef.current;
|
|
43
|
+
if (!a)
|
|
44
|
+
return;
|
|
45
|
+
const now = a.currentTime;
|
|
46
|
+
setT(now);
|
|
47
|
+
if (!startedRef.current)
|
|
48
|
+
return;
|
|
49
|
+
const lit = litWordIndex(words, now, true);
|
|
50
|
+
const begun = a.ended ? words.length : begunCount(words, now);
|
|
51
|
+
const was = shown.current;
|
|
52
|
+
if (lit === was.lit && begun === was.begun)
|
|
53
|
+
return;
|
|
54
|
+
if (was.lit !== lit) {
|
|
55
|
+
spans.current[was.lit]?.classList.remove('vn-word-lit');
|
|
56
|
+
spans.current[lit]?.classList.add('vn-word-lit');
|
|
57
|
+
}
|
|
58
|
+
const from = Math.max(0, Math.min(begun, was.begun < 0 ? 0 : was.begun));
|
|
59
|
+
const to = was.begun < 0 ? spans.current.length : Math.max(begun, was.begun);
|
|
60
|
+
for (let i = from; i < to; i++)
|
|
61
|
+
spans.current[i]?.classList.toggle('vn-ahead', i >= begun);
|
|
62
|
+
shown.current = { lit, begun };
|
|
63
|
+
};
|
|
64
|
+
const start = () => {
|
|
65
|
+
if (startedRef.current)
|
|
66
|
+
return;
|
|
67
|
+
startedRef.current = true;
|
|
68
|
+
setStarted(true);
|
|
69
|
+
};
|
|
70
|
+
// While it plays, follow the voice every frame; Safari's timeupdate alone is four times a second.
|
|
71
|
+
useEffect(() => {
|
|
72
|
+
if (!playing)
|
|
73
|
+
return;
|
|
74
|
+
let raf = 0;
|
|
75
|
+
const tick = () => {
|
|
76
|
+
sync();
|
|
77
|
+
raf = requestAnimationFrame(tick);
|
|
78
|
+
};
|
|
79
|
+
raf = requestAnimationFrame(tick);
|
|
80
|
+
return () => cancelAnimationFrame(raf);
|
|
81
|
+
});
|
|
82
|
+
const play = () => {
|
|
83
|
+
const a = audioRef.current;
|
|
84
|
+
if (!a)
|
|
85
|
+
return;
|
|
86
|
+
start();
|
|
87
|
+
// One voice at a time: the narration or another note stops when this one starts.
|
|
88
|
+
for (const other of Array.from(document.querySelectorAll('audio')))
|
|
89
|
+
if (other !== a && !other.paused)
|
|
90
|
+
other.pause();
|
|
91
|
+
void a.play()?.catch?.(() => setPlaying(false));
|
|
92
|
+
};
|
|
93
|
+
const toggle = () => {
|
|
94
|
+
const a = audioRef.current;
|
|
95
|
+
if (!a)
|
|
96
|
+
return;
|
|
97
|
+
if (a.paused)
|
|
98
|
+
play();
|
|
99
|
+
else
|
|
100
|
+
a.pause();
|
|
101
|
+
};
|
|
102
|
+
const seekTo = (s) => {
|
|
103
|
+
const a = audioRef.current;
|
|
104
|
+
if (!a)
|
|
105
|
+
return;
|
|
106
|
+
start();
|
|
107
|
+
const go = () => {
|
|
108
|
+
a.currentTime = s;
|
|
109
|
+
sync();
|
|
110
|
+
};
|
|
111
|
+
if (a.readyState >= 1)
|
|
112
|
+
go();
|
|
113
|
+
else
|
|
114
|
+
a.addEventListener('loadedmetadata', go, { once: true });
|
|
115
|
+
};
|
|
116
|
+
let k = 0;
|
|
117
|
+
spans.current = [];
|
|
118
|
+
return (_jsxs("div", { ...block, ref: rootRef, "data-nospeak": true, "data-artifact-voicenote": true, ...(started ? { 'data-started': '' } : {}), className: "my-8 rounded-xl border border-[color:var(--a-line)] bg-[color:var(--a-surface)] px-5 pb-5 pt-4 shadow-sm", children: [_jsx("style", { children: CSS }), _jsx("audio", { ref: audioRef, src: src, preload: "none", playsInline: true, "data-artifact-media": media, onPlay: () => { start(); setPlaying(true); }, onPause: () => { setPlaying(false); sync(); }, onEnded: () => { setPlaying(false); sync(); }, onTimeUpdate: sync, onSeeked: sync, onLoadedMetadata: (e) => { const d = e.currentTarget.duration; if (Number.isFinite(d))
|
|
119
|
+
setDuration(d); }, onDurationChange: (e) => { const d = e.currentTarget.duration; if (Number.isFinite(d))
|
|
120
|
+
setDuration(d); } }), _jsx("span", { className: "block text-[11px] uppercase tracking-[0.2em] text-[color:var(--a-accent)]", children: "Voice note" }), label ? _jsx("span", { className: "mt-1 block font-serif text-lg italic text-[color:var(--a-strong)]", children: label }) : null, _jsxs("div", { className: "mt-3 flex items-center gap-3", children: [_jsx("button", { type: "button", onClick: toggle, "aria-label": `${playing ? 'Pause' : 'Play'} voice note${label ? `: ${label}` : ''}`, className: "flex h-11 w-11 shrink-0 items-center justify-center rounded-full bg-[color:var(--a-accent)] text-[color:var(--a-on-accent)]", children: playing ? (_jsxs("svg", { width: "16", height: "16", viewBox: "0 0 24 24", fill: "currentColor", "aria-hidden": "true", children: [_jsx("rect", { x: "6", y: "5", width: "4", height: "14" }), _jsx("rect", { x: "14", y: "5", width: "4", height: "14" })] })) : (_jsx("svg", { width: "16", height: "16", viewBox: "0 0 24 24", fill: "currentColor", "aria-hidden": "true", children: _jsx("path", { d: "M8 5v14l11-7z" }) })) }), _jsx("input", { type: "range", "aria-label": "Seek", min: 0, max: total || 1, step: 0.01, value: Math.min(t, total || 1), onChange: (e) => seekTo(Number(e.currentTarget.value)), className: "min-w-0 flex-1" }), _jsxs("span", { className: "shrink-0 font-mono text-xs tabular-nums text-[color:var(--a-muted)]", children: [fmt(t), " / ", fmt(total)] })] }), _jsx("div", { className: "mt-4 leading-relaxed text-[color:var(--a-body)]", children: paragraphs.map((p, pi) => (_jsx("p", { "data-vn-paragraph": true, className: "my-2", children: p.map((w, wi) => {
|
|
121
|
+
const i = k++;
|
|
122
|
+
return (_jsxs("span", { children: [wi ? ' ' : null, _jsx("span", { ref: (el) => { if (el)
|
|
123
|
+
spans.current[i] = el; }, className: "vn-word", onClick: () => { seekTo(w.s); play(); }, children: w.w })] }, wi));
|
|
124
|
+
}) }, pi))) })] }));
|
|
125
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@supersuit/artifacts",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.44.0",
|
|
4
4
|
"description": "Publish markdown as branded, read-aloud pages from a Next.js app: the artifacts store, the route factory, the read-along reader, brand packs, password and confidential pages, and share cards.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -20,6 +20,9 @@
|
|
|
20
20
|
],
|
|
21
21
|
"main": "lib/index.js",
|
|
22
22
|
"types": "lib/index.d.ts",
|
|
23
|
+
"bin": {
|
|
24
|
+
"supersuit-transcript-fence": "lib/cli/transcript-fence.js"
|
|
25
|
+
},
|
|
23
26
|
"exports": {
|
|
24
27
|
".": {
|
|
25
28
|
"types": "./lib/index.d.ts",
|