@ericrisco/rsc 3.3.1 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. package/README.md +3 -3
  2. package/manifest.json +43 -22
  3. package/package.json +1 -1
  4. package/scripts/lib/default-skill-floor.js +5 -1
  5. package/scripts/lib/domains.js +2 -2
  6. package/scripts/lib/retired-skills.js +4 -0
  7. package/scripts/rsc.js +1 -1
  8. package/skills/design/SKILL.md +4 -3
  9. package/skills/design/evals/cases.yaml +2 -2
  10. package/skills/ftd/SKILL.md +1 -1
  11. package/skills/react/SKILL.md +2 -1
  12. package/skills/ux/SKILL.md +149 -0
  13. package/skills/ux/THIRD_PARTY.md +32 -0
  14. package/skills/ux/evals/cases.yaml +40 -0
  15. package/skills/ux/references/flows.md +124 -0
  16. package/skills/ux/upstream/UPSTREAM.json +59 -0
  17. package/skills/ux/upstream/ux-designer/GUIDE.md +180 -0
  18. package/skills/ux/upstream/ux-designer/LICENSE +21 -0
  19. package/skills/ux/upstream/ux-designer/references/01-core-principles.md +251 -0
  20. package/skills/ux/upstream/ux-designer/references/02-laws-of-ux.md +330 -0
  21. package/skills/ux/upstream/ux-designer/references/03-accessibility.md +411 -0
  22. package/skills/ux/upstream/ux-designer/references/04-visual-design.md +433 -0
  23. package/skills/ux/upstream/ux-designer/references/05-information-architecture.md +432 -0
  24. package/skills/ux/upstream/ux-designer/references/06-interaction-design.md +495 -0
  25. package/skills/ux/upstream/ux-designer/references/07-forms-and-inputs.md +491 -0
  26. package/skills/ux/upstream/ux-designer/references/08-mobile-ux.md +539 -0
  27. package/skills/ux/upstream/ux-designer/references/09-ux-writing.md +493 -0
  28. package/skills/ux/upstream/ux-designer/references/10-user-research.md +533 -0
  29. package/skills/ux/upstream/ux-designer/references/11-design-systems.md +618 -0
  30. package/skills/ux/upstream/ux-designer/references/12a-presence-awareness.md +395 -0
  31. package/skills/ux/upstream/ux-designer/references/12b-conflict-resolution-sync.md +420 -0
  32. package/skills/ux/upstream/ux-designer/references/13a-canvas-navigation.md +388 -0
  33. package/skills/ux/upstream/ux-designer/references/13b-canvas-objects-performance.md +559 -0
  34. package/skills/ux/upstream/ux-designer/references/14-ai-ux-patterns.md +511 -0
  35. package/skills/ux/upstream/ux-designer/references/15-ethical-design.md +419 -0
  36. package/skills/ux/upstream/ux-designer/references/16-onboarding.md +430 -0
  37. package/skills/ux/upstream/ux-designer/references/17-notifications.md +461 -0
  38. package/skills/ux/upstream/ux-designer/references/18-data-visualization.md +453 -0
  39. package/skills/ux/upstream/ux-designer/references/19-search-ux.md +479 -0
  40. package/skills/ux/upstream/ux-designer/references/20-emotional-design.md +382 -0
  41. package/skills/ux/upstream/ux-designer/references/21-data-tables.md +629 -0
  42. package/skills/ux/upstream/ux-designer/references/22-performance-ux.md +626 -0
  43. package/skills/ux/upstream/ux-designer/references/23-internationalization.md +347 -0
  44. package/skills/ux/upstream/ux-designer/references/24-voice-and-multimodal.md +308 -0
  45. package/skills/ux/upstream/ux-designer/references/25a-barcode-camera-scanning.md +342 -0
  46. package/skills/ux/upstream/ux-designer/references/25b-barcode-scanner-workflows.md +412 -0
  47. package/skills/ux/upstream/ux-designer/references/25c-barcode-display.md +277 -0
  48. package/skills/ux/upstream/ux-usability-foundations/GUIDE.md +285 -0
  49. package/skills/ux/upstream/ux-usability-foundations/LICENSE +21 -0
  50. package/skills/ux/upstream/ux-usability-foundations/references/anti-patterns.md +267 -0
  51. package/skills/ux/upstream/ux-usability-foundations/references/checklists.md +130 -0
  52. package/skills/ux/upstream/ux-usability-foundations/references/decision-prompts.md +340 -0
  53. package/skills/ux/upstream/ux-usability-foundations/references/principle-cards.md +449 -0
  54. package/skills/worktrees/SKILL.md +19 -5
  55. package/targets/branch-guard.mjs +59 -4
  56. package/targets/clone-bootstrap.mjs +1 -1
  57. package/targets/session-memory-adapter.mjs +2 -2
  58. package/targets/team-safe-start.mjs +1 -0
  59. package/targets/worktree-reaper.mjs +14 -1
  60. package/skills/ui-engineering/SKILL.md +0 -113
  61. package/skills/ui-engineering/evals/cases.yaml +0 -38
@@ -0,0 +1,347 @@
1
+ # Internationalization & Localization
2
+
3
+ Internationalization (**i18n**) is designing and building a product so it *can* be
4
+ adapted to different languages, regions, and cultures without re-engineering.
5
+ Localization (**l10n**) is the adaptation itself—translating strings, swapping
6
+ formats, adjusting layout. Get i18n wrong up front and every locale becomes a
7
+ rewrite; get it right and adding a language is a content task, not a code task.
8
+
9
+ > Rule of thumb: assume your UI will run right-to-left, in a language whose words
10
+ > are 40% longer than English, with dates and numbers formatted by the user's
11
+ > locale—*before* you have a single translation.
12
+
13
+ ---
14
+
15
+ ## Contents
16
+
17
+ - RTL & Bidirectional Layout
18
+ - Text Expansion & Contraction
19
+ - Pluralization & Gender
20
+ - Locale-Aware Formatting
21
+ - Translation-Friendly UI
22
+ - Language Switching UX
23
+ - Quick Checklist
24
+ - Common Mistakes
25
+ - Sources
26
+
27
+ ---
28
+
29
+ ## RTL & Bidirectional Layout
30
+
31
+ Arabic, Hebrew, Persian, and Urdu read right-to-left (RTL). RTL is not "flip the
32
+ CSS"—it's a directional mirror of the *layout*, while some content stays unchanged.
33
+
34
+ ### Use logical CSS properties, not physical ones
35
+
36
+ Physical properties (`left`, `right`, `margin-left`) hard-code LTR assumptions.
37
+ Logical properties resolve against the document/element direction automatically.
38
+
39
+ ```css
40
+ /* ❌ Physical — breaks in RTL */
41
+ .card { margin-left: 16px; text-align: left; border-left: 2px solid; }
42
+
43
+ /* ✅ Logical — mirrors automatically with dir="rtl" */
44
+ .card {
45
+ margin-inline-start: 16px;
46
+ text-align: start;
47
+ border-inline-start: 2px solid;
48
+ padding-inline: 12px; /* start + end */
49
+ inset-inline-start: 0; /* replaces left/right for positioning */
50
+ }
51
+ ```
52
+
53
+ | Physical | Logical equivalent |
54
+ |----------|-------------------|
55
+ | `margin-left` / `margin-right` | `margin-inline-start` / `margin-inline-end` |
56
+ | `padding-left` / `padding-right` | `padding-inline-start` / `padding-inline-end` |
57
+ | `left` / `right` | `inset-inline-start` / `inset-inline-end` |
58
+ | `text-align: left` | `text-align: start` |
59
+ | `border-left` | `border-inline-start` |
60
+
61
+ ### Set direction at the root, let it cascade
62
+
63
+ ```html
64
+ <html lang="ar" dir="rtl">
65
+ ```
66
+
67
+ Use `dir="auto"` on user-generated content fields so a single input handles either
68
+ direction based on the first strong character. For mixed-direction strings (an
69
+ English brand name inside Arabic text), rely on the Unicode Bidirectional Algorithm
70
+ and isolate embedded runs with `<bdi>` or CSS `unicode-bidi: isolate`.
71
+
72
+ ### Mirror these — but not those
73
+
74
+ **Mirror** (they imply direction of reading/progress):
75
+ - Layout: navigation, sidebars, alignment, the whole grid
76
+ - Directional icons: back/forward arrows, chevrons, breadcrumb separators
77
+ - Progress bars, sliders, carousels, "next step" affordances
78
+
79
+ **Do NOT mirror** (direction is intrinsic, not reading-relative):
80
+ - Logos and brand marks
81
+ - Media playback controls (play ▶ still points "forward in time", not in reading direction)
82
+ - Clocks, musical notation, and most diagrams
83
+ - Numbers themselves (digits are written left-to-right even within RTL text)
84
+ - Checkmarks, and icons of real-world objects whose orientation is fixed
85
+
86
+ ```css
87
+ /* Mirror a directional icon only */
88
+ [dir="rtl"] .icon-chevron { transform: scaleX(-1); }
89
+ /* Logo and play button: leave alone */
90
+ ```
91
+
92
+ ### What a mirror actually looks like
93
+
94
+ ```
95
+ LTR (English) RTL (Arabic)
96
+ ┌───────────────────────────┐ ┌───────────────────────────┐
97
+ │ [Logo] Home About [☰] │ │ [☰] About Home [Logo] │
98
+ ├───────────────────────────┤ ├───────────────────────────┤
99
+ │ ‹ Back Title │ │ Title Back › │
100
+ │ │ │ │
101
+ │ • List item │ │ List item • │
102
+ │ [Avatar] Name │ │ Name [Avatar] │
103
+ │ [Next ›] │ │ [‹ Next] │
104
+ └───────────────────────────┘ └───────────────────────────┘
105
+ ▶ 0:12 ──────●──── 3:45 ▶ 0:12 ──────●──── 3:45
106
+ (media controls do NOT mirror — time still flows left→right)
107
+ ```
108
+
109
+ Note: the nav, alignment, list bullets, avatars, and the Back/Next chevrons all
110
+ flip; the logo and the media scrubber do not.
111
+
112
+ ---
113
+
114
+ ## Text Expansion & Contraction
115
+
116
+ Translated text changes length. UI laid out tightly around English will overflow,
117
+ truncate, or wrap badly. German, Finnish, and Russian commonly run **30–40% longer**
118
+ than English; short strings can expand far more.
119
+
120
+ ### IBM's rule of thumb for expansion
121
+
122
+ | English length | Expect up to |
123
+ |----------------|-------------|
124
+ | ≤ 10 chars | +100–200% |
125
+ | 11–20 chars | +80–100% |
126
+ | 21–30 chars | +60–80% |
127
+ | 31–50 chars | +40% |
128
+ | > 50 chars | +30% |
129
+
130
+ ### Design for it
131
+
132
+ - **No fixed-width text containers.** Let buttons, tabs, labels, and badges grow.
133
+ - **Avoid truncation on essential UI.** A clipped menu item is a broken menu item.
134
+ - **Test with pseudo-localization** early: replace `Settings` with
135
+ `[Ŝéttîng§ ──────]` to surface overflow and hard-coded strings before real
136
+ translation exists.
137
+ - **CJK contracts**—Chinese/Japanese/Korean are often *shorter and taller*; don't
138
+ assume single-line height either.
139
+
140
+ ### Never concatenate translated fragments
141
+
142
+ Word order differs across languages, so string-gluing produces nonsense.
143
+
144
+ ```js
145
+ // ❌ Grammatically broken in most languages
146
+ const msg = "You have " + count + " new " + itemType + " messages";
147
+
148
+ // ✅ One translatable message with named placeholders + ICU plurals
149
+ t('inbox.summary', { count, itemType });
150
+ // en: "{count, plural, one {# new message} other {# new messages}}"
151
+ // de: "{count, plural, one {# neue Nachricht} other {# neue Nachrichten}}"
152
+ ```
153
+
154
+ ---
155
+
156
+ ## Pluralization & Gender
157
+
158
+ English has two plural forms (1 / many). Many languages have more—Arabic has **six**
159
+ CLDR plural categories (zero, one, two, few, many, other); Polish and Russian have
160
+ distinct "few" vs. "many" forms. The naive `count + "s"` is wrong almost everywhere.
161
+
162
+ Use **ICU MessageFormat** (backed by CLDR plural rules), supported by `i18next`,
163
+ `react-intl`/FormatJS, `Intl.PluralRules`, and most modern libraries.
164
+
165
+ ```
166
+ {count, plural,
167
+ =0 {No items}
168
+ one {# item}
169
+ few {# items}
170
+ many {# items}
171
+ other {# items}
172
+ }
173
+ ```
174
+
175
+ > English resolves only to `one`/`other`; `few`/`many` (and `=0`) are listed here
176
+ > for the languages that need them (Polish, Russian, Arabic…). A locale picks the
177
+ > matching category automatically.
178
+
179
+ ```js
180
+ // Pick the right category programmatically when needed
181
+ new Intl.PluralRules('pl').select(2); // → "few"
182
+ new Intl.PluralRules('en').select(2); // → "other"
183
+ ```
184
+
185
+ **Gender & grammatical agreement:** adjectives, articles, and past tense can change
186
+ with the gender of a noun or the user. Use ICU `select` rather than building
187
+ sentences from parts, and avoid embedding the user's name mid-sentence where
188
+ grammar would need to agree with it.
189
+
190
+ ```
191
+ {gender, select,
192
+ female {She updated her profile}
193
+ male {He updated his profile}
194
+ other {They updated their profile}
195
+ }
196
+ ```
197
+
198
+ ---
199
+
200
+ ## Locale-Aware Formatting
201
+
202
+ Dates, numbers, currency, and units are **locale data**, not strings. Use the
203
+ built-in `Intl` APIs (or platform equivalents) instead of hand-rolling formats.
204
+
205
+ ```js
206
+ // Dates — never assume MM/DD/YYYY (that's US-only)
207
+ new Intl.DateTimeFormat('en-US').format(date); // 3/13/2026 (M/D)
208
+ new Intl.DateTimeFormat('en-GB').format(date); // 13/03/2026 (D/M)
209
+ new Intl.DateTimeFormat('ja-JP').format(date); // 2026/3/13
210
+ new Intl.DateTimeFormat('ar-EG').format(date); // ١٣‏/٣‏/٢٠٢٦
211
+
212
+ // Numbers — decimal/grouping separators vary
213
+ new Intl.NumberFormat('de-DE').format(1234567.89); // "1.234.567,89"
214
+ new Intl.NumberFormat('en-US').format(1234567.89); // "1,234,567.89"
215
+
216
+ // Currency — symbol, placement, and spacing are locale-specific
217
+ new Intl.NumberFormat('de-DE', { style: 'currency', currency: 'EUR' })
218
+ .format(99.5); // "99,50 €"
219
+
220
+ // Units, relative time, lists
221
+ new Intl.NumberFormat('en', { style: 'unit', unit: 'kilometer' }).format(5);
222
+ new Intl.RelativeTimeFormat('en').format(-1, 'day'); // "1 day ago"
223
+ new Intl.ListFormat('en').format(['a','b','c']); // "a, b, and c"
224
+ ```
225
+
226
+ ### Beyond format strings
227
+
228
+ | Concern | Watch out for |
229
+ |---------|---------------|
230
+ | **Name order** | Many cultures put family name first (China, Japan, Korea, Hungary). Don't label fields "First / Last"—use "Given / Family" and don't force a split. |
231
+ | **Address format** | Field order, postal-code position, and required fields differ; some countries have no postal codes or states. Use a flexible/freeform model. |
232
+ | **Calendars & weeks** | Week may start Sunday, Monday, or Saturday; non-Gregorian calendars exist (Islamic, Hebrew, Japanese eras). |
233
+ | **Time zones** | Store UTC, display in the user's zone; show the zone for events. |
234
+ | **Phone numbers** | Vary in length and grouping; validate per-region (libphonenumber), not with one regex. |
235
+ | **Measurement units** | Metric vs. imperial; convert, don't just relabel. |
236
+
237
+ ### Forms that survive every locale
238
+
239
+ ```
240
+ ❌ Locale-blind form ✅ Locale-flexible form
241
+ ───────────────────── ──────────────────────────────────
242
+ First name [______] Given name(s) [__________]
243
+ Last name [______] Family name [__________]
244
+ (don't require a split; allow one
245
+ State [__▼] full-name field for cultures that
246
+ ZIP [_____] don't separate them)
247
+
248
+ Country [__▼] ← drives the rest
249
+ Address [__________]
250
+ (reveal region/postal fields based on
251
+ country; some have neither)
252
+ ```
253
+
254
+ - Drive address fields from the selected **country**, not a fixed US template.
255
+ - Don't mark a postal code or "state/province" as required globally—many places
256
+ have neither.
257
+ - Validate phone numbers per region; store names as entered (don't force casing or
258
+ ASCII—accept Unicode, including non-Latin scripts).
259
+
260
+ ---
261
+
262
+ ## Translation-Friendly UI
263
+
264
+ The UI must give translators what they need and tolerate what they produce.
265
+
266
+ - **No text baked into images.** Text in raster images can't be translated, scaled,
267
+ or read by screen readers. Use real text over CSS, or SVG with translatable `<text>`.
268
+ - **Externalize every string.** No hard-coded user-facing text in components.
269
+ Reference keys (`t('checkout.payNow')`), not English literals scattered in code.
270
+ - **Provide context for translators.** A key named `submit` is ambiguous—"submit a
271
+ form" vs. "submit to authority" translate differently. Add descriptions/comments
272
+ and, ideally, screenshots so translators see where the string appears.
273
+ - **Allow for missing translations.** New strings ship before translations land—fall
274
+ back gracefully to a base locale, never to a blank or a raw key like
275
+ `checkout.payNow` shown to the user.
276
+ - **Avoid idioms, humor, and culture-bound metaphors** ("hit it out of the park",
277
+ "piece of cake")—they don't translate and waste translator effort.
278
+ - **Leave room in the layout** (see Text Expansion) and avoid embedding UI controls
279
+ mid-sentence, which forces translators to break grammar around them.
280
+ - **Colors, icons, and imagery carry cultural meaning.** Red = danger/luck/celebration
281
+ depending on region; a mailbox or trash-can icon may not be universally recognized.
282
+ Prefer well-established universal icons and don't rely on one alone.
283
+
284
+ ---
285
+
286
+ ## Language Switching UX
287
+
288
+ - **List languages in their own name (endonyms):** "Deutsch", "Français", "日本語",
289
+ "العربية"—not "German", "French", "Japanese", "Arabic". A user who needs another
290
+ language can't necessarily read the current one.
291
+ - **Don't use flags to represent languages.** Languages ≠ countries: one flag can't
292
+ represent Spanish (20+ countries) and one country may have many languages. Flags
293
+ are also politically loaded. Use language names.
294
+ - **Detection vs. explicit choice:** you may default from `Accept-Language`/device
295
+ locale, but always offer an explicit, easy-to-find switch and **persist** the
296
+ user's choice (per account and/or `localStorage`/cookie). Never trap users in a
297
+ language they didn't pick.
298
+ - **Reflect the choice in everything:** `lang`/`dir` attributes, formatting, and the
299
+ URL/route where relevant (good for SEO: `/de/`, `hreflang` tags).
300
+ - Put the switcher somewhere conventional (header or footer) and reachable without
301
+ understanding the current language—an icon + endonym works well.
302
+
303
+ ---
304
+
305
+ ## Quick Checklist
306
+
307
+ - [ ] All user-facing strings externalized (no hard-coded text)
308
+ - [ ] Logical CSS properties; layout verified in `dir="rtl"`
309
+ - [ ] Directional icons mirror; logos/media/numerals do not
310
+ - [ ] Containers tolerate 30–40%+ text expansion; no fixed-width labels/buttons
311
+ - [ ] ICU MessageFormat for plurals/gender (not `count + "s"`)
312
+ - [ ] Dates, numbers, currency, units via `Intl` / locale data
313
+ - [ ] No text baked into images
314
+ - [ ] Translator context (descriptions/screenshots) provided
315
+ - [ ] Graceful fallback for missing translations (never raw keys)
316
+ - [ ] Language switcher uses endonyms, not flags; choice persists
317
+ - [ ] Pseudo-localization run in CI to catch overflow and hard-coded strings
318
+
319
+ ---
320
+
321
+ ## Common Mistakes
322
+
323
+ 1. **Hard-coded strings** — English literals in components; impossible to translate.
324
+ 2. **Assuming LTR** — physical CSS, manual `left`/`right` everywhere; RTL breaks.
325
+ 3. **Flags as languages** — wrong, ambiguous, and politically fraught.
326
+ 4. **Fixed-width text containers** — translated text overflows or truncates.
327
+ 5. **Concatenating fragments** — string-gluing ignores word order and grammar.
328
+ 6. **English plural rules** — `count + "s"` fails in most languages.
329
+ 7. **Text in images** — untranslatable, inaccessible, doesn't scale.
330
+ 8. **Hard-coded date/number formats** — `MM/DD/YYYY` and `,`/`.` are locale-specific.
331
+ 9. **"First name / Last name" assumptions** — name order and structure vary widely.
332
+ 10. **No translation fallback** — users see blank UI or raw keys for new strings.
333
+ 11. **Idioms and humor** — don't survive translation; confuse translators.
334
+ 12. **No translator context** — ambiguous keys produce wrong translations.
335
+
336
+ ---
337
+
338
+ ## Sources
339
+
340
+ - [W3C Internationalization (i18n) Activity](https://www.w3.org/International/) — specs and best practices
341
+ - [MDN: CSS Logical Properties](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_logical_properties_and_values)
342
+ - [MDN: Intl object](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl)
343
+ - [Unicode CLDR](https://cldr.unicode.org/) — locale data and plural rules
344
+ - [ICU MessageFormat](https://unicode-org.github.io/icu/userguide/format_parse/messages/) — plurals, gender, formatting
345
+ - [Material Design: Bidirectionality](https://m2.material.io/design/usability/bidirectionality.html)
346
+ - [Nielsen Norman Group: Translation & localization](https://www.nngroup.com/topic/international-users/)
347
+ - [Smashing Magazine: RTL Styling 101](https://www.smashingmagazine.com/2017/11/right-to-left-mobile-design/)
@@ -0,0 +1,308 @@
1
+ # Voice & Multimodal Input
2
+
3
+ Most interfaces still assume one input channel (a pointer or a touchscreen) and one
4
+ output channel (a screen). Real usage is **multimodal**: people speak, type, tap,
5
+ glance, and switch devices mid-task. This reference treats input modality as a
6
+ first-class design concern—voice, touch, pointer, keyboard, and their combinations—
7
+ across the range of contexts a product runs in (phone, desktop, watch, TV, car).
8
+
9
+ > This file is about *input modality in general*. For conversational AI chat
10
+ > patterns specifically, see [14-ai-ux-patterns.md](14-ai-ux-patterns.md).
11
+ > The two overlap (a voice assistant is often LLM-backed) but the concerns here—
12
+ > discoverability, recognition feedback, modality fallback—apply to any voice or
13
+ > multimodal UI, AI or not.
14
+
15
+ ---
16
+
17
+ ## Contents
18
+
19
+ - Voice as a First-Class Input
20
+ - Voice Copy & Timing
21
+ - Multimodal Interaction
22
+ - Cross-Device Continuity
23
+ - Accessibility Intersection
24
+ - Spatial & AR (Emerging — Low Priority)
25
+ - Quick Checklist
26
+ - Common Mistakes
27
+ - Sources
28
+
29
+ ---
30
+
31
+ ## Voice as a First-Class Input
32
+
33
+ Voice fails most often not because recognition is bad, but because the UX around it
34
+ is an afterthought: a tiny mic icon, no feedback, no recovery. Treat voice like any
35
+ other primary input—discoverable, with clear state, confirmation, and error paths.
36
+
37
+ ### Discoverability
38
+
39
+ - Don't bury voice behind an unlabeled mic glyph. Surface what it can do—suggested
40
+ utterances ("Try: *set a timer for 10 minutes*"), an onboarding hint, or visible
41
+ example commands.
42
+ - Voice's weakness is that **the command space is invisible**. Unlike a menu, users
43
+ can't see their options. Show examples in context and after errors.
44
+ - Make the entry point obvious and reachable; size the affordance like a real button,
45
+ not a 16px icon.
46
+
47
+ ### Feedback: show every state
48
+
49
+ Users must always know whether the system is **idle, listening, processing, or
50
+ speaking**. Silence is ambiguous—did it hear me? is it broken?
51
+
52
+ ```
53
+ States to make visible:
54
+ ● Idle — mic available, not capturing
55
+ ◉ Listening — actively capturing (waveform / pulsing animation)
56
+ ⋯ Processing — recognized, working on it
57
+ ▸ Responding — speaking / showing the result
58
+ ✕ Error — didn't catch that, with a way forward
59
+ ```
60
+
61
+ - Show **live transcription** of what was heard so the user can catch misrecognition
62
+ immediately ("I said *Karen* not *Aaron*").
63
+ - Pair audio output with a **visual transcript** of what the system said.
64
+
65
+ ### Confirmation & error recovery
66
+
67
+ - **Confirm consequential actions** before executing ("Delete all photos from
68
+ June—are you sure?"), but don't nag for trivial, reversible ones.
69
+ - Always show what was *recognized*, not just what was done, so errors are traceable.
70
+ - On failure, **offer a path forward**: re-prompt with an example, fall back to a
71
+ visual chooser, or let the user edit the transcribed text—never a dead-end
72
+ "Sorry, I didn't understand."
73
+ - **Support barge-in:** let users interrupt a long spoken response or prompt and
74
+ start their next command without waiting for it to finish.
75
+
76
+ ### A complete voice interaction, end to end
77
+
78
+ ```
79
+ ┌──────────────────────────────────────────────────────────┐
80
+ │ ● Tap to speak (idle — clear, button-sized entry) │
81
+ └──────────────────────────────────────────────────────────┘
82
+ │ user taps / says wake word
83
+ ▼
84
+ ┌──────────────────────────────────────────────────────────┐
85
+ │ ◉ Listening… ∿∿∿∿∿∿ (waveform reacts to voice) │
86
+ │ "set a timer for ten minutes" ← live transcript │
87
+ └──────────────────────────────────────────────────────────┘
88
+ │ endpoint detected (or user taps stop)
89
+ ▼
90
+ ┌──────────────────────────────────────────────────────────┐
91
+ │ ⋯ Working on it… │
92
+ └──────────────────────────────────────────────────────────┘
93
+ │
94
+ ▼
95
+ ┌──────────────────────────────────────────────────────────┐
96
+ │ ▸ Timer set for 10:00. [⏸ Pause] [✕ Cancel] │
97
+ │ (spoken aloud AND shown as text + a visual control) │
98
+ └──────────────────────────────────────────────────────────┘
99
+ │ low confidence? → don't guess silently:
100
+ ▼
101
+ ┌──────────────────────────────────────────────────────────┐
102
+ │ ✕ Did you mean a timer or an alarm? [Timer] [Alarm] │
103
+ │ (recover with a visual chooser, not a dead end) │
104
+ └──────────────────────────────────────────────────────────┘
105
+ ```
106
+
107
+ The result is **multimodal by default**: voice in, but the response is spoken,
108
+ shown as text, *and* leaves behind tappable controls so the user can continue by
109
+ whichever modality suits the moment.
110
+
111
+ ### Activation: wake words vs. push-to-talk
112
+
113
+ - **Push-to-talk** (tap/hold to speak) is explicit, privacy-friendly, and reliable
114
+ in noisy/public settings—prefer it for on-screen UIs and anything sensitive.
115
+ - **Wake words** ("Hey …") enable hands-free use but require always-listening, which
116
+ raises privacy stakes—be explicit about on-device detection and show a clear
117
+ live-mic indicator. Offer a way to disable the wake word entirely.
118
+ - Whichever you use, provide an **unambiguous stop**, and never begin acting on audio
119
+ captured before the explicit trigger.
120
+
121
+ ### Recognition feedback in code
122
+
123
+ ```js
124
+ // Web Speech API — surface interim results so users see misrecognition live
125
+ const rec = new (window.SpeechRecognition || window.webkitSpeechRecognition)();
126
+ rec.interimResults = true; // stream partial transcripts → live feedback
127
+ rec.lang = navigator.language; // honor the user's locale (see i18n reference)
128
+
129
+ rec.onstart = () => setState('listening'); // show waveform / pulsing mic
130
+ rec.onresult = (e) => {
131
+ const t = Array.from(e.results).map(r => r[0].transcript).join('');
132
+ showTranscript(t); // user can catch "Aaron" vs "Karen"
133
+ };
134
+ rec.onerror = (e) => recoverWithVisualFallback(e.error); // never a dead end
135
+ rec.onend = () => setState('idle'); // always return to a known state
136
+ ```
137
+
138
+ ### Privacy is part of voice UX
139
+
140
+ - Make it unambiguous when the mic is **live** (persistent indicator, not just at start).
141
+ - Be explicit about whether audio is processed on-device or sent to a server, and
142
+ whether it's retained. Don't capture ambient audio before an explicit trigger.
143
+ - Offer a visible, one-tap way to stop listening and to review/delete history.
144
+
145
+ ---
146
+
147
+ ## Voice Copy & Timing
148
+
149
+ Spoken language is linear and unscannable—a user can't skim a sentence they hear the
150
+ way they skim text. Voice copy and timing need their own discipline.
151
+
152
+ - **Be brief and front-load the answer.** Say the result first, details after
153
+ ("Done—your timer's set for 10 minutes" not a 20-word preamble). Long spoken
154
+ responses are painful; offer "want the details?" rather than dumping them.
155
+ - **Write for the ear.** Short sentences, no nested clauses, no jargon, no
156
+ "click here" (there's nothing to click). Numbers and dates should be speakable.
157
+ - **One question at a time.** Don't ask "what, when, and where?" in one prompt—the
158
+ user can only hold so much, and won't remember which to answer first.
159
+ - **Tune endpointing (silence timeout).** Cut off too early and you clip the user;
160
+ wait too long and it feels broken. Allow for natural pauses and "thinking" silence,
161
+ and let users signal they're done (a tap, or a stop word).
162
+ - **Set expectations on latency.** If processing will take more than ~1s, fill the
163
+ gap with a clear "working on it" state (visual + optional audio cue), not silence.
164
+
165
+ ```
166
+ ❌ "I have located three results matching your query. The first result is…"
167
+ ✅ "Found three. Top one: … Want the others?"
168
+ ```
169
+
170
+ ---
171
+
172
+ ## Multimodal Interaction
173
+
174
+ Multimodal means combining input channels so each does what it's best at—and letting
175
+ users move fluidly between them. The classic example: "put **that** [point] **there**
176
+ [point]" combines voice with pointing; neither alone is as fast.
177
+
178
+ ### Combine voice + touch + pointer + keyboard
179
+
180
+ - Let modalities **complement**, not compete: voice for intent ("make this blue"),
181
+ pointing/selection for the target.
182
+ - Don't force a single modality for a whole task. A user dictating a message should
183
+ still be able to tap to fix one word instead of re-recording the sentence.
184
+ - Keep state shared across modalities—what you select by touch is the referent for
185
+ the next voice command.
186
+
187
+ ### Modality-appropriate design per context
188
+
189
+ The right primary modality depends on the device and the user's situation. Design
190
+ the *same capability* to suit each.
191
+
192
+ | Context | Constraints | Lean on |
193
+ |---------|-------------|---------|
194
+ | **Phone (on the go)** | One hand, glancing, noisy/quiet | Touch primary; voice for hands-busy |
195
+ | **Desktop** | Keyboard + precise pointer, focused | Keyboard shortcuts, pointer; voice optional |
196
+ | **Watch** | Tiny screen, brief glances | Voice + a few taps; minimal text entry |
197
+ | **TV / 10-foot UI** | Distance, D-pad/remote, no keyboard | Voice search >> on-screen keyboard; large focus targets |
198
+ | **Car** | Eyes/hands occupied, safety-critical | Voice primary; minimal-glance visuals, large touch zones |
199
+ | **Kiosk / public** | Shared, may be loud, privacy concerns | Touch primary; voice often inappropriate |
200
+
201
+ - **"Eyes-free / hands-free" contexts** (driving, cooking) make voice essential and
202
+ visual-only UIs unusable—but a noisy or public context can make voice useless.
203
+ Design for the situation, not the gadget.
204
+
205
+ ### Graceful degradation when a modality is unavailable
206
+
207
+ - A mic may be denied, broken, muted, or socially inappropriate; a screen may be off;
208
+ a keyboard may be absent. **Every core task must be completable in more than one way.**
209
+ - If voice is unavailable, the same action must be reachable by touch/pointer/keyboard,
210
+ and vice versa. Detect capability and adapt the offered affordances; don't show a
211
+ dead voice button when there's no mic permission.
212
+
213
+ ---
214
+
215
+ ## Cross-Device Continuity
216
+
217
+ Users start a task on one device and finish on another. Continuity is a multimodal
218
+ problem because the *input* changes across devices, not just the layout.
219
+
220
+ - **Handoff:** let a task move between devices with state intact (a started form, a
221
+ half-written message, a playback position). Surface "continue on this device".
222
+ - **Responsive *input*, not just responsive *layout*:** a layout that reflows but
223
+ still assumes hover, precise clicks, or a physical keyboard fails on touch/TV/voice.
224
+ Adapt interactions (tap targets, focus order, voice entry) to the device's modality.
225
+ - **Context preservation:** carry selection, scroll position, undo history, and auth
226
+ across the handoff so the user resumes rather than restarts.
227
+ - Keep identity and sync trustworthy—show what synced and when (see
228
+ [12b-conflict-resolution-sync.md](12b-conflict-resolution-sync.md)).
229
+
230
+ ---
231
+
232
+ ## Accessibility Intersection
233
+
234
+ Voice and multimodal design overlap heavily with accessibility (see
235
+ [03-accessibility.md](03-accessibility.md)). They help some users and can
236
+ exclude others—design for both directions.
237
+
238
+ - **Voice as an assistive input:** for users with motor or dexterity impairments,
239
+ voice can be the *primary* way to operate an interface. Voice control
240
+ (Voice Control, Voice Access) needs every actionable element to be reliably named
241
+ and reachable—the same semantics screen readers need.
242
+ - **Captions and visual alternatives for audio output:** anything the system *says*
243
+ must also be available as text for Deaf and hard-of-hearing users, and for anyone
244
+ in a sound-off context.
245
+ - **Never make voice the only path.** Speech-impaired users, people with strong
246
+ accents, non-native speakers, and those in quiet/public/noisy settings need a
247
+ non-voice route to the same outcome. Voice is an *addition*, not a replacement.
248
+ - Visual recognition feedback (live transcript) doubles as an accessibility feature.
249
+
250
+ ---
251
+
252
+ ## Spatial & AR (Emerging — Low Priority)
253
+
254
+ Spatial computing (headsets, AR overlays) adds gaze, hand-tracking, and 3D gesture
255
+ as input modalities. It's worth acknowledging but, for most products in 2026, **not
256
+ worth investing in yet**—the audience is small and patterns are unsettled. Keep two
257
+ durable principles in mind in case it becomes relevant:
258
+
259
+ - **Gaze ≠ intent.** Looking at something isn't selecting it; require a deliberate
260
+ confirm (pinch, dwell-with-feedback, voice) to avoid the "Midas touch" problem.
261
+ - **Comfort and fatigue are UX constraints.** "Gorilla arm" from mid-air gestures and
262
+ motion discomfort are real; favor small, low-effort gestures and respect reduced-motion.
263
+
264
+ Treat AR/spatial as a place to watch, not a default investment.
265
+
266
+ ---
267
+
268
+ ## Quick Checklist
269
+
270
+ - [ ] Voice entry is discoverable, with example commands (not just a bare mic icon)
271
+ - [ ] Idle / listening / processing / responding / error states are all visible
272
+ - [ ] Live transcription shows what was recognized
273
+ - [ ] Consequential voice actions are confirmed; trivial ones are not
274
+ - [ ] Errors offer a path forward (re-prompt, edit, or fall back to visual)
275
+ - [ ] Barge-in supported; mic-live state and privacy are clear
276
+ - [ ] Every core task is completable in ≥ 2 modalities (graceful degradation)
277
+ - [ ] Interactions—not just layout—adapt to device modality
278
+ - [ ] Spoken output has a text/caption equivalent
279
+ - [ ] Voice is never the only path to any outcome
280
+
281
+ ---
282
+
283
+ ## Common Mistakes
284
+
285
+ 1. **Voice-only flows** — no fallback excludes speech-impaired, accented, non-native,
286
+ and silent-context users.
287
+ 2. **Hidden mic / no discoverability** — users can't find voice or guess its commands.
288
+ 3. **No recognition feedback** — no live transcript, so misrecognition goes unnoticed.
289
+ 4. **Ambiguous state** — user can't tell if the system is listening, thinking, or dead.
290
+ 5. **Dead-end errors** — "Sorry, I didn't understand" with no way forward.
291
+ 6. **No confirmation for destructive voice commands** — easy to delete by mishearing.
292
+ 7. **Ignoring privacy** — unclear mic-live state, ambient capture, no history controls.
293
+ 8. **Responsive layout, non-responsive input** — reflows but still assumes hover/keyboard.
294
+ 9. **Audio-only output** — no captions/transcript for spoken responses.
295
+ 10. **Forcing one modality per task** — can't tap to fix a single dictated word.
296
+
297
+ ---
298
+
299
+ ## Sources
300
+
301
+ - [Nielsen Norman Group: Voice Interfaces & Usability](https://www.nngroup.com/topic/voice-interaction/)
302
+ - [Google: Conversation Design Guidelines](https://developers.google.com/assistant/conversation-design/welcome)
303
+ - [Apple HIG: Siri & Voice](https://developer.apple.com/design/human-interface-guidelines/siri)
304
+ - [Apple HIG: Multimodal & Inputs](https://developer.apple.com/design/human-interface-guidelines/inputs)
305
+ - [W3C: Voice & Multimodal Interaction](https://www.w3.org/standards/) — standards activity
306
+ - [MDN: Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API)
307
+ - [Microsoft: Multimodal interaction guidance](https://learn.microsoft.com/en-us/windows/apps/design/input/)
308
+ - [Bolt, R. (1980). "Put-That-There"](https://dl.acm.org/doi/10.1145/800250.807503) — foundational multimodal research