@gmessier/nitro-speech 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +116 -27
- package/android/src/main/AndroidManifest.xml +1 -0
- package/android/src/main/java/com/margelo/nitro/nitrospeech/recognizer/HapticImpact.kt +66 -0
- package/android/src/main/java/com/margelo/nitro/nitrospeech/recognizer/HybridRecognizer.kt +11 -0
- package/ios/HapticImpact.swift +23 -0
- package/ios/HybridRecognizer.swift +28 -10
- package/lib/commonjs/index.js +44 -22
- package/lib/commonjs/index.js.map +1 -1
- package/lib/module/index.js +42 -21
- package/lib/module/index.js.map +1 -1
- package/lib/tsconfig.tsbuildinfo +1 -1
- package/lib/typescript/index.d.ts +23 -3
- package/lib/typescript/index.d.ts.map +1 -1
- package/lib/typescript/specs/NitroSpeech.nitro.d.ts +13 -0
- package/lib/typescript/specs/NitroSpeech.nitro.d.ts.map +1 -1
- package/nitrogen/generated/android/c++/JHapticFeedbackStyle.hpp +62 -0
- package/nitrogen/generated/android/c++/JHybridRecognizerSpec.cpp +4 -0
- package/nitrogen/generated/android/c++/JSpeechToTextParams.hpp +11 -1
- package/nitrogen/generated/android/kotlin/com/margelo/nitro/nitrospeech/HapticFeedbackStyle.kt +22 -0
- package/nitrogen/generated/android/kotlin/com/margelo/nitro/nitrospeech/SpeechToTextParams.kt +8 -2
- package/nitrogen/generated/ios/NitroSpeech-Swift-Cxx-Bridge.hpp +18 -0
- package/nitrogen/generated/ios/NitroSpeech-Swift-Cxx-Umbrella.hpp +3 -0
- package/nitrogen/generated/ios/c++/HybridRecognizerSpecSwift.hpp +3 -0
- package/nitrogen/generated/ios/swift/HapticFeedbackStyle.swift +44 -0
- package/nitrogen/generated/ios/swift/SpeechToTextParams.swift +47 -1
- package/nitrogen/generated/shared/c++/HapticFeedbackStyle.hpp +80 -0
- package/nitrogen/generated/shared/c++/SpeechToTextParams.hpp +12 -2
- package/package.json +4 -4
- package/src/index.ts +43 -21
- package/src/specs/NitroSpeech.nitro.ts +14 -0
package/README.md
CHANGED
|
@@ -4,11 +4,30 @@
|
|
|
4
4
|
[](https://github.com/NotGeorgeMessier/nitro-speech/blob/main/LICENSE)
|
|
5
5
|
[](https://www.npmjs.com/package/@gmessier/nitro-speech)
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
>
|
|
9
|
-
>
|
|
10
|
-
|
|
11
|
-
|
|
7
|
+
|
|
8
|
+
> If you hit an issue, please open a GitHub issue or reach out to me on Discord / Twitter (X) — response is guaranteed.
|
|
9
|
+
>
|
|
10
|
+
> - GitHub Issues: [NotGeorgeMessier/nitro-speech/issues](https://github.com/NotGeorgeMessier/nitro-speech/issues)
|
|
11
|
+
> - Discord: `gmessier`
|
|
12
|
+
> - Twitter (X): `SufferingGeorge`
|
|
13
|
+
|
|
14
|
+
React Native Real-Time Speech Recognition Library, powered by [Nitro Modules](https://github.com/mrousavy/nitro).
|
|
15
|
+
|
|
16
|
+
#### Key Features:
|
|
17
|
+
|
|
18
|
+
- Built on Nitro Modules for low-overhead native bridging
|
|
19
|
+
- Configurable Timer for silence (default: 8 sec)
|
|
20
|
+
- Callback `onAutoFinishProgress` for progress bars, etc...
|
|
21
|
+
- Method `addAutoFinishTime` for single timer update
|
|
22
|
+
- Method `updateAutoFinishTime` for constant timer update
|
|
23
|
+
- Optional Haptic Feedback on start and finish
|
|
24
|
+
- Speech-quality configurations:
|
|
25
|
+
- Result is grouped by speech segments into Batches.
|
|
26
|
+
- Param `disableRepeatingFilter` for consecutive duplicate-word filtering.
|
|
27
|
+
- Param `androidDisableBatchHandling` for removing empty recognition result.
|
|
28
|
+
- Embedded Permission handling
|
|
29
|
+
- Callback `onPermissionDenied` - if user denied the request
|
|
30
|
+
- Everything else that could be found in Expo or other libraries
|
|
12
31
|
|
|
13
32
|
## Table of Contents
|
|
14
33
|
|
|
@@ -17,7 +36,9 @@ Speech recognition for React Native, powered by [Nitro Modules](https://github.c
|
|
|
17
36
|
- [Features](#features)
|
|
18
37
|
- [Usage](#usage)
|
|
19
38
|
- [Recommended: useRecognizer Hook](#recommended-userecognizer-hook)
|
|
20
|
-
- [
|
|
39
|
+
- [With React Navigation (important)](#with-react-navigation-important)
|
|
40
|
+
- [Cross-component control: RecognizerRef](#cross-component-control-recognizerref)
|
|
41
|
+
- [Unsafe: RecognizerSession](#unsafe-recognizersession)
|
|
21
42
|
- [API Reference](#api-reference)
|
|
22
43
|
- [Requirements](#requirements)
|
|
23
44
|
- [Troubleshooting](#troubleshooting)
|
|
@@ -60,6 +81,7 @@ The library declares the required permission in its `AndroidManifest.xml` (merge
|
|
|
60
81
|
|
|
61
82
|
```xml
|
|
62
83
|
<uses-permission android:name="android.permission.RECORD_AUDIO" />
|
|
84
|
+
<uses-permission android:name="android.permission.VIBRATE" />
|
|
63
85
|
```
|
|
64
86
|
|
|
65
87
|
### iOS
|
|
@@ -81,12 +103,13 @@ Both permissions are required for speech recognition to work on iOS.
|
|
|
81
103
|
|---------|-------------|-----|---------|
|
|
82
104
|
| **Real-time transcription** | Get partial results as the user speaks, enabling live UI updates | ✅ | ✅ |
|
|
83
105
|
| **Auto-stop on silence** | Automatically stops recognition after configurable inactivity period (default: 8s) | ✅ | ✅ |
|
|
84
|
-
| **Auto-finish progress** | Progress callbacks showing countdown until auto-stop | ✅ |
|
|
85
|
-
| **
|
|
106
|
+
| **Auto-finish progress** | Progress callbacks showing countdown until auto-stop | ✅ | *(TODO)* |
|
|
107
|
+
| **Haptic feedback** | Optional haptics on recording start/stop | ✅ | ✅ |
|
|
86
108
|
| **Background handling** | Auto-stop when app loses focus/goes to background | ✅ | Not Safe *(TODO)* |
|
|
87
|
-
| **Contextual strings** | Domain-specific vocabulary for improved accuracy | ✅ | ✅ |
|
|
88
|
-
| **Repeating word filter** | Removes consecutive duplicate words from artifacts | ✅ | ✅ |
|
|
89
109
|
| **Permission handling** | Dedicated `onPermissionDenied` callback | ✅ | ✅ |
|
|
110
|
+
| **Repeating word filter** | Removes consecutive duplicate words from artifacts | ✅ | ✅ |
|
|
111
|
+
| **Locale support** | Configure speech recognizer for different languages | ✅ | ✅ |
|
|
112
|
+
| **Contextual strings** | Domain-specific vocabulary for improved accuracy | ✅ | ✅ |
|
|
90
113
|
| **Automatic punctuation** | Adds punctuation to transcription (iOS 16+) | ✅ | Auto |
|
|
91
114
|
| **Language model selection** | Choose between web search vs free-form models | Auto | ✅ |
|
|
92
115
|
| **Offensive word masking** | Control whether offensive words are masked | Auto | ✅ |
|
|
@@ -96,6 +119,9 @@ Both permissions are required for speech recognition to work on iOS.
|
|
|
96
119
|
|
|
97
120
|
### Recommended: useRecognizer Hook
|
|
98
121
|
|
|
122
|
+
`useRecognizer` is lifecycle-aware. It calls `stopListening()` during cleanup (unmount or `destroyDeps` change).
|
|
123
|
+
Because of that, treat it as a **single session owner** setup hook: use it once per recognition session/screen, where you define callbacks.
|
|
124
|
+
|
|
99
125
|
```typescript
|
|
100
126
|
import { useRecognizer } from '@gmessier/nitro-speech';
|
|
101
127
|
|
|
@@ -130,14 +156,20 @@ function MyComponent() {
|
|
|
130
156
|
<View>
|
|
131
157
|
<TouchableOpacity onPress={() => startListening({
|
|
132
158
|
locale: 'en-US',
|
|
159
|
+
disableRepeatingFilter: false,
|
|
133
160
|
autoFinishRecognitionMs: 8000,
|
|
161
|
+
|
|
134
162
|
contextualStrings: ['custom', 'words'],
|
|
163
|
+
// Haptics (both platforms)
|
|
164
|
+
startHapticFeedbackStyle: 'medium',
|
|
165
|
+
stopHapticFeedbackStyle: 'light',
|
|
135
166
|
// iOS specific
|
|
136
167
|
iosAddPunctuation: true,
|
|
137
168
|
// Android specific
|
|
138
169
|
androidMaskOffensiveWords: false,
|
|
139
170
|
androidFormattingPreferQuality: false,
|
|
140
171
|
androidUseWebSearchModel: false,
|
|
172
|
+
androidDisableBatchHandling: false,
|
|
141
173
|
})}>
|
|
142
174
|
<Text>Start Listening</Text>
|
|
143
175
|
</TouchableOpacity>
|
|
@@ -155,64 +187,106 @@ function MyComponent() {
|
|
|
155
187
|
}
|
|
156
188
|
```
|
|
157
189
|
|
|
158
|
-
|
|
190
|
+
Use the handlers returned by this single hook instance inside that owner component.
|
|
191
|
+
For other components, avoid creating another `useRecognizer` instance for the same session.
|
|
192
|
+
|
|
193
|
+
### With React Navigation (important)
|
|
194
|
+
|
|
195
|
+
React Navigation **doesn’t unmount screens** when you navigate — the screen can stay mounted in the background and come back without remounting. See: [Navigation lifecycle (React Navigation)](https://reactnavigation.org/docs/8.x/navigation-lifecycle/#summary).
|
|
196
|
+
|
|
197
|
+
Because of that, prefer tying recognition cleanup to **focus state**, not just component unmount. A simple approach is `useIsFocused()` and passing it into `useRecognizer`’s `destroyDeps` so recognition stops when the screen blurs. See: `[useIsFocused` (React Navigation)](https://reactnavigation.org/docs/8.x/use-is-focused).
|
|
198
|
+
|
|
199
|
+
```typescript
|
|
200
|
+
const isFocused = useIsFocused();
|
|
201
|
+
const {
|
|
202
|
+
// ...
|
|
203
|
+
} = useRecognizer(
|
|
204
|
+
{
|
|
205
|
+
// ...
|
|
206
|
+
},
|
|
207
|
+
[isFocused]
|
|
208
|
+
);
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
### Cross-component control: RecognizerRef
|
|
212
|
+
|
|
213
|
+
If you need to call recognizer methods from other components without prop drilling, use `RecognizerRef`.
|
|
214
|
+
|
|
215
|
+
```typescript
|
|
216
|
+
import { RecognizerRef } from '@gmessier/nitro-speech';
|
|
217
|
+
|
|
218
|
+
RecognizerRef.startListening({ locale: 'en-US' });
|
|
219
|
+
RecognizerRef.addAutoFinishTime(5000);
|
|
220
|
+
RecognizerRef.updateAutoFinishTime(10000, true);
|
|
221
|
+
RecognizerRef.stopListening();
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
`RecognizerRef` exposes only method handlers and is safe for cross-component method access.
|
|
225
|
+
|
|
226
|
+
### Unsafe: RecognizerSession
|
|
227
|
+
|
|
228
|
+
`RecognizerSession` is the hybrid object. It gives direct access to callbacks and control methods, but it is unsafe to orchestrate the full session directly from it.
|
|
159
229
|
|
|
160
230
|
```typescript
|
|
161
|
-
import {
|
|
231
|
+
import { RecognizerSession } from '@gmessier/nitro-speech';
|
|
162
232
|
|
|
163
233
|
// Set up callbacks
|
|
164
|
-
|
|
234
|
+
RecognizerSession.onReadyForSpeech = () => {
|
|
165
235
|
console.log('Listening...');
|
|
166
236
|
};
|
|
167
237
|
|
|
168
|
-
|
|
238
|
+
RecognizerSession.onResult = (textBatches) => {
|
|
169
239
|
console.log('Result:', textBatches.join('\n'));
|
|
170
240
|
};
|
|
171
241
|
|
|
172
|
-
|
|
242
|
+
RecognizerSession.onRecordingStopped = () => {
|
|
173
243
|
console.log('Stopped');
|
|
174
244
|
};
|
|
175
245
|
|
|
176
|
-
|
|
246
|
+
RecognizerSession.onAutoFinishProgress = (timeLeftMs) => {
|
|
177
247
|
console.log('Auto-stop in:', timeLeftMs, 'ms');
|
|
178
248
|
};
|
|
179
249
|
|
|
180
|
-
|
|
250
|
+
RecognizerSession.onError = (error) => {
|
|
181
251
|
console.log('Error:', error);
|
|
182
252
|
};
|
|
183
253
|
|
|
184
|
-
|
|
254
|
+
RecognizerSession.onPermissionDenied = () => {
|
|
185
255
|
console.log('Permission denied');
|
|
186
256
|
};
|
|
187
257
|
|
|
188
258
|
// Start listening
|
|
189
|
-
|
|
259
|
+
RecognizerSession.startListening({
|
|
190
260
|
locale: 'en-US',
|
|
191
261
|
});
|
|
192
262
|
|
|
193
263
|
// Stop listening
|
|
194
|
-
|
|
264
|
+
RecognizerSession.stopListening();
|
|
195
265
|
|
|
196
266
|
// Manually add time to auto finish timer
|
|
197
|
-
|
|
198
|
-
|
|
267
|
+
RecognizerSession.addAutoFinishTime(5000); // Add 5 seconds
|
|
268
|
+
RecognizerSession.addAutoFinishTime(); // Reset to original time
|
|
199
269
|
|
|
200
270
|
// Update auto finish time
|
|
201
|
-
|
|
202
|
-
|
|
271
|
+
RecognizerSession.updateAutoFinishTime(10000); // Set to 10 seconds
|
|
272
|
+
RecognizerSession.updateAutoFinishTime(10000, true); // Set to 10 seconds and refresh progress
|
|
203
273
|
```
|
|
204
274
|
|
|
205
275
|
### ⚠️ About dispose()
|
|
206
276
|
|
|
207
|
-
The `
|
|
277
|
+
The `RecognizerSession.dispose()` method is **NOT SAFE** and should rarely be used. Hybrid Objects in Nitro are typically managed by the JS garbage collector automatically. Only call `dispose()` in performance-critical scenarios where you need to eagerly destroy objects.
|
|
208
278
|
|
|
209
279
|
**See:** [Nitro dispose() documentation](https://nitro.margelo.com/docs/hybrid-objects#dispose)
|
|
210
280
|
|
|
211
281
|
## API Reference
|
|
212
282
|
|
|
213
|
-
### `useRecognizer(callbacks)`
|
|
283
|
+
### `useRecognizer(callbacks, destroyDeps?)`
|
|
214
284
|
|
|
215
|
-
|
|
285
|
+
#### Usage notes
|
|
286
|
+
|
|
287
|
+
- Use `useRecognizer` once per session/screen as the session setup owner.
|
|
288
|
+
- Cleanup stops recognition, so mounting multiple instances can unexpectedly end an active session.
|
|
289
|
+
- For method access in non-owner components, use `RecognizerRef`.
|
|
216
290
|
|
|
217
291
|
#### Parameters
|
|
218
292
|
|
|
@@ -223,6 +297,7 @@ A React hook that provides lifecycle-aware access to the speech recognizer.
|
|
|
223
297
|
- `onAutoFinishProgress?: (timeLeftMs: number) => void` - Called each second during auto-finish countdown
|
|
224
298
|
- `onError?: (message: string) => void` - Called when an error occurs
|
|
225
299
|
- `onPermissionDenied?: () => void` - Called if microphone permission is denied
|
|
300
|
+
- `destroyDeps` (array, optional) - Additional dependencies for the cleanup effect. When any of these change (or the component unmounts), recognition is stopped.
|
|
226
301
|
|
|
227
302
|
#### Returns
|
|
228
303
|
|
|
@@ -231,6 +306,18 @@ A React hook that provides lifecycle-aware access to the speech recognizer.
|
|
|
231
306
|
- `addAutoFinishTime(additionalTimeMs?: number)` - Add time to the auto-finish timer (or reset to original if no parameter)
|
|
232
307
|
- `updateAutoFinishTime(newTimeMs: number, withRefresh?: boolean)` - Update the auto-finish timer
|
|
233
308
|
|
|
309
|
+
### `RecognizerRef`
|
|
310
|
+
|
|
311
|
+
- `startListening(params: SpeechToTextParams)`
|
|
312
|
+
- `stopListening()`
|
|
313
|
+
- `addAutoFinishTime(additionalTimeMs?: number)`
|
|
314
|
+
- `updateAutoFinishTime(newTimeMs: number, withRefresh?: boolean)`
|
|
315
|
+
|
|
316
|
+
### `RecognizerSession`
|
|
317
|
+
|
|
318
|
+
- Exposes callbacks (`onReadyForSpeech`, `onResult`, etc.) and control methods.
|
|
319
|
+
- Prefer `useRecognizer` (single owner) + `RecognizerRef` for app-level usage.
|
|
320
|
+
|
|
234
321
|
### `SpeechToTextParams`
|
|
235
322
|
|
|
236
323
|
Configuration object for speech recognition.
|
|
@@ -241,6 +328,8 @@ Configuration object for speech recognition.
|
|
|
241
328
|
- `autoFinishRecognitionMs?: number` - Auto-stop timeout in milliseconds (default: `8000`)
|
|
242
329
|
- `contextualStrings?: string[]` - Array of domain-specific words for better recognition
|
|
243
330
|
- `disableRepeatingFilter?: boolean` - Disable filter that removes consecutive duplicate words (default: `false`)
|
|
331
|
+
- `startHapticFeedbackStyle?: 'light' | 'medium' | 'heavy'` - Haptic feedback style when microphone starts recording (default: `null` / disabled)
|
|
332
|
+
- `stopHapticFeedbackStyle?: 'light' | 'medium' | 'heavy'` - Haptic feedback style when microphone stops recording (default: `null` / disabled)
|
|
244
333
|
|
|
245
334
|
#### iOS-Specific Parameters
|
|
246
335
|
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
package com.margelo.nitro.nitrospeech.recognizer
|
|
2
|
+
|
|
3
|
+
import android.content.Context
|
|
4
|
+
import android.os.Build
|
|
5
|
+
import android.os.VibrationEffect
|
|
6
|
+
import android.os.Vibrator
|
|
7
|
+
import android.os.VibratorManager
|
|
8
|
+
import com.margelo.nitro.nitrospeech.HapticFeedbackStyle
|
|
9
|
+
|
|
10
|
+
class HapticImpact(
|
|
11
|
+
private val style: HapticFeedbackStyle = HapticFeedbackStyle.MEDIUM,
|
|
12
|
+
) {
|
|
13
|
+
private data class LegacyOneShot(
|
|
14
|
+
val durationMs: Long,
|
|
15
|
+
val amplitude: Int,
|
|
16
|
+
)
|
|
17
|
+
|
|
18
|
+
fun trigger(context: Context) {
|
|
19
|
+
val vibrator = getVibrator(context) ?: return
|
|
20
|
+
if (!vibrator.hasVibrator()) return
|
|
21
|
+
|
|
22
|
+
try {
|
|
23
|
+
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
|
|
24
|
+
val effect = when (style) {
|
|
25
|
+
HapticFeedbackStyle.LIGHT -> VibrationEffect.EFFECT_TICK
|
|
26
|
+
HapticFeedbackStyle.MEDIUM -> VibrationEffect.EFFECT_CLICK
|
|
27
|
+
HapticFeedbackStyle.HEAVY -> VibrationEffect.EFFECT_HEAVY_CLICK
|
|
28
|
+
}
|
|
29
|
+
vibrator.vibrate(VibrationEffect.createPredefined(effect))
|
|
30
|
+
return
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
val legacyOneShot = when (style) {
|
|
34
|
+
HapticFeedbackStyle.LIGHT -> LegacyOneShot(durationMs = 12L, amplitude = 50)
|
|
35
|
+
HapticFeedbackStyle.MEDIUM -> LegacyOneShot(durationMs = 18L, amplitude = 100)
|
|
36
|
+
HapticFeedbackStyle.HEAVY -> LegacyOneShot(durationMs = 28L, amplitude = 180)
|
|
37
|
+
}
|
|
38
|
+
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
|
|
39
|
+
vibrator.vibrate(
|
|
40
|
+
VibrationEffect.createOneShot(
|
|
41
|
+
legacyOneShot.durationMs,
|
|
42
|
+
legacyOneShot.amplitude
|
|
43
|
+
)
|
|
44
|
+
)
|
|
45
|
+
} else {
|
|
46
|
+
@Suppress("DEPRECATION")
|
|
47
|
+
vibrator.vibrate(legacyOneShot.durationMs)
|
|
48
|
+
}
|
|
49
|
+
} catch (_: SecurityException) {
|
|
50
|
+
// Missing android.permission.VIBRATE or disallowed by device policy.
|
|
51
|
+
} catch (_: Throwable) {
|
|
52
|
+
// Never crash the recognition flow because of haptics.
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
private fun getVibrator(context: Context): Vibrator? {
|
|
57
|
+
return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
|
|
58
|
+
val manager =
|
|
59
|
+
context.getSystemService(Context.VIBRATOR_MANAGER_SERVICE) as? VibratorManager
|
|
60
|
+
manager?.defaultVibrator
|
|
61
|
+
} else {
|
|
62
|
+
@Suppress("DEPRECATION")
|
|
63
|
+
context.getSystemService(Context.VIBRATOR_SERVICE) as? Vibrator
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
@@ -84,6 +84,11 @@ class HybridRecognizer: HybridRecognizerSpec() {
|
|
|
84
84
|
if (!isActive) return
|
|
85
85
|
onFinishRecognition(null, null, true)
|
|
86
86
|
mainHandler.postDelayed({
|
|
87
|
+
val context = NitroModules.applicationContext
|
|
88
|
+
val hapticImpact = config?.stopHapticFeedbackStyle
|
|
89
|
+
if (hapticImpact != null && context != null) {
|
|
90
|
+
HapticImpact(hapticImpact).trigger(context)
|
|
91
|
+
}
|
|
87
92
|
cleanup()
|
|
88
93
|
}, POST_RECOGNITION_DELAY)
|
|
89
94
|
}
|
|
@@ -156,6 +161,12 @@ class HybridRecognizer: HybridRecognizerSpec() {
|
|
|
156
161
|
|
|
157
162
|
speechRecognizer?.startListening(intent)
|
|
158
163
|
isActive = true
|
|
164
|
+
|
|
165
|
+
val hapticImpact = config?.startHapticFeedbackStyle
|
|
166
|
+
if (hapticImpact != null) {
|
|
167
|
+
HapticImpact(hapticImpact).trigger(context)
|
|
168
|
+
}
|
|
169
|
+
|
|
159
170
|
mainHandler.postDelayed({
|
|
160
171
|
if (isActive) {
|
|
161
172
|
onReadyForSpeech?.invoke()
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
import Foundation
|
|
2
|
+
import UIKit
|
|
3
|
+
|
|
4
|
+
class HapticImpact {
|
|
5
|
+
private let impactGenerator: UIImpactFeedbackGenerator
|
|
6
|
+
|
|
7
|
+
init(style: HapticFeedbackStyle) {
|
|
8
|
+
let hapticStyle = switch style {
|
|
9
|
+
case .light:
|
|
10
|
+
UIImpactFeedbackGenerator.FeedbackStyle.light
|
|
11
|
+
case .medium:
|
|
12
|
+
UIImpactFeedbackGenerator.FeedbackStyle.medium
|
|
13
|
+
case .heavy:
|
|
14
|
+
UIImpactFeedbackGenerator.FeedbackStyle.heavy
|
|
15
|
+
}
|
|
16
|
+
self.impactGenerator = UIImpactFeedbackGenerator(style: hapticStyle)
|
|
17
|
+
}
|
|
18
|
+
|
|
19
|
+
func trigger() {
|
|
20
|
+
impactGenerator.prepare()
|
|
21
|
+
impactGenerator.impactOccurred()
|
|
22
|
+
}
|
|
23
|
+
}
|
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
import Foundation
|
|
2
2
|
import Speech
|
|
3
3
|
import NitroModules
|
|
4
|
+
import os.log
|
|
4
5
|
|
|
5
6
|
class HybridRecognizer: HybridRecognizerSpec {
|
|
7
|
+
private let logger = Logger(subsystem: "com.margelo.nitro.nitrospeech", category: "AutoStopper")
|
|
6
8
|
private static let defaultAutoFinishRecognitionMs = 8000.0
|
|
7
9
|
|
|
8
10
|
var onReadyForSpeech: (() -> Void)?
|
|
@@ -19,10 +21,11 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
19
21
|
private var appStateObserver: AppStateObserver?
|
|
20
22
|
private var isActive: Bool = false
|
|
21
23
|
private var isStopping: Bool = false
|
|
24
|
+
private var config: SpeechToTextParams?
|
|
22
25
|
|
|
23
26
|
func startListening(params: SpeechToTextParams) {
|
|
24
27
|
if isActive {
|
|
25
|
-
|
|
28
|
+
// Previous recognition session is still active
|
|
26
29
|
return
|
|
27
30
|
}
|
|
28
31
|
|
|
@@ -30,9 +33,11 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
30
33
|
DispatchQueue.main.async {
|
|
31
34
|
guard let self = self else { return }
|
|
32
35
|
|
|
36
|
+
self.config = params
|
|
37
|
+
|
|
33
38
|
switch authStatus {
|
|
34
39
|
case .authorized:
|
|
35
|
-
self.requestMicrophonePermission(
|
|
40
|
+
self.requestMicrophonePermission()
|
|
36
41
|
case .denied, .restricted:
|
|
37
42
|
self.onPermissionDenied?()
|
|
38
43
|
case .notDetermined:
|
|
@@ -48,6 +53,10 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
48
53
|
guard isActive, !isStopping else { return }
|
|
49
54
|
isStopping = true
|
|
50
55
|
|
|
56
|
+
if let hapticStyle = config?.stopHapticFeedbackStyle {
|
|
57
|
+
HapticImpact(style: hapticStyle).trigger()
|
|
58
|
+
}
|
|
59
|
+
|
|
51
60
|
// Signal end of audio and request graceful finish
|
|
52
61
|
recognitionRequest?.endAudio()
|
|
53
62
|
recognitionTask?.finish()
|
|
@@ -79,13 +88,13 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
79
88
|
stopListening()
|
|
80
89
|
}
|
|
81
90
|
|
|
82
|
-
private func requestMicrophonePermission(
|
|
91
|
+
private func requestMicrophonePermission() {
|
|
83
92
|
AVAudioSession.sharedInstance().requestRecordPermission { [weak self] granted in
|
|
84
93
|
DispatchQueue.main.async {
|
|
85
94
|
guard let self = self else { return }
|
|
86
95
|
|
|
87
96
|
if granted {
|
|
88
|
-
self.startRecognition(
|
|
97
|
+
self.startRecognition()
|
|
89
98
|
} else {
|
|
90
99
|
self.onPermissionDenied?()
|
|
91
100
|
}
|
|
@@ -93,17 +102,17 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
93
102
|
}
|
|
94
103
|
}
|
|
95
104
|
|
|
96
|
-
private func startRecognition(
|
|
105
|
+
private func startRecognition() {
|
|
97
106
|
isStopping = false
|
|
98
107
|
|
|
99
|
-
let locale = Locale(identifier:
|
|
108
|
+
let locale = Locale(identifier: config?.locale ?? "en-US")
|
|
100
109
|
guard let speechRecognizer = SFSpeechRecognizer(locale: locale), speechRecognizer.isAvailable else {
|
|
101
110
|
onError?("Speech recognizer not available")
|
|
102
111
|
return
|
|
103
112
|
}
|
|
104
113
|
|
|
105
114
|
autoStopper = AutoStopper(
|
|
106
|
-
silenceThresholdMs:
|
|
115
|
+
silenceThresholdMs: config?.autoFinishRecognitionMs ?? Self.defaultAutoFinishRecognitionMs,
|
|
107
116
|
onProgress: { [weak self] timeLeftMs in
|
|
108
117
|
self?.onAutoFinishProgress?(timeLeftMs)
|
|
109
118
|
},
|
|
@@ -115,6 +124,10 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
115
124
|
do {
|
|
116
125
|
let audioSession = AVAudioSession.sharedInstance()
|
|
117
126
|
try audioSession.setCategory(.record, mode: .measurement, options: .duckOthers)
|
|
127
|
+
if #available(iOS 13.0, *) {
|
|
128
|
+
// Without this, iOS may suppress haptics while recording.
|
|
129
|
+
try audioSession.setAllowHapticsAndSystemSoundsDuringRecording(true)
|
|
130
|
+
}
|
|
118
131
|
try audioSession.setActive(true, options: .notifyOthersOnDeactivation)
|
|
119
132
|
} catch {
|
|
120
133
|
onError?("Failed to set up audio session: \(error.localizedDescription)")
|
|
@@ -131,19 +144,19 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
131
144
|
|
|
132
145
|
recognitionRequest.shouldReportPartialResults = true
|
|
133
146
|
|
|
134
|
-
if let contextualStrings =
|
|
147
|
+
if let contextualStrings = config?.contextualStrings, !contextualStrings.isEmpty {
|
|
135
148
|
recognitionRequest.contextualStrings = contextualStrings
|
|
136
149
|
}
|
|
137
150
|
|
|
138
151
|
if #available(iOS 16, *) {
|
|
139
|
-
if let addPunctiation =
|
|
152
|
+
if let addPunctiation = config?.iosAddPunctuation, addPunctiation == false {
|
|
140
153
|
recognitionRequest.addsPunctuation = false
|
|
141
154
|
} else {
|
|
142
155
|
recognitionRequest.addsPunctuation = true
|
|
143
156
|
}
|
|
144
157
|
}
|
|
145
158
|
|
|
146
|
-
let disableRepeatingFilter =
|
|
159
|
+
let disableRepeatingFilter = config?.disableRepeatingFilter ?? false
|
|
147
160
|
|
|
148
161
|
recognitionTask = speechRecognizer.recognitionTask(with: recognitionRequest) { [weak self] result, error in
|
|
149
162
|
guard let self = self else { return }
|
|
@@ -197,6 +210,11 @@ class HybridRecognizer: HybridRecognizerSpec {
|
|
|
197
210
|
audioEngine.prepare()
|
|
198
211
|
try audioEngine.start()
|
|
199
212
|
isActive = true
|
|
213
|
+
|
|
214
|
+
if let hapticStyle = config?.startHapticFeedbackStyle {
|
|
215
|
+
HapticImpact(style: hapticStyle).trigger()
|
|
216
|
+
}
|
|
217
|
+
|
|
200
218
|
autoStopper?.indicateRecordingActivity(
|
|
201
219
|
from: "startListening",
|
|
202
220
|
addMsToThreshold: nil
|
package/lib/commonjs/index.js
CHANGED
|
@@ -3,66 +3,78 @@
|
|
|
3
3
|
Object.defineProperty(exports, "__esModule", {
|
|
4
4
|
value: true
|
|
5
5
|
});
|
|
6
|
-
exports.useRecognizer = exports.
|
|
6
|
+
exports.useRecognizer = exports.RecognizerSession = exports.RecognizerRef = void 0;
|
|
7
7
|
var _react = _interopRequireDefault(require("react"));
|
|
8
8
|
var _reactNativeNitroModules = require("react-native-nitro-modules");
|
|
9
9
|
function _interopRequireDefault(e) { return e && e.__esModule ? e : { default: e }; }
|
|
10
|
+
/* eslint-disable react-hooks/exhaustive-deps */
|
|
11
|
+
|
|
10
12
|
const NitroSpeech = _reactNativeNitroModules.NitroModules.createHybridObject('NitroSpeech');
|
|
11
13
|
|
|
12
14
|
/**
|
|
13
|
-
* Unsafe access to the
|
|
15
|
+
* Unsafe access to the Recognizer Session.
|
|
14
16
|
*/
|
|
15
|
-
const
|
|
17
|
+
const RecognizerSession = exports.RecognizerSession = NitroSpeech.recognizer;
|
|
16
18
|
const recognizerStartListening = params => {
|
|
17
|
-
|
|
19
|
+
RecognizerSession.startListening(params);
|
|
18
20
|
};
|
|
19
21
|
const recognizerStopListening = () => {
|
|
20
|
-
|
|
22
|
+
RecognizerSession.stopListening();
|
|
21
23
|
};
|
|
22
24
|
const recognizerAddAutoFinishTime = additionalTimeMs => {
|
|
23
|
-
|
|
25
|
+
RecognizerSession.addAutoFinishTime(additionalTimeMs);
|
|
24
26
|
};
|
|
25
27
|
const recognizerUpdateAutoFinishTime = (newTimeMs, withRefresh) => {
|
|
26
|
-
|
|
28
|
+
RecognizerSession.updateAutoFinishTime(newTimeMs, withRefresh);
|
|
27
29
|
};
|
|
28
30
|
|
|
29
31
|
/**
|
|
30
32
|
* Safe, lifecycle-aware hook to use the recognizer.
|
|
33
|
+
*
|
|
34
|
+
* @param callbacks - The callbacks to use for the recognizer.
|
|
35
|
+
* @param destroyDeps - The additional dependencies to use for the cleanup effect.
|
|
36
|
+
*
|
|
37
|
+
* Example: To cleanup when the screen is unfocused.
|
|
38
|
+
*
|
|
39
|
+
* ```typescript
|
|
40
|
+
* const isFocused = useIsFocused()
|
|
41
|
+
* useRecognizer({ ... }, [isFocused])
|
|
42
|
+
* ```
|
|
31
43
|
*/
|
|
32
|
-
const useRecognizer = callbacks => {
|
|
44
|
+
const useRecognizer = (callbacks, destroyDeps = []) => {
|
|
33
45
|
_react.default.useEffect(() => {
|
|
34
|
-
|
|
46
|
+
RecognizerSession.onReadyForSpeech = () => {
|
|
35
47
|
callbacks.onReadyForSpeech?.();
|
|
36
48
|
};
|
|
37
|
-
|
|
49
|
+
RecognizerSession.onRecordingStopped = () => {
|
|
38
50
|
callbacks.onRecordingStopped?.();
|
|
39
51
|
};
|
|
40
|
-
|
|
52
|
+
RecognizerSession.onResult = resultBatches => {
|
|
41
53
|
callbacks.onResult?.(resultBatches);
|
|
42
54
|
};
|
|
43
|
-
|
|
55
|
+
RecognizerSession.onAutoFinishProgress = timeLeftMs => {
|
|
44
56
|
callbacks.onAutoFinishProgress?.(timeLeftMs);
|
|
45
57
|
};
|
|
46
|
-
|
|
58
|
+
RecognizerSession.onError = message => {
|
|
47
59
|
callbacks.onError?.(message);
|
|
48
60
|
};
|
|
49
|
-
|
|
61
|
+
RecognizerSession.onPermissionDenied = () => {
|
|
50
62
|
callbacks.onPermissionDenied?.();
|
|
51
63
|
};
|
|
52
64
|
return () => {
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
65
|
+
RecognizerSession.onReadyForSpeech = undefined;
|
|
66
|
+
RecognizerSession.onRecordingStopped = undefined;
|
|
67
|
+
RecognizerSession.onResult = undefined;
|
|
68
|
+
RecognizerSession.onAutoFinishProgress = undefined;
|
|
69
|
+
RecognizerSession.onError = undefined;
|
|
70
|
+
RecognizerSession.onPermissionDenied = undefined;
|
|
59
71
|
};
|
|
60
72
|
}, [callbacks]);
|
|
61
73
|
_react.default.useEffect(() => {
|
|
62
74
|
return () => {
|
|
63
|
-
|
|
75
|
+
RecognizerSession.stopListening();
|
|
64
76
|
};
|
|
65
|
-
}, []);
|
|
77
|
+
}, [...destroyDeps]);
|
|
66
78
|
return {
|
|
67
79
|
startListening: recognizerStartListening,
|
|
68
80
|
stopListening: recognizerStopListening,
|
|
@@ -70,5 +82,15 @@ const useRecognizer = callbacks => {
|
|
|
70
82
|
updateAutoFinishTime: recognizerUpdateAutoFinishTime
|
|
71
83
|
};
|
|
72
84
|
};
|
|
85
|
+
|
|
86
|
+
/**
|
|
87
|
+
* Safe reference to the Recognizer methods.
|
|
88
|
+
*/
|
|
73
89
|
exports.useRecognizer = useRecognizer;
|
|
90
|
+
const RecognizerRef = exports.RecognizerRef = {
|
|
91
|
+
startListening: recognizerStartListening,
|
|
92
|
+
stopListening: recognizerStopListening,
|
|
93
|
+
addAutoFinishTime: recognizerAddAutoFinishTime,
|
|
94
|
+
updateAutoFinishTime: recognizerUpdateAutoFinishTime
|
|
95
|
+
};
|
|
74
96
|
//# sourceMappingURL=index.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"names":["_react","_interopRequireDefault","require","_reactNativeNitroModules","e","__esModule","default","NitroSpeech","NitroModules","createHybridObject","
|
|
1
|
+
{"version":3,"names":["_react","_interopRequireDefault","require","_reactNativeNitroModules","e","__esModule","default","NitroSpeech","NitroModules","createHybridObject","RecognizerSession","exports","recognizer","recognizerStartListening","params","startListening","recognizerStopListening","stopListening","recognizerAddAutoFinishTime","additionalTimeMs","addAutoFinishTime","recognizerUpdateAutoFinishTime","newTimeMs","withRefresh","updateAutoFinishTime","useRecognizer","callbacks","destroyDeps","React","useEffect","onReadyForSpeech","onRecordingStopped","onResult","resultBatches","onAutoFinishProgress","timeLeftMs","onError","message","onPermissionDenied","undefined","RecognizerRef"],"sourceRoot":"../../src","sources":["index.ts"],"mappings":";;;;;;AACA,IAAAA,MAAA,GAAAC,sBAAA,CAAAC,OAAA;AACA,IAAAC,wBAAA,GAAAD,OAAA;AAAyD,SAAAD,uBAAAG,CAAA,WAAAA,CAAA,IAAAA,CAAA,CAAAC,UAAA,GAAAD,CAAA,KAAAE,OAAA,EAAAF,CAAA;AAFzD;;AASA,MAAMG,WAAW,GACfC,qCAAY,CAACC,kBAAkB,CAAkB,aAAa,CAAC;;AAEjE;AACA;AACA;AACO,MAAMC,iBAAiB,GAAAC,OAAA,CAAAD,iBAAA,GAAGH,WAAW,CAACK,UAAU;AAoBvD,MAAMC,wBAAwB,GAAIC,MAA0B,IAAK;EAC/DJ,iBAAiB,CAACK,cAAc,CAACD,MAAM,CAAC;AAC1C,CAAC;AAED,MAAME,uBAAuB,GAAGA,CAAA,KAAM;EACpCN,iBAAiB,CAACO,aAAa,CAAC,CAAC;AACnC,CAAC;AAED,MAAMC,2BAA2B,GAAIC,gBAAyB,IAAK;EACjET,iBAAiB,CAACU,iBAAiB,CAACD,gBAAgB,CAAC;AACvD,CAAC;AAED,MAAME,8BAA8B,GAAGA,CACrCC,SAAiB,EACjBC,WAAqB,KAClB;EACHb,iBAAiB,CAACc,oBAAoB,CAACF,SAAS,EAAEC,WAAW,CAAC;AAChE,CAAC;;AAED;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACA;AACO,MAAME,aAAa,GAAGA,CAC3BC,SAA8B,EAC9BC,WAAiC,GAAG,EAAE,KACf;EACvBC,cAAK,CAACC,SAAS,CAAC,MAAM;IACpBnB,iBAAiB,CAACoB,gBAAgB,GAAG,MAAM;MACzCJ,SAAS,CAACI,gBAAgB,GAAG,CAAC;IAChC,CAAC;IACDpB,iBAAiB,CAACqB,kBAAkB,GAAG,MAAM;MAC3CL,SAAS,CAACK,kBAAkB,GAAG,CAAC;IAClC,CAAC;IACDrB,iBAAiB,CAACsB,QAAQ,GAAIC,aAAuB,IAAK;MACxDP,SAAS,CAACM,QAAQ,GAAGC,aAAa,CAAC;IACrC,CAAC;IACDvB,iBAAiB,CAACwB,oBAAoB,GAAIC,UAAkB,IAAK;MAC/DT,SAAS,CAACQ,oBAAoB,GAAGC,UAAU,CAAC;IAC9C,CAAC;IACDzB,iBAAiB,CAAC0B,OAAO,GAAIC,OAAe,IAAK;MAC/CX,SAAS,CAACU,OAAO,GAAGC,OAAO,CAAC;IAC9B,CAAC;IACD3B,iBAAiB,CAAC4B,kBAAkB,GAAG,MAAM;MAC3CZ,SAAS,CAACY,kBAAkB,GAAG,CAAC;IAClC,CAAC;IACD,OAAO,MAAM;MACX5B,iBAAiB,CAACoB,gBAAgB,GAAGS,SAAS;MAC9C7B,iBAAiB,CAACqB,kBAAkB,GAAGQ,SAAS;MAChD7B,iBAAiB,CAACsB,QAAQ,GAAGO,SAAS;MACtC7B,iBAAiB,CAACwB,oBAAoB,GAAGK,SAAS;MAClD7B,iBAAiB,CAAC0B,OAAO,GAAGG,SAAS;MACrC7B,iBAAiB,CAAC4B,kBAAkB,GAAGC,SAAS;IAClD,CAAC;EACH,CAAC,EAAE,CAACb,SAAS,CAAC,CAAC;EAEfE,cAAK,CAACC,SAAS,CAAC,MAAM;IACpB,OAAO,MAAM;MACXnB,iBAAiB,CAACO,aAAa,CAAC,CAAC;IACnC,CAAC;EACH,CAAC,EAAE,CAAC,GAAGU,WAAW,CAAC,CAAC;EAEpB,OAAO;IACLZ,cAAc,EAAEF,wBAAwB;IACxCI,aAAa,EAAED,uBAAuB;IACtCI,iBAAiB,EAAEF,2BAA2B;IAC9CM,oBAAoB,EAAEH;EACxB,CAAC;AACH,CAAC;;AAED;AACA;AACA;AAFAV,OAAA,CAAAc,aAAA,GAAAA,aAAA;AAGO,MAAMe,aAAa,GAAA7B,OAAA,CAAA6B,aAAA,GAAG;EAC3BzB,cAAc,EAAEF,wBAAwB;EACxCI,aAAa,EAAED,uBAAuB;EACtCI,iBAAiB,EAAEF,2BAA2B;EAC9CM,oBAAoB,EAAEH;AACxB,CAAC","ignoreList":[]}
|