apple-notes-mcp 2.9.10 → 2.9.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +34 -3
- package/build/index.js +472 -35
- package/native/public-helper/apple-notes-public-helper.swift +331 -5
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1857,6 +1857,37 @@ Decodes a note's classic PencilKit drawings (`com.apple.drawing.2` and the older
|
|
|
1857
1857
|
|
|
1858
1858
|
---
|
|
1859
1859
|
|
|
1860
|
+
#### `transcribe-note-audio`
|
|
1861
|
+
|
|
1862
|
+
Transcribes a note's voice recordings and audio attachments now, on this Mac, with Apple's Speech framework through the [public native helper](#public-native-helper). Recognition is on-device only: `SpeechAnalyzer` on macOS 26 and later, or `SFSpeechRecognizer` with on-device recognition required on older systems (a locale without on-device support is refused, never sent to a server). The audio files are opened read-only where Notes keeps them.
|
|
1863
|
+
|
|
1864
|
+
**Requires:** Full Disk Access for the MCP host process, and the public native helper built once with `apple-notes-mcp setup --public-helper`.
|
|
1865
|
+
|
|
1866
|
+
| Parameter | Type | Required | Description |
|
|
1867
|
+
|-----------|------|----------|-------------|
|
|
1868
|
+
| `id` | string | Yes | Note ID (use `search-notes` to find it first) |
|
|
1869
|
+
| `locale` | string | No | BCP-47 language of the speech, such as `"en-US"` (default), `"it-IT"`, or `"fr-FR"` |
|
|
1870
|
+
| `attachmentId` | string | No | Only transcribe this audio attachment (an `x-coredata://…/ICAttachment/pN` id) |
|
|
1871
|
+
| `includeText` | boolean | No | Include transcript text (default `true`); `false` returns statuses and word counts only |
|
|
1872
|
+
| `downloadAssets` | boolean | No | Let macOS download the language's on-device speech model when it is missing (default `false`: the call returns `asset_unavailable` at once) |
|
|
1873
|
+
| `maxSeconds` | integer | No | Total time budget for the call, 30 to 3600 seconds (default 900) |
|
|
1874
|
+
|
|
1875
|
+
**Returns:** Overall `status` and, per audio attachment, `status`, `durationSeconds`, `wordCount`, `transcript`, and `takes` (a Notes recording can hold several takes; each is transcribed and the texts are joined in stored order). Statuses:
|
|
1876
|
+
|
|
1877
|
+
- `ok`: the whole recording was transcribed.
|
|
1878
|
+
- `partial`: some text came back, but a take failed or the helper stopped at its deadline (`code: "incomplete"`).
|
|
1879
|
+
- `error`: nothing was transcribed; `code` says why (`asset_unavailable` when the audio file is not on this Mac or the language's speech model is not installed, `permission_required`, `unsupported_locale`, `unsupported_audio`, `time_limit`, ...).
|
|
1880
|
+
- `indeterminate`: the helper did not answer in time, so the outcome is unknown and a retry may succeed.
|
|
1881
|
+
- `none` (overall only): the note has no audio.
|
|
1882
|
+
|
|
1883
|
+
Each take gets a deadline of 1.5 times its length plus a minute (at most 30 minutes), shortened to what is left of `maxSeconds`. A take that cannot start before the budget runs out reports `code: "time_limit"`. The helper runs as a separate process without blocking the server, and cancelling the request stops it. On macOS 27 a 16-minute recording took about 22 seconds. Some MCP clients stop waiting for a tool after a fixed time, so transcribe long recordings one at a time with `attachmentId`. Transcripts longer than `APPLE_NOTES_MCP_EXPORT_MAX_BYTES` are shortened and marked `transcriptTruncated`.
|
|
1884
|
+
|
|
1885
|
+
**Speech models:** the call never starts a download on its own. When the language's on-device model is not installed, it returns `asset_unavailable` right away. Pass `downloadAssets: true` to let macOS download it (a one-time download); if it is still downloading when the call ends, try again shortly.
|
|
1886
|
+
|
|
1887
|
+
**Speech Recognition permission:** the server never shows the Speech Recognition prompt, because nobody may be watching an MCP server to answer it. The helper reads the current authorization first. On older macOS, the `SFSpeechRecognizer` path needs that access, and macOS attributes the grant to the app that launches the MCP server (Claude Desktop, Codex, Terminal, and so on), not to the helper. Without it the call returns `code: "permission_required"`; allow the app under System Settings > Privacy & Security > Speech Recognition. On macOS 26 and later, `SpeechAnalyzer` transcribed files without any grant in testing (authorization stayed "not determined"), so only an explicit refusal (denied or restricted) stops it.
|
|
1888
|
+
|
|
1889
|
+
---
|
|
1890
|
+
|
|
1860
1891
|
#### `add-attachment`
|
|
1861
1892
|
|
|
1862
1893
|
Adds one nonempty local file of at most 64 MiB to an exact note using `id`, the
|
|
@@ -2496,13 +2527,13 @@ Every tool that does not read the Notes database works normally without Full Dis
|
|
|
2496
2527
|
- `get-note-link` returns an error on macOS 26+; on macOS 12–15 it still works via the AppleScript `note link` fallback
|
|
2497
2528
|
- `get-note-markdown` returns plain list items without `[x]`/`[ ]` annotations (graceful fallback)
|
|
2498
2529
|
- `get-sync-status` still answers, but reports no pending uploads and no active sync — treat that as "unknown", not "idle"
|
|
2499
|
-
- `get-note-drawings`
|
|
2530
|
+
- `get-note-drawings` and `transcribe-note-audio` return the same Full Disk Access error
|
|
2500
2531
|
|
|
2501
2532
|
---
|
|
2502
2533
|
|
|
2503
2534
|
## Public native helper
|
|
2504
2535
|
|
|
2505
|
-
`get-note-drawings` needs Apple's PencilKit framework
|
|
2536
|
+
`get-note-drawings` needs Apple's PencilKit framework and `transcribe-note-audio` needs the Speech framework; neither has an AppleScript or command-line interface. For them, the server uses a small Swift helper that links public Apple frameworks only (AppKit, PencilKit, AVFoundation, and Speech). No prebuilt binary ships with the package. Build it once on your Mac:
|
|
2506
2537
|
|
|
2507
2538
|
```bash
|
|
2508
2539
|
apple-notes-mcp setup --public-helper # compile, sign, verify, install
|
|
@@ -2511,7 +2542,7 @@ apple-notes-mcp setup --public-helper --check # report the installed state only
|
|
|
2511
2542
|
|
|
2512
2543
|
Setup compiles `native/public-helper/apple-notes-public-helper.swift` with `xcrun swiftc` (install the Command Line Tools with `xcode-select --install` if it is missing), signs it ad hoc, runs its `hello` handshake, and installs it in `~/Library/Application Support/apple-notes-mcp/public-helper/` next to a manifest recording the SHA-256 of the source and the binary. Before every use the server re-checks both digests: after an upgrade that changes the helper source, or if the binary is replaced, the helper is refused until you run setup again. `APPLE_NOTES_MCP_PUBLIC_HELPER_DIR` overrides the install folder and `APPLE_NOTES_MCP_PUBLIC_HELPER_TIMEOUT_MS` the per-call timeout.
|
|
2513
2544
|
|
|
2514
|
-
The helper never opens the Notes database and never writes under the Notes group container. The server reads the bytes it needs (read-only) and passes them to the helper on stdin; the helper answers with one JSON object on stdout.
|
|
2545
|
+
The helper never opens the Notes database and never writes under the Notes group container. The server reads the bytes it needs (read-only) and passes them to the helper on stdin, or, for transcription, names one audio file that the helper opens for reading; the helper answers with one JSON object on stdout.
|
|
2515
2546
|
|
|
2516
2547
|
---
|
|
2517
2548
|
|
package/build/index.js
CHANGED
|
@@ -24465,14 +24465,14 @@ var require_turndown_cjs = __commonJS({
|
|
|
24465
24465
|
} else if (node.nodeType === 1) {
|
|
24466
24466
|
replacement = replacementForNode.call(self, node);
|
|
24467
24467
|
}
|
|
24468
|
-
return
|
|
24468
|
+
return join28(output, replacement);
|
|
24469
24469
|
}, "");
|
|
24470
24470
|
}
|
|
24471
24471
|
function postProcess(output) {
|
|
24472
24472
|
var self = this;
|
|
24473
24473
|
this.rules.forEach(function(rule) {
|
|
24474
24474
|
if (typeof rule.append === "function") {
|
|
24475
|
-
output =
|
|
24475
|
+
output = join28(output, rule.append(self.options));
|
|
24476
24476
|
}
|
|
24477
24477
|
});
|
|
24478
24478
|
return output.replace(/^[\t\r\n]+/, "").replace(/[\t\r\n\s]+$/, "");
|
|
@@ -24484,7 +24484,7 @@ var require_turndown_cjs = __commonJS({
|
|
|
24484
24484
|
if (whitespace.leading || whitespace.trailing) content = content.trim();
|
|
24485
24485
|
return whitespace.leading + rule.replacement(content, node, this.options) + whitespace.trailing;
|
|
24486
24486
|
}
|
|
24487
|
-
function
|
|
24487
|
+
function join28(output, replacement) {
|
|
24488
24488
|
var s1 = trimTrailingNewlines(output);
|
|
24489
24489
|
var s2 = trimLeadingNewlines(replacement);
|
|
24490
24490
|
var nls = Math.max(output.length - s1.length, replacement.length - s2.length);
|
|
@@ -54680,7 +54680,7 @@ function formatShortcutSetup(report) {
|
|
|
54680
54680
|
}
|
|
54681
54681
|
|
|
54682
54682
|
// src/services/publicHelper.ts
|
|
54683
|
-
import { spawnSync as spawnSync3 } from "node:child_process";
|
|
54683
|
+
import { spawn, spawnSync as spawnSync3 } from "node:child_process";
|
|
54684
54684
|
import { createHash as createHash5 } from "node:crypto";
|
|
54685
54685
|
import {
|
|
54686
54686
|
chmodSync,
|
|
@@ -54702,7 +54702,11 @@ var PUBLIC_HELPER_BINARY = "apple-notes-public-helper";
|
|
|
54702
54702
|
var PUBLIC_HELPER_SOURCE = "native/public-helper/apple-notes-public-helper.swift";
|
|
54703
54703
|
var PUBLIC_HELPER_MANIFEST = "manifest.json";
|
|
54704
54704
|
var PUBLIC_HELPER_SETUP_COMMAND = "apple-notes-mcp setup --public-helper";
|
|
54705
|
-
var PUBLIC_HELPER_ACTIONS = /* @__PURE__ */ new Set([
|
|
54705
|
+
var PUBLIC_HELPER_ACTIONS = /* @__PURE__ */ new Set([
|
|
54706
|
+
"hello",
|
|
54707
|
+
"decode_drawing",
|
|
54708
|
+
"transcribe"
|
|
54709
|
+
]);
|
|
54706
54710
|
var DEFAULT_TIMEOUT_MS2 = 3e4;
|
|
54707
54711
|
var MAX_OUTPUT_BYTES = 256 * 1024 * 1024;
|
|
54708
54712
|
var publicManifestSchema = external_exports.object({
|
|
@@ -54738,6 +54742,7 @@ function defaultPublicHelperDeps(overrides = {}) {
|
|
|
54738
54742
|
exists: existsSync14,
|
|
54739
54743
|
readFile: (path10) => readFileSync4(path10),
|
|
54740
54744
|
spawn: spawnSync3,
|
|
54745
|
+
spawnAsync: spawn,
|
|
54741
54746
|
...overrides
|
|
54742
54747
|
};
|
|
54743
54748
|
}
|
|
@@ -54802,10 +54807,14 @@ var UNAVAILABLE_CODES = /* @__PURE__ */ new Set([
|
|
|
54802
54807
|
"helper_modified",
|
|
54803
54808
|
"helper_manifest_invalid"
|
|
54804
54809
|
]);
|
|
54810
|
+
var ENVELOPE_CODES = {
|
|
54811
|
+
invalid_request: "validation_error",
|
|
54812
|
+
attachment_not_found: "not_found"
|
|
54813
|
+
};
|
|
54805
54814
|
var PublicHelperError = class extends CodedError {
|
|
54806
54815
|
constructor(code, message) {
|
|
54807
54816
|
super(message, {
|
|
54808
|
-
code: UNAVAILABLE_CODES.has(code) ? "unsupported" : "operation_failed",
|
|
54817
|
+
code: UNAVAILABLE_CODES.has(code) ? "unsupported" : ENVELOPE_CODES[code] ?? "operation_failed",
|
|
54809
54818
|
helperCode: code
|
|
54810
54819
|
});
|
|
54811
54820
|
this.code = code;
|
|
@@ -54821,21 +54830,9 @@ var publicHelloSchema = external_exports.object({
|
|
|
54821
54830
|
actions: external_exports.array(external_exports.string())
|
|
54822
54831
|
}).passthrough();
|
|
54823
54832
|
function callPublicHelper(action, fields = {}, deps = defaultPublicHelperDeps(), options = {}) {
|
|
54824
|
-
|
|
54825
|
-
throw new PublicHelperError(
|
|
54826
|
-
"unknown_action",
|
|
54827
|
-
`"${action}" is not an action the server sends to the public helper.`
|
|
54828
|
-
);
|
|
54829
|
-
let binaryPath = options.binaryPath;
|
|
54830
|
-
if (!binaryPath) {
|
|
54831
|
-
const install = inspectPublicHelper(deps);
|
|
54832
|
-
if (!install.ready)
|
|
54833
|
-
throw new PublicHelperError(install.reason ?? "helper_not_installed", install.detail ?? "");
|
|
54834
|
-
binaryPath = install.binaryPath;
|
|
54835
|
-
}
|
|
54836
|
-
const timeout = Number.parseInt(deps.env[PUBLIC_HELPER_TIMEOUT_ENV] || "", 10) || options.timeoutMs || DEFAULT_TIMEOUT_MS2;
|
|
54833
|
+
const { binaryPath, timeout, input } = prepareCall(action, fields, deps, options);
|
|
54837
54834
|
const result = deps.spawn(binaryPath, [], {
|
|
54838
|
-
input
|
|
54835
|
+
input,
|
|
54839
54836
|
encoding: "utf8",
|
|
54840
54837
|
timeout,
|
|
54841
54838
|
killSignal: "SIGKILL",
|
|
@@ -54849,24 +54846,100 @@ function callPublicHelper(action, fields = {}, deps = defaultPublicHelperDeps(),
|
|
|
54849
54846
|
"helper_unreachable",
|
|
54850
54847
|
`Could not run the helper: ${result.error.message}`
|
|
54851
54848
|
);
|
|
54849
|
+
return parseHelperOutput(result.status, String(result.stdout ?? ""));
|
|
54850
|
+
}
|
|
54851
|
+
async function callPublicHelperAsync(action, fields = {}, deps = defaultPublicHelperDeps(), options = {}) {
|
|
54852
|
+
const { binaryPath, timeout, input } = prepareCall(action, fields, deps, options);
|
|
54853
|
+
const { signal } = options;
|
|
54854
|
+
const aborted2 = () => new PublicHelperError("aborted", "The request was cancelled; the helper was stopped.");
|
|
54855
|
+
if (signal?.aborted) throw aborted2();
|
|
54856
|
+
return new Promise((resolvePromise, reject) => {
|
|
54857
|
+
const child = (deps.spawnAsync ?? spawn)(binaryPath, [], {
|
|
54858
|
+
stdio: ["pipe", "pipe", "ignore"]
|
|
54859
|
+
});
|
|
54860
|
+
const chunks = [];
|
|
54861
|
+
let size = 0;
|
|
54862
|
+
let failure = null;
|
|
54863
|
+
const stop = (error2) => {
|
|
54864
|
+
failure ??= error2;
|
|
54865
|
+
child.kill("SIGKILL");
|
|
54866
|
+
};
|
|
54867
|
+
const timer = setTimeout(
|
|
54868
|
+
() => stop(new PublicHelperError("timeout", `The helper did not answer within ${timeout} ms.`)),
|
|
54869
|
+
timeout
|
|
54870
|
+
);
|
|
54871
|
+
const onAbort = () => stop(aborted2());
|
|
54872
|
+
signal?.addEventListener("abort", onAbort, { once: true });
|
|
54873
|
+
const settle = () => {
|
|
54874
|
+
clearTimeout(timer);
|
|
54875
|
+
signal?.removeEventListener("abort", onAbort);
|
|
54876
|
+
};
|
|
54877
|
+
child.stdout?.on("data", (chunk) => {
|
|
54878
|
+
size += chunk.length;
|
|
54879
|
+
if (size > MAX_OUTPUT_BYTES)
|
|
54880
|
+
stop(new PublicHelperError("invalid_response", "The helper response is too large."));
|
|
54881
|
+
else chunks.push(chunk);
|
|
54882
|
+
});
|
|
54883
|
+
child.on("error", (error2) => {
|
|
54884
|
+
settle();
|
|
54885
|
+
reject(
|
|
54886
|
+
failure ?? new PublicHelperError("helper_unreachable", `Could not run the helper: ${error2.message}`)
|
|
54887
|
+
);
|
|
54888
|
+
});
|
|
54889
|
+
child.on("close", (status, exitSignal) => {
|
|
54890
|
+
settle();
|
|
54891
|
+
if (failure) return reject(failure);
|
|
54892
|
+
if (status === null && exitSignal)
|
|
54893
|
+
return reject(
|
|
54894
|
+
new PublicHelperError("helper_crashed", `The helper stopped on signal ${exitSignal}.`)
|
|
54895
|
+
);
|
|
54896
|
+
try {
|
|
54897
|
+
resolvePromise(parseHelperOutput(status, Buffer.concat(chunks).toString("utf8")));
|
|
54898
|
+
} catch (error2) {
|
|
54899
|
+
reject(error2);
|
|
54900
|
+
}
|
|
54901
|
+
});
|
|
54902
|
+
child.stdin?.on("error", () => {
|
|
54903
|
+
});
|
|
54904
|
+
child.stdin?.end(input);
|
|
54905
|
+
});
|
|
54906
|
+
}
|
|
54907
|
+
function prepareCall(action, fields, deps, options) {
|
|
54908
|
+
if (!PUBLIC_HELPER_ACTIONS.has(action))
|
|
54909
|
+
throw new PublicHelperError(
|
|
54910
|
+
"unknown_action",
|
|
54911
|
+
`"${action}" is not an action the server sends to the public helper.`
|
|
54912
|
+
);
|
|
54913
|
+
let binaryPath = options.binaryPath;
|
|
54914
|
+
if (!binaryPath) {
|
|
54915
|
+
const install = inspectPublicHelper(deps);
|
|
54916
|
+
if (!install.ready)
|
|
54917
|
+
throw new PublicHelperError(install.reason ?? "helper_not_installed", install.detail ?? "");
|
|
54918
|
+
binaryPath = install.binaryPath;
|
|
54919
|
+
}
|
|
54920
|
+
const timeout = options.timeoutMs || Number.parseInt(deps.env[PUBLIC_HELPER_TIMEOUT_ENV] || "", 10) || DEFAULT_TIMEOUT_MS2;
|
|
54921
|
+
const input = JSON.stringify({ protocol: PUBLIC_HELPER_PROTOCOL, action, ...fields });
|
|
54922
|
+
return { binaryPath, timeout, input };
|
|
54923
|
+
}
|
|
54924
|
+
function parseHelperOutput(status, stdout) {
|
|
54852
54925
|
let parsed;
|
|
54853
54926
|
try {
|
|
54854
|
-
parsed = JSON.parse(
|
|
54927
|
+
parsed = JSON.parse(stdout.trim());
|
|
54855
54928
|
} catch {
|
|
54856
54929
|
throw new PublicHelperError(
|
|
54857
54930
|
"invalid_response",
|
|
54858
|
-
`The helper exited with status ${
|
|
54931
|
+
`The helper exited with status ${status} and no JSON response.`
|
|
54859
54932
|
);
|
|
54860
54933
|
}
|
|
54861
54934
|
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed))
|
|
54862
54935
|
throw new PublicHelperError("invalid_response", "The helper response is not a JSON object.");
|
|
54863
54936
|
const object3 = parsed;
|
|
54864
|
-
if (
|
|
54937
|
+
if (status !== 0 || object3.status !== "ok") {
|
|
54865
54938
|
const error2 = errorSchema.safeParse(object3);
|
|
54866
54939
|
if (!error2.success)
|
|
54867
54940
|
throw new PublicHelperError(
|
|
54868
54941
|
"invalid_response",
|
|
54869
|
-
`The helper failed with an unrecognized response (exit ${
|
|
54942
|
+
`The helper failed with an unrecognized response (exit ${status}).`
|
|
54870
54943
|
);
|
|
54871
54944
|
throw new PublicHelperError(error2.data.code, error2.data.message);
|
|
54872
54945
|
}
|
|
@@ -54913,6 +54986,10 @@ function publicHelperCompileArguments(sourcePath, digestPath, plistPath, outputP
|
|
|
54913
54986
|
"AppKit",
|
|
54914
54987
|
"-framework",
|
|
54915
54988
|
"PencilKit",
|
|
54989
|
+
"-framework",
|
|
54990
|
+
"AVFoundation",
|
|
54991
|
+
"-framework",
|
|
54992
|
+
"Speech",
|
|
54916
54993
|
"-Xlinker",
|
|
54917
54994
|
"-sectcreate",
|
|
54918
54995
|
"-Xlinker",
|
|
@@ -55279,6 +55356,313 @@ function formatNoteDrawings(result) {
|
|
|
55279
55356
|
return lines.join("\n");
|
|
55280
55357
|
}
|
|
55281
55358
|
|
|
55359
|
+
// src/utils/noteAudio.ts
|
|
55360
|
+
import { readdirSync as readdirSync4, statSync as statSync4 } from "node:fs";
|
|
55361
|
+
import { join as join25 } from "node:path";
|
|
55362
|
+
var EXTRA_AUDIO_UTIS = [
|
|
55363
|
+
"public.mp3",
|
|
55364
|
+
"public.aiff-audio",
|
|
55365
|
+
"public.aifc-audio",
|
|
55366
|
+
"com.microsoft.waveform-audio"
|
|
55367
|
+
];
|
|
55368
|
+
function audioRowsSql(columns, generationColumn) {
|
|
55369
|
+
const generation = generationColumn ? `m.${generationColumn}` : "NULL";
|
|
55370
|
+
const media = (alias) => `json((SELECT json_object('identifier', m.ZIDENTIFIER, 'generation', ${generation}, 'filename', m.ZFILENAME) FROM ZICCLOUDSYNCINGOBJECT m WHERE m.Z_PK = ${alias}.ZMEDIA))`;
|
|
55371
|
+
const live = (alias) => notTombstonedSql(columns, alias);
|
|
55372
|
+
const utis = EXTRA_AUDIO_UTIS.map((u) => `'${u}'`).join(", ");
|
|
55373
|
+
const accountCols = [...columns].filter((c) => /^ZACCOUNT\d*$/.test(c)).sort();
|
|
55374
|
+
const account = accountCols.length ? `SELECT (SELECT acc.ZIDENTIFIER FROM ZICCLOUDSYNCINGOBJECT acc WHERE acc.Z_ENT = ${entity("ICAccount")} AND acc.Z_PK IN (${accountCols.map((c) => `n.${c}`).join(", ")}) LIMIT 1) FROM ZICCLOUDSYNCINGOBJECT n WHERE n.Z_PK = @pk;` : "SELECT NULL;";
|
|
55375
|
+
return [
|
|
55376
|
+
NOTE_STATE_SQL,
|
|
55377
|
+
`SELECT json_group_array(json_object('pk', a.Z_PK, 'identifier', a.ZIDENTIFIER, 'uti', a.ZTYPEUTI, 'duration', a.ZDURATION, 'media', ${media("a")}, 'children', json((SELECT json_group_array(json_object('pk', c.Z_PK, 'identifier', c.ZIDENTIFIER, 'duration', c.ZDURATION, 'media', ${media("c")})) FROM (SELECT * FROM ZICCLOUDSYNCINGOBJECT k WHERE k.ZPARENTATTACHMENT = a.Z_PK AND ${live("k")} ORDER BY k.Z_PK) c)))) FROM (SELECT * FROM ZICCLOUDSYNCINGOBJECT t WHERE t.ZNOTE = @pk AND t.ZPARENTATTACHMENT IS NULL AND ${live("t")} AND (t.ZTYPEUTI LIKE '%audio%' OR t.ZTYPEUTI IN (${utis})) ORDER BY t.Z_PK) a;`,
|
|
55378
|
+
account
|
|
55379
|
+
].join("\n");
|
|
55380
|
+
}
|
|
55381
|
+
function accountDirsFor(containerDir, accountIdentifier) {
|
|
55382
|
+
const own = resolveAccountDir(containerDir, accountIdentifier);
|
|
55383
|
+
if (own) return [own];
|
|
55384
|
+
let names;
|
|
55385
|
+
try {
|
|
55386
|
+
names = readdirSync4(join25(containerDir, "Accounts")).sort();
|
|
55387
|
+
} catch {
|
|
55388
|
+
return [];
|
|
55389
|
+
}
|
|
55390
|
+
return names.map((name) => safeComponent(name) ? resolveAccountDir(containerDir, name) : null).filter((dir) => dir !== null);
|
|
55391
|
+
}
|
|
55392
|
+
function resolveMediaPath(media, accountIdentifier = null, containerDir = NOTES_CONTAINER_DIR) {
|
|
55393
|
+
const id2 = safeComponent(media?.identifier);
|
|
55394
|
+
const filename = safeComponent(media?.filename);
|
|
55395
|
+
if (!id2 || !filename) return null;
|
|
55396
|
+
const generation = safeComponent(media?.generation);
|
|
55397
|
+
for (const dir of accountDirsFor(containerDir, accountIdentifier)) {
|
|
55398
|
+
const base = join25(dir, "Media", id2);
|
|
55399
|
+
const candidates = generation ? [join25(base, generation, filename), join25(base, filename)] : [join25(base, filename)];
|
|
55400
|
+
for (const candidate of candidates) {
|
|
55401
|
+
const real = realInside(candidate, dir);
|
|
55402
|
+
if (real && isFile2(real)) return real;
|
|
55403
|
+
}
|
|
55404
|
+
}
|
|
55405
|
+
return null;
|
|
55406
|
+
}
|
|
55407
|
+
function isFile2(path10) {
|
|
55408
|
+
try {
|
|
55409
|
+
return statSync4(path10).isFile();
|
|
55410
|
+
} catch {
|
|
55411
|
+
return false;
|
|
55412
|
+
}
|
|
55413
|
+
}
|
|
55414
|
+
var positive = (value) => typeof value === "number" && Number.isFinite(value) && value > 0 ? value : null;
|
|
55415
|
+
function readAudioAssets(noteId3, options = {}) {
|
|
55416
|
+
const { store, pk } = parseNoteId2(noteId3);
|
|
55417
|
+
const dbPath2 = options.dbPath ?? NOTES_DB_PATH7;
|
|
55418
|
+
const columns = readColumns(dbPath2);
|
|
55419
|
+
const missing = ["ZMEDIA", "ZPARENTATTACHMENT", "ZFILENAME"].filter((c) => !columns.has(c));
|
|
55420
|
+
if (missing.length)
|
|
55421
|
+
throw new NoteStoreError(
|
|
55422
|
+
`This macOS version's Notes database lacks ${missing.join(", ")}; audio files cannot be located.`,
|
|
55423
|
+
"schema"
|
|
55424
|
+
);
|
|
55425
|
+
const generation = columns.has("ZGENERATION1") ? "ZGENERATION1" : columns.has("ZGENERATION") ? "ZGENERATION" : null;
|
|
55426
|
+
const [stateLine, rowsLine, accountLine] = queryNoteScoped(
|
|
55427
|
+
audioRowsSql(columns, generation),
|
|
55428
|
+
pk,
|
|
55429
|
+
dbPath2
|
|
55430
|
+
);
|
|
55431
|
+
assertNoteReadable(stateLine, noteId3);
|
|
55432
|
+
const rows = JSON.parse(rowsLine || "[]");
|
|
55433
|
+
const account = accountLine?.trim() || null;
|
|
55434
|
+
const attachmentId = (rowPk) => `x-coredata://${store}/ICAttachment/p${rowPk}`;
|
|
55435
|
+
return rows.map((row) => {
|
|
55436
|
+
const sources = row.children?.length ? row.children : [{ pk: row.pk, identifier: row.identifier, duration: row.duration, media: row.media }];
|
|
55437
|
+
const takes = sources.map((take) => ({
|
|
55438
|
+
attachmentId: attachmentId(take.pk),
|
|
55439
|
+
identifier: take.identifier ?? "",
|
|
55440
|
+
durationSeconds: positive(take.duration),
|
|
55441
|
+
path: resolveMediaPath(take.media, account, options.containerDir)
|
|
55442
|
+
}));
|
|
55443
|
+
const total = takes.reduce((sum, t) => sum + (t.durationSeconds ?? 0), 0);
|
|
55444
|
+
return {
|
|
55445
|
+
pk: row.pk,
|
|
55446
|
+
attachmentId: attachmentId(row.pk),
|
|
55447
|
+
identifier: row.identifier ?? "",
|
|
55448
|
+
typeUti: row.uti,
|
|
55449
|
+
durationSeconds: positive(row.duration) ?? positive(total),
|
|
55450
|
+
takes
|
|
55451
|
+
};
|
|
55452
|
+
});
|
|
55453
|
+
}
|
|
55454
|
+
function countWords2(text2) {
|
|
55455
|
+
return text2.split(/\s+/u).filter((w) => /[\p{L}\p{N}]/u.test(w)).length;
|
|
55456
|
+
}
|
|
55457
|
+
|
|
55458
|
+
// src/services/noteTranscription.ts
|
|
55459
|
+
var DEFAULT_TRANSCRIPTION_LOCALE = "en-US";
|
|
55460
|
+
var LOCALE_PATTERN = /^[A-Za-z]{2,3}(?:[-_][A-Za-z0-9]{2,8}){0,3}$/;
|
|
55461
|
+
var TRANSCRIBE_MAX_SECONDS = { default: 900, min: 30, max: 3600 };
|
|
55462
|
+
function takeDeadlineSeconds(durationSeconds) {
|
|
55463
|
+
const duration3 = durationSeconds ?? 600;
|
|
55464
|
+
return Math.min(1800, Math.max(60, Math.ceil(duration3 * 1.5) + 60));
|
|
55465
|
+
}
|
|
55466
|
+
var KILL_GRACE_MS = 3e4;
|
|
55467
|
+
var BUDGET_MARGIN_MS = 1e4;
|
|
55468
|
+
var MIN_TAKE_SECONDS = 5;
|
|
55469
|
+
var CALL_WIDE_CODES = /* @__PURE__ */ new Set([
|
|
55470
|
+
"permission_required",
|
|
55471
|
+
"asset_unavailable",
|
|
55472
|
+
"unsupported_locale",
|
|
55473
|
+
"speech_unavailable"
|
|
55474
|
+
]);
|
|
55475
|
+
var helperTranscriptSchema = external_exports.object({
|
|
55476
|
+
status: external_exports.literal("ok"),
|
|
55477
|
+
transcript: external_exports.string(),
|
|
55478
|
+
complete: external_exports.boolean(),
|
|
55479
|
+
stopReason: external_exports.string().optional(),
|
|
55480
|
+
engine: external_exports.string().optional(),
|
|
55481
|
+
durationSeconds: external_exports.number().optional()
|
|
55482
|
+
});
|
|
55483
|
+
async function transcribeTake(take, ctx) {
|
|
55484
|
+
const base = {
|
|
55485
|
+
attachmentId: take.attachmentId,
|
|
55486
|
+
identifier: take.identifier,
|
|
55487
|
+
status: "error",
|
|
55488
|
+
...take.durationSeconds !== null ? { durationSeconds: Math.round(take.durationSeconds) } : {}
|
|
55489
|
+
};
|
|
55490
|
+
const failed = (code, message) => ({
|
|
55491
|
+
take: { ...base, code, message },
|
|
55492
|
+
text: ""
|
|
55493
|
+
});
|
|
55494
|
+
if (!take.path)
|
|
55495
|
+
return failed(
|
|
55496
|
+
"asset_unavailable",
|
|
55497
|
+
"The audio file is not on this Mac (it may not have downloaded from iCloud yet)."
|
|
55498
|
+
);
|
|
55499
|
+
if (ctx.callWide) return failed(ctx.callWide.code, ctx.callWide.message);
|
|
55500
|
+
const remainingMs = ctx.deadlineAt - ctx.now();
|
|
55501
|
+
const deadline = Math.min(
|
|
55502
|
+
takeDeadlineSeconds(take.durationSeconds),
|
|
55503
|
+
Math.floor((remainingMs - BUDGET_MARGIN_MS) / 1e3)
|
|
55504
|
+
);
|
|
55505
|
+
if (deadline < MIN_TAKE_SECONDS)
|
|
55506
|
+
return failed(
|
|
55507
|
+
"time_limit",
|
|
55508
|
+
`Not started: the call's ${ctx.maxSeconds}-second limit was reached. Transcribe this recording on its own with attachmentId, or raise maxSeconds.`
|
|
55509
|
+
);
|
|
55510
|
+
let raw;
|
|
55511
|
+
try {
|
|
55512
|
+
raw = await callPublicHelperAsync(
|
|
55513
|
+
"transcribe",
|
|
55514
|
+
{
|
|
55515
|
+
path: take.path,
|
|
55516
|
+
locale: ctx.locale,
|
|
55517
|
+
timeoutSeconds: deadline,
|
|
55518
|
+
downloadAssets: ctx.downloadAssets
|
|
55519
|
+
},
|
|
55520
|
+
ctx.deps,
|
|
55521
|
+
{ timeoutMs: Math.min(deadline * 1e3 + KILL_GRACE_MS, remainingMs), signal: ctx.signal }
|
|
55522
|
+
);
|
|
55523
|
+
} catch (error2) {
|
|
55524
|
+
const code = error2 instanceof PublicHelperError ? error2.code : "internal_error";
|
|
55525
|
+
if (code === "aborted") throw error2;
|
|
55526
|
+
const message = error2 instanceof Error ? error2.message : String(error2);
|
|
55527
|
+
if (CALL_WIDE_CODES.has(code)) ctx.callWide = { code, message };
|
|
55528
|
+
return {
|
|
55529
|
+
take: {
|
|
55530
|
+
...base,
|
|
55531
|
+
// A timeout means the helper was killed mid-run: the outcome is unknown.
|
|
55532
|
+
status: code === "timeout" ? "indeterminate" : "error",
|
|
55533
|
+
code,
|
|
55534
|
+
message
|
|
55535
|
+
},
|
|
55536
|
+
text: ""
|
|
55537
|
+
};
|
|
55538
|
+
}
|
|
55539
|
+
const parsed = helperTranscriptSchema.safeParse(raw);
|
|
55540
|
+
if (!parsed.success) return failed("invalid_response", "Unexpected helper response.");
|
|
55541
|
+
const { transcript, complete, stopReason, engine, durationSeconds } = parsed.data;
|
|
55542
|
+
const text2 = transcript.trim();
|
|
55543
|
+
return {
|
|
55544
|
+
take: {
|
|
55545
|
+
...base,
|
|
55546
|
+
status: complete ? "ok" : text2 ? "partial" : "indeterminate",
|
|
55547
|
+
...complete ? {} : { code: "incomplete", message: stopReason ?? "stopped early" },
|
|
55548
|
+
...durationSeconds !== void 0 ? { durationSeconds } : {},
|
|
55549
|
+
wordCount: countWords2(text2),
|
|
55550
|
+
...engine ? { engine } : {}
|
|
55551
|
+
},
|
|
55552
|
+
text: text2
|
|
55553
|
+
};
|
|
55554
|
+
}
|
|
55555
|
+
function combineStatus(parts) {
|
|
55556
|
+
if (parts.length && parts.every((p) => p.status === "ok")) return "ok";
|
|
55557
|
+
if (parts.some((p) => p.hasText)) return "partial";
|
|
55558
|
+
if (parts.some((p) => p.status === "indeterminate")) return "indeterminate";
|
|
55559
|
+
return "error";
|
|
55560
|
+
}
|
|
55561
|
+
async function transcribeRecording(asset, includeText, ctx) {
|
|
55562
|
+
const outcomes = [];
|
|
55563
|
+
for (const take of asset.takes) outcomes.push(await transcribeTake(take, ctx));
|
|
55564
|
+
const transcript = outcomes.map((o) => o.text).filter(Boolean).join("\n\n");
|
|
55565
|
+
const status = combineStatus(
|
|
55566
|
+
outcomes.map((o) => ({ status: o.take.status, hasText: o.text.length > 0 }))
|
|
55567
|
+
);
|
|
55568
|
+
const firstProblem = outcomes.find((o) => o.take.status !== "ok")?.take;
|
|
55569
|
+
return {
|
|
55570
|
+
attachmentId: asset.attachmentId,
|
|
55571
|
+
identifier: asset.identifier,
|
|
55572
|
+
typeUti: asset.typeUti,
|
|
55573
|
+
status,
|
|
55574
|
+
...status !== "ok" && firstProblem?.code ? { code: firstProblem.code, message: firstProblem.message } : {},
|
|
55575
|
+
...asset.durationSeconds !== null ? { durationSeconds: Math.round(asset.durationSeconds) } : {},
|
|
55576
|
+
wordCount: countWords2(transcript),
|
|
55577
|
+
...includeText ? { transcript } : {},
|
|
55578
|
+
takes: outcomes.map((o) => o.take)
|
|
55579
|
+
};
|
|
55580
|
+
}
|
|
55581
|
+
async function transcribeNoteAudio(noteId3, options = {}) {
|
|
55582
|
+
const locale = options.locale ?? DEFAULT_TRANSCRIPTION_LOCALE;
|
|
55583
|
+
if (!LOCALE_PATTERN.test(locale))
|
|
55584
|
+
throw new PublicHelperError("invalid_request", `"${locale}" is not a BCP-47 locale.`);
|
|
55585
|
+
const maxSeconds = options.maxSeconds ?? TRANSCRIBE_MAX_SECONDS.default;
|
|
55586
|
+
if (!Number.isInteger(maxSeconds) || maxSeconds < TRANSCRIBE_MAX_SECONDS.min || maxSeconds > TRANSCRIBE_MAX_SECONDS.max)
|
|
55587
|
+
throw new PublicHelperError(
|
|
55588
|
+
"invalid_request",
|
|
55589
|
+
`maxSeconds must be a whole number from ${TRANSCRIBE_MAX_SECONDS.min} to ${TRANSCRIBE_MAX_SECONDS.max}.`
|
|
55590
|
+
);
|
|
55591
|
+
const deps = options.deps ?? defaultPublicHelperDeps();
|
|
55592
|
+
let assets = (options.readAssets ?? ((id2) => readAudioAssets(id2)))(noteId3);
|
|
55593
|
+
if (options.attachmentId) {
|
|
55594
|
+
assets = assets.filter((a) => a.attachmentId === options.attachmentId);
|
|
55595
|
+
if (assets.length === 0)
|
|
55596
|
+
throw new PublicHelperError(
|
|
55597
|
+
"attachment_not_found",
|
|
55598
|
+
`No audio attachment ${options.attachmentId} in note ${noteId3}.`
|
|
55599
|
+
);
|
|
55600
|
+
}
|
|
55601
|
+
const empty = { id: noteId3, locale, recordingCount: 0, recordings: [] };
|
|
55602
|
+
if (assets.length === 0) return { ...empty, status: "none" };
|
|
55603
|
+
const install = inspectPublicHelper(deps);
|
|
55604
|
+
if (!install.ready)
|
|
55605
|
+
throw new PublicHelperError(install.reason ?? "helper_not_installed", install.detail ?? "");
|
|
55606
|
+
const now = options.now ?? Date.now;
|
|
55607
|
+
const ctx = {
|
|
55608
|
+
locale,
|
|
55609
|
+
downloadAssets: options.downloadAssets ?? false,
|
|
55610
|
+
deps,
|
|
55611
|
+
signal: options.signal,
|
|
55612
|
+
now,
|
|
55613
|
+
deadlineAt: now() + maxSeconds * 1e3,
|
|
55614
|
+
maxSeconds,
|
|
55615
|
+
callWide: null
|
|
55616
|
+
};
|
|
55617
|
+
const recordings = [];
|
|
55618
|
+
for (const asset of assets)
|
|
55619
|
+
recordings.push(await transcribeRecording(asset, options.includeText ?? true, ctx));
|
|
55620
|
+
return {
|
|
55621
|
+
...empty,
|
|
55622
|
+
status: combineStatus(
|
|
55623
|
+
recordings.map((r) => ({ status: r.status, hasText: (r.wordCount ?? 0) > 0 }))
|
|
55624
|
+
),
|
|
55625
|
+
recordingCount: recordings.length,
|
|
55626
|
+
recordings
|
|
55627
|
+
};
|
|
55628
|
+
}
|
|
55629
|
+
function fitTranscriptions(result, maxBytes, measure = (r) => Buffer.byteLength(JSON.stringify(r))) {
|
|
55630
|
+
if (measure(result) <= maxBytes) return result;
|
|
55631
|
+
let next = result;
|
|
55632
|
+
for (let share = 0.5; measure(next) > maxBytes && share > 1e-4; share /= 2) {
|
|
55633
|
+
next = {
|
|
55634
|
+
...result,
|
|
55635
|
+
recordings: result.recordings.map(
|
|
55636
|
+
(r) => r.transcript ? {
|
|
55637
|
+
...r,
|
|
55638
|
+
transcript: r.transcript.slice(0, Math.floor(r.transcript.length * share)),
|
|
55639
|
+
transcriptTruncated: true
|
|
55640
|
+
} : r
|
|
55641
|
+
)
|
|
55642
|
+
};
|
|
55643
|
+
}
|
|
55644
|
+
return next;
|
|
55645
|
+
}
|
|
55646
|
+
function formatTranscription(result) {
|
|
55647
|
+
if (result.recordingCount === 0) return `No audio attachments in note ${result.id}.`;
|
|
55648
|
+
const lines = [
|
|
55649
|
+
`${result.recordingCount} audio attachment${result.recordingCount === 1 ? "" : "s"} in note ${result.id} (${result.status}, locale ${result.locale}):`
|
|
55650
|
+
];
|
|
55651
|
+
result.recordings.forEach((r, i) => {
|
|
55652
|
+
const facts = [
|
|
55653
|
+
`status ${r.status}`,
|
|
55654
|
+
...r.durationSeconds !== void 0 ? [`${r.durationSeconds} s`] : [],
|
|
55655
|
+
`${r.wordCount ?? 0} words`,
|
|
55656
|
+
...r.takes.length > 1 ? [`${r.takes.length} takes`] : [],
|
|
55657
|
+
...r.code ? [`${r.code}: ${r.message}`] : []
|
|
55658
|
+
];
|
|
55659
|
+
lines.push("", `[${i + 1}] ${r.attachmentId} (${facts.join(", ")})`);
|
|
55660
|
+
if (r.transcript)
|
|
55661
|
+
lines.push(r.transcriptTruncated ? `${r.transcript} [truncated]` : r.transcript);
|
|
55662
|
+
});
|
|
55663
|
+
return lines.join("\n");
|
|
55664
|
+
}
|
|
55665
|
+
|
|
55282
55666
|
// src/services/privateHelperBuild.ts
|
|
55283
55667
|
import { spawnSync as spawnSync5 } from "node:child_process";
|
|
55284
55668
|
import {
|
|
@@ -55291,14 +55675,14 @@ import {
|
|
|
55291
55675
|
writeFileSync as writeFileSync5
|
|
55292
55676
|
} from "node:fs";
|
|
55293
55677
|
import { release as release4 } from "node:os";
|
|
55294
|
-
import { join as
|
|
55678
|
+
import { join as join27 } from "node:path";
|
|
55295
55679
|
|
|
55296
55680
|
// src/services/privateHelper.ts
|
|
55297
55681
|
import { spawnSync as spawnSync4 } from "node:child_process";
|
|
55298
55682
|
import { createHash as createHash6 } from "node:crypto";
|
|
55299
55683
|
import { existsSync as existsSync15, readFileSync as readFileSync5 } from "node:fs";
|
|
55300
55684
|
import { homedir as homedir20 } from "node:os";
|
|
55301
|
-
import { dirname as dirname9, join as
|
|
55685
|
+
import { dirname as dirname9, join as join26, resolve as resolve6 } from "node:path";
|
|
55302
55686
|
import { fileURLToPath as fileURLToPath3 } from "node:url";
|
|
55303
55687
|
var PRIVATE_HELPER_PROTOCOL = 1;
|
|
55304
55688
|
var ENABLE_ENV = "APPLE_NOTES_MCP_ENABLE_PRIVATE";
|
|
@@ -55326,7 +55710,7 @@ var manifestSchema = external_exports.object({
|
|
|
55326
55710
|
function packageRoot2(fromDir = dirname9(fileURLToPath3(import.meta.url))) {
|
|
55327
55711
|
let dir = fromDir;
|
|
55328
55712
|
for (; ; ) {
|
|
55329
|
-
const candidate =
|
|
55713
|
+
const candidate = join26(dir, "package.json");
|
|
55330
55714
|
if (existsSync15(candidate)) {
|
|
55331
55715
|
try {
|
|
55332
55716
|
const pkg = JSON.parse(readFileSync5(candidate, "utf8"));
|
|
@@ -55343,7 +55727,7 @@ function defaultDeps2(overrides = {}) {
|
|
|
55343
55727
|
return {
|
|
55344
55728
|
env: process.env,
|
|
55345
55729
|
platform: process.platform,
|
|
55346
|
-
sourcePath:
|
|
55730
|
+
sourcePath: join26(packageRoot2(), HELPER_SOURCE_RELATIVE),
|
|
55347
55731
|
exists: existsSync15,
|
|
55348
55732
|
readFile: (path10) => readFileSync5(path10),
|
|
55349
55733
|
spawn: spawnSync4,
|
|
@@ -55356,14 +55740,14 @@ function privateHelperEnabled(env = process.env) {
|
|
|
55356
55740
|
function helperInstallDir(env = process.env) {
|
|
55357
55741
|
const override = env[HELPER_DIR_ENV]?.trim();
|
|
55358
55742
|
if (override) return override;
|
|
55359
|
-
return
|
|
55743
|
+
return join26(homedir20(), "Library", "Application Support", "apple-notes-mcp", "private-helper");
|
|
55360
55744
|
}
|
|
55361
55745
|
function sha256Hex2(data) {
|
|
55362
55746
|
return createHash6("sha256").update(data).digest("hex");
|
|
55363
55747
|
}
|
|
55364
55748
|
function inspectInstallation(deps = defaultDeps2()) {
|
|
55365
55749
|
const installDir = helperInstallDir(deps.env);
|
|
55366
|
-
const binaryPath =
|
|
55750
|
+
const binaryPath = join26(installDir, HELPER_BINARY_NAME);
|
|
55367
55751
|
const base = {
|
|
55368
55752
|
installDir,
|
|
55369
55753
|
binaryPath,
|
|
@@ -55381,7 +55765,7 @@ function inspectInstallation(deps = defaultDeps2()) {
|
|
|
55381
55765
|
if (!deps.exists(deps.sourcePath))
|
|
55382
55766
|
return fail("helper_not_installed", `Packaged helper source is missing: ${deps.sourcePath}`);
|
|
55383
55767
|
base.expectedSourceSha256 = sha256Hex2(deps.readFile(deps.sourcePath));
|
|
55384
|
-
const manifestPath =
|
|
55768
|
+
const manifestPath = join26(installDir, MANIFEST_NAME);
|
|
55385
55769
|
if (!deps.exists(binaryPath) || !deps.exists(manifestPath))
|
|
55386
55770
|
return fail(
|
|
55387
55771
|
"helper_not_installed",
|
|
@@ -55696,9 +56080,9 @@ function buildPrivateHelper(checkOnly, deps = defaultBuildDeps()) {
|
|
|
55696
56080
|
steps.push({ step: "find compiler", ok: true, detail: compiler });
|
|
55697
56081
|
const installDir = helperInstallDir(deps.env);
|
|
55698
56082
|
mkdirSync7(installDir, { recursive: true, mode: 448 });
|
|
55699
|
-
const staging = mkdtempSync6(
|
|
56083
|
+
const staging = mkdtempSync6(join27(installDir, ".staging-"));
|
|
55700
56084
|
try {
|
|
55701
|
-
const stagedBinary =
|
|
56085
|
+
const stagedBinary = join27(staging, HELPER_BINARY_NAME);
|
|
55702
56086
|
const compile = deps.spawn(
|
|
55703
56087
|
"/usr/bin/xcrun",
|
|
55704
56088
|
compileArguments(deps.sourcePath, stagedBinary, sourceSha),
|
|
@@ -55775,8 +56159,8 @@ function buildPrivateHelper(checkOnly, deps = defaultBuildDeps()) {
|
|
|
55775
56159
|
compiler
|
|
55776
56160
|
};
|
|
55777
56161
|
chmodSync2(stagedBinary, 448);
|
|
55778
|
-
renameSync2(stagedBinary,
|
|
55779
|
-
writeFileSync5(
|
|
56162
|
+
renameSync2(stagedBinary, join27(installDir, HELPER_BINARY_NAME));
|
|
56163
|
+
writeFileSync5(join27(installDir, MANIFEST_NAME), JSON.stringify(manifest, null, 2) + "\n", {
|
|
55780
56164
|
mode: 384
|
|
55781
56165
|
});
|
|
55782
56166
|
steps.push({ step: "install", ok: true, detail: installDir });
|
|
@@ -55970,6 +56354,16 @@ function withErrorHandling(handler, errorPrefix) {
|
|
|
55970
56354
|
}
|
|
55971
56355
|
};
|
|
55972
56356
|
}
|
|
56357
|
+
function withAsyncErrorHandling(handler, errorPrefix) {
|
|
56358
|
+
return async (params, extra) => {
|
|
56359
|
+
try {
|
|
56360
|
+
return await handler(params, extra?.signal);
|
|
56361
|
+
} catch (error2) {
|
|
56362
|
+
const message = error2 instanceof Error ? error2.message : "Unknown error";
|
|
56363
|
+
return errorResponse(`${errorPrefix}: ${message}`, error2);
|
|
56364
|
+
}
|
|
56365
|
+
};
|
|
56366
|
+
}
|
|
55973
56367
|
var MAX = {
|
|
55974
56368
|
TITLE: 2e3,
|
|
55975
56369
|
CONTENT: 5 * 1024 * 1024,
|
|
@@ -59006,6 +59400,49 @@ registerTool(
|
|
|
59006
59400
|
});
|
|
59007
59401
|
}, "Error reading drawings")
|
|
59008
59402
|
);
|
|
59403
|
+
registerTool(
|
|
59404
|
+
"transcribe-note-audio",
|
|
59405
|
+
{
|
|
59406
|
+
description: "Use when: you need the words spoken in a note's voice recordings or audio attachments, transcribed now on this Mac, by note id.\nReturns: per audio attachment a status (ok / partial / error / indeterminate), duration, word count, per-take results, and the transcript; overall status ok / partial / error / indeterminate / none.\nDo not use when: you only need the audio file (save-attachment) or a note has no audio.\nNote: read-only; recognition runs entirely on-device (never sent to a server). Needs Full Disk Access and the public native helper built once with `apple-notes-mcp setup --public-helper`. Long recordings take time: pass attachmentId to transcribe one at a time. An indeterminate result means the helper timed out; retrying may succeed. Never prompts: code permission_required means the user must allow the host app under System Settings > Privacy & Security > Speech Recognition. asset_unavailable can mean the language's speech model is not installed; pass downloadAssets: true only if the user agrees to the download.",
|
|
59407
|
+
inputSchema: {
|
|
59408
|
+
id: noteIdInput,
|
|
59409
|
+
locale: external_exports.string().max(35).optional().describe('BCP-47 language of the speech, e.g. "en-US" (default), "it-IT", "fr-FR"'),
|
|
59410
|
+
attachmentId: external_exports.string().max(MAX.ATTACHMENT_ID).optional().describe("Only transcribe this audio attachment (x-coredata ICAttachment id)"),
|
|
59411
|
+
includeText: external_exports.boolean().optional().describe("Include transcript text (default true); false returns statuses and counts only"),
|
|
59412
|
+
downloadAssets: external_exports.boolean().optional().describe(
|
|
59413
|
+
"Let macOS download the language's on-device speech model if it is missing (default false: returns asset_unavailable at once)"
|
|
59414
|
+
),
|
|
59415
|
+
maxSeconds: external_exports.number().int().min(TRANSCRIBE_MAX_SECONDS.min).max(TRANSCRIBE_MAX_SECONDS.max).optional().describe(
|
|
59416
|
+
`Total time budget for the call in seconds (default ${TRANSCRIBE_MAX_SECONDS.default}); recordings not started in time report code time_limit`
|
|
59417
|
+
)
|
|
59418
|
+
},
|
|
59419
|
+
outputSchema: {
|
|
59420
|
+
id: external_exports.string().optional(),
|
|
59421
|
+
locale: external_exports.string().optional(),
|
|
59422
|
+
status: external_exports.string().optional(),
|
|
59423
|
+
recordingCount: external_exports.number().optional(),
|
|
59424
|
+
recordings: external_exports.array(external_exports.object({}).passthrough()).optional()
|
|
59425
|
+
}
|
|
59426
|
+
},
|
|
59427
|
+
withAsyncErrorHandling(async (params, signal) => {
|
|
59428
|
+
const { id: id2, locale, attachmentId, includeText, downloadAssets, maxSeconds } = params;
|
|
59429
|
+
const result = fitTranscriptions(
|
|
59430
|
+
await transcribeNoteAudio(id2, {
|
|
59431
|
+
locale,
|
|
59432
|
+
attachmentId,
|
|
59433
|
+
includeText,
|
|
59434
|
+
downloadAssets,
|
|
59435
|
+
maxSeconds,
|
|
59436
|
+
signal
|
|
59437
|
+
}),
|
|
59438
|
+
exportMaxResponseBytes()
|
|
59439
|
+
);
|
|
59440
|
+
return successResponse(
|
|
59441
|
+
formatTranscription(result),
|
|
59442
|
+
result
|
|
59443
|
+
);
|
|
59444
|
+
}, "Error transcribing audio")
|
|
59445
|
+
);
|
|
59009
59446
|
registerTool(
|
|
59010
59447
|
"list-recent-notes",
|
|
59011
59448
|
{
|
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
// A small command-line helper for apple-notes-mcp that uses PUBLIC Apple
|
|
4
4
|
// frameworks only. It never opens the Notes database and never writes to the
|
|
5
5
|
// Notes group container: the TypeScript server reads the bytes it needs
|
|
6
|
-
// (read-only) and hands them over on stdin
|
|
6
|
+
// (read-only) and hands them over on stdin, or names an audio file that the
|
|
7
|
+
// helper opens for reading only.
|
|
7
8
|
//
|
|
8
9
|
// Protocol: one JSON object on stdin, one JSON object on stdout.
|
|
9
10
|
// request: {"protocol": 1, "action": "<name>", ...fields}
|
|
@@ -15,6 +16,7 @@
|
|
|
15
16
|
// decode_drawing PencilKit drawing bytes (base64) -> strokes
|
|
16
17
|
// encode_drawing strokes -> PencilKit drawing bytes (base64); used to build
|
|
17
18
|
// synthetic test fixtures, never to write to Notes
|
|
19
|
+
// transcribe on-device Speech transcription of one audio file
|
|
18
20
|
//
|
|
19
21
|
// Build (done by `apple-notes-mcp setup --public-helper`, which also generates
|
|
20
22
|
// the one-line source-digest file that defines helperSourceSHA256 and the
|
|
@@ -23,9 +25,11 @@
|
|
|
23
25
|
// -Xlinker -sectcreate -Xlinker __TEXT -Xlinker __info_plist -Xlinker Info.plist \
|
|
24
26
|
// apple-notes-public-helper.swift source-digest.swift -o apple-notes-public-helper
|
|
25
27
|
|
|
28
|
+
import AVFoundation
|
|
26
29
|
import AppKit
|
|
27
30
|
import Foundation
|
|
28
31
|
import PencilKit
|
|
32
|
+
import Speech
|
|
29
33
|
|
|
30
34
|
let protocolVersion = 1
|
|
31
35
|
let maxInputBytes = 96 * 1024 * 1024
|
|
@@ -211,11 +215,331 @@ func encodeDrawing(_ request: [String: Any]) throws -> [String: Any] {
|
|
|
211
215
|
return ["status": "ok", "strokeCount": strokes.count, "dataBase64": data.base64EncodedString()]
|
|
212
216
|
}
|
|
213
217
|
|
|
218
|
+
// MARK: - Transcription
|
|
219
|
+
//
|
|
220
|
+
// On-device only. On macOS 26 and later, SpeechAnalyzer with SpeechTranscriber
|
|
221
|
+
// runs entirely on the Mac (its language model is a local asset). On older
|
|
222
|
+
// systems, SFSpeechRecognizer is used with requiresOnDeviceRecognition = true,
|
|
223
|
+
// and a locale without on-device support is refused rather than sent to a
|
|
224
|
+
// server. The helper stops at its own deadline and returns what it has with
|
|
225
|
+
// complete = false, so the caller can report a partial transcript.
|
|
226
|
+
|
|
227
|
+
/// Final transcript segments gathered so far; an actor so the deadline path can read them.
|
|
228
|
+
actor TranscriptCollector {
|
|
229
|
+
private var segments: [String] = []
|
|
230
|
+
func add(_ segment: String) { segments.append(segment) }
|
|
231
|
+
func text() -> String { segments.joined(separator: " ") }
|
|
232
|
+
func isEmpty() -> Bool { segments.isEmpty }
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
/// The newest partial result (legacy recognizer), guarded by a lock because the
|
|
236
|
+
/// recognizer calls back on its own queue.
|
|
237
|
+
final class LatestText: @unchecked Sendable {
|
|
238
|
+
private let lock = NSLock()
|
|
239
|
+
private var value = ""
|
|
240
|
+
func set(_ text: String) { lock.withLock { value = text } }
|
|
241
|
+
func get() -> String { lock.withLock { value } }
|
|
242
|
+
}
|
|
243
|
+
|
|
244
|
+
enum RaceOutcome {
|
|
245
|
+
case completed
|
|
246
|
+
case failed(Error)
|
|
247
|
+
case timedOut
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
/// Resumes a continuation exactly once, whichever side of a race finishes first.
|
|
251
|
+
final class RaceGate: @unchecked Sendable {
|
|
252
|
+
private let lock = NSLock()
|
|
253
|
+
private var continuation: CheckedContinuation<RaceOutcome, Never>?
|
|
254
|
+
private var expired = false
|
|
255
|
+
init(_ continuation: CheckedContinuation<RaceOutcome, Never>) { self.continuation = continuation }
|
|
256
|
+
/// Called when the deadline passes, before `stop`: whatever `work` reports
|
|
257
|
+
/// afterwards (usually a cancellation error) is a timeout, not a failure.
|
|
258
|
+
func expire() { lock.withLock { expired = true } }
|
|
259
|
+
func resume(_ outcome: RaceOutcome) {
|
|
260
|
+
let (pending, final): (CheckedContinuation<RaceOutcome, Never>?, RaceOutcome) = lock.withLock {
|
|
261
|
+
defer { continuation = nil }
|
|
262
|
+
return (continuation, expired ? .timedOut : outcome)
|
|
263
|
+
}
|
|
264
|
+
pending?.resume(returning: final)
|
|
265
|
+
}
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
/// Runs `work` against a deadline without waiting for it after the deadline:
|
|
269
|
+
/// some framework calls (asset downloads) ignore cancellation, and the process
|
|
270
|
+
/// exits right after it answers anyway. On timeout, `stop` runs first so a
|
|
271
|
+
/// cooperative `work` can hand back what it has.
|
|
272
|
+
func race(
|
|
273
|
+
seconds: Double,
|
|
274
|
+
work: @escaping @Sendable () async throws -> Void,
|
|
275
|
+
stop: @escaping @Sendable () async -> Void
|
|
276
|
+
) async -> RaceOutcome {
|
|
277
|
+
await withCheckedContinuation { continuation in
|
|
278
|
+
let gate = RaceGate(continuation)
|
|
279
|
+
Task {
|
|
280
|
+
do {
|
|
281
|
+
try await work()
|
|
282
|
+
gate.resume(.completed)
|
|
283
|
+
} catch {
|
|
284
|
+
gate.resume(.failed(error))
|
|
285
|
+
}
|
|
286
|
+
}
|
|
287
|
+
Task {
|
|
288
|
+
try? await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
|
|
289
|
+
gate.expire()
|
|
290
|
+
await stop()
|
|
291
|
+
gate.resume(.timedOut)
|
|
292
|
+
}
|
|
293
|
+
}
|
|
294
|
+
}
|
|
295
|
+
|
|
296
|
+
func authorizationName(_ status: SFSpeechRecognizerAuthorizationStatus) -> String {
|
|
297
|
+
switch status {
|
|
298
|
+
case .authorized: return "authorized"
|
|
299
|
+
case .denied: return "denied"
|
|
300
|
+
case .restricted: return "restricted"
|
|
301
|
+
case .notDetermined: return "notDetermined"
|
|
302
|
+
@unknown default: return "unknown"
|
|
303
|
+
}
|
|
304
|
+
}
|
|
305
|
+
|
|
306
|
+
// Speech Recognition access. The helper runs under an MCP server that nobody
|
|
307
|
+
// may be watching, so it never calls SFSpeechRecognizer.requestAuthorization:
|
|
308
|
+
// a system prompt would block the helper until the server killed it. It reads
|
|
309
|
+
// the current status instead and stops with `permission_required` when the
|
|
310
|
+
// engine it is about to use needs a grant that is missing.
|
|
311
|
+
|
|
312
|
+
/// The failure returned instead of prompting. The grant belongs to the app that
|
|
313
|
+
/// runs the MCP server (Claude, a terminal, ...), which macOS treats as responsible.
|
|
314
|
+
func permissionRequired(_ status: SFSpeechRecognizerAuthorizationStatus) -> HelperFailure {
|
|
315
|
+
let settings = "System Settings > Privacy & Security > Speech Recognition"
|
|
316
|
+
let detail: String
|
|
317
|
+
switch status {
|
|
318
|
+
case .denied:
|
|
319
|
+
detail = "Speech Recognition access was denied for the app running this server. Turn it on in \(settings), then try again."
|
|
320
|
+
case .restricted:
|
|
321
|
+
detail = "Speech Recognition is restricted on this Mac (for example by a device management profile), so it cannot be used."
|
|
322
|
+
default:
|
|
323
|
+
detail = "Speech Recognition access has not been granted to the app running this server, and the server never shows the permission prompt. Allow the app in \(settings), then try again."
|
|
324
|
+
}
|
|
325
|
+
return HelperFailure(code: "permission_required", message: detail)
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
/// Stops before any Speech call when access is missing. The legacy recognizer
|
|
329
|
+
/// needs `.authorized`. SpeechAnalyzer (macOS 26+) runs on-device without a
|
|
330
|
+
/// grant and does not prompt, so there only an explicit refusal (`.denied` or
|
|
331
|
+
/// `.restricted`) stops it; that also covers a future macOS that gates it.
|
|
332
|
+
func checkSpeechAccess(requireGrant: Bool) throws {
|
|
333
|
+
let status = SFSpeechRecognizer.authorizationStatus()
|
|
334
|
+
if status == .authorized { return }
|
|
335
|
+
if requireGrant || status == .denied || status == .restricted {
|
|
336
|
+
throw permissionRequired(status)
|
|
337
|
+
}
|
|
338
|
+
}
|
|
339
|
+
|
|
340
|
+
/// Maps a Speech error to a stable code. Permission is decided from the
|
|
341
|
+
/// explicit authorization status, never from the error's text.
|
|
342
|
+
func speechFailure(_ error: Error, requireGrant: Bool) -> HelperFailure {
|
|
343
|
+
if let failure = error as? HelperFailure { return failure }
|
|
344
|
+
do {
|
|
345
|
+
try checkSpeechAccess(requireGrant: requireGrant)
|
|
346
|
+
} catch let failure as HelperFailure {
|
|
347
|
+
return failure
|
|
348
|
+
} catch {}
|
|
349
|
+
return HelperFailure(code: "transcription_failed", message: error.localizedDescription)
|
|
350
|
+
}
|
|
351
|
+
|
|
352
|
+
/// Makes sure the locale's on-device model is installed. A download starts only
|
|
353
|
+
/// when the caller opted in: otherwise a missing model is `asset_unavailable` at once.
|
|
354
|
+
@available(macOS 26, *)
|
|
355
|
+
func ensureSpeechAssets(_ transcriber: SpeechTranscriber, localeID: String, allowDownload: Bool) async throws {
|
|
356
|
+
switch await AssetInventory.status(forModules: [transcriber]) {
|
|
357
|
+
case .installed:
|
|
358
|
+
return
|
|
359
|
+
case .unsupported:
|
|
360
|
+
throw HelperFailure(code: "unsupported_locale", message: "No on-device speech model for \(localeID)")
|
|
361
|
+
case .downloading where !allowDownload:
|
|
362
|
+
throw HelperFailure(
|
|
363
|
+
code: "asset_unavailable",
|
|
364
|
+
message: "macOS is still downloading the on-device speech model for \(localeID); try again shortly"
|
|
365
|
+
)
|
|
366
|
+
default:
|
|
367
|
+
if !allowDownload {
|
|
368
|
+
throw HelperFailure(
|
|
369
|
+
code: "asset_unavailable",
|
|
370
|
+
message: "The on-device speech model for \(localeID) is not installed. Retry with downloadAssets: true to let macOS download it."
|
|
371
|
+
)
|
|
372
|
+
}
|
|
373
|
+
}
|
|
374
|
+
guard let installation = try await AssetInventory.assetInstallationRequest(supporting: [transcriber]) else {
|
|
375
|
+
return
|
|
376
|
+
}
|
|
377
|
+
let outcome = await race(seconds: 90, work: { try await installation.downloadAndInstall() }, stop: {})
|
|
378
|
+
switch outcome {
|
|
379
|
+
case .completed:
|
|
380
|
+
return
|
|
381
|
+
case .failed(let error):
|
|
382
|
+
throw HelperFailure(
|
|
383
|
+
code: "asset_unavailable",
|
|
384
|
+
message: "Could not install the on-device speech model for \(localeID): \(error.localizedDescription)"
|
|
385
|
+
)
|
|
386
|
+
case .timedOut:
|
|
387
|
+
throw HelperFailure(
|
|
388
|
+
code: "asset_unavailable",
|
|
389
|
+
message: "The on-device speech model for \(localeID) is still downloading; try again shortly"
|
|
390
|
+
)
|
|
391
|
+
}
|
|
392
|
+
}
|
|
393
|
+
|
|
394
|
+
@available(macOS 26, *)
|
|
395
|
+
func transcribeWithAnalyzer(
|
|
396
|
+
file: AVAudioFile, localeID: String, deadline: Double, allowDownload: Bool
|
|
397
|
+
) async throws -> [String: Any] {
|
|
398
|
+
try checkSpeechAccess(requireGrant: false)
|
|
399
|
+
guard SpeechTranscriber.isAvailable else {
|
|
400
|
+
throw HelperFailure(code: "speech_unavailable", message: "On-device transcription is not available on this Mac")
|
|
401
|
+
}
|
|
402
|
+
guard let locale = await SpeechTranscriber.supportedLocale(equivalentTo: Locale(identifier: localeID)) else {
|
|
403
|
+
throw HelperFailure(code: "unsupported_locale", message: "No on-device transcription for \(localeID)")
|
|
404
|
+
}
|
|
405
|
+
let transcriber = SpeechTranscriber(locale: locale, preset: .transcription)
|
|
406
|
+
try await ensureSpeechAssets(transcriber, localeID: localeID, allowDownload: allowDownload)
|
|
407
|
+
let analyzer = SpeechAnalyzer(modules: [transcriber])
|
|
408
|
+
let collector = TranscriptCollector()
|
|
409
|
+
let reader = Task {
|
|
410
|
+
for try await result in transcriber.results where result.isFinal {
|
|
411
|
+
let segment = String(result.text.characters).trimmingCharacters(in: .whitespacesAndNewlines)
|
|
412
|
+
if !segment.isEmpty { await collector.add(segment) }
|
|
413
|
+
}
|
|
414
|
+
}
|
|
415
|
+
let outcome = await race(
|
|
416
|
+
seconds: deadline,
|
|
417
|
+
work: {
|
|
418
|
+
_ = try await analyzer.analyzeSequence(from: file)
|
|
419
|
+
try await analyzer.finalizeAndFinishThroughEndOfInput()
|
|
420
|
+
try await reader.value
|
|
421
|
+
},
|
|
422
|
+
stop: {
|
|
423
|
+
await analyzer.cancelAndFinishNow()
|
|
424
|
+
reader.cancel()
|
|
425
|
+
}
|
|
426
|
+
)
|
|
427
|
+
var response: [String: Any] = ["status": "ok", "engine": "SpeechAnalyzer", "locale": locale.identifier(.bcp47)]
|
|
428
|
+
switch outcome {
|
|
429
|
+
case .completed:
|
|
430
|
+
response["complete"] = true
|
|
431
|
+
case .timedOut:
|
|
432
|
+
response["complete"] = false
|
|
433
|
+
response["stopReason"] = "deadline"
|
|
434
|
+
case .failed(let error):
|
|
435
|
+
await analyzer.cancelAndFinishNow()
|
|
436
|
+
reader.cancel()
|
|
437
|
+
let failure = speechFailure(error, requireGrant: false)
|
|
438
|
+
if await collector.isEmpty() { throw failure }
|
|
439
|
+
response["complete"] = false
|
|
440
|
+
response["stopReason"] = failure.message
|
|
441
|
+
}
|
|
442
|
+
response["transcript"] = await collector.text()
|
|
443
|
+
return response
|
|
444
|
+
}
|
|
445
|
+
|
|
446
|
+
/// Pre-macOS 26 path: SFSpeechRecognizer, forced on-device. It needs a Speech
|
|
447
|
+
/// Recognition grant, which is checked here and never requested.
|
|
448
|
+
func transcribeWithRecognizer(url: URL, localeID: String, deadline: Double) async throws -> [String: Any] {
|
|
449
|
+
try checkSpeechAccess(requireGrant: true)
|
|
450
|
+
guard let recognizer = SFSpeechRecognizer(locale: Locale(identifier: localeID)) else {
|
|
451
|
+
throw HelperFailure(code: "unsupported_locale", message: "No speech recognizer for \(localeID)")
|
|
452
|
+
}
|
|
453
|
+
guard recognizer.supportsOnDeviceRecognition else {
|
|
454
|
+
throw HelperFailure(code: "unsupported_locale", message: "No on-device recognition for \(localeID)")
|
|
455
|
+
}
|
|
456
|
+
recognizer.queue = OperationQueue()
|
|
457
|
+
let request = SFSpeechURLRecognitionRequest(url: url)
|
|
458
|
+
request.requiresOnDeviceRecognition = true
|
|
459
|
+
request.shouldReportPartialResults = true
|
|
460
|
+
let latest = LatestText()
|
|
461
|
+
final class TaskBox: @unchecked Sendable { var task: SFSpeechRecognitionTask? }
|
|
462
|
+
let box = TaskBox()
|
|
463
|
+
let outcome = await race(
|
|
464
|
+
seconds: deadline,
|
|
465
|
+
work: {
|
|
466
|
+
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
|
|
467
|
+
var resumed = false
|
|
468
|
+
box.task = recognizer.recognitionTask(with: request) { result, error in
|
|
469
|
+
if resumed { return }
|
|
470
|
+
if let result {
|
|
471
|
+
latest.set(result.bestTranscription.formattedString)
|
|
472
|
+
if result.isFinal {
|
|
473
|
+
resumed = true
|
|
474
|
+
continuation.resume()
|
|
475
|
+
return
|
|
476
|
+
}
|
|
477
|
+
}
|
|
478
|
+
if let error {
|
|
479
|
+
resumed = true
|
|
480
|
+
continuation.resume(throwing: error)
|
|
481
|
+
}
|
|
482
|
+
}
|
|
483
|
+
}
|
|
484
|
+
},
|
|
485
|
+
stop: { box.task?.cancel() }
|
|
486
|
+
)
|
|
487
|
+
// Partial results replace each other; the newest one is the transcript so far.
|
|
488
|
+
let transcript = await latest.get()
|
|
489
|
+
var response: [String: Any] = ["status": "ok", "engine": "SFSpeechRecognizer", "locale": localeID]
|
|
490
|
+
switch outcome {
|
|
491
|
+
case .completed:
|
|
492
|
+
response["complete"] = true
|
|
493
|
+
case .timedOut:
|
|
494
|
+
response["complete"] = false
|
|
495
|
+
response["stopReason"] = "deadline"
|
|
496
|
+
case .failed(let error):
|
|
497
|
+
let failure = speechFailure(error, requireGrant: true)
|
|
498
|
+
if transcript.isEmpty { throw failure }
|
|
499
|
+
response["complete"] = false
|
|
500
|
+
response["stopReason"] = failure.message
|
|
501
|
+
}
|
|
502
|
+
response["transcript"] = transcript
|
|
503
|
+
return response
|
|
504
|
+
}
|
|
505
|
+
|
|
506
|
+
func transcribe(_ request: [String: Any]) async throws -> [String: Any] {
|
|
507
|
+
guard let path = request["path"] as? String, path.hasPrefix("/") else {
|
|
508
|
+
throw HelperFailure(code: "invalid_request", message: "path must be an absolute file path")
|
|
509
|
+
}
|
|
510
|
+
let localeID = request["locale"] as? String ?? "en-US"
|
|
511
|
+
let deadline = min(max((request["timeoutSeconds"] as? NSNumber)?.doubleValue ?? 300, 5), 3600)
|
|
512
|
+
var isDirectory: ObjCBool = false
|
|
513
|
+
guard FileManager.default.fileExists(atPath: path, isDirectory: &isDirectory), !isDirectory.boolValue else {
|
|
514
|
+
throw HelperFailure(code: "file_not_found", message: "The audio file does not exist")
|
|
515
|
+
}
|
|
516
|
+
let url = URL(fileURLWithPath: path)
|
|
517
|
+
let file: AVAudioFile
|
|
518
|
+
do {
|
|
519
|
+
file = try AVAudioFile(forReading: url)
|
|
520
|
+
} catch {
|
|
521
|
+
throw HelperFailure(code: "unsupported_audio", message: "Could not read the audio: \(error.localizedDescription)")
|
|
522
|
+
}
|
|
523
|
+
let duration = file.fileFormat.sampleRate > 0 ? Double(file.length) / file.fileFormat.sampleRate : 0
|
|
524
|
+
var response: [String: Any]
|
|
525
|
+
if #available(macOS 26, *) {
|
|
526
|
+
response = try await transcribeWithAnalyzer(
|
|
527
|
+
file: file, localeID: localeID, deadline: deadline,
|
|
528
|
+
allowDownload: request["downloadAssets"] as? Bool ?? false
|
|
529
|
+
)
|
|
530
|
+
} else {
|
|
531
|
+
response = try await transcribeWithRecognizer(url: url, localeID: localeID, deadline: deadline)
|
|
532
|
+
}
|
|
533
|
+
response["durationSeconds"] = Int(duration.rounded())
|
|
534
|
+
response["speechAuthorization"] = authorizationName(SFSpeechRecognizer.authorizationStatus())
|
|
535
|
+
return response
|
|
536
|
+
}
|
|
537
|
+
|
|
214
538
|
// MARK: - Dispatch
|
|
215
539
|
|
|
216
|
-
let actions = ["hello", "decode_drawing", "encode_drawing"]
|
|
540
|
+
let actions = ["hello", "decode_drawing", "encode_drawing", "transcribe"]
|
|
217
541
|
|
|
218
|
-
func handle(_ request: [String: Any]) throws -> [String: Any] {
|
|
542
|
+
func handle(_ request: [String: Any]) async throws -> [String: Any] {
|
|
219
543
|
guard (request["protocol"] as? NSNumber)?.intValue == protocolVersion else {
|
|
220
544
|
throw HelperFailure(code: "protocol_mismatch", message: "expected protocol \(protocolVersion)")
|
|
221
545
|
}
|
|
@@ -231,6 +555,8 @@ func handle(_ request: [String: Any]) throws -> [String: Any] {
|
|
|
231
555
|
return try decodeDrawing(request)
|
|
232
556
|
case "encode_drawing":
|
|
233
557
|
return try encodeDrawing(request)
|
|
558
|
+
case "transcribe":
|
|
559
|
+
return try await transcribe(request)
|
|
234
560
|
default:
|
|
235
561
|
throw HelperFailure(code: "unknown_action", message: "unknown action")
|
|
236
562
|
}
|
|
@@ -238,7 +564,7 @@ func handle(_ request: [String: Any]) throws -> [String: Any] {
|
|
|
238
564
|
|
|
239
565
|
@main
|
|
240
566
|
struct PublicHelper {
|
|
241
|
-
static func main() {
|
|
567
|
+
static func main() async {
|
|
242
568
|
let input = FileHandle.standardInput.readDataToEndOfFile()
|
|
243
569
|
if input.count > maxInputBytes {
|
|
244
570
|
fail(HelperFailure(code: "invalid_request", message: "request exceeds \(maxInputBytes) bytes"))
|
|
@@ -247,7 +573,7 @@ struct PublicHelper {
|
|
|
247
573
|
fail(HelperFailure(code: "invalid_request", message: "stdin must be one JSON object"))
|
|
248
574
|
}
|
|
249
575
|
do {
|
|
250
|
-
finish(try handle(request))
|
|
576
|
+
finish(try await handle(request))
|
|
251
577
|
} catch let failure as HelperFailure {
|
|
252
578
|
fail(failure)
|
|
253
579
|
} catch {
|
package/package.json
CHANGED