create-pmos 0.114.0 → 0.116.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/payload/KIT-VERSION +1 -1
- package/payload/planning/evals/eval-calibration.md +1 -1
- package/payload/planning/okrs/health-budgets.md +1 -1
- package/payload/state/discoveries.md +1 -1
- package/workspace/guide/notes.md +13 -1
- package/workspace/lib/audio-resample.js +76 -0
- package/workspace/lib/notes-audio.js +87 -19
- package/workspace/lib/notes-echo.js +32 -14
- package/workspace/lib/notes-hires.js +94 -0
- package/workspace/lib/notes-polish.js +75 -0
- package/workspace/lib/routes-notes.js +31 -8
- package/workspace/notes-test-signals.js +65 -0
- package/workspace/package-lock.json +2 -2
- package/workspace/package.json +1 -1
- package/workspace/public/pm/audio-capture.js +20 -6
- package/workspace/public/pm/view-notes.js +43 -10
- package/workspace/test-lists/meeting-notes.json +7 -3
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "create-pmos",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.116.0",
|
|
4
4
|
"description": "Scaffold the PMOS portable kit — run the PMOS product-management workflow (initiative → lane-fit spec → contract → build → three gates, with work-type lanes for features, research, maintenance, incidents, and content) entirely inside your own repo, with all state as git-versioned files. Includes `workflow-sweep`, which audits the state every board derives from and separates real defects from work you simply have not done. `create-pmos workspace` also installs + launches the local Terminal Workspace GUI. No database, no network, no telemetry.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"bin": {
|
package/payload/KIT-VERSION
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
type: Reference
|
|
3
3
|
title: "Eval calibration log"
|
|
4
4
|
description: "Append-only record of every PM override of an evaluator verdict."
|
|
5
|
-
generated: {by: process:create-pmos, at: "2026-10-
|
|
5
|
+
generated: {by: process:create-pmos, at: "2026-10-08T06:17:49Z"}
|
|
6
6
|
status: stable
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
type: Reference
|
|
3
3
|
title: "Health budgets"
|
|
4
4
|
description: "Closed registry of health-budget anchors (D59), edited only by the PM."
|
|
5
|
-
generated: {by: process:create-pmos, at: "2026-10-
|
|
5
|
+
generated: {by: process:create-pmos, at: "2026-10-08T06:17:49Z"}
|
|
6
6
|
status: stable
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
type: Reference
|
|
3
3
|
title: "Discoveries and friction log"
|
|
4
4
|
description: "Append-only record of what runs discovered and where the process rubbed."
|
|
5
|
-
generated: {by: process:create-pmos, at: "2026-10-
|
|
5
|
+
generated: {by: process:create-pmos, at: "2026-10-08T06:17:49Z"}
|
|
6
6
|
status: stable
|
|
7
7
|
---
|
|
8
8
|
|
package/workspace/guide/notes.md
CHANGED
|
@@ -132,8 +132,20 @@ When a meeting was heard with local transcription, its audio is kept on this mac
|
|
|
132
132
|
|
|
133
133
|
When you listened to both your microphone and the meeting, the workspace removes the meeting's echo from your microphone after you press **Stop**. A speaker plays the meeting, your microphone hears it a second time, and this second copy is the echo. The workspace uses the meeting's own recording as the guide. This takes a few minutes for each hour of meeting. It runs in the background, and the replay changes when it is done. Your own voice stays, because it is not in the meeting's recording. A part of the meeting where the clean-up would not help stays as it was recorded. The transcript does not change, because it is made from each source on its own. Headphones are still the best fix, because they stop the echo at its source.
|
|
134
134
|
|
|
135
|
+
The replay of a new note is kept at 24 kHz, so it keeps the high sounds of a voice that a 16 kHz file loses. The transcript is still made from 16 kHz audio, as before. A note you recorded before this change keeps its 16 kHz replay and plays as it did. The 24 kHz file is about 173 MB for each hour, and the workspace deletes it with the note's audio.
|
|
136
|
+
|
|
135
137
|
The replay is your microphone and the meeting added together. When both are loud at the same moment, the workspace lowers the peaks softly, so the sound does not distort. A quiet voice is not changed. The transcript is made from each source on its own, so this does not change what is written.
|
|
136
138
|
|
|
139
|
+
### Clean up the replay sound
|
|
140
|
+
|
|
141
|
+
The replay can play a cleaned copy of the sound. The copy has less rumble and hiss, and an even loudness. It needs **ffmpeg** on your machine. To install it, run `brew install ffmpeg` in a terminal.
|
|
142
|
+
|
|
143
|
+
1. Open the settings. Select **Clean up the replay sound**. It is off until you select it. The setting is for this machine.
|
|
144
|
+
2. After **Stop**, the workspace makes the cleaned copy when the echo is out of the replay. It makes the copy of a new note by itself.
|
|
145
|
+
3. For a note you already have, open the note's settings and press **Clean up this note's sound**. The replay changes when the copy is ready.
|
|
146
|
+
|
|
147
|
+
The workspace makes the copy from the replay only. It does not change the transcript. If ffmpeg is not on your machine, the replay plays as it is, and the settings say that ffmpeg was not found. When you clear the setting, the replay plays the plain sound again, and the cleaned copy stays on your machine until you delete the note's audio.
|
|
148
|
+
|
|
137
149
|
To stop keeping audio, clear **Keep the audio for replay** in the settings. To delete one note's audio, press **Delete this note's audio** in the settings; the notes and the transcript stay. The browser's own recognition keeps no audio, so a meeting heard that way cannot be replayed.
|
|
138
150
|
|
|
139
151
|
## Capture highlights during the meeting
|
|
@@ -195,7 +207,7 @@ To take notes for a calendar event, open the event's detail on the Agenda and pr
|
|
|
195
207
|
|
|
196
208
|
## What stays on your machine
|
|
197
209
|
|
|
198
|
-
- Audio never leaves this machine. The page sends it to the workspace on this machine, which gives it to whisper.cpp in a temporary file and deletes that file at once. For replay, the workspace keeps one recording of each note in `notes/recordings/` in the state folder, until you delete the audio, the note, or mark the note private. The browser's own recognition is used only on the device, never in the cloud.
|
|
210
|
+
- Audio never leaves this machine. The page sends it to the workspace on this machine, which gives it to whisper.cpp in a temporary file and deletes that file at once. For replay, the workspace keeps one recording of each note in `notes/recordings/` in the state folder, with a cleaned copy beside it when you use **Clean up the replay sound**, until you delete the audio, the note, or mark the note private. The cleaned copy is made by ffmpeg on this machine, and nothing is sent anywhere. The browser's own recognition is used only on the device, never in the cloud.
|
|
199
211
|
- Notes are files in the workspace's own folder on this machine (`notes/` in the state folder). They are never written into a repository.
|
|
200
212
|
- Only Jev, with the switches on, and your agent session (the final notes, **Summary by agent**, and the Wispr Flow search) receive any text from a note. The Wispr Flow search receives the meeting's time and title only.
|
|
201
213
|
- The final pass runs Qwen3-ASR, Parakeet, Whisper and the speaker models on this machine, from temporary files that it deletes when it is done.
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
/* audio-resample.js — the server's copy of the low-pass resampler in public/pm/audio-capture.js (notes-audio-quality Run 4).
|
|
3
|
+
*
|
|
4
|
+
* makeResampler(inRate, outRate) → { push(Float32Array|Float64Array) → Int16Array, flush() → Int16Array }
|
|
5
|
+
*
|
|
6
|
+
* The page cannot require() a CommonJS file and the server cannot import() a page module without a warning on every start, so the
|
|
7
|
+
* function exists twice. Its text must stay the same in both files: notes-hires.test.js compares the two and runs them on the same
|
|
8
|
+
* sound. Change one, change the other.
|
|
9
|
+
*/
|
|
10
|
+
const RATE = 16000;
|
|
11
|
+
const CUT = 0.45; // the filter cuts at this share of the output rate
|
|
12
|
+
const CROSSINGS = 16; // zero crossings of the sinc on each side of the centre
|
|
13
|
+
|
|
14
|
+
const toS16 = (v) => { const c = v > 1 ? 1 : v < -1 ? -1 : v; return c < 0 ? c * 0x8000 : c * 0x7fff; };
|
|
15
|
+
|
|
16
|
+
function makeResampler(inRate, outRate = RATE) {
|
|
17
|
+
const ratio = inRate / outRate;
|
|
18
|
+
if (ratio === 1) {
|
|
19
|
+
return { push(x) { const o = new Int16Array(x.length); for (let i = 0; i < x.length; i++) o[i] = toS16(x[i]); return o; }, flush() { return new Int16Array(0); } };
|
|
20
|
+
}
|
|
21
|
+
const fc = CUT / Math.max(1, ratio); // cycles per input sample
|
|
22
|
+
const half = Math.ceil(CROSSINGS / (2 * fc)); // input samples either side of the centre
|
|
23
|
+
let buf = new Float64Array(0); // input not yet used up; buf[0] is input sample `base`
|
|
24
|
+
let base = 0;
|
|
25
|
+
let centre = 0; // the next output sample sits at this input position
|
|
26
|
+
let seen = 0; // input samples received
|
|
27
|
+
const weight = (d) => {
|
|
28
|
+
const x = 2 * fc * d;
|
|
29
|
+
const sinc = x === 0 ? 1 : Math.sin(Math.PI * x) / (Math.PI * x);
|
|
30
|
+
const w = 0.42 + 0.5 * Math.cos(Math.PI * d / half) + 0.08 * Math.cos(2 * Math.PI * d / half); // Blackman, centred
|
|
31
|
+
return sinc * w;
|
|
32
|
+
};
|
|
33
|
+
// The weights depend only on the fractional part of the centre, so they are made once for each fraction (3 or 160 of them for the
|
|
34
|
+
// rates in use) and kept: 48 kHz to 16 kHz, or 24 kHz to 16 kHz, would otherwise compute a sine and two cosines per tap per sample.
|
|
35
|
+
const phases = new Map();
|
|
36
|
+
const phaseOf = (frac) => {
|
|
37
|
+
const key = Math.round(frac * 1e6);
|
|
38
|
+
let ph = phases.get(key);
|
|
39
|
+
if (!ph) {
|
|
40
|
+
const c = Math.floor(frac * 1e6 + 0.5) / 1e6; const lo = Math.ceil(c - half) - 0; const hi = Math.floor(c + half);
|
|
41
|
+
const w = new Float64Array(hi - lo + 1); let norm = 0;
|
|
42
|
+
for (let i = lo; i <= hi; i++) { const k = weight(i - c); w[i - lo] = k; norm += k; }
|
|
43
|
+
for (let i = 0; i < w.length; i++) w[i] /= norm;
|
|
44
|
+
ph = { lo, w };
|
|
45
|
+
if (phases.size < 4096) phases.set(key, ph);
|
|
46
|
+
}
|
|
47
|
+
return ph;
|
|
48
|
+
};
|
|
49
|
+
function run(limit, out) {
|
|
50
|
+
while (centre < limit && Math.floor(centre) + half < base + buf.length) {
|
|
51
|
+
const fl = Math.floor(centre); const ph = phaseOf(centre - fl);
|
|
52
|
+
const first = fl + ph.lo - base; let sum = 0;
|
|
53
|
+
for (let k = 0; k < ph.w.length; k++) { const j = first + k; if (j >= 0 && j < buf.length) sum += buf[j] * ph.w[k]; }
|
|
54
|
+
out.push(toS16(sum));
|
|
55
|
+
centre += ratio;
|
|
56
|
+
}
|
|
57
|
+
}
|
|
58
|
+
function trim() {
|
|
59
|
+
const keep = Math.max(base, Math.floor(centre) - half);
|
|
60
|
+
if (keep > base) { buf = buf.slice(keep - base); base = keep; }
|
|
61
|
+
}
|
|
62
|
+
return {
|
|
63
|
+
push(x) {
|
|
64
|
+
const merged = new Float64Array(buf.length + x.length); merged.set(buf); merged.set(x, buf.length); buf = merged; seen += x.length;
|
|
65
|
+
const out = []; run(Infinity, out); trim();
|
|
66
|
+
return Int16Array.from(out);
|
|
67
|
+
},
|
|
68
|
+
flush() {
|
|
69
|
+
const pad = new Float64Array(buf.length + half + 1); pad.set(buf); buf = pad;
|
|
70
|
+
const out = []; run(seen, out); trim();
|
|
71
|
+
return Int16Array.from(out);
|
|
72
|
+
},
|
|
73
|
+
};
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
module.exports = { makeResampler, RATE };
|
|
@@ -28,27 +28,42 @@ const RATE = 16000;
|
|
|
28
28
|
const BYTES_PER_MS = (RATE * 2) / 1000;
|
|
29
29
|
const ON_TIME_MS = 1500; // a chunk this close to where its source left off is placed straight after it
|
|
30
30
|
const MAX_BYTES = RATE * 2 * 60 * 60 * 6; // six hours; a longer meeting keeps its first six hours
|
|
31
|
+
const HI_RATE = 24000; // the replay may be stored at this rate (D92 as amended 2026-10-08)
|
|
32
|
+
const HI_BYTES_PER_MS = (HI_RATE * 2) / 1000;
|
|
33
|
+
const HI_MAX_BYTES = HI_RATE * 2 * 60 * 60 * 6;
|
|
31
34
|
const ID_RE = /^n-[0-9a-z]{4,16}-[0-9a-f]{4}$/;
|
|
32
35
|
|
|
33
36
|
const dirOf = (runtimeDir) => path.join(runtimeDir, 'notes', 'recordings');
|
|
34
37
|
const fileOf = (runtimeDir, id) => path.join(dirOf(runtimeDir), id + '.pcm');
|
|
35
38
|
const SOURCES = ['mic', 'meeting'];
|
|
39
|
+
const hiFileOf = (runtimeDir, id) => path.join(dirOf(runtimeDir), id + '.hi.pcm');
|
|
40
|
+
const sideFileOf = (runtimeDir, id, suffix) => path.join(dirOf(runtimeDir), id + suffix);
|
|
41
|
+
const cleanFileOf = (runtimeDir, id) => path.join(dirOf(runtimeDir), id + '.clean.pcm');
|
|
36
42
|
const sourceFileOf = (runtimeDir, id, source) => path.join(dirOf(runtimeDir), id + '.' + source + '.pcm');
|
|
37
43
|
const settingsFile = (runtimeDir) => path.join(runtimeDir, 'notes', 'audio-settings.json');
|
|
38
44
|
const next = new Map(); // `${file}|${source}` → the byte offset where that source's last chunk ended
|
|
39
45
|
|
|
40
|
-
function
|
|
41
|
-
|
|
42
|
-
}
|
|
43
|
-
function setKeep(runtimeDir, keep) {
|
|
44
|
-
if (typeof keep !== 'boolean') return { ok: false, error: 'say keep: true or keep: false' };
|
|
46
|
+
function readSettings(runtimeDir) { try { const s = JSON.parse(fs.readFileSync(settingsFile(runtimeDir), 'utf8')); return s && typeof s === 'object' ? s : {}; } catch { return {}; } }
|
|
47
|
+
function writeSettings(runtimeDir, patch) {
|
|
45
48
|
const f = settingsFile(runtimeDir);
|
|
46
49
|
fs.mkdirSync(path.dirname(f), { recursive: true });
|
|
47
50
|
const tmp = f + '.' + process.pid + '.tmp';
|
|
48
|
-
fs.writeFileSync(tmp, JSON.stringify({
|
|
51
|
+
fs.writeFileSync(tmp, JSON.stringify({ ...readSettings(runtimeDir), ...patch }), { mode: 0o600 });
|
|
49
52
|
fs.renameSync(tmp, f);
|
|
53
|
+
}
|
|
54
|
+
function getKeep(runtimeDir) { return readSettings(runtimeDir).keep !== false; }
|
|
55
|
+
function setKeep(runtimeDir, keep) {
|
|
56
|
+
if (typeof keep !== 'boolean') return { ok: false, error: 'say keep: true or keep: false' };
|
|
57
|
+
writeSettings(runtimeDir, { keep });
|
|
50
58
|
return { ok: true, keep };
|
|
51
59
|
}
|
|
60
|
+
/** "Clean up the replay sound" (notes-audio-quality Run 3): off until the PM turns it on, for this machine. */
|
|
61
|
+
function getClean(runtimeDir) { return readSettings(runtimeDir).clean === true; }
|
|
62
|
+
function setClean(runtimeDir, clean) {
|
|
63
|
+
if (typeof clean !== 'boolean') return { ok: false, error: 'say clean: true or clean: false' };
|
|
64
|
+
writeSettings(runtimeDir, { clean });
|
|
65
|
+
return { ok: true, clean };
|
|
66
|
+
}
|
|
52
67
|
|
|
53
68
|
// The mixed file passes every sample through this soft knee (notes-audio-quality R1, R2). Below KNEE the sample is unchanged, so one
|
|
54
69
|
// source alone is not made quieter. Above it the sample bends towards CEILING and never reaches full scale, so two loud sources
|
|
@@ -65,7 +80,7 @@ function softLimit(x) {
|
|
|
65
80
|
/** Add one chunk at meeting time `t`. Samples already there (the other source) are added to, then limited by softLimit. With
|
|
66
81
|
* `mixed: false` only the source's own file is written: the PM keeps no audio for replay, and the final pass deletes it when done.
|
|
67
82
|
* The per-source files are never limited: they are what the transcription reads. */
|
|
68
|
-
function write(runtimeDir, { id, source, pcm, t, mixed = true }) {
|
|
83
|
+
function write(runtimeDir, { id, source, pcm, pcm24 = null, t, mixed = true }) {
|
|
69
84
|
if (!ID_RE.test(String(id || '')) || !pcm || !pcm.length || pcm.length % 2) return { ok: false, error: 'nothing to keep' };
|
|
70
85
|
const f = fileOf(runtimeDir, id);
|
|
71
86
|
const key = f + '|' + source;
|
|
@@ -76,7 +91,10 @@ function write(runtimeDir, { id, source, pcm, t, mixed = true }) {
|
|
|
76
91
|
if (at >= MAX_BYTES) return { ok: true, kept: false };
|
|
77
92
|
const len = Math.min(pcm.length, MAX_BYTES - at);
|
|
78
93
|
fs.mkdirSync(dirOf(runtimeDir), { recursive: true });
|
|
79
|
-
|
|
94
|
+
// With 24 kHz audio the replay is the 24 kHz mix and no 16 kHz mix is written; the 16 kHz source files are written either way
|
|
95
|
+
const hiOk = mixed && pcm24 && pcm24.length && pcm24.length % 2 === 0;
|
|
96
|
+
if (hiOk) writeHi(runtimeDir, id, source, pcm24, t);
|
|
97
|
+
if (mixed && !hiOk) {
|
|
80
98
|
const fd = fs.openSync(f, fs.existsSync(f) ? 'r+' : 'w+', 0o600);
|
|
81
99
|
try {
|
|
82
100
|
const old = Buffer.alloc(len);
|
|
@@ -95,9 +113,38 @@ function write(runtimeDir, { id, source, pcm, t, mixed = true }) {
|
|
|
95
113
|
return { ok: true, kept: true, at_ms: at / BYTES_PER_MS };
|
|
96
114
|
}
|
|
97
115
|
|
|
98
|
-
|
|
116
|
+
/** Add one chunk of 24 kHz audio to the note's 24 kHz mix at meeting time `t`, placed as write() places the 16 kHz audio. */
|
|
117
|
+
function writeHi(runtimeDir, id, source, pcm24, t) {
|
|
118
|
+
const f = hiFileOf(runtimeDir, id);
|
|
119
|
+
const key = f + '|' + source;
|
|
120
|
+
let at = Math.max(0, Math.round((Number(t) || 0) * HI_BYTES_PER_MS / 2) * 2);
|
|
121
|
+
const prev = next.get(key);
|
|
122
|
+
if (prev !== undefined && at <= prev + ON_TIME_MS * HI_BYTES_PER_MS) at = prev;
|
|
123
|
+
if (at >= HI_MAX_BYTES) return;
|
|
124
|
+
const len = Math.min(pcm24.length, HI_MAX_BYTES - at);
|
|
125
|
+
const fd = fs.openSync(f, fs.existsSync(f) ? 'r+' : 'w+', 0o600);
|
|
126
|
+
try {
|
|
127
|
+
const old = Buffer.alloc(len);
|
|
128
|
+
fs.readSync(fd, old, 0, len, at);
|
|
129
|
+
const out = Buffer.alloc(len);
|
|
130
|
+
for (let i = 0; i < len; i += 2) out.writeInt16LE(softLimit(old.readInt16LE(i) + pcm24.readInt16LE(i)), i);
|
|
131
|
+
fs.writeSync(fd, out, 0, len, at);
|
|
132
|
+
} finally { fs.closeSync(fd); }
|
|
133
|
+
next.set(key, at + len);
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
/** The file the replay is made from: the 24 kHz mix when the note has one, else the 16 kHz mix. { file, bytes, rate } | null. */
|
|
137
|
+
function baseOf(runtimeDir, id) {
|
|
99
138
|
if (!ID_RE.test(String(id || ''))) return null;
|
|
100
|
-
|
|
139
|
+
for (const [file, rate] of [[hiFileOf(runtimeDir, id), HI_RATE], [fileOf(runtimeDir, id), RATE]]) {
|
|
140
|
+
try { const st = fs.statSync(file); if (st.size) return { file, bytes: st.size, rate }; } catch { /* none */ }
|
|
141
|
+
}
|
|
142
|
+
return null;
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
function info(runtimeDir, id) {
|
|
146
|
+
const b = baseOf(runtimeDir, id);
|
|
147
|
+
return b ? { bytes: b.bytes, ms: Math.round(b.bytes / (b.rate * 2 / 1000)), rate: b.rate } : null;
|
|
101
148
|
}
|
|
102
149
|
function removeSources(runtimeDir, id) {
|
|
103
150
|
if (!ID_RE.test(String(id || ''))) return;
|
|
@@ -105,10 +152,14 @@ function removeSources(runtimeDir, id) {
|
|
|
105
152
|
}
|
|
106
153
|
function remove(runtimeDir, id) {
|
|
107
154
|
if (!ID_RE.test(String(id || ''))) return false;
|
|
108
|
-
for (const k of [...next.keys()]) if (k.startsWith(fileOf(runtimeDir, id) + '|')) next.delete(k);
|
|
155
|
+
for (const k of [...next.keys()]) if (k.startsWith(fileOf(runtimeDir, id) + '|') || k.startsWith(hiFileOf(runtimeDir, id) + '|')) next.delete(k);
|
|
109
156
|
for (const src of SOURCES) { try { fs.unlinkSync(sourceFileOf(runtimeDir, id, src)); } catch { /* none */ } }
|
|
110
|
-
|
|
111
|
-
|
|
157
|
+
let removed = false;
|
|
158
|
+
for (const extra of ['.echo.json', '.pcm.echo.tmp', '.hi.pcm.echo.tmp', '.hi.pcm', '.clean.pcm', '.clean.pcm.tmp']) {
|
|
159
|
+
try { fs.unlinkSync(path.join(dirOf(runtimeDir), id + extra)); if (extra === '.hi.pcm') removed = true; } catch { /* none */ }
|
|
160
|
+
}
|
|
161
|
+
try { fs.unlinkSync(fileOf(runtimeDir, id)); removed = true; } catch { /* none */ }
|
|
162
|
+
return removed;
|
|
112
163
|
}
|
|
113
164
|
/** The note's audio, one file per source: [{ source, file, bytes, ms }], the sources that were heard. */
|
|
114
165
|
function sources(runtimeDir, id) {
|
|
@@ -121,19 +172,36 @@ function sources(runtimeDir, id) {
|
|
|
121
172
|
return out;
|
|
122
173
|
}
|
|
123
174
|
|
|
124
|
-
|
|
175
|
+
/** The file the replay plays: the cleaned copy when the PM switched cleaning on and the copy is as long as the replay's base (a copy
|
|
176
|
+
* made before listening went on is shorter, so it is stale and the plain replay plays); otherwise the base. { file, bytes, rate,
|
|
177
|
+
* cleaned } | null. */
|
|
178
|
+
function replayOf(runtimeDir, id) {
|
|
179
|
+
const base = baseOf(runtimeDir, id);
|
|
180
|
+
if (!base) return null;
|
|
181
|
+
if (getClean(runtimeDir)) {
|
|
182
|
+
try { const st = fs.statSync(cleanFileOf(runtimeDir, id)); if (st.size === base.bytes) return { file: cleanFileOf(runtimeDir, id), bytes: st.size, rate: base.rate, cleaned: true }; } catch { /* none */ }
|
|
183
|
+
}
|
|
184
|
+
return { file: base.file, bytes: base.bytes, rate: base.rate, cleaned: false };
|
|
185
|
+
}
|
|
186
|
+
/** True when a cleaned copy exists and is as long as the replay's base. */
|
|
187
|
+
function hasClean(runtimeDir, id) {
|
|
188
|
+
const base = baseOf(runtimeDir, id);
|
|
189
|
+
try { return !!base && fs.statSync(cleanFileOf(runtimeDir, id)).size === base.bytes; } catch { return false; }
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
function wavHeader(dataBytes, rate = RATE) {
|
|
125
193
|
const h = Buffer.alloc(44);
|
|
126
194
|
h.write('RIFF', 0); h.writeUInt32LE(36 + dataBytes, 4); h.write('WAVE', 8);
|
|
127
195
|
h.write('fmt ', 12); h.writeUInt32LE(16, 16); h.writeUInt16LE(1, 20); h.writeUInt16LE(1, 22);
|
|
128
|
-
h.writeUInt32LE(
|
|
196
|
+
h.writeUInt32LE(rate, 24); h.writeUInt32LE(rate * 2, 28); h.writeUInt16LE(2, 32); h.writeUInt16LE(16, 34);
|
|
129
197
|
h.write('data', 36); h.writeUInt32LE(dataBytes, 40);
|
|
130
198
|
return h;
|
|
131
199
|
}
|
|
132
200
|
/** The audio as audio/wav. A `Range: bytes=a-b` request gets 206 with that part, which is how the page seeks. */
|
|
133
201
|
function serve(req, res, runtimeDir, id) {
|
|
134
|
-
const inf =
|
|
202
|
+
const inf = replayOf(runtimeDir, id);
|
|
135
203
|
if (!inf) { res.writeHead(404, { 'Content-Type': 'application/json' }); res.end(JSON.stringify({ ok: false, error: 'this note has no audio' })); return; }
|
|
136
|
-
const header = wavHeader(inf.bytes);
|
|
204
|
+
const header = wavHeader(inf.bytes, inf.rate);
|
|
137
205
|
const total = header.length + inf.bytes;
|
|
138
206
|
let start = 0; let end = total - 1; let partial = false;
|
|
139
207
|
const m = /^bytes=(\d*)-(\d*)$/.exec(String(req.headers.range || '').trim());
|
|
@@ -149,7 +217,7 @@ function serve(req, res, runtimeDir, id) {
|
|
|
149
217
|
if (start < header.length) res.write(header.subarray(start, Math.min(header.length, end + 1)));
|
|
150
218
|
const from = Math.max(0, start - header.length); const to = end - header.length;
|
|
151
219
|
if (to < from) { res.end(); return; }
|
|
152
|
-
fs.createReadStream(
|
|
220
|
+
fs.createReadStream(inf.file, { start: from, end: to }).on('error', () => res.end()).pipe(res);
|
|
153
221
|
}
|
|
154
222
|
|
|
155
|
-
module.exports = { write, info, remove, removeSources, sources, serve, getKeep, setKeep, wavHeader, softLimit, replayFile: fileOf, RATE, BYTES_PER_MS, MAX_BYTES };
|
|
223
|
+
module.exports = { write, info, remove, removeSources, sources, serve, getKeep, setKeep, getClean, setClean, hasClean, replayOf, wavHeader, softLimit, replayFile: (dir, id) => (baseOf(dir, id) || { file: fileOf(dir, id) }).file, mixFile: fileOf, hiFile: hiFileOf, sideFile: sideFileOf, baseOf, cleanFile: cleanFileOf, HI_RATE, RATE, BYTES_PER_MS, MAX_BYTES };
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
* cleanStream({ n, readFar, readMic, onChunk }) the canceller, over any two tracks of n samples → a report
|
|
5
5
|
* cleanReplay(runtimeDir, id) rebuild the note's replay file from its two source files → { ok, ... }
|
|
6
6
|
* cleanReplayInWorker(runtimeDir, id) the same, on a worker thread, so the server stays awake → Promise
|
|
7
|
-
* report(runtimeDir, id) what the last clean-up did, or null
|
|
7
|
+
* report(runtimeDir, id) what the last clean-up did (or why it failed), or null
|
|
8
8
|
*
|
|
9
9
|
* THE PROBLEM. The microphone hears the meeting through the speakers, so each voice is recorded twice, the second copy late.
|
|
10
10
|
* lib/notes-audio.js keeps the meeting on its own (<id>.meeting.pcm), which is exactly the signal that leaked into the microphone.
|
|
@@ -26,6 +26,7 @@ const fs = require('fs');
|
|
|
26
26
|
const path = require('path');
|
|
27
27
|
const { Worker } = require('worker_threads');
|
|
28
28
|
const notesAudio = require('./notes-audio');
|
|
29
|
+
const hires = require('./notes-hires');
|
|
29
30
|
|
|
30
31
|
const FS = notesAudio.RATE; // 16000
|
|
31
32
|
const L = 8192; // the echo path kept: 512 ms
|
|
@@ -259,7 +260,7 @@ function cleanStream({ n, readFar, readMic, onChunk }) {
|
|
|
259
260
|
}
|
|
260
261
|
|
|
261
262
|
/* ── the replay file ──────────────────────────────────────────────────────────────────────────── */
|
|
262
|
-
const reportFile = (runtimeDir, id) => notesAudio.
|
|
263
|
+
const reportFile = (runtimeDir, id) => notesAudio.sideFile(runtimeDir, id, '.echo.json');
|
|
263
264
|
function report(runtimeDir, id) { try { return JSON.parse(fs.readFileSync(reportFile(runtimeDir, id), 'utf8')); } catch { return null; } }
|
|
264
265
|
|
|
265
266
|
function readSamples(fd, size, from, count) {
|
|
@@ -282,9 +283,18 @@ function cleanReplay(runtimeDir, id, { beforeCommit } = {}) {
|
|
|
282
283
|
const srcs = notesAudio.sources(runtimeDir, id);
|
|
283
284
|
const mic = srcs.find((s) => s.source === 'mic'); const meet = srcs.find((s) => s.source === 'meeting');
|
|
284
285
|
if (!mic || !meet) return { ok: true, skipped: 'only one source was heard, so there is no echo to take out' };
|
|
285
|
-
|
|
286
|
-
|
|
286
|
+
const base = notesAudio.baseOf(runtimeDir, id);
|
|
287
|
+
if (!base) return { ok: true, skipped: 'the note has no replay file' };
|
|
288
|
+
// A note with a 24 kHz replay is cleaned at 16 kHz, from the source files, and the cleaned speech is then put together with the
|
|
289
|
+
// band above 8 kHz of the 24 kHz recording (lib/notes-hires.js). An older note has a 16 kHz replay and is rebuilt as it always was.
|
|
290
|
+
const hi = base.rate === notesAudio.HI_RATE;
|
|
291
|
+
const mixPath = notesAudio.mixFile(runtimeDir, id); const hiPath = notesAudio.hiFile(runtimeDir, id);
|
|
292
|
+
const out = hi ? hiPath : mixPath; const tmp = mixPath + '.echo.tmp'; const tmpHi = hiPath + '.echo.tmp';
|
|
287
293
|
let fdM = null; let fdX = null; let fdO = null;
|
|
294
|
+
const fail = (error) => {
|
|
295
|
+
for (const f of [tmp, tmpHi]) { try { fs.unlinkSync(f); } catch { /* none */ } }
|
|
296
|
+
return { ok: false, error: String(error).slice(0, 300) };
|
|
297
|
+
};
|
|
288
298
|
try {
|
|
289
299
|
fdM = fs.openSync(mic.file, 'r'); fdX = fs.openSync(meet.file, 'r'); fdO = fs.openSync(tmp, 'w', 0o600);
|
|
290
300
|
const n = Math.floor(Math.max(mic.bytes, meet.bytes) / 2);
|
|
@@ -303,22 +313,30 @@ function cleanReplay(runtimeDir, id, { beforeCommit } = {}) {
|
|
|
303
313
|
},
|
|
304
314
|
});
|
|
305
315
|
fs.closeSync(fdO); fdO = null;
|
|
316
|
+
if (hi) {
|
|
317
|
+
const c = hires.compose({ cleanFile: tmp, hiFile: hiPath, outFile: tmpHi });
|
|
318
|
+
if (!c.ok) { const r = fail(c.error); writeReport(runtimeDir, id, r); return r; }
|
|
319
|
+
}
|
|
306
320
|
if (beforeCommit) beforeCommit(); // the suites use this to make listening go on at the last moment
|
|
307
|
-
// the sources must be as they were: if listening went on meanwhile, the
|
|
308
|
-
const now = notesAudio.sources(runtimeDir, id);
|
|
309
|
-
const same = now.length === srcs.length && now.every((s) => { const o = srcs.find((x) => x.source === s.source); return o && o.bytes === s.bytes; })
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
321
|
+
// the sources and the replay must be as they were: if listening went on meanwhile, the replay as it is is the truth and this one is dropped
|
|
322
|
+
const now = notesAudio.sources(runtimeDir, id); const nowBase = notesAudio.baseOf(runtimeDir, id);
|
|
323
|
+
const same = now.length === srcs.length && now.every((s) => { const o = srcs.find((x) => x.source === s.source); return o && o.bytes === s.bytes; })
|
|
324
|
+
&& (!hi || (nowBase && nowBase.bytes === base.bytes));
|
|
325
|
+
if (!same) return fail('listening went on while the echo was taken out');
|
|
326
|
+
if (hi) { fs.renameSync(tmpHi, hiPath); try { fs.unlinkSync(tmp); } catch { /* none */ } } else fs.renameSync(tmp, mixPath);
|
|
327
|
+
const result = { ok: true, at: new Date().toISOString(), rate: hi ? notesAudio.HI_RATE : notesAudio.RATE, ...rep, took_ms: Date.now() - began };
|
|
328
|
+
writeReport(runtimeDir, id, result);
|
|
314
329
|
return result;
|
|
315
330
|
} catch (e) {
|
|
316
|
-
|
|
317
|
-
return { ok: false, error: String((e && e.message) || e).slice(0, 300) };
|
|
331
|
+
return fail((e && e.message) || e);
|
|
318
332
|
} finally {
|
|
319
333
|
for (const fd of [fdM, fdX, fdO]) if (fd !== null) { try { fs.closeSync(fd); } catch { /* closed */ } }
|
|
320
334
|
}
|
|
321
335
|
}
|
|
336
|
+
/** What the last clean-up did, kept beside the audio so a failure is not silent. */
|
|
337
|
+
function writeReport(runtimeDir, id, result) {
|
|
338
|
+
try { fs.writeFileSync(reportFile(runtimeDir, id), JSON.stringify({ at: new Date().toISOString(), ...result }), { mode: 0o600 }); } catch { /* best effort */ }
|
|
339
|
+
}
|
|
322
340
|
|
|
323
341
|
const running = new Map(); // `${dir}|${id}` → the promise of a clean-up that is under way
|
|
324
342
|
/** cleanReplay on a worker thread. A second call for the same note while one runs gets the same promise. Never rejects. */
|
|
@@ -329,7 +347,7 @@ function cleanReplayInWorker(runtimeDir, id) {
|
|
|
329
347
|
let done = false;
|
|
330
348
|
const w = new Worker(path.join(__dirname, 'notes-echo-worker.js'), { workerData: { runtimeDir, id } });
|
|
331
349
|
const finish = (r) => { if (done) return; done = true; clearTimeout(timer); resolve(r); };
|
|
332
|
-
const timer = setTimeout(() => { w.terminate();
|
|
350
|
+
const timer = setTimeout(() => { w.terminate(); for (const t of [notesAudio.mixFile(runtimeDir, id) + '.echo.tmp', notesAudio.hiFile(runtimeDir, id) + '.echo.tmp']) { try { fs.unlinkSync(t); } catch { /* none */ } } finish({ ok: false, error: 'it took too long' }); }, TIMEOUT_MS);
|
|
333
351
|
w.once('message', finish);
|
|
334
352
|
w.once('error', (e) => finish({ ok: false, error: String((e && e.message) || e).slice(0, 300) }));
|
|
335
353
|
w.once('exit', () => finish({ ok: false, error: 'the worker ended without a result' }));
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
/* notes-hires.js — put the cleaned speech back into the 24 kHz replay (notes-audio-quality Run 4, D92 as amended 2026-10-08).
|
|
3
|
+
*
|
|
4
|
+
* compose({ cleanFile, hiFile, outFile }) → { ok, samples } | { ok: false, error }
|
|
5
|
+
*
|
|
6
|
+
* THE PROBLEM. The echo clean-up (lib/notes-echo.js) works at 16 kHz, on the two source files, so it cleans the band below 8 kHz.
|
|
7
|
+
* The replay the PM hears is the 24 kHz mix, which also holds the band from 8 to 12 kHz: the air and the hiss of the voices. Rebuilding
|
|
8
|
+
* the replay from the 16 kHz clean-up alone would throw that band away and bring the replay back to 16 kHz quality.
|
|
9
|
+
*
|
|
10
|
+
* THE METHOD. Split the 24 kHz mix at the filter's edge (7.2 kHz): low = the mix taken down to 16 kHz and back up to 24 kHz, high = the
|
|
11
|
+
* mix minus low. The replay is then the cleaned 16 kHz mix taken up to 24 kHz, plus high. The low band is the cleaned speech; the high
|
|
12
|
+
* band is exactly what was recorded. Running it again on its own output changes nothing, because the high band of the result is the
|
|
13
|
+
* high band of the recording, so a second final pass (Listen again) can use it.
|
|
14
|
+
*
|
|
15
|
+
* WHAT IT NEVER DOES. It reads the two files and writes the output file; the caller renames it into place. It does not touch the
|
|
16
|
+
* 16 kHz source files. It opens no network module.
|
|
17
|
+
*/
|
|
18
|
+
const fs = require('fs');
|
|
19
|
+
const resample = require('./audio-resample');
|
|
20
|
+
const { softLimit } = require('./notes-audio');
|
|
21
|
+
|
|
22
|
+
const HI = 24000;
|
|
23
|
+
const LO = 16000;
|
|
24
|
+
const BLOCK_HI = 480000; // 20 s of the 24 kHz mix at a time
|
|
25
|
+
const BLOCK_LO = BLOCK_HI * LO / HI;
|
|
26
|
+
const TOLERANCE_S = 0.5; // the two files may differ in length by this much: more means they are not the same recording
|
|
27
|
+
|
|
28
|
+
const toFloat = (i16) => { const f = new Float32Array(i16.length); for (let i = 0; i < i16.length; i++) f[i] = i16[i] / 32768; return f; };
|
|
29
|
+
|
|
30
|
+
/** A queue of int16 samples that grows at the back and is eaten from the front. */
|
|
31
|
+
function queue() {
|
|
32
|
+
let a = new Int16Array(0);
|
|
33
|
+
return { get length() { return a.length; }, push(x) { const m = new Int16Array(a.length + x.length); m.set(a); m.set(x, a.length); a = m; },
|
|
34
|
+
take(n) { const out = a.subarray(0, n); a = a.slice(n); return out; } };
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
function readBlock(fd, size, from, count) {
|
|
38
|
+
const bytes = Math.max(0, Math.min(count * 2, size - from * 2));
|
|
39
|
+
const out = new Int16Array(bytes / 2);
|
|
40
|
+
if (bytes <= 0) return out;
|
|
41
|
+
const buf = Buffer.alloc(bytes);
|
|
42
|
+
fs.readSync(fd, buf, 0, bytes, from * 2);
|
|
43
|
+
for (let i = 0; i < out.length; i++) out[i] = buf.readInt16LE(i * 2);
|
|
44
|
+
return out;
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function compose({ cleanFile, hiFile, outFile }) {
|
|
48
|
+
let fdC = null; let fdH = null; let fdO = null;
|
|
49
|
+
try {
|
|
50
|
+
const sizeC = fs.statSync(cleanFile).size; const sizeH = fs.statSync(hiFile).size;
|
|
51
|
+
const nC = Math.floor(sizeC / 2); const nH = Math.floor(sizeH / 2);
|
|
52
|
+
if (!nH) return { ok: false, error: 'the 24 kHz recording is empty' };
|
|
53
|
+
if (Math.abs(nC / LO - nH / HI) > TOLERANCE_S) return { ok: false, error: 'the two recordings differ in length by more than ' + TOLERANCE_S + ' s' };
|
|
54
|
+
fdC = fs.openSync(cleanFile, 'r'); fdH = fs.openSync(hiFile, 'r'); fdO = fs.openSync(outFile, 'w', 0o600);
|
|
55
|
+
const down = resample.makeResampler(HI, LO); const up = resample.makeResampler(LO, HI); const upClean = resample.makeResampler(LO, HI);
|
|
56
|
+
const raw = queue(); const low = queue(); const clean = queue();
|
|
57
|
+
let written = 0;
|
|
58
|
+
const emit = (n) => {
|
|
59
|
+
if (n <= 0) return;
|
|
60
|
+
const r = raw.take(n); const l = low.take(n); const c = clean.take(n);
|
|
61
|
+
const buf = Buffer.alloc(n * 2);
|
|
62
|
+
for (let i = 0; i < n; i++) buf.writeInt16LE(softLimit(c[i] + (r[i] - l[i])), i * 2);
|
|
63
|
+
fs.writeSync(fdO, buf, 0, buf.length, written * 2); written += n;
|
|
64
|
+
};
|
|
65
|
+
const drain = () => emit(Math.min(raw.length, low.length, clean.length));
|
|
66
|
+
for (let a = 0, b = 0; a < nH || b < nC; a += BLOCK_HI, b += BLOCK_LO) {
|
|
67
|
+
const h = readBlock(fdH, sizeH, a, BLOCK_HI); const c = readBlock(fdC, sizeC, b, BLOCK_LO);
|
|
68
|
+
raw.push(h);
|
|
69
|
+
low.push(up.push(toFloat(down.push(toFloat(h)))));
|
|
70
|
+
clean.push(upClean.push(toFloat(c)));
|
|
71
|
+
drain();
|
|
72
|
+
}
|
|
73
|
+
low.push(up.push(toFloat(down.flush()))); low.push(up.flush());
|
|
74
|
+
clean.push(upClean.flush());
|
|
75
|
+
drain();
|
|
76
|
+
// the last few samples may lack a partner (the filters stop a hair short): the band-limited parts are then taken as silent
|
|
77
|
+
const rest = raw.length;
|
|
78
|
+
if (rest > 0) {
|
|
79
|
+
if (rest > 8) return { ok: false, error: 'the recordings did not line up to the end' };
|
|
80
|
+
const r = raw.take(rest); const l = low.take(Math.min(rest, low.length)); const c = clean.take(Math.min(rest, clean.length));
|
|
81
|
+
const buf = Buffer.alloc(rest * 2);
|
|
82
|
+
for (let i = 0; i < rest; i++) buf.writeInt16LE(softLimit((i < c.length ? c[i] : 0) + (i < l.length ? r[i] - l[i] : 0)), i * 2);
|
|
83
|
+
fs.writeSync(fdO, buf, 0, buf.length, written * 2); written += rest;
|
|
84
|
+
}
|
|
85
|
+
fs.closeSync(fdO); fdO = null;
|
|
86
|
+
return { ok: true, samples: written };
|
|
87
|
+
} catch (e) {
|
|
88
|
+
return { ok: false, error: String((e && e.message) || e).slice(0, 300) };
|
|
89
|
+
} finally {
|
|
90
|
+
for (const fd of [fdC, fdH, fdO]) if (fd !== null) { try { fs.closeSync(fd); } catch { /* closed */ } }
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
module.exports = { compose, HI, LO };
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
/* notes-polish.js — an optional cleaned copy of a note's replay (notes-audio-quality Run 3). Local only.
|
|
3
|
+
*
|
|
4
|
+
* detect({ env? }) → { bin } | null the ffmpeg the PM installed
|
|
5
|
+
* build(runtimeDir, id, { run?, bin?, force? }) → Promise<{ ok, ... }> make <id>.clean.pcm from the replay mix
|
|
6
|
+
* (`bin: null` means no ffmpeg; leave bin out to look for one)
|
|
7
|
+
*
|
|
8
|
+
* WHAT IT DOES. When the PM turned "Clean up the replay sound" on and has ffmpeg, the server runs ffmpeg on the note's replay mix:
|
|
9
|
+
* a high-pass filter at 80 Hz (rumble and desk thumps), light noise reduction, and loudness normalization to -16 LUFS with
|
|
10
|
+
* a true peak of -1.5 dB. The result is a second file next to the mix. The replay plays it while the setting is on and the copy is
|
|
11
|
+
* as long as the mix; with the setting off, or with no copy, it plays the plain mix.
|
|
12
|
+
*
|
|
13
|
+
* WHAT IT NEVER DOES. It reads only the replay mix, never the two source files, so the transcription input is not touched. It runs
|
|
14
|
+
* a program the PM installed (as whisper-cli is run), downloads nothing, installs nothing, and opens no network module. With ffmpeg
|
|
15
|
+
* missing it does nothing and says so. The copy is deleted with the note's audio.
|
|
16
|
+
*/
|
|
17
|
+
const fs = require('fs');
|
|
18
|
+
const path = require('path');
|
|
19
|
+
const notesAudio = require('./notes-audio');
|
|
20
|
+
const { runProgram } = require('./notes-final');
|
|
21
|
+
|
|
22
|
+
const FILTER = 'highpass=f=80,afftdn=nr=10:nf=-45,loudnorm=I=-16:TP=-1.5:LRA=11';
|
|
23
|
+
const TIMEOUT_MS = 20 * 60 * 1000;
|
|
24
|
+
const LENGTH_SLACK = 0.02; // the copy may differ from the mix by this share of its length, no more
|
|
25
|
+
|
|
26
|
+
const isExec = (p) => { try { fs.accessSync(p, fs.constants.X_OK); return fs.statSync(p).isFile(); } catch { return false; } };
|
|
27
|
+
|
|
28
|
+
/** The ffmpeg on this machine: PMOS_NOTES_FFMPEG_BIN, else the PATH, else where Homebrew puts it. */
|
|
29
|
+
function detect({ env = process.env, fallbacks = ['/opt/homebrew/bin', '/usr/local/bin', '/usr/bin'] } = {}) {
|
|
30
|
+
const named = env.PMOS_NOTES_FFMPEG_BIN;
|
|
31
|
+
if (named) return isExec(named) ? { bin: named } : null;
|
|
32
|
+
const dirs = String(env.PATH || '').split(path.delimiter).concat(fallbacks);
|
|
33
|
+
for (const d of dirs) { if (!d) continue; const f = path.join(d, 'ffmpeg'); if (isExec(f)) return { bin: f }; }
|
|
34
|
+
return null;
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
const running = new Map();
|
|
38
|
+
/** Make the cleaned copy of a note's replay. Resolves with { ok: true, ... } or { ok: false, error } or { ok: true, skipped }. Never rejects. */
|
|
39
|
+
function build(runtimeDir, id, opts = {}) {
|
|
40
|
+
const { run = runProgram, force = false } = opts;
|
|
41
|
+
const key = runtimeDir + '|' + id;
|
|
42
|
+
if (running.has(key)) return running.get(key);
|
|
43
|
+
const p = (async () => {
|
|
44
|
+
if (!force && !notesAudio.getClean(runtimeDir)) return { ok: true, skipped: 'cleaning the replay sound is off' };
|
|
45
|
+
const found = 'bin' in opts ? (opts.bin ? { bin: opts.bin } : null) : detect();
|
|
46
|
+
if (!found) return { ok: true, skipped: 'ffmpeg was not found on this machine' };
|
|
47
|
+
const inf = notesAudio.info(runtimeDir, id);
|
|
48
|
+
if (!inf) return { ok: true, skipped: 'the note has no replay file' };
|
|
49
|
+
const input = notesAudio.replayFile(runtimeDir, id); const out = notesAudio.cleanFile(runtimeDir, id); const tmp = out + '.tmp';
|
|
50
|
+
const began = Date.now();
|
|
51
|
+
try {
|
|
52
|
+
const r = await run(found.bin, ['-nostdin', '-hide_banner', '-loglevel', 'error', '-y', '-f', 's16le', '-ar', String(inf.rate), '-ac', '1', '-i', input,
|
|
53
|
+
'-af', FILTER, '-f', 's16le', '-ar', String(inf.rate), '-ac', '1', tmp], { timeout: TIMEOUT_MS });
|
|
54
|
+
if (!r.ok) { try { fs.unlinkSync(tmp); } catch { /* none */ } return { ok: false, error: r.error || 'ffmpeg failed' }; }
|
|
55
|
+
const size = fs.statSync(tmp).size;
|
|
56
|
+
const now = notesAudio.info(runtimeDir, id);
|
|
57
|
+
if (!now || now.bytes !== inf.bytes) { fs.unlinkSync(tmp); return { ok: false, error: 'listening went on while the sound was cleaned up' }; }
|
|
58
|
+
if (Math.abs(size - inf.bytes) > LENGTH_SLACK * inf.bytes) { fs.unlinkSync(tmp); return { ok: false, error: 'the cleaned copy came out a different length' }; }
|
|
59
|
+
// the copy has the mix's length to the byte, so the replay can seek in it with the same times
|
|
60
|
+
if (size !== inf.bytes) {
|
|
61
|
+
const fd = fs.openSync(tmp, 'r+');
|
|
62
|
+
try { if (size > inf.bytes) fs.ftruncateSync(fd, inf.bytes); else fs.writeSync(fd, Buffer.alloc(inf.bytes - size), 0, inf.bytes - size, size); } finally { fs.closeSync(fd); }
|
|
63
|
+
}
|
|
64
|
+
fs.renameSync(tmp, out);
|
|
65
|
+
return { ok: true, bytes: inf.bytes, took_ms: Date.now() - began };
|
|
66
|
+
} catch (e) {
|
|
67
|
+
try { fs.unlinkSync(tmp); } catch { /* none */ }
|
|
68
|
+
return { ok: false, error: String((e && e.message) || e).slice(0, 300) };
|
|
69
|
+
}
|
|
70
|
+
})().finally(() => running.delete(key));
|
|
71
|
+
running.set(key, p);
|
|
72
|
+
return p;
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
module.exports = { detect, build, FILTER };
|
|
@@ -25,6 +25,7 @@
|
|
|
25
25
|
* POST /api/notes/line { id, index, cat?, summary?, tags?, people?, text? } → { ok, segment, summary } the PM's correction (Run 3)
|
|
26
26
|
* GET /api/notes/recording?id=… → audio/wav, with byte ranges the note's kept audio (Run 3, D92)
|
|
27
27
|
* POST /api/notes/recording-delete { id } → { ok } deletes the note's audio and keeps the rest
|
|
28
|
+
* POST /api/notes/clean-audio { id } → { ok, started } makes the cleaned copy of the replay now (Run 3 of notes-audio-quality; needs ffmpeg)
|
|
28
29
|
* GET /api/notes/engine?ws=… → { ok, whisper, jev } is Whisper set up; which Jev gates hold (booleans only)
|
|
29
30
|
* POST /api/notes/switch { ws, jev?, check? } → { ok, jev } the two D91 switches, default off
|
|
30
31
|
* { language?, model?, accuracy?, keep_audio? } → { ok, whisper } the language, the model, the decoding
|
|
@@ -67,6 +68,7 @@ const whisper = require('./notes-whisper.js');
|
|
|
67
68
|
const notesRepo = require('./notes-repo.js');
|
|
68
69
|
const notesAudio = require('./notes-audio.js');
|
|
69
70
|
const notesEcho = require('./notes-echo.js');
|
|
71
|
+
const notesPolish = require('./notes-polish.js');
|
|
70
72
|
const notesFinal = require('./notes-final.js');
|
|
71
73
|
const notesQwen = require('./notes-qwen.js');
|
|
72
74
|
const notesLinks = require('./notes-links.js');
|
|
@@ -82,7 +84,7 @@ const VOCAB_TTL_MS = 60 * 1000;
|
|
|
82
84
|
const prefixes = ['/api/notes'];
|
|
83
85
|
const POSTS = new Set(['/api/notes/new', '/api/notes/save', '/api/notes/dictate', '/api/notes/record', '/api/notes/transcript',
|
|
84
86
|
'/api/notes/private', '/api/notes/pause', '/api/notes/summarize', '/api/notes/summarize-agent', '/api/notes/delete', '/api/notes/audio', '/api/notes/label',
|
|
85
|
-
'/api/notes/speaker', '/api/notes/switch', '/api/notes/whisper-setup', '/api/notes/final', '/api/notes/final-setup', '/api/notes/line', '/api/notes/recording-delete', '/api/notes/link', '/api/notes/bullet', '/api/notes/highlight', '/api/notes/highlight-remove', '/api/notes/final-edit', '/api/notes/wispr-fetch']);
|
|
87
|
+
'/api/notes/speaker', '/api/notes/switch', '/api/notes/whisper-setup', '/api/notes/final', '/api/notes/final-setup', '/api/notes/line', '/api/notes/recording-delete', '/api/notes/clean-audio', '/api/notes/link', '/api/notes/bullet', '/api/notes/highlight', '/api/notes/highlight-remove', '/api/notes/final-edit', '/api/notes/wispr-fetch']);
|
|
86
88
|
|
|
87
89
|
/* ── labels and checks, one note at a time ───────────────────────────────────────────────────── */
|
|
88
90
|
const chains = new Map(); // note id → the promise of its work so far (minute batches, imports, labels, checks)
|
|
@@ -355,7 +357,7 @@ function engineInfo(ctx, w) {
|
|
|
355
357
|
const out = { ready: live.ready, live: live.ready ? live.engine : null, missing: d.missing, setup: d.setup, model_dir: d.model_dir, model_size: d.model_size,
|
|
356
358
|
language: whisper.getLanguage(ctx.runtimeDir), languages: whisper.LANGUAGES.map(([value, label]) => ({ value, label })),
|
|
357
359
|
model: whisper.getModel(ctx.runtimeDir), models: whisper.MODELS.map((m) => ({ value: m.id, label: m.label })),
|
|
358
|
-
accuracy: whisper.getAccuracy(ctx.runtimeDir), accuracies: whisper.ACCURACIES.map((a) => ({ value: a.id, label: a.label })), keep_audio: notesAudio.getKeep(ctx.runtimeDir),
|
|
360
|
+
accuracy: whisper.getAccuracy(ctx.runtimeDir), accuracies: whisper.ACCURACIES.map((a) => ({ value: a.id, label: a.label })), keep_audio: notesAudio.getKeep(ctx.runtimeDir), clean_audio: notesAudio.getClean(ctx.runtimeDir), ffmpeg: !!notesPolish.detect(),
|
|
359
361
|
final: finalInfo(ctx), wispr: wisprOn(ctx) };
|
|
360
362
|
if (!w) return { whisper: out, jev: null };
|
|
361
363
|
const repo = tsSettings.getRepo(ctx.runtimeDir, w.root);
|
|
@@ -393,7 +395,9 @@ function finalFor(ctx) {
|
|
|
393
395
|
const dir = ctx.runtimeDir;
|
|
394
396
|
const r = notes.setFinal(dir, { id, final });
|
|
395
397
|
if (!notesAudio.getKeep(dir)) notesAudio.removeSources(dir, id);
|
|
396
|
-
else
|
|
398
|
+
else { // best effort, off the main thread: on any failure the replay keeps its plain mix. The echo comes out first, then the sound is cleaned up
|
|
399
|
+
(echoOff(ctx) ? Promise.resolve() : notesEcho.cleanReplayInWorker(dir, id)).then(() => notesPolish.build(dir, id)).catch(() => {});
|
|
400
|
+
}
|
|
397
401
|
if (!r.ok) return;
|
|
398
402
|
const cur = notes.read(dir, id);
|
|
399
403
|
const w = cur.ok && cur.note.ws ? ctx.getWs(cur.note.ws) : null;
|
|
@@ -544,7 +548,7 @@ async function handle(req, res, url, ctx) {
|
|
|
544
548
|
// the page is open: the agent that wrote the notes before Wispr Flow's arrived writes them again, merged
|
|
545
549
|
if (r.ok && mergeWispr(ctx, nid)) r = notes.read(dir, nid);
|
|
546
550
|
if (r.ok) {
|
|
547
|
-
r.audio = { pending: transcriber ? transcriber.pending() : 0, error: audioErrors.get(nid) || null }; r.recording = notesAudio.info(dir, nid);
|
|
551
|
+
r.audio = { pending: transcriber ? transcriber.pending() : 0, error: audioErrors.get(nid) || null }; r.recording = notesAudio.info(dir, nid); if (r.recording) r.recording = { ...r.recording, cleaned: notesAudio.hasClean(dir, nid) };
|
|
548
552
|
if (!Array.isArray(r.note.bullets)) r.note.bullets = [];
|
|
549
553
|
r.bullets_state = bulletsState(ctx, r.note);
|
|
550
554
|
}
|
|
@@ -650,13 +654,21 @@ async function handle(req, res, url, ctx) {
|
|
|
650
654
|
if (b64.length > Math.ceil(MAX_PCM_BYTES / 3) * 4) return done({ ok: false, status: 413, error: 'that audio chunk is too long' });
|
|
651
655
|
const pcm = Buffer.from(b64, 'base64');
|
|
652
656
|
if (pcm.length % 2) return done({ ok: false, status: 400, error: 'the audio must be 16-bit samples' });
|
|
657
|
+
// the same time at 24 kHz, for the replay (D92 as amended 2026-10-08); the transcription never reads it
|
|
658
|
+
const b24 = typeof body.pcm24 === 'string' ? body.pcm24 : '';
|
|
659
|
+
if (b24.length > Math.ceil((MAX_PCM_BYTES * 1.5) / 3) * 4) return done({ ok: false, status: 413, error: 'that audio chunk is too long' });
|
|
660
|
+
const pcm24 = b24 ? Buffer.from(b24, 'base64') : null;
|
|
661
|
+
if (pcm24 && pcm24.length % 2) return done({ ok: false, status: 400, error: 'the audio must be 16-bit samples' });
|
|
653
662
|
if (!liveDetect(ctx).ready) return done({ ok: false, status: 409, error: 'local transcription is not set up' });
|
|
654
663
|
const ms = (pcm.length / 2 / whisper.RATE) * 1000;
|
|
655
664
|
const t = Math.max(0, notes.meetingMs(note.note) - ms);
|
|
656
665
|
// the audio is kept for replay unless the PM switched that off (D92); a private note never reaches this line
|
|
657
666
|
arm(ctx, id);
|
|
658
667
|
// each source is also kept apart for the final pass after Stop, which deletes it when the PM keeps no audio (Run 9)
|
|
659
|
-
|
|
668
|
+
// a 24 kHz chunk must cover the same time as the 16 kHz chunk (to a tenth of a second); one that does not is not kept
|
|
669
|
+
const sameTime = !!pcm24 && pcm24.length > 0 && Math.abs(pcm24.length / 2 / notesAudio.HI_RATE - pcm.length / 2 / whisper.RATE) <= 0.1;
|
|
670
|
+
if (pcm24 && pcm24.length && !sameTime) audioErrors.set(id, { at: new Date().toISOString(), error: 'the 24 kHz audio of one chunk did not match its 16 kHz audio, so the replay has a gap there' });
|
|
671
|
+
{ try { notesAudio.write(dir, { id, source, pcm, pcm24: sameTime ? pcm24 : null, t, mixed: notesAudio.getKeep(dir) }); } catch (e) { audioErrors.set(id, { at: new Date().toISOString(), error: 'the audio was not kept: ' + ((e && e.message) || e) }); } }
|
|
660
672
|
return done(transcriberFor(ctx).push({ id, source, pcm, t, final: body.final === true }));
|
|
661
673
|
}
|
|
662
674
|
if (p === '/api/notes/label') return done(notes.setLabel(dir, { id, index: Number(body.index), cat: body.cat, drop: body.drop === true }));
|
|
@@ -680,22 +692,33 @@ async function handle(req, res, url, ctx) {
|
|
|
680
692
|
if (!r.ok) return done(r);
|
|
681
693
|
return done({ ok: true, deleted: notesAudio.remove(dir, id) });
|
|
682
694
|
}
|
|
695
|
+
if (p === '/api/notes/clean-audio') {
|
|
696
|
+
// the PM asks for the cleaned copy of one note now; the work goes on in the background and the replay changes when it ends
|
|
697
|
+
const r = notes.read(dir, id);
|
|
698
|
+
if (!r.ok) return done(r);
|
|
699
|
+
if (!notesAudio.getClean(dir)) return done({ ok: false, status: 409, error: 'Clean up the replay sound is switched off in Settings' });
|
|
700
|
+
if (!notesPolish.detect()) return done({ ok: false, status: 409, error: 'ffmpeg was not found on this machine' });
|
|
701
|
+
if (!notesAudio.info(dir, id)) return done({ ok: false, status: 404, error: 'this note has no audio' });
|
|
702
|
+
notesPolish.build(dir, id);
|
|
703
|
+
return done({ ok: true, started: true });
|
|
704
|
+
}
|
|
683
705
|
if (p === '/api/notes/switch') {
|
|
684
706
|
const o = wsOf();
|
|
685
707
|
// the spoken language belongs to this machine, not to a repository
|
|
686
708
|
if (body.wispr !== undefined) {
|
|
687
709
|
if (typeof body.wispr !== 'boolean') { ctx.json(res, { ok: false, error: 'say wispr: true or wispr: false' }, 400); return true; }
|
|
688
710
|
setWisprOn(dir, body.wispr);
|
|
689
|
-
if (body.language === undefined && body.model === undefined && body.accuracy === undefined && body.keep_audio === undefined && body.jev === undefined && body.check === undefined) {
|
|
711
|
+
if (body.language === undefined && body.model === undefined && body.accuracy === undefined && body.keep_audio === undefined && body.clean_audio === undefined && body.jev === undefined && body.check === undefined) {
|
|
690
712
|
ctx.json(res, { ok: true, ...engineInfo(ctx, o.ok ? o.w : null) }); return true;
|
|
691
713
|
}
|
|
692
714
|
}
|
|
693
|
-
if (body.language !== undefined || body.model !== undefined || body.accuracy !== undefined || body.keep_audio !== undefined) {
|
|
715
|
+
if (body.language !== undefined || body.model !== undefined || body.accuracy !== undefined || body.keep_audio !== undefined || body.clean_audio !== undefined) {
|
|
694
716
|
const r = body.language !== undefined ? whisper.setLanguage(dir, String(body.language)) : { ok: true };
|
|
695
717
|
const m = r.ok && body.model !== undefined ? whisper.setModel(dir, String(body.model)) : { ok: true };
|
|
696
718
|
const a = r.ok && m.ok && body.accuracy !== undefined ? whisper.setAccuracy(dir, String(body.accuracy)) : { ok: true };
|
|
697
719
|
const k = r.ok && m.ok && a.ok && body.keep_audio !== undefined ? notesAudio.setKeep(dir, body.keep_audio) : { ok: true };
|
|
698
|
-
|
|
720
|
+
const c = r.ok && m.ok && a.ok && k.ok && body.clean_audio !== undefined ? notesAudio.setClean(dir, body.clean_audio) : { ok: true };
|
|
721
|
+
if (!r.ok || !m.ok || !a.ok || !k.ok || !c.ok) { ctx.json(res, { ok: false, error: r.error || m.error || a.error || k.error || c.error }, 400); return true; }
|
|
699
722
|
if (body.jev === undefined && body.check === undefined) { ctx.json(res, { ok: true, ...engineInfo(ctx, o.ok ? o.w : null) }); return true; }
|
|
700
723
|
}
|
|
701
724
|
if (!o.ok || !o.w) { ctx.json(res, { ok: false, error: 'This repository is not registered.' }, 404); return true; }
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
/* notes-test-signals.js — synthetic sound for the notes audio suites (notes-echo.test.js, notes-hires.test.js). Not a test itself.
|
|
3
|
+
*
|
|
4
|
+
* speechLike(seed, secs, talkShare) → { x, active } noise through three resonances, in phrases with pauses
|
|
5
|
+
* makePath(seed, delayMs, gainDb) an echo path: a direct arrival and a decaying tail
|
|
6
|
+
* conv(x, h) conv by overlap-add over FFTs
|
|
7
|
+
* scene({ secs, delayMs, gainDb, tanh, change, noiseDb, seed }) → { far, near, mic, echo, n }
|
|
8
|
+
* the meeting `far`, the PM `near`, and the microphone that hears both
|
|
9
|
+
*/
|
|
10
|
+
const E = require('./lib/notes-echo');
|
|
11
|
+
|
|
12
|
+
const FS = 16000;
|
|
13
|
+
function rng(seed) { let a = seed >>> 0; return () => { a |= 0; a = a + 0x6D2B79F5 | 0; let t = Math.imul(a ^ a >>> 15, 1 | a); t = t + Math.imul(t ^ t >>> 7, 61 | t) ^ t; return ((t ^ t >>> 14) >>> 0) / 4294967296; }; }
|
|
14
|
+
const gauss = (r) => Math.sqrt(-2 * Math.log(r() + 1e-12)) * Math.cos(2 * Math.PI * r());
|
|
15
|
+
/** Speech-like: noise through three resonances, in phrases of 1 to 3 s with pauses. { x, active }. */
|
|
16
|
+
function speechLike(seed, secs, talkShare) {
|
|
17
|
+
const r = rng(seed); const n = secs * FS; const x = new Float64Array(n); const active = new Uint8Array(n);
|
|
18
|
+
const res = [[500, 80], [1500, 120], [2600, 200]].map(([f, bw]) => { const R = Math.exp(-Math.PI * bw / FS); return { a1: 2 * R * Math.cos(2 * Math.PI * f / FS), a2: -R * R, y1: 0, y2: 0 }; });
|
|
19
|
+
let t = 0;
|
|
20
|
+
while (t < n) {
|
|
21
|
+
const talk = Math.round((1 + 2 * r()) * FS); const quiet = Math.round(talk * (1 - talkShare) / talkShare * (0.5 + r())); const syll = 3 + 3 * r();
|
|
22
|
+
for (let i = 0; i < talk && t + i < n; i++) {
|
|
23
|
+
const env = 0.55 + 0.45 * Math.sin(2 * Math.PI * syll * i / FS) ** 2; const edge = Math.min(1, i / 800, (talk - i) / 800); const e = gauss(r);
|
|
24
|
+
let y = 0; for (const q of res) { const o = e * 0.2 + q.a1 * q.y1 + q.a2 * q.y2; q.y2 = q.y1; q.y1 = o; y += o; }
|
|
25
|
+
x[t + i] = y * env * edge * 0.04; active[t + i] = 1;
|
|
26
|
+
}
|
|
27
|
+
t += talk + quiet;
|
|
28
|
+
}
|
|
29
|
+
let pk = 0; for (let i = 0; i < n; i++) pk = Math.max(pk, Math.abs(x[i]));
|
|
30
|
+
for (let i = 0; i < n; i++) x[i] = x[i] / pk * 0.4;
|
|
31
|
+
return { x, active };
|
|
32
|
+
}
|
|
33
|
+
/** An echo path: a direct arrival after delayMs, then a decaying tail, rolled off at the top. */
|
|
34
|
+
function makePath(seed, delayMs, gainDb) {
|
|
35
|
+
const d = Math.round(delayMs * FS / 1000); const len = d + 4800; const r = rng(seed); const h = new Float64Array(len); const g = 10 ** (gainDb / 20);
|
|
36
|
+
for (let i = 0; i < len - d; i++) h[d + i] = (i === 0 ? 1 : 0.35 * gauss(r) * Math.exp(-i / FS / 0.08)) * g;
|
|
37
|
+
let y = 0; for (let i = 0; i < len; i++) { y = 0.55 * y + 0.45 * h[i]; h[i] = y; }
|
|
38
|
+
return h;
|
|
39
|
+
}
|
|
40
|
+
/** conv(x, h) by overlap-add over FFTs. */
|
|
41
|
+
function conv(x, h) {
|
|
42
|
+
const F = 16384; const B = F - h.length; if (B <= 0) throw new Error('path too long');
|
|
43
|
+
const out = new Float64Array(x.length); const Hr = new Float64Array(F); const Hi = new Float64Array(F); Hr.set(h); E.fft(Hr, Hi, false);
|
|
44
|
+
for (let s = 0; s < x.length; s += B) {
|
|
45
|
+
const re = new Float64Array(F); const im = new Float64Array(F);
|
|
46
|
+
for (let i = 0; i < B && s + i < x.length; i++) re[i] = x[s + i];
|
|
47
|
+
E.fft(re, im, false);
|
|
48
|
+
for (let k = 0; k < F; k++) { const a = re[k] * Hr[k] - im[k] * Hi[k]; im[k] = re[k] * Hi[k] + im[k] * Hr[k]; re[k] = a; }
|
|
49
|
+
E.fft(re, im, true);
|
|
50
|
+
for (let i = 0; i < F && s + i < out.length; i++) out[s + i] += re[i];
|
|
51
|
+
}
|
|
52
|
+
return out;
|
|
53
|
+
}
|
|
54
|
+
/** A scene: the meeting `far`, the PM `near`, and the microphone that hears both, plus the echo alone. */
|
|
55
|
+
function scene({ secs, delayMs = 120, gainDb = -9, tanh = 0, change = null, noiseDb = -50, seed = 1 }) {
|
|
56
|
+
const far = speechLike(11 + seed, secs, 0.65); const near = speechLike(29 + seed, secs, 0.45); const n = far.x.length;
|
|
57
|
+
const driven = tanh ? Float64Array.from(far.x, (v) => Math.tanh(tanh * v) / tanh) : far.x;
|
|
58
|
+
const echo = conv(driven, makePath(5, delayMs, gainDb));
|
|
59
|
+
if (change) { const second = conv(driven, makePath(6, delayMs + 20, gainDb - 6)); for (let i = Math.round(change * FS); i < n; i++) echo[i] = second[i]; }
|
|
60
|
+
const r = rng(77); const nz = 10 ** (noiseDb / 20) * 0.05; const mic = new Float64Array(n);
|
|
61
|
+
for (let i = 0; i < n; i++) mic[i] = echo[i] + near.x[i] * 0.5 + nz * gauss(r);
|
|
62
|
+
return { far, near, mic, echo, n };
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
module.exports = { FS, rng, gauss, speechLike, makePath, conv, scene };
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pmos-workspace",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.116.0",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "pmos-workspace",
|
|
9
|
-
"version": "0.
|
|
9
|
+
"version": "0.116.0",
|
|
10
10
|
"license": "MIT",
|
|
11
11
|
"dependencies": {
|
|
12
12
|
"@xterm/addon-fit": "^0.11.0",
|
package/workspace/package.json
CHANGED
|
@@ -32,14 +32,28 @@ export function makeResampler(inRate, outRate = RATE) {
|
|
|
32
32
|
const w = 0.42 + 0.5 * Math.cos(Math.PI * d / half) + 0.08 * Math.cos(2 * Math.PI * d / half); // Blackman, centred
|
|
33
33
|
return sinc * w;
|
|
34
34
|
};
|
|
35
|
-
|
|
35
|
+
// The weights depend only on the fractional part of the centre, so they are made once for each fraction (3 or 160 of them for the
|
|
36
|
+
// rates in use) and kept: 48 kHz to 16 kHz, or 24 kHz to 16 kHz, would otherwise compute a sine and two cosines per tap per sample.
|
|
37
|
+
const phases = new Map();
|
|
38
|
+
const phaseOf = (frac) => {
|
|
39
|
+
const key = Math.round(frac * 1e6);
|
|
40
|
+
let ph = phases.get(key);
|
|
41
|
+
if (!ph) {
|
|
42
|
+
const c = Math.floor(frac * 1e6 + 0.5) / 1e6; const lo = Math.ceil(c - half) - 0; const hi = Math.floor(c + half);
|
|
43
|
+
const w = new Float64Array(hi - lo + 1); let norm = 0;
|
|
44
|
+
for (let i = lo; i <= hi; i++) { const k = weight(i - c); w[i - lo] = k; norm += k; }
|
|
45
|
+
for (let i = 0; i < w.length; i++) w[i] /= norm;
|
|
46
|
+
ph = { lo, w };
|
|
47
|
+
if (phases.size < 4096) phases.set(key, ph);
|
|
48
|
+
}
|
|
49
|
+
return ph;
|
|
50
|
+
};
|
|
36
51
|
function run(limit, out) {
|
|
37
52
|
while (centre < limit && Math.floor(centre) + half < base + buf.length) {
|
|
38
|
-
const
|
|
39
|
-
|
|
40
|
-
for (let
|
|
41
|
-
|
|
42
|
-
out.push(toS16(sum / norm));
|
|
53
|
+
const fl = Math.floor(centre); const ph = phaseOf(centre - fl);
|
|
54
|
+
const first = fl + ph.lo - base; let sum = 0;
|
|
55
|
+
for (let k = 0; k < ph.w.length; k++) { const j = first + k; if (j >= 0 && j < buf.length) sum += buf[j] * ph.w[k]; }
|
|
56
|
+
out.push(toS16(sum));
|
|
43
57
|
centre += ratio;
|
|
44
58
|
}
|
|
45
59
|
}
|
|
@@ -52,6 +52,7 @@ const POLL_MS = 1500; // while listening, and until the last line is lab
|
|
|
52
52
|
const AGENT_POLL_MS = 3000;
|
|
53
53
|
const CHUNK_SAMPLES = 16000; // one second at 16 kHz
|
|
54
54
|
const RATE = 16000;
|
|
55
|
+
const HI_RATE = 24000; // the replay is kept at this rate when the device has it (D92 as amended 2026-10-08)
|
|
55
56
|
const SEND_TRIES = 3; // a chunk that the server did not take is sent again this many times in all …
|
|
56
57
|
const SEND_BACKOFF_MS = 400; // … after 400 ms, then 1200 ms
|
|
57
58
|
const SECTIONS = [['decision', 'Decisions'], ['action', 'Next steps'], ['question', 'Open questions']];
|
|
@@ -214,7 +215,7 @@ async function loadList(ctx) {
|
|
|
214
215
|
if (!currentId && list.notes.length) currentId = list.notes[0].id;
|
|
215
216
|
ctx.setState({});
|
|
216
217
|
}
|
|
217
|
-
const sig = (n) => JSON.stringify([n.final_state, n.final && n.final.at, n.final && n.final.names, n.final_notes && n.final_notes.at, n.wispr && [n.wispr.state, n.wispr.tries, n.wispr.reason], n.transcript, n.bullets, n.bullets_state && n.bullets_state.agent, n.summary && n.summary.at, n.agent && n.agent.markdown, n.private, n.speakers, n.title, (n.versions || []).length, n.__recording && n.__recording.bytes, n.meeting, n.initiative]);
|
|
218
|
+
const sig = (n) => JSON.stringify([n.final_state, n.final && n.final.at, n.final && n.final.names, n.final_notes && n.final_notes.at, n.wispr && [n.wispr.state, n.wispr.tries, n.wispr.reason], n.transcript, n.bullets, n.bullets_state && n.bullets_state.agent, n.summary && n.summary.at, n.agent && n.agent.markdown, n.private, n.speakers, n.title, (n.versions || []).length, n.__recording && [n.__recording.bytes, n.__recording.cleaned], n.meeting, n.initiative]);
|
|
218
219
|
async function loadNote(ctx, id) {
|
|
219
220
|
const r = await pmGet('/api/notes/one?id=' + encodeURIComponent(id));
|
|
220
221
|
if (r && r.ok) {
|
|
@@ -516,10 +517,10 @@ function dropPlayer() { if (player) { try { player.el.pause(); } catch { /* gone
|
|
|
516
517
|
function playerFor(ctx, n) {
|
|
517
518
|
const rec = n.__recording;
|
|
518
519
|
if (!rec || n.private) { if (player && player.id === n.id) dropPlayer(); return null; }
|
|
519
|
-
if (player && (player.id !== n.id || (player.bytes !== rec.bytes && !player.playing))) dropPlayer();
|
|
520
|
+
if (player && (player.id !== n.id || ((player.bytes !== rec.bytes || player.cleaned !== !!rec.cleaned) && !player.playing))) dropPlayer();
|
|
520
521
|
if (player) return player;
|
|
521
|
-
const a = el('audio', { controls: true, preload: 'metadata', class: 'nt-audio', 'data-nt-audio': '', src: '/api/notes/recording?id=' + encodeURIComponent(n.id) + '&b=' + rec.bytes });
|
|
522
|
-
const p = { id: n.id, bytes: rec.bytes, el: a, playing: false };
|
|
522
|
+
const a = el('audio', { controls: true, preload: 'metadata', class: 'nt-audio', 'data-nt-audio': '', src: '/api/notes/recording?id=' + encodeURIComponent(n.id) + '&b=' + rec.bytes + (rec.cleaned ? '&c=1' : '') });
|
|
523
|
+
const p = { id: n.id, bytes: rec.bytes, cleaned: !!rec.cleaned, el: a, playing: false };
|
|
523
524
|
a.addEventListener('play', () => { p.playing = true; ctx.setState({}); });
|
|
524
525
|
a.addEventListener('pause', () => { p.playing = false; ctx.setState({}); });
|
|
525
526
|
a.addEventListener('ended', () => { p.playing = false; ctx.setState({}); });
|
|
@@ -561,6 +562,21 @@ async function deleteAudio(ctx, n) {
|
|
|
561
562
|
ctx.setState({});
|
|
562
563
|
}
|
|
563
564
|
|
|
565
|
+
/** Ask for the cleaned copy of one note's replay, then read the note until the copy is there. */
|
|
566
|
+
async function cleanNow(ctx, n) {
|
|
567
|
+
const r = await pmPost('/api/notes/clean-audio', { id: n.id });
|
|
568
|
+
if (!r || !r.ok) { banner = { tone: 'bad', text: (r && r.error) || 'the sound was not cleaned up' }; ctx.setState({}); return; }
|
|
569
|
+
banner = { tone: 'ok', text: 'Cleaning up the sound. The replay changes when this is done; it takes a minute or two for a long meeting.' };
|
|
570
|
+
ctx.setState({});
|
|
571
|
+
for (let i = 0; i < 60; i++) {
|
|
572
|
+
await new Promise((res) => setTimeout(res, 3000));
|
|
573
|
+
const o = await loadNote(ctx, n.id);
|
|
574
|
+
if (o && o.ok && o.recording && o.recording.cleaned) { banner = { tone: 'ok', text: 'The replay now plays the cleaned sound.' }; ctx.setState({}); return; }
|
|
575
|
+
}
|
|
576
|
+
banner = { tone: 'bad', text: 'The cleaned sound is not ready yet. Open Settings and try again later.' };
|
|
577
|
+
ctx.setState({});
|
|
578
|
+
}
|
|
579
|
+
|
|
564
580
|
/* ── listening with local Whisper ─────────────────────────────────────────────────────────────── */
|
|
565
581
|
async function openSource(kind) {
|
|
566
582
|
const md = navigator.mediaDevices;
|
|
@@ -573,7 +589,7 @@ async function openSource(kind) {
|
|
|
573
589
|
return new MediaStream(tracks);
|
|
574
590
|
}
|
|
575
591
|
async function startAudio(ctx, n, want) {
|
|
576
|
-
const a = { id: n.id, state: 'starting', sources: [], streams: [], ctx: null, started: Date.now(), clock: null, msg: null, queue: {}, buf: {}, rs: {}, lost: 0, ended: 0 };
|
|
592
|
+
const a = { id: n.id, state: 'starting', sources: [], streams: [], ctx: null, started: Date.now(), clock: null, msg: null, queue: {}, buf: {}, rs: {}, rs24: {}, buf24: {}, lost: 0, ended: 0 };
|
|
577
593
|
const opened = [];
|
|
578
594
|
// the meeting first, straight from the press: the browser lets a page share a screen only right after a click
|
|
579
595
|
for (const kind of want.slice().sort((x, y) => (x === 'meeting' ? -1 : y === 'meeting' ? 1 : 0))) {
|
|
@@ -589,17 +605,22 @@ async function startAudio(ctx, n, want) {
|
|
|
589
605
|
// the clock counts the meeting time, as the line times do: Listen again carries on from where it stopped
|
|
590
606
|
a.started = Date.parse(r.clock_from || r.recorded_from) || a.started;
|
|
591
607
|
let ac;
|
|
592
|
-
|
|
608
|
+
// the device's own rate: the 16 kHz audio for transcription and the 24 kHz audio for the replay are both made from the same blocks (D92 as amended)
|
|
609
|
+
try { ac = new AudioContext(); } catch { ac = new AudioContext({ sampleRate: RATE }); }
|
|
593
610
|
a.ctx = ac;
|
|
594
611
|
const sink = ac.createGain(); sink.gain.value = 0; sink.connect(ac.destination);
|
|
595
612
|
for (const [kind, stream] of opened) {
|
|
596
613
|
a.streams.push(stream); a.sources.push(kind); a.buf[kind] = []; a.queue[kind] = Promise.resolve();
|
|
597
614
|
const src = ac.createMediaStreamSource(stream);
|
|
598
615
|
const rs = a.rs[kind] = makeResampler(ac.sampleRate);
|
|
616
|
+
// a device slower than 24 kHz has nothing above 12 kHz to keep, so it sends 16 kHz audio only and the replay stays 16 kHz
|
|
617
|
+
const rs24 = ac.sampleRate >= HI_RATE ? (a.rs24[kind] = makeResampler(ac.sampleRate, HI_RATE)) : null;
|
|
618
|
+
a.buf24[kind] = [];
|
|
599
619
|
await openCapture(ac, src, sink, (block) => {
|
|
600
620
|
if (audio !== a) return;
|
|
601
621
|
const s = rs.push(block);
|
|
602
622
|
const b = a.buf[kind]; for (let i = 0; i < s.length; i++) b.push(s[i]);
|
|
623
|
+
if (rs24) { const h = rs24.push(block); const b24 = a.buf24[kind]; for (let i = 0; i < h.length; i++) b24.push(h[i]); }
|
|
603
624
|
if (b.length >= CHUNK_SAMPLES) sendChunk(ctx, a, kind, false);
|
|
604
625
|
});
|
|
605
626
|
for (const t of stream.getTracks()) t.addEventListener('ended', () => { if (audio !== a) return; a.ended++; if (a.ended >= a.streams.length) stopAudio(ctx); });
|
|
@@ -612,11 +633,13 @@ async function startAudio(ctx, n, want) {
|
|
|
612
633
|
function sendChunk(ctx, a, kind, final) {
|
|
613
634
|
const b = a.buf[kind] || []; a.buf[kind] = [];
|
|
614
635
|
const pcm = b64(Int16Array.from(b));
|
|
636
|
+
const b24 = a.buf24[kind] || []; a.buf24[kind] = [];
|
|
637
|
+
const pcm24 = b24.length ? b64(Int16Array.from(b24)) : undefined;
|
|
615
638
|
const seconds = b.length / RATE;
|
|
616
639
|
a.queue[kind] = a.queue[kind].then(async () => {
|
|
617
640
|
let r = null;
|
|
618
641
|
for (let tries = 0; tries < SEND_TRIES; tries++) {
|
|
619
|
-
r = await pmPost('/api/notes/audio', { id: a.id, source: kind, pcm, final });
|
|
642
|
+
r = await pmPost('/api/notes/audio', { id: a.id, source: kind, pcm, pcm24, final });
|
|
620
643
|
if (r && r.ok) return;
|
|
621
644
|
if (r && (r.status === 409 || r.status === 404)) break; // the server refused this note's audio: sending again changes nothing
|
|
622
645
|
if (tries + 1 < SEND_TRIES) await new Promise((res) => setTimeout(res, SEND_BACKOFF_MS * 3 ** tries));
|
|
@@ -641,7 +664,10 @@ async function stopAudio(ctx, { discard = false } = {}) {
|
|
|
641
664
|
try { if (a.ctx) await a.ctx.close(); } catch { /* closed */ }
|
|
642
665
|
if (discard) { ctx.setState({}); return; }
|
|
643
666
|
// the rest of what was said, then the summary once the last line is transcribed
|
|
644
|
-
for (const k of a.sources) {
|
|
667
|
+
for (const k of a.sources) {
|
|
668
|
+
const tail = a.rs[k] ? a.rs[k].flush() : []; const b = a.buf[k]; for (let i = 0; i < tail.length; i++) b.push(tail[i]);
|
|
669
|
+
const tail24 = a.rs24[k] ? a.rs24[k].flush() : []; const b24 = a.buf24[k]; for (let i = 0; i < tail24.length; i++) b24.push(tail24[i]);
|
|
670
|
+
}
|
|
645
671
|
await Promise.all(a.sources.map((k) => sendChunk(ctx, a, k, true)));
|
|
646
672
|
await pmPost('/api/notes/pause', { id: a.id });
|
|
647
673
|
ctx.setState({});
|
|
@@ -892,7 +918,7 @@ async function chooseSources(ctx, n) {
|
|
|
892
918
|
}
|
|
893
919
|
async function setSwitch(ctx, patch) {
|
|
894
920
|
const ws = wsOf(ctx);
|
|
895
|
-
if (!ws && patch.language === undefined && patch.model === undefined && patch.keep_audio === undefined) return;
|
|
921
|
+
if (!ws && patch.language === undefined && patch.model === undefined && patch.keep_audio === undefined && patch.clean_audio === undefined) return;
|
|
896
922
|
const r = await pmPost('/api/notes/switch', ws ? { ws, ...patch } : patch);
|
|
897
923
|
if (r && r.ok) engine = { ws, whisper: r.whisper, jev: r.jev, at: Date.now() };
|
|
898
924
|
ctx.setState({});
|
|
@@ -926,7 +952,13 @@ function openSettings(ctx, n) {
|
|
|
926
952
|
onCommit: (v) => setSwitch(ctx, { keep_audio: v }).then(() => { h.close(); openSettings(ctx, n); }), attrs: { 'data-nt-keep-audio': '' } }) : null,
|
|
927
953
|
w ? el('p', { class: 'nt-hint', 'data-nt-keep-hint': '' }, w.keep_audio !== false
|
|
928
954
|
? 'The meeting audio is kept on this machine, outside every repository, until you delete it. It is never sent anywhere. Browser recognition keeps no audio.'
|
|
929
|
-
: 'No audio is kept: each clip is deleted once it is transcribed, and a meeting cannot be replayed.') : null
|
|
955
|
+
: 'No audio is kept: each clip is deleted once it is transcribed, and a meeting cannot be replayed.') : null,
|
|
956
|
+
w && w.keep_audio !== false ? field({ label: 'Clean up the replay sound', kind: 'check', value: w.clean_audio === true,
|
|
957
|
+
onCommit: (v) => setSwitch(ctx, { clean_audio: v }).then(() => { h.close(); openSettings(ctx, n); }), attrs: { 'data-nt-clean-audio': '' } }) : null,
|
|
958
|
+
w && w.keep_audio !== false ? el('p', { class: 'nt-hint', 'data-nt-clean-hint': w.ffmpeg ? 'ready' : 'missing' }, w.ffmpeg
|
|
959
|
+
? (w.clean_audio === true ? 'After Stop, the replay gets a cleaned copy: less rumble and hiss, and an even loudness. The transcript is not changed.'
|
|
960
|
+
: 'Off. When on, the replay gets a cleaned copy after Stop: less rumble and hiss, and an even loudness.')
|
|
961
|
+
: 'This needs ffmpeg, which was not found on this machine. Install it with brew install ffmpeg, then open Settings again.') : null));
|
|
930
962
|
const fp = w && w.final;
|
|
931
963
|
if (fp) {
|
|
932
964
|
const have = [fp.transcriber === 'qwen' ? 'Qwen3-ASR' : fp.transcriber === 'parakeet' ? 'Parakeet' : fp.transcriber === 'whisper' ? 'Whisper with beam search' : null,
|
|
@@ -958,6 +990,7 @@ function openSettings(ctx, n) {
|
|
|
958
990
|
button({ label: n.private ? 'Mark not private' : 'Mark private', onClick: () => { h.close(); setPrivate(ctx, n, !n.private); }, attrs: { [n.private ? 'data-nt-unprivate' : 'data-nt-mark-private']: '' } }),
|
|
959
991
|
button({ label: busy === 'agent' ? 'Starting …' : 'Summary by agent', disabled: n.private || busy === 'agent', onClick: () => { h.close(); summarizeAgent(ctx, n); }, attrs: { 'data-nt-summarize-agent': '' } }),
|
|
960
992
|
n.__recording || (n.final_state && n.final_state.state === 'failed') ? button({ label: 'Transcribe again', disabled: n.private || finalPending(n), onClick: () => { h.close(); runFinal(ctx, n); }, attrs: { 'data-nt-final-again': '' } }) : null,
|
|
993
|
+
n.__recording && w && w.clean_audio === true && w.ffmpeg && !n.__recording.cleaned ? button({ label: 'Clean up this note\'s sound', disabled: n.private || listening(n), onClick: () => { h.close(); cleanNow(ctx, n); }, attrs: { 'data-nt-clean-now': '' } }) : null,
|
|
961
994
|
n.__recording ? button({ label: 'Delete this note\'s audio', variant: 'ghost', onClick: () => { h.close(); deleteAudio(ctx, n); }, attrs: { 'data-nt-delete-audio': '' } }) : null,
|
|
962
995
|
button({ label: 'Delete', variant: 'ghost', onClick: () => { h.close(); deleteNote(ctx, n); }, attrs: { 'data-nt-delete': '' } }))));
|
|
963
996
|
return parts;
|
|
@@ -1,11 +1,13 @@
|
|
|
1
1
|
{
|
|
2
|
-
"_comment": "meeting-notes Runs 1 to 10: the Notes engine and routes (notes.test.js), local Whisper, line labels and repository checks (notes-listen.test.js), the final pass after Stop and the Wispr Flow fetch (notes-final.test.js), Qwen3-ASR kept loaded (notes-qwen.test.js), and the Notes view in a browser (pm-notes-test.js), the echo canceller for the replay (notes-echo.test.js, notes-audio-quality Run 2), plus the note pop-up patched in place while it listens (pm-note-dock-test.js).",
|
|
2
|
+
"_comment": "meeting-notes Runs 1 to 10: the Notes engine and routes (notes.test.js), local Whisper, line labels and repository checks (notes-listen.test.js), the final pass after Stop and the Wispr Flow fetch (notes-final.test.js), Qwen3-ASR kept loaded (notes-qwen.test.js), and the Notes view in a browser (pm-notes-test.js), the echo canceller for the replay (notes-echo.test.js, notes-audio-quality Run 2) and the optional cleaned copy of the replay (notes-polish.test.js, Run 3) and the 24 kHz replay (notes-hires.test.js, Run 4), plus the note pop-up patched in place while it listens (pm-note-dock-test.js).",
|
|
3
3
|
"modules": [
|
|
4
4
|
"notes.test.js",
|
|
5
5
|
"notes-listen.test.js",
|
|
6
6
|
"notes-final.test.js",
|
|
7
7
|
"notes-qwen.test.js",
|
|
8
|
-
"notes-echo.test.js"
|
|
8
|
+
"notes-echo.test.js",
|
|
9
|
+
"notes-polish.test.js",
|
|
10
|
+
"notes-hires.test.js"
|
|
9
11
|
],
|
|
10
12
|
"panels": [
|
|
11
13
|
"pm-notes-test.js",
|
|
@@ -18,6 +20,8 @@
|
|
|
18
20
|
"notes-final.test.js": 6,
|
|
19
21
|
"notes-qwen.test.js": 4,
|
|
20
22
|
"pm-note-dock-test.js": 20,
|
|
21
|
-
"notes-echo.test.js": 25
|
|
23
|
+
"notes-echo.test.js": 25,
|
|
24
|
+
"notes-polish.test.js": 3,
|
|
25
|
+
"notes-hires.test.js": 10
|
|
22
26
|
}
|
|
23
27
|
}
|