@crievo/omni 0.1.4 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +5 -3
- package/reference/learnings.md +12 -0
- package/reference/studio.md +10 -5
- package/studio/api/services/queue.ts +2 -0
- package/studio/api/services/root.ts +2 -3
- package/studio/runner/runner.ts +1 -1
- package/studio/studio.ts +5 -3
- package/studio/web/lib/studio-offline.ts +31 -7
- package/tools/job-queue.mjs +25 -7
- package/tools/job_queue.py +100 -24
- package/tools/lib/deps.mjs +21 -10
- package/tools/lib/oauth-client.mjs +18 -45
- package/tools/lib/platform.mjs +102 -0
- package/tools/lib/service-run.mjs +124 -0
- package/tools/lib/service-windows.mjs +240 -0
- package/tools/lib/service.mjs +74 -4
- package/tools/lib/up.mjs +32 -17
- package/tools/lib/vault.mjs +116 -0
- package/tools/metrics.mjs +7 -2
- package/tools/omni.mjs +6 -8
- package/tools/omnichat.mjs +28 -10
- package/tools/package.mjs +4 -3
- package/tools/publish.mjs +13 -7
- package/tools/runner.mjs +6 -6
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@crievo/omni",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.6",
|
|
4
4
|
"description": "OmniVideo no seu computador: o CLI omni, o studio local e as ferramentas do pipeline.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -10,7 +10,8 @@
|
|
|
10
10
|
"node": ">=22"
|
|
11
11
|
},
|
|
12
12
|
"os": [
|
|
13
|
-
"darwin"
|
|
13
|
+
"darwin",
|
|
14
|
+
"win32"
|
|
14
15
|
],
|
|
15
16
|
"license": "UNLICENSED",
|
|
16
17
|
"homepage": "https://omni.crievo.com",
|
|
@@ -39,6 +40,7 @@
|
|
|
39
40
|
"voice-glow": "0.3.0",
|
|
40
41
|
"zod": "4.6.5",
|
|
41
42
|
"bun": "1.4.3",
|
|
42
|
-
"@puppeteer/browsers": "3.2.3"
|
|
43
|
+
"@puppeteer/browsers": "3.2.3",
|
|
44
|
+
"@napi-rs/keyring": "2.1.1"
|
|
43
45
|
}
|
|
44
46
|
}
|
package/reference/learnings.md
CHANGED
|
@@ -802,3 +802,15 @@ these voices).
|
|
|
802
802
|
- **Symptom:** in `2026-10-18-roteiro-pronto-guia-de-motion`, c06's first frame was empty cream: the symbol that should be born at scale 0.35 on frame 0 only showed from the second frame, and the c05 → c06 cut sheet read as "ring gone, blank, symbol pops".
|
|
803
803
|
- **Cause:** floating point. The build sums scene durations, so `OMNIVIDEO.start` for c06 was `16.799999999999997`, while the beat `simbolo` (from `video.scenes[].beats`) was `16.8`. A helper gated with `if (T >= t0)` (`OV.signAt`) was false at `t = 0`. Progress helpers (`seg`, clamped) don't care; only hard `>=` gates do.
|
|
804
804
|
- **Fixed:** the gate takes a tolerance, `T >= t0 - 1e-4`. Rule: any on/off gate on a beat time in a continuous composition compares with an epsilon, never with a bare `>=`.
|
|
805
|
+
|
|
806
|
+
## The Keychain asked for a password at every read when I swapped `security` for a library (18/10/2026)
|
|
807
|
+
|
|
808
|
+
- **Symptom:** testing `@napi-rs/keyring` as the login vault on the Mac (the plan was one library for macOS, Windows and Linux), a dozen authorization dialogs opened and the reads hung: `node` reading an item `security` had written, `bun` reading an item `node` had written, and `security` reading an item `node` had written. The studio service has nobody to click them, so its token refresh would have frozen.
|
|
809
|
+
- **Cause:** a Keychain item's access list names the program that created it. `security add-generic-password` makes `/usr/bin/security` the trusted reader, which is why every process (the CLI's `node`, the service's `bun`, the hook) reads the login through `security` without a prompt. Any other program, even the same library from a different binary, is a stranger to the item.
|
|
810
|
+
- **Fixed:** `tools/lib/vault.mjs` keeps `security` on macOS and uses `@napi-rs/keyring` only on Windows (Credential Manager) and Linux (Secret Service), where an item belongs to the user, not to the program. Moving macOS to the library would mean every item rewritten by one binary and the studio running under that same binary, which the service and the CLI do not share. Rule: test anything that touches the Keychain with a throwaway service name, with a timeout, never the real item, and warn the user first, because every blocked read is a dialog on their screen.
|
|
811
|
+
|
|
812
|
+
## `os.kill(pid, 0)` is not a probe on Windows (18/10/2026)
|
|
813
|
+
|
|
814
|
+
- **Symptom:** none yet; found reading the queue for a Windows port.
|
|
815
|
+
- **Cause:** on Windows, Python's `os.kill(pid, sig)` treats `sig == 0` as `CTRL_C_EVENT` and sends it to the process group of `pid`; any other number is a `TerminateProcess`. The Node `process.kill(pid, 0)` is a real probe on both systems.
|
|
816
|
+
- **Fixed:** `tools/job_queue.py` asks the kernel (`OpenProcess` + `GetExitCodeProcess`, `STILL_ACTIVE`) on Windows and keeps `os.kill` plus the `ps` zombie check elsewhere. Also on Windows: `select()` works on sockets only (the guardian reads the Node pipe in a thread), an atomic replace fails while a reader has the file open (retry), and a text file is written in the code page unless `encoding="utf-8"` says otherwise.
|
package/reference/studio.md
CHANGED
|
@@ -300,9 +300,10 @@ studio crashes, restarts or starts in the middle of a task.
|
|
|
300
300
|
|
|
301
301
|
1. The tool writes a ticket, `.queue/<id>.json`: tool, video, command, pid,
|
|
302
302
|
origin (`chat`, `terminal`, `studio`), arrival.
|
|
303
|
-
2. It waits to be the oldest live ticket waiting and then takes the
|
|
304
|
-
on `.queue/machine.lock
|
|
305
|
-
|
|
303
|
+
2. It waits to be the oldest live ticket waiting and then takes the lock
|
|
304
|
+
on `.queue/machine.lock` (`flock` on macOS and Linux, `msvcrt.locking` on
|
|
305
|
+
Windows). The kernel releases it when the process dies, even with
|
|
306
|
+
`kill -9`: there is no forgotten lock.
|
|
306
307
|
3. On its turn, the tool's output goes to the terminal as always **and** to
|
|
307
308
|
`.queue/<id>.log`, which the screen shows live.
|
|
308
309
|
4. On exit, the ticket becomes `ok`, `error` (with the exit code) or
|
|
@@ -316,8 +317,12 @@ finished tickets are kept, with their logs.
|
|
|
316
317
|
The protocol lives in `tools/job_queue.py` (`with turn("voice", dir):`). The
|
|
317
318
|
Node tools use `tools/job-queue.mjs` (`await turn("render", dir)`), which
|
|
318
319
|
runs `job_queue.py` as a guardian: a Python process that waits for the turn,
|
|
319
|
-
holds the
|
|
320
|
-
pipe closes.
|
|
320
|
+
holds the lock while the Node process lives and releases it when the stdin
|
|
321
|
+
pipe closes. On Windows the same protocol runs with two substitutions: the
|
|
322
|
+
liveness probe asks the kernel (`GetExitCodeProcess`; `os.kill(pid, 0)` there
|
|
323
|
+
would send Ctrl+C to the process group) and there is no zombie check. The
|
|
324
|
+
tests (`tools/test/test_job_queue.py`, `tools/job-queue.test.mjs`) run real
|
|
325
|
+
processes on all three systems.
|
|
321
326
|
|
|
322
327
|
Details:
|
|
323
328
|
|
|
@@ -42,6 +42,8 @@ export function isAlive(pid: number) {
|
|
|
42
42
|
} catch (e: any) {
|
|
43
43
|
if (e.code !== "EPERM") return false;
|
|
44
44
|
}
|
|
45
|
+
/* Windows has no zombies, and no `ps` */
|
|
46
|
+
if (process.platform === "win32") return true;
|
|
45
47
|
const ps = Bun.spawnSync(["ps", "-o", "stat=", "-p", String(pid)]);
|
|
46
48
|
return !ps.stdout.toString().trim().startsWith("Z");
|
|
47
49
|
}
|
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
import { readFileSync, existsSync, copyFileSync } from "node:fs";
|
|
4
4
|
import { join, resolve } from "node:path";
|
|
5
5
|
import { homedir } from "node:os";
|
|
6
|
+
import { pickPython } from "../../../tools/lib/platform.mjs";
|
|
6
7
|
|
|
7
8
|
/* Two roots, as in tools/lib/paths.mjs. CODE_ROOT holds tools/, templates/, docs/ and the .venv: the
|
|
8
9
|
project dir (studio/api/services → ../../..), or OMNI_CODE_ROOT when this code runs bundled into the
|
|
@@ -22,9 +23,7 @@ if (!existsSync(join(ROOT, "omnivideo.config.json")) && existsSync(join(CODE_ROO
|
|
|
22
23
|
export const CONFIG = JSON.parse(readFileSync(join(ROOT, "omnivideo.config.json"), "utf8"));
|
|
23
24
|
/* the project's Python (.venv, created by `node tools/setup.mjs`); without it, the one in the config
|
|
24
25
|
(the VoiceStudio venv) and, last, the system python3 */
|
|
25
|
-
export const PYTHON: string =
|
|
26
|
-
[join(CODE_ROOT, ".venv", "bin", "python"), CONFIG.voicestudio?.python ?? ""].find(p => p && existsSync(p)) ??
|
|
27
|
-
"python3";
|
|
26
|
+
export const PYTHON: string = pickPython(CODE_ROOT, [CONFIG.voicestudio?.python ?? ""]);
|
|
28
27
|
/* config path: relative paths are resolved from the project root */
|
|
29
28
|
export const resolvePath = (p: string) => resolve(ROOT, p.replace(/^~(?=\/|$)/, homedir()));
|
|
30
29
|
/* the node on PATH is the same one the chat uses; without PATH (studio opened outside a
|
package/studio/runner/runner.ts
CHANGED
|
@@ -114,7 +114,7 @@ export function startRunner(o: RunnerOptions) {
|
|
|
114
114
|
/* an unreachable studio while refreshing says nothing about the login: try again later */
|
|
115
115
|
const token = await Promise.resolve(o.token()).catch(() => "");
|
|
116
116
|
if (!token || token === revoked) {
|
|
117
|
-
if (!hinted) log("aviso: este Mac não está pareado com o studio hospedado:
|
|
117
|
+
if (!hinted) log("aviso: este Mac não está pareado com o studio hospedado: npx -y @crievo/omni login");
|
|
118
118
|
hinted = true;
|
|
119
119
|
timer = setTimeout(connect, KEY_CHECK_MS);
|
|
120
120
|
return;
|
package/studio/studio.ts
CHANGED
|
@@ -36,6 +36,7 @@ import { startRunner } from "./runner/runner";
|
|
|
36
36
|
import { cloudSettings, env, writeEnv } from "../tools/platforms.mjs";
|
|
37
37
|
import { checkKey, finishPairing, startPairing } from "../tools/lib/pairing.mjs";
|
|
38
38
|
import { ACCESS_PREFIX, accessToken } from "../tools/lib/oauth-client.mjs";
|
|
39
|
+
import { openUrl } from "../tools/lib/platform.mjs";
|
|
39
40
|
|
|
40
41
|
const args = process.argv.slice(2);
|
|
41
42
|
const option = (name: string) => {
|
|
@@ -93,7 +94,8 @@ try {
|
|
|
93
94
|
});
|
|
94
95
|
} catch (e) {
|
|
95
96
|
if ((e as { code?: string })?.code === "EADDRINUSE") {
|
|
96
|
-
|
|
97
|
+
const who = process.platform === "win32" ? `netstat -ano | findstr :${PORT}` : `lsof -i :${PORT}`;
|
|
98
|
+
console.error(`erro: a porta ${PORT} já está em uso. O studio já está aberto? (${who})`);
|
|
97
99
|
process.exit(1);
|
|
98
100
|
}
|
|
99
101
|
throw e;
|
|
@@ -117,7 +119,7 @@ console.log(`OmniVideo · ${address}`);
|
|
|
117
119
|
console.log(
|
|
118
120
|
`vídeos em ${videoRoots.map(r => (relative(ROOT, r).startsWith("..") ? r : relative(ROOT, r) + "/")).join(", ")} · fila em .queue/`,
|
|
119
121
|
);
|
|
120
|
-
if (args.includes("--open"))
|
|
122
|
+
if (args.includes("--open")) openUrl(address);
|
|
121
123
|
|
|
122
124
|
/* --- runner ---------------------------------------------------------------------
|
|
123
125
|
The hosted studio reaches this same app through the channel (docs/specs/phase-1-split-api.md).
|
|
@@ -137,7 +139,7 @@ if (args.includes("--runner") || (hosted.on && option("port") === undefined)) {
|
|
|
137
139
|
const pair = async () => {
|
|
138
140
|
const code = await startPairing(url);
|
|
139
141
|
console.log(`cloud: código do pareamento ${code.user_code} · ${code.verification_uri_complete}`);
|
|
140
|
-
|
|
142
|
+
openUrl(code.verification_uri_complete);
|
|
141
143
|
const paired = await finishPairing(url, code, hostname().replace(/\.local$/, ""));
|
|
142
144
|
writeEnv({ OMNIVIDEO_RUNNER_TOKEN: paired.key });
|
|
143
145
|
return paired.key;
|
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
/* What the "studio sem conexão" screen (components/StudioOffline.tsx) says, from what the hub knows
|
|
2
|
-
(cloud/studio/src/presence.mjs): the reason the Mac is out and the step that brings it back. A
|
|
3
|
-
|
|
2
|
+
(cloud/studio/src/presence.mjs): the reason the Mac is out and the step that brings it back. A Mac that
|
|
3
|
+
never connected gets the install (the plugin, or the CLI alone); a refused key repairs itself (the runner
|
|
4
|
+
opens the approval page on the Mac); the rest needs one thing on the Mac. The commands are the installed
|
|
5
|
+
CLI's (tools/lib/cli.mjs: `npx -y @crievo/omni …`), with the repository's own as the last step for the
|
|
6
|
+
machine that runs from a checkout. */
|
|
4
7
|
|
|
5
8
|
import type { RunnerProblem } from "../types/api";
|
|
6
9
|
import { formatDayTime } from "./format";
|
|
@@ -8,6 +11,10 @@ import { formatDayTime } from "./format";
|
|
|
8
11
|
export type OfflineStep = { text: string; code?: string };
|
|
9
12
|
export type OfflineAdvice = { title: string; text: string; steps: OfflineStep[] };
|
|
10
13
|
|
|
14
|
+
/* the plugin's marketplace and the CLI's package (tools/lib/cli.mjs, tools/omni.mjs) */
|
|
15
|
+
const PLUGIN_INSTALL = "claude plugin install omnivideo --marketplace pedroflp/omni-plugin";
|
|
16
|
+
const CLI = "npx -y @crievo/omni";
|
|
17
|
+
|
|
11
18
|
const theMac = (name: string | null | undefined) => (name ? `O Mac “${name}”` : "O Mac que roda o studio");
|
|
12
19
|
|
|
13
20
|
export function offlineAdvice(o: {
|
|
@@ -22,22 +29,25 @@ export function offlineAdvice(o: {
|
|
|
22
29
|
text: `${theMac(p.host || o.name)} tentou conectar em ${formatDayTime(p.at)}, mas o servidor não aceitou a chave dele. Ele pede uma nova sozinho: a página de aprovação abre no navegador do Mac.`,
|
|
23
30
|
steps: [
|
|
24
31
|
{ text: "No Mac, aprove a conexão na página que abriu (entre com o Google se ela pedir)." },
|
|
25
|
-
{ text: "Se a página não abriu, no Mac:", code:
|
|
32
|
+
{ text: "Se a página não abriu, no Mac:", code: `${CLI} login` },
|
|
26
33
|
],
|
|
27
34
|
};
|
|
28
35
|
if (p?.reason === "revoked")
|
|
29
36
|
return {
|
|
30
37
|
title: "Este Mac foi desconectado",
|
|
31
38
|
text: `${theMac(p.name || o.name)} foi desconectado do studio em ${formatDayTime(p.at)}. Ele só volta quando for pareado de novo.`,
|
|
32
|
-
steps: [
|
|
39
|
+
steps: [
|
|
40
|
+
{ text: "No Mac:", code: `${CLI} login` },
|
|
41
|
+
{ text: "Se o studio roda do repositório:", code: "node tools/runner.mjs login" },
|
|
42
|
+
],
|
|
33
43
|
};
|
|
34
44
|
if (p?.reason === "protocol" && p.side === "mac")
|
|
35
45
|
return {
|
|
36
46
|
title: "O Mac precisa de atualização",
|
|
37
47
|
text: `${theMac(p.host || o.name)} está com uma versão do studio anterior à do servidor, e os dois não se entendem.`,
|
|
38
48
|
steps: [
|
|
39
|
-
{ text: "No Mac, atualize
|
|
40
|
-
{ text: "
|
|
49
|
+
{ text: "No Mac, atualize o OmniVideo (ele reinicia o studio sozinho):", code: `${CLI}@latest up` },
|
|
50
|
+
{ text: "Se o studio roda do repositório:", code: "git pull && node tools/runner.mjs service restart" },
|
|
41
51
|
],
|
|
42
52
|
};
|
|
43
53
|
if (p?.reason === "protocol")
|
|
@@ -48,12 +58,26 @@ export function offlineAdvice(o: {
|
|
|
48
58
|
{ text: "Suba o servidor: o merge na main faz o deploy sozinho, ou GitHub → Actions → Deploy → Run workflow." },
|
|
49
59
|
],
|
|
50
60
|
};
|
|
61
|
+
/* the hub has never seen a Mac for this account: the first run, so the way in is the install */
|
|
62
|
+
if (!o.name && !o.lastSeen)
|
|
63
|
+
return {
|
|
64
|
+
title: "Falta ligar o OmniVideo no seu Mac",
|
|
65
|
+
text: "O vídeo é feito no seu computador (a narração, o motion e a montagem), e este studio só mostra e comanda. Nenhum Mac entrou nesta conta ainda.",
|
|
66
|
+
steps: [
|
|
67
|
+
{ text: "Instale o plugin no Claude Code (uma vez):", code: PLUGIN_INSTALL },
|
|
68
|
+
{
|
|
69
|
+
text: "Abra uma sessão do Claude Code. O plugin abre o navegador para você aprovar o Mac, baixa o que o vídeo precisa (alguns minutos) e liga o studio sozinho. Esta tela some quando ele conectar.",
|
|
70
|
+
},
|
|
71
|
+
{ text: "Sem o plugin, pelo terminal:", code: `${CLI} setup` },
|
|
72
|
+
],
|
|
73
|
+
};
|
|
51
74
|
return {
|
|
52
75
|
title: "O studio perdeu a conexão",
|
|
53
76
|
text: `${theMac(o.name)} não está conectado ${o.lastSeen ? `desde ${formatDayTime(o.lastSeen)}` : "agora"}. Ele pode estar desligado, dormindo ou sem internet, ou o studio está fechado. O que você pediu espera e volta a andar quando ele reconectar.`,
|
|
54
77
|
steps: [
|
|
55
78
|
{ text: "Confira se o Mac está ligado, acordado e com internet." },
|
|
56
|
-
{ text: "Deixe o studio sempre aberto (uma vez só, no Mac):", code:
|
|
79
|
+
{ text: "Deixe o studio sempre aberto (uma vez só, no Mac):", code: `${CLI} up` },
|
|
80
|
+
{ text: "Se o studio roda do repositório:", code: "node tools/runner.mjs service install" },
|
|
57
81
|
{ text: "Ou abra à mão:", code: "cd studio && bun start" },
|
|
58
82
|
],
|
|
59
83
|
};
|
package/tools/job-queue.mjs
CHANGED
|
@@ -25,17 +25,19 @@ import {
|
|
|
25
25
|
} from "node:fs";
|
|
26
26
|
import { join, dirname } from "node:path";
|
|
27
27
|
import { CODE_ROOT, DATA_ROOT as ROOT } from "./lib/paths.mjs";
|
|
28
|
+
import { IS_WINDOWS, pickPython, sleepSync } from "./lib/platform.mjs";
|
|
28
29
|
|
|
29
30
|
const CFG = JSON.parse(readFileSync(join(ROOT, "omnivideo.config.json"), "utf8"));
|
|
30
31
|
const QUEUE_DIR = join(ROOT, ".queue");
|
|
31
32
|
|
|
32
|
-
/* a zombie counts as dead (see studio/api/services/queue.ts) */
|
|
33
|
+
/* a zombie counts as dead (see studio/api/services/queue.ts); Windows has none, and no `ps` */
|
|
33
34
|
function alive(pid) {
|
|
34
35
|
try {
|
|
35
36
|
process.kill(pid, 0);
|
|
36
37
|
} catch (e) {
|
|
37
38
|
if (e.code !== "EPERM") return false;
|
|
38
39
|
}
|
|
40
|
+
if (IS_WINDOWS) return true;
|
|
39
41
|
const ps = spawnSync("ps", ["-o", "stat=", "-p", String(pid)], { encoding: "utf8" });
|
|
40
42
|
return !String(ps.stdout).trim().startsWith("Z");
|
|
41
43
|
}
|
|
@@ -86,6 +88,19 @@ function mirror(log) {
|
|
|
86
88
|
}
|
|
87
89
|
}
|
|
88
90
|
|
|
91
|
+
/* the atomic swap. On Windows a reader that has the ticket open (the guardian's Python, without share-delete)
|
|
92
|
+
makes the rename fail for a moment: try again, synchronously, because this runs on 'exit' */
|
|
93
|
+
function swap(tmp, file) {
|
|
94
|
+
for (let attempt = 0; ; attempt++) {
|
|
95
|
+
try {
|
|
96
|
+
return renameSync(tmp, file);
|
|
97
|
+
} catch (e) {
|
|
98
|
+
if (!IS_WINDOWS || attempt >= 24 || !["EPERM", "EACCES", "EBUSY"].includes(e.code)) throw e;
|
|
99
|
+
sleepSync(20);
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
|
|
89
104
|
/* on 'exit' only synchronous code runs; death by signal doesn't pass here, and
|
|
90
105
|
the guardian closes the ticket as interrupted */
|
|
91
106
|
function close(file, exitCode) {
|
|
@@ -98,7 +113,7 @@ function close(file, exitCode) {
|
|
|
98
113
|
});
|
|
99
114
|
const tmp = join(dirname(file), `.${ticket.id}.${process.pid}.tmp`);
|
|
100
115
|
writeFileSync(tmp, JSON.stringify(ticket, null, 1) + "\n");
|
|
101
|
-
|
|
116
|
+
swap(tmp, file);
|
|
102
117
|
} catch {
|
|
103
118
|
/* the guardian closes it */
|
|
104
119
|
}
|
|
@@ -128,10 +143,8 @@ function checkVoicePause(tool, dir) {
|
|
|
128
143
|
export async function turn(tool, dir) {
|
|
129
144
|
checkVoicePause(tool, dir);
|
|
130
145
|
if (process.env.OMNIVIDEO_QUEUE === "0" || insideTurn()) return null;
|
|
131
|
-
/* the project's Python (.venv); without it, the config's and the system python3 */
|
|
132
|
-
const python =
|
|
133
|
-
[join(CODE_ROOT, ".venv", "bin", "python"), CFG.voicestudio?.python ?? ""].find(p => p && existsSync(p)) ??
|
|
134
|
-
"python3";
|
|
146
|
+
/* the project's Python (.venv, laid out per OS); without it, the config's and the system python3 */
|
|
147
|
+
const python = pickPython(CODE_ROOT, [CFG.voicestudio?.python ?? ""]);
|
|
135
148
|
const guardian = spawn(
|
|
136
149
|
python,
|
|
137
150
|
[
|
|
@@ -143,7 +156,12 @@ export async function turn(tool, dir) {
|
|
|
143
156
|
"--",
|
|
144
157
|
...process.argv.slice(1),
|
|
145
158
|
],
|
|
146
|
-
{
|
|
159
|
+
{
|
|
160
|
+
stdio: ["pipe", "pipe", "inherit"],
|
|
161
|
+
windowsHide: true,
|
|
162
|
+
/* Windows Python writes the ticket and its notices in the console's code page unless told */
|
|
163
|
+
env: IS_WINDOWS ? { ...process.env, PYTHONUTF8: "1" } : process.env,
|
|
164
|
+
},
|
|
147
165
|
);
|
|
148
166
|
const ticket = await new Promise((resolveTicket, reject) => {
|
|
149
167
|
let buf = "";
|
package/tools/job_queue.py
CHANGED
|
@@ -14,9 +14,10 @@ In the Python tools:
|
|
|
14
14
|
|
|
15
15
|
Heavy: voice, captions, final render, sound, assembly and Premiere
|
|
16
16
|
(assemble, check, export). Each writes a ticket, .queue/<id>.json, and waits to
|
|
17
|
-
be the oldest live one waiting; then it takes the
|
|
18
|
-
|
|
19
|
-
|
|
17
|
+
be the oldest live one waiting; then it takes the lock on .queue/machine.lock
|
|
18
|
+
(flock on macOS and Linux, msvcrt.locking on Windows). The kernel releases the
|
|
19
|
+
lock when the process dies, even on kill -9: there is no forgotten lock. The
|
|
20
|
+
task output also goes to .queue/<id>.log.
|
|
20
21
|
|
|
21
22
|
The studio only reads this dir. Running, stopped or restarting in the middle of
|
|
22
23
|
a task, the queue is the same.
|
|
@@ -26,10 +27,8 @@ holding the turn passes to its children, keeps a child from waiting for its own
|
|
|
26
27
|
parent's turn.
|
|
27
28
|
"""
|
|
28
29
|
|
|
29
|
-
import fcntl
|
|
30
30
|
import json
|
|
31
31
|
import os
|
|
32
|
-
import select
|
|
33
32
|
import shlex
|
|
34
33
|
import signal
|
|
35
34
|
import subprocess
|
|
@@ -42,6 +41,12 @@ from pathlib import Path
|
|
|
42
41
|
|
|
43
42
|
from omni_paths import DATA_ROOT as ROOT
|
|
44
43
|
|
|
44
|
+
IS_WINDOWS = sys.platform == "win32"
|
|
45
|
+
if IS_WINDOWS:
|
|
46
|
+
import msvcrt
|
|
47
|
+
else:
|
|
48
|
+
import fcntl
|
|
49
|
+
|
|
45
50
|
QUEUE_DIR = ROOT / ".queue"
|
|
46
51
|
LOCK = QUEUE_DIR / "machine.lock"
|
|
47
52
|
|
|
@@ -56,9 +61,35 @@ def now():
|
|
|
56
61
|
return datetime.now(timezone.utc).isoformat(timespec="milliseconds").replace("+00:00", "Z")
|
|
57
62
|
|
|
58
63
|
|
|
64
|
+
def alive_windows(pid):
|
|
65
|
+
"""os.kill(pid, 0) is not a probe on Windows: signal 0 is CTRL_C_EVENT, sent to the process group.
|
|
66
|
+
Ask the kernel instead: the process exists and has not exited (STILL_ACTIVE)."""
|
|
67
|
+
import ctypes
|
|
68
|
+
from ctypes import wintypes
|
|
69
|
+
|
|
70
|
+
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
|
71
|
+
kernel32.OpenProcess.restype = wintypes.HANDLE
|
|
72
|
+
kernel32.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
|
|
73
|
+
kernel32.GetExitCodeProcess.argtypes = [wintypes.HANDLE, ctypes.POINTER(wintypes.DWORD)]
|
|
74
|
+
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
|
|
75
|
+
handle = kernel32.OpenProcess(0x1000, False, pid) # PROCESS_QUERY_LIMITED_INFORMATION
|
|
76
|
+
if not handle:
|
|
77
|
+
# ERROR_ACCESS_DENIED: it exists and is not ours to open; anything else (invalid parameter): gone
|
|
78
|
+
return ctypes.get_last_error() == 5
|
|
79
|
+
try:
|
|
80
|
+
code = wintypes.DWORD()
|
|
81
|
+
if not kernel32.GetExitCodeProcess(handle, ctypes.byref(code)):
|
|
82
|
+
return True
|
|
83
|
+
return code.value == 259 # STILL_ACTIVE
|
|
84
|
+
finally:
|
|
85
|
+
kernel32.CloseHandle(handle)
|
|
86
|
+
|
|
87
|
+
|
|
59
88
|
def alive(pid):
|
|
60
89
|
if not pid:
|
|
61
90
|
return False
|
|
91
|
+
if IS_WINDOWS:
|
|
92
|
+
return alive_windows(pid)
|
|
62
93
|
try:
|
|
63
94
|
os.kill(pid, 0)
|
|
64
95
|
except ProcessLookupError:
|
|
@@ -78,8 +109,19 @@ def write(ticket):
|
|
|
78
109
|
"""Atomic swap: a reader never gets a half-written ticket."""
|
|
79
110
|
dest = ticket_path(ticket["id"])
|
|
80
111
|
tmp = dest.with_name(f".{ticket['id']}.{os.getpid()}.tmp")
|
|
81
|
-
tmp.write_text(json.dumps(ticket, ensure_ascii=False, indent=1) + "\n")
|
|
82
|
-
|
|
112
|
+
tmp.write_text(json.dumps(ticket, ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
|
|
113
|
+
if not IS_WINDOWS:
|
|
114
|
+
tmp.replace(dest)
|
|
115
|
+
return
|
|
116
|
+
# a reader that has the destination open (without share-delete) makes the replace fail for a moment
|
|
117
|
+
for attempt in range(25):
|
|
118
|
+
try:
|
|
119
|
+
tmp.replace(dest)
|
|
120
|
+
return
|
|
121
|
+
except PermissionError:
|
|
122
|
+
if attempt == 24:
|
|
123
|
+
raise
|
|
124
|
+
time.sleep(0.02)
|
|
83
125
|
|
|
84
126
|
|
|
85
127
|
def read_all():
|
|
@@ -90,7 +132,7 @@ def read_all():
|
|
|
90
132
|
tickets = []
|
|
91
133
|
for p in QUEUE_DIR.glob("*.json"):
|
|
92
134
|
try:
|
|
93
|
-
ticket = json.loads(p.read_text())
|
|
135
|
+
ticket = json.loads(p.read_text(encoding="utf-8"))
|
|
94
136
|
mtime = p.stat().st_mtime
|
|
95
137
|
except (OSError, ValueError):
|
|
96
138
|
continue
|
|
@@ -109,7 +151,7 @@ def sort_key(ticket):
|
|
|
109
151
|
def close_dead(id_):
|
|
110
152
|
"""Re-read after learning the owner died: what it wrote before exiting stands."""
|
|
111
153
|
try:
|
|
112
|
-
ticket = json.loads(ticket_path(id_).read_text())
|
|
154
|
+
ticket = json.loads(ticket_path(id_).read_text(encoding="utf-8"))
|
|
113
155
|
except (OSError, ValueError):
|
|
114
156
|
return
|
|
115
157
|
if ticket.get("state") in ACTIVE:
|
|
@@ -128,7 +170,7 @@ def prune():
|
|
|
128
170
|
for p in (ticket_path(ticket["id"]), QUEUE_DIR / f"{ticket['id']}.log"):
|
|
129
171
|
try:
|
|
130
172
|
p.unlink()
|
|
131
|
-
except FileNotFoundError:
|
|
173
|
+
except (FileNotFoundError, PermissionError):
|
|
132
174
|
pass
|
|
133
175
|
|
|
134
176
|
|
|
@@ -205,9 +247,25 @@ def summary(ticket):
|
|
|
205
247
|
return f"{ticket['tool']} · {ticket.get('video') or '-'} ({who}, pid {ticket['pid']})"
|
|
206
248
|
|
|
207
249
|
|
|
250
|
+
def try_lock(fd):
|
|
251
|
+
"""Takes the machine lock if it is free, without waiting. True when it is ours now."""
|
|
252
|
+
if IS_WINDOWS:
|
|
253
|
+
os.lseek(fd, 0, os.SEEK_SET)
|
|
254
|
+
try:
|
|
255
|
+
msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
|
|
256
|
+
except OSError:
|
|
257
|
+
return False
|
|
258
|
+
return True
|
|
259
|
+
try:
|
|
260
|
+
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
|
261
|
+
return True
|
|
262
|
+
except BlockingIOError:
|
|
263
|
+
return False
|
|
264
|
+
|
|
265
|
+
|
|
208
266
|
def wait(ticket, fd, pause):
|
|
209
267
|
"""Returns the seconds waited when this ticket is the oldest live one waiting
|
|
210
|
-
and the
|
|
268
|
+
and the lock is its own; None if `pause` gives up midway."""
|
|
211
269
|
t0 = time.time()
|
|
212
270
|
said = None
|
|
213
271
|
lock_held_since = None
|
|
@@ -218,12 +276,8 @@ def wait(ticket, fd, pause):
|
|
|
218
276
|
for t in tickets
|
|
219
277
|
if t["effective"] == "waiting" and t["id"] != ticket["id"] and sort_key(t) < sort_key(ticket)
|
|
220
278
|
]
|
|
221
|
-
if not ahead:
|
|
222
|
-
|
|
223
|
-
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
|
224
|
-
return time.time() - t0
|
|
225
|
-
except BlockingIOError:
|
|
226
|
-
pass
|
|
279
|
+
if not ahead and try_lock(fd):
|
|
280
|
+
return time.time() - t0
|
|
227
281
|
running = [t for t in tickets if t["effective"] == "running"]
|
|
228
282
|
# lock busy with nobody running is usually the handover between two tasks
|
|
229
283
|
# (Node closes the ticket an instant before the guardian releases the lock)
|
|
@@ -245,7 +299,7 @@ def wait(ticket, fd, pause):
|
|
|
245
299
|
def begin(ticket, waited):
|
|
246
300
|
ticket.update(state="running", started_at=now())
|
|
247
301
|
write(ticket)
|
|
248
|
-
# with the
|
|
302
|
+
# with the lock in hand, another "running" ticket can only belong to a dead process
|
|
249
303
|
for t in read_all():
|
|
250
304
|
if t["id"] != ticket["id"] and t["state"] == "running" and not alive(t.get("pid")):
|
|
251
305
|
close_dead(t["id"])
|
|
@@ -291,7 +345,7 @@ def inside_turn():
|
|
|
291
345
|
if not id_:
|
|
292
346
|
return False
|
|
293
347
|
try:
|
|
294
|
-
ticket = json.loads(ticket_path(id_).read_text())
|
|
348
|
+
ticket = json.loads(ticket_path(id_).read_text(encoding="utf-8"))
|
|
295
349
|
except (OSError, ValueError):
|
|
296
350
|
return False
|
|
297
351
|
return ticket.get("state") == "running" and alive(ticket.get("pid"))
|
|
@@ -354,13 +408,35 @@ def turn(tool, dir=None):
|
|
|
354
408
|
del os.environ["OMNIVIDEO_TURN"]
|
|
355
409
|
|
|
356
410
|
|
|
411
|
+
class StdinWatcher(threading.Thread):
|
|
412
|
+
"""Reads the pipe from the Node process until it closes. A thread because select() on a pipe is
|
|
413
|
+
POSIX only; `closed` is set when the Node process is gone, in any way."""
|
|
414
|
+
|
|
415
|
+
def __init__(self):
|
|
416
|
+
super().__init__(daemon=True)
|
|
417
|
+
self.closed = threading.Event()
|
|
418
|
+
self.start()
|
|
419
|
+
|
|
420
|
+
def run(self):
|
|
421
|
+
try:
|
|
422
|
+
while os.read(0, 4096):
|
|
423
|
+
pass
|
|
424
|
+
except OSError:
|
|
425
|
+
pass
|
|
426
|
+
self.closed.set()
|
|
427
|
+
|
|
428
|
+
|
|
357
429
|
def guardian(tool, dir, pid, command):
|
|
358
430
|
"""Holds the turn of a Node process (tools/job-queue.mjs).
|
|
359
431
|
|
|
360
432
|
Waits like any task; on its turn, writes one JSON line to stdout and keeps
|
|
361
433
|
reading stdin, which is a pipe from the Node process. The pipe closes when
|
|
362
|
-
it exits, in any way, and then the ticket is closed and the
|
|
434
|
+
it exits, in any way, and then the ticket is closed and the lock released.
|
|
363
435
|
"""
|
|
436
|
+
if IS_WINDOWS:
|
|
437
|
+
# the pipe to Node carries text in the console's code page unless told otherwise
|
|
438
|
+
for stream in (sys.stdout, sys.stderr):
|
|
439
|
+
stream.reconfigure(encoding="utf-8")
|
|
364
440
|
# Ctrl-C kills Node; the guardian stays to close the ticket when the pipe closes
|
|
365
441
|
signal.signal(signal.SIGINT, signal.SIG_IGN)
|
|
366
442
|
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
|
@@ -369,9 +445,10 @@ def guardian(tool, dir, pid, command):
|
|
|
369
445
|
heartbeat = Heartbeat(ticket["id"])
|
|
370
446
|
fd = os.open(LOCK, os.O_RDWR | os.O_CREAT, 0o644)
|
|
371
447
|
|
|
448
|
+
watcher = StdinWatcher()
|
|
449
|
+
|
|
372
450
|
def pause(seconds):
|
|
373
|
-
|
|
374
|
-
return not (readable and not os.read(0, 4096))
|
|
451
|
+
return not watcher.closed.wait(seconds)
|
|
375
452
|
|
|
376
453
|
waited = wait(ticket, fd, pause)
|
|
377
454
|
if waited is None:
|
|
@@ -383,8 +460,7 @@ def guardian(tool, dir, pid, command):
|
|
|
383
460
|
json.dumps({"id": ticket["id"], "file": str(ticket_path(ticket["id"])), "log": str(ROOT / ticket["log"])}),
|
|
384
461
|
flush=True,
|
|
385
462
|
)
|
|
386
|
-
|
|
387
|
-
pass
|
|
463
|
+
watcher.closed.wait()
|
|
388
464
|
close_dead(ticket["id"])
|
|
389
465
|
heartbeat.stop()
|
|
390
466
|
os.close(fd)
|
package/tools/lib/deps.mjs
CHANGED
|
@@ -30,6 +30,9 @@ import {
|
|
|
30
30
|
import { homedir, tmpdir } from "node:os";
|
|
31
31
|
import { delimiter, join } from "node:path";
|
|
32
32
|
import { gunzipSync } from "node:zlib";
|
|
33
|
+
import { venvPython } from "./platform.mjs";
|
|
34
|
+
|
|
35
|
+
export { venvPython };
|
|
33
36
|
|
|
34
37
|
export const OMNI_DIR = process.env.OMNI_DIR || join(homedir(), ".omni");
|
|
35
38
|
export const BIN_DIR = join(OMNI_DIR, "bin");
|
|
@@ -44,10 +47,6 @@ const exe = (/** @type {string} */ name) => (win ? `${name}.exe` : name);
|
|
|
44
47
|
/* ffmpeg-static has no Windows arm64 build: the x64 one runs under emulation */
|
|
45
48
|
const arch = () => (process.arch === "arm64" && !win ? "arm64" : "x64");
|
|
46
49
|
|
|
47
|
-
/** the app's venv Python, as each OS lays a venv out @param {string} codeRoot */
|
|
48
|
-
export const venvPython = codeRoot =>
|
|
49
|
-
win ? join(codeRoot, ".venv", "Scripts", "python.exe") : join(codeRoot, ".venv", "bin", "python");
|
|
50
|
-
|
|
51
50
|
/** @returns {{ chrome?: string }} */
|
|
52
51
|
const readState = () => {
|
|
53
52
|
try {
|
|
@@ -163,8 +162,10 @@ async function ensureUv() {
|
|
|
163
162
|
join(tmp, archive),
|
|
164
163
|
await fetchBytes(`https://github.com/astral-sh/uv/releases/download/${UV_VERSION}/${archive}`),
|
|
165
164
|
);
|
|
166
|
-
/* tar reads both .tar.gz and .zip on macOS and Windows 10+, and .tar.gz on Linux
|
|
167
|
-
|
|
165
|
+
/* tar reads both .tar.gz and .zip on macOS and Windows 10+, and .tar.gz on Linux. On Windows it is the
|
|
166
|
+
system's own bsdtar: a Git Bash `tar` earlier on PATH is GNU tar, which can't open a .zip */
|
|
167
|
+
const tar = win ? join(process.env.SystemRoot ?? "C:\\Windows", "System32", "tar.exe") : "tar";
|
|
168
|
+
const x = spawnSync(tar, ["-xf", archive], { cwd: tmp, encoding: "utf8" });
|
|
168
169
|
if (x.status !== 0) throw new Error(`não consegui abrir o ${archive}: ${x.stderr.trim()}`);
|
|
169
170
|
const found = readdirSync(tmp, { recursive: true, withFileTypes: true }).find(
|
|
170
171
|
e => e.isFile() && e.name === exe("uv"),
|
|
@@ -177,6 +178,17 @@ async function ensureUv() {
|
|
|
177
178
|
return file;
|
|
178
179
|
}
|
|
179
180
|
|
|
181
|
+
/** the arguments that make the app's venv @param {string} codeRoot */
|
|
182
|
+
export const uvVenvArgs = codeRoot => [
|
|
183
|
+
"venv",
|
|
184
|
+
"--python",
|
|
185
|
+
PYTHON_VERSION,
|
|
186
|
+
"--python-preference",
|
|
187
|
+
"only-managed",
|
|
188
|
+
"--allow-existing",
|
|
189
|
+
join(codeRoot, ".venv"),
|
|
190
|
+
];
|
|
191
|
+
|
|
180
192
|
/** the venv with requirements.txt installed @param {string} codeRoot */
|
|
181
193
|
export async function ensurePython(codeRoot) {
|
|
182
194
|
const python = venvPython(codeRoot);
|
|
@@ -187,10 +199,9 @@ export async function ensurePython(codeRoot) {
|
|
|
187
199
|
UV_PYTHON_INSTALL_DIR: join(OMNI_DIR, "python"),
|
|
188
200
|
UV_CACHE_DIR: join(OMNI_DIR, "uv-cache"),
|
|
189
201
|
};
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
});
|
|
202
|
+
/* only-managed: the venv's Python is always uv's own build, never one found on the machine (on Windows the
|
|
203
|
+
Microsoft Store's python.exe stub, which opens the Store instead of running) */
|
|
204
|
+
const venv = spawnSync(uv, uvVenvArgs(codeRoot), { encoding: "utf8", env });
|
|
194
205
|
if (venv.status !== 0) throw new Error(`uv venv falhou: ${venv.stderr.trim().split("\n").at(-1)}`);
|
|
195
206
|
const pip = spawnSync(uv, ["pip", "install", "--python", python, "-r", join(codeRoot, "requirements.txt")], {
|
|
196
207
|
encoding: "utf8",
|