@naiba/pi-durable-background-tasks 0.0.0-stage → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +59 -2
- package/dist/index.d.ts +88 -0
- package/dist/index.js +165 -0
- package/dist/pi.d.ts +3 -0
- package/dist/pi.js +19 -0
- package/dist/process-supervisor.py +94 -0
- package/package.json +47 -4
- package/src/index.ts +99 -0
- package/src/pi.ts +20 -0
- package/src/process-supervisor.py +94 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 naiba
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,60 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Pi Durable Background Tasks
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Supervised background shell commands for Pi Durable. No delegated agents, model configuration, or implicit model calls.
|
|
4
|
+
|
|
5
|
+
**Initial development version. Linux only; requires Python 3.9+ and Linux pidfd support.**
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
bun add @naiba/pi-durable-background-tasks
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
## Usage
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
import {BackgroundTasks} from "@naiba/pi-durable-background-tasks";
|
|
15
|
+
import {backgroundTasksExtension} from "@naiba/pi-durable-background-tasks/pi";
|
|
16
|
+
|
|
17
|
+
const tasks = new BackgroundTasks("/persistent/session/background");
|
|
18
|
+
registry.install(backgroundTasksExtension(tasks, "/workspace"));
|
|
19
|
+
// Register your normal Pi beforeTool approval policy. This package does not bypass it.
|
|
20
|
+
const unsubscribe = tasks.subscribe(() => render(tasks.list()));
|
|
21
|
+
// Host shutdown must await cleanup:
|
|
22
|
+
await tasks.close();
|
|
23
|
+
unsubscribe();
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
Tools: `background_start`, `background_status`, `background_output`, `background_wait`, `background_stop`.
|
|
27
|
+
Wait requires `timeout_ms` between 1 and 600000. Observation timeout/cancellation does not stop the command. Commands have no implicit execution deadline.
|
|
28
|
+
|
|
29
|
+
The core API works without creating a model session. `start` requires a stable `requestId`; repeats return the original record and mismatching arguments are rejected. Recovered records never automatically relaunch a command. Use one service instance per directory and one writer process; concurrent writers are not supported.
|
|
30
|
+
|
|
31
|
+
## Process ownership
|
|
32
|
+
|
|
33
|
+
Each command has a Linux Python subreaper supervisor. A private stdin pipe ties ownership to the host: explicit stop or host pipe loss (including host SIGKILL) initiates cleanup. The supervisor reaps descendants, including ordinary double-fork/setsid escapes. TERM gets a grace interval before KILL; pidfd signalling avoids signalling a recycled PID. Even normal shell completion cleans up descendants before publishing a settled result.
|
|
34
|
+
|
|
35
|
+
This is **lifecycle management, not an OS security sandbox**. Privileged or hostile commands can kill the supervisor, leave its PID namespace, or start processes through an external daemon. Killing both the supervisor and host, uninterruptible kernel tasks, machine failure, and external-service side effects are not covered. Use delegated cgroups/container isolation for stronger enforcement. A supervisor without confirmed cleanup yields `unknown`, never fabricated success. The service never kills from a persisted PID after restart.
|
|
36
|
+
|
|
37
|
+
Records and logs survive reopening; live process continuity across host restart is deliberately not promised. Recovered active records become `unknown`. No automatic model wakeup or replay occurs. Hosts choose how to consume lifecycle events and display unknown execution ownership.
|
|
38
|
+
|
|
39
|
+
Output reads are bounded to 64 KiB, with byte offsets. Logs are retained without automatic deletion; configure filesystem quotas at the host level. Commands run with inherited environment and host permissions. Do not expose this service directly to untrusted clients; isolate task directories by session and authorize every tool/HTTP action.
|
|
40
|
+
|
|
41
|
+
## Development
|
|
42
|
+
|
|
43
|
+
```sh
|
|
44
|
+
bun install
|
|
45
|
+
bun run build
|
|
46
|
+
bun test
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
Tests exercise real Linux processes rather than mocked cancellation. The build includes the Python supervisor beside the JavaScript distribution.
|
|
50
|
+
|
|
51
|
+
## Publishing
|
|
52
|
+
|
|
53
|
+
Releases use `.github/workflows/publish.yml` on pushed `v*` tags, with npm Trusted Publishing (OIDC); no npm token secret is required. Configure this package's GitHub Actions trusted publisher on npm:
|
|
54
|
+
|
|
55
|
+
- Organization or user: `naiba`
|
|
56
|
+
- Repository: `pi-durable-background-tasks`
|
|
57
|
+
- Workflow filename: `publish.yml`
|
|
58
|
+
- Environment: leave empty (the workflow does not declare an environment)
|
|
59
|
+
|
|
60
|
+
Update `package.json` to a new stable version, refresh `bun.lock` if needed, commit and push, then push the matching tag (for example, `v0.1.1` for version `0.1.1`). The workflow rejects mismatching or prerelease versions, runs type checks, build and process tests, then publishes publicly with provenance. Published versions are immutable: `0.1.0` already exists and must not be republished. Never move a published release tag. There is no manual-dispatch publish path.
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
export type BackgroundTask = {
|
|
2
|
+
id: string;
|
|
3
|
+
command: string;
|
|
4
|
+
cwd: string;
|
|
5
|
+
name: string;
|
|
6
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
7
|
+
createdAt: string;
|
|
8
|
+
updatedAt: string;
|
|
9
|
+
exitCode?: number | null;
|
|
10
|
+
error?: string;
|
|
11
|
+
};
|
|
12
|
+
export declare class BackgroundTasks {
|
|
13
|
+
private python;
|
|
14
|
+
private records;
|
|
15
|
+
private owned;
|
|
16
|
+
private listeners;
|
|
17
|
+
private closed;
|
|
18
|
+
readonly directory: string;
|
|
19
|
+
constructor(directory: string, python?: string);
|
|
20
|
+
subscribe(listener: () => void): () => void;
|
|
21
|
+
list(): {
|
|
22
|
+
id: string;
|
|
23
|
+
command: string;
|
|
24
|
+
cwd: string;
|
|
25
|
+
name: string;
|
|
26
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
27
|
+
createdAt: string;
|
|
28
|
+
updatedAt: string;
|
|
29
|
+
exitCode?: number | null;
|
|
30
|
+
error?: string;
|
|
31
|
+
}[];
|
|
32
|
+
get(id: string): {
|
|
33
|
+
id: string;
|
|
34
|
+
command: string;
|
|
35
|
+
cwd: string;
|
|
36
|
+
name: string;
|
|
37
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
38
|
+
createdAt: string;
|
|
39
|
+
updatedAt: string;
|
|
40
|
+
exitCode?: number | null;
|
|
41
|
+
error?: string;
|
|
42
|
+
};
|
|
43
|
+
get active(): boolean;
|
|
44
|
+
private update;
|
|
45
|
+
start(input: {
|
|
46
|
+
requestId: string;
|
|
47
|
+
command: string;
|
|
48
|
+
cwd: string;
|
|
49
|
+
name?: string;
|
|
50
|
+
}): {
|
|
51
|
+
id: string;
|
|
52
|
+
command: string;
|
|
53
|
+
cwd: string;
|
|
54
|
+
name: string;
|
|
55
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
56
|
+
createdAt: string;
|
|
57
|
+
updatedAt: string;
|
|
58
|
+
exitCode?: number | null;
|
|
59
|
+
error?: string;
|
|
60
|
+
};
|
|
61
|
+
output(id: string, offset?: number, limit?: number): Promise<{
|
|
62
|
+
text: string;
|
|
63
|
+
nextOffset: number;
|
|
64
|
+
}>;
|
|
65
|
+
wait(id: string, timeoutMs: number, signal?: AbortSignal): Promise<{
|
|
66
|
+
id: string;
|
|
67
|
+
command: string;
|
|
68
|
+
cwd: string;
|
|
69
|
+
name: string;
|
|
70
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
71
|
+
createdAt: string;
|
|
72
|
+
updatedAt: string;
|
|
73
|
+
exitCode?: number | null;
|
|
74
|
+
error?: string;
|
|
75
|
+
}>;
|
|
76
|
+
stop(id: string): Promise<{
|
|
77
|
+
id: string;
|
|
78
|
+
command: string;
|
|
79
|
+
cwd: string;
|
|
80
|
+
name: string;
|
|
81
|
+
status: "starting" | "running" | "stopping" | "completed" | "failed" | "stopped" | "unknown";
|
|
82
|
+
createdAt: string;
|
|
83
|
+
updatedAt: string;
|
|
84
|
+
exitCode?: number | null;
|
|
85
|
+
error?: string;
|
|
86
|
+
}>;
|
|
87
|
+
close(): Promise<void>;
|
|
88
|
+
}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
import { spawn } from "node:child_process";
|
|
2
|
+
import { mkdirSync, readFileSync, readdirSync, writeFileSync, renameSync } from "node:fs";
|
|
3
|
+
import { open } from "node:fs/promises";
|
|
4
|
+
import { join, resolve } from "node:path";
|
|
5
|
+
import { fileURLToPath } from "node:url";
|
|
6
|
+
import { createHash } from "node:crypto";
|
|
7
|
+
export class BackgroundTasks {
|
|
8
|
+
python;
|
|
9
|
+
records = new Map();
|
|
10
|
+
owned = new Map();
|
|
11
|
+
listeners = new Set();
|
|
12
|
+
closed = false;
|
|
13
|
+
directory;
|
|
14
|
+
constructor(directory, python = "python3") {
|
|
15
|
+
this.python = python;
|
|
16
|
+
if (process.platform !== "linux")
|
|
17
|
+
throw new Error("Background task supervision currently requires Linux");
|
|
18
|
+
this.directory = resolve(directory);
|
|
19
|
+
mkdirSync(this.directory, { recursive: true, mode: 0o700 });
|
|
20
|
+
for (const file of readdirSync(this.directory).filter(value => /^[a-f0-9]{64}\.json$/.test(value))) {
|
|
21
|
+
const task = JSON.parse(readFileSync(join(this.directory, file), "utf8"));
|
|
22
|
+
if (task.id + ".json" !== file)
|
|
23
|
+
throw new Error("Invalid background task identity");
|
|
24
|
+
this.records.set(task.id, task);
|
|
25
|
+
// Persisted PIDs are never used for control after restart: ownership is unknown.
|
|
26
|
+
if (["starting", "running", "stopping"].includes(task.status))
|
|
27
|
+
this.update(task, { status: "unknown", error: "Host restarted; command is not replayed and process ownership cannot be confirmed" });
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
subscribe(listener) { this.listeners.add(listener); return () => { this.listeners.delete(listener); }; }
|
|
31
|
+
list() { return [...this.records.values()].map(task => ({ ...task })); }
|
|
32
|
+
get(id) { const task = this.records.get(id); if (!task)
|
|
33
|
+
throw new Error("Background task not found"); return { ...task }; }
|
|
34
|
+
get active() { return this.owned.size > 0; }
|
|
35
|
+
update(task, changes = {}) {
|
|
36
|
+
Object.assign(task, changes, { updatedAt: new Date().toISOString() });
|
|
37
|
+
const path = join(this.directory, task.id + ".json");
|
|
38
|
+
writeFileSync(path + ".pending", JSON.stringify(task), { mode: 0o600 });
|
|
39
|
+
renameSync(path + ".pending", path);
|
|
40
|
+
for (const listener of this.listeners) {
|
|
41
|
+
try {
|
|
42
|
+
listener();
|
|
43
|
+
}
|
|
44
|
+
catch { /* Observers cannot break process cleanup. */ }
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
start(input) {
|
|
48
|
+
if (this.closed)
|
|
49
|
+
throw new Error("Background task service is closed");
|
|
50
|
+
if (!input.requestId || !input.command.trim())
|
|
51
|
+
throw new Error("requestId and command are required");
|
|
52
|
+
const id = createHash("sha256").update(input.requestId).digest("hex");
|
|
53
|
+
const cwd = resolve(input.cwd), name = input.name || input.command.slice(0, 100);
|
|
54
|
+
const existing = this.records.get(id);
|
|
55
|
+
if (existing) {
|
|
56
|
+
if (existing.command !== input.command || existing.cwd !== cwd || existing.name !== name)
|
|
57
|
+
throw new Error("Request identity conflicts with earlier command");
|
|
58
|
+
return { ...existing };
|
|
59
|
+
}
|
|
60
|
+
const task = { id, command: input.command, cwd, name, status: "starting", createdAt: new Date().toISOString(), updatedAt: new Date().toISOString() };
|
|
61
|
+
this.records.set(id, task);
|
|
62
|
+
this.update(task);
|
|
63
|
+
const child = spawn(this.python, [fileURLToPath(new URL("./process-supervisor.py", import.meta.url))], { stdio: "pipe" });
|
|
64
|
+
let finish;
|
|
65
|
+
const done = new Promise(resolve => { finish = resolve; });
|
|
66
|
+
this.owned.set(id, { child, done });
|
|
67
|
+
let buffer = "", diagnostic = "", settled = false;
|
|
68
|
+
child.stdout.on("data", chunk => {
|
|
69
|
+
buffer += chunk.toString();
|
|
70
|
+
for (let position; (position = buffer.indexOf("\n")) >= 0;) {
|
|
71
|
+
const line = buffer.slice(0, position);
|
|
72
|
+
buffer = buffer.slice(position + 1);
|
|
73
|
+
try {
|
|
74
|
+
const event = JSON.parse(line);
|
|
75
|
+
if (event.event === "started" && task.status === "starting")
|
|
76
|
+
this.update(task, { status: "running" });
|
|
77
|
+
if (event.event === "settled") {
|
|
78
|
+
settled = true;
|
|
79
|
+
this.update(task, { status: event.status, exitCode: event.exitCode });
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
catch {
|
|
83
|
+
diagnostic = "Invalid supervisor response";
|
|
84
|
+
child.stdin.end();
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
});
|
|
88
|
+
child.stderr.on("data", chunk => { diagnostic = (diagnostic + chunk.toString()).slice(-4000); });
|
|
89
|
+
child.stdin.on("error", () => { });
|
|
90
|
+
child.on("error", error => { diagnostic = error.message; });
|
|
91
|
+
child.on("close", () => {
|
|
92
|
+
try {
|
|
93
|
+
if (!settled)
|
|
94
|
+
this.update(task, { status: "unknown", error: diagnostic || "Supervisor exited without confirming cleanup" });
|
|
95
|
+
}
|
|
96
|
+
finally {
|
|
97
|
+
this.owned.delete(id);
|
|
98
|
+
finish();
|
|
99
|
+
for (const listener of this.listeners) {
|
|
100
|
+
try {
|
|
101
|
+
listener();
|
|
102
|
+
}
|
|
103
|
+
catch { }
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
});
|
|
107
|
+
child.stdin.write(JSON.stringify({ command: task.command, cwd, log: join(this.directory, id + ".log") }) + "\n");
|
|
108
|
+
return { ...task };
|
|
109
|
+
}
|
|
110
|
+
async output(id, offset = 0, limit = 32768) {
|
|
111
|
+
this.get(id);
|
|
112
|
+
if (!Number.isSafeInteger(offset) || offset < 0 || !Number.isInteger(limit) || limit < 1 || limit > 65536)
|
|
113
|
+
throw new Error("Invalid output bounds");
|
|
114
|
+
let file;
|
|
115
|
+
try {
|
|
116
|
+
file = await open(join(this.directory, id + ".log"), "r");
|
|
117
|
+
}
|
|
118
|
+
catch (error) {
|
|
119
|
+
if (error.code === "ENOENT")
|
|
120
|
+
return { text: "", nextOffset: offset };
|
|
121
|
+
throw error;
|
|
122
|
+
}
|
|
123
|
+
try {
|
|
124
|
+
const buffer = Buffer.alloc(limit);
|
|
125
|
+
const { bytesRead } = await file.read(buffer, 0, limit, offset);
|
|
126
|
+
return { text: buffer.subarray(0, bytesRead).toString("utf8"), nextOffset: offset + bytesRead };
|
|
127
|
+
}
|
|
128
|
+
finally {
|
|
129
|
+
await file.close();
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
async wait(id, timeoutMs, signal) {
|
|
133
|
+
this.get(id);
|
|
134
|
+
if (!Number.isInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 600000)
|
|
135
|
+
throw new Error("timeoutMs must be 1..600000");
|
|
136
|
+
signal?.throwIfAborted();
|
|
137
|
+
const owner = this.owned.get(id);
|
|
138
|
+
if (!owner)
|
|
139
|
+
return this.get(id);
|
|
140
|
+
await new Promise((resolve, reject) => {
|
|
141
|
+
const finish = () => { clearTimeout(timer); signal?.removeEventListener("abort", abort); resolve(); };
|
|
142
|
+
const abort = () => { clearTimeout(timer); signal?.removeEventListener("abort", abort); reject(signal?.reason); };
|
|
143
|
+
const timer = setTimeout(finish, timeoutMs);
|
|
144
|
+
signal?.addEventListener("abort", abort, { once: true });
|
|
145
|
+
void owner.done.then(finish);
|
|
146
|
+
});
|
|
147
|
+
return this.get(id);
|
|
148
|
+
}
|
|
149
|
+
async stop(id) {
|
|
150
|
+
const task = this.records.get(id);
|
|
151
|
+
if (!task)
|
|
152
|
+
throw new Error("Background task not found");
|
|
153
|
+
const owner = this.owned.get(id);
|
|
154
|
+
if (!owner) {
|
|
155
|
+
if (task.status === "unknown")
|
|
156
|
+
throw new Error("Cannot stop a process without verified ownership");
|
|
157
|
+
return { ...task };
|
|
158
|
+
}
|
|
159
|
+
this.update(task, { status: "stopping" });
|
|
160
|
+
owner.child.stdin.end();
|
|
161
|
+
await owner.done;
|
|
162
|
+
return this.get(id);
|
|
163
|
+
}
|
|
164
|
+
async close() { this.closed = true; await Promise.all([...this.owned.keys()].map(id => this.stop(id))); }
|
|
165
|
+
}
|
package/dist/pi.d.ts
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
import { Type } from "@earendil-works/pi-ai";
|
|
2
|
+
import { BackgroundTasks } from "./index.js";
|
|
3
|
+
export declare function backgroundTasksExtension(service: BackgroundTasks, cwd: string): import("@earendil-works/pi-durable").Extension<import("@earendil-works/pi-durable").ToolRegistration<Type.TObject<{}>, import("@earendil-works/chord").JsonValue>>;
|
package/dist/pi.js
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
import { defineExtension, defineTool } from "@earendil-works/pi-durable";
|
|
2
|
+
import { Type } from "@earendil-works/pi-ai";
|
|
3
|
+
const result = (value) => ({ content: [{ type: "text", text: JSON.stringify(value) }] });
|
|
4
|
+
export function backgroundTasksExtension(service, cwd) {
|
|
5
|
+
return defineExtension({ name: "background-tasks", tools: [
|
|
6
|
+
defineTool({ name: "background_start", description: "Start a supervised background shell command. Returns immediately; no model delegation. Commands do not survive host loss. Approval belongs to the host tool policy.", replay: "unsafe",
|
|
7
|
+
parameters: Type.Object({ command: Type.String(), name: Type.Optional(Type.String()) }),
|
|
8
|
+
execute: async (args, api) => result(service.start({ ...args, cwd, requestId: String(api.taskId) })) }),
|
|
9
|
+
defineTool({ name: "background_status", description: "List background commands owned by this session.", replay: "safe", parameters: Type.Object({}), execute: async () => result(service.list()) }),
|
|
10
|
+
defineTool({ name: "background_output", description: "Read bounded background output using byte offsets.", replay: "safe",
|
|
11
|
+
parameters: Type.Object({ id: Type.String(), offset: Type.Optional(Type.Integer({ minimum: 0 })), limit: Type.Optional(Type.Integer({ minimum: 1, maximum: 65536 })) }),
|
|
12
|
+
execute: async (args) => result(await service.output(args.id, args.offset, args.limit)) }),
|
|
13
|
+
defineTool({ name: "background_wait", description: "Wait for a background command. Explicit timeout ends observation only, not execution.", replay: "safe",
|
|
14
|
+
parameters: Type.Object({ id: Type.String(), timeout_ms: Type.Integer({ minimum: 1, maximum: 600000 }) }),
|
|
15
|
+
execute: async (args, api, ctx) => result(await service.wait(args.id, args.timeout_ms, ctx.abortSignal)) }),
|
|
16
|
+
defineTool({ name: "background_stop", description: "Stop a session-owned command and reap its descendants. Returns after supervisor cleanup.", replay: "unsafe",
|
|
17
|
+
parameters: Type.Object({ id: Type.String() }), execute: async (args) => result(await service.stop(args.id)) }),
|
|
18
|
+
] });
|
|
19
|
+
}
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
"""Linux command owner. EOF on the host pipe means cleanup, never detach."""
|
|
2
|
+
import ctypes
|
|
3
|
+
import json
|
|
4
|
+
import os
|
|
5
|
+
import selectors
|
|
6
|
+
import signal
|
|
7
|
+
import subprocess
|
|
8
|
+
import sys
|
|
9
|
+
import time
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def children():
|
|
13
|
+
# Subreaper adoption keeps double-fork/setsid descendants under our ownership.
|
|
14
|
+
found = set()
|
|
15
|
+
pending = [os.getpid()]
|
|
16
|
+
while pending:
|
|
17
|
+
parent = pending.pop()
|
|
18
|
+
try:
|
|
19
|
+
with open(f"/proc/{parent}/task/{parent}/children") as stream:
|
|
20
|
+
direct = [int(value) for value in stream.read().split()]
|
|
21
|
+
except FileNotFoundError:
|
|
22
|
+
continue
|
|
23
|
+
for pid in direct:
|
|
24
|
+
if pid not in found:
|
|
25
|
+
found.add(pid)
|
|
26
|
+
pending.append(pid)
|
|
27
|
+
return found
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
def reap():
|
|
31
|
+
while True:
|
|
32
|
+
try:
|
|
33
|
+
if os.waitpid(-1, os.WNOHANG)[0] == 0:
|
|
34
|
+
return
|
|
35
|
+
except ChildProcessError:
|
|
36
|
+
return
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
def cleanup(grace):
|
|
40
|
+
deadline = time.monotonic() + grace
|
|
41
|
+
while True:
|
|
42
|
+
owned = children()
|
|
43
|
+
if not owned:
|
|
44
|
+
reap()
|
|
45
|
+
return
|
|
46
|
+
for pid in owned:
|
|
47
|
+
try:
|
|
48
|
+
# pidfds prevent a recycled PID from targeting an unrelated process.
|
|
49
|
+
fd = os.pidfd_open(pid)
|
|
50
|
+
try:
|
|
51
|
+
if pid in children():
|
|
52
|
+
signal.pidfd_send_signal(fd, signal.SIGTERM if time.monotonic() < deadline else signal.SIGKILL)
|
|
53
|
+
finally:
|
|
54
|
+
os.close(fd)
|
|
55
|
+
except ProcessLookupError:
|
|
56
|
+
pass
|
|
57
|
+
reap()
|
|
58
|
+
time.sleep(0.02)
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def main():
|
|
62
|
+
libc = ctypes.CDLL(None, use_errno=True)
|
|
63
|
+
if libc.prctl(36, 1, 0, 0, 0) != 0: # PR_SET_CHILD_SUBREAPER
|
|
64
|
+
raise OSError(ctypes.get_errno(), "Cannot become a child subreaper")
|
|
65
|
+
os.umask(0o077)
|
|
66
|
+
config = json.loads(sys.stdin.buffer.readline())
|
|
67
|
+
stopped = False
|
|
68
|
+
def stop(signum, frame):
|
|
69
|
+
nonlocal stopped
|
|
70
|
+
stopped = True
|
|
71
|
+
signal.signal(signal.SIGTERM, stop)
|
|
72
|
+
signal.signal(signal.SIGINT, stop)
|
|
73
|
+
selector = selectors.DefaultSelector()
|
|
74
|
+
selector.register(sys.stdin, selectors.EVENT_READ)
|
|
75
|
+
process = None
|
|
76
|
+
try:
|
|
77
|
+
with open(config["log"], "ab", buffering=0) as output:
|
|
78
|
+
process = subprocess.Popen(["/bin/sh", "-c", config["command"]], cwd=config["cwd"],
|
|
79
|
+
stdin=subprocess.DEVNULL, stdout=output, stderr=output, start_new_session=True)
|
|
80
|
+
print(json.dumps({"event": "started", "pid": process.pid}), flush=True)
|
|
81
|
+
while process.poll() is None and not stopped:
|
|
82
|
+
if selector.select(0.05):
|
|
83
|
+
# A stop byte OR EOF (including host SIGKILL) requests cleanup.
|
|
84
|
+
os.read(sys.stdin.fileno(), 4096)
|
|
85
|
+
stopped = True
|
|
86
|
+
exit_code = process.poll()
|
|
87
|
+
finally:
|
|
88
|
+
cleanup(config.get("graceSeconds", 1))
|
|
89
|
+
selector.close()
|
|
90
|
+
print(json.dumps({"event": "settled", "status": "stopped" if stopped else "completed" if exit_code == 0 else "failed", "exitCode": exit_code}), flush=True)
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
if __name__ == "__main__":
|
|
94
|
+
main()
|
package/package.json
CHANGED
|
@@ -1,6 +1,49 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@naiba/pi-durable-background-tasks",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.1",
|
|
4
|
+
"description": "Background shell tools for Pi Durable; no model delegation",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"repository": {
|
|
8
|
+
"type": "git",
|
|
9
|
+
"url": "git+https://github.com/naiba/pi-durable-background-tasks.git"
|
|
10
|
+
},
|
|
11
|
+
"exports": {
|
|
12
|
+
".": {
|
|
13
|
+
"bun": "./src/index.ts",
|
|
14
|
+
"types": "./src/index.ts",
|
|
15
|
+
"default": "./dist/index.js"
|
|
16
|
+
},
|
|
17
|
+
"./pi": {
|
|
18
|
+
"bun": "./src/pi.ts",
|
|
19
|
+
"types": "./src/pi.ts",
|
|
20
|
+
"default": "./dist/pi.js"
|
|
21
|
+
}
|
|
22
|
+
},
|
|
23
|
+
"types": "./dist/index.d.ts",
|
|
24
|
+
"files": [
|
|
25
|
+
"dist",
|
|
26
|
+
"src",
|
|
27
|
+
"README.md",
|
|
28
|
+
"LICENSE"
|
|
29
|
+
],
|
|
30
|
+
"engines": {
|
|
31
|
+
"node": ">=22"
|
|
32
|
+
},
|
|
33
|
+
"scripts": {
|
|
34
|
+
"build": "tsc && cp src/process-supervisor.py dist/process-supervisor.py",
|
|
35
|
+
"test": "bun test",
|
|
36
|
+
"check": "tsc --noEmit"
|
|
37
|
+
},
|
|
38
|
+
"peerDependencies": {
|
|
39
|
+
"@earendil-works/pi-durable": "^1.0.4",
|
|
40
|
+
"@earendil-works/pi-ai": "^1.0.4"
|
|
41
|
+
},
|
|
42
|
+
"devDependencies": {
|
|
43
|
+
"@types/node": "^24.0.0",
|
|
44
|
+
"@types/bun": "latest",
|
|
45
|
+
"typescript": "~5.9.3",
|
|
46
|
+
"@earendil-works/pi-durable": "1.0.4",
|
|
47
|
+
"@earendil-works/pi-ai": "1.0.4"
|
|
48
|
+
}
|
|
49
|
+
}
|
package/src/index.ts
ADDED
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
import {spawn, type ChildProcessWithoutNullStreams} from "node:child_process";
|
|
2
|
+
import {mkdirSync, readFileSync, readdirSync, writeFileSync, renameSync} from "node:fs";
|
|
3
|
+
import {open} from "node:fs/promises";
|
|
4
|
+
import {join, resolve} from "node:path";
|
|
5
|
+
import {fileURLToPath} from "node:url";
|
|
6
|
+
import {createHash} from "node:crypto";
|
|
7
|
+
|
|
8
|
+
export type BackgroundTask = {
|
|
9
|
+
id:string; command:string; cwd:string; name:string;
|
|
10
|
+
status:"starting"|"running"|"stopping"|"completed"|"failed"|"stopped"|"unknown";
|
|
11
|
+
createdAt:string; updatedAt:string; exitCode?:number|null; error?:string;
|
|
12
|
+
};
|
|
13
|
+
type Owned = {child:ChildProcessWithoutNullStreams; done:Promise<void>};
|
|
14
|
+
export class BackgroundTasks {
|
|
15
|
+
private records = new Map<string,BackgroundTask>();
|
|
16
|
+
private owned = new Map<string,Owned>();
|
|
17
|
+
private listeners = new Set<()=>void>();
|
|
18
|
+
private closed=false;
|
|
19
|
+
readonly directory:string;
|
|
20
|
+
constructor(directory:string, private python="python3") {
|
|
21
|
+
if(process.platform!=="linux")throw new Error("Background task supervision currently requires Linux");
|
|
22
|
+
this.directory=resolve(directory);mkdirSync(this.directory,{recursive:true,mode:0o700});
|
|
23
|
+
for(const file of readdirSync(this.directory).filter(value=>/^[a-f0-9]{64}\.json$/.test(value))){
|
|
24
|
+
const task=JSON.parse(readFileSync(join(this.directory,file),"utf8")) as BackgroundTask;
|
|
25
|
+
if(task.id+".json"!==file)throw new Error("Invalid background task identity");
|
|
26
|
+
this.records.set(task.id,task);
|
|
27
|
+
// Persisted PIDs are never used for control after restart: ownership is unknown.
|
|
28
|
+
if(["starting","running","stopping"].includes(task.status))this.update(task,{status:"unknown",error:"Host restarted; command is not replayed and process ownership cannot be confirmed"});
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
subscribe(listener:()=>void){this.listeners.add(listener);return ()=>{this.listeners.delete(listener);};}
|
|
32
|
+
list(){return [...this.records.values()].map(task=>({...task}));}
|
|
33
|
+
get(id:string){const task=this.records.get(id);if(!task)throw new Error("Background task not found");return {...task};}
|
|
34
|
+
get active(){return this.owned.size>0;}
|
|
35
|
+
private update(task:BackgroundTask, changes:Partial<BackgroundTask>={}){
|
|
36
|
+
Object.assign(task,changes,{updatedAt:new Date().toISOString()});
|
|
37
|
+
const path=join(this.directory,task.id+".json");
|
|
38
|
+
writeFileSync(path+".pending",JSON.stringify(task),{mode:0o600});renameSync(path+".pending",path);
|
|
39
|
+
for(const listener of this.listeners){try{listener();}catch{/* Observers cannot break process cleanup. */}}
|
|
40
|
+
}
|
|
41
|
+
start(input:{requestId:string;command:string;cwd:string;name?:string}){
|
|
42
|
+
if(this.closed)throw new Error("Background task service is closed");
|
|
43
|
+
if(!input.requestId || !input.command.trim())throw new Error("requestId and command are required");
|
|
44
|
+
const id=createHash("sha256").update(input.requestId).digest("hex");
|
|
45
|
+
const cwd=resolve(input.cwd),name=input.name||input.command.slice(0,100);
|
|
46
|
+
const existing=this.records.get(id);
|
|
47
|
+
if(existing){if(existing.command!==input.command||existing.cwd!==cwd||existing.name!==name)throw new Error("Request identity conflicts with earlier command");return {...existing};}
|
|
48
|
+
const task:BackgroundTask={id,command:input.command,cwd,name,status:"starting",createdAt:new Date().toISOString(),updatedAt:new Date().toISOString()};
|
|
49
|
+
this.records.set(id,task);this.update(task);
|
|
50
|
+
const child=spawn(this.python,[fileURLToPath(new URL("./process-supervisor.py",import.meta.url))],{stdio:"pipe"});
|
|
51
|
+
let finish!:()=>void;const done=new Promise<void>(resolve=>{finish=resolve;});
|
|
52
|
+
this.owned.set(id,{child,done});
|
|
53
|
+
let buffer="",diagnostic="",settled=false;
|
|
54
|
+
child.stdout.on("data",chunk=>{
|
|
55
|
+
buffer+=chunk.toString();
|
|
56
|
+
for(let position;(position=buffer.indexOf("\n"))>=0;){
|
|
57
|
+
const line=buffer.slice(0,position);buffer=buffer.slice(position+1);
|
|
58
|
+
try{const event=JSON.parse(line);
|
|
59
|
+
if(event.event==="started"&&task.status==="starting")this.update(task,{status:"running"});
|
|
60
|
+
if(event.event==="settled"){settled=true;this.update(task,{status:event.status,exitCode:event.exitCode});}
|
|
61
|
+
}catch{diagnostic="Invalid supervisor response";child.stdin.end();}
|
|
62
|
+
}
|
|
63
|
+
});
|
|
64
|
+
child.stderr.on("data",chunk=>{diagnostic=(diagnostic+chunk.toString()).slice(-4000);});
|
|
65
|
+
child.stdin.on("error",()=>{/* close/error below owns final state */});
|
|
66
|
+
child.on("error",error=>{diagnostic=error.message;});
|
|
67
|
+
child.on("close",()=>{
|
|
68
|
+
try{if(!settled)this.update(task,{status:"unknown",error:diagnostic||"Supervisor exited without confirming cleanup"});}
|
|
69
|
+
finally{this.owned.delete(id);finish();for(const listener of this.listeners){try{listener();}catch{}}}
|
|
70
|
+
});
|
|
71
|
+
child.stdin.write(JSON.stringify({command:task.command,cwd,log:join(this.directory,id+".log")})+"\n");
|
|
72
|
+
return {...task};
|
|
73
|
+
}
|
|
74
|
+
async output(id:string,offset=0,limit=32768){
|
|
75
|
+
this.get(id);
|
|
76
|
+
if(!Number.isSafeInteger(offset)||offset<0||!Number.isInteger(limit)||limit<1||limit>65536)throw new Error("Invalid output bounds");
|
|
77
|
+
let file;try{file=await open(join(this.directory,id+".log"),"r");}catch(error){if((error as NodeJS.ErrnoException).code==="ENOENT")return {text:"",nextOffset:offset};throw error;}
|
|
78
|
+
try{const buffer=Buffer.alloc(limit);const {bytesRead}=await file.read(buffer,0,limit,offset);return {text:buffer.subarray(0,bytesRead).toString("utf8"),nextOffset:offset+bytesRead};}finally{await file.close();}
|
|
79
|
+
}
|
|
80
|
+
async wait(id:string,timeoutMs:number,signal?:AbortSignal){
|
|
81
|
+
this.get(id);
|
|
82
|
+
if(!Number.isInteger(timeoutMs)||timeoutMs<1||timeoutMs>600000)throw new Error("timeoutMs must be 1..600000");
|
|
83
|
+
signal?.throwIfAborted();
|
|
84
|
+
const owner=this.owned.get(id);if(!owner)return this.get(id);
|
|
85
|
+
await new Promise<void>((resolve,reject)=>{
|
|
86
|
+
const finish=()=>{clearTimeout(timer);signal?.removeEventListener("abort",abort);resolve();};
|
|
87
|
+
const abort=()=>{clearTimeout(timer);signal?.removeEventListener("abort",abort);reject(signal?.reason);};
|
|
88
|
+
const timer=setTimeout(finish,timeoutMs);signal?.addEventListener("abort",abort,{once:true});void owner.done.then(finish);
|
|
89
|
+
});
|
|
90
|
+
return this.get(id);
|
|
91
|
+
}
|
|
92
|
+
async stop(id:string){
|
|
93
|
+
const task=this.records.get(id);if(!task)throw new Error("Background task not found");
|
|
94
|
+
const owner=this.owned.get(id);
|
|
95
|
+
if(!owner){if(task.status==="unknown")throw new Error("Cannot stop a process without verified ownership");return {...task};}
|
|
96
|
+
this.update(task,{status:"stopping"});owner.child.stdin.end();await owner.done;return this.get(id);
|
|
97
|
+
}
|
|
98
|
+
async close(){this.closed=true;await Promise.all([...this.owned.keys()].map(id=>this.stop(id)));}
|
|
99
|
+
}
|
package/src/pi.ts
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
import {defineExtension,defineTool} from "@earendil-works/pi-durable";
|
|
2
|
+
import {Type} from "@earendil-works/pi-ai";
|
|
3
|
+
import {BackgroundTasks} from "./index.js";
|
|
4
|
+
const result=(value:unknown)=>({content:[{type:"text" as const,text:JSON.stringify(value)}]});
|
|
5
|
+
export function backgroundTasksExtension(service:BackgroundTasks,cwd:string){
|
|
6
|
+
return defineExtension({name:"background-tasks",tools:[
|
|
7
|
+
defineTool({name:"background_start",description:"Start a supervised background shell command. Returns immediately; no model delegation. Commands do not survive host loss. Approval belongs to the host tool policy.",replay:"unsafe",
|
|
8
|
+
parameters:Type.Object({command:Type.String(),name:Type.Optional(Type.String())}),
|
|
9
|
+
execute:async(args,api)=>result(service.start({...args,cwd,requestId:String(api.taskId)}))}),
|
|
10
|
+
defineTool({name:"background_status",description:"List background commands owned by this session.",replay:"safe",parameters:Type.Object({}),execute:async()=>result(service.list())}),
|
|
11
|
+
defineTool({name:"background_output",description:"Read bounded background output using byte offsets.",replay:"safe",
|
|
12
|
+
parameters:Type.Object({id:Type.String(),offset:Type.Optional(Type.Integer({minimum:0})),limit:Type.Optional(Type.Integer({minimum:1,maximum:65536}))}),
|
|
13
|
+
execute:async(args)=>result(await service.output(args.id,args.offset,args.limit))}),
|
|
14
|
+
defineTool({name:"background_wait",description:"Wait for a background command. Explicit timeout ends observation only, not execution.",replay:"safe",
|
|
15
|
+
parameters:Type.Object({id:Type.String(),timeout_ms:Type.Integer({minimum:1,maximum:600000})}),
|
|
16
|
+
execute:async(args,api,ctx)=>result(await service.wait(args.id,args.timeout_ms,ctx.abortSignal))}),
|
|
17
|
+
defineTool({name:"background_stop",description:"Stop a session-owned command and reap its descendants. Returns after supervisor cleanup.",replay:"unsafe",
|
|
18
|
+
parameters:Type.Object({id:Type.String()}),execute:async(args)=>result(await service.stop(args.id))}),
|
|
19
|
+
]});
|
|
20
|
+
}
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
"""Linux command owner. EOF on the host pipe means cleanup, never detach."""
|
|
2
|
+
import ctypes
|
|
3
|
+
import json
|
|
4
|
+
import os
|
|
5
|
+
import selectors
|
|
6
|
+
import signal
|
|
7
|
+
import subprocess
|
|
8
|
+
import sys
|
|
9
|
+
import time
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def children():
|
|
13
|
+
# Subreaper adoption keeps double-fork/setsid descendants under our ownership.
|
|
14
|
+
found = set()
|
|
15
|
+
pending = [os.getpid()]
|
|
16
|
+
while pending:
|
|
17
|
+
parent = pending.pop()
|
|
18
|
+
try:
|
|
19
|
+
with open(f"/proc/{parent}/task/{parent}/children") as stream:
|
|
20
|
+
direct = [int(value) for value in stream.read().split()]
|
|
21
|
+
except FileNotFoundError:
|
|
22
|
+
continue
|
|
23
|
+
for pid in direct:
|
|
24
|
+
if pid not in found:
|
|
25
|
+
found.add(pid)
|
|
26
|
+
pending.append(pid)
|
|
27
|
+
return found
|
|
28
|
+
|
|
29
|
+
|
|
30
|
+
def reap():
|
|
31
|
+
while True:
|
|
32
|
+
try:
|
|
33
|
+
if os.waitpid(-1, os.WNOHANG)[0] == 0:
|
|
34
|
+
return
|
|
35
|
+
except ChildProcessError:
|
|
36
|
+
return
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
def cleanup(grace):
|
|
40
|
+
deadline = time.monotonic() + grace
|
|
41
|
+
while True:
|
|
42
|
+
owned = children()
|
|
43
|
+
if not owned:
|
|
44
|
+
reap()
|
|
45
|
+
return
|
|
46
|
+
for pid in owned:
|
|
47
|
+
try:
|
|
48
|
+
# pidfds prevent a recycled PID from targeting an unrelated process.
|
|
49
|
+
fd = os.pidfd_open(pid)
|
|
50
|
+
try:
|
|
51
|
+
if pid in children():
|
|
52
|
+
signal.pidfd_send_signal(fd, signal.SIGTERM if time.monotonic() < deadline else signal.SIGKILL)
|
|
53
|
+
finally:
|
|
54
|
+
os.close(fd)
|
|
55
|
+
except ProcessLookupError:
|
|
56
|
+
pass
|
|
57
|
+
reap()
|
|
58
|
+
time.sleep(0.02)
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def main():
|
|
62
|
+
libc = ctypes.CDLL(None, use_errno=True)
|
|
63
|
+
if libc.prctl(36, 1, 0, 0, 0) != 0: # PR_SET_CHILD_SUBREAPER
|
|
64
|
+
raise OSError(ctypes.get_errno(), "Cannot become a child subreaper")
|
|
65
|
+
os.umask(0o077)
|
|
66
|
+
config = json.loads(sys.stdin.buffer.readline())
|
|
67
|
+
stopped = False
|
|
68
|
+
def stop(signum, frame):
|
|
69
|
+
nonlocal stopped
|
|
70
|
+
stopped = True
|
|
71
|
+
signal.signal(signal.SIGTERM, stop)
|
|
72
|
+
signal.signal(signal.SIGINT, stop)
|
|
73
|
+
selector = selectors.DefaultSelector()
|
|
74
|
+
selector.register(sys.stdin, selectors.EVENT_READ)
|
|
75
|
+
process = None
|
|
76
|
+
try:
|
|
77
|
+
with open(config["log"], "ab", buffering=0) as output:
|
|
78
|
+
process = subprocess.Popen(["/bin/sh", "-c", config["command"]], cwd=config["cwd"],
|
|
79
|
+
stdin=subprocess.DEVNULL, stdout=output, stderr=output, start_new_session=True)
|
|
80
|
+
print(json.dumps({"event": "started", "pid": process.pid}), flush=True)
|
|
81
|
+
while process.poll() is None and not stopped:
|
|
82
|
+
if selector.select(0.05):
|
|
83
|
+
# A stop byte OR EOF (including host SIGKILL) requests cleanup.
|
|
84
|
+
os.read(sys.stdin.fileno(), 4096)
|
|
85
|
+
stopped = True
|
|
86
|
+
exit_code = process.poll()
|
|
87
|
+
finally:
|
|
88
|
+
cleanup(config.get("graceSeconds", 1))
|
|
89
|
+
selector.close()
|
|
90
|
+
print(json.dumps({"event": "settled", "status": "stopped" if stopped else "completed" if exit_code == 0 else "failed", "exitCode": exit_code}), flush=True)
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
if __name__ == "__main__":
|
|
94
|
+
main()
|