@celilo/cli 0.25.0 → 0.26.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -27,7 +27,7 @@ Each entry: `module id` — what it is — **provides** / **requires** capabilit
27
27
  - **knot-unbound-internal** — split-horizon internal DNS via Knot (authoritative) + Unbound (recursive); lightweight, plain apt, no .NET. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/knot-unbound-internal/base-module-aspect/`).
28
28
  - **technitium** — internal split-horizon DNS resolver + authoritative server (web UI + HTTP API); heavier alternative to knot-unbound. **provides:** `dns_internal`. Ships a base-module-aspect (`modules/technitium/base-module-aspect/`).
29
29
  - **namecheap** — public DNS A-record management via Namecheap Dynamic DNS API (HTTP, no browser automation). A caller supplies a NAME and nothing else: the address is the source IP of celilo's own update, re-derived on every assert. Registering `<domain>` also claims `www.<domain>` and vice versa (best effort, reported back as `outputs.companion_fqdn` so the framework's `public_dns` check watches it — Namecheap answers `ErrCount 0` for `www` updates it does not apply). DDNS passwords are keyed by the **registrable domain**, never the FQDN. **provides:** `dns_registrar`.
30
- - **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. **REQUIRES the `control-plane-vpn` network and READS its range** (`requires.networks`; `client_subnet` is a `source: system` derive of `network.control-plane-vpn.subnet`). It does not write that network and has no way to — celilo owns the namespace, and the deploy will not reach any hook until the network is defined, asking for a range if one is missing. So `wg0` is attributable the moment it exists. This replaces a declare-before-you-create ordering inside `on_install`, which could only narrow the window and not close it: a consumer that captured config before the hook started could not see a value the hook wrote, whatever order it wrote it in, and that is exactly what left `wg0` unattributable (celilo#759). `health_check` still asserts the declaration matches what the tunnel serves — what it catches now is divergence, celilo's network having changed since the module resolved its config. That same key is what the internal resolver's split-horizon view consumes. Adopts a running tunnel in place (existing key and peers retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does; the ISP-router drivers `greenwave` and `axon` do not).
30
+ - **wireguard** — owns the admin WireGuard tunnel on the firewall host: interface, listen port, peers (as records), and client subnet are module config rather than hand-maintained state. Exposes the listen port and **registers the client subnet as a trusted source**, so VPN reach into the managed zones is in the firewall registry and every converge re-emits it. **REQUIRES the `control-plane-vpn` network and READS its range** (`requires.networks`; `client_subnet` is a `source: system` derive of `network.control-plane-vpn.subnet`). It does not write that network and has no way to — celilo owns the namespace, and the deploy will not reach any hook until the network is defined, asking for a range if one is missing. So `wg0` is attributable the moment it exists. This replaces a declare-before-you-create ordering inside `on_install`, which could only narrow the window and not close it: a consumer that captured config before the hook started could not see a value the hook wrote, whatever order it wrote it in, and that is exactly what left `wg0` unattributable (celilo#759). `health_check` still asserts the declaration matches what the tunnel serves — what it catches now is divergence, celilo's network having changed since the module resolved its config. That same key is what the internal resolver's split-horizon view consumes. Adopts a running tunnel in place (existing key retained; `wg syncconf`, never `wg-quick down`) because that tunnel is the operator's recovery path. **Adoption is a one-time IMPORT and it ENDS.** The marker is `registered_peers` — celilo's own config key, separate from the operator's `peers` so a machine can never rewrite what an operator typed. While that key has never been written the tunnel is not yet celilo's; the deploy that writes it (even as `[]`) CLAIMS the tunnel, importing whatever `[Peer]` blocks are running, and from then on the render is closed-world (`peers` ∪ `registered_peers`) with a `[Peer]` on the box in neither reported as drift by `on_install` and by `health_check`'s `unknown_peers`. An ABSENT `peers` declaration and a declared-EMPTY one are deliberately different (no `default: []` on either variable): they used to be the same value, so the module read every empty list as "adopt what is running", the last peer could not be revoked, and a hand-added peer rode along on every deploy unreported (celilo#765). On a tunnel celilo has not yet claimed, a declared-empty `peers` is REFUSED rather than obeyed — celilo's own variable-default seeding wrote a real `peers = []` row for every install of the previous version that never set one, so on the installed base an empty list cannot be told from a stored default, and obeying it would `wg syncconf` the admin tunnel down to zero peers. **requires:** `firewall` (and the provider must support trusted-source registration — `iptables` does; the ISP-router drivers `greenwave` and `axon` do not).
31
31
 
32
32
  ## Public edge (ingress / identity)
33
33
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@celilo/cli",
3
- "version": "0.25.0",
3
+ "version": "0.26.0",
4
4
  "description": "Celilo — home lab orchestration CLI",
5
5
  "type": "module",
6
6
  "bin": {
@@ -58,10 +58,10 @@
58
58
  "dependencies": {
59
59
  "@aws-sdk/client-s3": "^3.1109.0",
60
60
  "@aws-sdk/lib-storage": "^3.1101.0",
61
- "@celilo/capabilities": "^1.2.0",
61
+ "@celilo/capabilities": "^1.3.0",
62
62
  "@celilo/cli-display": "^0.2.0",
63
63
  "@celilo/core": "^0.8.0",
64
- "@celilo/event-bus": "^0.5.0",
64
+ "@celilo/event-bus": "^0.6.0",
65
65
  "ajv": "^8.18.0",
66
66
  "drizzle-orm": "^0.36.4",
67
67
  "ink": "^7.0.1",
@@ -75,10 +75,12 @@
75
75
  },
76
76
  "devDependencies": {
77
77
  "@biomejs/biome": "^1.9.4",
78
+ "@celilo/terraform-fake": "^0.3.0",
78
79
  "@types/bun": "^1.1.14",
79
80
  "@types/react": "^19.2.14",
80
81
  "drizzle-kit": "^0.30.0",
81
82
  "ink-testing-library": "^4.0.0",
83
+ "msw": "^2.15.0",
82
84
  "typescript": "^5.9.3",
83
85
  "zod-to-json-schema": "^3.25.2"
84
86
  }
@@ -0,0 +1,328 @@
1
+ /**
2
+ * The Proxmox client, exercised over a real socket — against the SAME handlers
3
+ * the e2e rig serves (D8).
4
+ *
5
+ * `proxmox.ts` hand-rolls `node:https` across 1,169 lines, and its existing
6
+ * 240-line test covers only the pure helpers: not one assertion reaches the
7
+ * wire. So the auth header, the `{data}` envelope, the 401 path and every
8
+ * non-2xx branch have gone unasserted — in the file that talks to the
9
+ * hypervisor.
10
+ *
11
+ * ## Why a real server rather than `setupServer`
12
+ *
13
+ * D8 proposed MSW's in-process `setupServer` for exactly this. **It does not
14
+ * work under Bun**, measured rather than assumed: `setupServer` intercepts
15
+ * global `fetch`, but a `node:https.request` goes straight past it and fails
16
+ * with `ECONNREFUSED` from Bun's own `node:_http_client`. `proxmox.ts` uses
17
+ * `node:https` directly, so the interceptor never sees it.
18
+ *
19
+ * `@celilo/terraform-fake` already exposes the same handlers as a real HTTPS
20
+ * server, so that is what these use. The D8 goal is met either way — ONE handler
21
+ * set behind both the client's unit tests and the rig's simulator, so the two
22
+ * cannot come to believe different things about what Proxmox returns — but it is
23
+ * met over a socket instead of an interceptor. The cost is a port and a
24
+ * self-signed certificate; the client already sets `rejectUnauthorized: false`,
25
+ * as every real consumer does.
26
+ */
27
+
28
+ import { afterAll, beforeAll, describe, expect, test } from 'bun:test';
29
+ import https from 'node:https';
30
+ import type { AddressInfo } from 'node:net';
31
+ import { type ProxmoxFake, createProxmoxFake } from '@celilo/terraform-fake';
32
+ import {
33
+ ProxmoxClient,
34
+ type ProxmoxCredentials,
35
+ buildProxmoxApiUrl,
36
+ getNodeForVmid,
37
+ listNodeStorage,
38
+ testProxmoxConnection,
39
+ } from './proxmox';
40
+
41
+ let fake: ProxmoxFake;
42
+ let credentials: ProxmoxCredentials;
43
+ let tls: { key: string; cert: string };
44
+
45
+ const TOKEN_ID = 'celilo@pve!e2e';
46
+ const TOKEN_SECRET = '00000000-0000-0000-0000-000000000000';
47
+
48
+ beforeAll(async () => {
49
+ tls = await generateSelfSigned();
50
+ fake = createProxmoxFake({
51
+ tls,
52
+ nodes: [{ name: 'pve1', cores: 8, memoryBytes: 16 * 1024 ** 3, diskBytes: 500 * 1024 ** 3 }],
53
+ storages: [{ name: 'local-lvm', content: 'images,rootdir' }],
54
+ });
55
+ const port = await fake.listen(0);
56
+ credentials = {
57
+ api_url: buildProxmoxApiUrl('127.0.0.1', port),
58
+ api_token_id: TOKEN_ID,
59
+ api_token_secret: TOKEN_SECRET,
60
+ };
61
+ });
62
+
63
+ afterAll(async () => {
64
+ await fake.close();
65
+ });
66
+
67
+ /**
68
+ * Run one request against a throwaway server that answers however the test
69
+ * needs, for the branches a well-behaved Proxmox never exercises.
70
+ */
71
+ async function withStubServer<T>(
72
+ reply: (res: import('node:http').ServerResponse) => void,
73
+ run: (credentials: ProxmoxCredentials) => Promise<T>,
74
+ ): Promise<T> {
75
+ const server = https.createServer(tls, (_req, res) => reply(res));
76
+ await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
77
+ const { port } = server.address() as AddressInfo;
78
+ try {
79
+ return await run({
80
+ api_url: buildProxmoxApiUrl('127.0.0.1', port),
81
+ api_token_id: TOKEN_ID,
82
+ api_token_secret: TOKEN_SECRET,
83
+ });
84
+ } finally {
85
+ await new Promise<void>((resolve) => server.close(() => resolve()));
86
+ }
87
+ }
88
+
89
+ describe('the request the client actually sends', () => {
90
+ test('carries the PVEAPIToken header Proxmox requires', async () => {
91
+ // Never asserted before. Proxmox rejects the session outright if this is
92
+ // malformed, and the client builds it by string concatenation.
93
+ let seen: string | undefined;
94
+ const server = https.createServer(tls, (req, res) => {
95
+ seen = req.headers.authorization;
96
+ res.writeHead(200, { 'content-type': 'application/json' });
97
+ res.end(JSON.stringify({ data: [] }));
98
+ });
99
+ await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
100
+ const { port } = server.address() as AddressInfo;
101
+
102
+ await new ProxmoxClient({
103
+ ...credentials,
104
+ api_url: buildProxmoxApiUrl('127.0.0.1', port),
105
+ }).clusterResources();
106
+ await new Promise<void>((resolve) => server.close(() => resolve()));
107
+
108
+ expect(seen).toBe(`PVEAPIToken=${TOKEN_ID}=${TOKEN_SECRET}`);
109
+ });
110
+
111
+ test('unwraps the `{data}` envelope rather than handing back the whole body', async () => {
112
+ const result = await listNodeStorage(credentials, 'pve1');
113
+
114
+ expect(result.success).toBe(true);
115
+ expect(Array.isArray(result.success && result.data)).toBe(true);
116
+ });
117
+
118
+ test('a query string is DROPPED — the constraint every caller works around', async () => {
119
+ // `makeProxmoxRequest` passes `url.pathname` and never `url.search`, so a
120
+ // filter silently does not arrive. `getNodeForVmid` documents this and
121
+ // fetches the whole inventory instead. Pinned here so that adding `?type=vm`
122
+ // somewhere and quietly getting unfiltered results is a test failure rather
123
+ // than a puzzling runtime one.
124
+ let seenUrl: string | undefined;
125
+ const server = https.createServer(tls, (req, res) => {
126
+ seenUrl = req.url;
127
+ res.writeHead(200, { 'content-type': 'application/json' });
128
+ res.end(JSON.stringify({ data: [] }));
129
+ });
130
+ await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
131
+ const { port } = server.address() as AddressInfo;
132
+
133
+ await getNodeForVmid({ ...credentials, api_url: buildProxmoxApiUrl('127.0.0.1', port) }, 1);
134
+ await new Promise<void>((resolve) => server.close(() => resolve()));
135
+
136
+ expect(seenUrl).toBe('/api2/json/cluster/resources');
137
+ expect(seenUrl).not.toContain('?');
138
+ });
139
+ });
140
+
141
+ describe('a POST — the other half of the wire contract', () => {
142
+ test('sends form-encoded parameters, not JSON', async () => {
143
+ // Proxmox only accepts `application/x-www-form-urlencoded`; sending JSON
144
+ // gets a 400 that names no field. Never asserted before.
145
+ //
146
+ // The stub answers BOTH paths, because `setGuestPower` first resolves the
147
+ // node via `/cluster/resources` and only then issues the POST.
148
+ let contentType: string | undefined;
149
+ let postedTo: string | undefined;
150
+ let body = '';
151
+ const server = https.createServer(tls, (req, res) => {
152
+ if (req.method === 'GET') {
153
+ res.writeHead(200, { 'content-type': 'application/json' });
154
+ res.end(JSON.stringify({ data: [{ vmid: 403, node: 'pve1', type: 'lxc' }] }));
155
+ return;
156
+ }
157
+ contentType = req.headers['content-type'];
158
+ postedTo = req.url;
159
+ req.on('data', (c) => {
160
+ body += c;
161
+ });
162
+ req.on('end', () => {
163
+ res.writeHead(200, { 'content-type': 'application/json' });
164
+ res.end(JSON.stringify({ data: 'UPID:pve1:00000001:0:66BF0000:vzshutdown:403:root@pam:' }));
165
+ });
166
+ });
167
+ await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
168
+ const { port } = server.address() as AddressInfo;
169
+
170
+ const result = await new ProxmoxClient({
171
+ ...credentials,
172
+ api_url: buildProxmoxApiUrl('127.0.0.1', port),
173
+ }).setGuestPower(403, 'lxc', 'shutdown');
174
+ await new Promise<void>((resolve) => server.close(() => resolve()));
175
+
176
+ expect(result.success).toBe(true);
177
+ expect(contentType).toBe('application/x-www-form-urlencoded');
178
+ expect(body).not.toStartWith('{');
179
+ // `shutdown`, not `stop`: a pause is planned, so the guest is asked to go
180
+ // quietly rather than having its power pulled.
181
+ expect(postedTo).toBe('/api2/json/nodes/pve1/lxc/403/status/shutdown');
182
+ }, 30_000);
183
+ });
184
+
185
+ describe('what the client does with a response it did not want', () => {
186
+ test('401 says the credentials are wrong, not merely that something failed', async () => {
187
+ const result = await withStubServer(
188
+ (res) => {
189
+ res.writeHead(401);
190
+ res.end();
191
+ },
192
+ (creds) => new ProxmoxClient(creds).clusterResources(),
193
+ );
194
+
195
+ expect(result.success).toBe(false);
196
+ expect(result.success === false && result.message).toContain('Authentication failed');
197
+ });
198
+
199
+ test('a 500 reports the status, so a caller can tell it from a refusal', async () => {
200
+ const result = await withStubServer(
201
+ (res) => {
202
+ res.writeHead(500);
203
+ res.end('boom');
204
+ },
205
+ (creds) => new ProxmoxClient(creds).clusterResources(),
206
+ );
207
+
208
+ expect(result.success).toBe(false);
209
+ expect(result.success === false && result.message).toContain('500');
210
+ });
211
+
212
+ test('a non-JSON body is a parse failure, not a crash', async () => {
213
+ // Proxmox behind a misconfigured proxy answers HTML. The client must still
214
+ // return a result, because every caller branches on `.success`.
215
+ const result = await withStubServer(
216
+ (res) => {
217
+ res.writeHead(200, { 'content-type': 'text/html' });
218
+ res.end('<html>nope</html>');
219
+ },
220
+ (creds) => new ProxmoxClient(creds).clusterResources(),
221
+ );
222
+
223
+ expect(result.success).toBe(false);
224
+ expect(result.success === false && result.message).toContain('parse');
225
+ });
226
+
227
+ test('an unreachable host resolves to a failure instead of rejecting', async () => {
228
+ // The whole function is a `new Promise(resolve)` with no reject path, so a
229
+ // throw here would surface as an unhandled rejection inside a CLI command.
230
+ const result = await new ProxmoxClient({
231
+ ...credentials,
232
+ // Reserved for documentation (RFC 5737) and never routable.
233
+ api_url: buildProxmoxApiUrl('192.0.2.1', 9),
234
+ }).clusterResources();
235
+
236
+ expect(result.success).toBe(false);
237
+ }, 30_000);
238
+ });
239
+
240
+ describe('against the real handler set', () => {
241
+ test('listNodeStorage returns the storages the fake declares', async () => {
242
+ const result = await listNodeStorage(credentials, 'pve1');
243
+ const names = result.success
244
+ ? (result.data as Array<{ storage: string }>).map((s) => s.storage)
245
+ : [];
246
+
247
+ expect(names).toContain('local-lvm');
248
+ });
249
+
250
+ test('listClusterResources gets node rows AND guest rows from one unfiltered call', async () => {
251
+ // Why the fake returns both when no `?type=` is given, even though the
252
+ // Terraform provider needs the filtered form: this client cannot ask for a
253
+ // filter (see the dropped-query test) and needs both kinds at once.
254
+ fake.state.addGuest({
255
+ vmid: 401,
256
+ node: 'pve1',
257
+ kind: 'lxc',
258
+ hostname: 'client-test',
259
+ status: 'running',
260
+ config: { cores: '2', memory: '1024' },
261
+ });
262
+
263
+ const result = await new ProxmoxClient(credentials).clusterResources();
264
+ const kinds = new Set(result.success ? result.data.map((r) => r.type) : []);
265
+
266
+ expect(kinds.has('node')).toBe(true);
267
+ expect(kinds.has('lxc')).toBe(true);
268
+
269
+ fake.state.removeGuest(401);
270
+ });
271
+
272
+ test('getNodeForVmid finds the node a guest lives on', async () => {
273
+ fake.state.addGuest({
274
+ vmid: 402,
275
+ node: 'pve1',
276
+ kind: 'lxc',
277
+ hostname: 'placed',
278
+ status: 'running',
279
+ config: {},
280
+ });
281
+
282
+ const result = await getNodeForVmid(credentials, 402);
283
+
284
+ expect(result.success && result.data).toBe('pve1');
285
+
286
+ fake.state.removeGuest(402);
287
+ });
288
+
289
+ test('getNodeForVmid returns null for a vmid that does not exist yet', async () => {
290
+ // First deploy: the container has not been created. Distinct from an error,
291
+ // and the callers rely on the difference.
292
+ const result = await getNodeForVmid(credentials, 9999);
293
+
294
+ expect(result.success).toBe(true);
295
+ expect(result.success && result.data).toBeNull();
296
+ });
297
+ });
298
+
299
+ describe('testProxmoxConnection', () => {
300
+ test('succeeds against a token with full permissions', async () => {
301
+ const result = await testProxmoxConnection(credentials);
302
+
303
+ expect(result.success).toBe(true);
304
+ });
305
+
306
+ test('a 401 on the probe fails the connection test', async () => {
307
+ const result = await withStubServer(
308
+ (res) => {
309
+ res.writeHead(401);
310
+ res.end();
311
+ },
312
+ (creds) => testProxmoxConnection(creds),
313
+ );
314
+
315
+ expect(result.success).toBe(false);
316
+ });
317
+ });
318
+
319
+ /** A throwaway pair. Validity beyond "parses as a cert" is not the point. */
320
+ async function generateSelfSigned(): Promise<{ key: string; cert: string }> {
321
+ const dir = `/tmp/proxmox-client-test-${process.pid}`;
322
+ await Bun.$`mkdir -p ${dir}`.quiet();
323
+ await Bun.$`openssl req -x509 -newkey rsa:2048 -keyout ${dir}/key.pem -out ${dir}/cert.pem -days 1 -nodes -subj /CN=proxmox.test`.quiet();
324
+ return {
325
+ key: await Bun.file(`${dir}/key.pem`).text(),
326
+ cert: await Bun.file(`${dir}/cert.pem`).text(),
327
+ };
328
+ }
@@ -54,7 +54,13 @@ describe('celilo#699 — stdout is undecorated', () => {
54
54
  // an empty roster would pass every assertion below without testing them.
55
55
  const imported = celilo('module import ../../modules/celilo-mgmt');
56
56
  expect(imported.status, `module import failed:\n${imported.stderr}`).toBe(0);
57
- });
57
+ // The spawn above is budgeted 60s, but bun's DEFAULT hook timeout is 5s, so
58
+ // the hook killed the import at 5001ms and the failure surfaced as
59
+ // `status: null` with an empty stderr — which reads as "module import is
60
+ // broken" rather than "this hook is not allowed to take as long as the work
61
+ // inside it". `module import` runs ansible-galaxy, measured at ~7s here; CI
62
+ // is under 5s, so the suite is green there and red on a slower machine.
63
+ }, 90_000);
58
64
 
59
65
  afterAll(async () => {
60
66
  await ctx.cleanup();
@@ -0,0 +1,63 @@
1
+ import { Database } from 'bun:sqlite';
2
+ import { afterEach, describe, expect, test } from 'bun:test';
3
+ import { rmSync } from 'node:fs';
4
+ import { tmpdir } from 'node:os';
5
+ import { join } from 'node:path';
6
+ import { createDbClient } from './client';
7
+
8
+ // #798: a command that opened the db while a deploy still held it died on
9
+ // SQLITE_BUSY immediately, because the connection carried no busy timeout.
10
+ describe('createDbClient concurrency', () => {
11
+ const paths: string[] = [];
12
+
13
+ const freshPath = () => {
14
+ const path = join(tmpdir(), `celilo-client-test-${Bun.nanoseconds()}.db`);
15
+ paths.push(path);
16
+ return path;
17
+ };
18
+
19
+ afterEach(() => {
20
+ for (const path of paths.splice(0)) {
21
+ for (const suffix of ['', '-wal', '-shm']) {
22
+ rmSync(`${path}${suffix}`, { force: true });
23
+ }
24
+ }
25
+ });
26
+
27
+ test('sets a non-zero busy timeout so a contended open waits instead of throwing', () => {
28
+ const { $client } = createDbClient({ path: freshPath() });
29
+
30
+ const timeout = $client.query<{ timeout: number }, []>('PRAGMA busy_timeout').get()?.timeout;
31
+
32
+ // Red before #798: bun:sqlite defaults to 0, i.e. fail on first contention.
33
+ expect(timeout).toBeGreaterThan(0);
34
+ });
35
+
36
+ test('opens in WAL mode', () => {
37
+ const { $client } = createDbClient({ path: freshPath() });
38
+
39
+ const mode = $client.query<{ journal_mode: string }, []>('PRAGMA journal_mode').get();
40
+
41
+ expect(mode?.journal_mode.toLowerCase()).toBe('wal');
42
+ });
43
+
44
+ test('a second open succeeds while another connection holds a write transaction', () => {
45
+ const path = freshPath();
46
+ createDbClient({ path });
47
+
48
+ // Hold the write lock the way a deploy in progress does.
49
+ const writer = new Database(path);
50
+ writer.run(`PRAGMA busy_timeout = ${0}`);
51
+ writer.run('BEGIN IMMEDIATE');
52
+
53
+ try {
54
+ // Reading is what `module show-config` does; it must not throw.
55
+ const { $client } = createDbClient({ path, readonly: true });
56
+ const mode = $client.query<{ journal_mode: string }, []>('PRAGMA journal_mode').get();
57
+ expect(mode?.journal_mode.toLowerCase()).toBe('wal');
58
+ } finally {
59
+ writer.run('ROLLBACK');
60
+ writer.close();
61
+ }
62
+ });
63
+ });
package/src/db/client.ts CHANGED
@@ -2,6 +2,7 @@ import { Database } from 'bun:sqlite';
2
2
  import { existsSync, mkdirSync } from 'node:fs';
3
3
  import { dirname, join } from 'node:path';
4
4
  import { fileURLToPath } from 'node:url';
5
+ import { BUSY_TIMEOUT_MS, ensureWalMode } from '@celilo/event-bus/wal';
5
6
  import { drizzle } from 'drizzle-orm/bun-sqlite';
6
7
  import { migrate } from 'drizzle-orm/bun-sqlite/migrator';
7
8
  import { getDbPath } from '../config/paths';
@@ -60,12 +61,18 @@ export function createDbClient(config?: Partial<DatabaseConfig>) {
60
61
  create: true,
61
62
  });
62
63
 
64
+ // Set FIRST so it covers every statement below, including the migrations. A
65
+ // command that opens the db while a deploy still holds it used to die on
66
+ // SQLITE_BUSY immediately instead of waiting (#798).
67
+ sqlite.run(`PRAGMA busy_timeout = ${BUSY_TIMEOUT_MS}`);
68
+
63
69
  // Enable foreign keys
64
70
  sqlite.run('PRAGMA foreign_keys = ON');
65
71
 
66
- // Enable WAL mode for better concurrency
72
+ // Enable WAL mode for better concurrency. busy_timeout does NOT cover this
73
+ // transition, so the switch needs its own retry — see ensureWalMode.
67
74
  if (!readonly) {
68
- sqlite.run('PRAGMA journal_mode = WAL');
75
+ ensureWalMode(sqlite);
69
76
  }
70
77
 
71
78
  const db = drizzle(sqlite, { schema });
@@ -109,6 +109,10 @@ export const CAPABILITY_MODULE_MAP: Record<string, { script: string; legacyFacto
109
109
  script: 'scripts/publish-functions.ts',
110
110
  legacyFactoryName: 'default',
111
111
  },
112
+ control_plane_vpn: {
113
+ script: 'scripts/control-plane-vpn-functions.ts',
114
+ legacyFactoryName: 'default',
115
+ },
112
116
  };
113
117
 
114
118
  /**
@@ -76,22 +76,19 @@ export function addCIDR(ip: string, maskBits: number): string {
76
76
  }
77
77
 
78
78
  /**
79
- * Check if an IP address belongs to a subnet
80
- * Example: isInSubnet("10.0.10.50/24", "10.0.10.0/24") → true
79
+ * Check if an IP address belongs to a subnet.
80
+ *
81
+ * Re-exported from `@celilo/capabilities` rather than implemented here, so the
82
+ * one answer serves both celilo and module scripts — which cannot import from
83
+ * the backend and would otherwise carry a second copy (celilo#809).
84
+ *
85
+ * The implementation that used to live here compared the first three octets and
86
+ * threw away the prefix it had parsed, so it was right for /24 and wrong in both
87
+ * directions for anything else. It also routed through `parseSubnet`, which
88
+ * REFUSES anything smaller than a /24 — an allocation rule that has no business
89
+ * constraining a containment question.
81
90
  */
82
- export function isInSubnet(ipWithMask: string, subnet: string): boolean {
83
- const ip = stripCIDR(ipWithMask);
84
- const subnetInfo = parseSubnet(subnet);
85
-
86
- const ipOctets = ip.split('.').map((o) => Number.parseInt(o, 10));
87
-
88
- // For /24 subnet, just check first 3 octets
89
- return (
90
- ipOctets[0] === subnetInfo.octets[0] &&
91
- ipOctets[1] === subnetInfo.octets[1] &&
92
- ipOctets[2] === subnetInfo.octets[2]
93
- );
94
- }
91
+ export { isInSubnet } from '@celilo/capabilities';
95
92
 
96
93
  /**
97
94
  * Generate all possible IPs in a subnet range
@@ -0,0 +1,123 @@
1
+ /**
2
+ * The DNS-ingress allocate-and-reserve guard (ISS-0156).
3
+ *
4
+ * This invariant had NO test. Losing it is not a crash: `module generate`
5
+ * re-allocates a different address on every run, silently moving the address
6
+ * internal clients use to reach DNS, while every command still reports success.
7
+ * `module generate` runs repeatedly over a module's life, so "on the second
8
+ * run" is the normal case, not an edge one.
9
+ */
10
+
11
+ import { beforeEach, describe, expect, test } from 'bun:test';
12
+ import type { DbClient } from '../db/client';
13
+ import type { ModuleManifest } from '../manifest/schema';
14
+ import { getModuleConfigValue } from '../services/module-config';
15
+ import { setupTestDatabase } from '../test-utils/database';
16
+ import { ensureDnsIngressIp } from './generator';
17
+
18
+ let db: DbClient;
19
+
20
+ /** A manifest that opts in the way a `dns_internal` provider does. */
21
+ const wantsIngress = {
22
+ variables: { owns: [{ name: 'dns_ingress_ip', source: 'infrastructure' }] },
23
+ } as unknown as ModuleManifest;
24
+
25
+ /** Same shape, but the variable is operator input rather than infrastructure. */
26
+ const operatorSupplied = {
27
+ variables: { owns: [{ name: 'dns_ingress_ip', source: 'user_input' }] },
28
+ } as unknown as ModuleManifest;
29
+
30
+ /**
31
+ * Typed as `string | undefined` rather than `unknown`: every assertion here is
32
+ * about an address, and an untyped read pushes a cast onto each one.
33
+ */
34
+ const storedIp = (moduleId: string): string | undefined => {
35
+ const value = getModuleConfigValue(moduleId, 'dns_ingress_ip', db)?.value;
36
+ return typeof value === 'string' ? value : undefined;
37
+ };
38
+
39
+ beforeEach(async () => {
40
+ db = await setupTestDatabase();
41
+ db.$client
42
+ .prepare('INSERT OR REPLACE INTO system_config (key, value) VALUES (?, ?)')
43
+ .run('network.internal.subnet', '10.226.1.0/24');
44
+ });
45
+
46
+ describe('ensureDnsIngressIp', () => {
47
+ test('allocates an address from the internal subnet on first generate', async () => {
48
+ const result = await ensureDnsIngressIp('technitium', wantsIngress, db);
49
+
50
+ expect(result.success).toBe(true);
51
+ expect(storedIp('technitium')).toMatch(/^10\.226\.1\.\d+$/);
52
+ });
53
+
54
+ test('REUSES the same address on a second generate', async () => {
55
+ // The guard itself. Re-allocating here moves the resolver's DNAT ingress
56
+ // every time the module is regenerated, and nothing reports a problem.
57
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
58
+ const first = storedIp('technitium');
59
+
60
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
61
+ const second = storedIp('technitium');
62
+
63
+ expect(second).toBe(first);
64
+ });
65
+
66
+ test('stays stable across many generates, not just two', async () => {
67
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
68
+ const first = storedIp('technitium');
69
+
70
+ for (let i = 0; i < 5; i++) {
71
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
72
+ }
73
+
74
+ expect(storedIp('technitium')).toBe(first);
75
+ });
76
+
77
+ test('RESERVES the address, so it is never handed out to something else', async () => {
78
+ // Allocation without reservation is the same bug one step later: a
79
+ // container gets the resolver's ingress address and DNS goes dark.
80
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
81
+ const ip = storedIp('technitium');
82
+
83
+ const reserved = db.$client
84
+ .prepare('SELECT ip_start, reason FROM ip_reservations WHERE ip_start = ?')
85
+ .get(ip ?? '') as { ip_start: string; reason: string } | undefined;
86
+
87
+ expect(reserved?.ip_start).toBe(ip);
88
+ // The reason names the owner, so an operator reading the table can tell
89
+ // what an otherwise anonymous held address is for.
90
+ expect(reserved?.reason).toBe('dns-ingress:technitium');
91
+ });
92
+
93
+ test('two modules get two different addresses', async () => {
94
+ await ensureDnsIngressIp('technitium', wantsIngress, db);
95
+ await ensureDnsIngressIp('knot-unbound-internal', wantsIngress, db);
96
+
97
+ expect(storedIp('knot-unbound-internal')).not.toBe(storedIp('technitium'));
98
+ });
99
+
100
+ test('does nothing for a module that never asked for one', async () => {
101
+ const result = await ensureDnsIngressIp('caddy', {} as ModuleManifest, db);
102
+
103
+ expect(result.success).toBe(true);
104
+ expect(storedIp('caddy')).toBeUndefined();
105
+ });
106
+
107
+ test('only `source: infrastructure` opts in', async () => {
108
+ // A same-named variable the operator supplies is theirs to set; allocating
109
+ // over it would overwrite an operator's deliberate choice.
110
+ await ensureDnsIngressIp('technitium', operatorSupplied, db);
111
+
112
+ expect(storedIp('technitium')).toBeUndefined();
113
+ });
114
+
115
+ test('fails with an actionable message when the internal subnet is unset', async () => {
116
+ db.$client.prepare('DELETE FROM system_config WHERE key = ?').run('network.internal.subnet');
117
+
118
+ const result = await ensureDnsIngressIp('technitium', wantsIngress, db);
119
+
120
+ expect(result.success).toBe(false);
121
+ expect(result.success === false && result.error).toContain('network.internal.subnet');
122
+ });
123
+ });
@@ -653,6 +653,70 @@ celilo module import modules/${moduleId}
653
653
  * @param options - Generation options
654
654
  * @returns Generation result
655
655
  */
656
+ /** Narrower than `GenerateResult`: this step produces no files, only an outcome. */
657
+ export type DnsIngressResult = { success: true } | { success: false; error: string };
658
+
659
+ /**
660
+ * Allocate-and-reserve the module's dedicated DNS-ingress IP, ONCE (ISS-0156).
661
+ *
662
+ * A `dns_internal` provider deploys into a PROTECTED zone (dmz) so it can see
663
+ * protected-zone query sources for split-horizon views. `internal` devices have
664
+ * no route into the 10-net, so they reach the resolver through a firewall DNAT
665
+ * on a dedicated `internal`-subnet address. A module opts in by declaring a
666
+ * `dns_ingress_ip` infrastructure variable.
667
+ *
668
+ * **Idempotence is the whole point, and it is load-bearing.** `module generate`
669
+ * runs repeatedly over a module's life. Re-allocating here on the second run
670
+ * would move the address internal clients use to reach DNS, every time — while
671
+ * every command still reports success. That is why the stored value is reused
672
+ * rather than re-derived, and why this is a named function instead of a branch
673
+ * buried in `generateTemplates`: an invariant nothing can call is an invariant
674
+ * nothing can test, and this one had no test at all.
675
+ */
676
+ export async function ensureDnsIngressIp(
677
+ moduleId: string,
678
+ manifest: ModuleManifest,
679
+ db: DbClient,
680
+ ): Promise<DnsIngressResult> {
681
+ const wantsDnsIngress = manifest.variables?.owns?.some(
682
+ (v) => v.name === 'dns_ingress_ip' && v.source === 'infrastructure',
683
+ );
684
+ if (!wantsDnsIngress) return { success: true };
685
+
686
+ const existing = getModuleConfigValue(moduleId, 'dns_ingress_ip', db)?.value;
687
+ if (typeof existing === 'string' && existing.length > 0) {
688
+ log.success(`Using existing DNS-ingress IP ${existing} for ${moduleId}`);
689
+ return { success: true };
690
+ }
691
+
692
+ const subnetRow = db.$client
693
+ .prepare('SELECT value FROM system_config WHERE key = ?')
694
+ .get('network.internal.subnet') as { value: string } | undefined;
695
+ if (!subnetRow?.value) {
696
+ return {
697
+ success: false,
698
+ error:
699
+ 'network.internal.subnet is not configured — required to allocate the ' +
700
+ 'dns_internal DNS-ingress IP (ISS-0156). Ensure the internal network is set up first.',
701
+ };
702
+ }
703
+
704
+ const { allocateIPFromSubnet, reserveIP } = await import('../ipam/allocator');
705
+ const { stripCIDR } = await import('../ipam/subnet-parser');
706
+ try {
707
+ const ip = stripCIDR(await allocateIPFromSubnet(subnetRow.value, 'internal', db));
708
+ await reserveIP(ip, 'internal', `dns-ingress:${moduleId}`, null, db);
709
+ upsertModuleConfig(db, moduleId, 'dns_ingress_ip', ip);
710
+ log.success(`Allocated DNS-ingress IP ${ip} (internal subnet) for ${moduleId}`);
711
+ return { success: true };
712
+ } catch (error) {
713
+ return {
714
+ success: false,
715
+ error: `DNS-ingress IP allocation failed: ${error instanceof Error ? error.message : String(error)}`,
716
+ };
717
+ }
718
+ }
719
+
656
720
  export async function generateTemplates(options: GenerateOptions): Promise<GenerateResult> {
657
721
  const { moduleId, modulePath, outputPath, db = getDb() } = options;
658
722
 
@@ -815,49 +879,8 @@ export async function generateTemplates(options: GenerateOptions): Promise<Gener
815
879
  }
816
880
  }
817
881
 
818
- // Dedicated DNS-ingress IP (ISS-0156). A dns_internal provider now deploys into
819
- // a PROTECTED zone (dmz) so it can see protected-zone query sources for
820
- // split-horizon views (openspec/specs/internal-dns-zone-views/spec.md). `internal` devices have
821
- // no route into the 10-net, so they reach the resolver through a firewall DNAT
822
- // on a dedicated `internal`-subnet address. A module opts in by declaring a
823
- // `dns_ingress_ip` infrastructure variable; we allocate a free IP from the
824
- // `internal` subnet via IPAM and RESERVE it (so it's never re-handed-out),
825
- // idempotently (reuse the stored value on re-generate). The resolver's
826
- // on_install passes it to firewall.exposeService({ ingressIp }).
827
- const wantsDnsIngress = manifest.variables?.owns?.some(
828
- (v) => v.name === 'dns_ingress_ip' && v.source === 'infrastructure',
829
- );
830
- if (wantsDnsIngress) {
831
- const existing = getModuleConfigValue(moduleId, 'dns_ingress_ip', db)?.value;
832
- if (typeof existing === 'string' && existing.length > 0) {
833
- log.success(`Using existing DNS-ingress IP ${existing} for ${moduleId}`);
834
- } else {
835
- const subnetRow = db.$client
836
- .prepare('SELECT value FROM system_config WHERE key = ?')
837
- .get('network.internal.subnet') as { value: string } | undefined;
838
- if (!subnetRow?.value) {
839
- return {
840
- success: false,
841
- error:
842
- 'network.internal.subnet is not configured — required to allocate the ' +
843
- 'dns_internal DNS-ingress IP (ISS-0156). Ensure the internal network is set up first.',
844
- };
845
- }
846
- const { allocateIPFromSubnet, reserveIP } = await import('../ipam/allocator');
847
- const { stripCIDR } = await import('../ipam/subnet-parser');
848
- try {
849
- const ip = stripCIDR(await allocateIPFromSubnet(subnetRow.value, 'internal', db));
850
- await reserveIP(ip, 'internal', `dns-ingress:${moduleId}`, null, db);
851
- upsertModuleConfig(db, moduleId, 'dns_ingress_ip', ip);
852
- log.success(`Allocated DNS-ingress IP ${ip} (internal subnet) for ${moduleId}`);
853
- } catch (error) {
854
- return {
855
- success: false,
856
- error: `DNS-ingress IP allocation failed: ${error instanceof Error ? error.message : String(error)}`,
857
- };
858
- }
859
- }
860
- }
882
+ const dnsIngress = await ensureDnsIngressIp(moduleId, manifest, db);
883
+ if (!dnsIngress.success) return dnsIngress;
861
884
 
862
885
  // Infrastructure Properties Resolution (Proxmox provider config)
863
886
  // For Proxmox services, extract provider config and store as temporary values