@serve.zone/pallet-bundle 35.3.0

package/ts_bundle/readme.md

@@ -0,0 +1,176 @@
+# @serve.zone/pallet-bundle
+
+The install surface of Pallet's sealed control bundle, released from Pallet's repository with
+Pallet's own version. An installer — Spark on fleet nodes, Onebox on its own host — uses it to
+
+- download a pinned Pallet release and verify it with tspack,
+- verify the extracted bundle: its control record, every executable's role and build provenance, the
+  component manifests and the notice and source inventories,
+- write the service units Pallet's "Service unit contract" states, and read them back from the
+  service manager,
+- run and supervise `pallet-control` modes over its `PALLET_CONTROL` process protocol.
+
+Pallet is the source of truth for all of it. Pallet's own packaging runs this package's verifier over
+every bundle it seals, so a release never disagrees with the version of this package that carries
+the same version number.
+
+## Issue Reporting and Security
+
+For reporting bugs, issues, or security vulnerabilities, please visit
+[community.foss.global/](https://community.foss.global/). This is the central community hub for all
+issue reporting. Developers who sign and comply with our contribution agreement and go through
+identification can also get a [code.foss.global/](https://code.foss.global/) account to submit Pull
+Requests directly.
+
+## Install
+
+```sh
+pnpm install @serve.zone/pallet-bundle
+```
+
+The package runs under Node.js and Deno; it uses `node:` APIs and the global `fetch` only.
+
+It reads its own version at runtime from the `package.json` beside its compiled modules
+(`version.ts`, resolved relative to `import.meta.url`). Consume it as an npm dependency and do not
+bundle it into another file: a bundler that inlines the modules loses that manifest, and
+`palletBundleVersion` and every call that takes a pin (the release client and the verifiers) then fail. `deno compile`
+keeps it: a compiled binary embeds each npm package it
+uses with its `package.json`, whether the dependency comes from a `node_modules` directory
+installed by pnpm or from Deno's own `nodeModulesDir: "auto"` install.
+
+## Pins
+
+A release is selected only by its pin, the values a consumer compiles in:
+
+```typescript
+import type { IPalletBundlePin } from '@serve.zone/pallet-bundle';
+
+const pin: IPalletBundlePin = {
+  version: '35.3.0',
+  sourceCommit: '<the 40-hex commit of the v35.3.0 tag>',
+  manifestSha256: '<SHA-256 of that release\'s tspack-manifest.json>',
+};
+```
+
+Every entry point refuses a pin whose `version` differs from this package's own version
+(`version_mismatch`): the inventory and contracts it states belong to that one Pallet version.
+Upgrade the package and the pin together.
+
+## Download, extract and verify
+
+```typescript
+import { PalletReleaseClient, extractPalletControlBundle } from '@serve.zone/pallet-bundle';
+
+const client = new PalletReleaseClient();
+const bundle = await client.run(pin, 'linux-amd64', '/opt/onebox/pallet', (download) =>
+  extractPalletControlBundle(download, 'linux-amd64', '/opt/onebox/pallet', pin, signal), signal);
+// bundle.directory is a fresh private directory below /opt/onebox/pallet; the caller owns it.
+console.log(bundle.executables.control.path, bundle.executables.control.sha256);
+console.log(bundle.containerdManifestSha256, bundle.provenance.store.projectVersion);
+```
+
+`PalletReleaseClient.run` resolves the release `v<version>` of `serve.zone/pallet` on
+`https://code.foss.global`, requires its tag to name the pinned commit, verifies the release metadata
+against the pinned manifest digest (tspack `verifyMetadata`), downloads the manifest, the checksums
+and the target's archive into a private directory below the given parent, verifies the archive
+(tspack `verifyBundle`) and hands the directory to the callback. It removes the download directory
+when the callback settles. A release that exists without all of its assets answers
+`release_pending` with the missing asset names.
+
+`extractPalletControlBundle` extracts the target's archive with tspack `extractBundle`, then runs
+`verifyPalletControlBundle` over it and removes the extraction again if that fails. For a bundle
+that is already extracted, call `verifyPalletControlBundle(directory, sealedFiles, pin, target)`
+with the file records of the pinned manifest.
+
+The verifier checks, against the sealed records of the bundle's files:
+
+- `control-build.json`: exactly its keys, Pallet's package name, the pinned version and commit, a
+  clean source, the target and the fixed host paths; every file it declares carries the sealed
+  digest and size, and every executable in the bundle is one it declares;
+- the executables `pallet-control` starts by name (`pallet-control`, `pallet-smartdb`,
+  `pallet-runtime`, `pallet-guard`, `pallet-dns`, `pallet-vpn`) and their tsrust provenance: the
+  sealed digest, the owning project and binary, the target, a clean build, and for `pallet-runtime`
+  the pinned version and commit;
+- the clock and containerd manifests, which list every file below their directories with digest,
+  size and mode, and the digests the control record pins for them and for the DNS and VPN notices;
+- the notice trees (`native-notices/`, `notices/smartnftables/`) and the clock and containerd notice
+  and corresponding-source inventories: each manifest names exactly the files below it.
+
+Refusals are `PalletBundleError` values with a fixed `code`; see `TPalletBundleErrorCode`.
+
+## Service units
+
+```typescript
+import { SystemdUnit, SystemdUnitFile } from '@push.rocks/smartdaemon';
+import {
+  createPalletGuardServiceDefinition, createPalletContainerdServiceDefinition,
+  createPalletNodeServiceDefinition, palletContainerdServiceSettingsMatch,
+} from '@serve.zone/pallet-bundle';
+
+const control = '/opt/onebox/pallet/current/pallet-control';
+const guard = createPalletGuardServiceDefinition({ unitName: 'onebox-pallet-guard.service',
+  executable: control, workingDirectory: '/opt/onebox/pallet' });
+const containerd = createPalletContainerdServiceDefinition({ unitName: 'onebox-pallet-containerd.service',
+  controlExecutable: control, workingDirectory: '/opt/onebox/pallet',
+  guardUnitName: guard.unitName, nodeUnitName: 'onebox-pallet-node.service' });
+const node = createPalletNodeServiceDefinition({ unitName: 'onebox-pallet-node.service',
+  controlExecutable: control, workingDirectory: '/opt/onebox/pallet',
+  guardUnitName: guard.unitName, containerdUnitName: containerd.unitName });
+```
+
+The definitions are `SystemdServiceDefinition` values of `@push.rocks/smartdaemon`; install them
+with its `SystemdUnitFile`. The unit names and the install root are the installer's.
+
+- The guard unit is a oneshot that runs `guard-recover` before `network-pre.target` and
+  `systemd-networkd.service`, which both require it. Install it only after the first
+  `guard-commission`. Its executable is `pallet-control` or an installer's own executable that
+  verifies the installation before it starts that mode.
+- The containerd unit runs `containerd-serve` to the contract in Pallet's readme ("Pallet's own
+  containerd"). `palletContainerdServiceSettingsMatch` reads the loaded unit's settings, drop-ins
+  included, back against that contract.
+- The node unit runs `runtime-unit`, the node lifetime as a unit's main process: null stdin, stopped
+  by SIGTERM, `KillMode=mixed`, restarted on failure by the service manager, after the guard and
+  containerd units. `palletNodeServiceSettingsMatch` reads it back.
+
+## Process supervisors
+
+```typescript
+import { PalletControlOneshot, PalletControlLifetime } from '@serve.zone/pallet-bundle';
+
+// A null-stdin oneshot: guard-recover, guard-commission, network-acquire, network-acquire-local, store-provision.
+await new PalletControlOneshot({ binaryPath: control, binarySha256: bundle.executables.control.sha256 })
+  .run('store-provision', signal);
+
+// A supervised lifetime held by a stdin pipe: enrollment-serve, enrollment-provision, runtime-serve.
+const lifetime = new PalletControlLifetime({ binaryPath: control, binarySha256, mode: 'enrollment-serve' });
+await lifetime.run(signal, async (running) => { /* talk to the enrollment socket */ });
+```
+
+Both verify the executable's digest before every start, start it with a clean environment and `/`
+as its working directory, read the `PALLET_CONTROL` lines of the mode's protocol and forward its
+standard error line by line, bounded (`PalletControlStderr`). They never restart a child. A oneshot
+succeeds only on `ready`, `stopped`, drained pipes and exit status 0. A lifetime runs the operation
+while the child is `ready`; closing its stdin is the graceful stop, and a missed shutdown deadline
+signals SIGTERM and then SIGKILL, which fails the lifetime as `shutdown_failed`. Refusals are
+`PalletControlProcessError` values; `unsupported_kernel` carries the operator's remedy.
+
+## License and Legal Information
+
+This repository contains open-source code licensed under the MIT License. A copy of the license is included in this package as `license.md`.
+
+**Please note:** The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.
+
+### Trademarks
+
+This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.
+
+Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.
+
+### Company Information
+
+Task Venture Capital GmbH<br>
+Registered at District Court Bremen HRB 35230 HB, Germany
+
+For any legal inquiries or further information, please contact us via email at hello@task.vc.
+
+By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.

package/ts_bundle/units.ts

@@ -0,0 +1,165 @@
+import * as plugins from './plugins.js';
+import { PalletBundleError } from './errors.js';
+
+/**
+ * The service units an installer writes for Pallet, to the "Service unit contract" in Pallet's readme.
+ * Pallet never writes a unit file itself: Spark on fleet nodes and Onebox on its own host install
+ * these definitions under their own unit names and install roots.
+ */
+
+/** The one network provider the guard's supported host profile orders itself before. */
+export const palletGuardNetworkProviderName = 'systemd-networkd.service';
+
+/** How long a stop of the node may take: a five-minute native operation, then the database joins. */
+export const palletNodeStopTimeoutSeconds = 360;
+
+export interface IPalletGuardUnitOptions {
+  unitName: string;
+  /**
+   * The boot command's executable. It runs `guard-recover`: `pallet-control` itself, or an
+   * installer's own executable that verifies the installed bundle before it starts that mode.
+   */
+  executable: string;
+  workingDirectory: string;
+}
+
+export interface IPalletContainerdUnitOptions {
+  unitName: string;
+  /** `<bundle>/pallet-control` of the selected installation. */
+  controlExecutable: string;
+  workingDirectory: string;
+  guardUnitName: string;
+  /** The node unit, the only CRI client, which this unit starts before. */
+  nodeUnitName: string;
+}
+
+export interface IPalletNodeUnitOptions {
+  unitName: string;
+  /** `<bundle>/pallet-control` of the selected installation. */
+  controlExecutable: string;
+  workingDirectory: string;
+  guardUnitName: string;
+  containerdUnitName: string;
+}
+
+const definition = (optionsArg: plugins.smartdaemon.ISystemdServiceDefinitionOptions) => {
+  try { return new plugins.smartdaemon.SystemdServiceDefinition(optionsArg); }
+  catch (error) { throw new PalletBundleError('invalid_options', { cause: error }); }
+};
+
+/**
+ * The retained allocation-pool guard's boot unit: a oneshot that recovers the commissioned guard
+ * before networkd, required by `network-pre.target` and by networkd itself. The installer installs it
+ * only after the first `guard-commission`, and never replaces an installed one with other bytes.
+ */
+export const createPalletGuardServiceDefinition = (optionsArg: IPalletGuardUnitOptions) => definition({
+  unitName: optionsArg.unitName,
+  description: 'serve.zone retained allocation-pool guard',
+  executable: optionsArg.executable,
+  args: ['guard-recover'],
+  workingDirectory: optionsArg.workingDirectory,
+  user: 'root',
+  group: 'root',
+  restart: 'no',
+  killMode: 'mixed',
+  timeoutStopSeconds: 360,
+  startup: { type: 'oneshot', remainAfterExit: true, timeoutStartSeconds: 600 },
+  dependencies: {
+    defaultDependencies: false,
+    after: [],
+    before: ['network-pre.target', palletGuardNetworkProviderName, 'shutdown.target'],
+    requires: [],
+    wants: ['network-pre.target'],
+    conflicts: ['shutdown.target'],
+  },
+  install: { wantedBy: [], requiredBy: ['network-pre.target', palletGuardNetworkProviderName] },
+});
+
+/**
+ * Pallet's containerd unit: `pallet-control containerd-serve` as the root main process with null stdin,
+ * ready on the `READY=1` its native owner (a child of the main process) sends. Stopping the unit
+ * signals only `containerd-serve`, which stops containerd itself; every shim and workload keeps
+ * running and is reattached by the next containerd. containerd's exit fails the process and the
+ * service manager restarts it — Pallet never restarts containerd inside the process.
+ */
+export const createPalletContainerdServiceDefinition = (optionsArg: IPalletContainerdUnitOptions) => definition({
+  unitName: optionsArg.unitName,
+  description: 'serve.zone Pallet containerd',
+  executable: optionsArg.controlExecutable,
+  args: ['containerd-serve'],
+  workingDirectory: optionsArg.workingDirectory,
+  user: 'root',
+  group: 'root',
+  restart: 'always',
+  restartSeconds: 2,
+  killMode: 'process',
+  // Pallet's 30 s containerd grace plus the owner's joins; the contract's floor is 45 s.
+  timeoutStopSeconds: 60,
+  // containerd-serve waits up to 60 s for CRI readiness, then imports the pause image.
+  startup: { type: 'notify', notifyAccess: 'all', timeoutStartSeconds: 120 },
+  delegate: true,
+  limitNoFile: 'infinity',
+  tasksMax: 'infinity',
+  oomScoreAdjust: -999,
+  dependencies: {
+    defaultDependencies: true,
+    after: [optionsArg.guardUnitName],
+    before: [optionsArg.nodeUnitName],
+    requires: [optionsArg.guardUnitName],
+    wants: [],
+    conflicts: [],
+  },
+});
+
+/**
+ * The node unit: `pallet-control runtime-unit` as the root main process with null stdin. It runs the
+ * same node lifetime as `runtime-serve`, but standard input is not its parent's lifetime: SIGTERM or
+ * SIGINT stops it. `KillMode=mixed` signals only the main process, which joins every owner and child
+ * it started, and the service manager removes whatever remains of the unit's cgroup when the stop
+ * timeout ends, before any successor starts. A failed lifetime is restarted by the service manager,
+ * never inside the process.
+ */
+export const createPalletNodeServiceDefinition = (optionsArg: IPalletNodeUnitOptions) => definition({
+  unitName: optionsArg.unitName,
+  description: 'serve.zone Pallet node',
+  executable: optionsArg.controlExecutable,
+  args: ['runtime-unit'],
+  workingDirectory: optionsArg.workingDirectory,
+  user: 'root',
+  group: 'root',
+  restart: 'on-failure',
+  restartSeconds: 5,
+  killMode: 'mixed',
+  timeoutStopSeconds: palletNodeStopTimeoutSeconds,
+  dependencies: {
+    defaultDependencies: true,
+    after: [optionsArg.guardUnitName, optionsArg.containerdUnitName],
+    before: [],
+    requires: [optionsArg.guardUnitName],
+    wants: [optionsArg.containerdUnitName],
+    conflicts: [],
+  },
+});
+
+/**
+ * Whether the service manager's own reading of the containerd unit, drop-ins included, is the
+ * contract. Admission and activation read the loaded unit rather than the file, so a drop-in that
+ * weakens the contract is refused whatever file produced it.
+ */
+export const palletContainerdServiceSettingsMatch = (settingsArg: plugins.smartdaemon.ISystemdServiceSettings,
+  unitNameArg: string): boolean =>
+  settingsArg.unitName === unitNameArg && settingsArg.type === 'notify' && settingsArg.notifyAccess === 'all' &&
+  settingsArg.restart === 'always' && typeof settingsArg.restartMicroseconds === 'number' &&
+  settingsArg.restartMicroseconds > 0 && settingsArg.restartMicroseconds <= 10_000_000 && settingsArg.killMode === 'process' &&
+  settingsArg.delegate && typeof settingsArg.timeoutStopMicroseconds === 'number' &&
+  settingsArg.timeoutStopMicroseconds >= 45_000_000 && settingsArg.limitNoFile === 'infinity' &&
+  settingsArg.limitNoFileSoft === 'infinity' && settingsArg.tasksMax === 'infinity' && settingsArg.oomScoreAdjust === -999;
+
+/** The same reading of the node unit: no restart inside a stop, and a stop long enough to join. */
+export const palletNodeServiceSettingsMatch = (settingsArg: plugins.smartdaemon.ISystemdServiceSettings,
+  unitNameArg: string): boolean =>
+  settingsArg.unitName === unitNameArg && settingsArg.type === 'exec' && settingsArg.killMode === 'mixed' &&
+  settingsArg.restart === 'on-failure' &&
+  typeof settingsArg.timeoutStopMicroseconds === 'number' &&
+  settingsArg.timeoutStopMicroseconds >= palletNodeStopTimeoutSeconds * 1_000_000 && !settingsArg.delegate &&
+  Object.keys(settingsArg.environment).length === 0;