ever-works-node 0.0.0-stage → 0.2.2

package/package.json

@@ -1,6 +1,49 @@
 {
   "name": "ever-works-node",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.2.2",
+  "description": "Ever Works Node - headless execution node (CLI + long-running service): fleet enrollment, heartbeat and capability reporting",
+  "author": {
+    "name": "Ever Co. LTD",
+    "email": "ever@ever.co",
+    "url": "https://ever.co"
+  },
+  "license": "AGPL-3.0",
+  "homepage": "https://ever.works",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ever-works/ever-works.git",
+    "directory": "apps/node"
+  },
+  "bugs": {
+    "url": "https://github.com/ever-works/ever-works/issues"
+  },
+  "keywords": [
+    "ever-works",
+    "fleet",
+    "agent",
+    "node",
+    "runner",
+    "claude-code",
+    "codex"
+  ],
+  "bin": {
+    "ever-works-node": "cli.js"
+  },
+  "main": "./cli.js",
+  "files": [
+    "cli.js",
+    "packaging",
+    "README.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=22.0.0"
+  },
+  "optionalDependencies": {
+    "@napi-rs/keyring": "^1.1.0",
+    "@homebridge/node-pty-prebuilt-multiarch": "^0.13.1"
+  },
+  "publishConfig": {
+    "access": "public"
+  }
+}

package/packaging/README.md

@@ -0,0 +1,295 @@
+# Running the Ever Works node unattended
+
+`ever-works-node start` is a foreground process. Everything in this
+directory exists so it does not have to be — so an enrolled machine
+survives a reboot, restarts after a crash, and can be drained without
+throwing away work it is half-way through.
+
+Three shapes, one binary:
+
+| Host            | Mechanism                                        | Where                                     |
+| --------------- | ------------------------------------------------ | ----------------------------------------- |
+| Linux (systemd) | template unit `ever-works-node@<user>.service`   | `systemd/`                                |
+| Windows         | Windows service (NSSM) or Scheduled Task at boot | `windows/`                                |
+| Container / k8s | image built from the node's Dockerfile           | `../../../.deploy/docker/node/Dockerfile` |
+
+Enrollment is **never** part of installation. It consumes a one-time
+token from the platform's Fleet settings page and is an explicit,
+interactive act:
+
+```bash
+ever-works-node enroll --api-url https://api.ever.works --token <token>
+```
+
+Install the service afterwards.
+
+The scripts below live in this directory of the source tree **and** inside
+the published npm package: after `npm install -g ever-works-node` they are
+under `$(npm root -g)/ever-works-node/packaging/`.
+
+---
+
+## Linux — systemd
+
+```bash
+sudo apps/node/packaging/systemd/install.sh alice
+sudo -u alice ever-works-node enroll --api-url https://api.ever.works --token <token>
+sudo systemctl enable --now ever-works-node@alice
+journalctl -fu ever-works-node@alice
+```
+
+The unit is a **template** whose instance name is the user the node runs
+as. A fleet node executes that user's commands, so it runs as that user
+rather than as root or a shared service account.
+
+`TimeoutStopSec=900` is load-bearing: stopping the node stops the
+heartbeat and then **waits** for in-flight jobs to report their verdicts.
+If systemd `SIGKILL`s the drain half-way, those claims lapse and the
+platform re-runs the same work on another machine. Raise it above your
+longest expected job.
+
+`systemctl reload` is wired to `ever-works-node pause` — a drain, not a
+restart.
+
+Logs go to the journal (`StandardOutput=journal`), which journald already
+rotates and bounds (`SystemMaxUse=` in `journald.conf`), so the unit needs
+nothing of its own.
+
+## Windows
+
+Enroll first, **as the account the node will run as**, then install from an
+**elevated** PowerShell session:
+
+```powershell
+# 1. as the fleet user (NOT elevated) - this consumes a one-time token
+ever-works-node enroll --api-url https://api.ever.works --token <token>
+
+# 2. as an administrator: rehearse, then install
+$password = Read-Host -AsSecureString 'Password for the node account'
+.\apps\node\packaging\windows\install-service.ps1 -Work -ServicePassword $password -DryRun
+.\apps\node\packaging\windows\install-service.ps1 -Work -ServicePassword $password
+```
+
+### Who the node runs as
+
+This is the load-bearing decision, and it is why `-ServicePassword` is not
+optional. A fleet node executes the enrolling user's work with that user's
+credentials:
+
+- its config and heartbeat credential live in that user's
+  `%APPDATA%\ever-works-node`, written with inheritance stripped and exactly
+  one ACE for that user;
+- its git credentials live in that user's Windows Credential Manager vault,
+  which the credential helper reads per-user;
+- the Claude Code and Codex logins live in that user's profile.
+
+`nssm install` calls `CreateService` with a NULL `lpServiceStartName`, which
+the Win32 API defines as **LocalSystem** — an account that shares none of
+those. A node installed that way starts, finds no config, reports "not
+enrolled", and is restarted every 10 seconds forever.
+
+So `-ServiceAccount` defaults to the account running the installer and is
+applied on **every** run, including a re-run to change a pin. It accepts
+`DOMAIN\User`, `MACHINE\User`, `.\User`, a bare local name, a UPN, and a gMSA
+(`DOMAIN\name$`, which takes no password). The built-in service accounts are
+refused, with the reason.
+
+A gMSA is accepted but is an odd fit, and the preflight will say so: it is
+provisioned rather than signed in to, so it has no profile, no `%APPDATA%`,
+and nowhere for `ever-works-node enroll` to have put a credential. A fleet
+node's whole job is to run the **enrolling user's** work with that user's git
+and model-CLI logins.
+
+### How the password is handled
+
+`-ServicePassword` is a **SecureString**, so the plaintext never reaches your
+shell history or `powershell.exe`'s own command line — passing a literal is a
+parameter-binding error, not a history entry. It is never echoed, never
+written to a log file, and never rendered into a progress line or an
+exception.
+
+On the **service** path it never reaches a command line either. The obvious
+implementation, `nssm set <name> ObjectName <account> <password>`, puts it on
+a child process's command line, and that is written to persistent telemetry:
+Security event 4688 wherever command-line auditing is enabled (the Microsoft
+and CIS baselines enable it), and any EDR agent that captures
+`ProcessCommandLine` — Defender for Endpoint uploads that to its cloud and
+keeps it queryable. So the installer unwraps the SecureString into unmanaged
+memory and hands `ChangeServiceConfigW` a pointer, zeroing the buffer
+afterwards. No child process, no cmdlet parameter, and no managed `String`
+(which is immutable and could not be zeroed).
+
+That means the installer also has to do the half of the job NSSM was quietly
+doing for us: `ChangeServiceConfig` does **not** grant the "Log on as a
+service" right, and without it a correctly configured service fails to start
+with error 1069. The installer grants it with `LsaAddAccountRights`.
+
+On the **scheduled-task** path there is no such route: `Register-ScheduledTask`
+takes the password as an ordinary cmdlet parameter and the Task Scheduler API
+takes a `BSTR`, so a managed copy exists for the duration of the call and
+PowerShell module logging (event 4103) would record the bound value where that
+policy is enabled. One more reason the service is the recommended mechanism.
+
+### Preflight
+
+The installer refuses, before registering anything, when the chosen account
+could not actually run a node: no such account, no profile, no config at that
+account's `%APPDATA%` path, a config whose ACL does not grant it, a
+`-ClaudePath`/`-CodexPath` pin that does not resolve or is not launchable, or
+a workspace root it cannot write (or that would land inside the system
+profile). It warns — but proceeds — for the survivable cases, including a
+pinned CLI with no login: the node would advertise `claude-code` and then fail
+every job it was sent, which is worse than not advertising it at all.
+
+It also warns when the account cannot write `%ProgramData%\ever-works-node` or
+a log file already there, and then **fixes** it: files created in that folder
+by a privileged identity inherit only `BUILTIN\Users: ReadAndExecute`, so a
+`node.log` written by a LocalSystem-era install is read-only to the account
+the fixed installer picks. Left alone, the service could not open its
+`AppStdout` and the scheduled task would die inside `cmd.exe` before the node
+ran — with nothing in either log. The installer grants the account `Modify` on
+the directory and on any log left behind.
+
+### If a real run fails part-way
+
+A fresh install that fails after `nssm install` — most likely at the identity
+step, which is where a wrong account or password surfaces — is **undone**. The
+service is removed rather than left registered under the LocalSystem default
+this whole mechanism exists to avoid. A re-run over a service that already
+existed is not removed; the message says so, and names `uninstall-service.ps1`.
+
+### Dry run
+
+`-DryRun` (or the standard `-WhatIf`) runs the whole preflight and prints
+every command that would run, changing nothing. It does not require elevation,
+so it is the way to check five remote machines before touching any of them.
+
+### Other flags
+
+`-ClaudePath` / `-CodexPath` pin the model CLIs for the service (a service
+inherits the **machine** `PATH`, which normally lacks the per-user npm
+directory) and `-WorkspaceRoot` sets the absolute, drive- or UNC-rooted
+directory the per-Task worktrees live under; each becomes the matching `start`
+flag. `-CliPath` points the service at a specific `cli.js` (default: the
+globally installed package; from a source checkout use `apps\node\dist\cli.js`).
+
+### Mechanism
+
+The script registers `node.exe` running that `cli.js` directly. npm's global
+install also puts `ever-works-node.ps1` / `.cmd` shims on `PATH`, but a service
+manager cannot launch the `.ps1`, and the `.cmd` puts `cmd.exe` between the
+service and the node, where the console stop signal used for draining gets
+swallowed. The command line is quoted once, by the script, and handed to NSSM
+verbatim — Windows PowerShell 5.1 would otherwise re-quote pinned paths
+containing spaces without escaping them.
+
+The script prefers a real Windows service via [NSSM](https://nssm.cc).
+This matters: Node.js cannot answer Service Control Manager messages, so
+`sc.exe create` pointed straight at `node.exe` yields a service Windows
+reports as "did not respond in a timely fashion" (error 1053). A wrapper
+is what makes it a genuine service _and_ what gives the node a graceful
+stop long enough to drain.
+
+Without NSSM the script registers a **Scheduled Task** at boot and says so.
+That runs unattended and survives reboots, but it will not appear in
+`services.msc`. Both branches write `%ProgramData%\ever-works-node\node.log`
+and `node.err.log`; the task branch reaches them through `cmd.exe`, which is
+acceptable there only because `Stop-ScheduledTask` terminates the task
+outright and delivers no console event for `cmd.exe` to swallow — that branch
+never drained.
+
+Both logs **rotate past 10 MB** (one policy, `Get-NodeLogRotationPolicy`). The
+service rotates them with NSSM's own `AppRotateFiles` / `AppRotateOnline` /
+`AppRotateBytes` — at start and online while it runs. NSSM never deletes a
+rotated file, so the service install also registers `<Name>-LogPrune`, a
+small `SYSTEM` scheduled task that runs daily at 04:30 and at boot and keeps
+the newest five rotated generations of each log (NSSM's `node-<timestamp>.log`
+names and this installer's own); `uninstall-service.ps1` removes it with the
+service. The task cannot rotate while the node runs (`cmd.exe` holds
+the file open without delete sharing, and truncating under its write offset
+would leave a hole), so it rotates when the task STARTS — at boot and on every
+restart-on-failure — as a first action that runs to completion before the
+node action, renaming a full log to `<name>.<yyyyMMddTHHmmss>` and keeping
+the newest five. Re-run the installer to add rotation to a machine
+installed before it existed.
+
+Without `-ServicePassword` the task falls back to an **S4U** logon, which is
+minted without the account's password and therefore never unlocks its DPAPI
+master key: Windows Credential Manager reads fail there exactly as they do
+under LocalSystem. The preflight says so. (A gMSA is not S4U either — its
+password is held by the domain controller, so it registers as a `Password`
+logon with no value.)
+
+Remove with `uninstall-service.ps1`. That leaves the node enrolled — use
+`ever-works-node unenroll` to retire it on the platform too.
+
+### Tests
+
+`test-install-service-logic.ps1` covers the decision logic (account
+normalisation, the service configuration plan, the scheduled-task plan, the
+ACL plan, the access-verdict and pin rules, the preflight verdicts) without
+registering anything. It also exercises the `ChangeServiceConfigW` and
+`LsaAddAccountRights` signatures against a deliberately invalid handle, which
+proves the marshalling without touching a service or the local security
+policy. `.github/workflows/node-packaging-windows.yml` runs it on a
+`windows-2022` runner under Windows PowerShell 5.1 — the shell the fleet
+workstations actually have — and exercises the installer's `-DryRun` and
+`-WhatIf` paths. `apps/node/src/core/packaging/windows-service-contract.internal.spec.ts`
+pins the same invariants from the Linux CI, so they hold even if that
+workflow's path filter is edited.
+
+## Container
+
+```bash
+docker build -f .deploy/docker/node/Dockerfile -t ever-works-node .
+docker run -d --name ever-works-node \
+  -v ever-works-node-config:/var/lib/ever-works-node \
+  ever-works-node start --work
+```
+
+Enroll into the same volume first:
+
+```bash
+docker run --rm -it \
+  -v ever-works-node-config:/var/lib/ever-works-node \
+  ever-works-node enroll --api-url https://api.ever.works --token <token>
+```
+
+Build with `--build-arg WITH_BROWSER=true` to install Chromium, so the
+image can honour the `browser` capability it would otherwise not
+advertise. It roughly triples the image size, which is why it is opt-in.
+
+`tini` is PID 1 on purpose: PID 1 does not get default signal handling,
+and without it the node's SIGTERM drain would never run.
+
+---
+
+## Credential storage
+
+The heartbeat secret goes into the OS keychain (macOS Keychain, Windows
+Credential Manager, Linux Secret Service) when one is available, and the
+config file holds only a pointer. Where no keychain exists — headless
+servers, containers — it falls back into the config file and **says so**,
+loudly, on every load. The file is then restricted to its owner: `0600`
+on POSIX, and an inheritance-stripped owner-only ACL on Windows.
+
+Two environment variables change this:
+
+| Variable                             | Effect                                                       |
+| ------------------------------------ | ------------------------------------------------------------ |
+| `EVER_WORKS_NODE_DISABLE_KEYCHAIN=1` | Force the file fallback (set in the container image).        |
+| `EVER_WORKS_NODE_CONFIG`             | Absolute path of the config file, overriding the OS default. |
+
+## Draining
+
+`pause` is a drain, not a kill:
+
+```bash
+ever-works-node pause     # stop leasing; in-flight jobs finish and report
+ever-works-node resume    # take work again
+```
+
+It tells the platform (so the scheduler stops offering work) _and_
+records the intent locally (so a restart comes back paused). A paused
+node keeps heartbeating: a drained machine that vanishes from Fleet is
+indistinguishable from a dead one.

package/packaging/systemd/ever-works-node@.service

@@ -0,0 +1,64 @@
+# Template unit: the instance name IS the user the node runs as.
+#   systemctl enable --now ever-works-node@alice
+# A fleet node executes the OWNER'S commands, so it runs as that owner
+# rather than as root or as a shared service account.
+[Unit]
+Description=Ever Works execution node (running as %i)
+Documentation=https://docs.ever.works
+# Enrollment and heartbeat are outbound HTTPS; starting before the
+# network is up just burns the first few backoff intervals.
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=%i
+Group=%i
+
+# `--work` is what makes this machine capacity rather than inventory.
+# Drop it to enroll a node for visibility only.
+ExecStart=/usr/bin/env ever-works-node start --work
+# `pause` drains: leasing stops immediately, in-flight jobs finish and
+# report. Wired to ExecReload so `systemctl reload` is a drain, not a cut.
+ExecReload=/usr/bin/env ever-works-node pause
+
+# The node's own SIGTERM handler stops the heartbeat and then WAITS for
+# in-flight jobs to report their verdicts. TimeoutStopSec must therefore
+# be longer than the longest job you expect, or systemd SIGKILLs the
+# drain half-way through and the platform re-runs the work elsewhere.
+KillSignal=SIGTERM
+TimeoutStopSec=900
+
+Restart=on-failure
+RestartSec=10s
+# A node that crash-loops on a bad config should stop and be visible,
+# not hammer the API forever.
+StartLimitIntervalSec=300
+StartLimitBurst=5
+
+# Hardening. The node executes the OWNER'S commands by design, so this is
+# a blast-radius limit, not a sandbox: it keeps a check from touching the
+# rest of the system, it does not make an untrusted check safe.
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=full
+ProtectHome=read-only
+ProtectKernelTunables=true
+ProtectControlGroups=true
+RestrictSUIDSGID=true
+# Config (credential) and workspaces are the only writable paths.
+StateDirectory=ever-works-node
+ReadWritePaths=/var/lib/ever-works-node
+
+Environment=NODE_ENV=production
+Environment=EVER_WORKS_NODE_CONFIG=/var/lib/ever-works-node/node-config.json
+# Drop-in overrides go in
+# /etc/systemd/system/ever-works-node@.service.d/override.conf
+EnvironmentFile=-/etc/default/ever-works-node
+
+StandardOutput=journal
+StandardError=journal
+SyslogIdentifier=ever-works-node
+
+[Install]
+WantedBy=multi-user.target

package/packaging/systemd/install.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+#
+# Install the Ever Works node as a systemd service.
+#
+#   sudo ./install.sh <user>
+#
+# Idempotent: re-running refreshes the unit and reloads systemd. It does
+# NOT enroll the node — enrollment consumes a one-time token and is an
+# explicit, interactive act:
+#
+#   ever-works-node enroll --api-url https://api.ever.works --token <token>
+#
+# Run that as <user> first, then start the service.
+
+set -euo pipefail
+
+UNIT_NAME="ever-works-node@.service"
+UNIT_SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/${UNIT_NAME}"
+UNIT_DEST="/etc/systemd/system/${UNIT_NAME}"
+STATE_DIR="/var/lib/ever-works-node"
+
+RUN_AS="${1:-}"
+if [ -z "${RUN_AS}" ]; then
+    echo "usage: $0 <user>" >&2
+    echo "  the node runs as that user and executes ITS commands" >&2
+    exit 2
+fi
+
+if [ "$(id -u)" -ne 0 ]; then
+    echo "error: must run as root (installing a system unit)" >&2
+    exit 1
+fi
+
+if ! id "${RUN_AS}" >/dev/null 2>&1; then
+    echo "error: user '${RUN_AS}' does not exist" >&2
+    exit 1
+fi
+
+if ! command -v systemctl >/dev/null 2>&1; then
+    echo "error: systemd is not available on this host" >&2
+    echo "  use the container image (.deploy/docker/node/Dockerfile) instead" >&2
+    exit 1
+fi
+
+if ! command -v ever-works-node >/dev/null 2>&1; then
+    # Refuse rather than install a unit that will crash-loop on start.
+    echo "error: 'ever-works-node' is not on PATH" >&2
+    echo "  install it first, e.g.: npm install -g ever-works-node" >&2
+    exit 1
+fi
+
+install -d -m 0700 -o "${RUN_AS}" -g "${RUN_AS}" "${STATE_DIR}"
+install -m 0644 "${UNIT_SRC}" "${UNIT_DEST}"
+
+systemctl daemon-reload
+
+echo "Installed ${UNIT_DEST}"
+echo
+echo "Next:"
+echo "  1. enroll (as ${RUN_AS}):"
+echo "     sudo -u ${RUN_AS} ever-works-node enroll --api-url <url> --token <token>"
+echo "  2. start:"
+echo "     systemctl enable --now ever-works-node@${RUN_AS}"
+echo
+echo "Operating:"
+echo "  drain (finish in-flight work, take no more):  ever-works-node pause"
+echo "  take work again:                              ever-works-node resume"
+echo "  follow logs:                                  journalctl -fu ever-works-node@${RUN_AS}"