hugpy-ops 0.2.2__tar.gz

hugpy_ops-0.2.2/LICENSE

@@ -0,0 +1,41 @@
+hugpy — Source-Available License
+
+Copyright (c) 2026 putkoff (hugpy.ai). All rights reserved.
+
+Permission is granted, free of charge, to use this software ("hugpy") for
+personal and non-commercial purposes, and for time-limited commercial
+evaluation, subject to the following conditions:
+
+1. Non-commercial use means use by an individual for personal purposes, or
+   use by a non-profit or educational institution for its own internal
+   purposes. Any use by, for, or on behalf of a for-profit business or in
+   connection with revenue-generating activity is commercial use — including
+   internal business use, use in producing goods or services, and use on
+   paid engagements.
+
+2. Commercial use requires a commercial license from the copyright holder.
+   Exception: a business may evaluate the software internally for up to
+   thirty (30) days free of charge; continued use after that requires a
+   commercial license.
+
+3. Redistribution of this software, in source or binary form, modified or
+   unmodified, is not permitted without prior written permission from the
+   copyright holder. Downloading the software from an official distribution
+   channel (PyPI, npm, hugpy.ai) is not redistribution.
+
+4. Modification for personal use or internal evaluation is permitted;
+   distribution of modified versions is not.
+
+5. This notice must be retained in all copies or substantial portions of
+   the software.
+
+6. Any use outside these terms automatically terminates this license.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.
+
+For commercial licensing or redistribution permission: https://hugpy.ai

hugpy_ops-0.2.2/PKG-INFO

@@ -0,0 +1,63 @@
+Metadata-Version: 2.4
+Name: hugpy-ops
+Version: 0.2.2
+Summary: Hugpy operations: sentinel, chaos runner, keeper REPL, todo keeper and provisioner for declared-but-missing model weights
+Author-email: putkoff <support@hugpy.ai>
+License-Expression: LicenseRef-Proprietary
+Project-URL: Homepage, https://hugpy.ai
+Project-URL: Documentation, https://github.com/hugpy/hugpy/blob/main/py/operations/hugpy_ops/README.md
+Project-URL: Repository, https://github.com/hugpy/hugpy
+Project-URL: Source, https://github.com/hugpy/hugpy/tree/main/py/operations/hugpy_ops
+Project-URL: Issues, https://github.com/hugpy/hugpy/issues
+Project-URL: Changelog, https://github.com/hugpy/hugpy/releases
+Project-URL: Architecture, https://github.com/hugpy/hugpy/blob/main/PARTITION.md
+Keywords: hugpy,llm,self-hosted,operations,sentinel,chaos-testing,provisioning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Environment :: Console
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: hugpy-platform
+Requires-Dist: hugpy-control
+Requires-Dist: hugpy-storage
+Requires-Dist: hugpy-engine
+Requires-Dist: hugpy-video
+Requires-Dist: hugpy-fleet
+Requires-Dist: hugpy-curation
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-timeout; extra == "test"
+Dynamic: license-file
+
+# hugpy-ops
+
+`hugpy_ops` — the operational consumers of the Hugpy public APIs, extracted
+from `abstract_hugpy_dev` as part of the partition (`py/partition.toml`,
+`PARTITION.md`). Nothing in this package is imported by normal server
+startup; every tool is its own console script:
+
+| Script | Module | What it does |
+|---|---|---|
+| `hugpy-sentinel` | `hugpy_ops.sentinel` | one detect -> case -> diagnose pass over central's HTTP read surfaces (`run-once`, `status`, `record-scorecard`) |
+| `hugpy-chaos` | `hugpy_ops.chaos` | chaos-and-learn exerciser (`hugpy-chaos`) and the k7 offload speed-cliff sweep (`hugpy-chaos sweep`) |
+| `hugpy-keeper` | `hugpy_ops.keeper` | stationary terminal REPL in which a served model keeps a machine or an LXD instance |
+| `hugpy-todo-keeper` | `hugpy_ops.todo_keeper_daemon` | the enrolling todo-keeper agent node (`todo.add` / `todo.tidy`) |
+| `hugpy-provisioner` | `hugpy_ops.provisioner` | declared-but-missing weights across the engine, studio and comfy registries -> `hugpy_storage` download queue |
+
+Allowed ecosystem dependencies: `hugpy_platform`, `hugpy_control`,
+`hugpy_storage`, `hugpy_engine`, `hugpy_video`, `hugpy_fleet`,
+`hugpy_curation`. The central URL and every state root come from
+`hugpy_platform` (`central_base_url`, `hugpy_state_dir`, `config_dir`,
+`models_root`).
+
+Deploy units and the operator notes live in `deploy/`; mechanics in `docs/`.
+
+```sh
+cd py/operations/hugpy_ops && ../../../.venv/bin/python -m pytest -q --timeout=120
+```

hugpy_ops-0.2.2/README.md

@@ -0,0 +1,26 @@
+# hugpy-ops
+
+`hugpy_ops` — the operational consumers of the Hugpy public APIs, extracted
+from `abstract_hugpy_dev` as part of the partition (`py/partition.toml`,
+`PARTITION.md`). Nothing in this package is imported by normal server
+startup; every tool is its own console script:
+
+| Script | Module | What it does |
+|---|---|---|
+| `hugpy-sentinel` | `hugpy_ops.sentinel` | one detect -> case -> diagnose pass over central's HTTP read surfaces (`run-once`, `status`, `record-scorecard`) |
+| `hugpy-chaos` | `hugpy_ops.chaos` | chaos-and-learn exerciser (`hugpy-chaos`) and the k7 offload speed-cliff sweep (`hugpy-chaos sweep`) |
+| `hugpy-keeper` | `hugpy_ops.keeper` | stationary terminal REPL in which a served model keeps a machine or an LXD instance |
+| `hugpy-todo-keeper` | `hugpy_ops.todo_keeper_daemon` | the enrolling todo-keeper agent node (`todo.add` / `todo.tidy`) |
+| `hugpy-provisioner` | `hugpy_ops.provisioner` | declared-but-missing weights across the engine, studio and comfy registries -> `hugpy_storage` download queue |
+
+Allowed ecosystem dependencies: `hugpy_platform`, `hugpy_control`,
+`hugpy_storage`, `hugpy_engine`, `hugpy_video`, `hugpy_fleet`,
+`hugpy_curation`. The central URL and every state root come from
+`hugpy_platform` (`central_base_url`, `hugpy_state_dir`, `config_dir`,
+`models_root`).
+
+Deploy units and the operator notes live in `deploy/`; mechanics in `docs/`.
+
+```sh
+cd py/operations/hugpy_ops && ../../../.venv/bin/python -m pytest -q --timeout=120
+```

hugpy_ops-0.2.2/deploy/SENTINEL.md

@@ -0,0 +1,142 @@
+# hugpy-sentinel — bound-exceeded watcher (k95)
+
+Operator directive (2026-08-06): *"the hugpy-agent needs to be deployed under
+suspected hangs and edge cases where expected bounds are exceeded — to
+document, remedy, fix or other."* The sentinel formalizes the loop the keeper
+ran by hand on the 2026-08-06 vision wedge (repeated worker 500s → diagnose
+stale projector-less seat → evict → verify → document).
+
+## What one pass does (`hugpy-sentinel run-once`)
+
+1. **Detect** — four read surfaces of central, over HTTP (never imports
+   central's internals): `GET /llm/jobs?live=0` (stalled beyond grace,
+   expired-without-real-terminal), `GET /llm/workers` (offline/unreachable,
+   `version_ok: false` — the tri-state `null` is not an anomaly),
+   `GET /oracle/capabilities` (eligible→ineligible transition vs. the last
+   snapshot), plus a local scorecard history (`record-scorecard` feeds it
+   from observed `POST /oracle/route` responses; N consecutive
+   `hard_pass=false` for one (capability, model) is an anomaly).
+2. **Case** — one OPEN case per fingerprint in a SQLite store under
+   `$HUGPY_SENTINEL_DIR` (partial unique index; re-detection touches
+   `last_seen`, never re-spawns).
+3. **Diagnose** — for each NEWLY opened case, spawn ONE bounded
+   `hugpy-agent case <brief> --case-dir <dir>` subprocess (default 900 s
+   timeout). The agent runs under a pinned DOCUMENT-ONLY policy profile
+   (readonly + case-dir-jailed `fs_write` + `http_fetch`; shell/spawn
+   hard-denied) and must return a markdown case report, stored as
+   `report.md` in the case dir; a one-liner lands in `cases.md`.
+4. States: `open → agent_running → documented | escalated`; `remedied` and
+   `closed` are operator moves (`status` lists them all).
+
+## Remedies are OFF by default
+
+`hugpy_ops/sentinel/remedies.py` is the complete whitelist (worker
+`POST /models/unload`, `POST /slots/<id>/unload|relaunch`, central
+`POST /llm/chat/cancel/<id>` — all reversible). Execution raises
+`RemediesDisabled` unless `HUGPY_SENTINEL_REMEDIES=1` is set, and the prod
+worker **ae** is excluded structurally (`PROD_EXCLUDED_WORKERS`, enforced in
+both eligibility and execute) — ae cases are document+escalate only, always.
+
+**Exception — downloads are ON by default (k97, operator ruling 2026-08-06:
+"autonomous downloads should be greenlit — especially for ComfyUI").** The
+`enqueue_download` remedy rides its OWN gate, `HUGPY_SENTINEL_DOWNLOADS`
+(default ON; set `0` to turn it off), separate from the remedies gate above.
+Each pass runs the k97 provisioner scan (`hugpy_ops/provisioner.py`
+— declared-but-missing weights across the comfy/studio/tasks registries,
+read through `hugpy_engine`, `hugpy_video.intel.studio` and `hugpy_storage`) and
+opens one info-severity `weight_missing` case per missing weight. Those cases
+take a deterministic FAST PATH (no agent spawn): the sentinel POSTs
+`{central}/llm/repos/download` — the console's own add-models enqueue, so the
+job lands on the hugpy-downloader-dev queue with dedupe/progress/cancel —
+then documents the case as `remedied`. A want whose source can't be proven
+from its registry row (no HF hub id) is documented UNRESOLVED, never guessed.
+The ae exclusion is irrelevant here by construction: downloads land on
+central's shared storage, never on a worker. Dry-run the same scan by hand:
+`hugpy-provisioner` (add `--apply` to enqueue).
+
+## Install (operator; the keeper only writes these files)
+
+```sh
+pip install hugpy-ops            # provides the hugpy-sentinel console script
+sudo cp <hugpy_ops>/deploy/hugpy-sentinel.{service,timer} /etc/systemd/system/
+# edit ExecStart's venv path per box, then
+sudo systemctl daemon-reload
+sudo systemctl enable --now hugpy-sentinel.timer
+systemctl list-timers hugpy-sentinel*; journalctl -u hugpy-sentinel -f
+```
+
+Later, to arm remedies (deliberate, after reviewing documented cases): add
+`Environment=HUGPY_SENTINEL_REMEDIES=1` to the service, `daemon-reload`,
+done — nothing else changes.
+
+## Brain ladder + pilot light (k96, operator ruling 2026-08-06)
+
+The 2026-08-06 shakedown aborted its case runs because the agent's single
+configured brain (Qwen3-Coder-Next) was mid-reload. The fix is an ordered
+**brain ladder**: `HUGPY_AGENT_BRAINS` (csv, best-first) on the sentinel unit
+— spawned case agents inherit it. A run starts on the first entry that is
+**warm** on a fleet worker (`GET /llm/workers` allocations) and walks DOWN one
+entry per capacity-class / permanent-verdict refusal (forward-only). Every
+brain chat carries `"no_makeroom": true`, so a cold brain load can NEVER evict
+a fleet model — it takes genuinely free room or is refused fast (which the
+ladder walks). A run answered below ladder position 1 stamps
+`[answered by <model> (ladder position N of M)]` onto its report, so a
+pilot-light case report is visibly reduced-depth.
+
+**The pilot light** is the LAST ladder entry by convention: a model small
+enough that a cold load is cheap, so the sentinel always has *some* brain even
+when the fleet is wedged. Keep it warm with the existing keep-warm star (no
+new pinning machinery — reconcile reseats a star every beat, even after a
+pressure eviction):
+
+```sh
+# pin the pilot light on a worker (operator-gated route); example: computron
+curl -X POST http://127.0.0.1:7002/api/llm/workers/<worker_id>/boot-prewarm \
+     -H 'Content-Type: application/json' \
+     -d '{"model_key": "Qwen~Qwen2.5-3B-Instruct-GGUF"}'
+# verify: the star map, and the model's allocation on /llm/workers
+curl http://127.0.0.1:7002/api/llm/workers/boot-prewarm
+```
+
+Recommended ladder for THIS fleet (ae 3090 + computron 4060-8GB + op,
+2026-08-06 — warm today: Qwen3-Coder-Next on the ae slot, Qwen2.5-VL-3B,
+small LFM/3B models):
+
+```
+HUGPY_AGENT_BRAINS=Qwen~Qwen3-Coder-Next-GGUF,Qwen~Qwen2.5-3B-Instruct-GGUF
+```
+
+Coder stays the quality brain; the 3B instruct is the pilot light — star it on
+computron (8 GB holds a 3B GGUF comfortably beside the VL model) so ae's slot
+churn can never leave case runs brainless. (VL-3B is deliberately NOT on the
+ladder: it is a vision model holding the vision seat.)
+
+The sentinel watches the floor itself: `pilot_light_not_resident` (severity
+warn) fires when the pilot light (from `HUGPY_SENTINEL_PILOT_LIGHT`, else the
+last `HUGPY_AGENT_BRAINS` entry) has no healthy allocation on any worker, and
+the case evidence carries the exact boot-prewarm POST to fix it.
+
+## Knobs (env; all read by `hugpy_ops/sentinel/settings.py`)
+
+| var | default | meaning |
+|---|---|---|
+| `HUGPY_SENTINEL_CENTRAL` | `central_base_url()` → `http://127.0.0.1:7002` | central base |
+| `HUGPY_SENTINEL_DIR` | `<HUGPY_HOME>/state/sentinel` (`hugpy_platform.hugpy_state_dir()`) | case db, case dirs, `cases.md`, history |
+| `HUGPY_SENTINEL_REMEDIES` | unset (OFF) | THE remedy gate |
+| `HUGPY_SENTINEL_DOWNLOADS` | unset (ON) | k97 downloads gate — `enqueue_download` only; `0` disables |
+| `HUGPY_PROVISION_FLOOR_GB` | 500 | provisioner free-space floor on the destination volume |
+| `HUGPY_SENTINEL_STALLED_GRACE_S` | 120 | stall must persist this long past central's 90 s label |
+| `HUGPY_SENTINEL_HARD_FAIL_STREAK` | 3 | consecutive `hard_pass=false` to case |
+| `HUGPY_SENTINEL_AGENT` | `hugpy-agent` | absolute path in the unit |
+| `HUGPY_SENTINEL_AGENT_TIMEOUT_S` | 900 | one case run's wall clock |
+| `HUGPY_SENTINEL_AGENT_MAX_STEPS` | 20 | agent loop bound |
+| `HUGPY_AGENT_BRAINS` | unset | k96 brain ladder, inherited by case agents (csv, best-first; last = pilot light) |
+| `HUGPY_SENTINEL_PILOT_LIGHT` | last `HUGPY_AGENT_BRAINS` entry | override for the pilot-light-not-resident check; empty ladder + unset = check off |
+
+A worker `version_ok: false` case additionally carries the `hugpy-fleet` /
+`hugpy-server` distribution versions installed beside the sentinel
+(`hugpy_ops.versions`, resolved lazily through `importlib.metadata`).
+
+Tests: `python -m pytest tests/test_sentinel.py tests/test_sentinel_http.py -q`
+in `py/operations/hugpy_ops` and `python -m unittest test_case_cli` in
+`py/inference/hugpy_agent/tests` (the pinned agent-side profile).

hugpy_ops-0.2.2/deploy/bin/hugpy-chaos

@@ -0,0 +1,14 @@
+#!/usr/bin/env python3
+"""SUPERSEDED by the `hugpy-chaos` console script (hugpy-ops distribution).
+
+Kept only for operators who run the checkout without installing it:
+    <venv>/bin/python deploy/bin/hugpy-chaos --dry-run
+With `pip install -e py/operations/hugpy_ops` the console script covers this
+file completely; prefer `hugpy-chaos`.
+"""
+import sys
+
+from hugpy_ops.chaos.runner import main
+
+if __name__ == "__main__":
+    sys.exit(main())

hugpy_ops-0.2.2/deploy/bin/hugpy-todo-keeper

@@ -0,0 +1,14 @@
+#!/usr/bin/env python3
+"""SUPERSEDED by the `hugpy-todo-keeper` console script (hugpy-ops distribution).
+
+Kept only for operators who run the checkout without installing it:
+    <venv>/bin/python deploy/bin/hugpy-todo-keeper
+With `pip install -e py/operations/hugpy_ops` the console script covers this
+file completely; prefer `hugpy-todo-keeper` (the unit's ExecStart uses it).
+"""
+import sys
+
+from hugpy_ops.todo_keeper_daemon import main
+
+if __name__ == "__main__":
+    sys.exit(main())

hugpy_ops-0.2.2/deploy/hugpy-sentinel.service

@@ -0,0 +1,76 @@
+# hugpy-sentinel.service — one sentinel pass (k95): detect bound-exceeded
+# conditions on central's read surfaces, open deduplicated cases, spawn ONE
+# bounded document-only `hugpy-agent case` run per NEW case, exit.
+#
+# NOT INSTALLED/ENABLED BY THIS FILE. The keeper writes it; the OPERATOR
+# installs after review (a checker that spawns agent runs — and could one
+# day apply remedies — is an operator decision):
+#
+#   sudo cp <hugpy_ops>/deploy/hugpy-sentinel.{service,timer} /etc/systemd/system/
+#   sudo systemctl daemon-reload
+#   sudo systemctl enable --now hugpy-sentinel.timer
+#   systemctl list-timers hugpy-sentinel*; journalctl -u hugpy-sentinel -f
+#
+# The sentinel is the `hugpy-sentinel` console script of the hugpy-ops
+# distribution (`pip install hugpy-ops`); adjust the venv path below per box.
+#
+# REMEDIES ARE OFF BY DEFAULT and stay off until the operator adds
+# Environment=HUGPY_SENTINEL_REMEDIES=1 below — until then every case is
+# document+escalate only. The prod worker "ae" is excluded in code
+# (hugpy_ops/sentinel/remedies.py PROD_EXCLUDED_WORKERS), not by this file.
+
+[Unit]
+Description=hugpy sentinel: one detect -> case -> diagnose pass
+After=network-online.target hugpy-api-dev.service
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+User=ubuntu
+WorkingDirectory=/home/ubuntu
+ExecStartPre=/usr/bin/test -x /home/ubuntu/hugpy/venv/bin/hugpy-sentinel
+ExecStart=/home/ubuntu/hugpy/venv/bin/hugpy-sentinel run-once
+# One pass polls four HTTP GETs and possibly runs one diagnosis agent per new
+# case (each internally capped at HUGPY_SENTINEL_AGENT_TIMEOUT_S, default
+# 900s). 30m is the outer backstop for a pass that opens several cases.
+TimeoutStartSec=30m
+Nice=10
+
+Environment=PYTHONUNBUFFERED=1
+# Central base URL: HUGPY_SENTINEL_CENTRAL overrides; else the platform
+# resolver (HUGPY_BASE_URL and its legacy aliases), else loopback central.
+Environment=HUGPY_SENTINEL_CENTRAL=http://127.0.0.1:7002
+# Case db, per-case dirs, cases.md ledger, scorecard history — durable, and
+# outside any checkout. Default without this line: <HUGPY_HOME>/state/sentinel.
+Environment=HUGPY_SENTINEL_DIR=/home/ubuntu/.local/state/hugpy-sentinel
+# The one-shot diagnosis agent. Pin the absolute path (PATH inside systemd
+# is minimal); adjust per box.
+Environment=HUGPY_SENTINEL_AGENT=/home/ubuntu/.local/bin/hugpy-agent
+# The agent talks to the fleet through central too.
+Environment=HUGPY_BASE=http://127.0.0.1:7002
+# k96 BRAIN LADDER for spawned case agents (inherited env; csv, best-first).
+# The run starts on the first WARM entry and walks down one entry per
+# capacity/permanent-verdict refusal; the LAST entry is the PILOT LIGHT — keep
+# it small (cheap cold load) and pinned warm (see deploy/SENTINEL.md "Pilot
+# light"). The sentinel also derives its pilot-light-not-resident check from
+# this line (override with HUGPY_SENTINEL_PILOT_LIGHT). Recommended for THIS
+# fleet (ae 3090 + computron 4060-8GB + op; 2026-08-06):
+# Environment=HUGPY_AGENT_BRAINS=Qwen~Qwen3-Coder-Next-GGUF,Qwen~Qwen2.5-3B-Instruct-GGUF
+# Remedy gate — LEAVE UNSET until the operator deliberately enables:
+# Environment=HUGPY_SENTINEL_REMEDIES=1
+
+# Hardening: network + its own state dir, nothing else writable.
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=read-only
+ReadWritePaths=/home/ubuntu/.local/state/hugpy-sentinel
+# The platform's per-user dirs (HUGPY_HOME, default ~/.hugpy) are created on
+# first resolve; grant them so a ProtectHome box does not die at import.
+ReadWritePaths=/home/ubuntu/.hugpy
+ProtectKernelTunables=true
+ProtectControlGroups=true
+RestrictSUIDSGID=true
+
+[Install]
+WantedBy=multi-user.target

hugpy_ops-0.2.2/deploy/hugpy-sentinel.timer

@@ -0,0 +1,17 @@
+# hugpy-sentinel.timer — 10-minute cadence for the sentinel's run-once pass.
+#   sudo systemctl enable --now hugpy-sentinel.timer
+[Unit]
+Description=hugpy sentinel: periodic bound-exceeded sweep
+
+[Timer]
+# Every 10 minutes, first run 2m after boot. A pass that finds nothing is
+# four HTTP GETs; a pass that opens a case runs one bounded agent.
+OnBootSec=2m
+OnUnitActiveSec=10m
+# Jitter so several boxes running a sentinel don't all poll central at once.
+RandomizedDelaySec=60
+Persistent=true
+Unit=hugpy-sentinel.service
+
+[Install]
+WantedBy=timers.target

hugpy_ops-0.2.2/deploy/hugpy-todo-keeper.service

@@ -0,0 +1,78 @@
+# hugpy todo-keeper agent node — the standing P3 agent node that answers the
+# console's ☑ todo drawer assists (todo.add / todo.tidy) as durable dispatched
+# tasks instead of stateless one-shot completions.
+#
+# NOT INSTALLED/ENABLED BY THIS FILE. The keeper writes it; the OPERATOR installs
+# it after review (deliberate — a standing daemon that enrolls a node and holds a
+# credential is an operator decision, not a subagent's):
+#
+#   sudo cp <hugpy_ops>/deploy/hugpy-todo-keeper.service /etc/systemd/system/
+#   sudo systemctl daemon-reload
+#   sudo systemctl enable --now hugpy-todo-keeper
+#   systemctl status hugpy-todo-keeper; journalctl -u hugpy-todo-keeper -f
+#
+# The daemon is the `hugpy-todo-keeper` console script of the hugpy-ops
+# distribution (`pip install hugpy-ops`); adjust the venv path below per box.
+#
+# RESTART SURVIVAL: enrollment happens ONCE; the node id + one-time agt_ token
+# persist to ~/.config/hugpy/todo-keeper.json (0600, outside any checkout), so a
+# reboot resumes the SAME node rather than orphaning it. Restart=always + a
+# 10s backoff covers a central that is briefly down.
+
+[Unit]
+Description=hugpy todo-keeper agent node (P3 /agent node; answers todo.add + todo.tidy)
+After=network.target hugpy-api-dev.service
+Wants=hugpy-api-dev.service
+StartLimitIntervalSec=0
+
+[Service]
+Type=simple
+User=ubuntu
+WorkingDirectory=/home/ubuntu
+ExecStartPre=/usr/bin/test -x /home/ubuntu/hugpy/venv/bin/hugpy-todo-keeper
+ExecStart=/home/ubuntu/hugpy/venv/bin/hugpy-todo-keeper
+Restart=always
+RestartSec=10
+
+Environment=PYTHONUNBUFFERED=1
+# Central base URL — HUGPY_BASE_URL is canonical; the legacy HUGPY_CENTRAL
+# alias is still honoured by hugpy_platform.central_base_url.
+Environment=HUGPY_BASE_URL=http://127.0.0.1:7002
+Environment=HUGPY_TODO_KEEPER_MODEL=Qwen2.5-3B-Instruct-GGUF
+# The token/cursor state file. MUST be on durable storage under $HOME — /tmp and
+# XDG_RUNTIME_DIR are cleared on boot, which would orphan the node every reboot.
+Environment=HUGPY_TODO_KEEPER_STATE=/home/ubuntu/.config/hugpy/todo-keeper.json
+# REQUIRED — /agent/register is PERMANENTLY key-gated (operator ruling 2026-07-16:
+# agent keys are "a separate api category entirely… gated permanently"). The gate
+# ignores the sitewide api_key_required toggle AND HUGPY_AGENT_OPEN, because
+# register MINTS credentials and is publicly reachable. Enrolling bare 401s.
+#
+# Sourced from the share, NOT written here: d-env/env holds CONSOLE_API=<key>, so
+# the secret never enters git (the repo tracks this unit; it must stay clean).
+# EnvironmentFile is optional-prefixed (-) so a missing file doesn't block boot —
+# enrollment then fails loudly with an explicit 401 message instead, which is the
+# honest failure. Once enrolled the node uses its own agt_ token; the console key
+# is only needed for the one-time bootstrap (and any re-enroll).
+# ⚠️ systemd does NOT shell-expand: `Environment=HUGPY_API_KEY=${CONSOLE_API}` passes the
+# LITERAL string "${CONSOLE_API}" (verified 2026-07-16 via `systemctl show -p Environment`),
+# and the node would 401 on enroll. EnvironmentFile already puts CONSOLE_API in the
+# environment under its own name — so the daemon reads CONSOLE_API directly and no
+# expansion is needed. Don't "fix" this back into an Environment= line.
+EnvironmentFile=-/srv/share/projects/hugpy/d-env/env
+
+# Hardening: this daemon needs the network, its state file, and nothing else.
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=read-only
+# ...except its own state directory, which it must write (the token + cursor).
+ReadWritePaths=/home/ubuntu/.config/hugpy
+# The platform's per-user dirs (HUGPY_HOME, default ~/.hugpy) are created on
+# first resolve; grant them rather than loosening ProtectHome.
+ReadWritePaths=/home/ubuntu/.hugpy
+ProtectKernelTunables=true
+ProtectControlGroups=true
+RestrictSUIDSGID=true
+
+[Install]
+WantedBy=multi-user.target