dsh-autorestart 0.8.5 → 0.8.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +275 -5
- package/README.md +450 -61
- package/assets/icon-9c245e5b.ico +0 -0
- package/assets/icon-rescue-1902bb15.ico +0 -0
- package/assets/icon-rescue-1902bb15.png +0 -0
- package/client.js +2547 -287
- package/cordis.patch.yml +12 -0
- package/docs/README.zh.md +228 -28
- package/docs/platform-matrix.md +171 -55
- package/docs/real-machine-verification-0.8.6.md +202 -0
- package/index.js +13272 -9193
- package/lib/autostart-advice.js +31 -2
- package/lib/backup.js +2598 -66
- package/lib/checkpoint.js +121 -4
- package/lib/cookie-clear.js +260 -0
- package/lib/lifecycle.js +23 -40
- package/lib/plugin-compare.js +499 -0
- package/lib/plugin-failure.js +382 -0
- package/lib/plugin-recover.js +1557 -0
- package/lib/plugin-watch.js +213 -0
- package/lib/rescue-paths.js +982 -56
- package/lib/restart-verdict.js +379 -0
- package/lib/restart.js +244 -7
- package/lib/scheduler-capability.js +19 -3
- package/lib/sleep.js +190 -29
- package/lib/watchdog-disarm.js +46 -2
- package/lib/watchdog-schedule.js +834 -17
- package/lib/watchdog-tier.js +8 -1
- package/lib/watchdog.js +1222 -47
- package/locale/en.json +165 -78
- package/locale/zh.json +165 -78
- package/package.json +3 -2
- package/screenshots.json +6 -6
- package/scripts/README.md +6 -3
- package/scripts/build-icon.mjs +998 -0
- package/scripts/clear-dsh-auth-cookies.mjs +257 -0
- package/scripts/emergency-disable.sh +69 -2
- package/scripts/emergency-toggle.ps1 +85 -69
- package/scripts/rescue-strings.zh.json +36 -18
- package/scripts/restore-profile.ps1 +182 -1
- package/scripts/watchdog.mjs +432 -79
package/README.md
CHANGED
|
@@ -61,6 +61,42 @@ Two safety gates protect the restart. The host checks that no conversation is bu
|
|
|
61
61
|
helper checks the same gate over loopback immediately before it kills the process, so a turn that
|
|
62
62
|
starts in between is never killed.
|
|
63
63
|
|
|
64
|
+
## Supported platforms
|
|
65
|
+
|
|
66
|
+
One question decides everything: **which environment below are you in, and has anybody actually run it there?**
|
|
67
|
+
|
|
68
|
+
| Your machine | Run on a real machine | What you must know |
|
|
69
|
+
| --- | --- | --- |
|
|
70
|
+
| **Windows desktop** (the DeepSeek Harness client) | **yes — verified** | nothing to configure — `restartMode: app` is the default |
|
|
71
|
+
| **`dsh web` on WSL, with `systemd --user`** | **yes — verified** | you must set `restartMode: command` **and** a `restartCommand` that really restarts this host; remember the shutdown button switches off the **whole Windows machine**, not the distro |
|
|
72
|
+
| **`dsh web` on WSL, without `systemd --user`** | **not yet** | the same configuration as the row above; the automatic rollback falls back to a detached process of the plugin's own |
|
|
73
|
+
| **`dsh web` on Windows** | **not yet** | the same configuration as the row above |
|
|
74
|
+
| **`dsh web` on Linux** (not WSL) | **not yet — there is no such machine here** | the same configuration as the row above |
|
|
75
|
+
|
|
76
|
+
The last three rows are **implemented and covered by unit tests, but no one has run them on a real host**;
|
|
77
|
+
they take the same code path as the verified WSL row (`restartMode: command` plus your own restart
|
|
78
|
+
command), yet "it should work that way" is not "it was verified there", so it is not claimed. The
|
|
79
|
+
**with/without `systemd --user`** split changes only *how* the automatic rollback is triggered (a systemd
|
|
80
|
+
user timer versus a detached background process of the plugin's own); everything else is the same. The
|
|
81
|
+
settings page always says which one your machine got, and it never promises a rollback it cannot deliver.
|
|
82
|
+
|
|
83
|
+
### Out of scope — not tested, not supported
|
|
84
|
+
|
|
85
|
+
These three are **not** in the supported set. The plugin may appear to run there, but nothing is
|
|
86
|
+
promised: no real-machine test exists, and some abilities simply cannot be delivered.
|
|
87
|
+
|
|
88
|
+
| Environment | Status | What you lose |
|
|
89
|
+
| --- | --- | --- |
|
|
90
|
+
| **macOS** | **not tested / not supported** | no desktop rescue shortcut; the automatic rollback degrades to a background process of the plugin's own (and on a machine without `setsid` it degrades further to *no rollback at all*); the shutdown countdown has never been verified on a real Mac |
|
|
91
|
+
| **Remote SSH with no graphical session** | **not tested / not supported** | no desktop and no desktop rescue entry, so the only way back in is a `.sh` script — which means you must already be able to SSH in; the "start at login" advice starts the **host**, not your SSH session; the session resume only happens once *somebody* starts the host again |
|
|
92
|
+
| **Containers** (Docker / Compose / Kubernetes) | **not tested / not supported** | no "start at login" at all (the orchestrator owns that); the automatic rollback degrades to a background process of the plugin's own (often no `setsid` in a slim image → *no rollback*); the shutdown countdown is **not** container-aware — it really tries to power off the host, so do not press it in a container; a rebuilt container may take the checkpoint and the backups with it |
|
|
93
|
+
|
|
94
|
+
If you are on one of those three, install it as an experiment at your own risk, and use
|
|
95
|
+
`scripts/restore-profile.sh` / `scripts/emergency-disable.sh` as your way out.
|
|
96
|
+
|
|
97
|
+
The complete per-capability matrix — every ability × every environment, each cell with the
|
|
98
|
+
`file:line` it rests on — is in [docs/platform-matrix.md](docs/platform-matrix.md).
|
|
99
|
+
|
|
64
100
|
## The whole thing in one page (ordinary users can stop here)
|
|
65
101
|
|
|
66
102
|
### Why a restart does not lose your work
|
|
@@ -74,8 +110,11 @@ The plugin turns a restart into a **resumable** event, and the order never chang
|
|
|
74
110
|
itself; the hosted web client is covered below);
|
|
75
111
|
4. **The new process checks itself** — four things: is the port up, are the routes there, did the
|
|
76
112
|
plugin load, did the sessions resume;
|
|
77
|
-
5. **If it did not come up, it rolls back** —
|
|
78
|
-
|
|
113
|
+
5. **If it did not come up, it rolls back** — a snapshot (configuration + installed plugins) whose
|
|
114
|
+
identity is `known-good` (it was taken before a restart that then came up) is put back, and the
|
|
115
|
+
verdict of that restart is written down honestly.
|
|
116
|
+
**When there is not a single copy with that identity**, it does not put "the newest one" in its
|
|
117
|
+
place: it reports plainly that there is no restore point carrying an identity.
|
|
79
118
|
|
|
80
119
|
So "the restart lost my work" becomes "the restart carried on": you do not have to type "continue".
|
|
81
120
|
|
|
@@ -112,13 +151,17 @@ writes that sentence and the matching command into the settings page.
|
|
|
112
151
|
configuration, another plugin, a bad upgrade) → go back to the last good state; if **only this
|
|
113
152
|
plugin** is suspect → take this plugin out instead.
|
|
114
153
|
|
|
115
|
-
**And use the scripts of YOUR
|
|
116
|
-
|
|
154
|
+
**And use the scripts of YOUR host** — a Linux machine has no `.cmd` file, and a Windows machine has
|
|
155
|
+
no `.sh`. Pick by the host you are sitting in front of, not by a platform list:
|
|
117
156
|
|
|
118
|
-
| Your
|
|
157
|
+
| Your host | back to the last good state | take this plugin out |
|
|
119
158
|
|---|---|---|
|
|
120
159
|
| **Windows** (the desktop app, `dsh web` on Windows) | double-click `scripts/restore-profile.cmd` | double-click `scripts/emergency-disable.cmd` |
|
|
121
|
-
| **
|
|
160
|
+
| **WSL / Linux** (`dsh web`) | `sh scripts/restore-profile.sh` | `sh scripts/emergency-disable.sh` |
|
|
161
|
+
| **macOS** | `sh scripts/restore-profile.sh` | `sh scripts/emergency-disable.sh` |
|
|
162
|
+
|
|
163
|
+
On macOS the same two POSIX `sh` scripts apply; they have never been run on a real Mac, so they are
|
|
164
|
+
untested and nothing is promised.
|
|
122
165
|
|
|
123
166
|
Two scripts per platform, and they do different jobs:
|
|
124
167
|
|
|
@@ -138,6 +181,15 @@ in [`scripts/README.md`](scripts/README.md).
|
|
|
138
181
|
[`docs/platform-matrix.md`](docs/platform-matrix.md) — the settings page shows the paths of the
|
|
139
182
|
scripts that really exist on the machine you are on.
|
|
140
183
|
|
|
184
|
+
**The desktop rescue shortcut (Windows only).** The settings page can put a rescue shortcut on the
|
|
185
|
+
desktop; double-clicking it runs `restore-profile.cmd`. **Its icon is the plugin's own
|
|
186
|
+
`assets/icon.ico` — not DSH's executable.** That is deliberate: on a desktop where DSH will not
|
|
187
|
+
start, a shortcut wearing DSH's face reads as "this is the program", which is exactly the thing
|
|
188
|
+
that is broken. When an install ships no icon file, the `.lnk` is still created — it simply gets no
|
|
189
|
+
`IconLocation` instead of borrowing some other program's face, and nothing is generated at runtime.
|
|
190
|
+
Every other platform gets no desktop shortcut at all; the settings page gives a copyable
|
|
191
|
+
restore-script command instead.
|
|
192
|
+
|
|
141
193
|
### What has really been tested
|
|
142
194
|
|
|
143
195
|
| How it was started | Tested on a real machine? | Evidence |
|
|
@@ -149,16 +201,19 @@ scripts that really exist on the machine you are on.
|
|
|
149
201
|
| a Windows login task | **partly** | on a real machine one login-triggered task was created → queried (State=Ready, the logon trigger) → deleted (no leftovers) with the system's own tools, which proves that no administrator rights are needed; **"the fallback into the startup folder when the task is refused" is pinned by unit tests only** |
|
|
150
202
|
| the desktop client | **yes** | on a real machine it comes back by itself (there is a snapshot and a self-check record, no rollback was triggered, and it does not start twice); that is the client's own business, it is not part of the hosted matrix |
|
|
151
203
|
| pm2 / forever | no, unit tests only | the recognition and the behaviour are pinned by unit tests |
|
|
152
|
-
| container / compose | no
|
|
204
|
+
| container / compose | **no — out of scope** | see [Supported platforms](#supported-platforms): containers are **not** a supported environment |
|
|
153
205
|
| `npx` / `pnpm dlx` wrapper | no, unit tests only | same |
|
|
154
206
|
| started by another DSH plugin | no, unit tests only | same |
|
|
155
|
-
| tmux / screen / an SSH session | no
|
|
207
|
+
| tmux / screen / an SSH session | **no — out of scope** | see [Supported platforms](#supported-platforms): a remote SSH session without a GUI is **not** a supported environment |
|
|
156
208
|
| WSL started by a Windows-side process | no, unit tests only | same (it is recognised as "cannot tell", so nothing happens) |
|
|
157
209
|
|
|
158
|
-
**
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
210
|
+
**"Unit tests only" means "not verified on a real machine", not "unsupported"** — on the four
|
|
211
|
+
supported platforms those paths are pinned by unit tests, and "no full install/uninstall cycle" does
|
|
212
|
+
not mean unsupported either. The three environments listed under
|
|
213
|
+
[Supported platforms](#supported-platforms) (macOS, a remote SSH session without a GUI, containers)
|
|
214
|
+
are a different case: they are **outside** the supported set, and the table above says so.
|
|
215
|
+
If your environment is recognised wrongly, please send us the "startup mode + the evidence" block
|
|
216
|
+
from the settings page.
|
|
162
217
|
|
|
163
218
|
### Report a problem
|
|
164
219
|
|
|
@@ -199,9 +254,22 @@ what it adds before you install it.
|
|
|
199
254
|
| An agent owns a durable goal (`create_goal`) that should survive a restart | The goal activation lives in the process, so after the restart it is `disarmed` and the automatic continuation stops | The plugin re-arms the goal at boot and the goal round driver starts the next round right away |
|
|
200
255
|
| An agent wants to install or upgrade a plugin, but a restart would cut the work short | You either wait or take the risk | Call `continuum_restart`: it waits until every agent is idle and the session continues by itself afterwards |
|
|
201
256
|
|
|
257
|
+
## Host version requirement
|
|
258
|
+
|
|
259
|
+
**Requires DSH ≥ `0.1.6-alpha.2`.**
|
|
260
|
+
|
|
261
|
+
Since `0.1.6-alpha.2` the host handles "a plugin did not come up" in two different ways:
|
|
262
|
+
|
|
263
|
+
- **Ordinary plugins** (your own, or installed from npm) that fail to load **do not block startup**: the host boots anyway, logs `warning: N entry did not activate`, and leaves the other plugins running. That log line is exactly what this plugin reads to show "these features did not come up this time".
|
|
264
|
+
- **Required plugins** (the host's own parts, e.g. the web server) that fail **still fail the startup**: the log says `startup failed: N required plugins did not activate`. That is the case the watchdog handles — restore a verified backup, then bring the host back up.
|
|
265
|
+
|
|
266
|
+
On `0.1.6-alpha.1` and older there is no such distinction: any failing plugin throws and the host does not come up at all, so the signal this plugin depends on does not exist there.
|
|
267
|
+
|
|
268
|
+
**How this was established (stated plainly)**: by reading DSH's official `@deepseek-ai/dsh-app-boot` package release by release (`0.1.0-rc.2` / `0.1.3-alpha.2` / `0.1.5-rc.3` / `0.1.6-alpha.1` / `0.1.6-alpha.2` / `0.1.7-rc.2` / `0.2.0-rc.1` / `0.2.1-rc.3` / `0.2.2-alpha.2`); the boundary falls at `0.1.6-alpha.2`. The official DSH release notes do **not** state this: the `0.1.6-alpha.2` entry only says "Group startup errors and pending services, and save full diagnostics to a log file". So this is a source-level finding plus a local measurement, not an official declaration.
|
|
269
|
+
|
|
202
270
|
## Install
|
|
203
271
|
|
|
204
|
-
Current version: **0.8.
|
|
272
|
+
Current version: **0.8.6**.
|
|
205
273
|
|
|
206
274
|
From npm — the package is published under the name `dsh-autorestart`:
|
|
207
275
|
|
|
@@ -297,32 +365,45 @@ Two blocks of the settings page are worth a word, because they show facts instea
|
|
|
297
365
|
|
|
298
366
|
- **What is backed up right now** (right under the two backup switches): the HOST reads
|
|
299
367
|
`<DSH_HOME>/storages/dsh-autorestart/backups/`, so the number of copies, the **local time** of each
|
|
300
|
-
one,
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
Every
|
|
304
|
-
|
|
368
|
+
one, how much disk they take together and where the folder lives are all real. Only the directory
|
|
369
|
+
metadata is read — the contents are never walked (one copy can be 150 MB). With no backup at all it
|
|
370
|
+
says: "No backup yet; one is written before every restart."
|
|
371
|
+
Every row is one copy and carries two things: its backup **time**, and the one action that applies
|
|
372
|
+
to it ("Roll back to this copy"). A row carries **no size** — the total sits once, above the list,
|
|
373
|
+
as "Using 156 MB on disk, 2 copies right now". The list does **not** say which copy is the safe
|
|
374
|
+
one — the plugin no longer makes that call (user decision, 2026-11-02): a row is a time and a
|
|
375
|
+
button, and the judgement is the reader's:
|
|
305
376
|
|
|
306
377
|
```
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
Backup
|
|
378
|
+
Using 156 MB on disk, 2 copies right now
|
|
379
|
+
|
|
380
|
+
Backup time 2026-10-29 22:08 [Roll back to this copy]
|
|
381
|
+
Backup time 2026-10-29 21:44 [Roll back to this copy]
|
|
310
382
|
```
|
|
311
383
|
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
384
|
+
One row can carry one more neutral sentence, and it is not a verdict: the copy the host last
|
|
385
|
+
started from says "This is the state at the most recent startup. DSH starting does not mean every
|
|
386
|
+
plugin is healthy — a broken plugin is excluded and the host still boots. Which copy to roll back
|
|
387
|
+
to is your call." It is attached by the copy's own record, never by its position in the list.
|
|
388
|
+
|
|
389
|
+
The identity still exists, but **it is not on disk**. `meta.json` next to each copy records FACTS
|
|
390
|
+
only — when the copy was taken (`takenAt` / `at`), what put it there (`trigger`: `first-install` /
|
|
391
|
+
`pre-restart` / `manual-rollback`), `version`, the content fingerprint, and what the boot that
|
|
392
|
+
followed reported (`boot`: did the host come up, was it running this content, which plugin load
|
|
393
|
+
failures it listed). The three names are **derived from those facts when the copy is read** —
|
|
394
|
+
`known-good` (a boot is on record after it, or it is the first-install copy), `snapshot` (no boot
|
|
395
|
+
after it yet) or `unknown` (no readable `meta.json`, or no usable boot record on an old copy). Being
|
|
396
|
+
restorable is a **second, narrower check** on top of that name: an automatic rollback only restores
|
|
397
|
+
a copy whose recorded boot both started and ran this exact content. The value records what happened
|
|
398
|
+
after the copy was taken; it is not an evaluation of whether the plugins
|
|
399
|
+
inside it work. The browser cannot read `meta.json`, and no identity is printed on a row.
|
|
320
400
|
|
|
321
401
|
The time is the local clock of the machine that took the copy, **to the minute and with no
|
|
322
|
-
timezone offset** (a `+08:00` only adds noise — the reader is looking at their own clock). The
|
|
323
|
-
|
|
324
|
-
(`117.7 MB`, never `156.0 MB`)
|
|
325
|
-
|
|
402
|
+
timezone offset** (a `+08:00` only adds noise — the reader is looking at their own clock). The size
|
|
403
|
+
of the whole folder picks its own unit (`92 KB` / `156 MB`) and keeps the tenth only when it says
|
|
404
|
+
something (`117.7 MB`, never `156.0 MB`); it is printed once in the summary line above the rows,
|
|
405
|
+
never on a row, and it carries no "(plugins included)" bracket. Two copies are two groups with air
|
|
406
|
+
between them.
|
|
326
407
|
- **A setting that was saved says so.** Every control writes itself the moment it is changed, so the
|
|
327
408
|
panel has to say whether the write landed: once the HOST answers `ok: true`, a green strip at the
|
|
328
409
|
**top of the settings panel itself** (inside the panel — a `sticky` row, so it stays visible while
|
|
@@ -352,9 +433,9 @@ show up in the Plugin Manager and in the config editor.
|
|
|
352
433
|
| `maxSessions` | `4` | Maximum number of sessions resumed per boot. |
|
|
353
434
|
| `maxResumeAttempts` | `2` | Maximum resume attempts per session, counted across boots, so a bad session cannot loop forever. |
|
|
354
435
|
| `deferPollMs` | `3000` | Poll interval while waiting for the conversations to go idle. |
|
|
355
|
-
| `deferSoftMs` | `300000` |
|
|
356
|
-
| `deferHardMs` | `900000` |
|
|
357
|
-
| `deferPolicy` | `auto` |
|
|
436
|
+
| `deferSoftMs` | `300000` | **Deprecated — read but completely ignored.** A pending restart has no deadline. Kept only so existing configuration files stay valid. |
|
|
437
|
+
| `deferHardMs` | `900000` | **Deprecated — read but completely ignored.** A pending restart never gives up. Kept only so existing configuration files stay valid. |
|
|
438
|
+
| `deferPolicy` | `auto` | Kept for configuration compatibility. Waiting is the same under every value: a pending restart waits for the idle gate, with no deadline. |
|
|
358
439
|
| `forceRestartWhenBusy` | `false` | Whether a restart may be forced while a conversation is still running. The default `false` never interrupts a session: the plugin waits until the other conversations finish. |
|
|
359
440
|
| `restartMode` | `app` | `app` = restart the DeepSeek Harness desktop app. `command` = run `restartCommand`. |
|
|
360
441
|
| `restartCommand` | `''` | Custom restart command. Supports the `{pid}`, `{ppid}` and `{app}` placeholders. |
|
|
@@ -369,7 +450,7 @@ show up in the Plugin Manager and in the config editor.
|
|
|
369
450
|
| `continuationText` | `''` | Overrides the default continuation prompt text. |
|
|
370
451
|
| `backupOnRestart` | `true` | Snapshot the profile before every restart, so a restart that leaves the host unable to boot can be rolled back. |
|
|
371
452
|
| `backupIncludePlugins` | `true` | Include the installed plugin tree in that snapshot (about 117 MB here, ~3 s). `false` keeps only the configuration files. |
|
|
372
|
-
| `maxBackups` | `3` | How many snapshots are kept (the current one plus the two before it); older ones are deleted when a new one is written — **except the newest known-good
|
|
453
|
+
| `maxBackups` | `3` | How many snapshots are kept (the current one plus the two before it); older ones are deleted when a new one is written — **except the newest copy whose identity is `known-good`, which is never deleted**. |
|
|
373
454
|
| `rollbackTimeoutMs` | `90000` | How long the detached watcher waits for the new host before it restores the snapshot. |
|
|
374
455
|
| `restartCommandTimeoutMs` | `30000` | How long the restart command may run before the plugin stops waiting for it. A command that starts the host in the foreground never returns — that is reported as "still running", not as a failure. |
|
|
375
456
|
| `rollbackCopyTimeoutMs` | `120000` | How long ONE restore step (the configuration files, then the plugin tree) may take. A restore that does not finish in time is given up and the reason is written into the rollback report. |
|
|
@@ -424,7 +505,7 @@ delayed work of the other plugins — and it is the **only** such window:
|
|
|
424
505
|
for the restart gate. That is right while the message is about to be picked up — the agent claims
|
|
425
506
|
it within a second or two — but it is a deadlock when nobody ever claims it. The real case was a
|
|
426
507
|
fresh, empty conversation in the UI with one unclaimed message in its inbox: the gate stayed
|
|
427
|
-
closed
|
|
508
|
+
closed and the restart never happened.
|
|
428
509
|
|
|
429
510
|
**The upper bound.** `queuedGraceMs` (default **60000** ms) is how long an unclaimed message may
|
|
430
511
|
hold the restart back:
|
|
@@ -484,10 +565,13 @@ Both `…/health` and `…/health?state=1` carry `restartPhase` (stage, bilingua
|
|
|
484
565
|
line next to the restart settings — the pill is only there while a restart is pending, the settings
|
|
485
566
|
line is always there.
|
|
486
567
|
|
|
487
|
-
### What you see while the restart waits
|
|
568
|
+
### What you see while the restart waits
|
|
488
569
|
|
|
489
|
-
|
|
490
|
-
|
|
570
|
+
A restart waits for one thing only: **every conversation finishing its work**. There is no deadline
|
|
571
|
+
and nothing expires — it keeps waiting for as long as that takes, and only two things end it: the
|
|
572
|
+
gate opens (the restart runs) or you withdraw the request.
|
|
573
|
+
|
|
574
|
+
The wait is visible from the first second, not after some threshold:
|
|
491
575
|
|
|
492
576
|
- **The request is kept.** The pending restart stays pending, the requester stays checkpointed, and
|
|
493
577
|
the restart fires as soon as the gate is quiet again. Nothing is cancelled behind your back.
|
|
@@ -498,7 +582,8 @@ not pretend that everything is fine — and it does not give up either:
|
|
|
498
582
|
`<session>` keeps holding it" (「已等 17 分:`<会话>`一直占着」).
|
|
499
583
|
|
|
500
584
|
A conversation is never killed to make the restart happen: `forceRestartWhenBusy` stays off by
|
|
501
|
-
default, and this plugin never turns it on.
|
|
585
|
+
default, and this plugin never turns it on. When it IS turned on, it restarts over a running
|
|
586
|
+
conversation — it interrupts it. That is a choice about interrupting work, not a deadline.
|
|
502
587
|
|
|
503
588
|
### What the quiet window measures
|
|
504
589
|
|
|
@@ -563,20 +648,39 @@ it was started from, so a restart that leaves the host unable to boot can be und
|
|
|
563
648
|
`backupIncludePlugins` is turned off — the installed plugin tree (`node_modules`). Snapshots live in
|
|
564
649
|
`<storage>/dsh-autorestart/backups/<timestamp>/` with a `manifest.json` next to them. On a profile
|
|
565
650
|
like this one the full snapshot measures about 117 MB and takes under three seconds.
|
|
566
|
-
- **
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
651
|
+
- **PLUGINS INSTALLED WITH `link:` ARE BACKED UP TOO** (2026-10-30, measured gap). Five plugins on
|
|
652
|
+
this machine were installed as links/junctions (`dsh-autorestart` itself among them), and the tree
|
|
653
|
+
copy recorded the LINK and not one byte behind it — so the plugin that does the rescuing could not
|
|
654
|
+
be put back. The manifest now carries `links: [{name, target, kind, exists, readable, copied, files,
|
|
655
|
+
bytes}]` (+ `linkFiles` / `linkBytes`) and the CONTENTS of every enabled linked plugin are kept in
|
|
656
|
+
`node_modules-link/<name>/` (1 216 files / 69.5 MB for these five). On restore each link gets one
|
|
657
|
+
decision, printed as its own line — `kept` (target still there: the development directory keeps
|
|
658
|
+
working), `replaced` (the copy takes its place), `target-missing`, `no-backup`. The restore script
|
|
659
|
+
has `-ForceLinkPlugins` for "replace them all"; the watchdog uses the same rules
|
|
660
|
+
(`decideLinkRestore`) and does not give a link a second chance (`attempts >= 2` ⇒ replaced).
|
|
661
|
+
- **The rollback target is the copy taken before an earlier restart that came up, not the newest
|
|
662
|
+
copy** (user decision, 2026-10-30). Every copy carries a `meta.json`, and **it records facts
|
|
663
|
+
only**: when it was taken, what put it there (`trigger`), `version`, the content fingerprint, and
|
|
664
|
+
what the boot that followed reported (did the host come up, was it this content, which plugin load
|
|
665
|
+
failures). The three identity names — `known-good`, `snapshot`, `unknown` — are **derived from
|
|
666
|
+
those facts when the copy is read, never written to disk**: a copy with a boot on record after it
|
|
667
|
+
(or the first-install copy) is `known-good`; a copy with no boot after it yet is `snapshot`; a copy
|
|
668
|
+
with no readable `meta.json`, or an old one with no usable boot record, is `unknown`. That name is
|
|
669
|
+
not the same question as "can it be restored": the automatic rollback additionally requires the
|
|
670
|
+
recorded boot to have started and to have run this exact content. The identity records what
|
|
671
|
+
happened after the copy was taken; it is not an
|
|
672
|
+
evaluation of whether the plugins inside it work. The snapshot taken right before THIS restart is
|
|
673
|
+
never the rollback target — it may be the broken state itself (that is exactly the "I edited a
|
|
674
|
+
plugin and clicked restart" case). The first install seeds one copy with `trigger: first-install`,
|
|
675
|
+
and every boot whose checks pass on this exact content records a `boot` on that restart's snapshot.
|
|
676
|
+
- **No `known-good` copy, no restore at all.** When no copy has a boot recorded after it yet, the
|
|
677
|
+
plugin restores nothing and says why: the report carries `no-known-good: <reason>`, the profile is
|
|
678
|
+
left exactly as the failed restart left it, and no watcher is armed (a watcher with nothing to
|
|
679
|
+
restore would only write the broken state back).
|
|
577
680
|
- **How many are kept:** the newest `maxBackups` snapshots (default `3`: the current one plus the two
|
|
578
|
-
before it); older ones are deleted when a new one is written. The newest
|
|
579
|
-
never deleted, whatever the limit says — it is the only
|
|
681
|
+
before it); older ones are deleted when a new one is written. The newest copy whose identity is
|
|
682
|
+
`known-good` is never deleted, whatever the limit says — it is the only kind an automatic rollback
|
|
683
|
+
restores.
|
|
580
684
|
- **How a failed boot is detected:** a small watcher runs **outside** the host process (so it survives
|
|
581
685
|
the host being killed). It waits for the **new** instance and probes **its** health route for
|
|
582
686
|
`rollbackTimeoutMs` (default `90000`). The new instance publishes its own port and its own boot id
|
|
@@ -593,8 +697,8 @@ it was started from, so a restart that leaves the host unable to boot can be und
|
|
|
593
697
|
The watcher is therefore started **before** the command runs, and it is the one that decides
|
|
594
698
|
whether the new instance came up.
|
|
595
699
|
- **A manual rollback to ANY copy** (user decision, 2026-10-30): the settings page offers
|
|
596
|
-
"roll back to this copy" on every row, so a copy from last week — even
|
|
597
|
-
restored on purpose. It asks first (which copy, that the current state is replaced, that a restart
|
|
700
|
+
"roll back to this copy" on every row, so a copy from last week — even one with no boot recorded
|
|
701
|
+
after it — can be restored on purpose. It asks first (which copy, that the current state is replaced, that a restart
|
|
598
702
|
follows); the Host saves the current state as a fresh snapshot BEFORE it restores anything, writes
|
|
599
703
|
`last-manual-rollback.json`, and only then requests a normal restart (the plugin never kills the
|
|
600
704
|
host itself). The route (`POST /__dsh_autorestart/rollback-to`) takes a backup NAME only — a path
|
|
@@ -669,10 +773,166 @@ instance alive? (then there is nothing to restore) → **ask the health route of
|
|
|
669
773
|
the host itself** (only a host that does not answer may be rolled back; when
|
|
670
774
|
there is no port to ask, one more round is waited and the markers are looked at
|
|
671
775
|
again) → take the shared lock (`rollback.lock`, so exactly one of us restores) →
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
776
|
+
and then, **in this order**: restore the snapshot → **remove the stale
|
|
777
|
+
`dsh-auth-*` handshake cookies** → start the host again → wait for the ready
|
|
778
|
+
marker.
|
|
779
|
+
|
|
780
|
+
Each pass tries up to **`maxAttempts`** times (3 by default, with a hard ceiling
|
|
781
|
+
— never endless): a pass that did not come up backs off (5s / 10s / 20s, capped)
|
|
782
|
+
and the three questions above are asked AGAIN before the next start, because the
|
|
783
|
+
world changed in between (a slow boot is `not-needed`, an instance that came up
|
|
784
|
+
is `abandoned`). The snapshot is restored on the **first** try only (copying the
|
|
785
|
+
same tree twice changes nothing), **but the cookies are cleared before every
|
|
786
|
+
single start**: starting the host with 71 stale cookies is starting it to die
|
|
787
|
+
again (the Cookie header overflows 16 KB and the host answers 431).
|
|
788
|
+
|
|
789
|
+
**A failure always leaves a report.** Whatever the outcome — recovered,
|
|
790
|
+
not-needed, abandoned, decided, failed, or out of attempts — every pass writes
|
|
791
|
+
`<stateDir>/last-watchdog-report.json`: which tier it used, `healthCheck`, how
|
|
792
|
+
many attempts, and per attempt `restored` / `cookieClear` / `relaunched` /
|
|
793
|
+
`ready` / the backoff, plus why it stopped. A report that never reaches the disk
|
|
794
|
+
is a report that never happened; that was one of the three defects 0.8.6 fixes.
|
|
795
|
+
|
|
796
|
+
### What the watchdog does NOT cover
|
|
797
|
+
|
|
798
|
+
Be clear about the window, because the whole thing is a **window**:
|
|
799
|
+
|
|
800
|
+
- the watchdog is armed **only while this plugin is restarting the host** — it is
|
|
801
|
+
a one-off item scheduled just before the kill, and the host disarms it at boot
|
|
802
|
+
(`watchdog-pending.json` gone, ready marker written, the scheduling item
|
|
803
|
+
deleted);
|
|
804
|
+
- **outside that window nobody is watching.** A host that is killed by the system
|
|
805
|
+
at 03:00, a client that is start-and-die on a manual double-click, or a machine
|
|
806
|
+
that is found with 0 processes in the morning has **no watchdog to bring it
|
|
807
|
+
back**: the plugin is not running (it died with the host) and no one-off item
|
|
808
|
+
was ever registered. What is left in that case is `scripts/restore-profile.cmd`
|
|
809
|
+
(double-click it) and, when the profile itself was emptied, nothing at all;
|
|
810
|
+
- the watchdog restores **one snapshot** and starts the host — it repairs no bug
|
|
811
|
+
of the host. A host that dies for the same reason every time ends as `failed`
|
|
812
|
+
with a report that names the attempts;
|
|
813
|
+
- `watchdogEnabled: false` (and a machine whose tier is `none`) means no
|
|
814
|
+
automatic rollback at all, and the settings page says so.
|
|
815
|
+
|
|
816
|
+
### A plugin that did not load: the REPORTED kind of failure
|
|
817
|
+
|
|
818
|
+
Not every plugin failure keeps the host from booting. The other one is quieter: the
|
|
819
|
+
host comes up, one entry **never activates**, and the feature behind it simply 404s.
|
|
820
|
+
The startup log says so (`<plugin>: failed to import: <error>`, `<plugin>: import
|
|
821
|
+
failed …`, `- <plugin>: loading`, or the aggregate `dsh: warning: N entry did not
|
|
822
|
+
activate`) and, without this pass, nobody reads it.
|
|
823
|
+
|
|
824
|
+
The two failures are handled by two different mechanisms, and they never overlap:
|
|
825
|
+
|
|
826
|
+
| failure | who handles it |
|
|
827
|
+
|---|---|
|
|
828
|
+
| **process level** — the host does not come up at all | the pre-restart snapshot + the watchdog: one snapshot is restored and the host is started again |
|
|
829
|
+
| **reported** — the host is up, one plugin silently did not activate | **a report and a popup, nothing else** — the user decides, and one button puts ONE plugin back |
|
|
830
|
+
|
|
831
|
+
**THE BOOT PASS DOES NOT FIX ANYTHING** (user decision, 2026-11-01). It used to
|
|
832
|
+
put the backup whose identity is `known-good` back and ask for one restart;
|
|
833
|
+
measurement showed that was redundant work on top of what the host already does:
|
|
834
|
+
|
|
835
|
+
- the host **already survives** a broken plugin — measured at **22 s** on the
|
|
836
|
+
desktop (normal boot, the broken plugin's route 404s, the configuration is not
|
|
837
|
+
touched) and at **2.0–2.6 s** on a hosted `dsh web` instance (the profile's
|
|
838
|
+
`package.json` was byte-identical across five samples, and `supervise-web.sh`
|
|
839
|
+
started the host **once** in 62.9 s);
|
|
840
|
+
- so an automatic restore only added a second writer into the profile at boot,
|
|
841
|
+
and an automatic restart on top of it.
|
|
842
|
+
|
|
843
|
+
What the boot pass does now: **read the previous boot's log, recognise the
|
|
844
|
+
failed plugins, write `<stateDir>/last-plugin-recovery.json`.** It never calls
|
|
845
|
+
the restore, it never asks for a restart — both facts are pinned by tests (every
|
|
846
|
+
file in the profile keeps its sha256, no `<plugin>.broken-<ts>` evidence is
|
|
847
|
+
created, `state.restart` stays empty).
|
|
848
|
+
|
|
849
|
+
**The popup is the half that was missing.** A card in the composer dock says
|
|
850
|
+
which plugin did not load and what error it reported, and offers exactly one
|
|
851
|
+
action: *"Restore `<plugin>` only"*.
|
|
852
|
+
|
|
853
|
+
**`POST /__dsh_autorestart/plugin-restore`** with `{ "plugin": "<name>" }` is
|
|
854
|
+
the ONLY remaining writer into the profile, and it only runs because a human
|
|
855
|
+
pressed the button:
|
|
856
|
+
|
|
857
|
+
- **`overwrite`** — the newest backup whose identity is `known-good` holds a real copy: the live
|
|
858
|
+
directory is renamed to `<name>.broken-<ts>` (**evidence — nothing is ever
|
|
859
|
+
deleted**) and the backup copy takes its place;
|
|
860
|
+
- **`replace-link`** — the backup only holds the CONTENTS of a `link:` install
|
|
861
|
+
(`node_modules-link/<name>`): the live link is moved aside the same way and the
|
|
862
|
+
copied contents take its place;
|
|
863
|
+
- **`already-consistent`** — the live plugin already equals the backup: no file
|
|
864
|
+
is written and no restart is asked for (pressing twice is harmless);
|
|
865
|
+
- **`no-backup` / a copy that fails** — `ok:false` with a `reason` **and HTTP
|
|
866
|
+
200**: "there is no backup for this plugin" is an answer, not a server error.
|
|
867
|
+
|
|
868
|
+
A restore touches **that one plugin**: every other plugin, and the profile's
|
|
869
|
+
`package.json` / `cordis.patch.yml`, are byte-identical afterwards (asserted
|
|
870
|
+
file by file). Each attempt is appended to
|
|
871
|
+
`<stateDir>/plugin-restore-history.json` (newest 20) and the newest one is
|
|
872
|
+
carried by the health route as `pluginRestore`, so the card can say "already put
|
|
873
|
+
back — restart to load it".
|
|
874
|
+
|
|
875
|
+
The loop guards of the old automatic pass (`duplicate` / `needs-human`) are
|
|
876
|
+
**not** applied to a button press: "should the machine act on its own" is a
|
|
877
|
+
different question from "a person asked again", and the idempotency check still
|
|
878
|
+
protects the disk.
|
|
879
|
+
|
|
880
|
+
Two loops are possible, and both are stopped:
|
|
881
|
+
|
|
882
|
+
1. **the same old log line read again.** Every boot would "find" the line the
|
|
883
|
+
previous boot left behind. The pass only reads the log of the **previous** boot,
|
|
884
|
+
drops lines older than that boot, and gives every failure a **fingerprint**
|
|
885
|
+
(plugin + first error line + log time) that is recorded as handled once — the
|
|
886
|
+
second time it is a `duplicate` and nothing happens;
|
|
887
|
+
2. **the cause is not in the plugin** (a missing peer dependency, a missing host
|
|
888
|
+
capability). Swapping the copy back cannot fix that, so one plugin is restored
|
|
889
|
+
once per cycle: a second failure reports `needs-human`, touches nothing and asks
|
|
890
|
+
for no restart. One boot without any plugin failure resets the cycle.
|
|
891
|
+
|
|
892
|
+
It is also **idempotent**: before copying, the live directory is compared with the
|
|
893
|
+
backup copy (relative path + size + sha256). When they already match, nothing is
|
|
894
|
+
copied and no restart is asked for (`already-consistent`).
|
|
895
|
+
|
|
896
|
+
**A restart is requested only when files really changed.** `no-backup`,
|
|
897
|
+
`duplicate`, `already-consistent` and `needs-human` all leave `restartRequested:
|
|
898
|
+
false` — a restart that cannot change the outcome only costs another boot. The
|
|
899
|
+
restart goes through the **existing** entry (`requestRestart`, the same gate,
|
|
900
|
+
snapshot and resume path); there is no second restart mechanism.
|
|
901
|
+
|
|
902
|
+
Everything is written to `<stateDir>/last-plugin-recovery.json`:
|
|
903
|
+
|
|
904
|
+
```json
|
|
905
|
+
{
|
|
906
|
+
"at": 1761000900000,
|
|
907
|
+
"atIso": "2026-11-01T01:00:05.000Z",
|
|
908
|
+
"log": "…/logs/startup-…log",
|
|
909
|
+
"backupDir": "…/backups/20261101T000000Z",
|
|
910
|
+
"backupKind": "known-good",
|
|
911
|
+
"restoredAt": 1761000000000,
|
|
912
|
+
"failures": [{ "plugin": "broken-plugin", "error": "Error: Cannot find module 'x'" }],
|
|
913
|
+
"actions": [{
|
|
914
|
+
"plugin": "broken-plugin",
|
|
915
|
+
"mode": "overwrite",
|
|
916
|
+
"restoredFrom": "…/backups/…/node_modules/broken-plugin",
|
|
917
|
+
"restoredAt": 1761000000000,
|
|
918
|
+
"backupKind": "known-good",
|
|
919
|
+
"evidence": { "renamedTo": "…/node_modules/broken-plugin.broken-20261101T010005" }
|
|
920
|
+
}],
|
|
921
|
+
"restartRequested": true,
|
|
922
|
+
"needsHuman": false
|
|
923
|
+
}
|
|
924
|
+
```
|
|
925
|
+
|
|
926
|
+
`restoredAt` and `backupKind` come from the **backup's own `meta.json`**, so the
|
|
927
|
+
answer to "restored to which point in time" is that copy's own record and not
|
|
928
|
+
the moment the clock was read. The file holds facts (when the copy was taken, what
|
|
929
|
+
triggered it, `version`, the content fingerprint, what the boot after it reported);
|
|
930
|
+
`backupKind` is the identity the Host **derives** from those facts — the file does
|
|
931
|
+
not carry a `kind` field any more. The same object is exposed as `pluginRecovery` on
|
|
932
|
+
the health route and by `continuum_status` (`null` / omitted when this machine
|
|
933
|
+
never had such an incident). The reading of the report is not part of this
|
|
934
|
+
change: the interface part is a separate task.
|
|
935
|
+
|
|
676
936
|
|
|
677
937
|
**The disarming is the host's job, and it does it at boot.** As soon as this
|
|
678
938
|
plugin is loaded, it writes the ready marker, removes `watchdog-pending.json` and
|
|
@@ -914,6 +1174,135 @@ once, in the same composer dock, as a small card:
|
|
|
914
1174
|
and records it on `POST /announcements` (`{ feature }`). The `version` in that answer is the feature
|
|
915
1175
|
version — the same number that `SLEEP_FEATURE_VERSION` holds.
|
|
916
1176
|
|
|
1177
|
+
## 0.8.6: what is backed up, the three signals, and the edge of the watchdog
|
|
1178
|
+
|
|
1179
|
+
> This chapter covers the 0.8.6 release notes. The Chinese original of the same chapter is in
|
|
1180
|
+
> [docs/README.zh.md](docs/README.zh.md). It only answers the four questions an ordinary user asks:
|
|
1181
|
+
> **what is backed up, how many signal layers there are, what the watchdog can save, and what to do in
|
|
1182
|
+
> which order when something is off.** Implementation details live in the sections above.
|
|
1183
|
+
|
|
1184
|
+
### 1. What is backed up, and what is not
|
|
1185
|
+
|
|
1186
|
+
| | In the backup | Why |
|
|
1187
|
+
|---|---|---|
|
|
1188
|
+
| **Backed up** | the profile configuration (`cordis.patch.yml`, `package.json`, `pnpm-lock.yaml`) **plus the plugins you wrote yourself and installed as a link (`link:`)** — their contents are kept separately in `node_modules-link/<name>/` — **never their `.git`** (history is rebuildable from its remote, and it was the bulk of every copy: one linked plugin, 962 files / ~74 MB). A sweep also removes directories holding neither a manifest nor a configuration file, at most 32 dirs / 2 GB per pass, and reports every removal. An incomplete copy never carries a backup's name at all: it is written as `.staging-<stamp>` and renamed only once its manifest and configuration are on disk, so an interruption leaves a staging directory instead of something that looks restorable. **The boot sweep does not remove those `.staging-*` leftovers** (it skips every dot-named directory on purpose, so it never deletes a copy that is being written right now) — they are simply never listed and never restored from, and removing them by hand is safe | the plugins you write have **no definite source and no definite version**: delete one and nothing can fetch it again, so a backup is the only way |
|
|
1189
|
+
| **Not backed up** | **third-party plugins** (installed from npm) | they **do have a definite source and version**: `pnpm-lock.yaml` pins every package's version and integrity hash, so reinstalling brings back **the version from the backup, not the latest one** — a backup would be redundant, and reinstalling is more reliable than a copy |
|
|
1190
|
+
|
|
1191
|
+
- Measured (from 2026-11-01 the slim backup is the default): one backup went from 308 MB to about 100 MB.
|
|
1192
|
+
- `backupIncludePlugins: false` → configuration only (about 66 KB, finished instantly). With that setting
|
|
1193
|
+
**a plugin problem cannot be rolled back, it has to be reinstalled by hand**, and the settings page says so.
|
|
1194
|
+
- The old full-copy behaviour is still there: `slimBackup: false`.
|
|
1195
|
+
|
|
1196
|
+
### 2. The three signal layers: two are certain, one is only a clue
|
|
1197
|
+
|
|
1198
|
+
On every boot the plugin compares three layers against the list the previous boot recorded
|
|
1199
|
+
(`plugin-watch.json`):
|
|
1200
|
+
|
|
1201
|
+
- **tool names** (`tools`) — plugins that provide tools (codekg, vault, release-board and the like);
|
|
1202
|
+
- **interface modules** (`clientModules`) — plugins that draw something in the interface;
|
|
1203
|
+
- **registered packages** (`packages`) — packages that register a schema.
|
|
1204
|
+
|
|
1205
|
+
| Signal | Strength | What it proves | What it does not prove |
|
|
1206
|
+
|---|---|---|---|
|
|
1207
|
+
| **a tool name disappeared** | **certain** | these features really did not come up this time | which plugin caused it |
|
|
1208
|
+
| **a registered package disappeared** | **certain** | this package did not load this time | whether the files are wrong or the configuration disabled it |
|
|
1209
|
+
| **a plugin's files differ from the backup** | **clue** | this plugin may have been changed (it may also be your own edit, or an upgrade) | **not the same as broken** — a plugin whose files are all correct and that fails while loading is invisible to this layer |
|
|
1210
|
+
|
|
1211
|
+
Three more limits belong right here:
|
|
1212
|
+
|
|
1213
|
+
- **interface modules that map to no installed plugin** are counted and reported separately
|
|
1214
|
+
("also N interface module(s) did not appear this time"); they are **never attributed to a plugin** —
|
|
1215
|
+
when the owner cannot be identified, no owner is guessed.
|
|
1216
|
+
- **a layer that cannot be read means "no news", never "gone"**: when a host upgrade changes the shape of
|
|
1217
|
+
some API, the only possible answer is "this layer could not be seen this time", never "your plugin is gone".
|
|
1218
|
+
- **the file comparison uses the set of relative paths plus each file's size**, it never hashes a whole tree
|
|
1219
|
+
(a real profile is about 150 MB). So it does find "a file was changed / deleted / is absent from the backup",
|
|
1220
|
+
and it **does not find** a plugin whose files are all correct but which fails while loading (a missing peer
|
|
1221
|
+
dependency, a throwing initialiser, a version the host refuses).
|
|
1222
|
+
|
|
1223
|
+
### 3. "Inconsistent" is not the same as "broken" — the hard wording rule
|
|
1224
|
+
|
|
1225
|
+
The popup and the settings page may only say two things:
|
|
1226
|
+
|
|
1227
|
+
- "**these features did not come up this time**" (a certain fact — it already happened, it is not a guess);
|
|
1228
|
+
- "**these plugins differ from the backup**" (a difference list, with no conclusion drawn from it).
|
|
1229
|
+
|
|
1230
|
+
**Never write "the plugin is broken"**: differing from the backup may simply mean you edited it, or upgraded
|
|
1231
|
+
it. The status word of a row in the settings page is only
|
|
1232
|
+
`fine` / `differs from the backup` / `not in the backup` / `not installed` / `cannot tell (no backup to compare
|
|
1233
|
+
against)` — there is no "broken" state.
|
|
1234
|
+
|
|
1235
|
+
### 4. The watchdog: what it does
|
|
1236
|
+
|
|
1237
|
+
- **Configurable**: `watchdogEnabled`, **on by default**; with it off, a new version that does not start has
|
|
1238
|
+
to be handled by hand.
|
|
1239
|
+
- **It only stands watch inside one window**: a restart that **this plugin** started. After the host is
|
|
1240
|
+
replaced it waits for the new instance and asks **that instance's own** health route (never the old
|
|
1241
|
+
instance, never the old port), then works in a fixed order: **restore the newest backup whose
|
|
1242
|
+
identity is `known-good` → clear the stale `dsh-auth-*` handshake cookies → start the host again →
|
|
1243
|
+
wait for the ready marker**.
|
|
1244
|
+
- **Three possible triggers, one chosen from what this machine can do and reported honestly**: a Windows
|
|
1245
|
+
scheduled task / a systemd user timer / a background process of its own; **with none of the three this
|
|
1246
|
+
machine has no automatic rollback** and the settings page says exactly that.
|
|
1247
|
+
- **Bounded retries**: 3 by default (hard ceiling 10), with a backoff between passes, and the three questions
|
|
1248
|
+
(is it up / is an instance alive / does health answer) are asked again before every start.
|
|
1249
|
+
- **Every attempt reaches the disk**: `<stateDir>/last-watchdog-report.json` — which tier, how many attempts,
|
|
1250
|
+
and per attempt the restore / cookie clear / relaunch / ready, plus why it stopped. A report that never
|
|
1251
|
+
reaches the disk is a report that never happened.
|
|
1252
|
+
- **A restart that goes FINE writes its ending too**: when the new host comes up and the watchdog stands
|
|
1253
|
+
down, the report is updated to say so (`result`/`stage`/`outcome: 'disarmed'`, `hostReady`, `disarmed`,
|
|
1254
|
+
`disarmedAt`, and a `disarm` block). Before that, the file stayed at `armed` forever for a restart that
|
|
1255
|
+
had already finished — while `health` correctly answered `ready:true`. The old fields are kept
|
|
1256
|
+
untouched, and `previousResult` still says what the file said before (usually `armed`).
|
|
1257
|
+
- **The capability probe's tasks are cleaned up**: proving that this machine can create a one-off task
|
|
1258
|
+
means creating one — and deleting it as the probe's last step, which a crash leaves behind (two were
|
|
1259
|
+
found on a real machine). The boot disarm now removes every leftover task whose name starts with this
|
|
1260
|
+
plugin's own probe prefix (`dsh-autorestart-capability-probe-`) and **never** matches anything else;
|
|
1261
|
+
the name reported for a root-folder task carries a leading `\` (`schtasks` prints `\task-name`), so one
|
|
1262
|
+
leading backslash is stripped before comparing — and a name that still contains a `\` afterwards is a task
|
|
1263
|
+
inside a folder and is refused, never deleted;
|
|
1264
|
+
what it removed (and what it could not) is written into the disarm record and the log, and a refusal
|
|
1265
|
+
never prevents the host from coming up.
|
|
1266
|
+
|
|
1267
|
+
### 5. The watchdog's blind spots (say them out loud)
|
|
1268
|
+
|
|
1269
|
+
**What it can save**: **a plugin problem of its own making that keeps DSH from booting** — one snapshot is
|
|
1270
|
+
restored and the host is started again.
|
|
1271
|
+
|
|
1272
|
+
**What it cannot save** (this list matters more than the line above):
|
|
1273
|
+
|
|
1274
|
+
- **the DSH binary itself or the system environment** (a broken host upgrade, missing environment
|
|
1275
|
+
dependencies). It repairs no bug of the host: a host that dies for the same reason every time only ends as
|
|
1276
|
+
`failed`, with a report that names the attempts;
|
|
1277
|
+
- **third-party plugin problems** — it only looks at its own snapshot: it neither diagnoses which plugin
|
|
1278
|
+
caused the failure nor fixes a third-party plugin's bug;
|
|
1279
|
+
- **a full disk, permission problems, changes made by a system update** — restoring a snapshot does not solve
|
|
1280
|
+
any of those;
|
|
1281
|
+
- **everything outside the restart window** — a host killed by the system at 03:00, a client that is
|
|
1282
|
+
start-and-die on a manual double-click, a morning with 0 processes: at that moment **nobody is on watch**
|
|
1283
|
+
(the plugin died with the host and no one-off item was ever registered), and all that is left is
|
|
1284
|
+
`scripts/restore-profile.cmd`, double-clicked by hand;
|
|
1285
|
+
- **`watchdogEnabled: false`, or a machine whose tier is `none`** → no automatic rollback, and the settings
|
|
1286
|
+
page says so.
|
|
1287
|
+
|
|
1288
|
+
### 6. The four steps after something went wrong
|
|
1289
|
+
|
|
1290
|
+
1. **look at the popup** — it appears by itself after a restart and says two things separately: which
|
|
1291
|
+
features did not come up this time, and which plugins' files differ from the backup;
|
|
1292
|
+
2. **or open the "plugins" section in the settings page and press "check now"** — every installed plugin is
|
|
1293
|
+
listed with its current status (`fine` / `differs from the backup` / `not in the backup` / `not installed`
|
|
1294
|
+
/ `cannot tell`);
|
|
1295
|
+
3. **a plugin you wrote yourself → press "restore"** — only **that one** named plugin is put back to the
|
|
1296
|
+
version in the backup: the live directory is renamed to `<name>.broken-<ts>` as evidence (`broken` is just
|
|
1297
|
+
the name of that evidence directory, not a conclusion) and **nothing is ever deleted**; every other plugin
|
|
1298
|
+
and `package.json` / `cordis.patch.yml` stay byte-identical;
|
|
1299
|
+
4. **a third-party plugin → reinstall the version the popup names** — the panel gives the command, and what
|
|
1300
|
+
comes back is **the version recorded in the backup (not the latest one)**; when the backup does not record
|
|
1301
|
+
the version, the panel says so plainly ("it can only be reinstalled by name").
|
|
1302
|
+
|
|
1303
|
+
One precondition behind all four steps: **when there is no usable backup the plugin changes nothing for
|
|
1304
|
+
you** — it says "it has to be handled by hand" instead of pretending it restored something.
|
|
1305
|
+
|
|
917
1306
|
## Graceful shutdown (收工关机)
|
|
918
1307
|
|
|
919
1308
|
**What it does.** One moon button at the very bottom of the left sidebar: press it and the plugin
|