@mnstry/atelier 0.2.0-alpha.12 → 0.2.0-alpha.14

package/docs/obsidian-plugin.md

@@ -32,7 +32,24 @@ is `src/projection/obsidian/plugin-bridge/channel.mjs`; the service's half is
   a window with the view, its state and the reason in words, the committed and
   the prepared generation, when freshness was last checked, the held, retained
   and pending edits, the service address and state, and the plugin and app
-  versions.
+  versions. While the service is unreachable, current generations, check time
+  and edit counts are unknown. After a connection refusal or timeout on the
+  same channel, the window also shows the last authenticated state, reason,
+  generation and check under "Last observed" labels. Those rows are historical
+  and never establish current freshness, service health or permission.
+  History stays only in memory. It is cleared on a channel change, missing or
+  unsupported setup, authentication refusal, an untrusted answer, a missing or
+  malformed view, a report for another scope, a disabled view or unload.
+  A connection reset or any other failure outside connection refusal and
+  timeout also clears it. Without a prior usable report, "Last observed
+  status" says "not available". After startup admits the channel, a republish
+  that names a different channel (host, port, scope or key) resets the display
+  to `connecting` before its next answer; this reset does not repeat an
+  unchanged notice or lose the transition back to current. A republish of the
+  same channel keeps the display and history, and unusable data shows
+  `not set up`.
+  Republish keeps an unsupported-app or unavailable-Node setup label and its
+  channel closed. Plugin 1.1.2 adds this status continuity.
 - A notice appears when the view moves into a state that needs a person (held,
   stale, service unreachable, app too old) and when it is current again.
   `updating` is not announced: it is what every change looks like.
@@ -104,18 +121,48 @@ at the maintenance service's next tick, with nothing changed at the view's
 sources: the service compares the plugin files each committed generation pins
 with the disk, has a view whose files differ prepared again, and a generation
 that is already committed is published again, as it is, with its notes kept.
-A drift left for the person is asked about once, and again once it changes;
+A drift left for the person is asked about once, and again once it changes.
+What a publication could not change is not asked about again: a drift that
+publishing cannot repair (the data file of a vault root that is a link, say)
+is asked about at most once more after a publication that left it, which
+writes no vault file (the view is prepared again and takes the vault's lock,
+and for that tick shows as `stale`, then `updating`, before it is `current`
+again), and not again after that. A plugin file
+another writer removes or changes after the publication, even while the same
+tick goes on, unless it holds again what it held before the tick under the
+same pin and vault root, and a path
+the person repairs before the tick ends (the vault root made private,
+say), is asked about at the next look and written;
 where the plugin is turned off in the vault, a pinned file that is gone is no
 drift. Publishing a committed generation again needs the app when one runs;
 if that app does not qualify, nothing is written, the view stays `current`
 (a committed generation needs no app, so an app that cannot be qualified
 never makes a current view stale), and the file is written at a later
 attempt, on the retry schedule of an unsettled view, once the app
-qualifies. (The settings files are held to the same rule for a file nobody
+qualifies. A drift that publishing would only leave for the person again
+(the data file of a vault root that is a link or not private, a folder where
+a file goes) is no reason to publish again and needs no app: the path is
+left as it is, and the view keeps its plain reason. Nothing in `status` or
+`open` names such a path yet; without its data file the plugin cannot reach
+the service, so it shows as not present (`no-live-lease`). A few paths the
+publisher leaves for the person still count as a drift here, and wait for the
+app as one: a file nobody may read, a parent folder that cannot be looked
+into, and a vault root Atelier keeps whose mode cannot be changed. Until the app qualifies, a
+view waiting for it has the reason
+`verified-by-read-back-plugin-waits-for-app` rather than
+`verified-by-read-back`, and `status` and `open` add `plugin files wait for
+the app` to the plugin line (`files: "waits-for-app"` in `--json`), with a
+next step for the plugin. (The settings files are held to the same rule for a file nobody
 may read.) A plugin file that another program changes while a publication runs
 is a race and is handled like one on a note: that publication does not
 commit, the view is tried again, and the file is planned from the disk anew
 (`plugin-file-changed`); nothing reads a plugin file as an edit of a note.
+The view is then `publisher-conflict`, and its next step in `status` and
+`open` names that file as what stopped the publication (changed by another
+program or by the person; one change is enough), rather than asking the
+person to close another publisher; the same holds for a settings file
+(`settings-changed`). The reason is the first unit that stopped the
+publication, so other causes are not ruled out.
 Upgrades replace the plugin files the same way. The plugin's version in
 `manifest.json` changes whenever its code does, and the test suite holds the
 two together: the app keeps running the `main.js` it loaded until it reloads
@@ -306,6 +353,124 @@ the vault's key without sending it, and the service proves it first
 | `POST /plugin/release` | `sessionId`, `counter`, `mac` | sealed: whether the session was released |
 | `POST /plugin/status` | `sessionId`, `counter`, `mac` | sealed: the view's state, reason, verification, committed and prepared generation, held notes (counted, not named), retained edits, open pending edits, and the service's own state |
 
+A status answer carries the view and its pending edits only while the project
+allows that view at the moment of the request, not as of the last tick
+(`src/runtime/obsidian/view-permission.mjs`): its Obsidian settings are
+enabled and declare the view, and its workspace pointer still names the
+workspace this service runs for, with its private state where the service
+keeps it. Otherwise the answer is still sealed and still carries the service's
+own state, but `view` and `pendingEdits` are `null`, which the plugin already
+reads as nothing known about the view. That covers a view turned off or taken
+out of the settings before the next tick, and the freshness a later tick
+writes for it, which keeps its generations and counts. It also covers anything
+that cannot be established: settings that refuse, a project that does not
+load, a pointer that cannot be read. The plugin then shows `Atelier: stale`
+with the reason "no freshness recorded", keeps its session and asks again at
+its next round; it is not told why, and it remembers nothing of the view to
+show later.
+
+Loading the project runs Git in child processes, which would otherwise happen
+every two seconds for every open vault, so the project is kept:
+
+- **What decides it.** The files the engine observes as configuration,
+  compared at every request by a digest of what they hold, never by their
+  times. Each file holds where it really is and the bytes there: its path
+  resolved as the system resolves it when the loader opens it (`realpath(3)`,
+  through a link anywhere on the way, a `..` after a link included), so a
+  configuration kept in a dotfiles repository and linked into the project is
+  kept like any other, and a link pointed elsewhere, the file's own or one to
+  a folder above it, is a change even where every byte is the same. A folder
+  in a file's place, a link that leads nowhere or to a folder, and a file
+  that cannot be read vouch for nothing: then nothing is kept, and the
+  project is loaded at every request. Where the system resolves no path for a
+  regular file that is there and is not a link (some volumes on Windows
+  answer `EISDIR`; a C library that needs `/proc` and has none), the file is
+  compared by its own path as given and its bytes, so the project is still
+  kept; a link above such a file, pointed elsewhere, is not seen there. A
+  link the system does not resolve vouches for nothing. The pointer and the
+  place of the private state are read at every request.
+- **When it is kept.** Two loads in a row, each started after those files
+  were read and found to hold the same, must return the same project as far
+  as the decision reads one: the views it enables, where its pointer is, the
+  data root its local overlay prefers and the roots it protects. A field that
+  differs from load to load, such as a time or a personal-workspace binding
+  kept under a symbol key, does not keep a project from being kept. One
+  load is not enough: a file changed and changed back while it ran leaves the
+  same bytes around a project read from other ones. So three loads keep the
+  project when the service has just started, and two after a change.
+- **For how long.** A kept project is loaded again once it is two and a half
+  minutes old, and is never used once it is five, the interval of the
+  engine's full reconciliation. Whatever no file here names (an overlay named
+  by `MNSTRY_ATELIER_LOCAL_CONFIG` that did not exist at the last load, for
+  instance) is seen by then.
+- **A loader that answers a promise** is never waited for, because the plugin
+  gives a request a second and a half. Its view is told nothing until two of
+  its loads have agreed; one load runs at a time, and a kept project goes on
+  deciding, within its time, while it is loaded again. A load that has not
+  answered within two and a half minutes is given up and its later answer
+  ignored, whether a request or that answer is the first to come after the
+  deadline. A service bound to a personal home (`--personal-home`) has such a
+  loader: it starts from the home alone, and its plugins are told about a view
+  once two of the loads that compose the confirmed generation have agreed. A
+  service given only a loader that answers a promise still refuses to start
+  (see "Personal workspaces" in [obsidian.md](obsidian.md)).
+- **A load that fails** (it throws, it is refused, it is given up, or a file
+  vouches for nothing under a loader that answers a promise) ends whatever
+  was kept, and is asked for again after two seconds, then four, up to
+  thirty, while the files hold the same. A change to them is tried at once,
+  with two exceptions. Until a load has answered and named the files, a
+  corrected configuration waits for the next try. And while one of the files
+  vouches for nothing there is no digest to change, so after a load that
+  failed then (one that answers a promise, or one that answers at once and
+  threw) a change to another file waits for the next try as well.
+- **What the service shows of these loads.** Its status document counts
+  them (`statusLoads`: `answered`, `failed`, `givenUp`, `slow`, `notKept` and
+  `longestMs`). Its log gets a `status-project-loaded` line, with the
+  milliseconds, the outcome, flags and counts and never a path, only for a
+  load somebody has to see, each kind on its own row, written when the row is
+  1, 2, 4, 8 and so on long (from 2 for loads after which nothing is kept):
+  - a load that failed or was given up;
+  - a load that answered at once and took a second or more (`slow`). A second
+    is most of the second and a half the plugin gives a request, spent on the
+    listener's own event loop; a load that answers a promise holds nothing;
+  - a load after which nothing is kept although it named the files it
+    started over (`notKept`, from the second in a row, with `vouched` false
+    while a file vouches for nothing): a load at every request. A load that
+    named other files than it started over (the first, which started over
+    none, or the one after a change that names other files, such as a new
+    confirmed selection) neither counts nor ends that row, so loads that
+    each name other files than the one before are never counted. A start and
+    a change each have one load that names the files it started over and has
+    nothing to agree with yet, so neither writes such a line; each is
+    counted, so a healthy service shows one `notKept` at its start and one
+    after each change. A change that lands while a load reads the files,
+    names the same files, and changes the views or what the workspace is
+    resolved from, makes that load keep nothing as well, and so writes one
+    line and counts two. Any other change that lands then counts one, as a
+    change does, and writes nothing.
+
+  A row of failures goes on through loads that answer between them, and ends
+  at the first load that answers a minute or more after its last failure; a
+  line is written then (`afterFailures`). That line comes with the first load
+  after the minute, which under a kept project is the next reload, two and a
+  half minutes after the load that kept it, and none comes if no load
+  follows. A row of slow
+  loads ends the same way, at a load that answers in time a minute or more
+  after the last slow one. So, while loads come less than a minute apart (a
+  load at every request, or the retries after a failure), a loader that
+  fails, or is slow, every other load or every third keeps one row, written
+  less and less often, instead of a line per load. While a project is kept it
+  is loaded again every two and a half minutes, so there a row of failures
+  ends at the next reload that answers: a loader that answers at least one
+  reload between two failures writes each failure and the load that ends its
+  row, as before, and one that fails at every reload and answers only at its
+  retry keeps one row. Each slow load is written there, as before. A row
+  of loads after which nothing is kept ends at the next load that keeps the
+  project and named the files it started over. A healthy service writes none
+  of these lines. The log is cut over only when it is opened, so a line per
+  load, every two seconds while a file vouches for nothing, would have grown
+  it without bound.
+
 Refusals, before anything else is looked at: `Host` other than the listener's
 literal loopback authority, a cross-site `Origin` or `Sec-Fetch-Site`, a path
 other than one of the nine exactly (a query included), or another method (the
@@ -384,7 +549,11 @@ is authenticated, so they only ever decide what is shown.
   directory changed) and one HMAC per vault; at most four handshakes wait per
   view, and recorded challenges can keep them filled for at most about
   thirty seconds after a restart, since a challenge is answered only within
-  thirty seconds of the time it names.
+  thirty seconds of the time it names. A status request of a session costs a
+  read of the project's configuration files and of its workspace pointer; the
+  project itself is loaded again when one of those files changed, every two
+  and a half minutes, or at every request while one of them vouches for
+  nothing (see "The channel").
 - Egress. `plugins/` is in the egress scan, which fails a request whose target
   is not a literal loopback address, and the release audit requires the three
   plugin files in the package and scans them in the tarball.
@@ -423,10 +592,14 @@ answer decides wherever it gives one:
   and opens by path the vault the app lists. Where the settings file does
   not show it (a sandboxed build, say), or where it lists it and, once the
   vault is opened, the command line still does not answer for it while only
-  the plugin reports the version, `open` answers at once
+  the plugin reports the version, `open` answers
   `app-cli-unavailable` / `vault-open-cli-silent`: the plugin shows the vault
   open, so the command line is what did not answer, and waiting for the app
-  would change nothing.
+  would change nothing. After a launch that answer is given at the second
+  such round in a row, not the first: one tool call that timed out while the
+  app was busy opening its window looks the same once, and `open` then goes
+  on waiting for the vault. A wait that ends right after one such round gives
+  the same answer.
 - Whether an app runs is the process table's answer, which says no only when
   it finds no Obsidian process at all. A lease outlives its app by up to the
   lease time (six seconds after a crash), and any process that holds the
@@ -454,6 +627,72 @@ note open in the editor.
 
 ## Evidence
 
+- `test/obsidian-plugin-status-continuity.test.mjs` exercises transient history
+  and the shipped request flow through a finite, socket-free `http.request`
+  stand-in. Canonical channel computations seal the invented replies. It
+  checks refused and timed-out lease/status requests, rejected leases followed
+  by interrupted handshakes, listener proof and answer authentication,
+  republished data, notice deduplication and recovery, and late answers after
+  a channel change or unload. It opens no service or app and writes no plugin
+  data. These tests complement the real-listener and real-app checks below.
+- `test/obsidian-view-permission.test.mjs` asks whether the project allows a
+  view the way the service asks before a status answer, over real project
+  files, the real loader and the real workspace resolution, with no listener,
+  with time a number the test moves: every way a view or its
+  workspace is withdrawn and given back; the workspace's identity compared as
+  well as its place; two loads per change however often a view is asked
+  about; a change that keeps a file's size and modification time; a change
+  that lands during a load; a file the project comes to name; a configuration
+  file that is a link, retargeted, or a link to a link; a folder, a link that
+  leads nowhere and a link to a folder in a file's place; a kept project's
+  time; a loader that fails, with its backoff; and a loader that answers a
+  promise: one load at a time, a change undone and done again while it runs,
+  through a link, a load that never answers, one whose answer comes after its
+  deadline before any request, and one refused again and again; a link whose
+  target has a `..` after a link, and a link to a folder above a file pointed
+  at a byte-for-byte copy; two loads that differ in a time and a symbol-keyed
+  binding, or in a protected root; how long each load took and how it ended;
+  and the service refusing to start on a loader that answers a promise, a
+  refused promise or a thenable. Mutations of the module (a link that vouches
+  for nothing, a project kept from one load whose files are read again when
+  it answers, no deadline, no backoff, no identity check, the files not taken
+  from the load, no time limit, a late answer taken, JavaScript's realpath, no
+  real path for a regular file, the whole project compared, no protected
+  roots compared) and a service that does not refuse each fail at least one
+  of them. It also covers: a late refusal after the deadline; which loads the
+  log report writes (a row of failures or of slow loads kept through loads in
+  time between them until a minute without one, so a loader that fails every
+  other or every third load writes one row; loads after which nothing is
+  kept, from the second in a row, and loads that named other files, which
+  neither count nor end that row) and that a line holds no path; a load at
+  every request written while a file vouches for nothing and while loads
+  never agree, and none at a change that names other files; each input of the
+  workspace resolution (pointer folder, data-root preference, protected
+  repositories) keeping two loads apart; a system that resolves no path, for
+  a regular file and for a link; and, through the test service entry in a
+  child process, a start under `--startup` on a promise, a refused promise,
+  a thenable (its `then` called once) and a refused promise with a `then` of
+  its own that throws (never called), which asks the loader once, records the
+  refusal and ends with 0; and one on a loader that throws `undefined`, a
+  value with no prototype or one whose `code` cannot be read, or answers a
+  project whose `then` cannot be read, which logs `untyped-error` with the
+  text of what was thrown (its message, or the value itself, `undefined` for
+  `undefined`, and `unprintable` when it cannot be made text) and ends
+  with 1. `test/obsidian-personal-selection.test.mjs` covers a locator that
+  answers a thenable (its `then` called once) or throws `undefined`. Ten more
+  mutations (the loader asked twice at such a start, its
+  promise not let go, every load or every failure written, no fallback for a
+  regular file or one for a link too, a late refusal not given up, and each
+  of the three inputs dropped) each fail at least one case, and so do a row
+  of failures or of slow loads ended at any load in time, a load after which
+  nothing is kept written from the first in a row, the first load counted
+  among those, a kept project that does not end that row, a load that named
+  other files counted or ending it, and the files taken as vouching when they
+  did not.
+  `test/obsidian-personal-workspace.test.mjs` keeps a bound personal
+  workspace from the loader that composes in a worker: its views once two
+  compositions agree, nothing while its generation is refused, and the next
+  generation's views once that is bound.
 - `test/obsidian-plugin.test.mjs` runs the shipped `main.js` in a stand-in app
   against a real listener on an ephemeral loopback port: parity of the
   plugin's constants and of every proof and MAC with the channel contract (a
@@ -483,7 +722,13 @@ note open in the editor.
   drift left for the person asked about once),
   and an upgrade; the service publishing the plugin and the plugin it
   published holding the view; qualification from a plugin report, with one
-  launch or two; and `status` and `open` reporting it. A spawn guard refuses
+  launch or two; `status` and `open` reporting it; a view the project turns
+  off or no longer declares, told nothing of what the service stored about
+  it, before the next tick and after it, and told as before once it is
+  allowed again; a project whose pointer is pointed at another workspace,
+  told nothing until it names this one again; and a loader that answers a
+  promise, never waited for, with the plugin keeping its session meanwhile.
+  A spawn guard refuses
   any child of the suite that could reach an Obsidian with the developer's
   own `HOME`.
 - The same file has three real-app tests, skipped unless