blockyard 0.0.9 → 0.1.0

package/docs/MEASUREMENTS.md

@@ -1252,3 +1252,150 @@ showing STALLED. Block intervals are close to exponential with a 10-minute mean,
 minutes or more has probability e^-4, about 1.8% -- once in fifty blocks, a few times a day. Stalled
 now means a connected peer reports a higher tip (`getpeerinfo` `synced_headers`); peers agreeing on
 the tip is a long gap and synced; no peer height at all waits two hours.
+
+## 32. The DOOM Diversion's emulated PC (2026-09-15)
+
+Not the node: the i386 and the PC in `public/js/x86.js` and `dospc.js` running the shareware
+`DOOM.EXE` v1.9 (the Diversion). Taken on this box -- AMD Ryzen 9 9950X3D, Node v22.23.2, Chromium
+152 (snap, headless). Reproduce the headless figures with `node scripts/dos-bench.js doom`.
+
+**Speed of the interpreter**, 400 M instructions of DOOM's title and demos on a clock of 30 M
+instructions to the virtual second:
+
+| build | instructions a second |
+|---|---|
+| first cut: unsigned values (`>>> 0`), flag operands in closure variables, a try/catch per instruction | 66-68 M |
+| every value an int32, lazy flags in an `Int32Array`, one try/catch around the loop, 32-bit fast paths | 95 M |
+| the same with the Sound Blaster and OPL3 attached, synthesising at 44.1 kHz per call of `tick` | 73 M |
+| the OPL's per-register work hoisted out of the sample loop | 82 M |
+| `tick` working in batches of at least 128 frames (it had been called every 2,000 instructions: a sample or two each) | **92 M** |
+
+The try/catch alone was 14% of the profile, and the collector 1.5% from doubles boxed in closure
+variables; the rewrite was checked instruction by instruction against the first cut for 40 M
+instructions (identical except the start-up's environment read, which the first cut got wrong).
+
+**In the browser**, the machine in a worker, sound on: **94-110 M instructions a second, 34-37
+frames a second** -- DOOM's own cap is 35. The game needs about a million instructions a frame of
+real work; the rest of each tic it spends in its own busy wait for the timer. The page's CSP forbids
+eval, so a JIT was never an option; a slower machine than this one has roughly a factor of two in
+hand before DOOM drops below 35.
+
+**Start-up**: graphics mode after 8.0 M instructions without a sound card, 17.0 M with one (the
+DMX driver probes the DSP and the OPL); 20 pages flipped by 31 M.
+
+**Correctness of the CPU, measured against the host's own**: a differential fuzzer ran random
+instructions natively (a C harness in 64-bit mode, 32-bit operands, register forms) and in `x86.js`,
+comparing all registers and every flag the instruction defines -- ALU rows, immediates, shifts and
+rotates with counts past the width, MUL/IMUL/DIV/IDIV including divide faults, BT/BTS/BTR/BTC,
+BSF/BSR, SHLD/SHRD, MOVZX/MOVSX, SETcc, BSWAP, XADD, CMPXCHG, SAHF/LAHF. Two seeds, 78,565
+instructions, 2,338 divide faults agreed, **zero mismatches**. The harness needs a C compiler, so it
+is not in `npm test`; `test/x86.test.js` keeps a case from each class it covered.
+
+**The music is in tune**: over 10 s of the title music, 178 notes keyed on, 96 of them within
+2.5 cents of equal temperament; the rest spread to +-50 cents, which is DMX's pitch bends.
+
+## 33. Quake on the same PC (2026-09-15)
+
+`QUAKE.EXE` v1.06 (DJGPP, go32 stub) on the emulated PC, this box, Node v22.23.2. Reproduce with
+`node scripts/dos-bench.js quake`.
+
+**Speed**: 77 M instructions a second headless with the Sound Blaster attached, 72-79 in a Chromium
+worker -- lower than DOOM's 90-105 because Quake's code is FPU-heavy (the x87 was 15% of the profile)
+and every memory operand adds a segment base (DJGPP's DS is at its memory block; `ea` was 13%).
+Moving the FPU stack to a Float64Array with typed-array operand conversion and one base addition
+when DS and SS share it took 74 to 76: the interpreter's dispatch is the rest.
+
+**Frame rate**: `+timedemo demo1` on a clock of 74 M instructions to the virtual second -- the
+emulator's own speed -- reported **969 frames in 33.5 seconds, 28.9 fps**, about 2.6 M instructions a
+frame. In the browser, a new game on the start map drew 26 frames a second. Period hardware for
+comparison: a Pentium 90 ran the same demo at 320x200 at roughly that rate.
+
+**Start-up**: graphics mode after 244 M instructions on a 30 M clock (Quake pages its 27 MB heap in
+and times its hardware), twenty screens drawn by 285 M; 2.8 s of wall time headless.
+
+## 34. The decoded-instruction cache, and Quake's view size (2026-09-15)
+
+This box, Node v22.23.2, `node scripts/dos-bench.js doom 400` / `quake 1500` (M instructions a
+second, headless, Sound Blaster attached):
+
+| build | DOOM | Quake |
+|---|---|---|
+| before (the interpreter, §32-33) | 90 | 77 |
+| a closure per decoded instruction | 68 | 69 |
+| int32 decodings in a typed array per page, one switch; a write drops the whole page | 83 | 90 |
+| ...a write clears only the instructions it overlaps (DOOM patches its span drawer each call) | 100 | 90 |
+| ...the FPU decoded too, and the loop keeps the current page between instructions | 107 | 104 |
+| ...no page check after a cached handler | **111** | **108** |
+
+The page-drop row is the one to remember: 217,660 whole-page drops in 300 M instructions of DOOM,
+every one from `mov [eax],ebx` into the constants of its own span routine at 0x12bdaf.
+
+**Quake's view size**, `+viewsize N +timedemo demo1` (969 frames), on a clock set to the emulator's
+speed:
+
+| viewsize | before the cache (76 M clock) | after (105 M clock) |
+|---|---|---|
+| 100 | 29.7 fps | 40.9 fps |
+| 80 (the new default) | 32.5 | 44.7 |
+| 60 | 40.5 | -- |
+
+In a Chromium worker: Quake 99-106 MIPS and 40-42 frames a second in a new game at `viewsize 80`
+(26 before), DOOM 133 MIPS at its 35 fps cap.
+
+**Checked, not assumed**: both games lock-stepped against the uncached interpreter (DOOM 400 M, Quake
+1.5 G instructions; registers and flags compared every 10,000; memory identical at the end), and the
+native fuzzer re-run through `run(1)` over 118k instructions with no mismatch.
+
+## 35. A second pass on the CPU: what paid and what did not (2026-09-15)
+
+Asked for all five of: split the ALU routine, specialise the hot x87 forms, cheaper dispatch, dead
+flags, and a block copy of the frame into video memory. **Method, after the first readings misled:**
+each build against a copy of the previous one (`oracle4`), pinned to one core with `taskset`, best of
+three; unpinned runs on this shared box moved +-5% between identical runs. Quake throughput as M
+instructions a second over its timedemo, and as **frames a second of wall time** over 600 timedemo
+frames -- the second catches work a MIPS figure cannot, such as a `rep movsd` that is one instruction
+however many bytes it moves. DOOM on `-timedemo demo1` only: its normal MIPS depends on how much time
+lands in its cheap wait loop, which a change to the machine's slice size alone moved by 5%.
+
+| build | Quake MIPS | Quake fps (wall) | DOOM timedemo MIPS |
+|---|---|---|---|
+| before (the cache, §34) | 101 | 37.7 | 103 |
+| ALU split per operation + hot x87 forms decoded to their own handlers | 133 | -- | -- |
+| + fused cmp/test+Jcc dispatch and "no flags" forms by flag liveness | 131 | -- | 110 |
+| the same without the look-ahead (fusion and no-flags off) | 133 | -- | 114.5 |
+| + the address formed inline in the loop instead of a call per handler | 141.5 | -- | 112 |
+| + aligned reads/writes inline in the hottest moves | 141 | 52.5 | 115 |
+| the same without the VGA block copy | 141 | 51 (noise) | -- |
+
+**Kept**: the ALU split, the x87 handlers, the inline address and moves -- Quake's frames a second of
+wall time 37.7 -> 52.5 (+40%). **Removed**: the fused branches and the no-flags forms (nothing gained;
+the look-ahead re-ran on every re-decode of DOOM's self-patching drawer), and the block copy (within
+noise: the frame copy is 64,000 bytes against 140 M instructions a second).
+
+The reason inlining mattered: `run()` is one function with 170-odd cases, V8's cumulative inlining
+budget runs out long before the helpers it calls, and every uninlined `eaOf` was a real call.
+
+**In a Chromium worker** (new game, `viewsize 80`): Quake 115-118 MIPS and 39-49 frames a second on
+screen, once the worker looked for a finished frame every 50,000 instructions instead of once per
+10 ms slice (two frame copies inside one slice had been showing as one). DOOM 139 MIPS.
+
+Checked: DOOM 400 M and Quake 1.5 G instructions lock-stepped identical to `oracle4` (memory equal),
+and the native fuzzer through `run(1)`, 78k instructions, no mismatch.
+
+## 36. Wolfenstein 3D in real mode (2026-09-15)
+
+`WOLF3D.EXE` v1.4 (LZEXE-packed, Borland C, real mode) on the emulated PC, this box, Node v22.23.2,
+pinned to one core. It unpacks itself and reaches the sign-on screen in 1.0 s of wall time (60 M
+instructions on a 20 M clock).
+
+**Speed**: in a game, turning on the first map, **65 M instructions a second** headless (twice the same
+reading) and 68-69 in a Chromium worker. Real-mode code runs through the uncached `step()`, so this is
+the plain interpreter's speed; the game needs about **285 k instructions a frame**, so its 70 frames a
+second (the VGA's refresh, which it waits for) take 20 M a second -- a third of what is there, and the
+browser shows 70 frames a second.
+
+**DOOM and Quake unchanged**: best of three against a copy of the previous build (`oracle5`), DOOM
+timedemo 93.3 vs 93.0 M a second, Quake 126.3 vs 127.7. Quake lock-stepped identical for 600 M
+instructions, memory equal. DOOM lock-steps identical to 175 M and then differs, by design: at 173 M it
+copies between VGA pages in write mode 1, which the previous build wrote as plain data, and reads the
+planes back.

package/docs/SECURITY.md

@@ -5,7 +5,7 @@ page describes the access model, what protects it, and exactly what leaves your
 To report a vulnerability, see [SECURITY.md](../SECURITY.md) at the repository root.
 
 - [Threat model in one paragraph](#threat-model-in-one-paragraph)
-- [Access: open by default, accounts on request](#access-open-by-default-accounts-on-request)
+- [Access: sign-in by default, open on request](#access-sign-in-by-default-open-on-request)
 - [Accounts, sessions and passwords](#accounts-sessions-and-passwords)
 - [Talking to the node](#talking-to-the-node)
 - [Node writes](#node-writes)
@@ -26,10 +26,19 @@ monitor leaking information about you to third parties (limited to the on-demand
 connections listed below, which you can turn off). The monitor does not hold keys and has no
 wallet access.
 
-## Access: open by default, accounts on request
+## Access: sign-in by default, open on request
 
-With the default `auth.enabled: false`, anyone who can reach the port reads the monitor
-with the fixed role `viewer`:
+**Out of the box the monitor listens on `127.0.0.1` only and requires sign-in.** The first
+start creates an `admin` account and prints its password once (or takes it from
+`BLOCKYARD_ADMIN_PASSWORD`). Reach it from another machine over an SSH tunnel
+(`ssh -L 21000:127.0.0.1:21000 you@host`), or bind a LAN address with `BLOCKYARD_BIND` /
+`server.hosts` once you have decided who may see it. These are the defaults since
+2026-09-15; 0.0.9 shipped bound to every interface with no sign-in, which the first outside
+review rightly called out.
+
+Open access is still available as a posture you choose: with `auth.enabled: false`
+(`BLOCKYARD_AUTH=0`), anyone who can reach the port reads the monitor with the fixed role
+`viewer`:
 
 | open to anyone who can reach the port | still closed |
 |---|---|
@@ -42,7 +51,7 @@ The `viewer` ceiling cannot be raised by configuration or by any credential whil
 are off. The start-up log states which addresses are readable and how to close them, so
 "anyone on the LAN can read your node" is never a surprise.
 
-Set `BLOCKYARD_AUTH=1` (or `"auth": { "enabled": true }`) to require sign-in. Roles:
+With sign-in on (the default; `BLOCKYARD_AUTH=1` restores it after an override), roles:
 
 | role | may |
 |---|---|
@@ -124,9 +133,15 @@ Every call and every refusal is appended to the audit trail.
 
 ## Transport security
 
-- HTTPS is built in: name a certificate and key and **every** listener serves HTTPS. A
-  half-configured pair or an expired certificate stops the start-up; a certificate close to
-  expiry starts with a warning, and the log prints its fingerprint.
+- HTTPS is the default on **every** listener. With no certificate named, the server makes its
+  own self-signed one on first start (`server/tls/selfsigned.js`, under `<data>/tls/`, the key
+  mode 600, ECDSA P-256), naming the addresses it is reached on, and remakes it when it nears
+  expiry or stops naming a bound address; the log prints its fingerprint. A certificate of your
+  own replaces it (`BLOCKYARD_TLS_CERT`/`_KEY`); a half-configured pair or an expired
+  certificate stops the start-up. `BLOCKYARD_TLS=0` is plain HTTP, for a proxy in front.
+- A self-signed certificate proves nothing about who you are talking to the first time; it
+  does encrypt the session and pins the fingerprint after that. Compare the fingerprint the
+  log prints with the one the browser shows before trusting it on a network you do not own.
 - Over HTTPS the session cookie is `Secure` and `Strict-Transport-Security` is sent with a
   two-day lifetime, without `includeSubDomains` or `preload` — a LAN address can be reissued,
   and HSTS cannot be withdrawn once a browser has it.
@@ -150,13 +165,15 @@ Everything in the monitor talks only to your node, **except**:
 
 | when | to | what is sent |
 |---|---|---|
-| while someone has the **Markets**, **Kiosk** or **Overview** tab open — Overview's price line is **on by default**, so the landing page reaches out unless you switch it off — and for 10 minutes after the last request | `api.exchange.coinbase.com`, `api.kraken.com`, `www.bitstamp.net`, `api-pub.bitfinex.com`, `www.okx.com` (HTTPS) | public ticker, hourly candle and order-book requests with a `User-Agent` naming the software — nothing about your node |
-| when someone opens an **explorer** page and no fresh market price is at hand | two of the exchanges above, at most once a minute | a public ticker request |
+| only with **Enable market polling** ticked in Display settings (**off by default**), while someone has the **Markets**, **Kiosk** or **Overview** tab open — Overview's price line reads the same feed unless you switch it off — and for 10 minutes after the last request | `api.exchange.coinbase.com`, `api.kraken.com`, `www.bitstamp.net`, `api-pub.bitfinex.com`, `www.okx.com` (HTTPS) | public ticker, hourly candle and order-book requests with a `User-Agent` naming the software — nothing about your node |
+| with polling on, when someone opens an **explorer** page or the **Mining** tab and no fresh market price is at hand | two of the exchanges above, at most once a minute | a public ticker request |
 
-Nothing is fetched when nobody is looking. Your machine's public address is visible to those
-exchanges when a request is made, as with any web request. To make **no** outbound connections
-at all, set `BLOCKYARD_MARKETS=0` (or `"markets": { "enabled": false }`): the Markets and Kiosk
-tabs then say that market data is off, and the explorer shows no dollar figures.
+**Out of the box none of this happens**: polling is off until someone ticks **Display settings → Markets & Price → Enable market polling**, and until then the
+Markets and Kiosk tabs say so and the explorer shows no dollar figures. The switch is a Display
+setting shared by every screen, so anyone who can change settings on this monitor can turn it
+on; `BLOCKYARD_MARKETS=0` (or `"markets": { "enabled": false }`) removes the feed from the
+server so that no checkbox can. With polling on, nothing is fetched when nobody is looking. Your machine's public address is visible to those exchanges
+when a request is made, as with any web request.
 
 The monitor sends no telemetry, checks for no updates, and phones home to no one.
 
@@ -190,6 +207,6 @@ and errors; they never contain passwords, session tokens or RPC credentials.
 - Turn accounts on if anyone who can reach the port should not see your node.
 - Use HTTPS, a reverse proxy, or an SSH tunnel on untrusted networks.
 - Leave node actions off unless you have a specific need, and then enable only that action.
-- Set `BLOCKYARD_MARKETS=0` on machines that must not make outbound connections.
+- Set `BLOCKYARD_MARKETS=0` on machines that must not make outbound connections: the polling checkbox (off by default) then cannot turn the feed on.
 - Keep `config/local.json` and `data/` readable only by the service account.
 - Keep Node.js current within the 22.x line or later.

package/docs/TROUBLESHOOTING.md

@@ -54,6 +54,10 @@ The Node & RPC page shows the last error. Common causes:
 
 ## The page does not load from another machine
 
+0. Out of the box the monitor binds `127.0.0.1` and answers this machine only. Either reach
+   it over an SSH tunnel (`ssh -L 21000:127.0.0.1:21000 you@host`, then `https://localhost:21000`)
+   or bind a LAN address: `BLOCKYARD_BIND=192.0.2.10` (or `0.0.0.0`), or `server.hosts` in
+   `config/local.json`, and restart.
 1. From the other machine, check how it reaches the server: `ip route get <address>`.
 2. Make sure that address is one the monitor binds — see the start-up log. A LAN-only bind is
    not reachable over a VPN unless the VPN address is also listed in `server.host`.
@@ -159,6 +163,14 @@ so far and an ETA; the address page repeats it. Things it says, and what they me
   [INSTALL](INSTALL.md#bitcoinconf-settings-worth-having), or let it run overnight.
 - **The ETA is wrong at first** — it is computed from the files done so far in the current phase
   and settles after the first few; files are not all the same size.
+- **It stopped one short — "scan 5,720 of 5,721, about 1 s left" for an hour** — a worker
+  died, most likely killed for memory (four workers is about 10 GB beside the node). Since
+  2026-09-15 that fails the build at once with `an index worker exited with code N while on
+  {"type":"scan","file":...}` and the flag turns to *build failed*; before that fix the build
+  hung there for good. Either way: restart BlockYard with fewer workers (`addressIndexWorkers` on
+  the node entry in `config/local.json`). The build starts over; it is not resumable. The flag
+  also says **no progress for N min** whenever nothing has moved for two minutes, so a stall is
+  visible as one rather than as a stale ETA.
 - **Hours, not minutes** — expect **a few hours**: 29 min 45 s is 16 workers on NVMe, and four
   workers (the installer's default) are roughly four times slower; **spinning disks** are slower still whatever the
   number, and there one worker is the fast setting, because parallel readers only seek against
@@ -171,9 +183,25 @@ so far and an ETA; the address page repeats it. Things it says, and what they me
   restart (the server builds again), or run `node scripts/index-build.js --out <dir>` by hand.
   A pruned node, unreadable block files and a full disk are the usual causes.
 
+## "getrawmempool verbose dropped as stale" in Events, and the mempool panels look old
+
+The monitor keeps one RPC request in flight and serves the live polls first; the full-pool poll is
+the lowest priority, so when the node's RPC is slow it waits behind them and, past its freshness
+budget, is dropped rather than shown as current. A streak of drops is one warning event when it
+starts, a counter on **Node & RPC → data quality** while it lasts, and one event when the poll
+answers again with the count and the span. The Mempool, Block space and Mining panels show their
+last reading meanwhile and say how old it is.
+
+The cause is the node, not the monitor: look for what else is asking it. On 2026-09-15 an Umbrel
+node answered in seconds for thirteen hours while another BlockYard built its address index
+against it over the LAN, and the drops stopped the moment that build finished. A remote index
+build, a wallet rescan, `gettxoutsetinfo` from another tool, or an initial block download all
+show the same way.
+
 ## Markets or Kiosk show no prices
 
-- **"market data is off"** — `BLOCKYARD_MARKETS=0` or `markets.enabled: false` is set.
+- **"market polling is off"** — the default. Tick **Display settings → Markets & Price → Enable market polling**.
+- **"market data is off on this server"** — `BLOCKYARD_MARKETS=0` or `markets.enabled: false` is set on the server; the checkbox cannot override it.
 - **"asking the exchanges…" for a long time** — the server cannot reach the exchanges. Test
   from the server: `curl -sI https://api.exchange.coinbase.com/products/BTC-USD/ticker`. Check
   outbound firewall rules and DNS.
@@ -289,6 +317,12 @@ browser in kiosk mode pointed at `http://<host>:21000/#kiosk`.
 - **"too many attempts"** — the lockout lasts 10 minutes per username and per address.
 - **Sign-in does not stick** — over plain HTTP, make sure `BLOCKYARD_SECURE_COOKIE` is not set
   (a `Secure` cookie is never sent over HTTP). Behind a TLS proxy, set it.
+- **The browser warns about the certificate** — expected once per address: the monitor's own
+  certificate is self-signed. Compare the fingerprint in the start-up log with the browser's,
+  then accept it. To be rid of the warning, name a certificate of your own (INSTALL §9) or put
+  a proxy with a real one in front (§10).
+- **`https://` says the connection was reset, or `http://` shows nothing** — the port speaks
+  one or the other: HTTPS unless `BLOCKYARD_TLS=0`. Use the scheme the start-up log prints.
 
 ## Tests fail on a fresh clone