blockyard 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +929 -0
- package/LICENSE +202 -0
- package/NOTICE +4 -0
- package/README.md +191 -4
- package/SECURITY.md +38 -0
- package/bin/blockyard.js +41 -0
- package/config/pool-map.json +2620 -0
- package/docs/API.md +1577 -0
- package/docs/ARCHITECTURE.md +1394 -0
- package/docs/AUTO-UPDATE.md +269 -0
- package/docs/CONFIGURATION.md +847 -0
- package/docs/DEFECTS.md +813 -0
- package/docs/EFFECTS-AGENTS.md +448 -0
- package/docs/GETTING-STARTED.md +205 -0
- package/docs/INSTALL.md +547 -0
- package/docs/MEASUREMENTS.md +1401 -0
- package/docs/RULES.md +681 -0
- package/docs/SECURITY-AUDIT-2026-09-14.md +177 -0
- package/docs/SECURITY-AUDIT.md +258 -0
- package/docs/SECURITY.md +212 -0
- package/docs/TROUBLESHOOTING.md +332 -0
- package/docs/USER-GUIDE.md +1262 -0
- package/package.json +53 -5
- package/public/404.html +9 -0
- package/public/css/app.css +2009 -0
- package/public/donate-qr.png +0 -0
- package/public/index.html +1085 -0
- package/public/js/about.js +112 -0
- package/public/js/agents.js +1141 -0
- package/public/js/app.js +1386 -0
- package/public/js/arkanoid.js +806 -0
- package/public/js/blockanoid.js +347 -0
- package/public/js/blockout.js +347 -0
- package/public/js/blockpack.js +428 -0
- package/public/js/blockscene3d.js +2830 -0
- package/public/js/breakout.js +224 -0
- package/public/js/charts.js +635 -0
- package/public/js/depthchart.js +315 -0
- package/public/js/details3d.js +4342 -0
- package/public/js/doom.js +31 -0
- package/public/js/dosaudio.js +48 -0
- package/public/js/dosgame.js +389 -0
- package/public/js/dosio.js +186 -0
- package/public/js/dospc.js +1353 -0
- package/public/js/dosworker.js +196 -0
- package/public/js/explorer.js +405 -0
- package/public/js/feepalette.js +149 -0
- package/public/js/fmt.js +162 -0
- package/public/js/goggles.js +886 -0
- package/public/js/kiosk.js +41 -0
- package/public/js/login.js +88 -0
- package/public/js/markets.js +395 -0
- package/public/js/mining.js +1416 -0
- package/public/js/panels.js +970 -0
- package/public/js/pricechart.js +189 -0
- package/public/js/quake.js +20 -0
- package/public/js/settings.js +1096 -0
- package/public/js/soundcard.js +459 -0
- package/public/js/tetris.js +226 -0
- package/public/js/tetrust.js +356 -0
- package/public/js/tetsound.js +175 -0
- package/public/js/theme.js +235 -0
- package/public/js/wolf3d.js +22 -0
- package/public/js/x86.js +1978 -0
- package/public/login.html +33 -0
- package/scripts/blockfile-measure.js +156 -0
- package/scripts/browser-check.mjs +286 -0
- package/scripts/check.js +173 -0
- package/scripts/decode-check.js +81 -0
- package/scripts/doc-counts.js +109 -0
- package/scripts/donate-qr.py +23 -0
- package/scripts/dos-bench.js +56 -0
- package/scripts/fake-node.js +534 -0
- package/scripts/index-bench.js +216 -0
- package/scripts/index-benchmark.js +117 -0
- package/scripts/index-build.js +40 -0
- package/scripts/live-render-check.mjs +89 -0
- package/scripts/manage-users.js +132 -0
- package/scripts/motion-check.mjs +138 -0
- package/scripts/pool-map.js +157 -0
- package/scripts/setup.js +432 -0
- package/scripts/shots.mjs +278 -0
- package/scripts/smoke.sh +327 -0
- package/scripts/tls.js +31 -0
- package/scripts/ui.js +174 -0
- package/server/auth/sessions.js +221 -0
- package/server/auth/users.js +243 -0
- package/server/chain/blockfile.js +234 -0
- package/server/chain/index/build.js +210 -0
- package/server/chain/index/heights.js +36 -0
- package/server/chain/index/live.js +276 -0
- package/server/chain/index/rows.js +145 -0
- package/server/chain/index/store.js +154 -0
- package/server/chain/index/worker.js +109 -0
- package/server/chain/tx.js +310 -0
- package/server/collect/gbt.js +229 -0
- package/server/collect/logparse.js +765 -0
- package/server/collect/logtail.js +189 -0
- package/server/collect/markets.js +333 -0
- package/server/collect/mining.js +333 -0
- package/server/collect/monitor.js +2545 -0
- package/server/collect/network.js +295 -0
- package/server/collect/nextblock.js +275 -0
- package/server/collect/sync.js +386 -0
- package/server/config.js +644 -0
- package/server/http/api.js +1319 -0
- package/server/http/explorer.js +418 -0
- package/server/http/games.js +77 -0
- package/server/http/server.js +420 -0
- package/server/http/sse.js +176 -0
- package/server/http/static.js +212 -0
- package/server/main.js +673 -0
- package/server/netinfo.js +253 -0
- package/server/rpc/allowlist.js +130 -0
- package/server/rpc/client.js +414 -0
- package/server/store/audit.js +148 -0
- package/server/store/history.js +220 -0
- package/server/store/ledger.js +290 -0
- package/server/store/ring.js +173 -0
- package/server/tls/selfsigned.js +160 -0
- package/server/util/fmt.js +29 -0
- package/systemd/blockyard.service +102 -0
package/docs/DEFECTS.md
ADDED
|
@@ -0,0 +1,813 @@
|
|
|
1
|
+
# Defect and gap register
|
|
2
|
+
|
|
3
|
+
Open items, honestly stated. Checked items keep their entry so the reasoning is
|
|
4
|
+
not re-litigated.
|
|
5
|
+
|
|
6
|
+
**Scope, settled 2026-09-14.** BlockYard supports **Bitcoin Core, on the machine that runs it**:
|
|
7
|
+
the explorer's address index is built from the node's block files, so the monitor lives next
|
|
8
|
+
to the node. Two things this register was written against are therefore no longer supported and
|
|
9
|
+
no longer defects here: the **experimental node** whose RPC surface, log grammar and counters
|
|
10
|
+
several entries below were measured on (its measurements are kept, as records), and **reading a
|
|
11
|
+
node elsewhere over RPC alone** -- a node appliance on the LAN, tried on 2026-09-13 and dropped,
|
|
12
|
+
because real-time explorer data over RPC was a failed idea (INSTALL, "It runs on the node's
|
|
13
|
+
machine"). Entries whose only subject was one of those are closed below with that reason, and
|
|
14
|
+
the record says which.
|
|
15
|
+
|
|
16
|
+
## Done since the first draft
|
|
17
|
+
|
|
18
|
+
Kept as checked rather than deleted, so nobody re-derives them.
|
|
19
|
+
|
|
20
|
+
- [x] **`public/js/panels.js` written** — all eight pages render (`renderChain`,
|
|
21
|
+
`renderMempool`, `renderPeers`, `renderNetwork`, `renderLogs`, `renderNode`,
|
|
22
|
+
`renderAdmin`, `ensureLogsLoaded`).
|
|
23
|
+
- [x] **`login.html` + `js/login.js` + `404.html` exist.** The login script is a
|
|
24
|
+
separate file because the page's CSP is `script-src 'self'`: an inline script
|
|
25
|
+
would not run, and "the button does nothing" is a miserable bug to chase.
|
|
26
|
+
- [x] **git repo, `.gitignore`, and the private GitHub remote.** `data/` holds
|
|
27
|
+
`users.json` (scrypt hashes + salts), `sessions.json` (hashed session tokens)
|
|
28
|
+
and `audit.jsonl`; verified after pushing that `data/` is absent from the
|
|
29
|
+
remote tree entirely.
|
|
30
|
+
- [x] **`README.md` written** for humans; `AGENTS.md` stays the agent resume point.
|
|
31
|
+
- [x] **404 answers 404.** It was answering HTTP 200 with a page body, which makes
|
|
32
|
+
every crawler and uptime probe think the guessed path was correct.
|
|
33
|
+
- [x] **The bench node was being tailed from the wrong file.** `logFile` pointed at
|
|
34
|
+
`<datadir>/main/debug.log`, which is 144 bytes (three `node start` lines) and had
|
|
35
|
+
not moved since boot, while the node wrote to `console.log` in its run directory.
|
|
36
|
+
The monitor held the stub open on fd 22 for two hours and every log-derived panel
|
|
37
|
+
of that node sat still without a single warning. Verified with `ls -l /proc/<pid>/fd`
|
|
38
|
+
before and after. Now repointed, and `log-silent` exists so the next one speaks up.
|
|
39
|
+
- [x] **Log grammar drift, measured and covered.** The bench build of 2026-09-08
|
|
40
|
+
03:02 rewrote `[dlc]`: 30 of 1,702 lines parsed, and 1 of its 1,006 `[dlc]` lines.
|
|
41
|
+
Now 94.8% of that run parses (production went 82.4% → 90.5% from the same rules),
|
|
42
|
+
including per-peer download rates that exist in no RPC call. Frozen real fixtures
|
|
43
|
+
plus a ratio canary: `test/bench-log.test.js`, rule 15.
|
|
44
|
+
- [x] **Three log-health flags**: `log-missing` (no source, not a zero), `log-silent`
|
|
45
|
+
(file frozen — and it reports the chain delta observed during the silence rather
|
|
46
|
+
than asserting a cause, because the bench node block-buffers stdout to a file),
|
|
47
|
+
`log-unparsed` (lines arriving, <5% matching any rule). Plus `log.health.ratio`
|
|
48
|
+
published so coverage is visible on the Node panel instead of being internal.
|
|
49
|
+
- [x] **`cfg.log.tailBytes` never reached the tailer.** It was read as
|
|
50
|
+
`log.tailBytes` off the *logger function*, which has no such property, so the
|
|
51
|
+
configured value was ignored and LogTail's own default applied — the same number,
|
|
52
|
+
which is why it looked fine. Same trap would have swallowed `staleMs`; the monitor
|
|
53
|
+
now takes a `logCfg` of its own.
|
|
54
|
+
|
|
55
|
+
- [x] **RPC-only mode exists and is honest about what it loses.** `log.enabled: false`
|
|
56
|
+
/ `BLOCKYARD_LOG_SOURCE=0`: the monitor opens no log file at all (verified — the
|
|
57
|
+
process has no log fd), states the mode in `health.quality` (`log-source-disabled`)
|
|
58
|
+
naming the figures that lose their source, and `/api/config`'s provenance table is
|
|
59
|
+
generated per mode so no row can name a log source while the log is off. Two new
|
|
60
|
+
RPC sources came with it: `getaddrmaninfo` (11 ms) and `listbanned` (3 ms).
|
|
61
|
+
- [x] **`upload-unmeasurable`: the ratio check that stops a fake `0 B/s`.** The bench
|
|
62
|
+
build reports 12,896,531,244 bytes received against 1,129 sent with 21 peers
|
|
63
|
+
connected. That is not traffic, it is a counter missing the download worker, so
|
|
64
|
+
`outBps` is withheld (`uploadMeasured: false`) instead of rendering a calm zero.
|
|
65
|
+
- [x] **`peerinfo-partial`**: on builds that do publish rows, 21 peers sum to 70.29%
|
|
66
|
+
of `getnettotals` — labelled a subset, not presented as a breakdown.
|
|
67
|
+
- [x] **`nettotals-zero` and `peerinfo-empty` now say what happens *in the mode you
|
|
68
|
+
are running*.** Both used to end with "...comes from the node log instead", which is
|
|
69
|
+
a lie in RPC-only mode.
|
|
70
|
+
- [x] **`CounterRate.add` was called twice per `getnettotals` sample** (once discarded).
|
|
71
|
+
Harmless to the value, but it doubled the sample buffer; the RPC-only path made it
|
|
72
|
+
worth cleaning.
|
|
73
|
+
|
|
74
|
+
- [x] **Labelled log lines are scanned field by field, and the rigid rules were
|
|
75
|
+
deleted, not kept as fallbacks.** `bandwidthTickFields` and `dlcProgressFields`
|
|
76
|
+
decode the `[dlc]` tick and progress lines one labelled field at a time; unknown
|
|
77
|
+
fields are kept verbatim (`extraFields` / `extraValues`) and surfaced at
|
|
78
|
+
`log.unrecognisedTickFields`. Live coverage: bench **68.2% → 90.3%**,
|
|
79
|
+
production **73.6% → 89.7%**. A second code path for one line is how a rule can be
|
|
80
|
+
matching 185/185 and then 0/185 without anyone noticing, so there is now one path.
|
|
81
|
+
- [x] **Peers RPC cannot see are parsed**: 323 inbound connections on production were
|
|
82
|
+
accepted and dropped before BIP324 completed (`[serve] inbound … v2 handshake
|
|
83
|
+
failed`) — `getpeerinfo` never counts a peer it never negotiated with. Aggregated
|
|
84
|
+
into `inbound-handshake-failing` with a per-host breakdown (323 rows would have
|
|
85
|
+
pushed every real event out of the feed and said less).
|
|
86
|
+
- [x] **`node-restarting`**: `[serve] shutting down (signal 15): tip=965914` is
|
|
87
|
+
surfaced with the tip at shutdown, so the ECONNREFUSED that follows reads as the
|
|
88
|
+
restart instead of as a monitor or network fault. Measured while it happened: a
|
|
89
|
+
concurrent session deployed `deploy-20260908a` at 06:21 and the
|
|
90
|
+
monitor reported `online: false` with the reason intact.
|
|
91
|
+
- [x] **An unknown eta stays unknown**: `[dlc] ==` can print `eta --:--:--:--`;
|
|
92
|
+
decoded as `null`, never as zero or as a huge number.
|
|
93
|
+
|
|
94
|
+
- [x] **A production outage caused by two lost conf lines, surfaced correctly by the
|
|
95
|
+
monitor while the node looked "running" to systemd.** The 06:21 deploy dropped
|
|
96
|
+
`rpcport=8331`/`port=8332`; P2P then collided with the benchmark's Core on 8333
|
|
97
|
+
(`[boot] lsock failed: Address already in use`) and RPC drifted to 8332, so the
|
|
98
|
+
monitor reported `online false / ECONNREFUSED` about a node that was alive on a port
|
|
99
|
+
nobody was watching. Restoring the two lines fixed both. Full record in
|
|
100
|
+
`MEASUREMENTS 18`; the diagnostic note now sits in `server/config.js`, because this
|
|
101
|
+
is the third time those lines have been lost (2026-08-26, twice on 2026-09-08).
|
|
102
|
+
- [x] **Two timings that make a healthy node look broken**, now written down: RPC binds
|
|
103
|
+
**~100 s** after systemd says `running`, and `[utxo_live] init` blocks chain RPCs for
|
|
104
|
+
**~40 s** after that. A probe at 40 s sees `ECONNREFUSED`; the slow tier landing in
|
|
105
|
+
the init window produces `rpc timeout after 90000ms` while `getblockcount` answers in
|
|
106
|
+
0.15 ms moments later. Neither is a monitor fault and neither is now a mystery.
|
|
107
|
+
- [x] **The RPC peer/byte gap is closed on the current production build**
|
|
108
|
+
(`deploy-20260908a`: 4 peer rows, `bytesrecv` summing to exactly `getnettotals`,
|
|
109
|
+
nonzero sent) where the previous build answered `[]` and `0/0`. RPC-only mode is
|
|
110
|
+
therefore viable on production *today* — and left switched off by default, because
|
|
111
|
+
every build still reports the same non-Core `subversion` (`:0.0.1`) and the mode has
|
|
112
|
+
no way to know it is on a generous build. That asymmetry is the whole argument for
|
|
113
|
+
the log staying a primary source.
|
|
114
|
+
**Update 2026-09-11:** the "no way to know" half is false on `deploy-20260910ag`.
|
|
115
|
+
`getnetworkinfo` answers the build's own commit attestation "860b8fdd", dirty false
|
|
116
|
+
(MEASUREMENTS 27). The monitor does not read the field yet (see Functional gaps), and
|
|
117
|
+
older builds omit it.
|
|
118
|
+
|
|
119
|
+
## Security / correctness
|
|
120
|
+
|
|
121
|
+
- [x] **Listening address is now a decision on this box, and the failure modes are
|
|
122
|
+
loud.** Bound to the LAN address (`config/local.json` → `192.0.2.10`; the unit
|
|
123
|
+
agrees) instead of the wildcard, so tailscale and the docker bridges are no longer
|
|
124
|
+
served and `127.0.0.1` is not either — documented in `AGENTS.md` as the trap it is,
|
|
125
|
+
because a refused loopback curl looks exactly like a dead monitor. Two behaviours
|
|
126
|
+
added with it: binding an address the machine does not have **fails at boot and
|
|
127
|
+
prints the addresses it does have** (matters when the address came from DHCP), and
|
|
128
|
+
the boot log names the interfaces left unserved. `server.host` must now be an
|
|
129
|
+
address literal — a hostname is refused at load rather than at `listen()`, or worse,
|
|
130
|
+
at the next reboot after a DNS change. Covered by `test/bind.test.js`.
|
|
131
|
+
**Followed within the hour by the correction that mattered:** LAN-only locked the
|
|
132
|
+
operator out, because this box's browser traffic arrives over `tailscale0`
|
|
133
|
+
(`ip route get 198.51.100.8` → `dev tailscale0`) and the tailnet advertises no route
|
|
134
|
+
for `192.0.2.0/24` — a tailnet client cannot reach the LAN address at all. So
|
|
135
|
+
binding is multi-address now (`server.hosts`, one HTTP server per address; a single
|
|
136
|
+
socket cannot listen twice, which is what `ERR_SERVER_ALREADY_LISTEN` at 15:11 was
|
|
137
|
+
telling me), an address missing at boot is warned about and skipped rather than
|
|
138
|
+
fatal, and the configured set is LAN + tailnet — bridges and loopback still refused.
|
|
139
|
+
The lesson is recorded in `AGENTS.md`: *"no response from the LAN address" was
|
|
140
|
+
correct behaviour, not a fault*, and a bind change is a client-topology decision
|
|
141
|
+
that has to be checked against how clients actually arrive.
|
|
142
|
+
**Deliberately unchanged: the code default is still `0.0.0.0`**, so a fresh clone or
|
|
143
|
+
`npm run dev` binds everything. Changing that would break development on other
|
|
144
|
+
machines; the deployment decision lives in `config/local.json` (gitignored) and the
|
|
145
|
+
unit, which means *a new machine gets the wildcard until someone says otherwise*.
|
|
146
|
+
- [x] **TLS exists now; the lack of it stops being a documented shrug.** Serving
|
|
147
|
+
HTTPS was an option (`server.tls.cert`/`key`, `BLOCKYARD_TLS_CERT`/`_KEY`), and the
|
|
148
|
+
settings that must not be independent are wired together: turning TLS on forces the
|
|
149
|
+
session cookie `Secure` (a Secure cookie over HTTP is never sent, which reads as a
|
|
150
|
+
login that will not stick), HSTS is sent only over TLS (2 days, no `includeSubDomains`,
|
|
151
|
+
no `preload` — a LAN address can be reissued and HSTS cannot be unsaid), and every
|
|
152
|
+
listener is encrypted or none is ("encrypted on the tunnel, clear on the LAN" lets the
|
|
153
|
+
weakest address decide whether the cookie is a secret). Config mistakes that were
|
|
154
|
+
silent are now fatal: a **half** pair (cert without key) would otherwise fall back to
|
|
155
|
+
plaintext on a port someone believes is HTTPS; an **expired** certificate is fatal,
|
|
156
|
+
because a browser refuses and the dashboard — being behind that refusal — cannot say
|
|
157
|
+
so. Certs are parsed with `crypto.X509Certificate` (a builtin), so `selfSigned`,
|
|
158
|
+
`notAfter` and the fingerprint appear in the boot log; inside 14 days of expiry warns
|
|
159
|
+
rather than refusing. `test/tls.test.js` generates a throwaway cert with `openssl`
|
|
160
|
+
(and skips, naming the reason, where openssl is absent), and asserts plaintext
|
|
161
|
+
against the TLS listener *fails* rather than falling back. What remains true: the
|
|
162
|
+
default is still plaintext, and the boot now says so on stdout in words.
|
|
163
|
+
- [x] **CIDR membership is computed on bytes, for both families.** The old
|
|
164
|
+
implementation compared the TEXT of a client address with the text of a network
|
|
165
|
+
prefix (`net.split('::')[0]`, then `startsWith`). For `fd00::/8` that behaves; for
|
|
166
|
+
anything subtler it over-permits — the regression now pinned by `test/cidr.test.js`:
|
|
167
|
+
`2001:db8:1::/48` accepted `2001:db8:1f::1`, because the string starts with the
|
|
168
|
+
characters `2001:db8:1`. Text does not know where a nibble ends. Now: IPv4 and IPv6
|
|
169
|
+
parsed to bytes (compression, `%zone`, embedded IPv4, brackets), bit-exact
|
|
170
|
+
comparison, family never crossed (`0.0.0.0/0` does not admit `::1`), a malformed
|
|
171
|
+
entry admits nobody **and** stops the boot, and the refusal names the entry that
|
|
172
|
+
matched — for the log line, not the client.
|
|
173
|
+
- [x] **A per-response CSP script nonce.** `script-src 'self'` was fine *until*
|
|
174
|
+
someone added an inline script, at which point it would fail in the browser with
|
|
175
|
+
nothing recorded server-side — the same silently-dead shape as the module-scope
|
|
176
|
+
ReferenceError that blanked this UI for a day. HTML is now rewritten per response
|
|
177
|
+
(`%BLOCKYARD_NONCE%` placeholder + asset versioning), measured at 0.009 ms per request
|
|
178
|
+
(MEASUREMENTS §22), so the cost of the option is nil. The nonce lands in `script-src`
|
|
179
|
+
only — never in `style-src`, which would reopen the thing fixed below — and
|
|
180
|
+
`test/csp.test.js` asserts it differs between responses, because a constant nonce is
|
|
181
|
+
an open door with a documentation comment attached.
|
|
182
|
+
- [x] **`audit.jsonl` rotates by size** (8 MiB, keeping 5 ⇒ worst case 48 MiB), the
|
|
183
|
+
reader walks the chain newest-first so a rotation is invisible mid-page, the budget
|
|
184
|
+
is checked against the on-disk file rather than a drifting counter, a **failed**
|
|
185
|
+
rotation keeps appending and says so in `/api/telemetry` and the admin page, and a
|
|
186
|
+
restart adopts the existing file size. The reason for a hard bound is not tidiness:
|
|
187
|
+
this disk also holds the history snapshots and the node's datadir, so an unbounded
|
|
188
|
+
audit log's failure mode is not "no audit", it is "no node" (MEASUREMENTS §22).
|
|
189
|
+
- [x] **`/api/login` has its own per-address bucket** (capacity 10, refill 0.5/s),
|
|
190
|
+
separate from `LoginGuard`. The lockout answers "this username keeps failing" and the
|
|
191
|
+
per-user token bucket cannot apply pre-auth, so the unmitigated case was a slow grind
|
|
192
|
+
spread over addresses — each attempt costing a scrypt KDF, measured at **20 ms and
|
|
193
|
+
16 MB** on this box (that comment previously claimed ~50 ms and is corrected). Refusals
|
|
194
|
+
return `code:"throttled"` with a retry window and are audited as `login-throttled`.
|
|
195
|
+
Asserted in `test/http-app.test.js` and `scripts/smoke.sh`.
|
|
196
|
+
- [x] **Rehash-on-login, with the parameters stored per user.** Raising
|
|
197
|
+
`auth.scrypt` now actually raises it, one account at a time, at the only moment the
|
|
198
|
+
plaintext is in hand. Three details that matter: rehash happens on a **successful**
|
|
199
|
+
verify only (hashing a wrong password would store a hash of the wrong secret and
|
|
200
|
+
destroy the account), verification uses the *stored* parameters so an old hash never
|
|
201
|
+
stops working, and an unrecognised `scheme` is refused rather than assumed to be
|
|
202
|
+
scrypt. The admin page marks accounts whose stored cost is behind (`kdf behind`), and
|
|
203
|
+
the upgrade is logged and audited — "it applies on next login" is only checkable if
|
|
204
|
+
that login reports it.
|
|
205
|
+
|
|
206
|
+
- [x] **A latent one, found while testing something else: env vars whose cast was a
|
|
207
|
+
function were never cast.** `env()` implemented `Number` and `Boolean` and returned
|
|
208
|
+
the raw string otherwise, so `BLOCKYARD_ALLOW_CIDRS=a/24,b/8` arrived as a **string** —
|
|
209
|
+
and the gate iterated it character by character, parsed nothing, and refused every
|
|
210
|
+
address including the operator's. The documented way to restrict the monitor to a LAN
|
|
211
|
+
was a deny-all. `BLOCKYARD_ACTIONS` was the same shape, making `allow.includes(name)` a
|
|
212
|
+
*substring* test on a string. `BLOCKYARD_HOST`/`_BIND` survived only because `validate()`
|
|
213
|
+
re-splits a string there. Found by `test/cidr.test.js`, which was written to test CIDR
|
|
214
|
+
matching — the bug surfaced one layer below where the mistake was. Fixed in
|
|
215
|
+
`env()` (rule 20), with regressions for both variables.
|
|
216
|
+
|
|
217
|
+
- [x] **Access is now open by default, and the way it was done is the defect that
|
|
218
|
+
would have been.** `auth.enabled` flipped to `false`: no sign-in, like a block
|
|
219
|
+
explorer. The obvious implementation moves the session check behind the flag — and
|
|
220
|
+
takes the `route.auth === 'admin'` check down with it, so "no login required" becomes
|
|
221
|
+
"no login required, and anyone may create accounts". Guarded structurally instead:
|
|
222
|
+
the anonymous identity is a frozen `viewer` that no config or credential can raise;
|
|
223
|
+
the admin-role check runs on **both** branches; `/api/login` answers
|
|
224
|
+
`accounts_disabled` rather than pretending a credential exists; `/api/audit` declines
|
|
225
|
+
rather than serving a trail that can name nobody; CSRF is skipped only because there
|
|
226
|
+
is no cookie to ride (with the two checks that make that safe asserted next to it);
|
|
227
|
+
rate limits key per **address**, so one chatty tab cannot spend the LAN's bucket; and
|
|
228
|
+
node writes are refused twice over — fatal at config load, again at the route — unless
|
|
229
|
+
`BLOCKYARD_ALLOW_WRITES_WITHOUT_AUTH=1` is chosen deliberately. The boot line names the
|
|
230
|
+
readable addresses, what "read" grants, and the switch that closes it. Rules 23, 24;
|
|
231
|
+
`test/open-access.test.js`, 19 new smoke checks against a second, open instance.
|
|
232
|
+
|
|
233
|
+
- [x] **`scripts/manage-users.js` — the documented password-recovery CLI — had not
|
|
234
|
+
parsed for some time.** An editor put a real carriage return *inside a string
|
|
235
|
+
literal*. Legal in CommonJS, a syntax error in ESM, and this project is
|
|
236
|
+
`"type": "module"`, so `node scripts/manage-users.js passwd admin` threw before
|
|
237
|
+
`main()` ran. Nothing caught it because nothing imports a CLI: `npm test` never
|
|
238
|
+
executed it, and `node --check` on the same bytes **passes outside the project**
|
|
239
|
+
(parsed as CJS) — a check that succeeds for the wrong reason (rule 21). Fixed the
|
|
240
|
+
literal (raw DEL/ETX operands replaced with escapes while I was there) and added
|
|
241
|
+
`test/parse-all.test.js`: every shipped file under `server/`, `public/js/` and
|
|
242
|
+
`scripts/` must parse **as the module it will be loaded as**, no raw control
|
|
243
|
+
characters anywhere, and the CLI must actually run. It asserts how many files it
|
|
244
|
+
checked, so the glob cannot quietly match nothing.
|
|
245
|
+
|
|
246
|
+
- [x] **`CounterRate` fabricates a `0 B/s` download rate after every peer disconnect
|
|
247
|
+
on `deploy-20260910ag`, and understates it silently after smaller ones.** Found
|
|
248
|
+
2026-09-11 (MEASUREMENTS 27). **Closed 2026-09-14 by scope:** the defect is that build's
|
|
249
|
+
`getnettotals`, which sums the *live* peer table and so drops when a peer leaves. Bitcoin
|
|
250
|
+
Core's `getnettotals` is a lifetime counter for the process (it only ever grows), so on the
|
|
251
|
+
supported node the reset-on-decrease rule is exactly right: a decrease *is* a restart. The
|
|
252
|
+
candidate fixes stay below for whoever supports a node whose counters forget. `server/store/ring.js` `CounterRate.add`
|
|
253
|
+
treats any value below the window's oldest sample as a node restart:
|
|
254
|
+
`if (value < this.samples[0].value) { this.samples = [{ t, value }]; return 0; }`.
|
|
255
|
+
That is right for a lifetime counter, and this build's `getnettotals` is not one. It
|
|
256
|
+
sums the **live** peer table plus the download's bytes (`cmd_getnettotals`, whose own
|
|
257
|
+
comment calls this a documented divergence from Core), so it drops whenever a peer
|
|
258
|
+
leaves.
|
|
259
|
+
Measured: **109,036,122 → 89,979,098** bytes received between 09:22:07 and
|
|
260
|
+
09:23:03Z, when a peer holding **19,303,469** bytes disconnected. Two failure shapes:
|
|
261
|
+
- **Drop larger than the window's growth**: reset, and `rateIn.add()` returns `0`,
|
|
262
|
+
which `monitor.js` stores straight into `state.net.inBps`. That lands in the `net`
|
|
263
|
+
ring and on the chart as a calm idle sample. The node was moving 4,268 B/s over the
|
|
264
|
+
next clean interval. `rateOut` resets the same way. Its `add()` return is discarded
|
|
265
|
+
and `outBps = rateOut.rate()` reads `null` with one sample, so upload shows `–` for
|
|
266
|
+
that tick with no reason attached.
|
|
267
|
+
- **Drop smaller than the window's growth** (180 s window for both directions): no
|
|
268
|
+
reset. `rate()` is `(last − first) / span`, which silently includes the departed
|
|
269
|
+
peer's bytes as a negative step. It stays understated by
|
|
270
|
+
`bytes the peer held / window span` until the pre-disconnect samples age out of the
|
|
271
|
+
window. No flag, no zero, just a lower number: the shape nobody will catch by eye.
|
|
272
|
+
|
|
273
|
+
This breaks AGENTS.md rule 3 (never fabricate a number to fill a gap) and RULES rule 8
|
|
274
|
+
(absent beats zero). It is the same sin §23 removed for a counter stuck at 0, now
|
|
275
|
+
arriving through the restart heuristic.
|
|
276
|
+
Candidate fixes, to decide rather than implement blind:
|
|
277
|
+
1. **Per-peer deltas keyed by `getpeerinfo` `id`.** For control peers `id` is
|
|
278
|
+
`nodeid`, taken from `__sync_fetch_and_add(&next_nodeid, 1)` (node `main.c`) and
|
|
279
|
+
documented in `rpc_node.c` as unique for the process lifetime. A departed peer then
|
|
280
|
+
simply stops contributing. Two costs: download-worker rows are appended with
|
|
281
|
+
`id = 100000 + slot`, a slot index a different peer can reuse, so key those on
|
|
282
|
+
`(id, addr, conntime)`; and `getpeerinfo` sits in the 15-min rare tier while the log
|
|
283
|
+
is on, whereas this needs it on the 4 s fast tier (0.25–0.29 ms, 3.5–3.9 kB measured).
|
|
284
|
+
2. **Tell a restart from a departure.** `uptime` is already in the same fast-tier batch.
|
|
285
|
+
Reset only when it went backwards. Otherwise the decrease is a departure: withhold
|
|
286
|
+
the sample (`null` plus a reason) instead of returning `0`. The understated case
|
|
287
|
+
still needs option 1, because the drop in `getnettotals` mixes the departed peer's
|
|
288
|
+
bytes with everyone else's growth.
|
|
289
|
+
3. Whatever is chosen, `blockRate` (`monitor.js`) relies on the reset-on-decrease
|
|
290
|
+
behaviour deliberately and must keep it. A test should replay the measured sequence
|
|
291
|
+
above and assert that neither `0` nor an understated rate comes out.
|
|
292
|
+
|
|
293
|
+
- [x] **A node without `coinstatsindex` was sent a full UTXO-set walk every minute, and the index
|
|
294
|
+
build took the blame for an hour.** Found 2026-09-14 on the first fresh install (a Mac, Core
|
|
295
|
+
29.1, block files on a platter array). The symptoms -- 18 s RPC answers, 90 s timeouts, the
|
|
296
|
+
mempool read dropped, an empty block-space board -- appeared the moment the address index build
|
|
297
|
+
started, so the build was throttled four ways (its own connection, a pacer on the node's
|
|
298
|
+
latency, fewer workers, smaller batches) before anyone measured the node with nothing running:
|
|
299
|
+
`npm run check` then showed 10 ms / 921 ms / 1.0 s for chain info, a full block and the
|
|
300
|
+
mempool. The cause was `utxoStatsWanted` reading a missing `coinstatsindex` key as "not assumed
|
|
301
|
+
unindexed" and asking `gettxoutsetinfo` -- 165 M outputs without the index -- on the slow tier
|
|
302
|
+
every 60 s, past the timeout, with Core still walking and holding its chain lock. It was there on
|
|
303
|
+
the very first boot, before any build existed. Fixed: the indexes are asked first, alone, and the
|
|
304
|
+
UTXO figures only of a node that has said it keeps the index (`test/utxo-hashtype.test.js`).
|
|
305
|
+
The build throttling stays: it is right on a shared disk, it just was not the fault.
|
|
306
|
+
**The lesson, for the next slow node:** measure it alone first. `npm run check` now times every
|
|
307
|
+
call it makes, which is three lines that would have saved an hour.
|
|
308
|
+
- [x] **The index held 256 and more file descriptors open for the life of the process, and macOS
|
|
309
|
+
allows a process 256.** Found 2026-09-14 preparing the first macOS install: the store kept a
|
|
310
|
+
descriptor per segment and layer, and the build kept all 256 bucket files open through the scan --
|
|
311
|
+
on a stock macOS that is the whole soft limit (`ulimit -n`) before a socket is opened. Fixed the
|
|
312
|
+
same day: a lookup opens the one file it reads and closes it (`server/chain/index/store.js`;
|
|
313
|
+
measured on the full index afterwards, 0.02 ms median warm, 0.28 ms p90, 21 descriptors held by
|
|
314
|
+
the process), and the build keeps at most 64 buckets open, least recently written closed first
|
|
315
|
+
(`MAX_OPEN` in `build.js`).
|
|
316
|
+
- [x] **A 42-minute gap between blocks showed STALLED on two independent, healthy nodes.** Found
|
|
317
|
+
2026-09-14 (operator: "Production is fucked now"). The state came from the tip's age alone, and the
|
|
318
|
+
network finds no block for 40 minutes about once in fifty. Fixed the same day in
|
|
319
|
+
`server/collect/sync.js`: stalled only when a connected peer reports a tip above this node's
|
|
320
|
+
(`getpeerinfo` `synced_headers`, `startingheight` as the fallback); peers agreeing on the tip is a
|
|
321
|
+
long gap and synced, with a caveat saying so; no peer height at all waits two hours
|
|
322
|
+
(`STALL_ALONE_SEC`). The caveat names which of the three applies.
|
|
323
|
+
|
|
324
|
+
## Functional gaps
|
|
325
|
+
|
|
326
|
+
- [x] **Address history and balances on Bitcoin Core: resolved by an index of our own.** Found
|
|
327
|
+
2026-09-13 as "no address index, and cannot have one from Core"; closed 2026-09-14 by
|
|
328
|
+
`server/chain/index/` -- built from the node's block files, kept current by a follower, checked
|
|
329
|
+
to the satoshi against `scantxoutset` (MEASUREMENTS §30), reached by `npm run setup`. The two
|
|
330
|
+
things an address page still lacks are their own entry below. The record of how it got here: `getaddressbalance` and `getaddresstxids` are insight-style extensions
|
|
331
|
+
that Bitcoin Core has never carried at any setting; measured against both configured
|
|
332
|
+
nodes — an Umbrel and the local Core — each answers `Method not found`. mempool.space
|
|
333
|
+
shows balance, total received, UTXO counts and a balance history for the same address
|
|
334
|
+
because `electrs` builds that index itself from the block files; it does not ask Core,
|
|
335
|
+
because Core cannot answer.
|
|
336
|
+
**Fixed on the honesty axis** (commit `46e378f`): a refusal used to become `[]`, then
|
|
337
|
+
`txCount: 0`, then the words "no transactions in this node's address index" — a
|
|
338
|
+
fabricated zero indistinguishable from a genuinely unused address. The reply now
|
|
339
|
+
carries `indexed: false` with a **null** count, and the page says the index is absent.
|
|
340
|
+
**Fixed on the capability axis (2026-09-14)** where a node has `addressIndex` configured:
|
|
341
|
+
`server/chain/index/` builds the index from the node's own blk/rev files (30 minutes, 124 GB,
|
|
342
|
+
MEASUREMENTS §30) and the address page reads history, balance and each transaction's net amount
|
|
343
|
+
from it. Checked live: every row of four pages (including page 4 of a 2.3 M-transaction address)
|
|
344
|
+
matched the node's decoded transaction for txid, height and amount.
|
|
345
|
+
**It follows the chain (2026-09-14):** `server/chain/index/live.js`, started by the server for each
|
|
346
|
+
index directory, fetches every new block with `getblock <hash> 3`, logs its rows (CRC-framed,
|
|
347
|
+
replayed on restart) before serving them, rolls back on a reorganisation, folds blocks 100 deep
|
|
348
|
+
into immutable layers and merges layers past 32. Checked live: caught 12 blocks up in 6.5 s, then
|
|
349
|
+
40 of 40 balances equal `scantxoutset` at that tip, 38 of those addresses having rows only in the
|
|
350
|
+
new tail. A reorganisation below what is folded stops the follower and the page says to rebuild.
|
|
351
|
+
**The dead RPCs are no longer sent on every view** (2026-09-14). `xAddress` remembers a
|
|
352
|
+
"method not found" per node and skips `getaddressbalance`/`getaddresstxids` for
|
|
353
|
+
`INDEX_RECHECK_MS` (10 minutes), then asks again, because the daemon behind a node id can
|
|
354
|
+
change. A timeout or other error is not a refusal and is never remembered. Measured
|
|
355
|
+
against both configured nodes: view 1 sends `validateaddress` plus the two refused
|
|
356
|
+
calls; views 2 and 3 send `validateaddress` alone, with the same page result
|
|
357
|
+
(`indexed: false`, `txCount: null`).
|
|
358
|
+
|
|
359
|
+
- [x] **We can read the block files after all — `-blocksxor`, not an unknown format.** This finding
|
|
360
|
+
became the index build (`server/chain/blockfile.js`, `server/chain/index/build.js`): closed the
|
|
361
|
+
same day it was opened, 2026-09-14, and kept for the numbers.
|
|
362
|
+
Found 2026-09-14, correcting a wrong conclusion reached the same day. Every
|
|
363
|
+
`blk*.dat` on the local node opens with `1c33dfa8` rather than the mainnet magic
|
|
364
|
+
`f9beb4d9`, and a 126.5 MB file contains **zero** occurrences of any network magic,
|
|
365
|
+
which read as "this is not a Core datadir". It is: Core XOR-obfuscates block data at
|
|
366
|
+
rest (default since v28) with the 8-byte key in `blocks/xor.dat`. XOR the file by
|
|
367
|
+
`offset % 8` and the magic, the size field and every record appear.
|
|
368
|
+
**Verified rather than assumed**, because two earlier spikes produced numbers that
|
|
369
|
+
were void and I nearly published both: the first parsed **zero** blocks and still
|
|
370
|
+
reported "337 MB/s"; the second desynced and crashed on a garbage varint. A parser
|
|
371
|
+
that walks nothing measures an empty loop. The third decodes block records and checks
|
|
372
|
+
them against the node — heights 918042, 918304, 918323, transaction counts (3865,
|
|
373
|
+
2968, 4255) and byte sizes (1614012, 1746797, 1571713) all matching `getblock`
|
|
374
|
+
exactly. Only then were its timings used.
|
|
375
|
+
**Measured cost of a full read**, on 5,756 files / ~711 GB: **~0.7 h of disk** at the
|
|
376
|
+
296 MB/s this device sustains on files not already in cache (203 / 414 / 360 MB/s
|
|
377
|
+
across three), overlapping ~0.16 h of CPU for the XOR and record framing. A re-read of
|
|
378
|
+
the same file takes 30 ms against 630 ms cold — which is why the first throughput
|
|
379
|
+
figure (1258 MB/s, implying a 10-minute chain) was page cache and is not used here.
|
|
380
|
+
**Measured in full on 2026-09-14 (MEASUREMENTS 28):** decoding every block and every
|
|
381
|
+
undo record with addresses is **9.5 single-core hours** (5.9 of them block decoding),
|
|
382
|
+
over 1.44 billion transactions (calibrated against `getchaintxstats` to 0.7%), 3.63 B
|
|
383
|
+
funding rows and 3.51 B spending rows. Raw index rows: **85.6 GB** history-only, **142.7 GB**
|
|
384
|
+
with amounts, before storage-engine overhead. Undo data (`rev*.dat`) supplies every
|
|
385
|
+
input's spent output, checked against `getblock <hash> 3` with zero mismatches, so the
|
|
386
|
+
spending side needs no UTXO replay.
|
|
387
|
+
|
|
388
|
+
- [ ] **An address page has no mempool transactions.** A transaction still in the mempool is not
|
|
389
|
+
in the index, and the page says unconfirmed transactions are not included. Needs the follower
|
|
390
|
+
to watch the mempool for the address's scripts -- the arrivals per poll, decoded, with their
|
|
391
|
+
parents' outputs for the spending side -- or a ZMQ `rawtx` client; either is a day's work done
|
|
392
|
+
honestly.
|
|
393
|
+
**The unspent-output list is done (2026-09-14):** the index names every transaction that touched
|
|
394
|
+
the address, each one's outputs paying it are asked of `gettxout` (the UTXO set, less what the
|
|
395
|
+
mempool already spends), and the page lists them with the index's own height. That walk is the
|
|
396
|
+
whole history, so it is done for an address with at most 100 transactions and declined in
|
|
397
|
+
words for a longer one (`utxoNote`); an unspent flag in the index itself would need every
|
|
398
|
+
spend to update the row it spends, which the append-only layers do not do cheaply.
|
|
399
|
+
|
|
400
|
+
- [ ] **The explorer pays roughly 3–5x for verbose RPC it re-parses anyway.** Measured
|
|
401
|
+
2026-09-13/14. `fetchTxs` fetches up to 25 transactions per page with
|
|
402
|
+
`getrawtransaction <txid> 2`, and `xBlock` adds `getblock <hash> 1` plus
|
|
403
|
+
`getblockstats`. On the same block, verbosity 2 returns 10.77 MB against 3.17 MB of
|
|
404
|
+
raw hex at verbosity 0 — and the cost is worst where the node is slowest: on the
|
|
405
|
+
Umbrel, **1,843 ms for v2 against 373 ms for v0**, of which 880 ms is the node's own
|
|
406
|
+
serialization (time to first byte) versus 73 ms locally. Fetching raw and decoding
|
|
407
|
+
locally would cut both the bytes and the node's CPU. The disk parser in the entry
|
|
408
|
+
above is the same decoding work, so this is one piece of code, not two.
|
|
409
|
+
**The decoder exists** (2026-09-14): `server/chain/tx.js` decodes raw transactions and
|
|
410
|
+
blocks into Core's verbose field names -- txid, wtxid, size, vsize, weight, inputs,
|
|
411
|
+
witnesses, output values in integer satoshis, script type and address, with Core's
|
|
412
|
+
Solver order and names. `scripts/decode-check.js` replays whole blocks from a live node
|
|
413
|
+
and compares every field: 29 blocks from height 170 to the tip, 47,507 transactions and
|
|
414
|
+
124,129 outputs, zero disagreements. **The explorer does not use it yet**: raw bytes carry
|
|
415
|
+
no prevout values or addresses, so fees and input addresses still need the spent
|
|
416
|
+
outputs from somewhere, and choosing that source is the open part.
|
|
417
|
+
**Narrowed 2026-09-14:** the address page no longer asks the node for an address's history at
|
|
418
|
+
all -- the index supplies every (block, position, net amount) and the page fetches only the 25
|
|
419
|
+
transactions it shows, still at verbosity 2. Transaction and block pages are as measured.
|
|
420
|
+
|
|
421
|
+
- [ ] **The transaction cache does not survive a restart.** `server/http/explorer.js`
|
|
422
|
+
keeps a 3,000-entry in-memory LRU of confirmed transactions and nothing else. Every
|
|
423
|
+
restart re-asks the node for pages it served minutes earlier, which on the Umbrel is
|
|
424
|
+
seconds per page. Confirmed transactions are immutable, so they are the safest thing
|
|
425
|
+
in the system to persist; the ring/history machinery under `server/store/` already
|
|
426
|
+
writes to disk and is the obvious place to put them.
|
|
427
|
+
**Narrowed 2026-09-14:** the cost that made this hurt was the remote node -- seconds per page
|
|
428
|
+
over the LAN. On the node's machine, where BlockYard now runs, a page's re-ask is ~70 ms of
|
|
429
|
+
the node's time per transaction (measured on the local Core), so this is a nicety, not a wound.
|
|
430
|
+
|
|
431
|
+
- [x] **Coverage is now measured per shape, not globally.** `SHAPES` in
|
|
432
|
+
`logparse.js` declares which measurements must keep arriving with gates derived from
|
|
433
|
+
measured cadences (p95 x 8, clamped to [10, 30] min — table in `MEASUREMENTS 19`),
|
|
434
|
+
and the monitor raises `log-shape-silent` when a shape it has *seen* stops arriving
|
|
435
|
+
while the log keeps moving. A shape is armed on first match, so a build that never
|
|
436
|
+
emits a line is excused without a build table. Gates are overridable via
|
|
437
|
+
`logCfg.shapeGatesMs`, which is how the tests assert in milliseconds.
|
|
438
|
+
|
|
439
|
+
Four unparsed production shapes were found on the way and are now rules: `addrv2
|
|
440
|
+
gossip`, `outbound top-up` (aggregate, with a reason breakdown →
|
|
441
|
+
`outbound-dial-failing`), `[dl] updating utxo` (validation throughput, a third rate
|
|
442
|
+
kept separate per rule 9), `header mirror`, plus `[dial]` background attempts /
|
|
443
|
+
failures and the **`no global IPv6 route`** note (`ipv6-unreachable`, "a host
|
|
444
|
+
capability, not a node fault"). Production coverage 73.4% → **82.8%**.
|
|
445
|
+
|
|
446
|
+
Two bugs this caught, both mine, both now pinned by tests:
|
|
447
|
+
- it flagged a **synced** node (`initialblockdownload=false`, `vp=1`,
|
|
448
|
+
`blocks==headers`) for "bandwidth rate stopped arriving", because I had inferred
|
|
449
|
+
IBD from log shapes and the tail contained post-boot `[utxo_live] catchup
|
|
450
|
+
progress`. The node's own RPC answer now takes precedence; log shapes are the
|
|
451
|
+
fallback, and if even that is unknown, we do not watch.
|
|
452
|
+
- `[serve] shutting down` replayed from four hours of backfilled tail raised
|
|
453
|
+
`node-restarting` on a node that had been up since. The flag is now age-gated and
|
|
454
|
+
is cleared the moment `getblockchaininfo` answers. The event still reaches the
|
|
455
|
+
feed, where its timestamp speaks for itself.
|
|
456
|
+
- [x] **The RPC circuit breaker is per-node, so in principle one slow method can blind
|
|
457
|
+
every panel — unproven as harmful, and now measurable.** **Closed 2026-09-14:** measured not
|
|
458
|
+
harmful (0 trips in clean running, below), the policy kept on purpose, and `rpc.breaker` in
|
|
459
|
+
`/api/telemetry` names what opened it if it ever bites. Kept as the record of the question. Observed at 10:35–10:38:
|
|
460
|
+
`online false (breaker open, retry in 26s)` → `online true` → `online false` within
|
|
461
|
+
25 s, while a direct `getblockcount` answered in 1 ms in between. That looked like
|
|
462
|
+
granularity until it could be measured properly: over 15 minutes of clean running
|
|
463
|
+
after the per-node tagging landed, **0 breaker trips and 0 errors on either node**
|
|
464
|
+
(max avg latency 22 ms on production, 362 ms on bench, chain points continuing on
|
|
465
|
+
both). Every flap that day aligned with one of the nine real restarts. So the earlier
|
|
466
|
+
guess — that this was the mechanism behind "8088 is not updating" — is not supported,
|
|
467
|
+
and is corrected here rather than left in as a plausible-sounding defect.
|
|
468
|
+
Remaining question, worth answering only if it bites again: a slow-tier timeout
|
|
469
|
+
currently opens a breaker that also blocks the 4 s tier. Per-tier granularity is a
|
|
470
|
+
small change but it is not obviously better — blocking *everything* for 30 s after 3
|
|
471
|
+
consecutive failures is also what protects a single-threaded node from a client that
|
|
472
|
+
will not stop asking.
|
|
473
|
+
|
|
474
|
+
**Updated 2026-09-09 — the question is now answerable even though the policy is
|
|
475
|
+
unchanged.** The lane records what opened the breaker (`openedBy`: method label,
|
|
476
|
+
error kind, duration, and the calls that were queued when it opened), the last
|
|
477
|
+
failure, the consecutive count against the threshold, and the remaining cooldown;
|
|
478
|
+
`/api/telemetry` publishes it as `rpc.breaker`, and a call refused *because* the
|
|
479
|
+
breaker is open now says which method opened it instead of emitting a shrug. A gap
|
|
480
|
+
found while writing that: the breaker was checked at submit but **not** at dequeue,
|
|
481
|
+
so work already queued when it opened still fired at the node — "back off for 30 s"
|
|
482
|
+
meant "back off for new questions only". Fixed, with a test
|
|
483
|
+
(`test/breaker.test.js`).
|
|
484
|
+
- [x] **Production was restarted nine times before 10:39** -- **closed 2026-09-14 by scope:** the
|
|
485
|
+
deploy that did it belonged to the experimental node; what this repo owed (say the shape out
|
|
486
|
+
loud: `node-restart-storm`) was done on 2026-09-09. The record: (18 shutdown lines in one
|
|
487
|
+
log file: 06:21, 06:31, 06:35, 09:53, 10:11, 10:34, 10:36, 10:38, …) by a concurrent
|
|
488
|
+
deploying session. Not this repo's bug, but it is the single biggest determinant of
|
|
489
|
+
what the dashboard looks like, and every monitor reading during those windows is
|
|
490
|
+
correct-but-alarming. Whoever owns the deploy should know the box sees it as ~9
|
|
491
|
+
outages. **What this repo changed on 2026-09-09:** the shape is now said out loud —
|
|
492
|
+
3+ shutdowns in 60 min raises `node-restart-storm`, quoting the timestamps, with the
|
|
493
|
+
window on the **log's** clock so a backfill of archived shutdown lines cannot read as
|
|
494
|
+
nine restarts happening now. Nine individually-correct alarms in an hour were being
|
|
495
|
+
read as "the monitor is flapping", which is the opposite of what a true-but-noisy
|
|
496
|
+
signal should produce. The history that produced the shape table below:
|
|
497
|
+
|
|
498
|
+
| shape | what changed | rigid-rule result |
|
|
499
|
+
|---|---|---|
|
|
500
|
+
| `[dlc] --` tick | `+ staged 1` at 05:42 | 0 of 26 lines |
|
|
501
|
+
| `[dlc] --` tick | `staged N commit N` at 06:14 | 0 of 30 lines |
|
|
502
|
+
| `[dlc] ==` progress | `(no gap, 100.00% landed)`, `eta --:--:--:--` | 0 of 185 lines |
|
|
503
|
+
| `[serve] inbound` (new tag) | 646 lines, none previously parsed | — |
|
|
504
|
+
|
|
505
|
+
Each was fixed by scanning labelled fields one at a time (`bandwidthTickFields`,
|
|
506
|
+
`dlcProgressFields`): a new field now costs that field and lands in
|
|
507
|
+
`log.unrecognisedTickFields` verbatim (`{"staged":"26","commit":"6484"}`). A third
|
|
508
|
+
field, `cursorhelp`, appeared two hours later and cost nothing.
|
|
509
|
+
- [x] **`app.js` had zero coverage and died while evaluating — the page has never
|
|
510
|
+
rendered, since the first commit.** `app.js:3` imported five chart helpers;
|
|
511
|
+
`app.js:379` builds `helpers.charts = { lineChart, histogram, scatter, meter,
|
|
512
|
+
stackedBars, ... }` at **module scope**. Shorthand properties resolve identifiers, so
|
|
513
|
+
the module threw `ReferenceError: histogram is not defined` before `boot()`, before
|
|
514
|
+
login, before the stream — and therefore **before any HTTP request**. Every symptom
|
|
515
|
+
today ("does not update with any stats at all", "nothing refreshes", "can't get stats
|
|
516
|
+
for the production build") traces here. `git log -S` puts it in `256d970`, the initial
|
|
517
|
+
commit: it has never worked in a browser.
|
|
518
|
+
- Why nothing caught it: 60 smoke checks drive HTTP directly and never execute the
|
|
519
|
+
shipped entry point; `node --check` parses, and a missing import parses fine; the
|
|
520
|
+
static DOM-contract tests check identifiers used in strings, not bindings. A green
|
|
521
|
+
suite that never evaluates the thing the user runs is measuring the wrong thing.
|
|
522
|
+
- Why I did not find it: the server log was empty of requests, which is *evidence of
|
|
523
|
+
a dead client*, and I read it as "their tab isn't connecting" and went three layers
|
|
524
|
+
into sessions, cookies and bind addresses. The browser console settled it in one
|
|
525
|
+
line, and asking for the console should have been the **first** question, not the
|
|
526
|
+
sixth. Cheapest decisive evidence goes first — that is the whole reason this repo
|
|
527
|
+
writes down measurements before opinions.
|
|
528
|
+
- Fixed by importing the four names, and closed with `test/app-boot.test.js`, which
|
|
529
|
+
imports `app.js` under the DOM stub (`test/dom-stub.js`) — Node evaluates the same
|
|
530
|
+
ESM graph, so this is the browser's structural check with no browser. Verified in
|
|
531
|
+
both directions: with the import removed it fails with the browser's own message
|
|
532
|
+
(`ReferenceError: histogram is not defined`), and a second assertion names all four
|
|
533
|
+
missing imports. That second assertion was initially **vacuous** — it matched the
|
|
534
|
+
first `charts: {}` in the file, which is the empty one in `state`, and "passed"
|
|
535
|
+
while the bug was live; it now scans every occurrence and asserts it found
|
|
536
|
+
something to check.
|
|
537
|
+
- [x] **CSP blocked every data-driven style attribute**, so the layout collapsed on top
|
|
538
|
+
of the dead script: `style-src 'self'` also refuses `style="…"`, and the UI draws bar
|
|
539
|
+
and progress widths that way, producing a wall of "Refused to apply inline style".
|
|
540
|
+
Narrowed with `style-src-attr 'unsafe-inline'` — data-driven attributes allowed,
|
|
541
|
+
inline `<style>` elements and style text still blocked — rather than loosening
|
|
542
|
+
`script-src`, which is the directive that actually matters for XSS.
|
|
543
|
+
- [x] **`style-src-attr 'unsafe-inline'` is deleted, and a test keeps it dead.** The
|
|
544
|
+
static `style="…"`s became classes in `public/css/app.css`; the data-driven ones (the
|
|
545
|
+
sync bar's fill and the `verificationprogress` tick) travel as `data-w` / `data-left`
|
|
546
|
+
and are written through the CSSOM by `applyDataSizes()`, which CSP permits — a string
|
|
547
|
+
that never reaches an HTML attribute cannot be an injection site. Numbers only: a
|
|
548
|
+
value that will not parse is skipped rather than pasted into CSS, and out-of-range
|
|
549
|
+
values are clamped. `test/csp.test.js` asserts three separate things — no style
|
|
550
|
+
attribute in shipped HTML, none in injected markup, and that the CSSOM path actually
|
|
551
|
+
sets the width (without which "no style attributes" would just be a grep that broke
|
|
552
|
+
the sync bar).
|
|
553
|
+
- [ ] **No real-browser harness — narrowed, not closed.** The gap this entry
|
|
554
|
+
described ("the network/session layer between the renderers and the node") is now
|
|
555
|
+
covered, but by Node rather than by a browser: `app.shutdown()` exists (teardown was
|
|
556
|
+
the only thing preventing an in-process boot), so `test/helpers/http.js` boots the
|
|
557
|
+
real app — one node, two nodes, or HTTPS with a generated certificate — and speaks
|
|
558
|
+
HTTP to it: headers, cookies, `304` behaviour, nonces, the login throttle, the
|
|
559
|
+
drill-down routes. The DOM stub also parses the tags the app injects, which is how
|
|
560
|
+
the CSSOM-width claim is asserted as behaviour rather than as a grep.
|
|
561
|
+
**Still open, and not closeable without a browser engine:** layout, the CSS cascade,
|
|
562
|
+
real `EventSource` reconnect semantics, and anything a stub agrees with itself about.
|
|
563
|
+
The stub is a parser of the markup this app writes, not a layout engine; do not read
|
|
564
|
+
its green as "the page renders".
|
|
565
|
+
**Narrowed again 2026-09-14:** there is a browser now, outside `npm test` -- headless Chromium
|
|
566
|
+
driven over CDP by `scripts/browser-check.mjs`, `scripts/live-render-check.mjs`,
|
|
567
|
+
`scripts/motion-check.mjs` and `scripts/shots.mjs`, which is how the block-space paint order
|
|
568
|
+
was measured pixel by pixel, the README screenshots are taken, and the About page's
|
|
569
|
+
click-to-copy was proven by reading the clipboard back. Not in the suite because it needs a
|
|
570
|
+
browser on the box; still the only thing that sees layout.
|
|
571
|
+
- [x] **The page says which build it is running, and complains when it is not
|
|
572
|
+
current.** The static layer stamps `data-blockyard-build` and rewrites asset URLs to
|
|
573
|
+
`?v=<build>` (build id = version + digest of asset sizes and mtimes, `0.1.0-3faef7bb00`
|
|
574
|
+
form — not a git SHA, because this box deploys by copy and a SHA would describe a tree
|
|
575
|
+
that was never committed at boot). `/api/build?build=` answers `matchesClient`,
|
|
576
|
+
`checkBuild()` compares them on load and every 5 min, and on mismatch the header shows
|
|
577
|
+
`stale build — reload` and toasts once. The important part is the *comparison*: a
|
|
578
|
+
footer that shows a hash nobody can compare against anything is decoration. HTML
|
|
579
|
+
consequently never answers `304` (its body is per-response); assets keep their ETag.
|
|
580
|
+
Asserted in `test/http-app.test.js` and nine checks in `scripts/smoke.sh`.
|
|
581
|
+
- [x] **`rpc-lane.test.js` raced under load: fixed 2026-09-13.** "a long-running heavy call
|
|
582
|
+
does not starve a cheap high-priority poll" and its sibling each did `await sleep(5)` to let the
|
|
583
|
+
blocking task start before queueing behind it. That is a coin flip, not a wait: on a loaded
|
|
584
|
+
machine the 5 ms can elapse before the heavy task reaches its first `await`, so the lane is still
|
|
585
|
+
free and the cheap poll runs first -- `['cheap','heavy']` instead of `['heavy','cheap']`.
|
|
586
|
+
Measured across this session's logs: **4 failures in ~40 full-suite runs, 0 in 8 isolated runs**,
|
|
587
|
+
which is exactly the signature of a load-dependent race and exactly why it kept looking like
|
|
588
|
+
"just a flake". Both now await a promise the task itself resolves once it is genuinely holding
|
|
589
|
+
the lane. Nothing in `server/rpc/` changed: the lane was never wrong, the test was.
|
|
590
|
+
|
|
591
|
+
- [ ] Mempool **add/remove stream** is not real: the node refuses
|
|
592
|
+
`zmqpubsequence` (documented in its `docs/RPC_LIVE_NODE.md` slice 19 — it can
|
|
593
|
+
publish adds but has no clean "removed" choke point). So "realtime mempool"
|
|
594
|
+
here is a 60 s verbose poll plus log-derived ingest rate, not a per-tx stream.
|
|
595
|
+
The UI must not imply per-transaction add/remove events.
|
|
596
|
+
**Since 2026-09-09 the payload says so itself:** `/api/mempool` carries
|
|
597
|
+
`feed:{kind:'poll', cadenceSec:60, streamAvailable:false, why:…}`, naming the missing
|
|
598
|
+
removal choke point, so "don't imply a stream" is a field rather than a style guide
|
|
599
|
+
(`scripts/smoke.sh` asserts it). A poll-diff of successive `getrawmempool` snapshots
|
|
600
|
+
would report *some* removals and is deliberately not done: it would present
|
|
601
|
+
eviction as a remove only when the poll happened to straddle it, which is a fabricated
|
|
602
|
+
stream with better optics.
|
|
603
|
+
Re-checked 2026-09-11 on `deploy-20260910ag`: `getzmqnotifications` answers `[]`, and
|
|
604
|
+
none of the 165 methods is a sequence feed (MEASUREMENTS 27).
|
|
605
|
+
**Corrected 2026-09-13 -- that is a fact about THAT node, not about the feature.** Bitcoin
|
|
606
|
+
Core on an Umbrel answers `getzmqnotifications` with all five publishers:
|
|
607
|
+
`pubhashblock` 28334, `pubrawblock` 28332, `pubrawtx` 28333, `pubhashtx` 28336 and
|
|
608
|
+
**`pubsequence` 28335** -- the per-transaction add/remove/replace stream this entry says does
|
|
609
|
+
not exist. It does exist on Core; the experimental build is what lacked it. (Same shape of
|
|
610
|
+
error as the `getrawmempool … depends` claim corrected in MEASUREMENTS 26b the same day: a
|
|
611
|
+
true measurement on one node written up as a general limitation.)
|
|
612
|
+
What is still true is that we cannot *reach* it by default: bitcoind binds `0.0.0.0:2833x`
|
|
613
|
+
**inside its container**, and Umbrel publishes only 8332 to the LAN -- measured from this host,
|
|
614
|
+
all five ports answer `Connection refused` while 8332 is open. So a ZMQ feed has to be an
|
|
615
|
+
opt-in accelerator for operators who publish those ports, with the 20 s poll as the floor for
|
|
616
|
+
everyone else. The `feed:{kind:'poll', …}` field stays honest either way; it would report
|
|
617
|
+
`kind:'zmq'` when a stream is actually connected.
|
|
618
|
+
**2026-09-14:** BlockYard now runs on the node's machine, so the container objection is gone:
|
|
619
|
+
`zmqpubsequence=tcp://127.0.0.1:28335` in `bitcoin.conf` puts the stream within reach. What is
|
|
620
|
+
left is the client -- ZMTP is small enough to write without a dependency -- and the poll stays
|
|
621
|
+
the floor for a node that does not publish. Open, and now buildable.
|
|
622
|
+
- [x] No per-peer **byte** counters: **closed on `deploy-20260910ag`, still
|
|
623
|
+
build-dependent.** 2026-09-11 09:22:05Z: `getpeerinfo` 9 rows, and
|
|
624
|
+
`getconnectioncount` 9. `bytesrecv` summed to **109,027,561** against
|
|
625
|
+
`getnettotals` **109,036,122** 1.5 s later (99.992%; sent 34,804,574 vs 34,807,044),
|
|
626
|
+
and again 99.997% with 8 rows at 09:23:36Z (MEASUREMENTS 27). History kept so nobody
|
|
627
|
+
re-derives it: `deploy-20260907a` answered `[]` (so there per-peer bytes existed only
|
|
628
|
+
in the log's `[dlc] w<N>` lines); `deploy-20260908a` gave 4 rows for 6 connections;
|
|
629
|
+
the ~05:47 bench build gave 21 rows with `bytesrecv` up to 201,608,074 and a
|
|
630
|
+
vendor download-worker marker, summing to 70.29% of `getnettotals`.
|
|
631
|
+
**Why the ~30% "gap" vanished:** it was not filled. This build's `getnettotals`
|
|
632
|
+
forgets closed peers too, so rows and total agree by both leaving them out. That is a
|
|
633
|
+
divergence from Core, and it broke the rate maths (open item under Security /
|
|
634
|
+
correctness). Measured on a synced node only: no download-worker rows
|
|
635
|
+
(`id` ≥ 100000) were present, because the vendor download-info method was inactive.
|
|
636
|
+
- [x] No per-peer **relay transaction** counts -- **closed 2026-09-14:** Bitcoin Core exposes no
|
|
637
|
+
such figure either (`relaytxes` is a boolean), the panel claims none, and the log legs that
|
|
638
|
+
carried one belonged to the experimental node. Not in `getpeerinfo` on any build
|
|
639
|
+
measured (`[txrelay]` legs are the only source), and no method exposes them.
|
|
640
|
+
Re-checked on `deploy-20260910ag` 2026-09-11 09:22:05Z: rows carry `relaytxes: true`,
|
|
641
|
+
a per-peer boolean, and no count.
|
|
642
|
+
- [x] **Three RPC sources on `deploy-20260910ag` that nothing reads yet** -- **closed 2026-09-14
|
|
643
|
+
by scope:** all three are the experimental node's own methods and fields; Bitcoin Core has none
|
|
644
|
+
of them, and Core is what is supported. Kept for the record
|
|
645
|
+
(opportunities, not features; MEASUREMENTS 27). Verified by `grep` over `server/` on
|
|
646
|
+
2026-09-11: no call site for either method and no read of either field. The only
|
|
647
|
+
mention is the allowlist comment admitting that vendor read method to the read-only
|
|
648
|
+
console via its vendor read prefix.
|
|
649
|
+
- **A vendor download-info method** (0.175 ms; read-only, shared memory). It is aimed
|
|
650
|
+
at what RPC-only mode loses per MEASUREMENTS 23. By source, **not yet measured while
|
|
651
|
+
active**, it would supply: the worker ban count (`banned`); per-download-peer
|
|
652
|
+
identity and rate (`peers[].addr/subver/bytes_recv/bps_recv`); and the download's
|
|
653
|
+
own progress (`applied`, `end_height`, `window`, `first_hole`, `staged`,
|
|
654
|
+
`stall_evictions`). It does not supply the node's own ETA, disk writes, relay legs,
|
|
655
|
+
the accept/reject breakdown, archive holes or `sync_failing`, so those stay
|
|
656
|
+
log-only. Idle it answers `{active:false, bytes_total:0}`, so a poller can call it
|
|
657
|
+
unconditionally.
|
|
658
|
+
- **The build's own commit / dirty attestation fields** in `getnetworkinfo`: build
|
|
659
|
+
attestation over RPC, which MEASUREMENTS 15/18/23 said did not exist. It could stamp
|
|
660
|
+
provenance and history rows, and invalidate cached capability answers on change. It
|
|
661
|
+
is not a reason to stop re-asking: §23 saw a counter change inside one process.
|
|
662
|
+
- **`getorphantxs 1`** (~0.2 ms): `{txid, bytes, parents, age_ms}`. `parents` is the
|
|
663
|
+
count of missing inputs, i.e. *why* a transaction is stuck. It is an RPC source for
|
|
664
|
+
the orphan figure that is log-only today.
|
|
665
|
+
- [x] **The block map's cap is configured and measured.** `store.blockMapCap`
|
|
666
|
+
(default 12,000) replaces the hard-coded 3,000; the cost was measured — 12,000 rows =
|
|
667
|
+
**3.6 MB** (~310 B/row), so the old cap was saving ~2.7 MB while evicting blocks the
|
|
668
|
+
72 h retention would have kept. Trimming is a named method (`trimBlockMap()`), evicts
|
|
669
|
+
to 90% of the cap so it cannot thrash, keeps the tip, counts evictions, raises an
|
|
670
|
+
info quality flag, and publishes `{size, cap, evicted, oldestHeight}` in the snapshot.
|
|
671
|
+
Why it mattered: `gapSec` for the oldest surviving row is computed against the previous
|
|
672
|
+
height, and after a silent eviction that predecessor was gone.
|
|
673
|
+
- [x] **Suppressed during IBD, with the reason in the payload.** The figure is
|
|
674
|
+
difficulty × 2^32 ÷ the gap between blocks *we have*, which during IBD is the apply rate
|
|
675
|
+
wearing a network label — millions of EH/s. It is withheld (`null`) when
|
|
676
|
+
`initialblockdownload` is true or the node trails its own headers by more than 6, and
|
|
677
|
+
`hashrateNote` carries the sentence, so the UI renders `–` with an explanation rather
|
|
678
|
+
than a large orange number. A synced node still gets the figure, because there it is
|
|
679
|
+
honest. (The `2^32` was missing from the estimator itself until 2026-09-12 — a separate
|
|
680
|
+
defect, and one this page described correctly while the code did not: it understated the
|
|
681
|
+
rate by 4.29 billion and put `0.0 EH/s` on the page. Fixed and cross-checked against the
|
|
682
|
+
node's own `getnetworkhashps`, which the monitor had never read.)
|
|
683
|
+
- [x] **Two real nodes are wired, tested, and no longer the default.** With both
|
|
684
|
+
configured the monitor was verified against production *and* a live IBD node:
|
|
685
|
+
per-node series rings, the node switcher (which collapses to a single labelled
|
|
686
|
+
state dot at one node), `?node=` on every state fetch, and per-node SSE filtering.
|
|
687
|
+
The benchmark node was then removed from the defaults on measured grounds — RPC
|
|
688
|
+
latency ~18–32 s with 90 s timeouts, 25 failed tier runs and 9+ harness restarts in
|
|
689
|
+
about an hour, against production's 0 errors / ≤22 ms on the same code — and because
|
|
690
|
+
polling a benchmark from the machine running it consumes the single-threaded RPC lane
|
|
691
|
+
the benchmark needs (`MEASUREMENTS 21`). Multi-node stays a supported configuration:
|
|
692
|
+
`config.js` carries the exact `config/local.json` block to restore it, and
|
|
693
|
+
`test/config-nodes.test.js` asserts both halves — production-only by default, and
|
|
694
|
+
multi-node still working when configured.
|
|
695
|
+
- [x] **`/api/block` and `/api/tx`, plus a drill-down panel on Chain & Sync.**
|
|
696
|
+
Height or hash (blank = tip); header, `getblockstats`, and a page of 50 txids rendered
|
|
697
|
+
as buttons; a transaction pane drawn from the node's own verbosity-1 decode. Two
|
|
698
|
+
constraints, both about the node's single lane: **verbosity 2 is never asked**
|
|
699
|
+
(measured: 11 MB of hex per block, and it still omits Core's fee fields — the fake node
|
|
700
|
+
now *refuses* it so a test fails if the monitor ever asks), and no transaction hex is
|
|
701
|
+
ever proxied. The fee is absent and `notReported` says why: computing it needs every
|
|
702
|
+
input's prevout, N more turns on a one-threaded server, and this node omits it anyway.
|
|
703
|
+
- [x] **Unseen log lines are now counted *by tag*, not only by ratio.** The ratio
|
|
704
|
+
said coverage had fallen; it did not say what arrived, which is the difference
|
|
705
|
+
between "go read 5,000 lines" and "the node started printing `[migratetx]`". The
|
|
706
|
+
monitor keeps a census of lines no rule claimed — tag, count, first/last seen, one
|
|
707
|
+
sample line — publishes the top 8 in `log.unclaimed`, and raises `log-new-tag` when a
|
|
708
|
+
single unknown tag reaches 25 lines **while the rest of the log still parses**
|
|
709
|
+
(≥50% claimed). That second condition is what separates "the node added a subsystem"
|
|
710
|
+
from "the node rewrote everything", which `log-unparsed` already covers, and better.
|
|
711
|
+
Untagged lines are counted under `(untagged)` rather than dropped: a format change
|
|
712
|
+
with no tag is exactly as much news. Still unparsed on purpose: one-shot `[config]`,
|
|
713
|
+
`[boot]`, `[wallet]`, `[tor]`, `[serve]` lines carrying no figure any panel wants.
|
|
714
|
+
- [x] **History rings no longer average two daemons.** Rows are stamped with their
|
|
715
|
+
node and reads are filtered via `History.forNode(id)`; `history.summary()` reports
|
|
716
|
+
`unattributed` and `nodes` per ring. The measured mix this replaces: 2,308 production
|
|
717
|
+
rows against 1,816 bench rows in one `peers` ring, drawn as one line, with
|
|
718
|
+
`last()`/`tail()` returning whichever node wrote last (`MEASUREMENTS 20`).
|
|
719
|
+
|
|
720
|
+
Two consequences, both deliberate:
|
|
721
|
+
- **Pre-tagging history is excluded from per-node charts**, not guessed into them.
|
|
722
|
+
14,177 `node` / 18,022 `peers` / 16,384 `rpc` rows stay on disk and still count in
|
|
723
|
+
the unfiltered `summary()`; node-filtered reads refuse to claim them, because
|
|
724
|
+
picking an owner for them would put bench traffic on a production chart. Per-node
|
|
725
|
+
ranges refill from the restart: minutes for 1 h, up to a day/week for the long
|
|
726
|
+
windows. That is the honest cost of having stored the wrong thing, and it is
|
|
727
|
+
visible in the API rather than being discovered as missing history.
|
|
728
|
+
- **One ring per series is kept**, so two nodes halve each other's reach back in
|
|
729
|
+
time. Per-node rings would multiply memory per node and orphan a removed node's
|
|
730
|
+
history; the trade-off is asserted in `test/node-series.test.js` rather than left
|
|
731
|
+
as an opinion.
|
|
732
|
+
- [x] **The peer identity table exists.** The `[dl]` identity rows — user agent,
|
|
733
|
+
protocol version, the peer's own height, direction — are the only such figures on a
|
|
734
|
+
build where `getpeerinfo` answers `[]`, and they had been parsed, merged onto peer
|
|
735
|
+
rows, and returned by the API with nowhere to be seen: data collected and dropped on
|
|
736
|
+
the floor. The Peers page now has the table, flags a peer behind our tip, and says
|
|
737
|
+
which source it fell back to when no identity line is in the tail window. The fake node
|
|
738
|
+
was extended to print these lines, so the table is exercised by dev and smoke rather
|
|
739
|
+
than existing only in theory.
|
|
740
|
+
- [x] **They are in the `detail` view, each with its own label — not in the
|
|
741
|
+
strip.** `[dlc] ==` (the download worker's own percent, stored/total, its own ETA),
|
|
742
|
+
`[utxo_live] catchup progress:` (the applying thread's), and `[dl] updating utxo`
|
|
743
|
+
(validation throughput) are three threads reporting three rates; the strip's ≤60 px
|
|
744
|
+
budget has no room for a fourth figure and rules 4/9 forbid merging them. They now
|
|
745
|
+
appear as three separately-labelled rows behind `detail`, where "not printed by this
|
|
746
|
+
build" is a possible answer and the absence is visible.
|
|
747
|
+
- [ ] **An interrupted index build starts over.** Noted 2026-09-14. `buildIndex` empties its output
|
|
748
|
+
directory before it begins (`server/chain/index/build.js`), and the server treats an index as
|
|
749
|
+
built only when `manifest.json` exists (`server/main.js`), so a build stopped by Ctrl-C, a crash
|
|
750
|
+
or a reboot keeps nothing of its scan: the installer says so when it is stopped, and the server's
|
|
751
|
+
background build begins again from the first file on the next start. A resumable build would
|
|
752
|
+
need the per-file bucket output kept and a manifest of which files are done. A nicety on NVMe
|
|
753
|
+
(30 minutes on 16 workers), a real cost on the spinning disks the installer allows one worker for.
|
|
754
|
+
|
|
755
|
+
## Deliberate non-gaps (do not "fix")
|
|
756
|
+
|
|
757
|
+
- Upload rate is **not** estimated, and on the newest build there is a proof it
|
|
758
|
+
cannot be: 12,896,531,244 bytes received against 1,129 sent with 21 peers
|
|
759
|
+
connected. No source has it; see `docs/MEASUREMENTS.md` §4 and §15.
|
|
760
|
+
- Peer identity comes from the log, not from a synthesized table; see §3.
|
|
761
|
+
- The two sync percentages stay separate; see `docs/RULES.md` rule 9.
|
|
762
|
+
- The mempool scatter is absent from SSE frames; see rule 6.
|
|
763
|
+
|
|
764
|
+
- [x] **Multi-node is now exercised by the tests, not only by config.** The register
|
|
765
|
+
previously said multi-node "exists in config but is only exercised with one node".
|
|
766
|
+
`test/helpers/http.js` boots the real app with two fake daemons and asserts both are
|
|
767
|
+
monitored, that each `/api/state?node=` answers for its own node, that the sync object
|
|
768
|
+
carries the node it describes, that an optional node degrades to a note rather than a
|
|
769
|
+
red app, and that an unknown node is refused on both the stream and the read model
|
|
770
|
+
(`test/drilldown.test.js`). The measured 2,308-against-1,816-row mixing that motivated
|
|
771
|
+
per-node rings is still the reason the assertion exists.
|
|
772
|
+
|
|
773
|
+
## Node-side issues observed (not this repo's to fix)
|
|
774
|
+
|
|
775
|
+
- `getpeerinfo` empty while `getconnectioncount` is 13 (§3): **build-dependent**.
|
|
776
|
+
`deploy-20260907a` answered `[]` (2026-09-08 04:16 and 05:20), and so did the process
|
|
777
|
+
running on 2026-09-09 (`[]` with 17 connections at 17:44Z, §23). `deploy-20260908a`:
|
|
778
|
+
4 rows for 6 connections (2026-09-08 07:56, §18). `deploy-20260910ag`: 9 rows for 9
|
|
779
|
+
connections (2026-09-11 09:22:05Z, §27).
|
|
780
|
+
- `getnettotals` zeroed (§4): **build- and time-dependent**. 0/0 on `deploy-20260907a`
|
|
781
|
+
(2026-09-08, repeatedly). 0/0 → 23,955,131 received mid-uptime in one process on
|
|
782
|
+
2026-09-09 (§23). Nonzero on `deploy-20260910ag` (109,036,122 received at 09:22:07Z
|
|
783
|
+
on 2026-09-11), but it is a sum over **live** peers plus download bytes: it fell to
|
|
784
|
+
89,979,098 by 09:23:03Z when one peer left (§27). That divergence from Core's
|
|
785
|
+
lifetime counter is documented in the node's own source. Worth stating in its
|
|
786
|
+
`docs/RPC_LIVE_NODE.md` too, because every consumer that differences it is affected.
|
|
787
|
+
- `getblock verbosity=2` omits `fee`/`deltafee` that Core includes, and returns
|
|
788
|
+
11 MB of hex for one block (§6).
|
|
789
|
+
- `getrawmempool` verbose omits the ancestor/priority fields Core emits (§5). Unchanged
|
|
790
|
+
on `deploy-20260910ag`: 19,014 entries, keys `vsize/weight/time/fees.base`, 0.144 s
|
|
791
|
+
(2026-09-11 09:23:08Z). `getmempoolentry` does carry the full set (`depends`,
|
|
792
|
+
`ancestorcount`, `fees.ancestor`, …) in 0.69 ms, per txid (§27).
|
|
793
|
+
- `help <method>` carries no usage text on `deploy-20260910ag`: the same boilerplate for
|
|
794
|
+
every method. It is also cut at 255 characters, mid-word, for 9 of 11 sampled
|
|
795
|
+
methods, which truncates the pointer to `docs/RPC_LIVE_NODE.md` it exists to give
|
|
796
|
+
(2026-09-11, §27). Method safety has to be judged from the node's source.
|
|
797
|
+
- `getpeerinfo` **does** return rows on the 2026-09-08 03:02 bench build — but its
|
|
798
|
+
`bytessent`/`bytesrecv` sum to exactly `getnettotals` (1,153 / 3,232 bytes) on
|
|
799
|
+
peers last heard from 4,443 s ago, while the same run moved ~47 GB at 11.2 MB/s.
|
|
800
|
+
Still control-channel-only bytes; production still answers `[]` (§11).
|
|
801
|
+
- The node's `[dlc]` tick prints `(median 353.4)` **with no unit**, so the monitor
|
|
802
|
+
cannot report it as a rate without inventing one. If it is meant as KB/s, print
|
|
803
|
+
the unit (§11).
|
|
804
|
+
- `console.log` is block-buffered when stdout is a file: 2,719 bytes frozen for 18
|
|
805
|
+
minutes while the chain advanced ~15,000 blocks. Line flush should be unbuffered
|
|
806
|
+
or line-buffered when the destination is not a tty (§11).
|
|
807
|
+
- The benchmark harness truncates `console.log` on restart and archives the previous
|
|
808
|
+
run as `console.runNNN.log`, so a follower sees a truncation with counters back at
|
|
809
|
+
zero rather than a rotation. A stable path (a symlink, or append) would let the
|
|
810
|
+
monitor keep one continuous series per node.
|
|
811
|
+
- Production node's `[check] block data is NOT laid out monotonically (first
|
|
812
|
+
break at height 964924)` warning, present repeatedly in its log: the node itself
|
|
813
|
+
says truncation and pruning will refuse to run. Surfaced as `archive-hole`.
|