frugal-iot-server 0.3.5 → 0.3.6

package/INSTALLATION.md

@@ -5,40 +5,61 @@ an MQTT broker (Mosquitto), the Frugal IoT server, the logger that writes sensor
 and the web UI. Once installed, the Pi works **offline** — your sensor nodes talk to the Pi,
 and you view the data from a phone or laptop on the same Wi-Fi. No internet needed after setup.
 
-Two hardware paths are covered:
-
-* [Raspberry Pi 4](#part-a--raspberry-pi-4) — the recommended, tested path.
-* [Raspberry Pi Zero W](#part-b--raspberry-pi-zero-w) — smaller and cheaper, **not yet tested**, see the open questions in that section.
-
 If you already have a working Linux server (not a Pi), you do not need this document —
 see [README.md](https://github.com/mitra42/frugal-iot-server/blob/main/README.md) instead.
 
-> **Status:** the Raspberry Pi 4 path has been followed end to end on hardware (see
-> [Tested on](#tested-on)), and corrected from what that run found. Part B, and HTTPS/OTA in step
-> A10, are not written yet. Remaining uncertainties are listed under
-> [Open questions](#open-questions) — please add your findings there as you go.
+**Read the steps in order.** Anything specific to one board is marked in the step it belongs to,
+like this:
+
+> **Pi Zero W:** extra detail that only applies to a Zero W. Skip these on a Pi 4.
+
+> **Status:** followed on hardware — a Raspberry Pi 4 through step 9, and a Pi Zero W through step 8
+> (see [Tested on](#tested-on)) — and corrected from what those runs found. HTTPS/OTA in step 10 is
+> not written yet. Remaining uncertainties are under [Open questions](#open-questions) — please add
+> your findings there as you go.
 
 **Already have a Frugal IoT server running and just want a newer version?** Skip everything below
 and go to [Upgrading](#upgrading).
 
+**If a step does not do what it says here**, once you reach step 4 you can run
+`npx --no frugal-iot-diagnostic` from your install directory — it inspects the whole installation and
+reports what looks wrong. See [When something does not work](#when-something-does-not-work).
+
 ---
 
-## Part A — Raspberry Pi 4
+## Which Raspberry Pi
 
-### A0. What you need before you start
+Any of these work. What differs is only how long the install takes, and how much of a fight it is:
+
+| Board | Install | Notes |
+| --- | --- | --- |
+| **Pi 4** (or Pi 5, Pi 3) | About half an hour | The straightforward path. 64-bit, everything installs as ready-made binaries. |
+| **Pi Zero 2 W** | Expect longer | 64-bit, so it should avoid the long compile below, but shares the Zero's 512 MB. Not yet tested. |
+| **Pi Zero W** (the original) | Allow an afternoon | 32-bit ARMv6. One dependency has to be compiled — about 40 minutes — and 512 MB of RAM is not enough without adding swap. Tested and works. |
+
+A Zero W runs the server perfectly well once installed. It is *installing* that is slow, because a
+single 1 GHz core has to unpack several hundred packages and compile one of them.
+
+### 0. What you need before you start
 
 **Hardware**
 
-* Raspberry Pi 4 Model B. Any RAM size — a running server uses around 250 MB, so even the 1 GB
-  model has plenty of room. (Measured on a 4 GB Pi 4: 231 MB in use with the server running.)
+* A Raspberry Pi, per the table above. Any RAM size — a running server uses around 250 MB.
+  (Measured on a 4 GB Pi 4: 231 MB in use with the server running.)
 * A microSD card. 8 GB is enough for the system and software — a working install occupies about
   5 GB — but sensor data accumulates on this card for as long as the server runs, so 16 GB or larger
   is the safer choice. Class 10 / A1 or better.
-* The official Raspberry Pi USB-C power supply (5 V / 3 A). Phone chargers frequently cause
-  random reboots and corrupted SD cards — this is the single most common cause of "it doesn't work".
+* The official Raspberry Pi power supply for your board — USB-C on a Pi 4, micro-USB on a Zero.
+  Phone chargers frequently cause random reboots and corrupted SD cards — this is the single most
+  common cause of "it doesn't work".
 * A way to write the SD card from your laptop: a built-in SD slot or a USB card reader.
 * Note that you will often need an adapter from the SD format the Pi uses to the SD format of most laptop readers.
 
+> **Pi Zero W:** its Wi-Fi is 2.4 GHz only, so it cannot see a 5 GHz-only network. It has no
+> Ethernet socket either, so the cable trick in step 2 is unavailable — have the micro-HDMI cable
+> and a keyboard to hand instead. Note micro-USB for power and a micro-USB OTG adapter for the
+> keyboard: different cables from a Pi 4.
+
 **Software and information**
 
 * A laptop or desktop (Mac, Windows or Linux) with [Raspberry Pi Imager](https://www.raspberrypi.com/software/) installed.
@@ -56,18 +77,20 @@ and go to [Upgrading](#upgrading).
 **Have these to hand in case the headless setup does not come up**
 
 * An Ethernet cable from the Pi to your router — the simplest way in if the Pi does not appear on
-  the Wi-Fi, and you can then sort the Wi-Fi out over SSH (step A2).
+  the Wi-Fi, and you can then sort the Wi-Fi out over SSH (step 2).
 * A micro-HDMI to HDMI cable plus a monitor, and a USB keyboard — for when the Pi is nowhere near the
   router, or you want to see boot messages.
 
-### A1. Write the operating system to the SD card
+### 1. Write the operating system to the SD card
 
-We use **Raspberry Pi OS Lite (64-bit)** — the version with no desktop. The Pi is a server;
+We use **Raspberry Pi OS Lite** — the version with no desktop. The Pi is a server;
 a desktop would only consume memory and SD card space.
 
 1. Insert the SD card into your laptop and start Raspberry Pi Imager.
-2. **Choose Device** → *Raspberry Pi 4*.
+2. **Choose Device** → your board.
 3. **Choose OS** → *Raspberry Pi OS (other)* → *Raspberry Pi OS Lite (64-bit)*.
+   * **On a Pi Zero W choose *Raspberry Pi OS Lite (32-bit)*** — the 64-bit images will not boot on
+     its ARMv6 processor. (A Pi Zero **2** W is 64-bit, so it takes the 64-bit image like a Pi 4.)
 4. **Choose Storage** → your SD card. Check the size shown matches your card — this erases it.
 5. Click **Next**. When asked *"Would you like to apply OS customisation settings?"*, choose
    **Edit Settings** — everything below depends on it.
@@ -81,7 +104,7 @@ a desktop would only consume memory and SD card space.
      produces a key that fails even though the password was right.
    * If the Pi does not join the network, a mistyped SSID or password is much the likeliest cause -
      neither is echoed back to you here, and Imager combines the two into a key, so a slip in either
-     looks the same later. Step A2 fixes it in a couple of minutes with `nmtui`.
+     looks the same later. Step 2 fixes it in a couple of minutes with `nmtui`.
    * A WPA3 network cannot use the key Imager derives at all, so on one of those expect to finish the
      Wi-Fi setup with `nmtui` regardless.
 10. Enable SSH, either choose *Use password authentication* or paste your public key if you already use SSH keys.
@@ -95,7 +118,7 @@ Eject the card, put it in the Pi, connect power.
 The first boot resizes the filesystem and reboots itself. Give it **two to three minutes** before
 expecting it to answer.
 
-### A2. Log in over the network
+### 2. Log in over the network
 
 From your laptop's terminal:
 
@@ -105,6 +128,8 @@ ssh pi@frugaliot.local
 
 Say `yes` to the fingerprint question, then give the password you set in Imager.
 
+**If that logged you in, go straight to step 3.** The rest of this step is for when it did not.
+
 **If `frugaliot.local` is not found**, the `.local` (mDNS) name is not reaching you. In order of ease:
 
 * Wait another minute and try again — the Pi may still be on its first boot.
@@ -114,7 +139,7 @@ Say `yes` to the fingerprint question, then give the password you set in Imager.
   up an address, and `ssh pi@frugaliot.local` then works over the cable. This is the least effort
   way in if the Pi is within reach of the router, and once you are logged in you can sort the Wi-Fi
   out over SSH with `nmtui` as below, no monitor or keyboard needed. Unplug the cable afterwards and
-  check that Wi-Fi alone still gets you in.
+  check that Wi-Fi alone still gets you in. (Not an option on a Pi Zero — no Ethernet socket.)
 * Plug in the HDMI and keyboard, log in at the console, and run `ip addr` to read the IP address,
   and `sudo journalctl -b | grep -i wpa` to see why Wi-Fi failed. Use this when the Pi is nowhere
   near the router, or when you want to see boot messages.
@@ -150,7 +175,9 @@ the password — either way `nmtui` is the fix, and it is not worth more time th
 (`sudo nmcli device wifi list` shows each nearby network's SSID and whether it is WPA2 or WPA3;
 WPA3 cannot use a derived key at all.)
 
-Once logged in, bring the system up to date and reboot:
+### 3. Update the operating system and install the prerequisites
+
+Logged in, bring the system up to date and reboot:
 
 ```
 sudo apt update
@@ -174,49 +201,138 @@ which you may not have the password for.
 address does not change. Sensor nodes and phones then have something stable to talk to even where
 `.local` names do not work.
 
-### A3. Install the prerequisites
+Now the packages the server needs.
+
+**On a Pi 4** (or any 64-bit board):
 
 ```
 sudo apt install -y nodejs npm sqlite3 zsh
 node -v
 ```
 
+**On a Pi Zero W**, three more packages, because a 32-bit machine has to compile part of the server
+in step 4 and a Lite image has no compiler:
+
+```
+sudo apt install -y nodejs npm sqlite3 zsh build-essential python3-dev python3-setuptools
+node -v
+```
+
+What each is for:
+
 * `nodejs` — the server needs **Node 18 or later**. Current Raspberry Pi OS (Debian 13, trixie)
-  provides 20.19.2, which is fine. Older images shipped Node 18, also fine. If `node -v` reports
-  anything below 18, install a current version from
+  provides 20.19.2, which is fine, on 32-bit as well as 64-bit. Older images shipped Node 18, also
+  fine. If `node -v` reports anything below 18, install a current version from
   [NodeSource](https://github.com/nodesource/distributions) instead.
 * `npm` — installs the server; it is a separate package from `nodejs` on Debian.
 * `sqlite3` — the database the server keeps its accounts in.
-* `zsh` — the setup commands in step A6 are zsh scripts.
-
-### A4. Install the Frugal IoT server
+* `zsh` — the setup commands in step 6 are zsh scripts.
+* `build-essential`, `python3-dev`, `python3-setuptools` — only needed where something has to be
+  compiled. `python3-setuptools` is the non-obvious one: the build uses node-gyp 8, which imports
+  Python's `distutils`, removed in Python 3.12, and setuptools puts an importable `distutils` back.
+  Without it step 4 ends in `ModuleNotFoundError: No module named 'distutils'`.
+
+Now two settings that make the SD card last longer. A card wears out from being written to, and
+these are the two places the system writes constantly without being asked to:
+
+```
+# Cap the systemd journal, which by default is allowed to grow to a tenth of the card
+sudo mkdir -p /etc/systemd/journald.conf.d
+printf '[Journal]\nSystemMaxUse=16M\nSystemMaxFileSize=4M\n' | sudo tee /etc/systemd/journald.conf.d/frugal-iot.conf
+sudo systemctl restart systemd-journald
+
+# Swap out to the card only when there is genuinely no alternative
+echo 'vm.swappiness=1' | sudo tee /etc/sysctl.d/99-frugal-iot-swappiness.conf
+sudo sysctl --system | grep swappiness
+```
+
+The `grep` should print `vm.swappiness = 1`. Both survive a reboot. There is more about what wears a
+card out, and how to see whether yours is being written to hard, under
+[Wear and tear on the SD card](#wear-and-tear-on-the-sd-card) — but nothing else there needs doing
+during the install.
+
+> **Pi Zero W: add swap before going on.** 512 MB is not enough to unpack what step 4 downloads, and
+> running out does not fail cleanly — the board stops answering SSH and ping, and has to have its
+> power pulled. Raspberry Pi OS enables zram, which is **not** sufficient here, because compressed
+> RAM does not help when the working set is genuinely large. Add a real swap file (`dphys-swapfile`
+> is not on the Trixie Lite image, so make it directly):
+>
+> ```
+> sudo fallocate -l 2G /swapfile
+> sudo chmod 600 /swapfile
+> sudo mkswap /swapfile
+> sudo swapon /swapfile
+> free -h                                              # should show 2.0Gi of swap
+> echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
+> ```
+>
+> **No reboot needed** — `swapon` takes effect at once, which the `free -h` line confirms. The
+> `/etc/fstab` entry only matters later, so that the swap comes back after a reboot rather than
+> having to be turned on by hand. Go straight on to step 4.
+>
+> Swapping to an SD card is slow, and wears it. That is the trade: the install takes longer, but it
+> finishes instead of hanging. The `vm.swappiness=1` set above keeps the swap file available for
+> emergencies like this one without it being used routinely afterwards.
+
+### 4. Install the Frugal IoT server
 
 The server is an npm package. Make a directory for this server to live in and install it there —
 that directory will hold your configuration, your data, and your database, while npm looks after
 the software itself underneath it in `node_modules`.
 
+**On a Pi 4** (or any 64-bit board) — a few minutes, everything arrives as ready-made binaries and
+nothing is compiled:
+
 ```
 mkdir ~/frugal-iot
 cd ~/frugal-iot
 npm install frugal-iot-server
 ```
 
-That pulls in the web UI (`frugal-iot-client`) and the logger (`frugal-iot-logger`) as well.
-Expect a few minutes on a Pi. Then set the directory up:
+**On a Pi Zero W** — the extra flags make npm do one thing at a time, which lowers the peak memory
+as well as being kinder to the SD card. This is the slow part; leave it running:
 
 ```
-npx frugal-iot-init
+mkdir ~/frugal-iot
+cd ~/frugal-iot
+npm install --maxsockets 1 --no-audit --no-fund frugal-iot-server
+```
+
+That pulls in the web UI (`frugal-iot-client`) and the logger (`frugal-iot-logger`) as well.
+
+> **Pi Zero W: check that the compiled part actually built.** npm can report success while leaving
+> it out, and nothing else looks wrong when that happens — the server just refuses to start later.
+>
+> ```
+> node -e "require('sqlite3'); console.log('sqlite3 native module loads OK')"
+> ```
+>
+> If that throws `Could not locate the bindings file`, build just that package:
+>
+> ```
+> time npm rebuild sqlite3 --foreground-scripts
+> ```
+>
+> **This takes around 40 minutes on a Zero W** — it is compiling SQLite itself on one slow core. As
+> long as it is producing output it is working; `--foreground-scripts` is what lets you see that. If
+> the board locks up, pull the power, boot it, and run the same command again: work already done is
+> kept, so each attempt gets further.
+>
+> Most of what is being installed is not Frugal IoT. The logger depends on `firebase-admin`, which
+> brings in the Google Cloud SDK — around 40 packages and 30 MB — and that is what the board
+> struggles with. It is only used by organizations that configure a `firebase:` section.
+> TODO make that dependency optional in frugal-iot-logger, so small boards can skip it.
+
+Then set the directory up:
+
+```
+npx --no frugal-iot-init
 ```
 
 This copies in the configuration files, creates the `data`, `ota` and `config.d/organizations`
 directories, and creates the database. It never overwrites anything already there, so it is also
 what you run after an upgrade to pick up newly added configuration.
 
-> On a 64-bit Raspberry Pi OS this installs ready-made binaries and compiles nothing. If it ever
-> does stop while building `sqlite3`, no binary was available for your platform, so install the
-> compiler toolchain and try again: `sudo apt install -y build-essential python3` then
-> `npm install frugal-iot-server`.
-
 Everything from here on is run from `~/frugal-iot`, and `npx` is how you run the server's commands
 without having to know where npm put them.
 
@@ -240,10 +356,10 @@ works on the Pi.
 > `frugaliot.local` is known to work from a laptop and from an iPhone. Android phones generally
 > cannot resolve `.local` names, so if you will view the dashboard on Android, put the Pi's IP
 > address here instead — `broker: ws://192.168.1.42:9012`, using the address `ping -c1
-> frugaliot.local` reported in step A2 — and reserve that address in your router if you can, so it
+> frugaliot.local` reported in step 3 — and reserve that address in your router if you can, so it
 > does not change under you.
 
-### A5. Install and configure the MQTT broker (Mosquitto)
+### 5. Install and configure the MQTT broker (Mosquitto)
 
 Sensor nodes publish their readings to an MQTT broker; the Frugal IoT logger subscribes to it and
 writes the readings to disk; the web UI subscribes to it to show live values. On an offline Pi,
@@ -265,6 +381,12 @@ sudo cp extras/mosquitto.conf /etc/mosquitto/conf.d/frugal-iot.conf
 (Everything in `/etc/mosquitto/conf.d/` is read in addition to the packaged
 `/etc/mosquitto/mosquitto.conf`, which keeps its own settings for logging and persistence.)
 
+That file also turns off the broker's per-connection logging, because a node on a weak signal
+reconnects constantly and each of those lines is a write to the SD card. Nothing is lost — the
+readings themselves are recorded by the server, and everything else the broker says, including why
+it refused to start, still goes to `/var/log/mosquitto/mosquitto.log`. If you are chasing a node
+that keeps dropping off the network, comment out `connection_messages false` and restart the broker.
+
 That configuration names a password file, and Mosquitto will not start if the file is missing, so
 create an empty one. The accounts inside it get created for you in the next step:
 
@@ -279,7 +401,7 @@ instead of a `touch`, a `chown` and a `chmod`.
 > to whoever opened it, and the broker runs as the `mosquitto` user — so the file belongs to
 > `mosquitto`, and it lives under `/var/lib/mosquitto` (which that user owns) rather than
 > `/etc/mosquitto` (which root owns). `mosquitto_passwd` also writes a temporary backup file
-> alongside it, so it needs to write to that directory too, not just to the file. Step A6 runs it as
+> alongside it, so it needs to write to that directory too, not just to the file. Step 6 runs it as
 > the right user for you.
 
 Start the broker and have it start at every boot:
@@ -309,19 +431,24 @@ mosquitto_sub -h localhost -u nobody -P wrong -t '#'
 ```
 
 `Connection Refused: not authorised` is the **success** case here — the broker answered and
-requires an account. There are no accounts yet; step A6 creates the first one, and there is a
+requires an account. There are no accounts yet; step 6 creates the first one, and there is a
 fuller test at the end of it.
 
-### A6. Create your accounts and your organization
+> If Mosquitto did not start, `npx --no frugal-iot-diagnostic` will tell you why in one step — most often
+> that the password file its configuration names does not exist. Note that `systemctl status` and
+> the journal only show an exit code; the actual reason is in Mosquitto's own log, which the
+> diagnostic reads for you. See [When something does not work](#when-something-does-not-work).
 
-The database was created by `npx frugal-iot-init` in step A4, holding two accounts. One is
+### 6. Create your accounts and your organization
+
+The database was created by `npx --no frugal-iot-init` in step 4, holding two accounts. One is
 `everyone`, which nobody logs in as — it exists so that permissions granted to all logged-in users
 have somewhere to live. The other is `superuser`, this server's administrator, which is given admin
 rights over every organization you create. It starts with no password and cannot be logged into
 until you give it one:
 
 ```
-npx frugal-iot-setpassword superuser "<a-good-password>"
+npx --no frugal-iot-setpassword superuser "<a-good-password>"
 ```
 
 Use the same command later if you ever need to reset a password — for `superuser` or for any
@@ -330,19 +457,18 @@ other account.
 Frugal IoT groups devices as **organization → project → device**. Create yours:
 
 ```
-npx frugal-iot-addorganization dev "My Farm" you@example.com +61123456789 "<broker-password>"
+npx --no frugal-iot-addorganization myfarm "My Farm" you@example.com +61123456789 "<broker-password>"
 ```
 
 The arguments are: organization id, display name, your email, your phone (`+` and digits only),
-and a password. That one command writes `config.d/organizations/dev.yaml`, creates a login account
-named after the organization (`dev`), grants it admin rights, creates its OTA directory, **and adds
+and a password. That one command writes `config.d/organizations/myfarm.yaml`, creates a login account
+named after the organization (`myfarm`), grants it admin rights, creates its OTA directory, **and adds
 the organization's account to the broker's password file** — which is why the broker had to be
 installed first.
 
-* **Organization id `dev`** — must be 1–10 lower-case letters or digits. It becomes the first part
+* **Organization id `myfarm`** — must be 1–10 lower-case letters or digits. It becomes the first part
   of every MQTT topic, so it must match what your sensor nodes are configured to publish to.
-  `dev` is the default in the Frugal IoT node firmware, so it is the safe choice for a first install.
-* **The password** goes into `config.d/organizations/dev.yaml` as `mqtt_password` and into the
+* **The password** goes into `config.d/organizations/myfarm.yaml` as `mqtt_password` and into the
   broker's password file, so it is a *machine* credential: the server's logger, your sensor nodes,
   and any browser showing this organization's dashboard all authenticate to the broker with it.
   Treat it as shared, not personal.
@@ -353,45 +479,47 @@ Tell Mosquitto to re-read the password file, so the new account works:
 sudo systemctl restart mosquitto
 ```
 
-**Give your login its own password.** That command set the `dev` *web login* password to the same
+**Give your login its own password.** That command set the `myfarm` *web login* password to the same
 string as the broker password. They serve completely different purposes, so change the login one
 now to something only you know — the broker credential is unaffected, and nothing needs to be kept
 in step:
 
 ```
-npx frugal-iot-setpassword dev "<your-own-login-password>"
+npx --no frugal-iot-setpassword myfarm "<your-own-login-password>"
 ```
 
 **Now the full broker test.** In your SSH session subscribe as the organization, using the broker
 password you chose above:
 
 ```
-mosquitto_sub -h localhost -u dev -P '<broker-password>' -t '#' -v
+mosquitto_sub -h localhost -u myfarm -P '<broker-password>' -t '#' -v
 ```
 
 It should sit there silently — no error, no exit. Open a **second** SSH session to the Pi and
 publish something:
 
 ```
-mosquitto_pub -h localhost -u dev -P '<broker-password>' -t 'dev/test/hello' -m '42'
+mosquitto_pub -h localhost -u myfarm -P '<broker-password>' -t 'myfarm/test/hello' -m '42'
 ```
 
-`dev/test/hello 42` appearing in the first window proves the account, the password file and the
+`myfarm/test/hello 42` appearing in the first window proves the account, the password file and the
 port 1883 listener your sensor nodes use are all working. The WebSocket listener on 9012 gets
-exercised by the browser in step A7. Leave the subscriber running if you like — it is a useful
+exercised by the browser in step 7. Leave the subscriber running if you like — it is a useful
 window onto what your nodes are doing. (`Ctrl-C` stops it.)
 
 If you get `Connection Refused: not authorised`, the password does not match the one in
-`config.d/organizations/dev.yaml`, or Mosquitto has not re-read the file since it changed.
+`config.d/organizations/myfarm.yaml`, or Mosquitto has not re-read the file since it changed.
+`npx --no frugal-iot-diagnostic` checks this for every organization you have, and reports which ones the
+broker actually accepts.
 
-### A7. Start the server by hand and check it
+### 7. Start the server by hand and check it
 
 From your install directory — `npx` looks for the server in the current directory's `node_modules`,
 so this only works there:
 
 ```
 cd ~/frugal-iot
-npx frugal-iot-server
+npx --no frugal-iot-server
 ```
 
 > If npx answers with `Need to install the following packages: frugal-iot-server` and asks to
@@ -404,7 +532,7 @@ It lists each configuration file as it reads it, then:
 readYamlConfigFile ./config.yaml
 readYamlConfigDir ./config.d
     ... one line per configuration file ...
-Broker ws://frugaliot.local:9012 - organizations: dev
+Broker ws://frugaliot.local:9012 - organizations: myfarm
 Doing OTA updates at /ota_update from /home/pi/frugal-iot/ota
 Serving /node_modules from ./node_modules
 User Database exists
@@ -416,16 +544,16 @@ Mounted API routes at /api
 Added API error handler
 Serving /data from ./data
 Server starting on port 8080
-mqtt dev connecting
-mqtt dev connect
-Subscribing topic dev/# 0
+mqtt myfarm connecting
+mqtt myfarm connect
+Subscribing topic myfarm/# 0
 ```
 
 Check the `Broker` line names your own broker and your organization. The lines that matter most are
-the last three: `mqtt dev connect` means the server reached the broker and authenticated, and
-`Subscribing topic dev/#` means it is listening for your nodes. If instead you see repeated
-`mqtt dev close`, `offline`, or `Not authorized`, the broker URL or the password is wrong — recheck
-`config.d/mqtt.yaml` and that the password in `config.d/organizations/dev.yaml` matches the `dev`
+the last three: `mqtt myfarm connect` means the server reached the broker and authenticated, and
+`Subscribing topic myfarm/#` means it is listening for your nodes. If instead you see repeated
+`mqtt myfarm close`, `offline`, or `Not authorized`, the broker URL or the password is wrong — recheck
+`config.d/mqtt.yaml` and that the password in `config.d/organizations/myfarm.yaml` matches the `myfarm`
 broker account.
 
 Once nodes are reporting, each reading is logged as it arrives, so this output keeps scrolling.
@@ -442,7 +570,7 @@ http://frugaliot.local:8080
 > local networks"* the first time. **Allow it** — without that permission the browser cannot look up
 > `frugaliot.local`, nor reach the broker at that name, so the page and the live data both fail.
 
-You land on the Frugal IoT home page. Click **Dashboard**, and log in as username `dev` with the
+You land on the Frugal IoT home page. Click **Dashboard**, and log in as username `myfarm` with the
 login password you set with `frugal-iot-setpassword` — not the broker password. (`superuser` and its
 password work too.)
 
@@ -454,12 +582,17 @@ actually resolve: a phone that cannot look up `.local` names needs the Pi's IP a
 
 Until a sensor node reports in there will be no data to look at, but the dashboard should load.
 
+> Anything wrong here — the server not starting, `mqtt myfarm close` instead of `connect`, the page not
+> loading, the MQTT status not reaching *connected* — is worth a `npx --no frugal-iot-diagnostic` in
+> another terminal before digging in by hand. It tests the same chain from the Pi's side: broker
+> logins, the three ports, and the web server.
+
 Stop the server with `Ctrl-C` before continuing.
 
-### A8. Run the server as a service
+### 8. Run the server as a service
 
 So that it starts automatically at boot and restarts if it crashes. The file that
-`npx frugal-iot-init` put in `extras/` already describes this installation — user `pi`, installed
+`npx --no frugal-iot-init` put in `extras/` already describes this installation — user `pi`, installed
 into `/home/pi/frugal-iot` — so it needs no editing:
 
 ```
@@ -478,7 +611,7 @@ systemctl status frugaliot
 > `sudo systemctl restart frugaliot`. `WorkingDirectory` is the important one: it is where the
 > server looks for its configuration and database.
 
-To watch its log output, which is where the `mqtt dev connect` and incoming-reading messages now go:
+To watch its log output, which is where the `mqtt myfarm connect` and incoming-reading messages now go:
 
 ```
 journalctl -u frugaliot -f
@@ -487,7 +620,7 @@ journalctl -u frugaliot -f
 Reboot the Pi (`sudo reboot`), wait a couple of minutes, and check `http://frugaliot.local:8080`
 still answers. Your server is now installed.
 
-### A9. Point your sensor nodes at the Pi
+### 9. Point your sensor nodes at the Pi
 
 Your ESP8266/ESP32 nodes are told which broker to use in their sketch — `main.cpp`, or the `.ino`
 file if you build in the Arduino IDE. Look for a line like:
@@ -499,29 +632,29 @@ frugal_iot.configure_mqtt("frugaliot.naturalinnovation.org", "dev", "public");
 and point it at your Pi instead:
 
 ```cpp
-frugal_iot.configure_mqtt("frugaliot.local", "dev", "<broker-password>");
+frugal_iot.configure_mqtt("frugaliot.local", "myfarm", "<broker-password>");
 ```
 
 The three arguments are the broker's host, the organization, and that organization's broker
-password. The organization must be the one you created in step A6, because it is the first part of
+password. The organization must be the one you created in step 6, because it is the first part of
 every topic the node publishes to, and the password is the *broker* password from that step — not
 the login password. Then rebuild and flash the node as usual.
 
 > If the node does not connect, try the Pi's IP address in place of `frugaliot.local`. Resolving
 > `.local` names needs mDNS support in the firmware, which is not something this guide has
-> confirmed; an IP address avoids the question entirely, which is why step A2 suggests reserving one
+> confirmed; an IP address avoids the question entirely, which is why step 3 suggests reserving one
 > for the Pi in your router.
 
-You can confirm nodes are reporting without involving the UI, using the subscriber from step A6:
+You can confirm nodes are reporting without involving the UI, using the subscriber from step 6:
 
 ```
-mosquitto_sub -h localhost -u dev -P '<broker-password>' -t '#' -v
+mosquitto_sub -h localhost -u myfarm -P '<broker-password>' -t '#' -v
 ```
 
 Every reading from every node should scroll past. Seeing anything here also proves the broker's
 port 1883 is reachable from off the Pi, which is what the nodes need.
 
-### A10. HTTPS and over-the-air firmware updates
+### 10. HTTPS and over-the-air firmware updates
 
 **To be written.** Everything above gives you a plain HTTP server on your local network, which is
 all an offline installation needs. HTTPS matters for two things:
@@ -563,6 +696,80 @@ the time of the phone or laptop viewing it, so it could hand that to the server,
 it whenever its own clock looks implausible (a date far in the past). Not accurate to the second,
 but close enough for sensor data.
 
+---
+### Wear and tear on the SD card
+
+SD cards wear out from being written to, and a server recording sensor readings writes all the time.
+We have not yet seen a card fail, but plan for one eventually: keep a copy of `config.d/` and
+`frugal-iot.db` somewhere else, and treat the readings in `data/` as valuable but not irreplaceable.
+
+A new installation is set up to write as little as it reasonably can, and the settings are worth
+knowing about, because turning one of them back on for debugging and forgetting is easy to do:
+
+| What | Where | Shipped as |
+| ---- | ----- | ---------- |
+| A line logged for every MQTT message received | `verbose:` in `config.d/logger.yaml` | `false` |
+| Readings held in memory and written out periodically instead of one at a time | `flushseconds:` in `config.d/logger.yaml` | `300` (5 minutes) |
+| A line logged for every web request | `morgan:` in `config.d/server.yaml` | `false` |
+| Old readings compressed, and deleted if the disk fills | `housekeeping:` in `config.d/server.yaml` | compress after 2 days, never delete, keep 10% free |
+| A line logged for every device connect and disconnect | `connection_messages` in `extras/mosquitto.conf` | `false` |
+| A capped systemd journal, and swapping only as a last resort | `/etc/systemd/journald.conf.d/` and `/etc/sysctl.d/`, both set in step 3 | 16 MB journal, `vm.swappiness=1` |
+
+Nothing in that table needs doing — step 3 and step 5 set all of it up. It is here because turning
+one of them back on for debugging and then forgetting is easy to do.
+
+`flushseconds` is the one with a cost attached: readings that have not been written out yet are only
+in memory, so pulling the power loses up to five minutes of them. Stopping the server properly
+(`sudo service frugaliot stop`, or a restart) writes them out first, and so does looking at a graph.
+
+On a Pi 4 there should be very little swapping in any case — the server and Mosquitto together are a
+small load for 1 GB or more. The Pi Zero W is the one to watch, since step 3 adds a 2 GB swap file to
+get through the install. To see what is actually happening on your board:
+
+```
+cd ~/frugal-iot
+npx --no frugal-iot-diagnostic
+```
+
+and read the **Wear on the SD card** section, which reports how much has been written since boot,
+whether the machine is swapping, and whether each of the settings above is in force. Run it twice a
+few days apart — the interesting number is the rate, not the total.
+
+## When something does not work
+
+From your install directory:
+
+```
+cd ~/frugal-iot
+npx --no frugal-iot-diagnostic
+```
+
+It only looks — it changes nothing — and it ends with a **Summary** of anything it recognises as
+broken. It covers most of the checks scattered through this guide, in one pass:
+
+* the machine, the OS, free memory and disk, and whether the clock is synchronized
+* the versions of node, npm, sqlite3, mosquitto, and of the three Frugal IoT packages
+* what this directory holds: configuration, database, accounts, organizations
+* Mosquitto's configuration, **which password file it names and whether that file exists**, who owns
+  it, and the accounts in it
+* whether Mosquitto and the `frugaliot` service are running, with the last lines of the journal
+  **and of Mosquitto's own log**, which is where its startup errors actually appear
+* whether the broker refuses a wrong password, and accepts each organization's real one
+* whether ports 1883, 9012 and 8080 are listening, and whether the web server answers
+* whether the broker host in `config.d/mqtt.yaml` resolves from this machine
+* how much data the logger has written
+
+Two things to know:
+
+* Mosquitto's log belongs to the `mosquitto` user. The script reads it with `sudo` where it can; if
+  it says it could not, run the whole thing as `sudo npx --no frugal-iot-diagnostic`.
+* The output is safe to paste into a bug report. Passwords are deliberately not printed — the
+  password file is listed by account name only, and organizations by name.
+
+If you have not reached step 4 yet, the command does not exist. Copy
+[scripts/diagnostic.zsh](https://github.com/mitra42/frugal-iot-server/blob/main/scripts/diagnostic.zsh)
+to the Pi and run `zsh diagnostic.zsh` instead.
+
 ---
 
 ## Upgrading
@@ -572,7 +779,7 @@ To move an existing server to a newer release, from the directory you installed
 ```
 cd ~/frugal-iot
 npm update frugal-iot-server
-npx frugal-iot-init
+npx --no frugal-iot-init
 sudo systemctl restart frugaliot
 ```
 
@@ -603,68 +810,43 @@ sudo cp extras/mosquitto.conf /etc/mosquitto/conf.d/frugal-iot.conf && sudo syst
 sudo cp extras/frugaliot.service /etc/systemd/system/ && sudo systemctl daemon-reload && sudo systemctl restart frugaliot
 ```
 
-> It does not compare `config.yaml`, `config.d/mqtt.yaml` or `config.d/server.yaml`, because those
-> hold your own settings and would differ every time. If a release note mentions a new setting in
-> one of them, compare it yourself:
+> It does not compare `config.yaml`, `config.d/mqtt.yaml`, `config.d/logger.yaml` or
+> `config.d/server.yaml`, because those hold your own settings and would differ every time. If a
+> release note mentions a new setting in one of them, compare it yourself:
 > `diff config.d/server.yaml node_modules/frugal-iot-server/config.d/server.yaml`
 
+Because your own configuration is left alone, an upgraded server goes on behaving exactly as it did,
+which for the settings under [Wear and tear on the SD card](#wear-and-tear-on-the-sd-card) means it
+goes on writing as often as it did. A server that has been running since before those settings
+existed keeps logging every message and every web request, and writing every reading as it arrives —
+deliberately, since quietening somebody's server without being asked is not a thing an upgrade should
+do. On a Pi, compare the two files above and copy across the settings you want.
+
 Your organizations, accounts, database and logged data are untouched by an upgrade — they live in
 this directory, not in `node_modules`.
 
 ---
 
-## Part B — Raspberry Pi Zero W
-
-**Not yet tested — do not follow this section expecting it to work.** It records what is known
-and what has to be checked.
-
-First, work out which board you have, because they are very different:
-
-* **Raspberry Pi Zero 2 W** — 64-bit ARM (Cortex-A53), 512 MB RAM. Should follow **Part A**
-  almost unchanged: choose *Raspberry Pi Zero 2 W* in Imager, and everything else applies. Running
-  the server needs about 250 MB, so 512 MB should be enough; the pinch point is more likely to be
-  `npm install`, which is far hungrier than the running server.
-* **Raspberry Pi Zero W** (the original) — 32-bit ARMv6 (BCM2835), 512 MB RAM. This is the
-  awkward one: it needs the 32-bit Raspberry Pi OS, and **the official Node.js builds no longer
-  support ARMv6**. Unofficial ARMv6 builds exist at
-  [unofficial-builds.nodejs.org](https://unofficial-builds.nodejs.org/download/release/), but
-  which versions are available, and whether one recent enough for this server exists, needs checking.
-
-Differences to expect on either Zero:
-
-* Wi-Fi is 2.4 GHz only on the original Zero W — it cannot see a 5 GHz-only network.
-* 512 MB of RAM is tight for `npm install`. Increasing the swap file is likely to be necessary.
-* Micro-USB power and micro-USB OTG, not USB-C and USB-A — different cables from the Pi 4.
-* No Ethernet socket, so the Ethernet fallback in Part A is unavailable. (The original Zero W can
-  be reached as a USB gadget over the data port, which is an alternative worth documenting.)
-* Everything will be slow. `npm install` may take a long time, especially if `sqlite3` has to be
-  compiled from source — which is likely on ARMv6, since no ready-made binaries are published.
-
-**Both boards are to be tested**, one of each having been ordered for the purpose. This section gets
-written once we know how they behave — in particular whether a usable Node exists for the original
-Zero W's ARMv6.
-
----
-
 ## Open questions
 
 Things this guide currently states with less confidence than the rest, to be resolved by
 following it on real hardware. Please correct them in place, and delete them from this list, as
 they get settled.
 
-**Part A, to check while installing**
+**To check while installing**
 
-1. **`.local` from an Android phone** (step A2, A4) - confirmed working from a laptop (with the
+1. **`.local` from an Android phone** (steps 3, 4) - confirmed working from a laptop (with the
    local-network permission granted on a Mac) and from an iPhone. Android is expected to fail, which
-   is what the note in A4 assumes; worth confirming on a real Android phone.
+   is what the note in step 4 assumes; worth confirming on a real Android phone.
 
 **Needs information I do not have**
 
-2. **Pi Zero W and Zero 2 W** (Part B) — both boards are on order; Part B stays unwritten until they
-    have been tried. The open technical question is Node on the original Zero W's ARMv6.
+2. **Pi Zero 2 W** - the Zero W notes come from a real install; the Zero 2 W is untested. Being
+    64-bit it should take the 64-bit image and avoid the `sqlite3` compile entirely, which is the slow
+    part, but it has the same 512 MB and so probably still needs the swap file in step 3.
 3. **Bridging to the shared server** — the local broker could optionally bridge to
     naturalinnovation.org so data also reaches the shared server. Not covered here; a later task.
-4. **Organization naming** — this guide sets up exactly one organization named `dev`, because that
+4. **Organization naming** — this guide sets up exactly one organization named `myfarm`, because that
     is the node firmware's default. Is that the right default for a farm installation, or should the
     guide encourage a meaningful organization id (which then has to be set in the node firmware too)?
 
@@ -674,4 +856,5 @@ What the guide has actually been proven against — add a row for each run:
 
 | Date | Board | OS image | Node | Server | Result |
 | --- | --- | --- | --- | --- | --- |
-| 2026-07-31 | Pi 4 Model B, 4 GB | Raspberry Pi OS Lite 64-bit, Debian 13.6 (trixie) | 20.19.2 and npm 9.2.0, both from `apt` | frugal-iot-server 0.3.3 from npm | Steps A1–A8 completed, plus an upgrade over the top of it. Wi-Fi from Imager did not connect and was fixed with `nmtui` (A2). `sqlite3` installed as a prebuilt binary, nothing compiled. Mosquitto started with an empty password file, and `frugal-iot-addorganization` wrote the broker account without sudo. Dashboard reached over `frugaliot.local` from a Mac (Chrome, after allowing local network access) and from an iPhone, MQTT status *connected*. A9 (sensor nodes) and A10 (HTTPS/OTA) not exercised. |
+| 2026-08-17 | Pi Zero W (original), 512 MB | Raspberry Pi OS Lite **32-bit**, Debian 13 (trixie), kernel 6.18.39+rpt-rpi-v6 | 20.19.2 and npm 9.2.0, both from `apt` — they run on ARMv6 | frugal-iot-server 0.3.5 from npm | Steps 1-8 completed. `sqlite3` had to be compiled: about 40 minutes, after `build-essential python3-dev python3-setuptools` (without setuptools it fails on `distutils`). A 2 GB swap file was required — zram alone was not enough and the board locked up hard without it — and `npm install` was run with `--maxsockets 1`. Step 9 (sensor nodes) not exercised on this board, but the broker publish/subscribe tests in step 6 passed. |
+| 2026-08-16 | Pi 4 Model B, 4GB | Raspberryy Pi OS Lite 64-bit, Debian 13.6 (trixie) | 20.19.2 and npm 9.2.0, both from `apt` | frugal-iot-server 0.3.5 from npm | Steps 1-9 completed, plus upgrade over top.  Dashboard reached over `frugaliot.local` from a Mac (Chrome, after allowing local network access) and from an iPhone, MQTT status *connected*. 10 (HTTPS/OTA) not exercised.