frugal-iot-server 0.3.6 → 0.3.8

package/INSTALLATION.md

@@ -40,6 +40,44 @@ Any of these work. What differs is only how long the install takes, and how much
 A Zero W runs the server perfectly well once installed. It is *installing* that is slow, because a
 single 1 GHz core has to unpack several hundred packages and compile one of them.
 
+## The short way: one script
+
+Steps 3 to 8 below can be done for you. Once you can ssh into the Pi (steps 1 and 2, which need a
+person with an SD card):
+
+```
+curl -fsSLO https://raw.githubusercontent.com/mitra42/frugal-iot-server/main/scripts/install-pi.sh
+bash install-pi.sh --org myfarm --name "My Farm" --email you@example.com --phone +61123456789
+```
+
+It asks for anything it needs that you did not pass, says what it is doing as it goes, and stops at
+the first thing that fails - leaving a log, and telling you to run `frugal-iot-diagnostic`. Running
+it again after a failure carries on rather than starting over, which matters on a Pi Zero W where one
+step takes about 40 minutes. `--dry-run` checks the arguments and shows the plan without touching the
+machine.
+
+**Passwords** are asked for, not invented, unless you say otherwise: it prompts for each (twice, not
+echoed) and generates one if you just press Enter. Pass them as `--superuser-password`,
+`--login-password` and `--broker-password` to skip the prompts, `--random-passwords` to have all
+three generated, or `--yes` to ask nothing at all — which is what to use over ssh with no terminal.
+A password you chose is not written to the log; a generated one has to be, since otherwise you would
+have no way of knowing it.
+
+**It tells you whether a reboot is needed**, rather than leaving you to wonder — it notices a kernel
+or boot-firmware package in the upgrade, the images in `/boot` changing underneath it, and
+`/var/run/reboot-required`. If none of those happened it says so, and there is nothing more to do.
+
+Fetch it with `curl -O` and then run it, rather than piping curl into bash - piped, it has no
+terminal to ask questions at.
+
+It finishes with a live server and a tested broker. It cannot do step 9, pointing your nodes at it,
+because a node learns its broker by being flashed with it - so it prints the settings to give them.
+Nor step 10, HTTPS.
+
+The steps below are what it does, in the same order, if you would rather do it by hand or need to
+understand what went wrong.
+
+---
 ### 0. What you need before you start
 
 **Hardware**
@@ -667,6 +705,241 @@ That section will cover: a DNS name for the Pi, port forwarding on the router, n
 proxy in front of port 8080, and certificates from Let's Encrypt via certbot. Both require the Pi
 to have internet access, which is the opposite of the offline case this guide is aimed at.
 
+### 11. Optional: bridge this Pi to a production server
+
+Everything above gives a self-contained Pi. This step also relays its readings to a production
+Frugal IoT server, so the same nodes appear on a dashboard elsewhere and can be controlled from
+it, while the Pi carries on working on its own whenever the link is down. Skip it if you do not
+want that — nothing else depends on it.
+
+The relaying is done by the broker, not by the server: Mosquitto has a "bridge" feature for
+connecting to another broker, and the server, the logger and the nodes need no configuration
+change at all. This installation ships an example bridge configuration in
+`extras/mosquitto-bridge.conf.example`, which explains each setting in place; this section is the
+surrounding work.
+
+It comes in three parts, and only the first is a one-off:
+
+* **11a — preparing a production server to accept bridges.** A TLS listener, a certificate, and an
+  access-control file. Done once for a given production server, and not again until there is
+  another one.
+* **11b — authorizing this Pi on that server.** One command, run there, once per Pi.
+* **11c — pointing this Pi at it.** One command, run here, once per Pi.
+
+If someone has already done 11a for the server you are bridging to — the usual case once the first
+Pi is working — **start at 11b**.
+
+**What you get, and what you do not.** While the link is up, readings appear on production within
+a second or so. While it is down, the Pi records everything as usual and production simply has a
+gap — the readings taken during an outage never reach it. That is a deliberate choice: the
+alternative is for the broker to hold them and deliver the backlog on reconnect, but production
+timestamps each reading as it arrives, so a day's backlog would arrive claiming to have happened
+in the instant the link returned. A gap is honest; that would not be. Controls are treated the
+other way round, and *are* queued, so turning something on from production while the Pi is offline
+takes effect when it reconnects.
+
+#### 11a. Preparing a production server to accept bridges
+
+Once per production server, not per Pi — and someone has probably already done it for
+`frugaliot.naturalinnovation.org`. If so, skip to 11b.
+
+**Add a listener for bridges.** A Mosquitto bridge speaks MQTT or MQTT-over-TLS and cannot use
+WebSockets, so the existing `wss://` path that browsers and servers use cannot carry this. It needs
+a TLS listener of its own on 8883, and the sensible certificate to give it is the one the web
+server already uses for the same hostname — one certificate and one renewal, rather than two.
+
+Find where that certificate actually is, rather than assuming: certbot's own layout under
+`/etc/letsencrypt/live/` is only one of several, and a wildcard certificate covering more than one
+domain is often installed by hand somewhere else entirely.
+
+```
+sudo grep -rn 'SSLCertificateFile\|SSLCertificateKeyFile' /etc/apache2/   # Apache
+sudo nginx -T 2>/dev/null | grep ssl_certificate                          # nginx
+```
+
+Take the paths for the vhost serving this server's hostname — `sudo apache2ctl -S` says which
+vhost that is — and add to production's Mosquitto configuration:
+
+```
+listener 8883
+protocol mqtt
+certfile <the SSLCertificateFile path>
+keyfile  <the SSLCertificateKeyFile path>
+```
+
+then open 8883 on the firewall.
+
+> **`certfile` must contain the intermediate certificate as well as the server's own**, or bridges
+> will refuse to connect even though browsers are happy — a browser can often fill in a missing
+> intermediate from cache and a bridge never can. `sudo grep -c 'BEGIN CERTIFICATE' <certfile>`
+> should report 2 or more. If it reports 1, concatenate the issuer's chain onto a copy and point
+> `certfile` at that.
+
+> **The key's permissions are what usually goes wrong.** Unlike a web server, which reads its key
+> as root at startup, Mosquitto drops to the `mosquitto` user *first* and reads the key afterwards,
+> so a key left `root:root 0600` stops the broker starting:
+>
+> ```
+> Error: Unable to load server key file "...". Check keyfile.
+> OpenSSL Error[0]: error:8000000D:system library::Permission denied
+> ```
+>
+> On Debian the tidy fix is the `ssl-cert` group, which exists for exactly this and which
+> `/etc/ssl/private` is already set up to admit:
+>
+> ```
+> sudo adduser mosquitto ssl-cert
+> sudo chgrp ssl-cert <keyfile>
+> sudo chmod 640 <keyfile>
+> ```
+>
+> The web server is unaffected — root still reads the key as before. If you would rather not widen
+> access to a directory holding keys for other domains, copy the certificate and key into
+> `/etc/mosquitto/certs` owned `mosquitto:mosquitto` instead, and refresh that copy on renewal.
+
+**Add one line to however this certificate gets renewed:**
+
+```
+systemctl reload mosquitto
+```
+
+Mosquitto keeps the certificate it loaded at startup, so without this the broker goes on serving
+the old one until it happens to be restarted, and bridges start failing once it expires. A reload
+is enough — it picks up a replaced certificate without dropping a single connection. Where that
+line goes depends on how the certificate is renewed: a certbot deploy hook in
+`/etc/letsencrypt/renewal-hooks/deploy/`, `acme.sh`'s `--reloadcmd`, or, if the certificate is
+installed by hand, the written procedure for doing that.
+
+> Note the asymmetry with the Pi end below: a *certificate* change needs only `reload`, but adding
+> or changing a *bridge* needs a full `restart`, because Mosquitto does not reload bridges.
+
+**Turn on access control.** Without it, any account that can log in to the broker can publish and
+subscribe anywhere on it, so a per-Pi account would be no more confined than the organization's
+own. This is the part to plan carefully, because Mosquitto's ACL file is **deny-by-default**: the
+moment the broker names an `acl_file`, every existing account with no entry in it stops working.
+So the file that restricts new bridge accounts has to grant the existing organizations what they
+already have, in the same change.
+
+```
+sudo tee /etc/mosquitto/aclfile >/dev/null <<'EOF'
+# Deny-by-default: an account with no rule here can connect but reach no topic at all.
+
+# Each organization reaches only its own topic tree. "%u" is the connecting account name, and the
+# first element of every Frugal IoT topic is the organization id - so this is what the nodes,
+# dashboards and the server's own logger already do. It just stops being optional.
+pattern readwrite %u/#
+EOF
+sudo chmod 644 /etc/mosquitto/aclfile
+```
+
+`chmod 644` matters: like the password file and the TLS key, the broker reads this *after* dropping
+to the `mosquitto` user, so a root-only file stops it starting. Then name it in the configuration
+and restart:
+
+```
+echo 'acl_file /etc/mosquitto/aclfile' | sudo tee /etc/mosquitto/conf.d/zy-frugal-iot-acl.conf
+sudo systemctl restart mosquitto
+```
+
+Check every existing organization can still log in **before** connecting any Pi — a wrong ACL file
+locks out every node and dashboard at once, and the reason appears only in Mosquitto's own log. On a
+Frugal IoT server the quickest check is its journal, which logs one line per organization:
+
+```
+sudo journalctl -u frugaliot -n 40 --no-pager | grep -iE "mqtt |not authoris"
+```
+
+Every organization should show `connect` and none should show `not authorized`. To back the change
+out, delete `/etc/mosquitto/conf.d/zy-frugal-iot-acl.conf` and restart.
+
+Nothing needs adding here per Pi: `frugal-iot-addbridge-prod` in 11b appends each bridge's own rule.
+
+> One thing this costs: `pattern readwrite %u/#` also denies `$SYS`, so the production broker can no
+> longer tell anyone which bridges are currently connected. The Pi still reports its own bridge
+> state to `frugal-iot-diagnostic`, which is where you would look anyway, so this is a fair trade —
+> but it is why production's dashboard cannot show a site as up or down.
+
+#### 11b. Authorizing this Pi (run on the production server)
+
+Once per Pi. On the production server, from its own directory:
+
+```
+npx --no frugal-iot-addbridge-prod <org-id> <site-name>
+```
+
+The site name only distinguishes one Pi from another within an organization — `northfield`,
+`shed`, `village2` — and becomes part of the account name. The script creates the broker account,
+adds its access-control rule, reloads the broker, and prints both the password and the exact
+command to run in 11c.
+
+Three things it checks, because each of them is a way this quietly fails:
+
+* **The organization must already exist on the production server**, or the readings will arrive at
+  its broker and nothing will record them — that server's logger subscribes per organization,
+  driven by the files in its `config.d/organizations/`. If it is missing, add it there first with
+  `frugal-iot-addorganization`, using the *same* organization id as this Pi.
+* **Each Pi gets its own account**, rather than sharing the organization's. The organization's
+  broker password is handed to every browser that logs in, so it is not a secret; a per-Pi account
+  can be confined to one site and revoked on its own.
+* **An existing account is left alone.** Re-running it for a site that already has one is refused
+  rather than quietly issuing a new password, which would stop that Pi relaying without anyone
+  touching it.
+
+The password is shown once and is not stored anywhere you can read back — copy it before you lose
+the output.
+
+#### 11c. Bridging this Pi (run on the Pi)
+
+Once per Pi. Run the command 11b printed, from this installation's directory:
+
+```
+cd ~/frugal-iot
+npx --no frugal-iot-addbridge-pi --org <org-id> --host <prod-host> --account bridge-<site-name>
+```
+
+It asks for the password rather than taking it on the command line, so it stays out of your shell
+history. Then it checks the far end is reachable and that its certificate is valid for that name,
+writes `/etc/mosquitto/conf.d/frugal-iot-bridge.conf`, restarts the broker, and reports whether the
+bridge actually connected.
+
+The restart is unavoidable: Mosquitto does not pick up bridges on a reload signal, so `reload`
+appears to succeed and changes nothing. It briefly disconnects every node, which they recover from
+on their own.
+
+> The configuration goes into `/etc/mosquitto/conf.d/` and not into this installation's
+> `config.d/`, because it holds a password for the production server. Everything under `config.d/`
+> is served to any logged-in browser by `/config.json`, so a credential put there would not stay
+> private. The file is written mode 600 for the same reason.
+
+To undo it: `sudo rm /etc/mosquitto/conf.d/frugal-iot-bridge.conf && sudo systemctl restart
+mosquitto`. The Pi goes back to being self-contained and keeps everything it has recorded.
+
+#### Check it
+
+```
+npx --no frugal-iot-diagnostic
+```
+
+The **Bridge to a production server** section names the bridge, lists what it relays, and reports
+whether it is connected. `CONNECTED` means the link is up. `DOWN`, or a report that it has never
+connected, means the address, the credentials or the certificate is wrong — and Mosquitto's own log,
+quoted earlier in the same output, says which.
+
+Then confirm at the far end: log in to production's dashboard and look for this Pi's nodes. They
+should appear within a couple of minutes, which is however long it takes each node to next report.
+
+**One oddity worth expecting.** Each time the bridge reconnects, the most recent value of every
+reading is re-sent to production and recorded there with the reconnect time rather than the time
+it was measured. That is one row per topic per outage, and it happens because nodes publish
+readings retained. The `duplicates:` rules in `config.d/schema/topics.yaml` absorb it when the
+value has not moved and the outage was short; after a long outage the row is written.
+
+**The clock matters more once a Pi is bridged**, because its readings now sit alongside
+production's. A bridged Pi has internet access whenever the link is up, so it sets its time from
+the network and the problem below mostly goes away — but it still comes up after a power cut
+believing whatever it last saved, and anything recorded before it reaches a time server carries
+that wrong time.
+
 ### Known limitation: the clock on an offline Pi
 
 A Raspberry Pi has no battery-backed clock. While it has internet access it sets its time from the
@@ -755,6 +1028,8 @@ broken. It covers most of the checks scattered through this guide, in one pass:
 * whether Mosquitto and the `frugaliot` service are running, with the last lines of the journal
   **and of Mosquitto's own log**, which is where its startup errors actually appear
 * whether the broker refuses a wrong password, and accepts each organization's real one
+* whether a bridge to a production server is configured, what it relays, and whether it is
+  connected right now (step 11) — the bridge password is not printed
 * whether ports 1883, 9012 and 8080 are listening, and whether the web server answers
 * whether the broker host in `config.d/mqtt.yaml` resolves from this machine
 * how much data the logger has written
@@ -770,6 +1045,30 @@ If you have not reached step 4 yet, the command does not exist. Copy
 [scripts/diagnostic.zsh](https://github.com/mitra42/frugal-iot-server/blob/main/scripts/diagnostic.zsh)
 to the Pi and run `zsh diagnostic.zsh` instead.
 
+### A topic that will not go away
+
+Nodes publish almost everything **retained**, meaning the broker keeps the last value and gives it
+to every new subscriber — which is how a dashboard shows a reading immediately instead of waiting
+for the next one. The catch is that a topic published by mistake outlives the mistake. Misspell a
+field, rename a module, flash a node with the wrong id, and that topic sits on the broker for ever,
+appearing on every dashboard. Fixing the node does not remove it, and neither does restarting the
+broker: the only way is to publish an empty message to that exact topic.
+
+```
+cd ~/frugal-iot
+npx --no frugal-iot-clearretained 'myfarm/lotus/+/sht/temperture/#'
+```
+
+Quote the pattern or the shell will expand it. That lists what is retained and changes nothing;
+add `--delete` to the same command to remove it.
+
+> Look before deleting. A node's `min`, `max`, `color` and `wired` settings are retained messages
+> too, and they are how the dashboard knows how to draw it — delete those and the node has to be
+> restarted before it looks right again. A pattern of just `#` is refused outright.
+
+On a Pi that is bridged to a production server (step 11), deleting here deletes there too: the
+bridge forwards the empty message like any other.
+
 ---
 
 ## Upgrading

package/README.md

@@ -3,7 +3,8 @@
 ## Installation
 
 For a self-contained server on a Raspberry Pi - including the MQTT broker, and starting from a
-blank SD card - see [INSTALLATION.md](INSTALLATION.md).
+blank SD card - see [INSTALLATION.md](INSTALLATION.md), which also has a script that does the whole
+thing in one go once you can ssh in.
 
 On a unix box that is already running ...
 
@@ -20,15 +21,20 @@ npm install frugal-iot-server
 npx --no frugal-iot-init
 ```
 `frugal-iot-init` copies in the configuration files, creates the `data`, `ota` and
-`config.d/organizations` directories, and creates the database. It never overwrites anything
-already there, so run it again after an upgrade to pick up newly added configuration.
+`config.d/organizations` directories, points `config.d/server.yaml` at the web client npm just
+installed, and creates the database. It never overwrites anything already there, so run it again
+after an upgrade to pick up newly added configuration.
 
 Edit `config.yaml` and `config.d/mqtt.yaml` (which broker to talk to) if the defaults do not suit.
 
-The database is created with a `superuser` account that has no password, so give it one:
+A newly created database has two accounts in it, `everyone` (which holds the permissions every
+logged-in user gets) and `superuser`, which has no password - so give it one:
 ```
 npx --no frugal-iot-setpassword superuser "<a-good-password>"
 ```
+On a database that existed before this account was introduced, that command reports
+`no user 'superuser'`, because id 1 was already taken by whoever registered first - and that
+account already has the superuser's privileges. Log in as that one instead.
 
 Then add an organization - this writes its yaml file into `config.d/organizations`, creates a
 login account of the same name with its permissions, and sets its password on the MQTT broker:
@@ -52,24 +58,49 @@ readYamlConfigFile ./config.yaml
 ... and then reading each of the other files
 
 Broker wss://frugaliot.naturalinnovation.org/wss - organizations: dev
+Not logging HTTP requests (morgan: false in config.d/server.yaml)
 Doing OTA updates at /ota_update from ...some path.../ota
-Serving /node_modules from ./node_modules
+Serving /node_modules from node_modules/frugal-iot-client/node_modules then ./node_modules
 User Database exists
 Opened user database
 Exec-ed starting SQL
+Created logger client for API integration
+Mounted API routes at /api
 Serving /data from ./data
 Server starting on port 8080
+Logger not reporting individual messages (verbose: false in config.d/logger.yaml)
+Collecting readings in memory, writing them out every 300 seconds
 mqtt dev connecting
 mqtt dev connect
 Subscribing topic dev/# 0
-Received dev/lotus/esp8266-85ea2b/humidity   71.8
 ```
-Where the broker and organizations are reported back, 
-then it successfully connects to the mqtt server
-and receives data from nodes attached to it. 
+Where the broker and organizations are reported back,
+then it successfully connects to the mqtt server.
+
+**It then goes quiet, and that is what should happen.** A new installation is set up to write as
+little as it can, because the usual home for this is a Raspberry Pi running from an SD card, and
+cards wear out from being written to. So it does not log a line per message received, nor a line
+per web request, and readings are collected in memory and written out every five minutes rather
+than one at a time. Seeing nothing after the lines above does not mean nothing is arriving.
+
+To watch the messages while setting up, or to work out why a node's readings are not appearing,
+turn it back on in `config.d/logger.yaml` and restart:
+```
+verbose: true
+```
+and each message reappears as `Received dev/lotus/esp8266-85ea2b/humidity   71.8`. `morgan: true`
+in `config.d/server.yaml` does the same for web requests. Turn both off again afterwards.
+There is more on what gets written, and how to see how much, under
+[Wear and tear on the SD card](INSTALLATION.md#wear-and-tear-on-the-sd-card).
 
 Open a browser pointing at for example `localhost:8080` and you should see the UI.
 
+If something is not right, this reports on the whole installation - versions, configuration,
+broker logins, what has been logged - and changes nothing:
+```
+npx --no frugal-iot-diagnostic
+```
+
 To upgrade later: `npm update frugal-iot-server` then `npx --no frugal-iot-init`.
 
 #### Developing the server, client or logger
@@ -85,22 +116,66 @@ The commands above have in-repo equivalents - `scripts/addorganization.zsh` and
 `scripts/setpassword.zsh` - and the server is `node frugal-iot-server.js`. All of them work on
 the directory you run them in, which for a clone is the top of the repo.
 
-To work on the client or logger from sibling checkouts, switch two places to their commented-out
-development lines:
-- `config.d/server.yaml` - `htmldir` and `nodemodulesdir` point at `../frugal-iot-client`
-- `frugal-iot-server.js` - the `MqttLogger` import points at `../frugal-iot-logger/index.js`
+To work on the client or the logger from sibling checkouts as well, link them in. This needs no
+edit to any file, so there is nothing to remember to change back:
+```
+cd ../frugal-iot-client && npm link
+cd ../frugal-iot-logger && npm link
+cd ../frugal-iot-server && npm link frugal-iot-client frugal-iot-logger
+```
+The server then loads both from your checkouts. `ls -l node_modules/frugal-iot-*` shows whether the
+links are in place - a later `npm install` or `npm update` can replace them with the published
+packages, which looks like your changes having no effect.
+
+There are commented-out alternatives in `config.d/server.yaml` (for `htmldir`) and in
+`frugal-iot-server.js` (for the `MqttLogger` import) that do the same job by editing instead. They
+work, but they are easy to publish by accident, so prefer the links.
 
-Take care not to commit those local switches, and note that `npm publish` packages your working
-tree - so check with `npm pack --dry-run` before publishing.
+Before publishing, run:
+```
+npm run prerelease
+```
+It checks the sensor schema and copies it into the examples that ship with `frugal-iot-logger`, sets
+the service worker's cache version from the client version this release installs, and refuses if:
+- the package is wired to a local checkout - a `file:` dependency, an `npm link`, a switched import
+  or a switched `htmldir`;
+- it requires a version of the client or logger that has not been published (a warning if published
+  but not the newest, since a fresh install resolves to the newest anyway while an existing one may
+  not move);
+- anything git ignores would be published. `files` in `package.json` is an allow-list, which fails
+  closed, but a file sitting in one of the listed directories goes out whether or not you meant it
+  to - which is how a working copy of the user database was published in 0.3.2.
+
+It also lists anything in the repository that would *not* be published and is not already accounted
+for in `scripts/not-published.txt`, so a newly added file that installers need does not go missing
+unnoticed. `npm run check-files` runs just that part.
+
+`npm run check-schema` on its own reports sensor topics that do not say whether they are logged, or
+that are logged with no rule about how often - both of which are easy to add by accident and
+expensive on an SD card.
 
 #### Running a production server
 To set it up as a service that runs at startup (and instructions vary between flavors of Linux)
 
-copy and edit `extras/frugaliot.service` to `/etc/systemd/system/frugaliot.service`
-you'll need to change the user, the directory you installed into, and `ExecStart`
+copy and edit `extras/frugaliot.service` to `/etc/systemd/system/frugaliot.service`. As shipped it
+matches the Raspberry Pi install in [INSTALLATION.md](INSTALLATION.md) - user `pi`, installed in
+`/home/pi/frugal-iot` - so on any other machine change `User`, `WorkingDirectory` and `ExecStart`
+to the account and directory you installed into.
+```
+sudo cp extras/frugaliot.service /etc/systemd/system/frugaliot.service
+sudo systemctl daemon-reload
+sudo systemctl enable --now frugaliot
+```
+`daemon-reload` is the step that is easy to miss - without it systemd keeps using the version it
+read before, and your edits appear to do nothing. After that `service frugaliot start|stop|restart`
+works as usual, and `systemctl status frugaliot` reports whether it is running.
+
+If you already have the service installed and only want to change one setting, `sudo systemctl edit
+frugaliot` writes an override rather than touching the file, which is easier to undo
+(`sudo systemctl revert frugaliot`).
 
-You can run`service frugaliot start` to start it
-and `systemctl enable frugaliot` to make sure it starts at boot. 
+Stop it with `service frugaliot stop` rather than killing it, so that readings still held in memory
+are written out first.
 
 Note that this will give you a HTTP server, but OTA on ESP32 requires HTTPS.
 

package/config.d/schema/devices.yaml

@@ -0,0 +1,63 @@
+# Which readings appear on a device's card, and in what order.
+#
+# Keys match the OTA key, which the firmware builds as SYSTEM_OTA_PREFIX "_" SYSTEM_OTA_SUFFIX -
+# the application, then the board (see src/system/ota.cpp). So sht30 built for ten boards gives ten
+# keys, sht30_c3_pico, sht30_d1_mini, ... all wanting the same card. Resolution order is:
+#   1. an exact match on the device id         e.g. esp8266-fb94bb
+#   2. an exact match on the full OTA key      e.g. sht30_c3_pico
+#   3. the longest key the OTA key starts with e.g. sht30
+# Longest wins at step 3, so file order never matters and sht30 beats sht.
+#
+# front:   the ordered rows on the front of the card. An entry containing "/" is a twig
+#          (module/leaf); one without is a control module, shown as its single control row.
+# summary: the one-line summary. Optional, and usually best omitted - it defaults to the first few
+#          entries of front, which now includes a control, shown compactly as "Relay ✓".
+#
+# An application with no entry gets the default: graphable readings, then actuators, then controls.
+# That is the right answer whenever the order does not matter, so most applications belong here only
+# if they want something specific. Module ids are the keys of modules.yaml - note the BH1750 sensor
+# publishes under "lux", not "bh1750".
+#
+# Of the applications in the node library, three are deliberately absent: "all" is every sensor at
+# once and no order helps, "gsheets" carries only a logging control, and "lcd_sht" displays another
+# node's readings rather than its own.
+
+agri:
+  front:    [soil/soil, ds18b20/ds18b20, sht/temperature, sht/humidity, controlhysteresis]
+  summary:  [soil/soil, ds18b20/ds18b20]
+bh1750:
+  front:    [lux/lux, controlhysteresis]
+blinken:
+  front:    [controlblinken]
+commonground:
+  front:    [bme280/temperature, bme280/humidity, do/do, ultrasonic/ultrasonic, ds18b20/ds18b20, controlhysteresis]
+datalogger:
+  front:    [sht/temperature, sht/humidity]
+ensaht:
+  front:    [aht21/temperature, aht21/humidity, ens160/aqi, controlhysteresis]
+gps:
+  front:    [gps/satellites, gps/latitude, gps/longitude, gps/altitude]
+  summary:  [gps/satellites]
+lilygohigrow:
+  front:    [soil/soil, dht/temperature, dht/humidity, lux/lux]
+  summary:  [soil/soil]
+loadcell:
+  front:    [loadcell/loadcell]
+loramesher:
+  front:    [sht/temperature, sht/humidity]
+ms5803:
+  front:    [ms5803/pressure, ms5803/temperature]
+power:
+  front:    [sht/temperature, sht/humidity, controlblinken]
+remotedisplay:
+  front:    [sht/temperature, sht/humidity, controlhysteresis]
+sht30:
+  front:    [sht/temperature, sht/humidity, controlhysteresis]
+soil:
+  front:    [soil/soil, sht/temperature, sht/humidity, controlhysteresis]
+  summary:  [soil/soil]
+sonoff:
+  # The relay is what someone wants to see; the control that drives it comes second.
+  front:    [relay/on, controlhysteresis]
+temp:
+  front:    [aht20/temperature, bmp280/pressure, controlhysteresis]

package/config.d/schema/modules.yaml

@@ -7,6 +7,9 @@
 # topics: list of topics, each with leaf, optionally leaf_from if it should be copied, and optionally any fields of topics to be overridden
 # slot: if it is supposed to fit in a specific slot of frugaliot
 # insidefrugaliot: true if it should be hidden inside the Frugal_IoT drop-down on the name of a sensor, false or not set if it should be shown as a separate module
+# summary: false to keep this module off a device's one-line summary. Omitted means it contributes -
+#   default-visible, because a new sensor that is silently invisible until someone remembers a flag
+#   is the worse failure. insidefrugaliot modules never contribute; they feed the status strip.
 aht20:
   name: "AHT20"
   topics:
@@ -28,33 +31,30 @@ bme280:
     - leaf: temperature
     - leaf: humidity
     - leaf: pressure
-      max:  1100
-      min:  300
 bme680:
   name: "BME680"
   topics:
     - leaf: temperature
     - leaf: humidity
     - leaf: pressure
-      max:  1100
-      min:  300
     - leaf: gas
 bmp280:
   name: "BMP280"
   topics:
     - leaf: temperature
     - leaf: pressure
-      max:  1100
-      min:  300
 button:
+  summary: false   # transient - a press is not a state worth a summary line
   name: Button
   topics:
     - leaf:     button
 captive:
+  summary: false   # a language code is not a reading
   name: Captive
   topics:
     - leaf:     language_code
 carousel:
+  summary: false   # a display selector is not a reading
   name: Display carousel
   topics:
     - leaf:     select
@@ -64,37 +64,6 @@ controlblinken:
     - leaf:     timeon
     - leaf:     timeoff
     - leaf:     out
-climate:
-  name: Climate
-  topics:
-    - leaf: temperature_now
-      leaf_from: controlfloat
-      name: Temperature Now
-    - leaf: temperature_setpoint
-      leaf_from: controlfloat
-      wireable: false
-      name: Temperature Setpoint
-    - leaf: temperature_hysteresis
-      leaf_from: controlfloat
-      wireable: false
-      name: Temperature Hysteresis
-    - leaf: humidity_now
-      leaf_from: controlfloat
-      name: Humidity Now
-    - leaf: humidity_setpoint
-      leaf_from: controlfloat
-      name: Humidity Setpoint
-      wireable: false
-    - leaf: humidity_hysteresis
-      leaf_from: controlfloat
-      name: Humidity Hysteresis
-      wireable: false
-    - leaf: temperature_out
-      leaf_from: controlouttoggle
-      name: Temperature Out
-    - leaf: humidity_out
-      leaf_from: controlouttoggle
-      name: Humidity Out
 controlhysteresis:
   name: Control
   topics:
@@ -153,6 +122,20 @@ do:
 ens160:
   name: "ENS160"
   topics:
+    # Compensation inputs, not readings. The ENS160 has no thermometer of its own: it takes
+    # ambient temperature and humidity from whatever is wired to it - an AHT21 sitting on the
+    # same breakout, or any other sensor on the node. So these come from controlfloat (wireable,
+    # rw: w, not logged) rather than from the temperature/humidity topics, which are read-only
+    # readings - whichever sensor actually measures them is the one that logs them.
+    - leaf: temperature
+      leaf_from: controlfloat
+      name: Temperature
+      max:  50
+      color: red
+    - leaf: humidity
+      leaf_from: controlfloat
+      name: Humidity
+      color: blue
     - leaf: aqi
     - leaf: tvoc
     - leaf: eco2
@@ -169,6 +152,7 @@ ensaht:
     - leaf:  eco2
     - leaf:  aqi500
 frugal_iot:
+  summary: false   # this is the card identity, not a module on it
   name: XXX
   topics:
     - leaf:     id
@@ -207,6 +191,7 @@ ina219:
     - leaf: power
     - leaf: load
 lcd:
+  summary: false   # a message is not a reading
   name: LCD
   topics:
     - leaf: message