gea-program 0.54.0__tar.gz → 0.55.0__tar.gz

{gea_program-0.54.0 → gea_program-0.55.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gea-program
-Version: 0.54.0
+Version: 0.55.0
 Summary: GEA-Program: downhole gauge monitoring - canonical measurement records, quality rules, drift evaluation with a re-fit change log, well-test validation, alarm management, model cards, store-and-forward, monthly SLA measurement, FAT/SAT protocols and a dashboard, generated from live objects in the client's reporting outline.
 Author: Daniel T. Murphy
 License: MPL-2.0
@@ -243,7 +243,7 @@ disappearing. Every action is written to `records/audit.jsonl` with the
 actor's name and the SHA-256 of the inputs it used. The service is written on
 the standard library only (no new dependency, the standalone check still
 passes), listens on the loopback interface by default (`--host 0.0.0.0` for a
-site network, behind the site's TLS proxy), keeps sessions in HttpOnly
+site network, over HTTPS - its own, below, or the site's TLS proxy), keeps sessions in HttpOnly
 SameSite cookies, and requires the page's own header on every action so a
 foreign form cannot act. Accounts live in `users.json` as salted PBKDF2-SHA256
 hashes; `gea users --workspace C:\site --action add --name ... --role admin`
@@ -1249,11 +1249,35 @@ is recomputed in the index. On the Reports page: the packets on record,
 Assemble as a job, and Schedule monthly (day and hour, the previous month).
 `gea help month-end`.
 
+### HTTPS
+
+The service speaks TLS itself when the workspace names a certificate: the
+site's own (from the company's certificate authority, which managed
+computers already trust; read where the site keeps it, at every start) or a
+self-signed one it makes with the standard library alone (an RSA key and an
+X.509 v3 server certificate for every name and address it is reached by).
+TLS 1.2 is the floor; the cookie is marked Secure; every answer carries
+Strict-Transport-Security; a plain `http://` request on the same port is
+sent to HTTPS; a certificate in its last 30 days raises the `tls.expiring`
+alert; a workspace whose certificate files are broken does not start rather
+than fall back to plain HTTP.
+
+```
+gea tls --workspace C:\site --site-cert C:\certs\gea-pad3.crt --site-key C:\certs\gea-pad3.key
+gea tls --workspace C:\site --self-signed --name gea-pad3 --name 10.20.0.15 --export-cert C:\certs\gea-site.crt
+gea serve --workspace C:\site --host 0.0.0.0                       # https://gea-pad3:8765/
+```
+
+Or Administration -> **HTTPS** in the page (download the certificate, not
+the key, for the site's computers to trust; compare the SHA-256 shown).
+`gea help https`.
+
 ### Before a site goes live
 
-`deploy/README.md` is the checklist: keep the service on the loopback
-address and put a TLS proxy in front (`deploy/Caddyfile`,
-`deploy/nginx-gea.conf`), start with `--behind-proxy`. The service itself
+`deploy/README.md` is the checklist: HTTPS from the service itself (above),
+or keep the service on the loopback address and put the site's TLS proxy in
+front (`deploy/Caddyfile`, `deploy/nginx-gea.conf`), start with
+`--behind-proxy`. The service itself
 rate-limits sign-ins (five failures lock the name and the address for 15
 minutes, audited), lists and revokes live sessions, sends the security
 headers on every response, and `gea housekeeping --workspace C:\site --apply`

{gea_program-0.54.0 → gea_program-0.55.0}/README.md

@@ -204,7 +204,7 @@ disappearing. Every action is written to `records/audit.jsonl` with the
 actor's name and the SHA-256 of the inputs it used. The service is written on
 the standard library only (no new dependency, the standalone check still
 passes), listens on the loopback interface by default (`--host 0.0.0.0` for a
-site network, behind the site's TLS proxy), keeps sessions in HttpOnly
+site network, over HTTPS - its own, below, or the site's TLS proxy), keeps sessions in HttpOnly
 SameSite cookies, and requires the page's own header on every action so a
 foreign form cannot act. Accounts live in `users.json` as salted PBKDF2-SHA256
 hashes; `gea users --workspace C:\site --action add --name ... --role admin`
@@ -1210,11 +1210,35 @@ is recomputed in the index. On the Reports page: the packets on record,
 Assemble as a job, and Schedule monthly (day and hour, the previous month).
 `gea help month-end`.
 
+### HTTPS
+
+The service speaks TLS itself when the workspace names a certificate: the
+site's own (from the company's certificate authority, which managed
+computers already trust; read where the site keeps it, at every start) or a
+self-signed one it makes with the standard library alone (an RSA key and an
+X.509 v3 server certificate for every name and address it is reached by).
+TLS 1.2 is the floor; the cookie is marked Secure; every answer carries
+Strict-Transport-Security; a plain `http://` request on the same port is
+sent to HTTPS; a certificate in its last 30 days raises the `tls.expiring`
+alert; a workspace whose certificate files are broken does not start rather
+than fall back to plain HTTP.
+
+```
+gea tls --workspace C:\site --site-cert C:\certs\gea-pad3.crt --site-key C:\certs\gea-pad3.key
+gea tls --workspace C:\site --self-signed --name gea-pad3 --name 10.20.0.15 --export-cert C:\certs\gea-site.crt
+gea serve --workspace C:\site --host 0.0.0.0                       # https://gea-pad3:8765/
+```
+
+Or Administration -> **HTTPS** in the page (download the certificate, not
+the key, for the site's computers to trust; compare the SHA-256 shown).
+`gea help https`.
+
 ### Before a site goes live
 
-`deploy/README.md` is the checklist: keep the service on the loopback
-address and put a TLS proxy in front (`deploy/Caddyfile`,
-`deploy/nginx-gea.conf`), start with `--behind-proxy`. The service itself
+`deploy/README.md` is the checklist: HTTPS from the service itself (above),
+or keep the service on the loopback address and put the site's TLS proxy in
+front (`deploy/Caddyfile`, `deploy/nginx-gea.conf`), start with
+`--behind-proxy`. The service itself
 rate-limits sign-ins (five failures lock the name and the address for 15
 minutes, audited), lists and revokes live sessions, sends the security
 headers on every response, and `gea housekeeping --workspace C:\site --apply`

{gea_program-0.54.0 → gea_program-0.55.0}/gea/__init__.py

@@ -123,7 +123,7 @@ from .profile_catalog import (
     read_pangaea_txt,
 )
 
-__version__ = "0.54.0"
+__version__ = "0.55.0"
 __all__ = [
     "aging_rate", "rate_psi_yr", "accuracy_psi",
     "Sensor", "SimulatorConfig", "DownholeEngine",

{gea_program-0.54.0 → gea_program-0.55.0}/gea/__main__.py

@@ -756,13 +756,26 @@ def main(argv=None) -> int:
 
     p_sv = sub.add_parser("serve", help="serve the dashboard over a workspace: the page, the reports and the JSON API (standard library HTTP)")
     p_sv.add_argument("--workspace", type=str, required=True, help="the workspace folder (gea workspace --action init)")
-    p_sv.add_argument("--host", type=str, default="127.0.0.1", help="bind address; 0.0.0.0 exposes it on the site network (put TLS in front)")
+    p_sv.add_argument("--host", type=str, default="127.0.0.1", help="bind address; 0.0.0.0 exposes it on the site network (turn HTTPS on first: gea tls)")
     p_sv.add_argument("--port", type=int, default=8765)
     p_sv.add_argument("--workers", type=int, default=1, help="jobs run at once")
     p_sv.add_argument("--no-scheduler", action="store_true", help="do not run scheduled jobs from this process")
     p_sv.add_argument("--behind-proxy", action="store_true", help="a reverse proxy terminates TLS in front: trust X-Forwarded-For/-Proto, mark the cookie Secure (deploy/ has nginx and Caddy examples)")
     p_sv.add_argument("--no-browser", action="store_true", help="do not open the control panel in a browser once the service is listening (a scheduled start, a restart, a server with no desktop)")
 
+    p_tl = sub.add_parser("tls", help="HTTPS for the dashboard: the site's own certificate, a self-signed one made here, or off; with no option, how it stands")
+    p_tl.add_argument("--workspace", type=str, required=True)
+    p_tl.add_argument("--site-cert", type=str, default=None, help="the site's certificate (PEM, the server's first, then any chain) - read from where it is at every start")
+    p_tl.add_argument("--site-key", type=str, default=None, help="its private key (PEM, without a passphrase)")
+    p_tl.add_argument("--self-signed", action="store_true", help="make a self-signed certificate and key in the workspace's keys\\tls folder")
+    p_tl.add_argument("--name", action="append", default=[], help="--self-signed: a host name or address the dashboard is reached by (repeat; default: this machine's names, localhost, 127.0.0.1)")
+    p_tl.add_argument("--days", type=int, default=825, help="--self-signed: days valid (at most 825)")
+    p_tl.add_argument("--bits", type=int, default=2048, choices=[2048, 3072])
+    p_tl.add_argument("--off", action="store_true", help="plain HTTP from the next start (the files are left where they are)")
+    p_tl.add_argument("--export-cert", type=str, default=None, help="write the certificate (public, never the key) to this file, for the site's computers to trust")
+    p_tl.add_argument("--actor", type=str, default="cli")
+    p_tl.add_argument("--json", action="store_true")
+
     p_hk = sub.add_parser("housekeeping", help="rotate the append-only logs, prune finished job folders and old live recordings (dry run unless --apply)")
     p_hk.add_argument("--workspace", type=str, required=True)
     p_hk.add_argument("--apply", action="store_true")
@@ -2603,6 +2616,41 @@ def main(argv=None) -> int:
             raise SystemExit(f"pore-pressure: {e.args[0] if e.args else e}")
         print(_json.dumps(r, default=str) if a.json else _PPF.text(r))
         return 0
+    elif a.cmd == "tls":
+        import json as _json
+        from . import tls as _TLS
+        from .workspace import Workspace, WorkspaceError
+        try:
+            ws = Workspace(a.workspace)
+            if sum(bool(x) for x in (a.self_signed, a.site_cert or a.site_key, a.off)) > 1:
+                raise ValueError("one of --self-signed, --site-cert/--site-key or --off")
+            if a.site_cert or a.site_key:
+                if not (a.site_cert and a.site_key):
+                    raise ValueError("--site-cert and --site-key go together")
+                r = _TLS.use_site_certificate(ws, a.actor, a.site_cert, a.site_key)
+            elif a.self_signed:
+                r = _TLS.make_self_signed(ws, a.actor, a.name or None, a.days, a.bits)
+            elif a.off:
+                r = _TLS.turn_off(ws, a.actor)
+            else:
+                r = _TLS.status(ws)
+            if a.export_cert:
+                pem = _TLS.certificate_pem(ws)
+                if not pem:
+                    raise ValueError("HTTPS is off: there is no certificate to export")
+                with open(a.export_cert, "wb") as f_:
+                    f_.write(pem)
+        except (KeyError, ValueError, OSError, WorkspaceError) as e:
+            raise SystemExit(f"tls: {e.args[0] if e.args else e}")
+        if a.json:
+            print(_json.dumps(r, default=str))
+        else:
+            print(_TLS.text(r))
+            if a.self_signed or a.site_cert or a.off:
+                print("   takes effect at the next start of the control panel (gea serve, or Restart on the Administration page)")
+            if a.export_cert:
+                print(f"   certificate written to {a.export_cert} - hand this out, never the key")
+        return 0
     elif a.cmd == "frac-hits":
         import json as _json
         from . import frac_hits as _FH
@@ -2946,18 +2994,29 @@ def main(argv=None) -> int:
             print(f"serve: BLOCK {mism}")
         if pre or mism:
             raise SystemExit("serve: not starting - fix the above (gea doctor --workspace ... shows the full picture)")
+        from .tls import TlsError
         try:
             svc = Service(a.workspace, host=a.host, port=a.port, workers=a.workers, scheduler=not a.no_scheduler, behind_proxy=a.behind_proxy)
         except WorkspaceError as e:
             raise SystemExit(f"serve: {e}")
+        except TlsError as e:
+            raise SystemExit(f"serve: HTTPS is on for this workspace but {e}. Put the files right, or: gea tls --workspace \"{a.workspace}\" --off")
         n_users = svc.app.users.count()
         print(f"gea {__version__}: serving workspace '{svc.ws.manifest['name']}' at {svc.url}")
         print(f"   code: {os.path.dirname(os.path.abspath(__file__))}   (an installed copy serves the files it was installed from; pip install -e <clone> follows the clone)")
         print(f"   patches: {len(svc.app.patches.store.list())} defined, {len([s for s in svc.app.patches.states() if s['status'] != 'STOPPED'])} running")
         if n_users == 0:
             print("   no accounts yet: the page will ask for the first administrator's name and password (one time), or run: gea users --workspace ... --action add --role admin --name ...")
-        if a.host not in ("127.0.0.1", "localhost", "::1") and not a.behind_proxy:
-            print("   NOTE: bound to a network address without --behind-proxy: traffic is plain HTTP. Put TLS in front (deploy/README.md) for anything beyond a trusted LAN.")
+        if svc.tls_status:
+            ts_ = svc.tls_status
+            f_ = ts_.get("facts") or {}
+            print(f"   HTTPS ({ts_['state']}): {ts_['detail']}")
+            print(f"   names {', '.join(f_.get('names') or [])}; SHA-256 {f_.get('sha256_fingerprint')}")
+            if f_.get("self_signed"):
+                print(f"   the certificate to hand out (never the key): {ts_['cert']}  - or download it from the Administration page")
+        elif a.host not in ("127.0.0.1", "localhost", "::1") and not a.behind_proxy:
+            print("   NOTE: bound to a network address in plain HTTP. Turn HTTPS on (gea tls --workspace ... --site-cert/--site-key, or --self-signed),")
+            print("   or put the site's TLS proxy in front (--behind-proxy; deploy/README.md), for anything beyond a trusted LAN.")
         print("   Ctrl+C stops it; every action is in records/audit.jsonl, every start and stop in records/runlog.jsonl")
         print("   the control panel's Stop and Restart end this run with a stated code: 0 stop, 86 start me again -")
         print("   the launcher reads it, and on anything but 86 it hands the window to a PowerShell prompt, never a Python one")

{gea_program-0.54.0 → gea_program-0.55.0}/gea/acceptance_tests.py

@@ -12239,6 +12239,228 @@ def section_cw_decline(tmp: str) -> None:
        + (f" [failed: {', '.join(failed4)}]" if failed4 else ""))
 
 
+def section_cx_https(tmp: str) -> None:
+    """Section CX - HTTPS: the self-signed certificate made by the standard library and read back; the service speaking
+    TLS 1.2 or later with it - the Secure cookie, HSTS, a plain request sent to HTTPS, an untrusting client refused while
+    the service keeps answering; the site's own certificate read where it lives; the expiry alert, the doctor, a broken
+    pair that does not start, the CLI; the key in no dossier and from no route; the page and the help."""
+    import http.client
+    import socket as _so
+    import ssl as _ssl
+    import subprocess as _sp
+    import urllib.error
+    import zipfile as _zf
+    from datetime import datetime as _dt, timedelta as _td, timezone as _tz
+    from . import __version__ as _ver
+    from . import alerts as AL
+    from . import doctor as DR
+    from . import dossier as DZ
+    from . import helplib as H
+    from . import tls as TLS
+    from .service import Service, Users
+    from .workspace import Workspace
+    pkg = Path(__file__).parent
+    d = Path(tmp, "https"); d.mkdir()
+
+    # -- CX1 the certificate by hand ----------------------------------------------------------------------------------
+    key = TLS.rsa_key(2048)
+    sig = TLS.sign_pkcs1_sha256(key, b"pad 3")
+    cert, kpem, facts = TLS.self_signed(["gea-pad3", "gea-pad3.example.local", "10.20.0.15", "localhost", "127.0.0.1"], days=400)
+    (d / "a.crt").write_bytes(cert); (d / "a.key").write_bytes(kpem)
+    der = _ssl.PEM_cert_to_DER_cert(cert.decode("ascii"))
+    nb = _dt.strptime(facts["not_before"], "%Y-%m-%dT%H:%M:%SZ"); na = _dt.strptime(facts["not_after"], "%Y-%m-%dT%H:%M:%SZ")
+    cert_b, key_b, _ = TLS.self_signed(["other"], days=5)
+    (d / "b.crt").write_bytes(cert_b); (d / "b.key").write_bytes(key_b)
+    (d / "enc.key").write_bytes(b"-----BEGIN ENCRYPTED PRIVATE KEY-----\nMIIB\n-----END ENCRYPTED PRIVATE KEY-----\n")
+
+    def refused(fn, *a, **k):
+        try:
+            fn(*a, **k)
+        except TLS.TlsError as e:
+            return str(e)
+        return ""
+    c1 = {
+        "rsa": key["n"].bit_length() == 2048 and key["e"] == 65537 and pow(pow(12345, key["e"], key["n"]), key["d"], key["n"]) == 12345
+               and TLS.verify_pkcs1_sha256(key["n"], key["e"], b"pad 3", sig) and not TLS.verify_pkcs1_sha256(key["n"], key["e"], b"pad 4", sig),
+        "facts": facts["names"] == ["gea-pad3", "gea-pad3.example.local", "10.20.0.15", "localhost", "127.0.0.1"] and facts["subject"] == "gea-pad3"
+                 and facts["self_signed"] and facts["self_signature_ok"] and facts["ca"] is False and facts["bits"] == 2048 and facts["key"] == "RSA"
+                 and abs((na - nb).total_seconds() - (400 * 86400 + 300)) < 2
+                 and facts["sha256_fingerprint"].replace(":", "").lower() == __import__("hashlib").sha256(der).hexdigest(),
+        "loads": isinstance(TLS.server_context(str(d / "a.crt"), str(d / "a.key")), _ssl.SSLContext)
+                 and TLS.server_context(str(d / "a.crt"), str(d / "a.key")).minimum_version == _ssl.TLSVersion.TLSv1_2,
+        "refused": all(refused(*x) for x in ((TLS.self_signed, []), (TLS.self_signed, ["x"], 826), (TLS.self_signed, ["x"], 30, 1024),
+                                             (TLS.self_signed, ["bad name!"])))
+                   and "does not belong" in refused(TLS.server_context, str(d / "a.crt"), str(d / "b.key"))
+                   and "passphrase" in refused(TLS.server_context, str(d / "a.crt"), str(d / "enc.key"))
+                   and "not there" in refused(TLS.server_context, str(d / "nope.crt"), str(d / "a.key"))
+                   and "no certificate" in refused(TLS.read_certificate, b"not a certificate"),
+    }
+    failed1 = [k for k, v in c1.items() if not v]
+    ok(not failed1,
+       "CX1 the self-signed certificate is made by the standard library: an RSA key (e 65537; a PKCS #1 v1.5 SHA-256 signature that verifies, and not for "
+       "other bytes); an X.509 v3 server certificate with every name and address as a subject alternative name, its validity, not an authority, its own "
+       "signature good, its SHA-256 fingerprint; loaded by Python's TLS with 1.2 as the floor; no names, more than 825 days, a 1024-bit key, a bad name, a key "
+       "that is not the certificate's, a key with a passphrase and a missing file refused, each saying why" + (f" - failed: {failed1}" if failed1 else ""))
+
+    # -- CX2 the service over HTTPS ------------------------------------------------------------------------------------
+    wsp = str(Path(tmp, "ws_cx")); ws = Workspace.create(wsp, "HTTPS Site", actor="tester")
+    users = Users(str(Path(wsp, "users.json")))
+    users.add("adm", "admin-pass-123", "admin"); users.add("v", "viewer-pass-1", "viewer")
+    st0 = TLS.status(ws)
+    made = TLS.make_self_signed(ws, "tester", ["localhost", "127.0.0.1"], days=400)
+    svc = Service(wsp, host="127.0.0.1", port=0, scheduler=False).start()
+    port = svc.port
+    trust = _ssl.create_default_context(cafile=str(Path(wsp, TLS.SELF_CERT)))
+
+    def req(method, path, body=None, cookie=None, host="127.0.0.1", ctx=trust):
+        c = http.client.HTTPSConnection(host, port, context=ctx, timeout=30)
+        h = {"Content-Type": "application/json", "X-GEA-Action": "1"} if method == "POST" else {}
+        if cookie:
+            h["Cookie"] = cookie
+        c.request(method, path, body=(json.dumps(body) if body is not None else None), headers=h)
+        r = c.getresponse()
+        data = r.read()
+        ver = c.sock.version() if c.sock else None
+        c.close()
+        return r.status, dict(r.getheaders()), data, ver
+
+    def plain(path, host_header):
+        c = http.client.HTTPConnection("127.0.0.1", port, timeout=30)
+        c.request("GET", path, headers={"Host": host_header})
+        r = c.getresponse()
+        out = (r.status, r.getheader("Location"), r.read())
+        c.close()
+        return out
+    try:
+        idle = _so.create_connection(("127.0.0.1", port), timeout=30)            # connects and says nothing
+        s_s, h_s, _, ver = req("GET", "/api/session")
+        s_l, h_l, b_l, _ = req("POST", "/api/login", {"name": "adm", "password": "admin-pass-123"})
+        ck = h_l.get("Set-Cookie", "").split(";")[0]
+        s_v, h_v, _, _ = req("POST", "/api/login", {"name": "v", "password": "viewer-pass-1"})
+        vk = h_v.get("Set-Cookie", "").split(";")[0]
+        p301 = plain("/api/overview?x=1", f"127.0.0.1:{port}")
+        p400 = plain("/", "elsewhere.example")
+        try:
+            urllib.request.urlopen(svc.url + "api/session", context=_ssl.create_default_context(), timeout=30)
+            untrusted = "accepted"
+        except urllib.error.URLError as e:
+            untrusted = str(e.reason)
+        wrong_name = _ssl.create_default_context(cafile=str(Path(wsp, TLS.SELF_CERT)))
+        try:
+            with _so.create_connection(("127.0.0.1", port), timeout=30) as raw:
+                with wrong_name.wrap_socket(raw, server_hostname="gea-other.example"):
+                    pass
+            name_check = "accepted"
+        except _ssl.SSLError as e:
+            name_check = str(e)
+        s_after, _, _, _ = req("GET", "/api/session")
+        idle.close()
+        # -- CX3 the API ------------------------------------------------------------------------------------------------
+        s_ta, _h, ta_b, _ = req("GET", "/api/tls", cookie=ck)
+        ta = json.loads(ta_b)
+        s_tv, _, _, _ = req("GET", "/api/tls", cookie=vk)
+        s_cv, h_cv, pem_v, _ = req("GET", "/api/tls/certificate", cookie=vk)
+        s_pv, _, _, _ = req("POST", "/api/tls", {"action": "off"}, cookie=vk)
+        s_mm, _, b_mm, _ = req("POST", "/api/tls", {"action": "site", "cert": str(d / "a.crt"), "key": str(d / "b.key")}, cookie=ck)
+        s_bad, _, _, _ = req("POST", "/api/tls", {"action": "nonsense"}, cookie=ck)
+        s_ss, _, b_ss, _ = req("POST", "/api/tls", {"action": "self-signed", "names": "localhost 127.0.0.1", "days": 300}, cookie=ck)
+        s_t2, _, t2, _ = req("GET", "/api/tls", cookie=ck)
+        t2 = json.loads(t2)
+    finally:
+        svc.stop()
+    c2 = {
+        "off_first": st0["state"] == "OFF" and made["state"] == "OK" and made["mode"] == "self-signed",
+        "url": svc.url == f"https://127.0.0.1:{port}/",
+        "tls": s_s == 200 and ver in ("TLSv1.2", "TLSv1.3") and "max-age=" in h_s.get("Strict-Transport-Security", ""),
+        "cookie": s_l == 200 and s_v == 200 and "Secure" in h_l.get("Set-Cookie", "") and "HttpOnly" in h_l.get("Set-Cookie", ""),
+        "plain": p301[0] == 301 and p301[1] == f"https://127.0.0.1:{port}/api/overview?x=1" and p400[0] == 400 and b"speaks HTTPS" in p400[2],
+        "refused": "CERTIFICATE_VERIFY_FAILED" in untrusted and ("mismatch" in name_check.lower() or "doesn't match" in name_check or "not valid" in name_check),
+        "still": s_after == 200 and svc.httpd.tls_refused >= 2,
+    }
+    failed2 = [k for k, v in c2.items() if not v]
+    ok(not failed2,
+       "CX2 with HTTPS on, the service speaks TLS 1.2 or later at an https:// address: a client that trusts the certificate signs in and the cookie is Secure "
+       "and HttpOnly, with Strict-Transport-Security on every answer; plain HTTP on the same port is sent to the same address over HTTPS when it names the "
+       "machine by a name the certificate carries, and told to use HTTPS otherwise; a client that does not trust the certificate, or reaches it by another "
+       "name, is refused - and a silent connection and the refusals do not stop the service answering" + (f" - failed: {failed2}" if failed2 else ""))
+    page_src = (pkg / "web" / "app.html").read_text(encoding="utf-8")
+    adm_view = page_src[page_src.index("VIEWS.admin = "):page_src.index("VIEWS.help = ")]
+    audit = Path(ws.audit_path).read_text(encoding="utf-8")
+    c3 = {
+        "status": s_ta == 200 and ta["state"] == "OK" and ta["serving"]["https"] and ta["restart_needed"] is False
+                  and ta["facts"]["sha256_fingerprint"] == made["facts"]["sha256_fingerprint"],
+        "roles": s_tv == 403 and s_pv == 403 and s_bad == 400,
+        "certificate": s_cv == 200 and pem_v.startswith(b"-----BEGIN CERTIFICATE-----") and b"PRIVATE KEY" not in pem_v
+                       and TLS.read_certificate(pem_v)["sha256_fingerprint"] == made["facts"]["sha256_fingerprint"] and "attachment" in h_cv.get("Content-Disposition", ""),
+        "change": s_mm == 400 and b"does not belong" in b_mm and s_ss == 200 and json.loads(b_ss)["restart_needed"] is True
+                  and t2["restart_needed"] is True and t2["serving"]["https"] and t2["facts"]["sha256_fingerprint"] != made["facts"]["sha256_fingerprint"],
+        "audit": audit.count('"tls.self_signed"') == 2,
+        "page": all(x in adm_view for x in ("/api/tls", "tlsSite", "tlsSelf", "/api/tls/certificate", "const cStop = $('#ctlStop')", "cAuto.onchange"))
+                and "const cStop" not in page_src[page_src.index("VIEWS.prefs = "):page_src.index("VIEWS.search = ")],
+    }
+    failed3 = [k for k, v in c3.items() if not v]
+    ok(not failed3,
+       "CX3 the HTTPS card (the administrator role): how it stands and what this run serves; the certificate - public - downloads for any signed-in user "
+       "and carries no key; a viewer may not change it; a key that is not the certificate's is refused; a new certificate is kept and says it needs a "
+       "restart; each change is audited; the operations-control buttons on the Administration page are wired there"
+       + (f" - failed: {failed3}" if failed3 else ""))
+
+    # -- CX4 the site's certificate, the expiry alert, a broken pair, the CLI, the key kept in ----------------------------
+    site_dir = Path(tmp, "site_certs"); site_dir.mkdir()
+    sc, sk, sf = TLS.self_signed(["gea-pad3.example.local", "127.0.0.1"], days=60, org="Example Operator IT")
+    (site_dir / "gea.crt").write_bytes(sc); (site_dir / "gea.key").write_bytes(sk)
+    st_site = TLS.use_site_certificate(ws, "tester", str(site_dir / "gea.crt"), str(site_dir / "gea.key"))
+    copied = sorted(p.name for p in Path(wsp, TLS.TLS_DIR).iterdir())
+    exp_at = _dt.strptime(sf["not_after"], "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=_tz.utc)
+    st_exp = TLS.status(ws, exp_at - _td(days=10))
+    r_exp = AL.check_conditions(ws, None, now=exp_at - _td(days=10), force=True)
+    TLS.make_self_signed(ws, "tester", ["localhost", "127.0.0.1"], days=400)
+    r_clr = AL.check_conditions(ws, None, now=_dt.now(_tz.utc), force=True)
+    os.remove(Path(wsp, TLS.SELF_KEY))
+    st_broken = TLS.status(ws)
+    doc = DR.check_workspace(wsp, "127.0.0.1", 0)
+    try:
+        Service(wsp, host="127.0.0.1", port=0, scheduler=False)
+        start_broken = "started"
+    except TLS.TlsError as e:
+        start_broken = str(e)
+    cli_off = _sp.run([sys.executable, "-m", "gea", "tls", "--workspace", wsp, "--off"], capture_output=True, text=True, cwd=str(pkg.parent))
+    svc_plain = Service(wsp, host="127.0.0.1", port=0, scheduler=False)
+    url_plain = svc_plain.url
+    svc_plain.httpd.server_close()
+    out_crt = str(d / "handed_out.crt")
+    cli_ss = _sp.run([sys.executable, "-m", "gea", "tls", "--workspace", wsp, "--self-signed", "--name", "gea-pad3", "--name", "10.20.0.15", "--days", "90",
+                      "--export-cert", out_crt], capture_output=True, text=True, cwd=str(pkg.parent))
+    cli_st = _sp.run([sys.executable, "-m", "gea", "tls", "--workspace", wsp, "--json"], capture_output=True, text=True, cwd=str(pkg.parent))
+    cli_bad = _sp.run([sys.executable, "-m", "gea", "tls", "--workspace", wsp, "--site-cert", str(site_dir / "gea.crt")], capture_output=True, text=True, cwd=str(pkg.parent))
+    __import__("gea.frac_hits", fromlist=["write_report"]).write_report(ws, _ver, "tester")
+    dz = DZ.build(ws, "tester", _ver, None, include_sources=True)
+    zp = Path(ws.reports_dir, dz["path"])
+    with _zf.ZipFile(zp) as z:
+        names = z.namelist()
+        leaked = [n for n in names if b"PRIVATE KEY" in z.read(n)]
+    key_text = Path(wsp, TLS.SELF_KEY).read_bytes()
+    c4 = {
+        "site": st_site["state"] == "OK" and st_site["mode"] == "site" and st_site["cert"] == str(site_dir / "gea.crt")
+                and st_site["facts"]["issuer"] == "gea-pad3.example.local" and "self_signed_key.pem" in copied and "gea.key" not in copied,
+        "expiring": st_exp["state"] == "EXPIRING" and [e["event"] for e in r_exp if e["event"] == "tls.expiring"] == ["tls.expiring"]
+                    and any(e["event"] == "condition.cleared" and e.get("cleared") == "tls.expiring" for e in r_clr) and "tls.expiring" in AL.EVENTS,
+        "broken": st_broken["state"] == "BROKEN" and any(f["level"] == "block" and "HTTPS BROKEN" in f["what"] for f in doc) and "not there" in start_broken,
+        "cli": cli_off.returncode == 0 and "HTTPS: OFF" in cli_off.stdout and url_plain.startswith("http://127.0.0.1:")
+               and cli_ss.returncode == 0 and "gea-pad3, 10.20.0.15" in cli_ss.stdout and "next start" in cli_ss.stdout
+               and Path(out_crt).read_bytes().startswith(b"-----BEGIN CERTIFICATE-----") and b"PRIVATE" not in Path(out_crt).read_bytes()
+               and cli_st.returncode == 0 and json.loads(cli_st.stdout)["state"] == "OK" and cli_bad.returncode != 0 and "go together" in (cli_bad.stderr + cli_bad.stdout),
+        "kept_in": not leaked and not any(n.startswith("keys/tls") or n.endswith(".pem") for n in names) and b"PRIVATE KEY" in key_text,
+        "help": not H.check("https") and any(t["topic"] == "https" for t in H.topics()) and "gea help https" in (pkg / "help" / "upkeep.md").read_text(encoding="utf-8"),
+    }
+    failed4 = [k for k, v in c4.items() if not v]
+    ok(not failed4,
+       "CX4 the site's own certificate is used where it lives (not copied in); in its last 30 days it reads EXPIRING and raises tls.expiring once, cleared "
+       "when a new one is in place; with its key gone the doctor says BLOCK and the service does not start (never plain HTTP by surprise) until "
+       "`gea tls --off`; `gea tls --self-signed --name ... --export-cert` makes one and writes the certificate alone; --site-cert without its key is refused; "
+       "no dossier carries the key; the help page" + (f" - failed: {failed4}" if failed4 else ""))
+
+
 def PR_read_ok(path) -> bool:
     from . import pdf_report as PR
     try:
@@ -12351,6 +12573,7 @@ def main() -> int:
         section_cu_pore_pressure(tmp)
         section_cv_torque_drag(tmp)
         section_cw_decline(tmp)
+        section_cx_https(tmp)
     if _FAILS:
         print(f"[ACCEPTANCE] {len(_FAILS)} FAILURES ({_PASS} passed):")
         for f in _FAILS:

{gea_program-0.54.0 → gea_program-0.55.0}/gea/alerts.py

@@ -17,7 +17,10 @@ worth an event, for the conditions the operator does not watch by hand:
   attached);
 - `frac.hit` - a fracturing stage reads as a FRAC HIT on a monitored well's pressure (the Frac hits tab; checked every
   15 minutes, once per stage and well);
-- `condition.cleared` - one of motor.due, penetrator.outside, dossier.changed or invoice.differs is no longer so.
+- `tls.expiring` - the dashboard's HTTPS certificate is in its last 30 days, or has expired (checked every 15
+  minutes while HTTPS is on);
+- `condition.cleared` - one of motor.due, penetrator.outside, dossier.changed, invoice.differs or tls.expiring is no
+  longer so.
 
 Every alert is written to `records/alerts.jsonl` whether or not any notification channel is set up - the Alarms
 tab lists them - and is handed to the notification rules, which send it where the site said (a rule per event
@@ -32,7 +35,7 @@ from datetime import datetime, timezone
 from typing import Dict, List, Optional
 
 EVENTS = ('seismic.protocol', 'seismic.event', 'seismic.watch_down', 'seismic.watch_up', 'motor.due', 'penetrator.outside',
-          'dossier.changed', 'invoice.differs', 'frac.hit', 'condition.cleared')
+          'dossier.changed', 'invoice.differs', 'frac.hit', 'tls.expiring', 'condition.cleared')
 CONDITIONS_EVERY_S = 900          # motors and penetrators: every 15 minutes
 DOSSIERS_EVERY_S = 86400          # dossiers: once a day
 _LOCK = threading.Lock()
@@ -172,6 +175,16 @@ def check_conditions(ws, notifier, now: Optional[datetime] = None, force: bool =
                 st['frac_hits_seen'] = sorted(done)
         except Exception as ex:
             last['frac_error'] = f'{type(ex).__name__}: {ex}'
+        try:
+            from . import tls as TLS
+            ts = TLS.status(ws, now)
+            if ts.get('configured') or 'tls.expiring' in st['conditions']:
+                f_ = ts.get('facts') or {}
+                _transition(ws, notifier, st, 'tls.expiring', ts['state'] in ('EXPIRING', 'EXPIRED'),
+                            {'event': 'tls.expiring', 'what': f"the dashboard's HTTPS certificate {f_.get('subject', '')}".strip(),
+                             'not_after': f_.get('not_after'), 'days_left': ts.get('days_left'), 'detail': ts['detail']}, now, raised)
+        except Exception as ex:
+            last['tls_error'] = f'{type(ex).__name__}: {ex}'
         for key in [k for k in st['conditions'] if k.split(':')[0] in ('motor.due', 'penetrator.outside') and k not in seen]:
             st['conditions'].pop(key, None)                  # the run or the record was removed: nothing left to clear
     if due('dossiers', DOSSIERS_EVERY_S):

{gea_program-0.54.0 → gea_program-0.55.0}/gea/doctor.py

@@ -253,6 +253,18 @@ def check_workspace(path: str, host: str = '127.0.0.1', port: int = 8765) -> Lis
     else:
         f.append(_finding('block', f'port {port} on {host} is already in use (another gea serve, or another program)',
                           f'stop the other service, or start with --port {port + 1}'))
+    try:
+        from . import tls as TLS
+        ts = TLS.status(ws)
+        lvl = {'OFF': 'info', 'OK': 'ok', 'EXPIRING': 'warn', 'EXPIRED': 'warn', 'NOT_YET_VALID': 'warn', 'BROKEN': 'block'}[ts['state']]
+        fix = {'OFF': f'for the site network: gea tls --workspace "{path}" --site-cert <the .crt or .pem> --site-key <its key>, or --self-signed',
+               'EXPIRING': f'gea tls --workspace "{path}" shows the certificate; renew it (or --self-signed for a new one) and restart',
+               'EXPIRED': f'gea tls --workspace "{path}" --self-signed makes a new one; a site certificate is renewed by the site',
+               'NOT_YET_VALID': "set this machine's clock",
+               'BROKEN': f'put the certificate and key files right, or turn HTTPS off: gea tls --workspace "{path}" --off'}.get(ts['state'], '')
+        f.append(_finding(lvl, f"HTTPS {ts['state']}: {ts['detail']}", fix))
+    except Exception as e:
+        f.append(_finding('warn', f'the HTTPS setting could not be read: {e}'))
     prev = _SV.previous_run(ws.path)
     if prev['status'] == 'UNCLEAN':
         f.append(_finding('warn', 'the last run of the control panel was killed rather than stopped: ' + prev['detail'],

{gea_program-0.54.0 → gea_program-0.55.0}/gea/help/alerts.md

@@ -6,7 +6,7 @@
 
 **How the watch reads the catalogue**: it runs in the service's background poller, so only while `gea serve` (the control panel) is running. An operation is under way when a declared operation (Operations & Seismicity, or the declarations sheet's `[operations]`) has started and has not ended, at a well with a surface position. A poll reads the USGS real-time summary feed of the past day, all magnitudes, updated every minute - the feed USGS asks automated programs to use - and keeps the earthquakes inside the circle around the operating wells (out to the screening distance, 20 km by default, beyond the farthest). On the first poll, or after a gap of a day or more, it asks the FDSN event service for the whole gap instead (from the operation's start, at most 30 days back). USGS revises magnitudes and locations; the newest version wins, and a revision that reaches a higher protocol level is a new alert.
 
-**The alerts**: `seismic.protocol` (a protocol level reached, once per level), `seismic.event` (any earthquake within the association radius during or after an operation, once), `seismic.watch_down` (no answer from the catalogue for three intervals while an operation runs) and `seismic.watch_up`; `motor.due` (checked every 15 minutes), `penetrator.outside` (every 15 minutes), `dossier.changed` (the newest dossier of each well and of the pad, re-checked daily), `invoice.differs` (a well's invoice cross-check finds a day that differs; checked daily and when a file is attached), `frac.hit` (a fracturing stage reads as a frac hit on a monitored well's pressure; checked every 15 minutes, once per stage and well), and `condition.cleared` when one of motor.due, penetrator.outside, dossier.changed or invoice.differs ends.
+**The alerts**: `seismic.protocol` (a protocol level reached, once per level), `seismic.event` (any earthquake within the association radius during or after an operation, once), `seismic.watch_down` (no answer from the catalogue for three intervals while an operation runs) and `seismic.watch_up`; `motor.due` (checked every 15 minutes), `penetrator.outside` (every 15 minutes), `dossier.changed` (the newest dossier of each well and of the pad, re-checked daily), `invoice.differs` (a well's invoice cross-check finds a day that differs; checked daily and when a file is attached), `frac.hit` (a fracturing stage reads as a frac hit on a monitored well's pressure; checked every 15 minutes, once per stage and well), `tls.expiring` (the control panel's HTTPS certificate in its last 30 days or expired; every 15 minutes while HTTPS is on - `gea help https`), and `condition.cleared` when one of motor.due, penetrator.outside, dossier.changed, invoice.differs or tls.expiring ends.
 
 **Getting them to a person**: every alert is written to `records/alerts.jsonl` and shown on the Alarms tab. To have it sent, set up notifications (Administration, Notifications: load the example, edit, validate and commit) with an email (SMTP) or webhook channel and a rule naming the alert, for example `{"event": "seismic.protocol", "channels": ["mail"]}`; the password stays in an environment variable named by `password_env`. A text message: most carriers deliver email sent to the phone's email-to-text address, or a webhook to a messaging service. The `quiet_s` window keeps one alert from repeating.
 

gea_program-0.55.0/gea/help/https.md

@@ -0,0 +1,19 @@
+# HTTPS
+
+> How browsers on the site network reach the control panel without anything in plain text: the site's own certificate (best: the company's certificate authority, which managed computers already trust) or a self-signed one made here; the names it must carry; handing it out; renewal and the expiry alert.
+
+**The command**: Administration -> **HTTPS** (the administrator role): how it stands, the certificate's names, validity, issuer and SHA-256 fingerprint, **Download the certificate**, and under **Change it**: **Use the site's certificate** (the certificate file and its private key, as paths on the machine running the service), **Make a self-signed certificate** (the names it is reached by, the days), **Turn HTTPS off**. A change takes effect at the next start: **Restart** under Operations control. From a terminal: `gea tls --workspace C:\site` prints how it stands; `--site-cert C:\certs\gea-pad3.crt --site-key C:\certs\gea-pad3.key` uses the site's; `--self-signed --name gea-pad3 --name gea-pad3.example.local --name 10.20.0.15` makes one (with no `--name`: this machine's names, localhost and 127.0.0.1); `--off`; `--export-cert C:\certs\gea-site.crt` writes the certificate (never the key) for the site's computers. Then `gea serve --workspace C:\site --host 0.0.0.0` (or the kit's start-dashboard) and open `https://gea-pad3:8765/`.
+
+**What it needs**: for the site's certificate, PEM files: the server certificate first (then any chain), and its private key without a passphrase (the service starts unattended); a `.pfx` is exported to PEM first (`openssl pkcs12 -in gea.pfx -out gea.crt -nokeys` and `-nocerts -nodes -out gea.key`), and the key file is readable by the service's account only. The certificate must carry every name the browsers use - the machine's short name, its full name, its address - as a subject alternative name. The site's files are read where they are, at every start: a renewal put in their place needs only a restart.
+
+**How it is read**: TLS 1.2 is the floor (TLS 1.3 when both ends have it). With HTTPS on, the session cookie is marked Secure, every response tells the browser to use only HTTPS from then on (Strict-Transport-Security, 180 days), and a plain `http://` request on the same port is sent to the same address over HTTPS when it named the machine by a name the certificate carries (otherwise it is told to use HTTPS). Each connection's handshake runs in its own thread, so a scanner or an old client cannot hold up the others. The self-signed certificate is made by this program alone: an RSA key of 2048 bits (3072 on request), an X.509 v3 certificate for a TLS server (not a certificate authority), signed with SHA-256, valid for at most 825 days (Apple's systems refuse a longer one even when trusted by hand). A browser warns about a self-signed certificate until the computer trusts it: by group policy across the site, or by hand (open the downloaded `.crt`, Install Certificate, Local Machine, Trusted Root Certification Authorities) - and the fingerprint shown there is compared with the one on the HTTPS card before it is trusted. A workspace whose certificate files are missing, unreadable or not a pair does not start (`gea doctor` says BLOCK): a site that turned HTTPS on is never served in plain text by surprise. A site already running a reverse proxy can keep TLS there instead (`--behind-proxy`, `gea help upkeep`).
+
+**What it writes**: the setting in `workspace.json` (`control.tls`: the mode, the two files, when and by whom) with the audit lines `tls.site_certificate`, `tls.self_signed` and `tls.off`; for a self-signed certificate, `keys\tls\self_signed_certificate.pem` and `keys\tls\self_signed_key.pem`. The key is never in a report, an export, an evidence pack or a dossier, and the service never sends it; the backup copies it with the rest of the workspace, like the dossier signing key. The alert `tls.expiring` when the certificate is in its last 30 days or has expired, checked every 15 minutes, and cleared when a new one is in place.
+
+**The number to check**: the SHA-256 fingerprint on the HTTPS card against the one the browser shows for the page (the padlock -> the certificate) - the same, or it is not this machine answering; the days left.
+
+**What this page will not call a measurement**: a padlock is not proof of who is answering unless the computer trusts the certificate for a reason - the site's authority, or a fingerprint someone compared. A self-signed certificate encrypts; it does not identify until it is trusted that way.
+
+**Sources**: RFC 8446 (TLS 1.3) and RFC 5246 (TLS 1.2); RFC 5280 (the X.509 certificate and its extensions); RFC 8017 (RSA, PKCS #1 v1.5 signatures); RFC 6797 (Strict-Transport-Security); Apple, "Requirements for trusted certificates in iOS 13 and macOS 10.15" (HT210176: a server certificate valid for at most 825 days, with its names as subject alternative names and the server-authentication key usage).
+
+Related: `gea help upkeep`, `gea help alerts`, `gea help doctor`.

{gea_program-0.54.0 → gea_program-0.55.0}/gea/help/upkeep.md

@@ -2,7 +2,7 @@
 
 > Keeping a site running for years: logs, jobs, sessions, the proxy, the load it can carry.
 
-**The command**: `gea housekeeping --workspace C:\site` (a dry run that lists what it would do) and `--apply`; schedule it daily from Administration (preset "housekeeping"). `gea loadtest --patches 16 --seconds 60 --with-service` before a site goes live. Administration -> Live sessions lists who is signed in from where and revokes one or all of a user's sessions; "Your preferences" has "Sign out everywhere else". `deploy/README.md`, `Caddyfile` and `nginx-gea.conf` put TLS in front; start the service with `--behind-proxy`.
+**The command**: `gea housekeeping --workspace C:\site` (a dry run that lists what it would do) and `--apply`; schedule it daily from Administration (preset "housekeeping"). `gea loadtest --patches 16 --seconds 60 --with-service` before a site goes live. Administration -> Live sessions lists who is signed in from where and revokes one or all of a user's sessions; "Your preferences" has "Sign out everywhere else". HTTPS without a proxy: `gea help https` (the service speaks TLS itself with the site's certificate or a self-signed one). Or, where the site already runs one, `deploy/README.md`, `Caddyfile` and `nginx-gea.conf` put TLS in a reverse proxy in front; start the service with `--behind-proxy`.
 
 **What it writes**: housekeeping segments the append-only logs (`audit.jsonl.<stamp>`, `alarm_events.jsonl.<stamp>`; the fresh alarm log starts at the carried PROCESSED watermark so nothing is re-processed), removes finished job folders older than 30 days beyond the newest 500, removes live recordings older than 90 days except the newest stream file per well, and audits the run. The load test leaves nothing behind.
 
@@ -10,4 +10,4 @@
 
 **What this page will not call a measurement**: a segmented log is not a deleted one - every segment is the record. Deletion is only ever of finished job folders and old recordings whose samples are already folded into the stream.
 
-Related: `gea help site`, `gea help patches`.
+Related: `gea help site`, `gea help patches`, `gea help https`.

{gea_program-0.54.0 → gea_program-0.55.0}/gea/helplib.py

@@ -84,6 +84,7 @@ INDEX: List[tuple] = [
     ('doctor', 'Which code is running', 'gea doctor and the serve start-up gate'),
     ('audit-update', 'Audit / Update', 'the whole audit log with filters, the program against PyPI, every report against its source'),
     ('soak', 'The soak', 'the patches through link outages for hours: reconnection, resume, loss by protocol, the page answering - measured, not modelled'),
+    ('https', 'HTTPS', "the control panel over HTTPS on the site network: the site's own certificate or a self-signed one made here, the names it carries, handing it out, renewal and the expiry alert"),
     ('upkeep', 'Upkeep', 'housekeeping, sessions, the TLS proxy, the load test'),
 ]
 # the four lines every page must carry, in this order

{gea_program-0.54.0 → gea_program-0.55.0}/gea/notify.py

@@ -68,7 +68,7 @@ EXAMPLE = {
 }
 EVENTS = ('alarm.activated', 'alarm.shelved', 'job.failed', 'patch.down', 'patch.up', 'approval.pending', 'file.imported', 'test',
           'seismic.protocol', 'seismic.event', 'seismic.watch_down', 'seismic.watch_up', 'motor.due', 'penetrator.outside', 'dossier.changed',
-          'condition.cleared', 'pad.summary', 'invoice.differs', 'frac.hit')
+          'condition.cleared', 'pad.summary', 'invoice.differs', 'frac.hit', 'tls.expiring')
 _LOCK = threading.Lock()
 
 
@@ -185,6 +185,8 @@ def _subject(ev: dict) -> str:
         return f"[GEA] the workspace no longer matches dossier {ev.get('dossier_id', '')}"
     if k == 'invoice.differs':
         return f"[GEA] the invoice for {w} differs from the reports and the record: {ev.get('detail', '')}"
+    if k == 'tls.expiring':
+        return f"[GEA] HTTPS certificate: {ev.get('detail', '')}"
     if k == 'frac.hit':
         return f"[GEA] frac hit: stage {ev.get('stage', '')} of {w} on {ev.get('monitor', '')} - {ev.get('detail', '')}"
     if k == 'pad.summary':