gea-program 0.16.0__tar.gz → 0.17.1__tar.gz

{gea_program-0.16.0/gea_program.egg-info → gea_program-0.17.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gea-program
-Version: 0.16.0
+Version: 0.17.1
 Summary: GEA-Program: downhole gauge monitoring - canonical measurement records, quality rules, drift evaluation with a re-fit change log, well-test validation, alarm management, model cards, store-and-forward, monthly SLA measurement, FAT/SAT protocols and a dashboard, generated from live objects in the client's reporting outline.
 Author: Daniel T. Murphy
 License: MPL-2.0
@@ -72,7 +72,7 @@ reaches a client document. Nothing unmeasured is ever reported as met.
   rate-controlled replay, duplicate suppression, per-record latency.
 - **Configuration versioning, SBOM, monthly SLA, FAT/SAT**
   (`config_versioning.py`, `sbom.py`, `sla_report.py`, `fat_sat.py`).
-- **Live protocol ports** (`live_ports.py`, `wits0.py`, `witsml.py`, `etp.py`, `opcua_port.py`,
+- **Live protocol ports** (`live_ports.py`, `wits0.py`, `witsml.py`, `etp.py`, `seedlink.py`, `opcua_port.py`,
   `mqtt_port.py`, `modbus.py`): WITS Level 0 from the drill floor (TCP connect,
   TCP listen, serial), WITSML 1.4.1 stores (read-only, polled), OPC UA (read and
   subscribe), MQTT (number, JSON, Sparkplug B), Modbus TCP; every message becomes
@@ -125,7 +125,7 @@ gea dashboard --catalog-well volve_f12_f14_production_excerpt:15/9-F-12:10000 --
 gea client-report --report accuracy --out client_report
 gea model-cards --out model_cards
 gea sbom --out sbom
-gea accept                     # the product gate (337 checks)
+gea accept                     # the product gate (340 checks)
 gea help drift                 # the help library, by the job (16 pages; the same text is on every dashboard page)
 gea guide                      # the click-by-click tester guide (docs/TESTER_GUIDE.md)
 gea gui                        # the desktop window (pip install "gea-program[desktop]")
@@ -623,6 +623,31 @@ promoted; the group and the support are declarations; a matched frequency
 is not a fault size. CSV or miniSEED/SAC; `gea vibration` runs the labelled
 pump; `gea help vibration`.
 
+### Live stations (SeedLink)
+
+The seismic leg had run on records that were already files. The stations
+serve SeedLink - the real-time protocol every data centre and most
+digitisers speak - and `gea/seedlink.py` is the customer: the TCP session on
+port 18000, `HELLO` read for what the server offers, SeedLink 3 (`STATION
+STA NET`, `SELECT LLCCC.T`, `DATA` with a hexadecimal sequence, `END`, the
+8-byte `SL` header on each 512-byte record) or SeedLink 4.0 (`SLPROTO 4.0`,
+`STATION NET_STA`, `SELECT LOC_B_S_SS`, a decimal sequence, the `SE` header
+with its length, uint64 sequence and station id, `ERROR <CODE>`), whichever
+the server speaks. Every record is appended byte for byte to a day file per
+channel under the station's `live/` folder with the state beside it - the
+last sequence number per station, so a reconnection (with backoff) resumes
+where it stopped and no record is filed twice; per channel the last sample,
+the latency, the records, the samples, the gaps (counted, never bridged).
+On the dashboard's Seismic page a live station is added with its host, port,
+network, station and selectors, started and stopped (the serving process
+keeps every enabled one up), and **folded**: the finished day files move into
+the station's record list, hashed, vertical first, and Refresh runs the leg
+on them as on any brought file. `gea seedlink-sim` is a three-component
+synthetic station speaking both versions; `gea seedlink --selftest` runs both
+against it; `gea seedlink --config ... --hello` prints what a server says of
+itself; `gea help seedlink`. A TexNet station is served by EarthScope at
+`rtserve.iris.washington.edu:18000`, network `TX`.
+
 ### WITSML 2.x over ETP
 
 The newer rigs and stores do not poll: Energistics ETP v1.2 is a WebSocket

{gea_program-0.16.0 → gea_program-0.17.1}/README.md

@@ -33,7 +33,7 @@ reaches a client document. Nothing unmeasured is ever reported as met.
   rate-controlled replay, duplicate suppression, per-record latency.
 - **Configuration versioning, SBOM, monthly SLA, FAT/SAT**
   (`config_versioning.py`, `sbom.py`, `sla_report.py`, `fat_sat.py`).
-- **Live protocol ports** (`live_ports.py`, `wits0.py`, `witsml.py`, `etp.py`, `opcua_port.py`,
+- **Live protocol ports** (`live_ports.py`, `wits0.py`, `witsml.py`, `etp.py`, `seedlink.py`, `opcua_port.py`,
   `mqtt_port.py`, `modbus.py`): WITS Level 0 from the drill floor (TCP connect,
   TCP listen, serial), WITSML 1.4.1 stores (read-only, polled), OPC UA (read and
   subscribe), MQTT (number, JSON, Sparkplug B), Modbus TCP; every message becomes
@@ -86,7 +86,7 @@ gea dashboard --catalog-well volve_f12_f14_production_excerpt:15/9-F-12:10000 --
 gea client-report --report accuracy --out client_report
 gea model-cards --out model_cards
 gea sbom --out sbom
-gea accept                     # the product gate (337 checks)
+gea accept                     # the product gate (340 checks)
 gea help drift                 # the help library, by the job (16 pages; the same text is on every dashboard page)
 gea guide                      # the click-by-click tester guide (docs/TESTER_GUIDE.md)
 gea gui                        # the desktop window (pip install "gea-program[desktop]")
@@ -584,6 +584,31 @@ promoted; the group and the support are declarations; a matched frequency
 is not a fault size. CSV or miniSEED/SAC; `gea vibration` runs the labelled
 pump; `gea help vibration`.
 
+### Live stations (SeedLink)
+
+The seismic leg had run on records that were already files. The stations
+serve SeedLink - the real-time protocol every data centre and most
+digitisers speak - and `gea/seedlink.py` is the customer: the TCP session on
+port 18000, `HELLO` read for what the server offers, SeedLink 3 (`STATION
+STA NET`, `SELECT LLCCC.T`, `DATA` with a hexadecimal sequence, `END`, the
+8-byte `SL` header on each 512-byte record) or SeedLink 4.0 (`SLPROTO 4.0`,
+`STATION NET_STA`, `SELECT LOC_B_S_SS`, a decimal sequence, the `SE` header
+with its length, uint64 sequence and station id, `ERROR <CODE>`), whichever
+the server speaks. Every record is appended byte for byte to a day file per
+channel under the station's `live/` folder with the state beside it - the
+last sequence number per station, so a reconnection (with backoff) resumes
+where it stopped and no record is filed twice; per channel the last sample,
+the latency, the records, the samples, the gaps (counted, never bridged).
+On the dashboard's Seismic page a live station is added with its host, port,
+network, station and selectors, started and stopped (the serving process
+keeps every enabled one up), and **folded**: the finished day files move into
+the station's record list, hashed, vertical first, and Refresh runs the leg
+on them as on any brought file. `gea seedlink-sim` is a three-component
+synthetic station speaking both versions; `gea seedlink --selftest` runs both
+against it; `gea seedlink --config ... --hello` prints what a server says of
+itself; `gea help seedlink`. A TexNet station is served by EarthScope at
+`rtserve.iris.washington.edu:18000`, network `TX`.
+
 ### WITSML 2.x over ETP
 
 The newer rigs and stores do not poll: Energistics ETP v1.2 is a WebSocket

{gea_program-0.16.0 → gea_program-0.17.1}/gea/__init__.py

@@ -123,7 +123,7 @@ from .profile_catalog import (
     read_pangaea_txt,
 )
 
-__version__ = "0.16.0"
+__version__ = "0.17.1"
 __all__ = [
     "aging_rate", "rate_psi_yr", "accuracy_psi",
     "Sensor", "SimulatorConfig", "DownholeEngine",

{gea_program-0.16.0 → gea_program-0.17.1}/gea/__main__.py

@@ -284,7 +284,8 @@ def main(argv=None) -> int:
                       choices=["init", "add-file", "add-catalog", "add-live", "remove", "list", "migrate", "refresh", "audit",
                                "add-seismic", "refresh-seismic", "remove-seismic", "sar-film", "refresh-all", "add-track", "refresh-track", "remove-track",
                                "seismic-dataset", "seismic-field", "add-site", "sites", "site-report", "remove-site", "rename",
-                               "set-well", "sra-define", "sra-report", "associate", "osdu-export", "quakeml-export", "well-identity", "vibration-report"])
+                               "set-well", "sra-define", "sra-report", "associate", "osdu-export", "quakeml-export", "well-identity", "vibration-report",
+                               "add-live-station", "live-fold", "live-run"])
     p_ws.add_argument("--stations", type=str, nargs="+", default=None, help="add-track: two or more array station ids of this workspace")
     p_ws.add_argument("--site", type=str, default=None, help="site-report/remove-site: the site id")
     p_ws.add_argument("--api", type=str, default=None, help="set-well: the API number")
@@ -354,7 +355,10 @@ def main(argv=None) -> int:
     p_ws.add_argument("--unit", type=str, default="m", help="add-seismic with --permits and --zone: the grid unit (m, usft, ft)")
     p_ws.add_argument("--band", type=float, nargs=2, default=None, metavar=("LO", "HI"), help="add-seismic: the band, Hz (default 1 50)")
     p_ws.add_argument("--permits", type=str, default=None, help="add-seismic: a permit export to turn into the rigs CSV (instead of --sources)")
-    p_ws.add_argument("--station", type=str, default=None, help="refresh-seismic / remove-seismic: the station id")
+    p_ws.add_argument("--station", type=str, default=None, help="refresh-seismic / remove-seismic / live-fold / live-run: the station id")
+    p_ws.add_argument("--seedlink-config", type=str, default=None, help="add-live-station: the SeedLink config JSON (gea seedlink --write-example-config)")
+    p_ws.add_argument("--include-open", action="store_true", help="live-fold: fold the current day's files too (they are still being written)")
+    p_ws.add_argument("--seconds", type=float, default=None, help="live-run: how long to run (default: until Ctrl+C)")
     p_ws.add_argument("--name", type=str, default=None, help="init: site name; add-*: display name")
     p_ws.add_argument("--file", type=str, default=None, help="add-file: the client's data file; add-live: the port configuration JSON")
     p_ws.add_argument("--entry", type=str, default=None, help="add-catalog: catalogue entry")
@@ -407,6 +411,21 @@ def main(argv=None) -> int:
     p_es.add_argument("--frames", type=int, default=600)
     p_es.add_argument("--interval", type=float, default=1.0, help="seconds between samples")
     p_es.add_argument("--seed", type=int, default=1)
+    p_sl = sub.add_parser("seedlink", help="the seismic leg's live port: a SeedLink (3 / 4.0) station's records as it writes them, byte for byte into day files; --selftest against the simulator")
+    p_sl.add_argument("--config", type=str, default=None, help="the SeedLink config JSON (host, port, streams)")
+    p_sl.add_argument("--write-example-config", type=str, default=None, help="write an example config here and exit")
+    p_sl.add_argument("--out", type=str, default="seedlink", help="the folder for the day files and the state (default ./seedlink)")
+    p_sl.add_argument("--seconds", type=float, default=60.0, help="how long to run (the state resumes the next run from the last sequence number)")
+    p_sl.add_argument("--max-records", type=int, default=None, help="stop after this many records")
+    p_sl.add_argument("--selftest", action="store_true", help="both protocols against the simulator on the loopback")
+    p_sl.add_argument("--hello", action="store_true", help="connect, print what the server says of itself (HELLO) and its INFO ID, and exit")
+    p_ss = sub.add_parser("seedlink-sim", help="a SeedLink server (3 and 4.0) for rehearsing the live station without a data centre: one synthetic three-component station")
+    p_ss.add_argument("--port", type=int, default=18000)
+    p_ss.add_argument("--network", type=str, default="XX")
+    p_ss.add_argument("--station", type=str, default="PAD3")
+    p_ss.add_argument("--channels", type=str, default="HHZ,HHN,HHE")
+    p_ss.add_argument("--rate", type=float, default=100.0, help="samples per second")
+    p_ss.add_argument("--seed", type=int, default=7)
     p_w0 = sub.add_parser("wits0-sim", help="a WITS0 sender for testing a patch without a rig: listens once, then streams a deterministic drilling sequence")
     p_w0.add_argument("--port", type=int, default=5001)
     p_w0.add_argument("--frames", type=int, default=600)
@@ -1915,6 +1934,31 @@ def main(argv=None) -> int:
                     raise SystemExit("well-identity needs --well <id>")
                 r = ws.well_identity(a.well)
                 print(_ppdm_text(r)); return 0
+            elif a.action == "add-live-station":
+                if not a.seedlink_config or not a.name:
+                    raise SystemExit("add-live-station needs --name <display> and --seedlink-config <json>")
+                st = ws.add_live_station(a.name, a.seedlink_config, a.lat, a.lon, actor=a.actor, band=a.band, stationxml=a.stationxml, sources_csv=a.sources)
+                print(f"live station {st['id']}: {st['live']['host']}:{st['live']['port']} {st['live']['stations']} - run `gea workspace --path ... --action live-run --station {st['id']}` or Start it on the page")
+                return 0
+            elif a.action == "live-fold":
+                if not a.station:
+                    raise SystemExit("live-fold needs --station <id>")
+                r = ws.fold_live_station(a.station, actor=a.actor, include_open=a.include_open)
+                print(f"folded {len(r['folded'])} file(s) into {a.station}: {r['folded']}; still open: {r['kept_open']}")
+                return 0
+            elif a.action == "live-run":
+                from . import seedlink as _L
+                if not a.station:
+                    raise SystemExit("live-run needs --station <id>")
+                cfg = ws.live_station_config(a.station)
+                d = ws.live_dir(a.station)
+                tap = _L.SeedLinkTap(cfg, d, os.path.join(d, "seedlink_state.json"))
+                try:
+                    st = tap.run(a.seconds if a.seconds else None)
+                except KeyboardInterrupt:
+                    tap.stop(); st = tap.state
+                print(_L.report_text(st))
+                return 0 if st["records"] else 1
             elif a.action == "quakeml-export":
                 from .quakeml import report_text as _qml_text
                 if not a.site:
@@ -1973,6 +2017,60 @@ def main(argv=None) -> int:
             stop.set()
             print("wits0-sim: stopped")
         return 0
+    elif a.cmd == "seedlink":
+        import json as _json
+        from . import seedlink as _L
+        if a.selftest:
+            r = _L.selftest()
+            print(_json.dumps({k: r[k] for k in ("label", "status", "checks")}, indent=1))
+            print(_json.dumps(r["detail"], indent=1, default=str))
+            return 0 if r["status"] == "OK" else 1
+        if a.write_example_config:
+            print("config:", _L.write_example_config(a.write_example_config))
+            return 0
+        if not a.config:
+            raise SystemExit("seedlink needs --config (or --write-example-config to start one, or --selftest)")
+        try:
+            cfg = _L.load_config(a.config)
+        except (ValueError, OSError) as e:
+            raise SystemExit(f"seedlink: {e}")
+        if a.hello:
+            try:
+                cl = _L.SeedLinkClient(cfg["host"], cfg["port"], cfg["timeout_s"]).connect()
+                h = cl.say_hello()
+                print(f"seedlink: {cfg['host']}:{cfg['port']} says {h['software']!r}; organization {cl.organization!r}; protocols {h['protocols']}; capabilities {h['capabilities']}")
+                cl.info("ID")
+                cl.sock.settimeout(cfg["timeout_s"])
+                pkt = cl.read_packet()
+                print(f"  INFO ID: {pkt['kind']} packet, SeedLink {pkt.get('protocol')}, {len(pkt.get('payload') or b'')} bytes")
+                cl.close()
+            except (OSError, ValueError, _L.SeedLinkError) as e:
+                raise SystemExit(f"seedlink: {e}\n  no SeedLink server answers at that host:port - start `gea seedlink-sim --port {cfg['port']}` in a SECOND window first, or point the config at the data centre (rtserve.iris.washington.edu:18000 for TexNet)")
+            return 0
+        tap = _L.SeedLinkTap(cfg, a.out)
+        try:
+            st = tap.run(a.seconds, max_records=a.max_records)
+        except KeyboardInterrupt:
+            tap.stop(); st = tap.state
+        print(_L.report_text(st))
+        if st["status"] == "REFUSED" or (st["records"] == 0 and st.get("last_error")):
+            print(f"  no records: start `gea seedlink-sim --port {cfg['port']}` in a SECOND window first, or point the config at the data centre (rtserve.iris.washington.edu:18000, network TX for TexNet)")
+            return 1
+        return 0
+    elif a.cmd == "seedlink-sim":
+        from . import seedlink as _L
+        srv = _L.simulate_server(port=a.port, network=a.network, station=a.station, channels=[c.strip().upper() for c in a.channels.split(",") if c.strip()], rate=a.rate, seed=a.seed)
+        print(f"seedlink-sim: a SeedLink server (3.1 and 4.0) listening on 127.0.0.1:{srv.port} - this window is the STATION and stays here; it writes {a.network}.{a.station} "
+              f"({a.channels}) at {a.rate:g} Hz in 512-byte records and serves them as customers ask.\n"
+              f"  Open a SECOND window and run:  gea seedlink --write-example-config seedlink.json; gea seedlink --config seedlink.json --seconds 60 --out live\n"
+              f"  or on the dashboard: Seismic -> Live stations -> Add (host 127.0.0.1, port {srv.port}, network {a.network}, station {a.station}) -> Start. Ctrl+C stops it.", flush=True)
+        try:
+            while srv.is_alive():
+                srv.join(0.5)
+        except KeyboardInterrupt:
+            srv.stop()
+            print("seedlink-sim: stopped")
+        return 0
     elif a.cmd == "etp-sim":
         from . import etp as _E
         th, port, stop = _E.simulate_store(port=a.port, frames=a.frames, interval_s=a.interval, seed=a.seed, verbose=True)

{gea_program-0.16.0 → gea_program-0.17.1}/gea/acceptance_tests.py

@@ -50,6 +50,7 @@ import re
 import subprocess
 import sys
 import tempfile
+import threading
 import time
 from pathlib import Path
 
@@ -5581,6 +5582,143 @@ def section_bj_etp(tmp: str) -> None:
        "launcher while it runs, so the program cannot be updated in place - stop, update, start with `python -m gea serve`; a panel started "
        "with `python -m gea` draws no such finding, and `gea update` from a launcher-started panel on Windows refuses to run pip and says why")
 
+
+def section_bk_seedlink(tmp: str) -> None:
+    """Section BK - the seismic leg's live port: SeedLink 3 and 4.0 against the simulator on the loopback, the records
+    byte for byte into day files, the resume by sequence number across a dropped link with no record twice, the live
+    station in the workspace, the service and the page, folded into the record list and refreshed."""
+    import http.cookiejar
+    import subprocess as _sp
+    import urllib.request
+    import urllib.error
+    from . import seedlink as L
+    from . import seismic as S
+    from . import helplib as H
+    from .workspace import Workspace
+    from .service import Service, Users
+    pkg = Path(__file__).parent
+    d = Path(tmp, "bk"); d.mkdir()
+
+    # -- the protocol, both versions, on the loopback ----------------------------------------------------------------
+    r = L.selftest(tmp=str(d / "self"))
+    h4 = L.parse_hello("SeedLink v4.0 (RingServer/2022.075) :: SLPROTO:3.1 SLPROTO:4.0 CAP EXTREPLY NSWILDCARD BATCH WS:13")
+    h3 = L.parse_hello("SeedLink v3.3 (2020.122)")
+    import numpy as _np
+    tr = S.Trace("XX", "PAD3", "", "HHZ", 1.7e9, 100.0, _np.arange(500, dtype=_np.int64))
+    recs = S.build_records([tr], "INT32", 512)
+    back = S.read_mseed(b"".join(recs))
+    sel_err = None
+    try:
+        L.parse_selector("!HHZ")
+    except ValueError as e:
+        sel_err = str(e)
+    ok(r["status"] == "OK" and all(r["checks"].values()) and r["label"] == "SIMULATION_SELF_TEST"
+       and h4["protocols"] == ["3.1", "4.0"] and "NSWILDCARD" in h4["capabilities"] and h3["protocols"] == ["3.0"]
+       and L.selector_v4("HHZ.D") == "*_H_H_Z.2D" and L.selector_v4("00HHZ") == "00_H_H_Z" and L.parse_selector("HHZ.D") == ("", "HHZ", "D")
+       and sel_err and "negative" in sel_err
+       and len(recs) == 5 and all(len(x) == 512 for x in recs) and recs[0][:6] == b"000001" and recs[4][:6] == b"000005"
+       and len(back) == 1 and back[0].npts == 500 and not back[0].gaps,
+       "BK1 SeedLink 3 and 4.0 as the specifications have them, proven against the simulator on the loopback: HELLO parsed for the protocols and "
+       "capabilities offered (a 3 server lists none), `STATION STA NET` / `SELECT LLCCC.T` / `DATA hhhhhh` / `END` answered OK and the 8-byte "
+       "`SL` header with its hexadecimal sequence; `SLPROTO 4.0` / `STATION NET_STA` / `SELECT LOC_B_S_SS` / `DATA n` and the `SE` header with "
+       "its little-endian length and uint64 sequence and the station id; INFO answered in a log record (3) and in JSON (4.0); a selector honoured; "
+       "the records appended byte for byte to one day file per channel that reads back whole; a second run resuming from the saved sequence "
+       "number asks DATA for the next and adds no record twice; a station the server does not have is REFUSED and named, with no reconnect loop; "
+       "the 512-byte INT32 records the simulator serves are the writer's own and read back as one trace")
+
+    # -- a dropped link: the tap reconnects and resumes, nothing twice ------------------------------------------------
+    srv = L.simulate_server(interval_s=0.1, rate=250.0)
+    time.sleep(1.2)
+    cfg = dict(L.EXAMPLE_CONFIG, host="127.0.0.1", port=srv.port, protocol="3", streams=[{"network": "XX", "station": "PAD3", "selectors": ["HHZ.D"]}],
+               reconnect_s=[0.3, 0.3])
+    tap = L.SeedLinkTap(cfg, str(d / "drop"))
+    th = threading.Thread(target=tap.run, kwargs={"duration_s": 9.0}, daemon=True); th.start()
+    time.sleep(2.5)
+    n_before = tap.state["records"]
+    dropped = srv.drop_clients()
+    time.sleep(3.0)
+    tap.stop(); th.join(timeout=10)
+    st = tap.state
+    day = [f for f in os.listdir(d / "drop") if f.endswith(".mseed")]
+    trs = S.read_mseed(str(d / "drop" / day[0])) if day else []
+    ok(dropped >= 1 and n_before >= 2 and st["reconnects"] >= 1 and st["records"] > n_before and st["status"] == "STOPPED"
+       and len(day) == 1 and len(trs) == 1 and not trs[0].gaps and trs[0].npts == st["channels"]["XX.PAD3..HHZ"]["samples"] == st["records"] * 112
+       and any(l.startswith("> DATA ") and len(l.split()) == 3 for l in st["handshake"]) and st["channels"]["XX.PAD3..HHZ"]["gaps"] == 0,
+       "BK2 the link drops under a running session: the tap counts the reconnection, comes back with backoff, asks DATA for the sequence after "
+       "the last record it filed, and the day file reads back as one trace with no gap and no record twice - the records on disk equal the "
+       "records counted equal the samples over 112")
+
+    # -- the live station in the workspace, through the service and the page ---------------------------------------
+    wsp = str(d / "ws"); ws = Workspace.create(wsp, "Pad 3", actor="tester")
+    users = Users(str(Path(wsp, "users.json"))); users.add("op", "operator-pass-1", "operator")
+    svc = Service(wsp, port=0, scheduler=False).start()
+    base = svc.url.rstrip("/")
+    opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
+
+    def call(method, path, body=None):
+        req = urllib.request.Request(base + path, method=method, data=(json.dumps(body).encode() if body is not None else None))
+        req.add_header("Content-Type", "application/json"); req.add_header("X-GEA-Action", "1")
+        try:
+            with opener.open(req, timeout=120) as rr:
+                return rr.status, json.loads(rr.read())
+        except urllib.error.HTTPError as e:
+            return e.code, json.loads(e.read())
+    try:
+        call("POST", "/api/login", {"name": "op", "password": "operator-pass-1"})
+        st_bad = call("POST", "/api/seismic/live/add", {"display": "bad", "config": {"host": "127.0.0.1", "port": srv.port, "streams": []}})[0]
+        cfg_all = dict(L.EXAMPLE_CONFIG, host="127.0.0.1", port=srv.port, protocol="auto")
+        st_add, added = call("POST", "/api/seismic/live/add", {"display": "Pad 3 live", "config": cfg_all, "lat": 31.5, "lon": -103.2, "band": [1, 50]})
+        sid = added.get("id")
+        st_start, started = call("POST", f"/api/seismic/live/{sid}/start")
+        t0 = time.time(); row = {}
+        while time.time() - t0 < 20:                        # until the card shows six records and a latency, not a fixed sleep (Windows was slower)
+            st_list, lst = call("GET", "/api/seismic/live")
+            row = next((x for x in lst["stations"] if x["id"] == sid), {})
+            if row.get("records", 0) >= 6 and row.get("latency_p95_s") is not None and len(row.get("channels", {})) == 3:
+                break
+            time.sleep(0.5)
+        st_seis, seis = call("GET", "/api/seismic")
+        st_stop, stopped = call("POST", f"/api/seismic/live/{sid}/stop")
+        st_fold, folded = call("POST", f"/api/seismic/live/{sid}/fold", {"include_open": True})
+        st_fold2, folded2 = call("POST", f"/api/seismic/live/{sid}/fold", {"include_open": True})
+        st_404 = call("POST", f"/api/seismic/live/{sid}/dance")[0]
+    finally:
+        svc.stop()
+        srv.stop()
+    station = ws.seismic_station(sid)
+    ws.refresh_seismic(sid, actor="tester")
+    res = ws.seismic_results(sid)
+    audit = [a["action"] for a in ws.audit_log()]
+    page_src = (pkg / "web" / "app.html").read_text(encoding="utf-8")
+    r1 = _sp.run([sys.executable, "-m", "gea", "seedlink", "--write-example-config", str(d / "sl.json")], capture_output=True, text=True)
+    r2 = _sp.run([sys.executable, "-m", "gea", "seedlink", "--config", str(d / "sl.json"), "--seconds", "1", "--out", str(d / "none")], capture_output=True, text=True)
+    help_md = (pkg / "help" / "seedlink.md").read_text(encoding="utf-8")
+    c = {
+        "add_refused_and_added": st_bad == 409 and st_add == 200 and added["live"]["stations"] == ["XX_PAD3"] and added["files"] == [] and added["kind"] == "single",
+        "started": st_start == 200 and started["running"] is True,
+        "connected_on_card": st_list == 200 and row.get("status") == "CONNECTED" and row.get("protocol") == "4" and row.get("records", 0) >= 6 and row.get("running") is True,
+        "three_channels_latency": set(row.get("channels", {})) == {"XX.PAD3..HHZ", "XX.PAD3..HHN", "XX.PAD3..HHE"} and row.get("latency_p95_s") is not None,
+        "station_list_carries_live": st_seis == 200 and any(s_.get("id") == sid and s_.get("live_state", {}).get("status") == "CONNECTED" for s_ in seis["stations"]),
+        "stopped": st_stop == 200 and stopped["running"] is False and stopped["status"] == "STOPPED",
+        "folded_z_first": st_fold == 200 and len(folded["folded"]) == 3 and folded["folded"][0].split(".")[3] == "HHZ" and st_fold2 == 200 and folded2["folded"] == [],
+        "unknown_action_404": st_404 == 404,
+        "station_files_hashed": len(station["files"]) == 3 and all(n in station["sha256"] for n in station["files"]) and station["live"]["folds"] == 1 and station["live"]["enabled"] is False,
+        "refreshed_on_folded": res.get("summary", {}).get("records") == 3 and res.get("summary", {}).get("unit") == "counts",
+        "audited": all(x in audit for x in ("seismic.live.add", "seismic.live.start", "seismic.live.stop", "seismic.live.fold", "seismic.refresh")),
+        "page_card": "Live stations (SeedLink)" in page_src and "/api/seismic/live/add" in page_src,
+        "cli": r1.returncode == 0 and r2.returncode != 0 and "seedlink-sim" in (r2.stdout + r2.stderr),
+        "help": any(t["topic"] == "seedlink" for t in H.topics()) and not H.check("seedlink") and "sequence number" in help_md
+                and "across a gap" in help_md.split("will not call a measurement")[1],
+    }
+    failed = [k for k, v in c.items() if not v]
+    ok(not failed,
+       "BK3 a live station in the workspace: added through the service with its SeedLink config (an empty stream list refused), started - "
+       "CONNECTED on 4.0 with the three channels, the latency and the records on the live-station card and on the station list - stopped, "
+       "folded (the day files hashed into the record list, the vertical first; a second fold finds nothing), and refreshed by the leg on what "
+       "the station sent; every step in the audit log; the page carries the card; `gea seedlink` writes its example and, with no server, "
+       "names the simulator to start; the help page says a gap is counted, never bridged"
+       + (f" [failed: {', '.join(failed)}; row={json.dumps({k: row.get(k) for k in ('status', 'protocol', 'records', 'latency_p95_s', 'last_error')})}]" if failed else ""))
+
 def main() -> int:
     print("GEA-Program - ACCEPTANCE SUITE (the product gate)")
     # a handle a job subprocess still holds at the end must not turn a finished gate into a traceback on Windows:
@@ -5646,6 +5784,7 @@ def main() -> int:
         section_bh_vibration(tmp)
         section_bi_conformance(tmp)
         section_bj_etp(tmp)
+        section_bk_seedlink(tmp)
     if _FAILS:
         print(f"[ACCEPTANCE] {len(_FAILS)} FAILURES ({_PASS} passed):")
         for f in _FAILS:

gea_program-0.17.1/gea/help/seedlink.md

@@ -0,0 +1,11 @@
+# Live stations (SeedLink)
+
+> The seismic leg has run on records that were already files. The stations do not hand out files; they serve SeedLink - the real-time protocol every data centre and most digitisers speak - and this is the port that takes it: the customer names its stations and channels, says where it left off, and the server pushes each miniSEED record as it is written. SeedLink 3 (the one every SeisComP and ringserver serves) and SeedLink 4.0 (FDSN, 2022) are both spoken; the server's HELLO decides which.
+
+**The command**: `gea seedlink --write-example-config seedlink.json`, then put the server's `host` and `port` (a TexNet station is served by EarthScope at `rtserve.iris.washington.edu:18000`, network `TX`), the stations as `{"network", "station", "selectors"}` with selectors in SeedLink 3 form `LLCCC.T` (`HHZ.D`; `?` for one character; converted for a 4.0 server), and run `gea seedlink --config seedlink.json --seconds 3600 --out live`; the next run resumes from the last sequence number the state file holds. In a workspace: `gea workspace --path C:\site --action add-live-station --name "<display>" --seedlink-config seedlink.json --lat --lon`, then Start on the dashboard's Seismic page (the serving process keeps every enabled live station up and reconnects with backoff) or `--action live-run --station <id>`; `--action live-fold --station <id>` moves the finished day files into the station's record list, and Refresh runs the leg on them. `gea seedlink-sim --port 18000` is a three-component synthetic station to rehearse against; `gea seedlink --selftest` runs both protocols against it on the loopback; `gea seedlink --config ... --hello` prints what a server says of itself.
+
+**What it writes**: every record the server sends, byte for byte, appended to a day file per channel - `NET.STA.LOC.CHA.YYYY.DDD.mseed` - under the station's `live/` folder, and the state beside it: per station the last sequence number (hexadecimal and per station in SeedLink 3, decimal in 4.0; the resume asks `DATA` for the next), per channel the last sample time, the latency (arrival minus the record's last sample), the records, the samples, and the gaps and overlaps between consecutive records (a gap is counted and left, never bridged). A fold moves the day files that are no longer being written (every day but the current UTC day, unless asked) into `source/`, hashes each and appends it to the station's files, vertical channels first, with the audit line naming what was folded; what the leg then computes is computed from what the station sent. The handshake is kept on the state (`HELLO`, the protocol, each `STATION`/`SELECT`/`DATA`, `END`) and the server's own description with it.
+
+**The number to check**: on the live-station card, `CONNECTED` with the latency p95 and the records climbing; after a fold, the station's file count; `REFUSED` with the `STATION` line names a station the server does not have (no reconnection loop); `ENDED` is the server's own `END`.
+
+**What this page will not call a measurement**: a sample across a gap - it is counted and the file keeps the gap. A time the record's header does not carry - the record's own time is the time. A record in a payload format it does not know - counted with the format byte and left undecoded; miniSEED 2 is what is read, miniSEED 3 is named when a 4.0 server sends it. A session the server has ended - the reason is on the state and the port stops. A location or a response - the leg's refresh on the folded files, with a StationXML, is where counts become ground motion.

{gea_program-0.16.0 → gea_program-0.17.1}/gea/helplib.py

@@ -45,6 +45,7 @@ INDEX: List[tuple] = [
     ('vibration', 'Machine vibration', 'the pump\'s record through the ISO 20816-3 zones and bearing envelope analysis'),
     ('conformance', 'Conformity, in the standards\' words', 'certificates against ISO/IEC 17025 7.8 and the bias against its class under a named ILAC-G8 decision rule'),
     ('etp', 'WITSML 2.x over ETP', 'the live port on Energistics ETP v1.2: subscribe to the store\'s channels and it pushes'),
+    ('seedlink', 'Live stations (SeedLink)', 'the seismic leg\'s live port: a station\'s records as it writes them, resumed by sequence number, folded into its record list'),
     ('doctor', 'Which code is running', 'gea doctor and the serve start-up gate'),
     ('audit-update', 'Audit / Update', 'the whole audit log with filters, the program against PyPI, every report against its source'),
     ('upkeep', 'Upkeep', 'housekeeping, sessions, the TLS proxy, the load test'),