signalk-cli 2.0.1__tar.gz → 2.1.0__tar.gz

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: signalk-cli
-Version: 2.0.1
+Version: 2.1.0
 Summary: Query SignalK v2 APIs and export data as CSV, JSON or Apache Arrow Feather
 Keywords: signalk,sailing,marine,nmea,boating
 Author: jey burrows
@@ -71,6 +71,10 @@ Use `uv` to run without installing the module permanently, for example:
 uv run --with signalk-cli signalk_cli.history list-providers
 ```
 
+```bash
+uv run --with signalk-cli signalk_cli.stream deltas navigation.position --follow
+```
+
 ## Running
 
 Run via `python -m signalk_cli.history <command>` or `python -m signalk_cli.stream <command>` or without installing the module with `uv run --with signalk-cli signalk_cli.history <command>`.
@@ -436,8 +440,10 @@ against the server's enumerated `/paths` list:
 Quote wildcarded paths (e.g. `'navigation.*'`) so the shell doesn't expand them
 against local filenames first.
 
-`deltas` always sends its own explicit subscribe message for `--context`,
-covering PATH arguments if given, otherwise every path (`*`). `--policy`,
+`deltas` always sends its own explicit subscribe message for `--context`
+(default `vessels.self`), covering PATH arguments if given, otherwise every
+path (`*`). `--context` also accepts the SignalK wildcard: `'*'` or
+`'vessels.*'` subscribes to every vessel, not just your own. `--policy`,
 `--period`, and `--min-period` control that subscription per the
 [Subscription Protocol](https://signalk.org/specification/1.8.2/doc/subscription_protocol.html)'s
 `policy`/`period`/`minPeriod` fields:
@@ -451,11 +457,23 @@ this CLI doesn't expose it: signalk-server rejects `full` outright ("Only
 delta format supported, using it") and always sends delta messages
 regardless, so the choice would be misleading.
 
-`--subscribe` is separate: it's the connection-level `subscribe` query
-parameter (`none`/`self`/`all`, default `none`), controlling only whether the
-server *additionally* auto-subscribes the connection at its own default
-policy/period — useful with `--subscribe all` to also receive other vessels'
-default-policy updates alongside your explicit subscription.
+**`--subscribe` is a separate, easily-confused mechanism**: it's the
+connection-level `subscribe` query parameter (`none`/`self`/`all`), which
+controls whether the server *additionally* auto-subscribes the connection
+at **its own** default policy/period — you don't get to choose the rate.
+This CLI defaults it to `none`, diverging from the SignalK spec's own
+default of `self`, because `deltas` always sends its own explicit `--context`
+subscription anyway; leaving the connection-level default at `self` would
+just double up on updates for your own vessel.
+
+So there are two different ways to see every vessel, and they're not
+interchangeable:
+
+| Want | Use | Policy/period used |
+|---|---|---|
+| Just your own vessel | (defaults) | Yours (`--policy`/`--period`) |
+| Every vessel, at your chosen rate | `--context '*'` | Yours (`--policy`/`--period`) |
+| Every vessel, at whatever rate the server defaults to | `--subscribe all` | The server's own default |
 
 #### Options
 
@@ -470,6 +488,7 @@ default-policy updates alongside your explicit subscription.
 | `--min-period SECONDS` | — | Per-path subscribe `minPeriod` field, in seconds (converted to ms); only meaningful with `--policy instant` |
 | `--format [csv\|json\|raw\|feather]` | from extension, else csv | Output format. `json` is JSON Lines (one row object per line, suitable for a live stream). `raw` is the exact delta message text, one per line. `feather` requires `pip install 'signalk-cli[feather]'` and `--output` (cannot stream to stdout). |
 | `--no-header` | — | Suppress the CSV header row |
+| `--include-meta` | — | Also emit rows for `meta` entries (units, description, zones, etc.), not just `values`. Adds a `kind` column (`value`/`meta`) to csv/json/feather output. Ignored for `--format raw`, which always includes meta as-is. |
 | `-o, --output [FILE]` | stdout | Write to a file. Omit the filename (`--output` alone) to auto-name as `signalk-stream-<server>-<timestamp>.<ext>`. Required for `--format feather`. |
 | `-f, --follow` | — | Keep streaming until interrupted (Ctrl-C) or `--count` is reached. Without this, print the next message then exit. |
 | `-n, --count N` | 1 without `--follow`, unlimited with it | Number of delta messages to output |
@@ -477,7 +496,7 @@ default-policy updates alongside your explicit subscription.
 
 #### Output formats
 
-**csv** / **json**: `timestamp, context, source, path, value` — one row per `path`/`value` pair in each delta's updates, written incrementally as messages arrive. Structured values (e.g. `navigation.position`) are JSON-encoded in the `value` column.
+**csv** / **json**: `timestamp, context, source, path, value` — one row per `path`/`value` pair in each delta's updates, written incrementally as messages arrive. Structured values (e.g. `navigation.position`) are JSON-encoded in the `value` column. With `--include-meta`, a `kind` column (`value`/`meta`) is inserted before `value`, and `meta` entries (units, description, zones, etc. — also JSON-encoded) are included as rows too; without it, `meta` entries are silently dropped from csv/json/feather (they're still present in `raw`).
 
 **raw**: the exact delta message JSON as received from the server, one message per line.
 

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/README.md

@@ -43,6 +43,10 @@ Use `uv` to run without installing the module permanently, for example:
 uv run --with signalk-cli signalk_cli.history list-providers
 ```
 
+```bash
+uv run --with signalk-cli signalk_cli.stream deltas navigation.position --follow
+```
+
 ## Running
 
 Run via `python -m signalk_cli.history <command>` or `python -m signalk_cli.stream <command>` or without installing the module with `uv run --with signalk-cli signalk_cli.history <command>`.
@@ -408,8 +412,10 @@ against the server's enumerated `/paths` list:
 Quote wildcarded paths (e.g. `'navigation.*'`) so the shell doesn't expand them
 against local filenames first.
 
-`deltas` always sends its own explicit subscribe message for `--context`,
-covering PATH arguments if given, otherwise every path (`*`). `--policy`,
+`deltas` always sends its own explicit subscribe message for `--context`
+(default `vessels.self`), covering PATH arguments if given, otherwise every
+path (`*`). `--context` also accepts the SignalK wildcard: `'*'` or
+`'vessels.*'` subscribes to every vessel, not just your own. `--policy`,
 `--period`, and `--min-period` control that subscription per the
 [Subscription Protocol](https://signalk.org/specification/1.8.2/doc/subscription_protocol.html)'s
 `policy`/`period`/`minPeriod` fields:
@@ -423,11 +429,23 @@ this CLI doesn't expose it: signalk-server rejects `full` outright ("Only
 delta format supported, using it") and always sends delta messages
 regardless, so the choice would be misleading.
 
-`--subscribe` is separate: it's the connection-level `subscribe` query
-parameter (`none`/`self`/`all`, default `none`), controlling only whether the
-server *additionally* auto-subscribes the connection at its own default
-policy/period — useful with `--subscribe all` to also receive other vessels'
-default-policy updates alongside your explicit subscription.
+**`--subscribe` is a separate, easily-confused mechanism**: it's the
+connection-level `subscribe` query parameter (`none`/`self`/`all`), which
+controls whether the server *additionally* auto-subscribes the connection
+at **its own** default policy/period — you don't get to choose the rate.
+This CLI defaults it to `none`, diverging from the SignalK spec's own
+default of `self`, because `deltas` always sends its own explicit `--context`
+subscription anyway; leaving the connection-level default at `self` would
+just double up on updates for your own vessel.
+
+So there are two different ways to see every vessel, and they're not
+interchangeable:
+
+| Want | Use | Policy/period used |
+|---|---|---|
+| Just your own vessel | (defaults) | Yours (`--policy`/`--period`) |
+| Every vessel, at your chosen rate | `--context '*'` | Yours (`--policy`/`--period`) |
+| Every vessel, at whatever rate the server defaults to | `--subscribe all` | The server's own default |
 
 #### Options
 
@@ -442,6 +460,7 @@ default-policy updates alongside your explicit subscription.
 | `--min-period SECONDS` | — | Per-path subscribe `minPeriod` field, in seconds (converted to ms); only meaningful with `--policy instant` |
 | `--format [csv\|json\|raw\|feather]` | from extension, else csv | Output format. `json` is JSON Lines (one row object per line, suitable for a live stream). `raw` is the exact delta message text, one per line. `feather` requires `pip install 'signalk-cli[feather]'` and `--output` (cannot stream to stdout). |
 | `--no-header` | — | Suppress the CSV header row |
+| `--include-meta` | — | Also emit rows for `meta` entries (units, description, zones, etc.), not just `values`. Adds a `kind` column (`value`/`meta`) to csv/json/feather output. Ignored for `--format raw`, which always includes meta as-is. |
 | `-o, --output [FILE]` | stdout | Write to a file. Omit the filename (`--output` alone) to auto-name as `signalk-stream-<server>-<timestamp>.<ext>`. Required for `--format feather`. |
 | `-f, --follow` | — | Keep streaming until interrupted (Ctrl-C) or `--count` is reached. Without this, print the next message then exit. |
 | `-n, --count N` | 1 without `--follow`, unlimited with it | Number of delta messages to output |
@@ -449,7 +468,7 @@ default-policy updates alongside your explicit subscription.
 
 #### Output formats
 
-**csv** / **json**: `timestamp, context, source, path, value` — one row per `path`/`value` pair in each delta's updates, written incrementally as messages arrive. Structured values (e.g. `navigation.position`) are JSON-encoded in the `value` column.
+**csv** / **json**: `timestamp, context, source, path, value` — one row per `path`/`value` pair in each delta's updates, written incrementally as messages arrive. Structured values (e.g. `navigation.position`) are JSON-encoded in the `value` column. With `--include-meta`, a `kind` column (`value`/`meta`) is inserted before `value`, and `meta` entries (units, description, zones, etc. — also JSON-encoded) are included as rows too; without it, `meta` entries are silently dropped from csv/json/feather (they're still present in `raw`).
 
 **raw**: the exact delta message JSON as received from the server, one message per line.
 

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "signalk-cli"
-version = "2.0.1"
+version = "2.1.0"
 description = "Query SignalK v2 APIs and export data as CSV, JSON or Apache Arrow Feather"
 readme = "README.md"
 requires-python = ">=3.13"
@@ -102,13 +102,16 @@ mypy_path = ["./src"]
 [tool.uv]
 compile-bytecode = true
 managed = true
-exclude-newer = "5 days"
+exclude-newer = "7 days"
 add-bounds = "major"
 
+[tool.uv.exclude-newer-package]
+uv_build = "2 days"
+
 [tool.uv.build-backend]
 module-root = "src"
 module-name = "signalk_cli"
 
 [build-system]
-requires = ["uv_build>=0.11.13,<0.12.0"]
+requires = ["uv_build>=0.12.0,<0.13.0"]
 build-backend = "uv_build"

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "signalk-cli"
-version = "2.0.1"
+version = "2.1.0"
 description = "Query SignalK v2 APIs and export data as CSV, JSON or Apache Arrow Feather"
 readme = "README.md"
 requires-python = ">=3.13"
@@ -101,11 +101,12 @@ mypy_path=["./src"]
 [tool.uv]
 compile-bytecode = true
 managed = true
-exclude-newer = "5 days"
+exclude-newer = "7 days"
+exclude-newer-package = {"uv_build"="2 days"}
 add-bounds = "major"
 
 [build-system]
-requires = ["uv_build>=0.11.13,<0.12.0"]
+requires = ["uv_build>=0.12.0,<0.13.0"]
 build-backend = "uv_build"
 
 

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/src/signalk_cli/stream/cli.py

@@ -49,18 +49,26 @@ def cli():
 @host_option
 @click.option("--no-cache", is_flag=True, help="Ignore cached host")
 @click.option(
-    "--context", "-c", default="vessels.self", show_default=True, help="SignalK context"
+    "--context",
+    "-c",
+    default="vessels.self",
+    show_default=True,
+    help="SignalK context for the explicit subscription this command "
+    "always sends. Also accepts the SignalK wildcard '*' (or 'vessels.*') "
+    "to subscribe to every vessel at your own --policy/--period — the "
+    "alternative to --subscribe all, which uses the server's default rate.",
 )
 @click.option(
     "--subscribe",
     type=click.Choice(SUBSCRIBE_POLICIES, case_sensitive=False),
     default="none",
     show_default=True,
-    help="Connection-level subscribe policy (none/self/all): whether the "
-    "server also auto-subscribes this connection using its own default "
-    "policy/period, in addition to the explicit subscription this command "
-    "always sends for --context. Use 'all' to also receive other vessels' "
-    "default-policy updates.",
+    help="Connection-level auto-subscribe at the server's OWN default "
+    "policy/period — separate from, and in addition to, this command's "
+    "explicit --context subscription above. Defaults to 'none' (the "
+    "SignalK spec's own default is 'self') to avoid double-subscribing "
+    "your own context. 'all' adds other vessels at the server's rate; for "
+    "other vessels at your chosen rate, use --context '*' instead.",
 )
 @click.option(
     "--policy",
@@ -102,6 +110,13 @@ def cli():
     "pip install 'signalk-cli[feather]' and --output (cannot stream to stdout).",
 )
 @click.option("--no-header", is_flag=True, help="Suppress header row (CSV only)")
+@click.option(
+    "--include-meta",
+    is_flag=True,
+    help="Also emit rows for 'meta' entries (units, description, zones, etc.), "
+    "not just 'values'. Adds a 'kind' column (value/meta) to csv/json/feather "
+    "output. Ignored for --format raw, which always includes meta as-is.",
+)
 @click.option(
     "--output",
     "-o",
@@ -140,6 +155,7 @@ def deltas(
     min_period,
     fmt,
     no_header,
+    include_meta,
     output,
     follow,
     count,
@@ -233,7 +249,7 @@ def deltas(
         click.echo(f"Format:      {fmt}", err=True)
 
         try:
-            ws = open_stream(host, subscribe)
+            ws = open_stream(host, subscribe, timeout=None if follow else 30)
         except niquests.RequestException as e:
             click.echo(f"Error connecting to stream: {api_error(e)}", err=True)
             sys.exit(1)
@@ -252,7 +268,7 @@ def deltas(
         message_count = 0
         row_total = 0
         header_written = False
-        feather_rows: list[tuple[str, str, str, str, str]] = []
+        feather_rows: list[tuple[str, ...]] = []
         fh = (
             open(output, "w", newline="")  # noqa: SIM115
             if write_to_file and fmt != "feather"
@@ -263,26 +279,32 @@ def deltas(
             for raw, delta in iter_deltas(ws, effective_count):
                 message_count += 1
                 if fmt == "feather":
-                    feather_rows.extend(extract_delta_rows(delta))
+                    feather_rows.extend(
+                        extract_delta_rows(delta, include_meta=include_meta)
+                    )
                     row_total = len(feather_rows)
                 elif fmt == "raw":
                     click.echo(raw, file=sink)
                 elif fmt == "json":
-                    row_total += write_json_delta(delta, sink)
+                    row_total += write_json_delta(
+                        delta, sink, include_meta=include_meta
+                    )
                 else:
                     if not header_written and not no_header:
-                        write_csv_header(sink)
+                        write_csv_header(sink, include_meta=include_meta)
                         header_written = True
-                    row_total += write_csv_delta(delta, sink)
+                    row_total += write_csv_delta(delta, sink, include_meta=include_meta)
         except KeyboardInterrupt:
             pass
+        except niquests.RequestException as e:
+            click.echo(f"Stream connection lost: {api_error(e)}", err=True)
         finally:
             ws.close()
             if fh:
                 fh.close()
 
         if fmt == "feather":
-            write_feather_rows(feather_rows, output)
+            write_feather_rows(feather_rows, output, include_meta=include_meta)
 
         if write_to_file:
             click.echo(f"Wrote {output}", err=True)

signalk_cli-2.1.0/src/signalk_cli/stream/output.py

@@ -0,0 +1,108 @@
+"""Row extraction and CSV/JSON/Feather writers for SignalK delta messages."""
+
+import csv
+import json
+from typing import IO
+
+FEATHER_EXTENSIONS = {".feather", ".arrow", ".fea"}
+
+
+def _normalize_value(value: object) -> str:
+    if isinstance(value, (dict, list)):
+        return json.dumps(value)
+    if value is None:
+        return ""
+    return str(value)
+
+
+def extract_delta_rows(
+    delta: dict, *, include_meta: bool = False
+) -> list[tuple[str, ...]]:
+    """Flatten a single delta message into rows.
+
+    Without `include_meta`, rows are (timestamp, context, source, path,
+    value) from each update's "values" entries. With `include_meta`, a
+    "kind" column ("value"/"meta") is inserted before "value", and each
+    update's "meta" entries are included too — per the Streaming API spec,
+    "meta" entries have the same path/value shape but "value" is a metadata
+    object (units, description, zones, etc.), not a telemetry reading.
+    """
+    context = delta.get("context", "")
+    rows: list[tuple[str, ...]] = []
+    for update in delta.get("updates", []):
+        timestamp = update.get("timestamp", "")
+        source = update.get("$source") or json.dumps(update.get("source", {}))
+        for entry in update.get("values", []):
+            path = entry.get("path", "")
+            value = _normalize_value(entry.get("value"))
+            if include_meta:
+                rows.append((timestamp, context, source, path, "value", value))
+            else:
+                rows.append((timestamp, context, source, path, value))
+        if include_meta:
+            for entry in update.get("meta", []):
+                path = entry.get("path", "")
+                value = _normalize_value(entry.get("value"))
+                rows.append((timestamp, context, source, path, "meta", value))
+    return rows
+
+
+CSV_COLUMNS = ["timestamp", "context", "source", "path", "value"]
+CSV_COLUMNS_WITH_KIND = ["timestamp", "context", "source", "path", "kind", "value"]
+
+
+def _columns(include_meta: bool) -> list[str]:
+    return CSV_COLUMNS_WITH_KIND if include_meta else CSV_COLUMNS
+
+
+def write_csv_header(sink: IO[str], *, include_meta: bool = False) -> None:
+    csv.writer(sink).writerow(_columns(include_meta))
+    sink.flush()
+
+
+def write_csv_delta(delta: dict, sink: IO[str], *, include_meta: bool = False) -> int:
+    """Write one delta's rows as CSV lines. Returns the number of rows written."""
+    rows = extract_delta_rows(delta, include_meta=include_meta)
+    writer = csv.writer(sink)
+    for row in rows:
+        writer.writerow(row)
+    sink.flush()
+    return len(rows)
+
+
+def write_json_delta(delta: dict, sink: IO[str], *, include_meta: bool = False) -> int:
+    """Write one delta's rows as JSON Lines (one row object per line). Returns row count."""
+    rows = extract_delta_rows(delta, include_meta=include_meta)
+    columns = _columns(include_meta)
+    for row in rows:
+        sink.write(json.dumps(dict(zip(columns, row))))
+        sink.write("\n")
+    sink.flush()
+    return len(rows)
+
+
+def write_feather_rows(
+    rows: list[tuple[str, ...]], output: str, *, include_meta: bool = False
+) -> int:
+    """Write accumulated delta rows as Feather. Returns the number of rows written.
+
+    Unlike CSV/JSON, Feather cannot be appended to incrementally — callers must
+    buffer rows across messages and call this once at the end of the session.
+    """
+    try:
+        import pyarrow as pa
+        from pyarrow import feather
+    except ImportError:
+        raise ImportError(
+            "pyarrow is required for Feather output: pip install 'signalk-cli[feather]'"
+        ) from None
+    columns = _columns(include_meta)
+    row_columns = list(zip(*rows)) if rows else [()] * len(columns)
+    table = pa.table(
+        {
+            name: pa.array(values, type=pa.string())
+            for name, values in zip(columns, row_columns)
+        }
+    )
+    feather.write_feather(table, output)
+    return len(rows)

{signalk_cli-2.0.1 → signalk_cli-2.1.0}/src/signalk_cli/stream/stream_api.py

@@ -24,9 +24,15 @@ def to_ws_url(host: str) -> str:
     return urlunparse((scheme, parsed.netloc, STREAM_PATH, "", "", ""))
 
 
-def open_stream(host: str, subscribe: str, timeout: float = 30):
+def open_stream(host: str, subscribe: str, timeout: float | None = 30):
     """Open a WebSocket connection to the SignalK streaming endpoint.
 
+    `timeout` sets the socket's read timeout for the life of the connection —
+    every subsequent `next_payload()` read reuses it, not just the initial
+    handshake. Pass `None` when the caller intends to block indefinitely
+    between messages (e.g. `--follow`), since deltas can legitimately go
+    quiet for longer than any fixed timeout depending on subscribe policy.
+
     Returns the underlying HTTP extension object, used to send/receive frames
     via `send_payload`/`next_payload`.
     """

signalk_cli-2.0.1/src/signalk_cli/stream/output.py

@@ -1,89 +0,0 @@
-"""Row extraction and CSV/JSON/Feather writers for SignalK delta messages."""
-
-import csv
-import json
-from typing import IO
-
-FEATHER_EXTENSIONS = {".feather", ".arrow", ".fea"}
-
-
-def extract_delta_rows(delta: dict) -> list[tuple[str, str, str, str, str]]:
-    """Flatten a single delta message into (timestamp, context, source, path, value) rows."""
-    context = delta.get("context", "")
-    rows = []
-    for update in delta.get("updates", []):
-        timestamp = update.get("timestamp", "")
-        source = update.get("$source") or json.dumps(update.get("source", {}))
-        for entry in update.get("values", []):
-            path = entry.get("path", "")
-            value = entry.get("value")
-            if isinstance(value, (dict, list)):
-                value = json.dumps(value)
-            elif value is None:
-                value = ""
-            else:
-                value = str(value)
-            rows.append((timestamp, context, source, path, value))
-    return rows
-
-
-CSV_COLUMNS = ["timestamp", "context", "source", "path", "value"]
-
-
-def write_csv_header(sink: IO[str]) -> None:
-    csv.writer(sink).writerow(CSV_COLUMNS)
-    sink.flush()
-
-
-def write_csv_delta(delta: dict, sink: IO[str]) -> int:
-    """Write one delta's rows as CSV lines. Returns the number of rows written."""
-    rows = extract_delta_rows(delta)
-    writer = csv.writer(sink)
-    for row in rows:
-        writer.writerow(row)
-    sink.flush()
-    return len(rows)
-
-
-def write_json_delta(delta: dict, sink: IO[str]) -> int:
-    """Write one delta's rows as JSON Lines (one row object per line). Returns row count."""
-    rows = extract_delta_rows(delta)
-    for ts, context, source, path, value in rows:
-        sink.write(
-            json.dumps(
-                {
-                    "timestamp": ts,
-                    "context": context,
-                    "source": source,
-                    "path": path,
-                    "value": value,
-                }
-            )
-        )
-        sink.write("\n")
-    sink.flush()
-    return len(rows)
-
-
-def write_feather_rows(rows: list[tuple[str, str, str, str, str]], output: str) -> int:
-    """Write accumulated delta rows as Feather. Returns the number of rows written.
-
-    Unlike CSV/JSON, Feather cannot be appended to incrementally — callers must
-    buffer rows across messages and call this once at the end of the session.
-    """
-    try:
-        import pyarrow as pa
-        from pyarrow import feather
-    except ImportError:
-        raise ImportError(
-            "pyarrow is required for Feather output: pip install 'signalk-cli[feather]'"
-        ) from None
-    columns = list(zip(*rows)) if rows else [()] * len(CSV_COLUMNS)
-    table = pa.table(
-        {
-            name: pa.array(values, type=pa.string())
-            for name, values in zip(CSV_COLUMNS, columns)
-        }
-    )
-    feather.write_feather(table, output)
-    return len(rows)