@tailglow/cli 0.7.1 → 0.8.1

package/README.md

@@ -1,8 +1,8 @@
 # @tailglow/cli
 
-`tglow` is the Tailglow command line. Every command is generated from the API's own routes and
-validations, so the CLI, the docs site, `llms.txt` and the in-product AI assistant all describe the
-same API by construction.
+`tglow` is the Tailglow command line. API resource commands are generated from the API's own
+routes and validations, so the CLI, docs site and in-product AI assistant describe the same API.
+Local commands also support sign-in, bundled documentation and file imports.
 
 ```bash
 npm install -g @tailglow/cli
@@ -10,7 +10,7 @@ tglow login
 tglow projects list
 ```
 
-Node 18.3 or newer. `bun add -g` and `pnpm add -g` work the same way.
+Node 22 or newer. `bun add -g` and `pnpm add -g` work the same way.
 
 Prefer a shorter name? Alias it rather than installing over one:
 
@@ -135,8 +135,89 @@ Pages print as markdown even when piped; `--json` returns `{ path, url, content
 bun run tglow <resource> <command>   # run from source, without installing
 bun run test:cli                     # unit tests, no network
 bun run build:llms                   # regenerate src/commands.generated.js and src/docs.generated.js
+bun run build:import                 # regenerate the bundled importer and its XML worker
 ```
 
 `src/commands.generated.js` and `src/docs.generated.js` are written by `scripts/build/llms.build.js`
 and are never edited by hand. Adding a route to the API adds its command here with no change to this
-package, and editing a guide changes the bundled page the same way.
+package, and editing a guide changes the bundled page the same way. `src/import.generated.js`, its
+declarations and `src/import-worker.generated.js` are self-contained bundles of the workspace
+importer written by `scripts/build/import.build.js`.
+
+## Import files
+
+Scan and validate locally without credentials:
+
+```bash
+tglow import scan export.xml
+tglow import scan export.xml --record-path /HealthData/Record --full
+tglow import validate events.jsonl --timestamp-field /event/created_at
+```
+
+Content detection supports XML, JSONL, JSON, CSV and TSV; use `--format` to resolve ambiguity.
+Scan defaults to a bounded sample, reports its coverage, and discovers literal XML element paths.
+XML automatically imports every direct child of the document root, including different record
+types. Root attributes and meaningful direct text become a separate document record; a root-only
+document becomes one record. `--record-path` is an optional filter for importing just part of a
+file. Rows preserve the `{name, attributes, children}` tree, strings, qualified names and child
+order. Line-break indentation in element-only containers is removed. Mixed text, inline spaces,
+whitespace-only values, CDATA, character references and inherited `xml:space="preserve"` stay intact;
+formatting separators between top-level records are omitted.
+There is no Apple-specific flattening. DTDs use the documented ignore profile; declarations are not
+applied and declared entities are not expanded. JSON documents have a 50 MiB limit; the other
+formats stream in bounded memory. Every serialized row must fit within 1 MiB including its
+newline.
+
+Upload using an exact project, source and ingest key, with normal CLI authentication:
+
+```bash
+tglow import events.jsonl --project prj_x --source-id src_x --ingest-key-id ik_x --collection events
+```
+
+Alternatively pass an explicit `--ingest-url` containing its key and `--project prj_x` to verify
+the expected project. Treat the URL as a credential: passing it on the command line exposes it
+to shell history and process listings. The CLI never prints the URL or API credential. The
+project can come from the existing environment/config default; `--project-id` is an alias for
+`--project`. Source and ingest-key IDs never default or select themselves. A localhost API cannot
+automatically direct an upload to a remote URL; use an explicit ingest URL for that destination.
+
+Upload reads a bounded preview, checks destination readiness, then validates records and sends
+batches as it reads. It holds one file descriptor and checks for file changes before every request
+and at EOF. These checks detect ordinary replacement, edits, append and truncation; they are not
+an immutable snapshot guarantee. Do not edit the input during import. No full temporary copy is
+needed by default. A late parse error can leave earlier batches accepted.
+
+Add `--validate-first` to copy the file into a private temporary directory and check the whole
+snapshot before any upload. This needs free temporary disk space approximately equal to the input
+size. The same snapshot is used for checking and delivery, then removed on completion, failure or
+graceful Ctrl-C. A cleanup failure is reported separately as `cleanup_error` and a warning; it does
+not change delivery status or acknowledged counts. Changes to the original after copying do not change the uploaded bytes. A failed
+full check sends no rows; successful checking cannot prevent a later delivery failure.
+
+`--timestamp-field` accepts a JSON pointer or an exact top-level field name. Source fields remain
+unchanged; the selected value sets request-level `ts`. Missing, invalid or timezone-ambiguous
+values fail validation. Selected dates must be between 1970-01-01 UTC and the current time, since earlier or future ingest timestamps cannot be preserved by the server. No timestamp field is selected automatically. Batches split whenever
+the exact parsed timestamp changes, which can produce one request per row, and otherwise stay
+within 500 rows and 1 MiB.
+
+Only HTTP 202 acknowledges a batch. Upload stops on rejection or uncertain delivery and never
+retries automatically. The report includes `accepted_count` and the affected batch's inclusive
+`first_row`/`last_row`; these indices refer to the selected output rows, not physical file lines.
+A timeout, cancellation during a request, or missing acknowledgement may mean that batch arrived.
+Do not blindly rerun a partial upload: import idempotency is not provided yet.
+
+Full scans, validation and upload read large XML files with worker threads. `--workers auto`, the
+default, starts two for files of 32 MiB or more on machines with at least four logical processors;
+`--workers 0` reads on the main thread, and `--workers 1` to `16` sets the count for any XML file.
+Each pass checks that its workers start and answer before reading, falls back to the main thread
+if they cannot, and reports what ran on stderr (`XML workers: 2.`). Reports, rows, batches and
+errors are identical with any worker count; JSONL, JSON, CSV, TSV and sampled scans read on the
+main thread.
+
+Progress is written to stderr. In `--json` mode or when piped, import always writes its final
+JSON report to stdout, including failures and partial uploads. `total_rows` is `null` until a full
+check or clean EOF establishes it. `accepted_count` counts only acknowledged rows. The `outcome`
+field distinguishes validation failures, changed files, destination failures, rejected batches,
+uncertain delivery and cancellation. Exit code is 0 on success, 1 on
+failure, and 130 after cancellation. Other CLI commands retain the standard failure-to-stderr
+behavior described above. See `tglow import --help` for all flags.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tailglow/cli",
-  "version": "0.7.1",
+  "version": "0.8.1",
   "type": "module",
   "description": "The Tailglow command line. Every command is generated from the Tailglow API reference, so the CLI and the documentation can never describe different APIs.",
   "bin": {
@@ -15,7 +15,7 @@
     "src"
   ],
   "engines": {
-    "node": ">=18.3.0"
+    "node": ">=22.0.0"
   },
   "homepage": "https://docs.tailglow.io",
   "bugs": {

package/src/commands.generated.js

@@ -3334,6 +3334,7 @@ export const CLI_COMMANDS = [
       "`transform_script` is TGL and must compile. A script that does not parse is rejected rather than stored, because an unparseable script would fail on every record it later touched.",
       "The script must write every field the view's output schema declares.",
       "Omitting the script queues the transform for generation instead of rejecting it.",
+      "`collection_schema_version` must be settled. A candidate stays out of views until it settles, so naming one is refused with a 409; settle it with Update Collection Schema first.",
       "A transform on a collection-backed view must name the `collection_schema_version` it covers. View-to-view transforms pass null, because they read another view's output rather than a collection schema.",
       "`transform_script` cannot be whitespace-only. Omit it to have TGL generated instead."
     ],
@@ -3830,11 +3831,20 @@ export const CLI_COMMANDS = [
       "`complete_through_at` is the instant before which every number is final. Null means completeness is unknown, and then every bucket is incomplete.",
       "For complete buckets only, leave out every key at or after `first_incomplete_bucket_at`.",
       "`coverage_reasons` names unresolved view scopes; `excluded` is informational.",
+      "Series are ranked by their total over the range (the count for a `count` metric, the sum of Last readings for `last`, the sum otherwise), ties broken by series hash, and `limit` of them come back per page, the metric's `ui_chart_max_series` by default. Walk the pages with `pagination.next_cursor` and `pagination.prev_cursor`; `pagination.total` counts every series.",
+      "`__other__` folds in every series that is not on the current page, so the page plus Other always adds up to the range total and a share is right on every page.",
+      "When the metric has pinned series, they make up the whole first page and later pages walk the other series by rank.",
+      "`series` returns exactly the listed series on one page, unranked and without `__other__`.",
+      "A metric with a `time_dimension` returns its range folded instead of as a timeline: every hour merges into its hour of the day, weekday or hour of the week. Record keys are then the category as a decimal string (`\"0\"` to `\"23\"`, `\"1\"` for Monday to `\"7\"` for Sunday, `\"0\"` to `\"167\"`), every category is present whatever `normalize` says, and `time_dimension` echoes it.",
+      "Data is stored by UTC hour. A fold places each hour in its category in the request `timezone` (echoed as `timezone`), one hour at a time, so a daylight saving change moves only the hours after it.",
+      "A fold reads hourly data whatever `interval` says, covers at most 400 days (a longer range is refused), and includes the current hour's data so far.",
+      "In a fold an empty category is `0` when `empty_bucket_handling` is `zero` and `null` when it is `gaps`, cumulative values are `null`, and there is no trend line.",
       "The aggregation window is resolved in order: an explicit `start_at` plus `end_at` wins; otherwise `time_range` is used; otherwise the metric's saved chart range is used.",
       "`start_at` and `end_at` only take effect when sent together. Sending one without the other has no effect on the window.",
       "When both are provided, `start_at` must be earlier than `end_at`.",
       "`time_range=custom` without explicit dates falls back to the saved custom range of `page_id`, then the metric, then the project.",
-      "`timezone` must be a valid IANA timezone name, or `UTC`."
+      "`timezone` must be a valid IANA timezone name, or `UTC`.",
+      "`after` and `before` are mutually exclusive."
     ],
     "path_params": [
       {
@@ -3875,7 +3885,7 @@ export const CLI_COMMANDS = [
         "name": "interval",
         "type": "ChartInterval",
         "required": false,
-        "description": "Requested aggregation bucket interval. Optional.",
+        "description": "Requested aggregation bucket interval. A metric with a time dimension ignores it and always folds hourly data. Optional.",
         "allowed_values": [
           "minute",
           "hour",
@@ -3920,7 +3930,7 @@ export const CLI_COMMANDS = [
         "name": "timezone",
         "type": "string",
         "required": false,
-        "description": "IANA timezone used to calculate date boundaries. Optional.",
+        "description": "IANA timezone used to calculate date boundaries and to place each hour in a time dimension's categories. Optional.",
         "allowed_values": [],
         "deprecated": null
       },
@@ -3949,20 +3959,47 @@ export const CLI_COMMANDS = [
         "deprecated": null
       },
       {
-        "name": "max_series",
+        "name": "series",
+        "type": "string",
+        "required": false,
+        "description": "Comma-separated series hashes to return. Optional.",
+        "allowed_values": [],
+        "deprecated": null
+      },
+      {
+        "name": "limit",
         "type": "integer",
         "required": false,
-        "description": "Maximum number of individual series to return. Optional. Minimum: `1`. Maximum: `50`.",
+        "description": "Number of series to return per page. Defaults to the metric's `ui_chart_max_series`. Optional. Minimum: `1`. Maximum: `200`.",
         "allowed_values": [],
         "deprecated": null
       },
       {
-        "name": "series",
+        "name": "after",
         "type": "string",
         "required": false,
-        "description": "Comma-separated series hashes to return. Optional.",
+        "description": "Cursor from `pagination.next_cursor` of a previous response. Returns the resources after that page. Optional.",
         "allowed_values": [],
         "deprecated": null
+      },
+      {
+        "name": "before",
+        "type": "string",
+        "required": false,
+        "description": "Cursor from `pagination.prev_cursor` of a previous response. Returns the resources before that page. Optional.",
+        "allowed_values": [],
+        "deprecated": null
+      },
+      {
+        "name": "sort",
+        "type": "string",
+        "required": false,
+        "description": "Ranking direction of the series. `desc` returns the largest first. Optional. Defaults to `\"desc\"`.",
+        "allowed_values": [
+          "asc",
+          "desc"
+        ],
+        "deprecated": null
       }
     ],
     "body_fields": []
@@ -3984,6 +4021,11 @@ export const CLI_COMMANDS = [
       "Field references and filters must be compatible with the selected view.",
       "Custom chart ranges require both dates, with an interval of at least 30 minutes.",
       "Color-mode configuration must match the selected chart color mode.",
+      "`ui_chart_color_mode` must be a mode the chart type supports. A metric created without a `ui_chart_type` is a `bar` chart and is checked as one.",
+      "`time_dimension` must be one the chart type supports: `breakdown` takes `hour_of_day` or `weekday`, `heatmap` takes `hour_of_week`, `hour_of_day` or `weekday`, and every other chart type takes none.",
+      "A `heatmap` chart requires a `time_dimension`.",
+      "A `time_dimension` cannot be combined with a cumulative `ui_chart_value` (`cumulative_sum` or `cumulative_count`).",
+      "`ui_chart_order` is read by `breakdown` charts only; other chart types ignore it.",
       "`filters` must be a valid filter expression.",
       "`timestamp_field` cannot reference `__meta__` metadata.",
       "`group_by_display_fields` is set at create and cannot be changed afterwards, because each series takes its label from the first record it sees. Map a `group_by` field to a view field that carries a readable value for the same record."
@@ -4101,7 +4143,10 @@ export const CLI_COMMANDS = [
           "stat",
           "gauge",
           "calendar",
-          "uptime"
+          "uptime",
+          "breakdown",
+          "heatmap",
+          "table"
         ],
         "deprecated": null
       },
@@ -4305,6 +4350,29 @@ export const CLI_COMMANDS = [
         ],
         "deprecated": null
       },
+      {
+        "name": "time_dimension",
+        "type": "TimeDimension",
+        "required": false,
+        "description": "Calendar position the chart folds the metric's hourly data into, such as the hour of the day. Null charts the data over time. Optional.",
+        "allowed_values": [
+          "hour_of_day",
+          "weekday",
+          "hour_of_week"
+        ],
+        "deprecated": null
+      },
+      {
+        "name": "ui_chart_order",
+        "type": "ChartOrder",
+        "required": false,
+        "description": "Order a breakdown chart shows its groups in. Null keeps a time dimension in its natural order and ranks field groups by value. Optional.",
+        "allowed_values": [
+          "natural",
+          "value"
+        ],
+        "deprecated": null
+      },
       {
         "name": "ui_chart_custom_range_start_at",
         "type": "ISODateString",
@@ -4325,7 +4393,7 @@ export const CLI_COMMANDS = [
         "name": "ui_chart_max_series",
         "type": "integer",
         "required": false,
-        "description": "Maximum number of individual series displayed by default. Optional. Minimum: `1`. Maximum: `50`.",
+        "description": "Number of series an aggregation returns per page when the request sends no `limit`. Optional. Minimum: `1`. Maximum: `50`.",
         "allowed_values": [],
         "deprecated": null
       },
@@ -4359,6 +4427,11 @@ export const CLI_COMMANDS = [
       "Changing the metric's data feed rebuilds its historical rollups.",
       "Custom chart ranges require both dates, with an interval of at least 30 minutes.",
       "Color-mode configuration must match the selected chart color mode.",
+      "`ui_chart_color_mode` must be a mode the chart type supports. Changing `ui_chart_type` keeps the current mode, so send a supported one with it.",
+      "`time_dimension` must be one the chart type supports: `breakdown` takes `hour_of_day` or `weekday`, `heatmap` takes `hour_of_week`, `hour_of_day` or `weekday`, and every other chart type takes none. Send `time_dimension: null` when changing to a type that takes none.",
+      "A `heatmap` chart requires a `time_dimension`.",
+      "A `time_dimension` cannot be combined with a cumulative `ui_chart_value` (`cumulative_sum` or `cumulative_count`).",
+      "`ui_chart_order` is read by `breakdown` charts only; other chart types ignore it.",
       "`filters` must be a valid filter expression.",
       "`timestamp_field` cannot reference `__meta__` metadata."
     ],
@@ -4475,7 +4548,10 @@ export const CLI_COMMANDS = [
           "stat",
           "gauge",
           "calendar",
-          "uptime"
+          "uptime",
+          "breakdown",
+          "heatmap",
+          "table"
         ],
         "deprecated": null
       },
@@ -4679,6 +4755,29 @@ export const CLI_COMMANDS = [
         ],
         "deprecated": null
       },
+      {
+        "name": "time_dimension",
+        "type": "TimeDimension",
+        "required": false,
+        "description": "Calendar position the chart folds the metric's hourly data into, such as the hour of the day. Null charts the data over time. Optional.",
+        "allowed_values": [
+          "hour_of_day",
+          "weekday",
+          "hour_of_week"
+        ],
+        "deprecated": null
+      },
+      {
+        "name": "ui_chart_order",
+        "type": "ChartOrder",
+        "required": false,
+        "description": "Order a breakdown chart shows its groups in. Null keeps a time dimension in its natural order and ranks field groups by value. Optional.",
+        "allowed_values": [
+          "natural",
+          "value"
+        ],
+        "deprecated": null
+      },
       {
         "name": "ui_chart_custom_range_start_at",
         "type": "ISODateString",
@@ -4699,7 +4798,7 @@ export const CLI_COMMANDS = [
         "name": "ui_chart_max_series",
         "type": "integer",
         "required": false,
-        "description": "Maximum number of individual series displayed by default. Optional. Minimum: `1`. Maximum: `50`.",
+        "description": "Number of series an aggregation returns per page when the request sends no `limit`. Optional. Minimum: `1`. Maximum: `50`.",
         "allowed_values": [],
         "deprecated": null
       },
@@ -6953,7 +7052,10 @@ export const CLI_COMMANDS = [
     ],
     "notes": [
       "A component is a card on one page showing one to six metrics together.",
-      "To change a whole page's layout in one write, send `components` to Update Page instead."
+      "To change a whole page's layout in one write, send `components` to Update Page instead.",
+      "Every metric must belong to the same project as the page, and none can already be on another card of the page.",
+      "A metric can be listed only once in `metric_ids`.",
+      "A page holds a limited number of components. Adding one past the limit is rejected."
     ],
     "path_params": [
       {
@@ -6974,7 +7076,51 @@ export const CLI_COMMANDS = [
       }
     ],
     "query_params": [],
-    "body_fields": []
+    "body_fields": [
+      {
+        "name": "name",
+        "type": "string | null",
+        "required": false,
+        "description": "Title of a card that shows several metrics. Send null to title it with their names. Optional.",
+        "allowed_values": [],
+        "deprecated": null
+      },
+      {
+        "name": "ui_sort_index",
+        "type": "integer",
+        "required": false,
+        "description": "Position of the component on the page. Lower appears first. Optional.",
+        "allowed_values": [],
+        "deprecated": null
+      },
+      {
+        "name": "ui_card_size",
+        "type": "ComponentCardSize",
+        "required": false,
+        "description": "How much of the page width the component occupies. Optional.",
+        "allowed_values": [
+          "1/1",
+          "1/2"
+        ],
+        "deprecated": null
+      },
+      {
+        "name": "metric_ids",
+        "type": "string[]",
+        "required": true,
+        "description": "Metrics the card shows together, in order.",
+        "allowed_values": [],
+        "deprecated": null
+      },
+      {
+        "name": "page_id",
+        "type": "string",
+        "required": false,
+        "description": "Page the component is added to. Optional.",
+        "allowed_values": [],
+        "deprecated": null
+      }
+    ]
   },
   {
     "resource": "components",