syvain-metrics 0.0.360 → 0.0.367

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +63 -25
  2. package/package.json +7 -7
package/README.md CHANGED
@@ -1,7 +1,9 @@
1
1
  # syvain-metrics
2
2
 
3
- Command line tools for inspecting [Syvain Metrics](https://metrics.syvain.com/)
4
- experiments, folders, metric rows, metric metadata, and annotations.
3
+ Command line tool for [Syvain Metrics](https://metrics.syvain.com/). It is built
4
+ for agents and scripts: every command prints newline-delimited JSON on stdout,
5
+ errors go to stderr with exit code 1, and experiments and folders can be
6
+ addressed by slug or path instead of ids.
5
7
 
6
8
  ## Install
7
9
 
@@ -13,40 +15,76 @@ The package installs the `syvain-metrics` binary.
13
15
 
14
16
  ## Authenticate
15
17
 
16
- Create an API key from the Syvain Metrics dashboard, then log in:
18
+ Create an organization API key in the Syvain Metrics dashboard, then either save
19
+ it with a login or export it for the current shell:
17
20
 
18
21
  ```sh
19
- syvain-metrics auth login
20
- syvain-metrics auth status --human
22
+ syvain-metrics auth login # prompts for the key
23
+ printf '%s' "$KEY" | syvain-metrics auth login --stdin
24
+ syvain-metrics auth status
25
+
26
+ export SYVAIN_METRICS_API_KEY=ak_... # skips the saved login
27
+ export SYVAIN_METRICS_HOST=https://metrics.syvain.com # optional override
21
28
  ```
22
29
 
23
- For scripts or CI, pass the API key on stdin:
30
+ The CLI talks to `https://metrics.syvain.com` unless `--host` or
31
+ `SYVAIN_METRICS_HOST` says otherwise.
32
+
33
+ ## Addressing things
34
+
35
+ - An **experiment** is its id or its slug: `experiment get mamba-run-001`.
36
+ - A **folder** is its absolute path or its id: `/models/mamba`. The root is `/`.
37
+ Paths are derived from the organization tree, so names with spaces work when
38
+ quoted.
39
+
40
+ ## Folders
24
41
 
25
42
  ```sh
26
- printf '%s' "$SYVAIN_METRICS_API_KEY" | syvain-metrics auth login --stdin
43
+ syvain-metrics folder list # every folder with its path
44
+ syvain-metrics folder list /models # direct children of one folder
45
+ syvain-metrics folder create /models/mamba # creates missing parents; idempotent
46
+ syvain-metrics folder move /scratch/run-1 /models
47
+ syvain-metrics folder move /scratch/run-1 / # to the root
48
+ syvain-metrics folder rename /models/mamba mamba-v2
27
49
  ```
28
50
 
29
- By default the CLI talks to `https://metrics.syvain.com`.
51
+ `folder create` prints the leaf folder with `created: true` when it was made and
52
+ `created: false` when the path already existed.
53
+
54
+ ## Experiments
55
+
56
+ ```sh
57
+ syvain-metrics experiment list --search mamba # id, slug, folderId, folderPath
58
+ syvain-metrics experiment list --folder /models # direct members only
59
+ syvain-metrics experiment get mamba-run-001 # status, timestamps, meta, folderPath, url
60
+ syvain-metrics experiment move mamba-run-001 /models/mamba
61
+ syvain-metrics experiment catalog mamba-run-001 # series names with metadata keys and values
62
+ syvain-metrics experiment annotations mamba-run-001 --limit 20
63
+ ```
64
+
65
+ ## Series
66
+
67
+ `series query` prints one line per experiment, series, and metadata partition:
68
+ `{experimentId, seriesName, metadata, data: {step, timestampMs, value}}`.
69
+ Without `--name` it reads every series in the experiment's catalog.
70
+
71
+ ```sh
72
+ syvain-metrics series query mamba-run-001 --name loss --filter split=valid
73
+ syvain-metrics series query mamba-run-001 > run-001.jsonl
74
+ syvain-metrics series query run-a run-b --name loss --x-axis timestamp
75
+ ```
30
76
 
31
- ## Usage
77
+ For analysis across many experiments prefer the
78
+ [DuckDB extension](../../libs/metrics_duckdb/README.md); the CLI is the quick
79
+ way to pipe one experiment into a file.
32
80
 
33
- The default output is newline-delimited JSON so commands can be piped into
34
- automation. Add `--human` for readable tables.
81
+ `series render` charts the same selections as a PNG through the public Metrics
82
+ Renderer. Every experiment, series, and metadata partition becomes one line, so
83
+ use `--filter` to narrow what is drawn.
35
84
 
36
85
  ```sh
37
- syvain-metrics folders list --human
38
- syvain-metrics folders create --name dendro-mamba
39
- syvain-metrics experiments list --limit 20 --human
40
- syvain-metrics experiments get --slug dendro-mamba-run-001
41
- syvain-metrics experiments metrics --id exp_123 --metric-name loss
42
- syvain-metrics experiments query metric-series \
43
- --series '{"experimentId":"exp_123","metricName":"loss","metadataFilter":[{"key":"split","value":"\"train\""}]}'
44
- syvain-metrics experiments render metric-series \
45
- --series '{"experimentId":"exp_123","metricName":"loss","metadataFilter":[{"key":"model","value":"\"small\""}],"splitKeys":["data_split","curriculum_split"]}' \
46
- --output metrics.png
47
- syvain-metrics experiments query metric-catalog --experiment-ids exp_123,exp_456
48
- syvain-metrics experiments annotate --id exp_123 --annotation "Good validation run"
86
+ syvain-metrics series render run-a run-b --name loss --filter split=valid -o loss.png
87
+ syvain-metrics series render run-a --name loss --display-mode subplots -o loss.png
49
88
  ```
50
89
 
51
- Run `syvain-metrics --help` or `syvain-metrics <command> --help` for the full
52
- command list and options.
90
+ Run `syvain-metrics --help` or `syvain-metrics <command> --help` for every flag.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "syvain-metrics",
3
- "version": "0.0.360",
3
+ "version": "0.0.367",
4
4
  "description": "Syvain Metrics command line interface.",
5
5
  "repository": {
6
6
  "type": "git",
@@ -15,11 +15,11 @@
15
15
  "syvain-metrics": "bin/syvain-metrics.js"
16
16
  },
17
17
  "optionalDependencies": {
18
- "syvain-metrics-linux-x64": "0.0.360",
19
- "syvain-metrics-linux-arm64": "0.0.360",
20
- "syvain-metrics-darwin-x64": "0.0.360",
21
- "syvain-metrics-darwin-arm64": "0.0.360",
22
- "syvain-metrics-windows-x64": "0.0.360",
23
- "syvain-metrics-windows-arm64": "0.0.360"
18
+ "syvain-metrics-linux-x64": "0.0.367",
19
+ "syvain-metrics-linux-arm64": "0.0.367",
20
+ "syvain-metrics-darwin-x64": "0.0.367",
21
+ "syvain-metrics-darwin-arm64": "0.0.367",
22
+ "syvain-metrics-windows-x64": "0.0.367",
23
+ "syvain-metrics-windows-arm64": "0.0.367"
24
24
  }
25
25
  }