@moda-ai/cli 1.55.19 → 1.55.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/{cli-c5c6f0yw.js → cli-38p87ykc.js} +1 -1
- package/dist/{cli-5km7j04z.js → cli-4j8nffsm.js} +6 -2
- package/dist/{cli-ssfq84k5.js → cli-sf0g0fjg.js} +133 -18
- package/dist/cli.js +5 -5
- package/dist/{index-6qqmjqb2.js → index-ewqbpnwz.js} +3 -3
- package/dist/{skills-d35nnm66.js → skills-br82cpt7.js} +2 -2
- package/dist/{skills-bundled-mdmhzfsr.js → skills-bundled-r9a2vs07.js} +2 -2
- package/dist/{sync-n2psdffz.js → sync-25vn78je.js} +2 -2
- package/package.json +1 -1
- package/skills/integration/index.json +2 -2
- package/skills/moda-cli/SKILL.md +114 -1434
- package/skills/moda-cli/references/auth.md +176 -0
- package/skills/moda-cli/references/commands.md +74 -0
- package/skills/moda-cli/references/dashboard-parity.md +29 -0
- package/skills/moda-cli/references/detections.md +79 -0
- package/skills/moda-cli/references/feedback.md +45 -0
- package/skills/moda-cli/references/output-protocol.md +260 -0
- package/skills/moda-cli/references/production-intelligence.md +96 -0
- package/skills/moda-cli/references/prompts.md +104 -0
- package/skills/moda-cli/references/recipes.md +74 -0
- package/skills/moda-cli/references/replays-and-registry.md +173 -0
- package/skills/moda-cli/references/search-and-traces.md +180 -0
- package/skills/moda-cli/references/signals-and-false-positives.md +79 -0
- package/skills/moda-cli/references/troubleshooting.md +65 -0
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Setup and authentication
|
|
2
|
+
|
|
3
|
+
Loaded on demand from the moda-cli skill. Run `moda auth --help` for the current subcommands and flags.
|
|
4
|
+
|
|
5
|
+
## Setup
|
|
6
|
+
|
|
7
|
+
### First-time install
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npm install -g @moda-ai/cli
|
|
11
|
+
moda init # browser-based auth + key provisioning
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Or one-shot via npx without installing:
|
|
15
|
+
|
|
16
|
+
```bash
|
|
17
|
+
npx -p @moda-ai/cli moda <command>
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
### Manual key
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
export MODA_API_KEY="moda_sk_..." # from moda.dev/settings
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
### Headless provisioning (CI / agents): `moda provision`
|
|
27
|
+
|
|
28
|
+
`moda provision` mints (or reuses) a tenant API key with zero prompts and
|
|
29
|
+
prints a single JSON credentials document to stdout — in every output mode,
|
|
30
|
+
even on an interactive terminal, so it composes directly:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
KEY=$(moda provision --tenant-id t1 | jq -r .apiKey)
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Output shape:
|
|
37
|
+
|
|
38
|
+
```json
|
|
39
|
+
{
|
|
40
|
+
"apiKey": "moda_sk_...",
|
|
41
|
+
"tenantId": "t1",
|
|
42
|
+
"tenantName": "Acme",
|
|
43
|
+
"tenantSlug": "acme",
|
|
44
|
+
"ingestUrl": "https://moda-ingest.modas.workers.dev",
|
|
45
|
+
"baseUrl": "https://moda.dev",
|
|
46
|
+
"reused": false
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Rules and limitations:
|
|
51
|
+
|
|
52
|
+
- **Requires an existing CLI session.** Minting a *new* key is control-plane
|
|
53
|
+
work: with no stored session it exits `4` (`auth_required`) telling you to
|
|
54
|
+
run `moda auth login` on a machine with a browser first, or to skip
|
|
55
|
+
provisioning and export `MODA_API_KEY` directly. If you already hold a key,
|
|
56
|
+
you do not need this command at all — `moda auth login --api-key` gets you
|
|
57
|
+
to an authenticated state with no browser.
|
|
58
|
+
- **Never prompts, even on a TTY.** A single-tenant account auto-selects;
|
|
59
|
+
with multiple tenants you must pass `--tenant-id=<id>` (or set
|
|
60
|
+
`MODA_TENANT_ID`), otherwise it exits `5` (`needs_input`) with per-tenant
|
|
61
|
+
`resume_flags`.
|
|
62
|
+
- `reused: true` means a previously stored key for that tenant passed a
|
|
63
|
+
liveness probe and was reused instead of minting a new one (same dedup +
|
|
64
|
+
idempotent bootstrap machinery as `moda init`).
|
|
65
|
+
- **Does not write `~/.moda/config.json` by default** — headless callers own
|
|
66
|
+
their credential storage. Pass `--save` to opt into the init-style
|
|
67
|
+
persistence.
|
|
68
|
+
- `--label=<text>` names the created key (default: machine hostname);
|
|
69
|
+
`--profile=NAME` scopes the session lookup.
|
|
70
|
+
- stdout carries a live credential — a warning is printed to stderr; never
|
|
71
|
+
pipe provision stdout into logs.
|
|
72
|
+
|
|
73
|
+
**When to use which:** `moda init` is the interactive first-time onboarding
|
|
74
|
+
wizard for a repo (harness discovery, skills, prompts, validation);
|
|
75
|
+
`moda provision` is for CI jobs and agents that already have a session and
|
|
76
|
+
just need credentials.
|
|
77
|
+
|
|
78
|
+
### Auth session inspection and token composition
|
|
79
|
+
|
|
80
|
+
```bash
|
|
81
|
+
moda auth login # browser PKCE sign-in; stores a moda_ct_ session token
|
|
82
|
+
moda auth login --api-key # NON-INTERACTIVE: validate MODA_API_KEY and save it
|
|
83
|
+
moda auth login --non-interactive # never open a browser; fail if no key is available
|
|
84
|
+
printf %s "$KEY" | moda auth login --api-key-stdin # key never touches argv
|
|
85
|
+
moda auth login --api-key=<key> # inline; WARNS, key is visible in argv + shell history
|
|
86
|
+
moda auth logout # revoke the session server-side, then delete it locally
|
|
87
|
+
moda auth status # both credentials; exit 0 if either is usable, else 4
|
|
88
|
+
moda auth status --online # additionally probe GET /api/cli/context
|
|
89
|
+
moda auth whoami # identity from the session, or from the API key
|
|
90
|
+
moda auth token # print the raw bearer token + newline, nothing else
|
|
91
|
+
curl -H "Authorization: Bearer $(moda auth token)" https://moda.dev/api/cli/context
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
**If you are an agent with `MODA_API_KEY` and no browser, this is the whole
|
|
95
|
+
flow:**
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
moda auth whoami # exit 0 + "verified": true means you are authenticated
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The API key alone is sufficient for every data command; `moda auth login` is
|
|
102
|
+
only needed for control-plane work (`moda provision`, `moda init`). There are
|
|
103
|
+
two independent credentials and `auth` reports both:
|
|
104
|
+
|
|
105
|
+
| Credential | Obtained by | Used for |
|
|
106
|
+
| --- | --- | --- |
|
|
107
|
+
| `moda_ct_` session token | `moda auth login` (browser PKCE) | control plane: `provision`, `init`, `auth token` |
|
|
108
|
+
| `moda_sk_` API key | `MODA_API_KEY`, `moda provision`, `moda init` | every data command |
|
|
109
|
+
|
|
110
|
+
- `moda auth login` opens the browser consent flow at `{MODA_BASE_URL}/auth/cli`
|
|
111
|
+
with a localhost callback (ports 8432-8435, first free). Approving in the
|
|
112
|
+
browser hands the CLI a one-time code that it exchanges for a `moda_ct_...`
|
|
113
|
+
session token (sliding 30-day expiry, renewed automatically in the
|
|
114
|
+
background while you keep using the CLI). If the browser cannot reach the
|
|
115
|
+
machine, paste the callback URL or the code shown on the consent page into
|
|
116
|
+
the terminal.
|
|
117
|
+
- `moda auth logout` revokes the stored `moda_ct_` session server-side
|
|
118
|
+
(best-effort) and deletes the local credential; legacy sessions are just
|
|
119
|
+
deleted locally.
|
|
120
|
+
- `moda auth login --api-key` authenticates **without a browser**. Prefer
|
|
121
|
+
`MODA_API_KEY` (bare `--api-key`) or `--api-key-stdin`; `--api-key=<key>`
|
|
122
|
+
works but puts the secret in argv and shell history, so it warns. The key is
|
|
123
|
+
validated against the Data API before anything is written, then saved to
|
|
124
|
+
`~/.moda/config.json`. A rejected key (401/403) or an unreachable API both
|
|
125
|
+
fail without persisting. `--non-interactive` is the same path plus a
|
|
126
|
+
guarantee that no browser opens even when no key is found — it exits with
|
|
127
|
+
the commands that would mint one.
|
|
128
|
+
- If `MODA_API_KEY` holds a **different** key than the one being saved, the
|
|
129
|
+
environment still wins afterwards. Login says so instead of reporting a
|
|
130
|
+
clean success: `"shadowed_by_env": true`, `"effective_credential": "env"`,
|
|
131
|
+
exit `3`. Unset `MODA_API_KEY` to use the saved credential.
|
|
132
|
+
- `moda auth whoami` reports the browser session when there is one. With no
|
|
133
|
+
session but a usable API key it reports
|
|
134
|
+
`{ "credential": "api_key", "apiKeyCredential": { "source": <"env"|"profile"|"config">,
|
|
135
|
+
"masked": <string>, "verified": <bool> }, "tenant_id": <string|null> }`.
|
|
136
|
+
`verified: true` means the key was accepted by the API just now. If the API
|
|
137
|
+
is unreachable the command exits `3` (degraded) with `verified: false`
|
|
138
|
+
rather than claiming an identity it could not confirm; a rejected key is an
|
|
139
|
+
error, not a "logged out" state.
|
|
140
|
+
- `moda auth status` reports
|
|
141
|
+
`{ "loggedIn": <bool>, "authenticated": <bool>, "apiKeyCredential": {...},
|
|
142
|
+
"profile": <string|null>, "expiresAt": <ISO|null> }`
|
|
143
|
+
plus `"user": { "email": <string> }` when the stored session carries an
|
|
144
|
+
email (PKCE `moda_ct_` sessions do; legacy sessions omit it). `loggedIn` is
|
|
145
|
+
**browser-session only**; `authenticated` is true when *either* credential is
|
|
146
|
+
usable, and the exit code follows `authenticated` (`0` usable, `4` neither).
|
|
147
|
+
`apiKeyCredential` is `{ "present": false }` or
|
|
148
|
+
`{ "present": true, "source": <"env"|"profile"|"config">, "masked": <string>,
|
|
149
|
+
"tenantId": <string|null> }` — `source: "env"` tells you a `MODA_API_KEY` is
|
|
150
|
+
shadowing anything on disk. The raw key is never emitted. Exit `4` is an
|
|
151
|
+
**expected state signal, not an error**: the payload is clean data (no error
|
|
152
|
+
block, no stderr noise), and no browser is ever opened.
|
|
153
|
+
Without `--online` there is no network I/O, so `authenticated` means the
|
|
154
|
+
credential is **present**, not that it still works, and
|
|
155
|
+
`apiKeyCredential.verified` is `null`. With `--online` the key is probed too:
|
|
156
|
+
a rejected key gets `verified: false`, stops counting toward `authenticated`,
|
|
157
|
+
and exits `4`; an unreachable API leaves it counting, so a blip never kills a
|
|
158
|
+
good credential. With `--online` the payload adds
|
|
159
|
+
`{ "online": { "reachable": <bool>, "ok": <bool>, "tenantCount": <n> } }`:
|
|
160
|
+
`reachable: true` means any HTTP response arrived (even a 401/500), `ok`
|
|
161
|
+
reflects HTTP success, and `tenantCount` is present only when the response
|
|
162
|
+
was ok and carried a tenants array. A network failure or probe timeout
|
|
163
|
+
(5 s) yields `reachable: false`; the exit code always follows local session
|
|
164
|
+
validity only, never the probe.
|
|
165
|
+
- `moda auth token` writes exactly the bare token plus a trailing newline to
|
|
166
|
+
stdout in **every** output mode — including piped/json/agent, because
|
|
167
|
+
command substitution is a pipe — so `$(moda auth token)` composes safely
|
|
168
|
+
with no envelope to strip. When logged out it prints a stderr message and
|
|
169
|
+
exits `4`; with an explicit `--json` flag it instead prints
|
|
170
|
+
`{"error":"not_authenticated"}` to stdout (still exit `4`).
|
|
171
|
+
- `moda auth token` prints whichever credential kind is stored. Newer logins
|
|
172
|
+
store a `moda_ct_...` CLI session token, which authenticates **`/api/cli/*`
|
|
173
|
+
endpoints only** — it is not accepted by the Data API (`/api/v1/data/*`
|
|
174
|
+
wants a `moda_sk_...` API key) or other dashboard endpoints. Legacy
|
|
175
|
+
sessions print the Auth0 bearer token as before.
|
|
176
|
+
- Both respect `--profile=NAME` and `MODA_PROFILE`.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# All commands at a glance
|
|
2
|
+
|
|
3
|
+
Loaded on demand from the moda-cli skill. This table can lag the CLI: `moda <command> --help` is authoritative for flags, types, ranges, and defaults.
|
|
4
|
+
|
|
5
|
+
## Reference: all commands at a glance
|
|
6
|
+
|
|
7
|
+
| Command | Purpose |
|
|
8
|
+
|---|---|
|
|
9
|
+
| `moda init` | Browser auth + provision API key + install agent rules |
|
|
10
|
+
| `moda provision` | Headless tenant + API key provisioning; single JSON credentials doc on stdout (exit `4` without a session, `5` when `--tenant-id` is needed) |
|
|
11
|
+
| `moda auth login` | Browser PKCE sign-in; stores a revocable `moda_ct_` session token |
|
|
12
|
+
| `moda auth login --api-key` | Non-interactive: validate `MODA_API_KEY` (or `--api-key=<key>`) and save it; never opens a browser |
|
|
13
|
+
| `moda auth whoami` | Authenticated identity from the session, or from the API key when there is no session |
|
|
14
|
+
| `moda auth logout` | Revoke the CLI session server-side, then delete it locally |
|
|
15
|
+
| `moda auth status` | Both credentials; exit `0` if either is usable, `4` if neither (expected, not an error) |
|
|
16
|
+
| `moda auth token` | Print the raw bearer token for `$(moda auth token)` composition |
|
|
17
|
+
| `moda search "<query>"` | **Message-grain semantic/keyword/hybrid search (incl. tool calls); primary content-discovery path** |
|
|
18
|
+
| `moda overview` | Dashboard KPIs, top clusters, recent activity |
|
|
19
|
+
| `moda ask "<question>"` | Natural-language production/harness answer with evidence (exit `3` = degraded local fallback) |
|
|
20
|
+
| `moda investigate` | Rank production issues (tool failures, frustrations, top Problems) with evidence + next commands |
|
|
21
|
+
| `moda clusters` | Browse topic cluster hierarchy; `--search="q"` finds clusters by meaning, `--node-id=ID` resolves a deep link |
|
|
22
|
+
| `moda cluster-traces <node_id>` | Traces in a cluster (legacy alias: `moda cluster-conversations`) |
|
|
23
|
+
| `moda traces` | List/filter traces by structured fields (legacy alias: `moda conversations`) |
|
|
24
|
+
| `moda context <conversation_id>` | Windowed message context (max 5 per side); `--msg-index=N` (alias `--message-index`) anchors the window |
|
|
25
|
+
| `moda frustrations` | User frustration detections with evidence (the frustration family of `emotions`, legacy field names) |
|
|
26
|
+
| `moda emotions` | Multi-family emotion detections: frustration, disapproval, sadness, confusion, anxiety, trust, positive (`--family=F`, `--user-id`, `--conversation-id`, limit 1–20) |
|
|
27
|
+
| `moda hallucinations` | Grounding detections: contradicted/verified outputs, rule breakdown (`--kind=contradicted\|verified`, `--conversation-id=ID`) |
|
|
28
|
+
| `moda tool-failures` | Tool failure overview |
|
|
29
|
+
| `moda tool-failure-detail <tool_name>` | Per-tool failure breakdown + examples |
|
|
30
|
+
| `moda problems` | Rank cross-signal Problems by root cause (what to fix first); takes only `--days-back`/`--limit`. Filter by family with `moda problems remainder --family=F` or `moda problem <problem_id> --traces --family=F` |
|
|
31
|
+
| `moda problem <problem_id>` | One Problem: dossier, or `--evidence`/`--reports`/`--traces`/`--feedback` pages (`--conversations` is a legacy alias of `--traces`) (`--limit` 1–50, `--cursor` verbatim keyset token) |
|
|
32
|
+
| `moda problem-feedback <problem_id>` | Write: `--action=mark_fixed\|dismiss\|flag_attribution\|rename`. `--reason` required for flag_attribution, optional for dismiss; `--new-name` for rename; `--attribution-id` (UUID) required for flag_attribution |
|
|
33
|
+
| `moda false-positive <kind> <target>` | Write: kinds `problem\|problem-detection\|quote\|tool\|laziness\|hallucination\|custom-signal`; `--reason` optional; problem-detection needs `--conversation-id` + `--attribution-id`; quote needs `--family`, `--turn`, `--quote`; tool needs `--tool` plus `--tool-use-id` or `--msg-index`; custom-signal needs `--conversation-id`; `--undo` removes a mark (not for problem) |
|
|
34
|
+
| `moda false-positives [conversation_id]` | Current false-positive marks: all (`--kind=quote\|tool\|laziness\|hallucination\|custom-signal`, `--all` to include removed, `--limit`) or one trace with its problem detection reviews. Problem dismissals are read with `moda problem <problem_id> --feedback`, detection reviews with `moda detection-reviews <conversation_id>` |
|
|
35
|
+
| `moda laziness` | Laziness detections per trace (`--days-back`, `--limit` 1-50, `--conversation-id`, `--family`) |
|
|
36
|
+
| `moda problem-linear <problem_id>` | The Problem's linked Linear issue (key, state, URL) or `link: null`; needs Linear connected (Integrations → Linear) |
|
|
37
|
+
| `moda problem-linear-file <problem_id>` | Write: file the Problem as a Linear issue (`--team-id`, `--title`, `--priority=0-4`), or link an existing one with `--issue=ENG-123\|URL` (exclusive with the others). Idempotent per Problem |
|
|
38
|
+
| `moda detection-reviews <conversation_id>` | Problem detections on a trace with review state (`--attribution-id=UUID` narrows to one) |
|
|
39
|
+
| `moda detection-review <problem_id>` | Write: `--conversation-id`, `--attribution-id` (UUID), `--verdict=correct\|false_positive\|clear`, optional `--note` |
|
|
40
|
+
| `moda signals` | Custom signals: criteria, status, 7-day judged/matched stats |
|
|
41
|
+
| `moda signal <signal_id>` | One custom signal; `--detections` pages recent hits (`--days-back` 1–90, `--limit` 1–20, `--offset`) |
|
|
42
|
+
| `moda trace-signals <conversation_id>` | Custom signal hits on one trace |
|
|
43
|
+
| `moda signal-label <signal_id>` | Write: `--conversation-id`, `--label=fits\|does_not_fit` (false positive), optional `--note`, `--msg-index`/`--msg-dedup-token` |
|
|
44
|
+
| `moda world-state <conversation_id>` | Agent memory: slots/threads/events; `--summary-only`; `--snapshot --msg-index=N` (state at a turn); `--replay --message-count=N` (state over time) |
|
|
45
|
+
| `moda failures` | Tool failures only, ranked (no frustrations or Problems) |
|
|
46
|
+
| `moda tail` | Live tail: one JSON line per new trace/detection (`--signal=traces\|emotions\|all`, legacy alias `--signal=conversations`, `--interval=N`, `--once`, `--max-events=N`). **Emotions caveat:** `/emotions` is ranked by score with no time ordering or cursor, so the tail follows the *highest-scoring* detections rather than everything; each poll emits a `tail_coverage` line with `scanned`/`total`/`coverage_pct`/`complete`. Pass `--full-scan` for a complete window scan (many more requests, capped by the API's offset ceiling of 10000). |
|
|
47
|
+
| `moda feedback "<note>"` | Flag wrong/missing data or CLI quirks to the Moda team |
|
|
48
|
+
| `moda prompts status` | Read-only local prompt status |
|
|
49
|
+
| `moda prompts diff` | Read-only: only prompts changed/new/deleted since last sync (hash-level, not a text diff) |
|
|
50
|
+
| `moda prompts sync` | Upload changed prompt versions and update lockfile |
|
|
51
|
+
| `moda prompts promote <key>` | Move a remote prompt label |
|
|
52
|
+
| `moda registry template\|validate\|push\|status\|coverage\|log\|show\|prompt` | Register prompts, skills, tools, and agent wiring from one agent-written manifest (`--file`, `--dry-run`, `--live`, `--message`, `--days`); every push is a commit in the Registry history; no repo scan |
|
|
53
|
+
| `moda users` / `moda user <id>` | End users with health/activity (`--page-size=N`, alias `--limit=N`; `--page=N`); one user's profile |
|
|
54
|
+
| `moda problem-reopen <pid>` | Reopen a Problem marked fixed |
|
|
55
|
+
| `moda signals <create\|update\|activate\|pause\|delete\|examples\|refine\|revise\|proposals\|accept\|reject\|search\|preview>` | Manage custom signals (writes take `--dry-run`; LLM-backed ones are rate-limited per key) |
|
|
56
|
+
| `moda prompts list\|show\|proposals\|usage\|replay-runs\|decide` | Read the prompt registry; merge/reject proposals |
|
|
57
|
+
| `moda replays sets\|runs\|run\|playouts\|playout` | Replay results: run history, per-seed verdicts, playout transcripts with tool provenance (read-only) |
|
|
58
|
+
| `moda skills list\|show\|policy` / `moda harness list\|show\|versions\|diff` | Skill library and synced harnesses |
|
|
59
|
+
|
|
60
|
+
`moda search` flags: `--mode=keyword|semantic|hybrid` (default `hybrid`),
|
|
61
|
+
`--user-id`, `--time-range=<range>`, `--limit=N` (1–100, default 20).
|
|
62
|
+
|
|
63
|
+
There is no flag set common to every command: each command rejects a flag it
|
|
64
|
+
does not take, and the error lists the flags it does. For the time window see
|
|
65
|
+
[Time-window flags by command](output-protocol.md#time-window-flags-by-command) (`--days-back`
|
|
66
|
+
on most detection commands, `--time-range` on `traces`/`search`/`clusters`,
|
|
67
|
+
`--window` on `users`/`user`); `--limit`/`--offset` exist only on paged list
|
|
68
|
+
commands (`moda <command> --help` shows them). Boolean flags take `true`/`false`
|
|
69
|
+
and also `1`/`0`, `yes`/`no`, `on`/`off`.
|
|
70
|
+
`frustrations` and `tool-failure-detail` additionally accept
|
|
71
|
+
`--include-window` (attach a `window` slice per row) and `--window=N` (1–5,
|
|
72
|
+
default `1`; half-width, so `--window=1` ⇒ 3 messages). `emotions` accepts
|
|
73
|
+
`--include-window` to keep each detection's `conversation_context` in agent
|
|
74
|
+
output. `problem <id>` accepts `--full` for the whole dossier in agent output.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Dashboard parity
|
|
2
|
+
|
|
3
|
+
Loaded on demand from the moda-cli skill.
|
|
4
|
+
|
|
5
|
+
### 10. Everything the dashboard does, from the CLI
|
|
6
|
+
|
|
7
|
+
Every command below needs only an API key, and the tenant comes from the key.
|
|
8
|
+
Writes listed here accept `--dry-run` (prints the exact request, sends nothing) except `problem-feedback`, `false-positive`, `detection-review`, `signal-label`, `problem-linear-file`, and `prompts promote|ab|propose`. Those refuse `--dry-run` before sending anything, so run them only when you mean it.
|
|
9
|
+
|
|
10
|
+
| Dashboard | CLI |
|
|
11
|
+
|---|---|
|
|
12
|
+
| Traces list: sort, page | `moda traces --sort=recent\|longest\|slowest --offset=N` |
|
|
13
|
+
| Trace detail: whole transcript | `moda context <id> --all` (ordered, with tool calls/results) |
|
|
14
|
+
| Users page / user profile | `moda users [--window --cohort --sort --order --health-status]`, `moda user <id>` |
|
|
15
|
+
| Problems: unexplained remainder | `moda problems remainder [--family=emotion\|tool_failure\|laziness\|… --cursor]` |
|
|
16
|
+
| Reopen a problem marked fixed | `moda problem-reopen <problem_id> --reason="…"` (resolved ones return only via re-detection) |
|
|
17
|
+
| Custom signals: create | `moda signals draft --note="…"`, then `moda signals create --name --description --pass-criteria --fail-criteria [--tool-names=a,b]` or `--file=body.json` |
|
|
18
|
+
| Custom signals: edit, lifecycle | `moda signals update <id> …`, `moda signals activate\|pause\|delete <id>` (delete: drafts only; archiving is dashboard-only) |
|
|
19
|
+
| Custom signals: review & tune | `moda signals examples <id> add\|update\|remove --conversation-id=…`, `refine <id> --request="…"`, `revise <id> --conversation-ids=…`, `proposals <id>`, `accept\|reject <id> <proposal_id>` |
|
|
20
|
+
| Custom signals: find matches | `moda signals search <id>`, `search-status <id> <search_id>`, `search-stop <id> <search_id>`, `preview --file=detector.json` |
|
|
21
|
+
| Replays: run detail, playout transcripts | `moda replays runs <set_id>`, `run <set_id> <run_id>`, `playouts <set_id> <run_id> [--case --arm --seed --transcript]`, `playout <set_id> <run_id> <playout_id>` |
|
|
22
|
+
| Prompts registry | `moda prompts list [--label --search]`, `show <key>`, `proposals <key>`, `usage <key>`, `replay-runs <key>`, `decide <key> <proposal_id> --action=merge\|reject\|reopen` |
|
|
23
|
+
| Skills library | `moda skills list`, `show <key>`, `policy <key> --live\|--local` |
|
|
24
|
+
| Harness pages | `moda harness list`, `show <id>`, `versions <id>`, `diff <id> --from=… --to=…` |
|
|
25
|
+
|
|
26
|
+
Not available to API keys on purpose (a human does these in the dashboard):
|
|
27
|
+
members/invites/roles, API-key minting, billing, provider credentials,
|
|
28
|
+
OAuth connections (Slack/Linear/GitHub), alert rules that message people,
|
|
29
|
+
and archiving an active custom signal (pause it, or delete a draft).
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Detections: frustrations, emotions, tool failures
|
|
2
|
+
|
|
3
|
+
Loaded on demand from the moda-cli skill.
|
|
4
|
+
|
|
5
|
+
### 5. Frustration analysis
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
moda frustrations # last 7 days
|
|
9
|
+
moda frustrations --days-back=14 --limit=20
|
|
10
|
+
moda frustrations --include-window --window=1 --limit=5 # agent output omits windows unless asked
|
|
11
|
+
moda emotions --family=frustration --user-id=<id> # one end user
|
|
12
|
+
moda emotions --conversation-id=<id> # every family, one trace
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
In agent output, `frustrations` and `emotions` rows leave out the embedded
|
|
16
|
+
conversation window (most of each row) unless you pass `--include-window`;
|
|
17
|
+
each row keeps `anchor` and a `context_command` to read the turns around it.
|
|
18
|
+
|
|
19
|
+
`--user-id` and `--conversation-id` work on `emotions` and `frustrations`, and
|
|
20
|
+
scope *every* figure: summary, breakdown, list and total. `--limit` is 1–20 on
|
|
21
|
+
both, and on `hallucinations` and `tool-failure-detail`. Page with `--offset`.
|
|
22
|
+
`frustrations` reads the same data as `emotions --family=frustration` and the
|
|
23
|
+
dashboard, in the legacy field names: `frustrated_count` = `by_family.frustration`
|
|
24
|
+
(detected traces), `total_analyzed` = `conversations_analyzed`. In `emotions`,
|
|
25
|
+
`pagination.total` counts detected **plus** at-risk rows (`is_detected: false`,
|
|
26
|
+
`risk_score` >= 0.5); use `pagination.detected` / `pagination.at_risk` for the split.
|
|
27
|
+
|
|
28
|
+
Each result includes an inline trace snippet, user quotes, trajectory,
|
|
29
|
+
signal breakdown (exasperation, profanity, anger, sarcasm, giving_up, insult),
|
|
30
|
+
and primary cause.
|
|
31
|
+
|
|
32
|
+
Every row carries a top-level `anchor` block:
|
|
33
|
+
`{ kind: 'frustration', conversation_id, msg_index, signal, quote_preview,
|
|
34
|
+
all_turns, no_anchor }`. `msg_index` is the 0-indexed turn where the
|
|
35
|
+
frustration fired, derived from `key_turns[0]` (the same turn the API centers
|
|
36
|
+
the embedded window on), falling back to `user_quotes[0].turn` when
|
|
37
|
+
`key_turns` is empty — reference `anchor.conversation_id` and
|
|
38
|
+
`anchor.msg_index` directly instead of digging into `user_quotes` to locate
|
|
39
|
+
the moment.
|
|
40
|
+
`no_anchor: true` means no anchor could be derived.
|
|
41
|
+
|
|
42
|
+
Pass `--include-window` to attach a `window` field per row with the message
|
|
43
|
+
slice centered on the anchor. `--window=N` (1–5, default `1`) is the
|
|
44
|
+
half-width; `--window=1` ⇒ 3 messages total (anchor ± 1), the canonical
|
|
45
|
+
"three lines" view.
|
|
46
|
+
|
|
47
|
+
### 6. Tool failure investigation
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
moda tool-failures # which tools are breaking
|
|
51
|
+
moda tool-failure-detail <tool_name> # subtype breakdown + examples
|
|
52
|
+
moda tool-failure-detail search --subtype=dependency.timeout --limit=10 # a value from `subtypes`
|
|
53
|
+
moda tool-failure-detail search --include-window --window=1 --limit=5
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Every example row in `tool-failure-detail` output carries a top-level
|
|
57
|
+
`anchor` block:
|
|
58
|
+
`{ kind: 'tool_failure', conversation_id, msg_index, tool_name, tool_use_id,
|
|
59
|
+
error_subtype, no_anchor }`. `msg_index` is the 0-indexed turn of the failing
|
|
60
|
+
tool call — reference `anchor.conversation_id` and `anchor.msg_index`
|
|
61
|
+
directly instead of matching by `tool_use_id` against the trace.
|
|
62
|
+
`no_anchor: true` means no anchor could be derived.
|
|
63
|
+
|
|
64
|
+
By default each row embeds `conversation`: the anchor ± 2 messages (without
|
|
65
|
+
the duplicate `content_blocks`), plus `context_command`, the exact
|
|
66
|
+
`moda context <conversation_id> --msg-index=<msg_index> --window=2` read for
|
|
67
|
+
the untrimmed window. `--compact` (same as `--include-window=false` or
|
|
68
|
+
`--no-include-window`) drops `conversation` and keeps the anchor and
|
|
69
|
+
`context_command`.
|
|
70
|
+
|
|
71
|
+
Pass `--include-window` to attach a `window` field per row (instead of
|
|
72
|
+
`conversation`) with the message slice centered on the anchor. `--window=N`
|
|
73
|
+
(1–5, default `1`) is the half-width; `--window=1` ⇒ 3 messages total
|
|
74
|
+
(anchor ± 1).
|
|
75
|
+
|
|
76
|
+
Examples default to `--limit=5` (max 20). Example text is returned whole; if
|
|
77
|
+
the agent envelope goes over its byte limit, whole trailing examples are
|
|
78
|
+
dropped and `truncation.data.full_output_path` holds every example. Agent output lists the 20 most frequent
|
|
79
|
+
`subtypes` (`subtypes_total` has the full count); `--json` lists all.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Feedback
|
|
2
|
+
|
|
3
|
+
Loaded on demand from the moda-cli skill.
|
|
4
|
+
|
|
5
|
+
## Feedback: help us improve
|
|
6
|
+
|
|
7
|
+
Successful agent envelopes carry a `meta.tip` reminding you of this. When a
|
|
8
|
+
response looks wrong (a cluster label that doesn't match its traces,
|
|
9
|
+
a frustration whose causes don't match the transcript, an empty result that
|
|
10
|
+
should not be empty, an API quirk), flag it with `moda feedback`. The Moda
|
|
11
|
+
team reads these to fix data quality issues. Only submit genuine
|
|
12
|
+
observations; never run the command with placeholder text.
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
moda feedback "cluster 'billing' is mostly refund traces" \
|
|
16
|
+
--category=bad_cluster_label --cluster-id=<node_id>
|
|
17
|
+
moda feedback "search finds nothing for a trace I can open" \
|
|
18
|
+
--category=missing_data --conversation-id=<id>
|
|
19
|
+
moda feedback "a cancelled call is counted as a tool failure" \
|
|
20
|
+
--category=wrong_tool_failure --tool-name=<tool>
|
|
21
|
+
moda feedback "trace is a refund request, not this problem" \
|
|
22
|
+
--category=false_positive_attribution --problem-id=<id> --conversation-id=<id>
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
- `<note>` is required (max 4000 chars), as the argument or `--note=TEXT`.
|
|
26
|
+
Say what looked wrong and why.
|
|
27
|
+
- `--category`: `bad_cluster_label | mismatched_frustration | missing_data |
|
|
28
|
+
noisy_data | wrong_tool_failure | incorrect_loop | api_quirk |
|
|
29
|
+
false_positive_attribution | misrouted_fix | harness_bug | other`
|
|
30
|
+
(default `other`). Use `false_positive_attribution` when a trace is
|
|
31
|
+
attributed to the wrong problem, `misrouted_fix` when a fix targets the
|
|
32
|
+
wrong problem or file, and `harness_bug` for a bug in the agent harness
|
|
33
|
+
Moda analysed. An unknown category is still sent, as `other` with
|
|
34
|
+
`refs.raw_category`, and a warning lists the valid ones.
|
|
35
|
+
- `--severity`: `info | low | medium | high` (default `low`).
|
|
36
|
+
- Attach ids when you have them: `--conversation-id`, `--cluster-id`,
|
|
37
|
+
`--tool-name`, `--run-id`, or any other `--<name>-id` (`--problem-id`,
|
|
38
|
+
`--fix-id`, `--attribution-id`, `--trace-id`, `--span-id`,
|
|
39
|
+
`--message-index`); they go into `refs` and let the team reproduce what
|
|
40
|
+
you saw.
|
|
41
|
+
- Feedback does not change the problem itself. To act on a misroute directly,
|
|
42
|
+
use `moda problem-feedback <problem_id> --action=flag_attribution
|
|
43
|
+
--attribution-id=<uuid> --reason=TEXT` or `moda false-positive <kind> <target>
|
|
44
|
+
--reason=TEXT`.
|
|
45
|
+
- Never include secrets or full transcripts in the note.
|