@moda-ai/cli 1.0.4 → 1.0.5

package/skills/moda-cli/SKILL.md

@@ -0,0 +1,291 @@
+---
+name: moda-cli
+description: Query Moda's AI agent conversation analytics from the terminal — overview KPIs, topic clusters, conversation search, message context, user frustration detections, tool failures, agent jobs (per-run outcomes), recurring workflows, and active loop detections. Use when the user asks about moda, modaflows, conversation analytics, user frustrations, agent observability, tool failure debugging, why an agent run partially completed or looped, or wants to investigate how their AI agent is performing.
+---
+
+# Moda CLI
+
+## What this skill does
+
+Wraps the `moda` CLI so the agent can query a Moda tenant's conversation
+analytics directly. Every command returns JSON on stdout; pipe to `jq` to
+filter. Read-only — no command in this skill modifies data.
+
+## When to use
+
+Activate this skill when the user wants to:
+
+- Inspect agent health: conversation volume, frustration rate, tool failures
+- Debug a specific frustrated conversation
+- Investigate which tools are failing and why
+- Browse conversation topics (clusters)
+- Search conversations by content, user, environment, or time range
+
+**Do not** activate for: writing code that calls the Moda Data API directly
+(use the API docs instead), instrumenting an app with the Moda SDK (different
+skill / docs), or any modification action — this CLI is read-only.
+
+## Required inputs
+
+Before running any command except `moda init`:
+
+- `MODA_API_KEY` env var must be set (`moda_sk_...`). If missing, run
+  `moda init` to provision one, or have the user export it.
+
+Optional:
+
+- `MODA_BASE_URL` defaults to `https://modaflows.com`; override only for
+  self-hosted or staging.
+
+## Setup
+
+### First-time install
+
+```bash
+npm install -g @moda-ai/cli
+moda init                       # browser-based auth + key provisioning
+```
+
+Or one-shot via npx without installing:
+
+```bash
+npx -p @moda-ai/cli moda <command>
+```
+
+### Manual key
+
+```bash
+export MODA_API_KEY="moda_sk_..."   # from modaflows.com/settings
+```
+
+## Procedure
+
+Default investigation pattern: **broad → narrow → context**. Always start with
+`moda overview` to ground subsequent calls in real numbers.
+
+### 1. Overview (always start here)
+
+```bash
+moda overview                        # last 7 days
+moda overview --days-back=30
+```
+
+Returns total conversations, trend, frustration rate, tool failure summary,
+top clusters, recent activity.
+
+### 2. Browse topic clusters
+
+```bash
+moda clusters                                 # root-level
+moda clusters --parent-id=<node_id>           # drill in
+moda cluster-conversations <node_id>          # conversations in cluster
+```
+
+### 3. Search conversations
+
+```bash
+export MODA_API_KEY="moda_sk_..."   # from modaflows.com/settings
+```
+
+Filters: `--search`, `--cluster-id`, `--user-id`, `--time-range`
+(`all|1h|3d|7d|24h|30d|90d`), `--environment`
+(`all|development|staging|production`), `--limit`, `--offset`.
+
+### 4. Read a single conversation
+
+```bash
+moda context <conversation_id>                # default window around middle
+moda context <conversation_id> --msg-index=5  # center on message 5
+moda context <conversation_id> --window=3     # 3 messages each side (max 5)
+```
+
+### 5. Frustration analysis
+
+```bash
+moda frustrations                             # last 7 days
+moda frustrations --days-back=14 --limit=20
+```
+
+Each result includes inline conversation snippet, user quotes, trajectory,
+signal breakdown (exasperation, profanity, anger, sarcasm, giving_up, insult),
+and primary cause.
+
+### 6. Tool failure investigation
+
+```bash
+moda tool-failures                            # which tools are breaking
+moda tool-failure-detail <tool_name>          # subtype breakdown + examples
+moda tool-failure-detail search --subtype=SEARCH_NO_RESULTS --limit=10
+```
+
+### 7. Workflow runs and outcomes
+
+Every triggered run of an agent (cron invocation, scheduled task, or
+interactive session) is classified `succeeded | partial | failed | loop |
+abandoned`. Use this layer to answer "did the job actually do its work?",
+which is structurally different from "did it error?" — most jobs that
+partially completed never threw.
+
+```bash
+moda runs                                       # last 7d, all outcomes
+moda runs --time-range=24h --outcome=partial    # which runs only half-finished
+moda runs --outcome=loop                        # runs caught in active loops
+moda runs --class-id=cron:helpscout-triage-hourly --limit=20
+moda run <run_id>                               # one execution + its side-effect receipts
+```
+
+`moda run <run_id>` returns the `agent_jobs` row plus every
+`side_effect_receipts` entry for the run — i.e. every tool call the agent
+made (read / write / send / spawn / compute), with success status and
+target system. This is the agent-as-employee log; use it instead of (or
+alongside) `moda context` when debugging "what did the agent actually
+DO".
+
+### 8. Recurring workflows / cron status board
+
+```bash
+moda workflows                                  # health per recurring class
+moda workflows --time-range=7d                  # filter to last 7d of activity
+```
+
+Each row is one job class (a cron slug, idea-widget id, or sourceType)
+with 30d rolling success / partial / failed / loop counts and a
+`health_score` (0–1). Sort by health to find degraded customer-facing
+crons that aren't crashing but aren't completing either.
+
+### 9. Active loop investigation
+
+```bash
+moda loops                                      # everything currently stuck
+moda loops --limit=10
+moda loop <detection_id>                        # one loop signature in detail
+```
+
+Loops are detected in real time by the conversation worker (tier 0/1) and
+the analytics container (tier 2). A row with `cycle_visits=378` means the
+agent re-ran the same action 378 times. Use the `loop_signature_human`
+field to recognize cross-conversation patterns (one tool definition bug
+will produce identical signatures across many users).
+
+## Common workflow recipes
+
+### Daily health check
+
+```bash
+moda overview --days-back=1
+moda frustrations --days-back=1 --limit=5
+moda tool-failures --days-back=1
+```
+
+### Debug a frustrated user
+
+```bash
+moda frustrations --days-back=7 --limit=10
+# Pick a conversation_id from output
+moda context <conversation_id>
+moda context <conversation_id> --msg-index=<turn>  # zoom on the bad turn
+```
+
+### Diagnose a flaky tool
+
+```bash
+moda tool-failures
+moda tool-failure-detail <tool_name>
+# Pick an example conversation_id from the output
+moda context <conversation_id>
+```
+
+### Explore what users are doing
+
+```bash
+moda clusters
+moda clusters --parent-id=<node_id>
+moda cluster-conversations <node_id>
+```
+
+### Find degraded crons before customers do
+
+```bash
+# Health-sorted workflow board
+moda workflows | jq '.workflows | sort_by(.health) | .[] | select(.runs >= 4 and .health < 0.85)'
+
+# What loops are eating tokens right now?
+moda loops --limit=20 | jq '.loops[] | {sig: .loop_signature_human, visits: .cycle_visits, conv: .conversation_id}'
+
+# Drill into a partial run
+moda runs --outcome=partial --limit=5 | jq -r '.runs[].job_id' | head -1 | xargs -I{} moda run {}
+```
+
+### One-liner chains with jq
+
+```bash
+# Pull primary causes of recent frustrations
+moda frustrations --days-back=7 | jq -r '.frustrations[].primary_cause'
+
+# Get conversation IDs for a search and fetch context for each
+for id in $(moda conversations --search="timeout" --limit=3 | jq -r '.conversations[].id'); do
+  moda context "$id"
+done
+
+# Tool failure leaderboard
+moda tool-failures | jq '.tools[] | {tool: .tool_name, failures: .failure_count}'
+```
+
+## Validation — how to know it worked
+
+- Every successful command exits 0 and prints valid JSON to stdout.
+- Errors print to stderr with non-zero exit. Check stderr before claiming
+  success.
+- If the JSON has `"conversations": []` or `"frustrations": []`, the call
+  worked but the time window is empty — widen `--days-back` before
+  concluding "no data".
+
+## Troubleshooting
+
+**`Error: MODA_API_KEY environment variable is required`**
+Run `moda init`, or have the user export `MODA_API_KEY` from
+[modaflows.com/settings](https://modaflows.com/settings).
+
+**`API error (HTTP 401)`**
+Key is invalid or revoked. Re-run `moda init` to get a fresh one.
+
+**`API error (HTTP 404)` on `cluster-conversations`, `context`, or
+`tool-failure-detail`**
+The id/name doesn't exist in this tenant. Verify by listing first:
+`moda clusters`, `moda conversations`, or `moda tool-failures`.
+
+**`Validation error:`**
+A flag value didn't match the schema. Check enums (`time_range` must be
+`1h|24h|3d|7d|30d|90d|all`, not `"7 days"`) and numeric ranges
+(`days_back` is 1–90, `window` max 5).
+
+**Empty results**
+Widen the window: try `--days-back=30` or `--time-range=30d`. Confirm with
+`moda overview` that data exists in the tenant at all.
+
+**`moda: command not found`**
+Either install globally (`npm install -g @moda-ai/cli`) or invoke through
+npx: `npx -p @moda-ai/cli moda <command>`.
+
+## Reference: all commands at a glance
+
+| Command | Purpose |
+|---|---|
+| `moda init` | Browser auth + provision API key + install agent rules |
+| `moda overview` | Dashboard KPIs, top clusters, recent activity |
+| `moda clusters` | Browse topic cluster hierarchy |
+| `moda cluster-conversations <node_id>` | Conversations in a cluster |
+| `moda conversations` | Search/filter conversations |
+| `moda context <conversation_id>` | Windowed message context (max 5 per side) |
+| `moda frustrations` | User frustration detections with evidence |
+| `moda tool-failures` | Tool failure overview |
+| `moda tool-failure-detail <tool_name>` | Per-tool failure breakdown + examples |
+| `moda workflows` | Recurring workflow catalog with 30d health per class |
+| `moda runs` | Recent workflow executions with classified outcomes |
+| `moda run <run_id>` | One execution + its side-effect receipts |
+| `moda loops` | Currently-active loop detections |
+| `moda loop <detection_id>` | One loop signature with blast radius |
+
+Common flags across most commands: `--days-back=N` (1–90), `--limit=N`,
+`--offset=N`, `--time-range=<range>`. Runs commands also accept
+`--outcome=<outcome>`, `--class-id=<workflow_class_id>`, and `--user-id=<id>`.

package/dist/__tests__/api-client.test.d.ts

@@ -1,2 +0,0 @@
-export {};
-//# sourceMappingURL=api-client.test.d.ts.map

package/dist/__tests__/api-client.test.js

@@ -1,73 +0,0 @@
-import { describe, it, expect, vi, beforeEach } from 'vitest';
-// Mock config before importing api-client
-vi.mock('../config.js', () => ({
-    MODA_API_KEY: 'moda_sk_test_key',
-    MODA_BASE_URL: 'https://test.modaflows.com',
-}));
-import { callDataAPI, ApiError } from '../api-client.js';
-describe('callDataAPI', () => {
-    beforeEach(() => {
-        vi.restoreAllMocks();
-    });
-    it('constructs correct URL', async () => {
-        const mockFetch = vi.fn().mockResolvedValue({
-            ok: true,
-            json: () => Promise.resolve({ data: 'test' }),
-        });
-        vi.stubGlobal('fetch', mockFetch);
-        await callDataAPI('/overview?days_back=7');
-        expect(mockFetch).toHaveBeenCalledWith('https://test.modaflows.com/api/v1/data/overview?days_back=7', expect.objectContaining({
-            headers: expect.objectContaining({
-                'x-api-key': 'moda_sk_test_key',
-            }),
-        }));
-    });
-    it('throws ApiError on non-ok response', async () => {
-        vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
-            ok: false,
-            status: 401,
-            text: () => Promise.resolve('Unauthorized'),
-        }));
-        await expect(callDataAPI('/overview')).rejects.toThrow('API error (401): Unauthorized');
-        await vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
-            ok: false,
-            status: 401,
-            text: () => Promise.resolve('Unauthorized'),
-        }));
-        try {
-            await callDataAPI('/overview');
-        }
-        catch (e) {
-            expect(e).toBeInstanceOf(ApiError);
-            expect(e.statusCode).toBe(401);
-        }
-    });
-    it('retries on 500 errors', async () => {
-        const mockFetch = vi.fn()
-            .mockResolvedValueOnce({ ok: false, status: 500, text: () => Promise.resolve('Internal Server Error') })
-            .mockResolvedValueOnce({ ok: true, json: () => Promise.resolve({ data: 'ok' }) });
-        vi.stubGlobal('fetch', mockFetch);
-        const result = await callDataAPI('/overview');
-        expect(result).toEqual({ data: 'ok' });
-        expect(mockFetch).toHaveBeenCalledTimes(2);
-    });
-    it('does not retry on 4xx errors (except 429)', async () => {
-        const mockFetch = vi.fn().mockResolvedValue({
-            ok: false,
-            status: 403,
-            text: () => Promise.resolve('Forbidden'),
-        });
-        vi.stubGlobal('fetch', mockFetch);
-        await expect(callDataAPI('/overview')).rejects.toThrow('API error (403)');
-        expect(mockFetch).toHaveBeenCalledTimes(1);
-    });
-    it('returns parsed JSON on success', async () => {
-        vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
-            ok: true,
-            json: () => Promise.resolve({ conversations: { total: 100 } }),
-        }));
-        const result = await callDataAPI('/overview');
-        expect(result).toEqual({ conversations: { total: 100 } });
-    });
-});
-//# sourceMappingURL=api-client.test.js.map

package/dist/__tests__/cli.test.d.ts

@@ -1,2 +0,0 @@
-export {};
-//# sourceMappingURL=cli.test.d.ts.map

package/dist/__tests__/cli.test.js

@@ -1,109 +0,0 @@
-import { describe, it, expect, vi, beforeEach } from 'vitest';
-// Mock api-client (same pattern as handlers.test.ts)
-vi.mock('../api-client.js', () => ({
-    callDataAPI: vi.fn().mockResolvedValue({ mock: true }),
-}));
-// Mock config to prevent process.exit
-vi.mock('../config.js', () => ({
-    validateConfig: vi.fn(),
-    MODA_API_KEY: 'test-key',
-    MODA_BASE_URL: 'https://test.example.com',
-    PORT: 3003,
-}));
-// Import after mocks
-import { parseArgs, flagsToArgs } from '../cli.js';
-import { callDataAPI } from '../api-client.js';
-const mockCallDataAPI = vi.mocked(callDataAPI);
-describe('parseArgs', () => {
-    it('extracts command from argv', () => {
-        const result = parseArgs(['node', 'cli.js', 'overview']);
-        expect(result.command).toBe('overview');
-        expect(result.positional).toBeUndefined();
-        expect(result.flags).toEqual({});
-    });
-    it('extracts positional argument', () => {
-        const result = parseArgs(['node', 'cli.js', 'cluster-conversations', 'node-123']);
-        expect(result.command).toBe('cluster-conversations');
-        expect(result.positional).toBe('node-123');
-    });
-    it('parses --key=value flags', () => {
-        const result = parseArgs(['node', 'cli.js', 'overview', '--days-back=30']);
-        expect(result.flags).toEqual({ 'days-back': '30' });
-    });
-    it('parses --key value flags', () => {
-        const result = parseArgs(['node', 'cli.js', 'overview', '--days-back', '30']);
-        expect(result.flags).toEqual({ 'days-back': '30' });
-    });
-    it('parses mixed flag formats', () => {
-        const result = parseArgs(['node', 'cli.js', 'conversations', '--search=error', '--limit', '5']);
-        expect(result.flags).toEqual({ search: 'error', limit: '5' });
-    });
-    it('handles boolean flags (no value)', () => {
-        const result = parseArgs(['node', 'cli.js', '--help']);
-        expect(result.command).toBe('--help');
-    });
-    it('returns empty command for no args', () => {
-        const result = parseArgs(['node', 'cli.js']);
-        expect(result.command).toBe('');
-    });
-    it('handles positional before flags', () => {
-        const result = parseArgs(['node', 'cli.js', 'context', 'conv-abc', '--window=3']);
-        expect(result.command).toBe('context');
-        expect(result.positional).toBe('conv-abc');
-        expect(result.flags).toEqual({ window: '3' });
-    });
-});
-describe('flagsToArgs', () => {
-    it('converts kebab-case to snake_case', () => {
-        const result = flagsToArgs({ 'days-back': '30', 'time-range': '7d' });
-        expect(result).toEqual({ days_back: 30, time_range: '7d' });
-    });
-    it('parses numeric values', () => {
-        const result = flagsToArgs({ limit: '10', offset: '0' });
-        expect(result).toEqual({ limit: 10, offset: 0 });
-    });
-    it('keeps string values as strings', () => {
-        const result = flagsToArgs({ search: 'error message' });
-        expect(result).toEqual({ search: 'error message' });
-    });
-    it('includes positional key/value', () => {
-        const result = flagsToArgs({ limit: '5' }, 'node_id', 'node-123');
-        expect(result).toEqual({ node_id: 'node-123', limit: 5 });
-    });
-});
-describe('CLI command dispatch', () => {
-    beforeEach(() => {
-        vi.clearAllMocks();
-    });
-    // We test the commands by dynamically importing and calling runCommand.
-    // Since runCommand is not exported, we test indirectly via the module's behavior.
-    // The key thing to test is that each command constructs the correct API URL.
-    // This is tested at the integration level via handlers.test.ts already.
-    // Here we focus on the arg parsing + flag conversion that is unique to the CLI.
-    it('overview command builds correct args from flags', () => {
-        const args = flagsToArgs({ 'days-back': '14' });
-        expect(args).toEqual({ days_back: 14 });
-    });
-    it('cluster-conversations builds args with positional node_id', () => {
-        const args = flagsToArgs({ limit: '5', offset: '0' }, 'node_id', 'cluster-abc');
-        expect(args).toEqual({ node_id: 'cluster-abc', limit: 5, offset: 0 });
-    });
-    it('conversations builds args with multiple flags', () => {
-        const args = flagsToArgs({ search: 'error', 'user-id': 'user-1', 'time-range': '7d', limit: '10', offset: '0' });
-        expect(args).toEqual({ search: 'error', user_id: 'user-1', time_range: '7d', limit: 10, offset: 0 });
-    });
-    it('context builds args with positional conversation_id', () => {
-        const args = flagsToArgs({ 'msg-index': '5', window: '3' }, 'conversation_id', 'conv-xyz');
-        expect(args).toEqual({ conversation_id: 'conv-xyz', msg_index: 5, window: 3 });
-    });
-    it('tool-failure-detail builds args with positional tool_name', () => {
-        const args = flagsToArgs({ subtype: 'timeout', 'days-back': '14' }, 'tool_name', 'search');
-        expect(args).toEqual({ tool_name: 'search', subtype: 'timeout', days_back: 14 });
-    });
-    it('offset=0 is preserved (not dropped as falsy)', () => {
-        const args = flagsToArgs({ offset: '0' });
-        expect(args.offset).toBe(0);
-        expect(args.offset).not.toBeUndefined();
-    });
-});
-//# sourceMappingURL=cli.test.js.map

package/dist/__tests__/e2e.test.d.ts

@@ -1,2 +0,0 @@
-export {};
-//# sourceMappingURL=e2e.test.d.ts.map