@lightdash/cli 2.240.0 → 2.241.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -30,6 +30,7 @@ Users describe features by what they see in the Lightdash editor. Translate:
|
|
|
30
30
|
| "view underlying data", raw rows behind a point (viz only) | `viz-underlying-data` | app code opt-in |
|
|
31
31
|
| Lightdash-owned "View underlying data" dialog (viz only) | `viz-host-underlying-data` | app code opt-in |
|
|
32
32
|
| light/dark mode, "matches my Lightdash theme" | `follow-host-theme` | CSS tokens — see below |
|
|
33
|
+
| "executive summary", "what changed", anomaly callouts, "AI insight", "Analyse this view" | `ai-insights` | app code opt-in |
|
|
33
34
|
|
|
34
35
|
## Automatic (zero wiring — active on any current-SDK bundle)
|
|
35
36
|
|
|
@@ -115,6 +116,12 @@ order, `sortBy` describes result ordering, `totalColumnCount` exposes truncation
|
|
|
115
116
|
- `external-fetch`: `client.externalFetch(alias, opts)` calls an external
|
|
116
117
|
connection linked to this app. The connection must already be linked by the
|
|
117
118
|
host; you cannot add one from app code.
|
|
119
|
+
- `ai-insights`: `useInsights()` returns the host's AI analysis of the current
|
|
120
|
+
view (headline, summary, limitations, anomalies) plus `analyse()`;
|
|
121
|
+
`useInsights(result)` narrows to one chart and adds `matches(row)` so flagged
|
|
122
|
+
points can be marked, with `investigate(id)` / `continueInAskAi(id)` for the
|
|
123
|
+
action menu. Render a summary block and markers; render nothing when
|
|
124
|
+
`status === 'unavailable'`. Full contract in `/app/references/ai-analysis.md`.
|
|
118
125
|
- `delivery-render`: `useDeliveryRender()` is `true` during scheduled-delivery
|
|
119
126
|
and delivery-preview capture renders. Gate tab/slide DATA mounting on it so
|
|
120
127
|
every tab's queries execute during capture while only the active tab stays
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
# AI analysis (`useInsights`)
|
|
2
|
+
|
|
3
|
+
> Read this when the user wants an executive summary, "what changed and why", anomaly callouts, "flag anything unusual", or an "AI insight" block inside the app.
|
|
4
|
+
|
|
5
|
+
Lightdash analyses the queries the app ran for the viewer's current view and pushes the result into the app. The app **renders** that analysis and can trigger it; it never sends its own prompt, never calls a model provider, and never needs an API key. Do not wire an external connection to an LLM for this — the native hook is the supported path and respects the viewer's data permissions.
|
|
6
|
+
|
|
7
|
+
Two operations exist, both run by Lightdash:
|
|
8
|
+
|
|
9
|
+
- **Detect** — reads the results the app already loaded and returns a headline, a summary, a list of notable data points (anomalies) with the query, field and row they refer to, and the limitations of the data (no comparison period, truncation, and so on).
|
|
10
|
+
- **Investigate** — for one anomaly, an AI agent with read-only query tools looks for possible drivers using data beyond the page and returns a Markdown explanation with evidence and a confidence line. It runs asynchronously; the app shows progress from the pushed status.
|
|
11
|
+
|
|
12
|
+
```tsx
|
|
13
|
+
import { useInsights } from '@lightdash/query-sdk';
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
## View-level: the executive summary block
|
|
17
|
+
|
|
18
|
+
`useInsights()` with no argument returns the analysis of the whole view:
|
|
19
|
+
|
|
20
|
+
```ts
|
|
21
|
+
const view = useInsights();
|
|
22
|
+
// view.status: 'idle' | 'analysing' | 'ready' | 'error' | 'unavailable'
|
|
23
|
+
// view.headline one sentence with the main finding (ready only)
|
|
24
|
+
// view.summary two to four sentences an executive can read without the charts
|
|
25
|
+
// view.limitations string[] — what the data could not show
|
|
26
|
+
// view.dataAsOf period the data covers, in words, or null
|
|
27
|
+
// view.generatedAt ISO timestamp, or null
|
|
28
|
+
// view.stale true when the view changed since the analysis ran
|
|
29
|
+
// view.anomalies every notable data point (see below)
|
|
30
|
+
// view.error message when status is 'error'
|
|
31
|
+
// view.analyse() ask Lightdash to (re)run the analysis
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Render a block near the top of the page. It must handle every status:
|
|
35
|
+
|
|
36
|
+
```tsx
|
|
37
|
+
import { useInsights } from '@lightdash/query-sdk';
|
|
38
|
+
import { Button } from '@/components/ui/button';
|
|
39
|
+
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
|
|
40
|
+
import { Loader2, RefreshCw, Sparkles } from 'lucide-react';
|
|
41
|
+
|
|
42
|
+
function ExecutiveSummary() {
|
|
43
|
+
const view = useInsights();
|
|
44
|
+
|
|
45
|
+
// The org has not enabled AI analysis: render nothing, never a stub.
|
|
46
|
+
if (view.status === 'unavailable') return null;
|
|
47
|
+
|
|
48
|
+
return (
|
|
49
|
+
<Card>
|
|
50
|
+
<CardHeader className="flex flex-row items-center justify-between">
|
|
51
|
+
<CardTitle className="flex items-center gap-2">
|
|
52
|
+
<Sparkles className="h-4 w-4" /> Executive summary
|
|
53
|
+
</CardTitle>
|
|
54
|
+
<Button
|
|
55
|
+
variant="outline"
|
|
56
|
+
size="sm"
|
|
57
|
+
onClick={view.analyse}
|
|
58
|
+
disabled={view.status === 'analysing'}
|
|
59
|
+
>
|
|
60
|
+
{view.status === 'analysing' ? (
|
|
61
|
+
<Loader2 className="h-4 w-4 animate-spin" />
|
|
62
|
+
) : (
|
|
63
|
+
<RefreshCw className="h-4 w-4" />
|
|
64
|
+
)}
|
|
65
|
+
{view.status === 'ready' ? 'Refresh' : 'Analyse'}
|
|
66
|
+
</Button>
|
|
67
|
+
</CardHeader>
|
|
68
|
+
<CardContent>
|
|
69
|
+
{view.status === 'idle' && (
|
|
70
|
+
<p className="text-muted-foreground">
|
|
71
|
+
Click Analyse for a summary of what is on this page.
|
|
72
|
+
</p>
|
|
73
|
+
)}
|
|
74
|
+
{view.status === 'analysing' && (
|
|
75
|
+
<p className="text-muted-foreground">Reading the data on this page…</p>
|
|
76
|
+
)}
|
|
77
|
+
{view.status === 'error' && (
|
|
78
|
+
<p className="text-destructive">{view.error}</p>
|
|
79
|
+
)}
|
|
80
|
+
{view.status === 'ready' && (
|
|
81
|
+
<>
|
|
82
|
+
{view.stale && (
|
|
83
|
+
<p className="text-sm text-muted-foreground">
|
|
84
|
+
The view changed since this was generated. Refresh to update.
|
|
85
|
+
</p>
|
|
86
|
+
)}
|
|
87
|
+
<h3 className="text-lg font-semibold">{view.headline}</h3>
|
|
88
|
+
<p>{view.summary}</p>
|
|
89
|
+
{view.limitations.length > 0 && (
|
|
90
|
+
<ul className="text-sm text-muted-foreground">
|
|
91
|
+
{view.limitations.map((l) => (
|
|
92
|
+
<li key={l}>{l}</li>
|
|
93
|
+
))}
|
|
94
|
+
</ul>
|
|
95
|
+
)}
|
|
96
|
+
<p className="text-xs text-muted-foreground">
|
|
97
|
+
AI-generated from the data on this page
|
|
98
|
+
{view.dataAsOf ? ` · Data as of ${view.dataAsOf}` : ''}
|
|
99
|
+
</p>
|
|
100
|
+
</>
|
|
101
|
+
)}
|
|
102
|
+
</CardContent>
|
|
103
|
+
</Card>
|
|
104
|
+
);
|
|
105
|
+
}
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Rules for the block:
|
|
109
|
+
|
|
110
|
+
- Say "AI-generated" in the footer. Never name a model or provider.
|
|
111
|
+
- Show the limitations; they are what keeps the summary honest.
|
|
112
|
+
- Never mount it unconditionally in a modal or overlay; it is page content.
|
|
113
|
+
- `analyse()` re-runs even when a result exists (that is the Refresh case). Do not call it in an effect on mount; the viewer clicks.
|
|
114
|
+
|
|
115
|
+
## Per-chart: anomalies on the data points
|
|
116
|
+
|
|
117
|
+
`useInsights(result)` with a `useLightdash` result narrows to that chart:
|
|
118
|
+
|
|
119
|
+
```ts
|
|
120
|
+
const orders = useLightdash(ordersQuery);
|
|
121
|
+
const insights = useInsights(orders);
|
|
122
|
+
// insights.status same as the view
|
|
123
|
+
// insights.anomalies the anomalies that refer to this query
|
|
124
|
+
// insights.matches(row) the anomalies whose dimension values match this row
|
|
125
|
+
// insights.canInvestigate false when no agent is available
|
|
126
|
+
// insights.investigate(id) start an investigation of one anomaly
|
|
127
|
+
// insights.continueInAskAi(id) open the investigation's thread in Ask AI
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Each anomaly:
|
|
131
|
+
|
|
132
|
+
```ts
|
|
133
|
+
{
|
|
134
|
+
id: string;
|
|
135
|
+
severity: 'high' | 'medium' | 'positive' | 'info';
|
|
136
|
+
text: string; // one or two sentences, names period and scope
|
|
137
|
+
queryUuid: string;
|
|
138
|
+
fieldId: string; // the metric it is about
|
|
139
|
+
dimensionValues: Record<string, string>; // the row it refers to, by dimension field id
|
|
140
|
+
expected: string | null;
|
|
141
|
+
actual: string | null;
|
|
142
|
+
investigation: {
|
|
143
|
+
status: 'idle' | 'running' | 'ready' | 'error';
|
|
144
|
+
explanation: string | null; // Markdown, when ready
|
|
145
|
+
partial: boolean; // true when the query budget ran out
|
|
146
|
+
threadUuid: string | null;
|
|
147
|
+
error: string | null;
|
|
148
|
+
};
|
|
149
|
+
}
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Mark the flagged rows on the chart. `matches(row)` takes the raw result row, so call it per datum:
|
|
153
|
+
|
|
154
|
+
```tsx
|
|
155
|
+
import { Bar, BarChart, Cell } from 'recharts';
|
|
156
|
+
|
|
157
|
+
function OrdersByStatus() {
|
|
158
|
+
const orders = useLightdash(ordersQuery);
|
|
159
|
+
const insights = useInsights(orders);
|
|
160
|
+
|
|
161
|
+
return (
|
|
162
|
+
<BarChart data={orders.data}>
|
|
163
|
+
<Bar dataKey="orders_count">
|
|
164
|
+
{orders.data.map((row) => {
|
|
165
|
+
const flagged = insights.matches(row).length > 0;
|
|
166
|
+
return (
|
|
167
|
+
<Cell
|
|
168
|
+
key={row.orders_status}
|
|
169
|
+
stroke={flagged ? 'var(--destructive)' : undefined}
|
|
170
|
+
strokeWidth={flagged ? 2 : 0}
|
|
171
|
+
/>
|
|
172
|
+
);
|
|
173
|
+
})}
|
|
174
|
+
</Bar>
|
|
175
|
+
</BarChart>
|
|
176
|
+
);
|
|
177
|
+
}
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Keep the marker subtle (an outline, a dot, a badge in the tooltip), and show the anomaly `text` on hover so the viewer learns why the point is marked.
|
|
181
|
+
|
|
182
|
+
## Investigate from the action menu
|
|
183
|
+
|
|
184
|
+
When a clicked data point has a matching anomaly, add **Investigate** to the point's action menu next to "Filter by …" and "View underlying data". Gate it on `insights.canInvestigate` and on the investigation not already running or ready:
|
|
185
|
+
|
|
186
|
+
```tsx
|
|
187
|
+
const anomaly = insights.matches(row)[0];
|
|
188
|
+
|
|
189
|
+
{anomaly && insights.canInvestigate && anomaly.investigation.status === 'idle' && (
|
|
190
|
+
<DropdownMenuItem onSelect={() => insights.investigate(anomaly.id)}>
|
|
191
|
+
Investigate with AI
|
|
192
|
+
</DropdownMenuItem>
|
|
193
|
+
)}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Then render the investigation state wherever the anomaly is shown (a card under the chart, a side sheet):
|
|
197
|
+
|
|
198
|
+
- `running` — a spinner and "Investigating…". It can take a minute.
|
|
199
|
+
- `ready` — render `explanation` as Markdown (it contains a summary, a "Possible drivers" list with the evidence for each, an "Evidence" list, and a "Confidence: …" line). If `partial` is true, say the query budget ran out. Offer a **Continue in Ask AI** button that calls `insights.continueInAskAi(anomaly.id)`; Lightdash opens the thread.
|
|
200
|
+
- `error` — show `error` and offer Investigate again.
|
|
201
|
+
|
|
202
|
+
Never open a dialog for the investigation yourself; the app renders state, Lightdash owns the run.
|
|
203
|
+
|
|
204
|
+
## What the app must not do
|
|
205
|
+
|
|
206
|
+
- Do not send prompts, rows or field values anywhere. The hook exposes results; Lightdash already has the data.
|
|
207
|
+
- Do not add an external connection to a model provider for summaries or explanations. Use this hook.
|
|
208
|
+
- Do not fabricate insights when the status is `idle` or `unavailable`; render the state.
|
|
209
|
+
- Do not render `explanation` as HTML. It is Markdown; render it as text or through a Markdown component.
|
|
210
|
+
- Do not call `analyse()` on mount or on a timer.
|
|
211
|
+
|
|
212
|
+
## When nothing is notable
|
|
213
|
+
|
|
214
|
+
`status === 'ready'` with an empty `anomalies` list is a normal outcome. The headline will say so; render it, do not treat it as an error.
|
|
@@ -689,7 +689,11 @@ const [period, setPeriod] = useUrlState('period', 'last_month');
|
|
|
689
689
|
|
|
690
690
|
### External APIs
|
|
691
691
|
|
|
692
|
-
For any external HTTP API call, read `/app/references/external-apis.md` and use `externalFetch` with a configured connection alias — never raw `fetch`/`axios` (blocked by the sandbox) and never hardcoded credentials.
|
|
692
|
+
For any external HTTP API call, read `/app/references/external-apis.md` and use `externalFetch` with a configured connection alias — never raw `fetch`/`axios` (blocked by the sandbox) and never hardcoded credentials. For AI summaries, explanations or anomaly callouts, do not wire an external connection to a model provider — use the native hook below.
|
|
693
|
+
|
|
694
|
+
### AI analysis — executive summary and anomalies
|
|
695
|
+
|
|
696
|
+
When the user asks for an executive summary, "what changed and why", anomaly callouts, or any AI-generated insight about the data on the page, read `/app/references/ai-analysis.md` and use `useInsights` from the SDK. Lightdash runs the analysis over the queries the app already loaded and pushes the result in; the app renders it (a summary block, markers on flagged points, an Investigate action) and never sends prompts, rows or keys. Render nothing when `status === 'unavailable'`.
|
|
693
697
|
|
|
694
698
|
## Visual Design
|
|
695
699
|
|
|
@@ -1274,6 +1278,8 @@ The action-menu example above shows typical `drillDown()` usage. For the full AP
|
|
|
1274
1278
|
| Metric action menu missing "View underlying data" | Users cannot inspect the rows behind a value unless they explicitly ask for it | Add `getUnderlyingData({ row, metric })` to default metric-value menus when the source row and metric are unambiguous |
|
|
1275
1279
|
| Underlying-data table has no Download button | Users can inspect rows but cannot export them through the backend pipeline | Add a button in the underlying-data table/dialog header that calls `downloadUnderlyingData({ row, metric, fileType, values, limit, filename })` and shows an exporting state |
|
|
1276
1280
|
| Using a formatted display value in `addFilter` | Filter never matches raw rows (e.g. `"$1,234"` vs `1234`) | Pass the raw row value into `addFilter`; only use `format()` for the menu label |
|
|
1281
|
+
| Wiring an LLM external connection for summaries or explanations | Sends the viewer's data to a third party with the author's key, bypasses org consent and permissions, and duplicates a native feature | Use `useInsights()` from the SDK (see `/app/references/ai-analysis.md`); Lightdash runs the analysis |
|
|
1282
|
+
| Calling `useInsights().analyse()` on mount or on a timer | Every viewer pays for a model call on every load | Only call `analyse()` from a viewer's click |
|
|
1277
1283
|
| `<XAxis dataKey="order_date_month" />` with no `tickFormatter` | Recharts renders the raw ISO timestamp (e.g. `2025-03-01T00:00:00Z`) as labels | `tickFormatter={(v) => formatDate(v, getColumn(columns, 'order_date_month'), 'axis')}` from `@/lib/format` |
|
|
1278
1284
|
| Skipping `tickFormatter` on a year axis because "year is just a number" | Year dimensions (`*_year`) are still full ISO timestamps at the data layer, so the axis renders `2025-01-01T00:00:00Z` | Apply the same `formatDate(...)` `tickFormatter` to year axes — `formatDate` will collapse to `2025` for year-grain columns |
|
|
1279
1285
|
| Using `format(row, 'order_date')` for a date column in a table cell | Renders `2025-03-17` (zero-padded tabular) — readable but ugly in dashboards | `formatField(row, col, format, 'cell')` from `@/lib/format` — renders `Mar 17, 2025` while still routing currency/% metrics through the server format |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@lightdash/cli",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.241.1",
|
|
4
4
|
"description": "Lightdash CLI tool",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -22,8 +22,8 @@
|
|
|
22
22
|
"@actions/core": "1.11.1",
|
|
23
23
|
"@aws-sdk/credential-providers": "3.995.0",
|
|
24
24
|
"@casl/ability": "6.8.0",
|
|
25
|
-
"@lightdash/common": "2.
|
|
26
|
-
"@lightdash/warehouses": "2.
|
|
25
|
+
"@lightdash/common": "2.241.1",
|
|
26
|
+
"@lightdash/warehouses": "2.241.1",
|
|
27
27
|
"@sidvind/better-ajv-errors": "7.0.0",
|
|
28
28
|
"@types/columnify": "1.5.1",
|
|
29
29
|
"ajv": "8.18.0",
|