@intflow/sentinelctl 0.4.0 → 0.5.0

package/README.md

@@ -46,39 +46,43 @@ Mistyped commands and options get a suggestion (`Did you mean --range?`), and ev
 ## Fleet overview and metrics
 
 ```bash
-sentinelctl fleet summary                                   # status, attention, busiest devices, log levels, top/rising/new error patterns
-sentinelctl fleet summary -r 1h -o json                     # about 5KB
+sentinelctl fleet summary                                   # status, offline split, attention, busiest devices, trends, log levels, error patterns
+sentinelctl fleet summary -r 1h -o json                     # about 10KB
 sentinelctl metrics top memory -r 7d                        # devices ranked by 7-day average memory + fleet distribution
 sentinelctl metrics top disk --agg last --above 85 --top 50
+sentinelctl metrics top diskDaysLeft --below 14             # disks that fill up within two weeks at the current growth
 sentinelctl metrics top cpu --agg p95 -r 24h --compare --sort change
-sentinelctl metrics top reboots -r 7d --above 0
+sentinelctl metrics top reboots -r 7d --above 0             # many devices reboot daily on schedule; compare with p50
 sentinelctl metrics top networkTx -r 24h --label fleet=edge -o json
 ```
 
-`metrics top` runs one query over every device: `--agg avg|max|min|p95|last` per device over `-r 1h|6h|24h|7d|30d` (or `--from/--to`, up to 30 days), ranked, with min/p50/p90/p95/max across devices. Metrics: cpu, memory, disk, swap, load, temperature, processes, tcpConnections, networkRx, networkTx, reboots. `--compare` adds the previous window of the same length. Device filters work like `devices list`; retired devices are excluded by default.
+`metrics top` runs one query over every device: `--agg avg|max|min|p95|last` per device over `-r 1h|6h|24h|7d|30d` (or `--from/--to`, up to 30 days), ranked, with min/p50/p90/p95/max across devices. Metrics: cpu, memory, disk, swap, load, temperature, processes, tcpConnections, networkRx, networkTx, reboots, diskDaysLeft. `--compare` adds the previous window of the same length. Device filters work like `devices list`; retired devices are excluded by default.
 
-## Logs
+## Logs: count first, read last
 
 ```bash
-sentinelctl logs search timeout -r 1h
-sentinelctl logs search -a <agent-id> -l error -r 24h --all
-sentinelctl logs search -s sshd.service --from 2026-09-30T00:00:00Z --to 2026-09-30T06:00:00Z
-sentinelctl logs search -l error -f                        # follow new errors (Ctrl+C to stop)
-sentinelctl logs search -q boom -o ndjson | jq .message
+sentinelctl logs stats -l error -r 24h --by message --compare        # which patterns, what changed
+sentinelctl logs groups -l error -r 24h                              # every device x source x pattern, with count, first/last and latest line
+sentinelctl logs stats -p "<pattern>" --by agent --timeline -r 7d    # where and since when
+sentinelctl logs search -p "<pattern>" -a <agent-id> -r 24h -n 50    # a few raw lines
 sentinelctl logs context -a <agent-id> -s app.service -t 2026-09-30T01:02:03.456Z
-sentinelctl logs histogram -l error -r 24h
-sentinelctl logs stats -l error -r 1h --by source          # exact counts, one server-side query
-sentinelctl logs stats -l error -r 24h --by agent --top 10
-sentinelctl logs stats -a <agent-id> -r 7d --by message    # message patterns, numbers collapsed to <N>
-sentinelctl logs stats -l error -r 24h --by message --compare --sort change   # what is new or growing
+sentinelctl logs facets -l error -r 24h                              # top devices, sources, severities at a glance
+sentinelctl logs histogram -a <agent-id> -r 30d
 sentinelctl logs sources -r 24h
+sentinelctl logs stats --service payment-api --env prod -l error -r 7d --by message   # one application across devices (needs service/env labels)
+sentinelctl logs stats -l error -r 24h --by service
+sentinelctl logs search -l error --since <nextSince> -o ndjson       # poll only new logs
 ```
 
-`logs stats --compare` also counts the previous window of the same length and adds `previousCount`, `change` and `isNew` per group. Message groups carry `searchText`, ready for `logs search -q`.
+- `logs search` returns at most 10,000 lines (250 per request, `--pages N` up to 40 or `--all`). On a busy fleet that covers minutes, so never download logs to count or summarize them: `logs stats` and `logs groups` count every log in the window on the server.
+- `-p/--pattern` takes a pattern exactly as `logs stats --by message` or `logs groups` print it (numbers as `<N>`) and matches only that pattern. `-q` matches words case-insensitively; add `--case-sensitive` for a much faster exact-case match.
+- `--dedupe` folds lines repeated within the same second; `--collapse` on `logs search` is the same as `logs groups`.
+- Windows: `15m`, `1h`, `6h`, `24h`, `7d`, `30d` or `--from/--to` up to the retention period (31 days by default). Fleet-wide aggregations over 7 days need `-a`, `-s`, `-p` or `--case-sensitive`; the server answers `log_query_too_broad` otherwise.
+- Aggregations over a day are computed per day and finished days are cached, so repeating a long query is cheap; the CLI continues automatically when the server needs several requests.
+- Levels: `emergency`, `critical` and `error` include more severe levels; `warning`, `info` (with notice) and `debug` match that level only.
+- `--fields` (raw fields) is admin-only and audited. `--follow` polls at most every 5 seconds and is meant for humans.
 
-Windows are `15m`, `1h`, `6h`, `24h`, `7d`, or `--from`/`--to` up to 30 days. One page holds 50, 100 or 250 logs; when you ask for several pages (`--pages N`, at most 40, or `--all`) the CLI fetches 250 per request and stops at 10,000 logs. To answer "how many" or "which devices the most", use `logs stats`: it aggregates on the server and returns exact counts instead of downloading logs. `--fields` (raw fields) is admin-only and audited. `--follow` polls at most every 5 seconds.
-
-Each user can run 2 log queries and 2 metric aggregations at a time, web and CLI combined; a third concurrent query gets 429 and GET requests retry automatically with backoff.
+Each user can run 2 log queries and 2 metric aggregations at a time (web and CLI combined) and spend 60 seconds of log query time per minute; GET requests that get 429 retry automatically with backoff.
 
 ## Devices and metadata
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@intflow/sentinelctl",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Sentinel Fleet Console CLI: search device logs, inspect status and metrics, edit metadata and manage roles",
   "type": "module",
   "bin": {

package/src/commands.mjs

@@ -4,7 +4,7 @@ import { ageText, localTime, percentText, renderKeyValues, renderTable, singleLi
 
 export const roleLabels = { viewer: 'viewer (read-only)', editor: 'editor', admin: 'admin' };
 export const outputChoices = ['table', 'json', 'ndjson'];
-const logRanges = ['15m', '1h', '6h', '24h', '7d'];
+const logRanges = ['15m', '1h', '6h', '24h', '7d', '30d'];
 const deviceRanges = ['30m', '1h', '6h', '24h', '7d', '30d'];
 const logLevels = ['emergency', 'critical', 'error', 'warning', 'info', 'debug'];
 const deviceStatuses = ['all', 'online', 'delayed', 'offline', 'unknown', 'maintenance', 'retired'];
@@ -13,15 +13,29 @@ const maxPages = 40;
 const deviceSorts = ['attention', 'status', 'displayName', 'cpu', 'memory', 'disk', 'load', 'temperature', 'ageSeconds', 'role', 'site', 'group', 'incidentCount'];
 
 const logFilterOptions = {
-  query: { alias: 'q', type: 'string', placeholder: 'text', description: 'Case-insensitive text in the message. Positional words are appended.' },
+  query: { alias: 'q', type: 'string', placeholder: 'text', description: 'Words or phrase in the message, case-insensitive by default. Positional words are appended.' },
+  pattern: { alias: 'p', type: 'string', placeholder: 'pattern', description: 'Exact message pattern with numbers as <N>, copied from `logs stats --by message` or `logs groups`. Fast and precise.' },
+  'case-sensitive': { type: 'boolean', description: 'Match -q case-sensitively. Much faster on long windows, but misses other casings.' },
   agent: { alias: 'a', type: 'string', placeholder: 'id', description: 'Exact agent ID (find it with `devices lookup`).' },
-  level: { alias: 'l', type: 'string', choices: logLevels, description: 'This severity and above (error includes emergency/alert/critical).' },
+  level: { alias: 'l', type: 'string', choices: logLevels, description: 'emergency/critical/error include more severe levels; warning, info (with notice) and debug match that level only.' },
   source: { alias: 's', type: 'string', placeholder: 'source', description: 'Exact source: systemd unit, container name, syslog identifier (see `logs sources`).' },
+  service: { type: 'string', placeholder: 'name', description: 'Logical service label (`service` field) across all devices, e.g. payment-api. Logs without the label never match.' },
+  env: { type: 'string', placeholder: 'name', description: 'Environment label (`env` field), e.g. prod.' },
   range: { alias: 'r', type: 'string', choices: logRanges, description: 'Relative window. Default 15m when --from/--to are absent.' },
-  from: { type: 'string', placeholder: 'ISO', description: 'Absolute window start, e.g. 2026-09-30T00:00:00+09:00. Requires --to; at most 30 days.' },
+  from: { type: 'string', placeholder: 'ISO', description: 'Absolute window start, e.g. 2026-09-30T00:00:00+09:00. Requires --to; up to the retention period (31 days by default).' },
   to: { type: 'string', placeholder: 'ISO', description: 'Absolute window end.' },
 };
 
+async function getComplete(context, path, query) {
+  let body = await context.client.get(path, query);
+  for (let round = 0; body.complete === false && round < 20; round += 1) {
+    context.note(`Aggregating step ${body.coverage?.done ?? 0}/${body.coverage?.chunks ?? '?'}, continuing…`);
+    body = await context.client.get(path, query);
+  }
+  if (body.complete === false) context.hint(`Partial result: ${body.coverage?.done}/${body.coverage?.chunks} steps aggregated. Run the same command again to continue (finished days are cached).`);
+  return body;
+}
+
 function logCriteria(options, positionals) {
   const text = [options.query, ...positionals].filter(Boolean).join(' ');
   if ((options.from && !options.to) || (!options.from && options.to)) {
@@ -33,8 +47,10 @@ function logCriteria(options, positionals) {
       throw new UsageError(`Cannot parse --${name}: ${value}`, { hint: 'Use ISO 8601, e.g. 2026-09-30T09:00:00+09:00.' });
     }
   }
+  if (options['case-sensitive'] && !text) throw new UsageError('--case-sensitive needs search text (-q).');
   return {
-    q: text || undefined, agentId: options.agent, level: options.level, source: options.source,
+    q: text || undefined, agentId: options.agent, level: options.level, source: options.source, service: options.service, env: options.env,
+    pattern: options.pattern, match: options['case-sensitive'] ? 'exact' : undefined,
     range: options.from ? undefined : (options.range ?? '15m'),
     from: options.from ? new Date(options.from).toISOString() : undefined,
     to: options.to ? new Date(options.to).toISOString() : undefined,
@@ -94,14 +110,43 @@ function logKey(log) {
 function emptyLogHint(criteria) {
   const tips = [];
   if (criteria.range && criteria.range !== '7d') tips.push('widen the window (-r 24h or -r 7d)');
-  if (criteria.q) tips.push('shorten or drop the search text');
+  if (criteria.q) tips.push(criteria.match === 'exact' ? 'drop --case-sensitive' : 'shorten or drop the search text');
+  if (criteria.pattern) tips.push('check the pattern with `sentinelctl logs stats --by message`');
   if (criteria.source) tips.push('check source names with `sentinelctl logs sources -r 24h`');
   if (criteria.agentId) tips.push('check the agent ID with `sentinelctl devices lookup <name>`');
   if (criteria.level) tips.push('drop -l');
   return tips.length ? `No logs matched. Try: ${tips.join('; ')}.` : 'No logs matched.';
 }
 
+function timestampNanos(value) {
+  const match = /^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2})(?:\.(\d{1,9}))?(Z|[+-]\d{2}:\d{2})$/.exec(String(value));
+  if (!match) return BigInt(Date.parse(value)) * 1_000_000n;
+  return BigInt(Date.parse(`${match[1]}${match[3]}`)) * 1_000_000n + BigInt((match[2] ?? '').padEnd(9, '0'));
+}
+
+function dedupeLogs(logs) {
+  const result = [];
+  for (const log of logs) {
+    const previous = result.at(-1);
+    const sameSecond = previous && String(previous.timestamp).slice(0, 19) === String(log.timestamp).slice(0, 19);
+    const strip = (message) => String(message).replace(/^\[[^\]]+\]\s*/, '');
+    if (sameSecond && previous.agentId === log.agentId && previous.source === log.source && strip(previous.message) === strip(log.message)) {
+      previous.repeat = (previous.repeat ?? 1) + 1;
+      continue;
+    }
+    result.push({ ...log });
+  }
+  return result;
+}
+
 async function logsSearch(context, { options, positionals }) {
+  if (options.collapse) return logsGroups(context, { options, positionals });
+  if (options.since) {
+    if (options.from || options.to || options.range) throw new UsageError('--since cannot be combined with --range/--from/--to.');
+    if (!Number.isFinite(Date.parse(options.since))) throw new UsageError(`Cannot parse --since: ${options.since}`, { hint: 'Pass the nextSince value printed by the previous run.' });
+    options.from = options.since;
+    options.to = new Date().toISOString();
+  }
   const criteria = logCriteria(options, positionals);
   const paging = options.all || options.pages > 1;
   const limit = options.fields ? 50 : (options.limit ?? (paging ? 250 : 100));
@@ -120,8 +165,17 @@ async function logsSearch(context, { options, positionals }) {
     if (!last.hasMore || !last.nextCursor) break;
     cursor = last.nextCursor;
   }
-  emit(context, context.global.output === 'json' ? { criteria: last.criteria, window: last.window, count: logs.length, hasMore: last.hasMore, logs } : logs,
-    () => renderTable(logs, logColumns, { width: context.width, wide: options.wide }));
+  if (options.since) {
+    const since = timestampNanos(options.since);
+    for (let index = logs.length - 1; index >= 0; index -= 1) if (timestampNanos(logs[index].timestamp) <= since) logs.splice(index, 1);
+  }
+  const shown = options.dedupe ? dedupeLogs(logs) : logs;
+  const nextSince = logs[0]?.timestamp ?? options.since ?? null;
+  emit(context, context.global.output === 'json'
+    ? { criteria: last.criteria, window: last.window, count: logs.length, ...(options.dedupe ? { rows: shown.length } : {}), hasMore: last.hasMore, ...(options.since ? { nextSince } : {}), logs: shown }
+    : shown,
+  () => renderTable(shown, options.dedupe ? [...logColumns.slice(0, 4), { header: 'x', value: (log) => (log.repeat ? `×${log.repeat}` : ''), max: 5 }, logColumns[4]] : logColumns, { width: context.width, wide: options.wide }));
+  if (options.since) context.hint(`Next time: --since ${nextSince}`);
   if (!logs.length) context.hint(emptyLogHint(criteria));
   else context.note(`${logs.length} logs · ${localTime(last.window.start)} → ${localTime(last.window.end)}`);
   if (last.hasMore && options.all) context.hint(`Stopped at the ${maxPages * 250}-log cap. Narrow the filters, or use \`sentinelctl logs stats\` for counts.`);
@@ -157,17 +211,39 @@ async function followLogs(context, query, options) {
 }
 
 
-const statsDimensions = ['source', 'agent', 'level', 'message'];
+const statsDimensions = ['source', 'agent', 'level', 'message', 'service'];
+
+const sparkBlocks = ['▁', '▂', '▃', '▄', '▅', '▆', '▇', '█'];
+
+function sparkline(points, window, step) {
+  if (!points?.length || !step) return '';
+  const start = Math.floor(Date.parse(window.start) / 1000 / step) * step;
+  const end = Date.parse(window.end) / 1000;
+  const counts = new Map(points);
+  const values = [];
+  for (let at = start; at < end; at += step) values.push(counts.get(at) ?? 0);
+  const peak = Math.max(1, ...values);
+  return values.map((value) => (value ? sparkBlocks[Math.min(7, Math.floor((value / peak) * 7.999))] : ' ')).join('');
+}
+
+function quoteArgument(text) {
+  return `"${String(text).replaceAll('\\', '\\\\').replaceAll('"', '\\"')}"`;
+}
+
+function narrowingHint(criteria) {
+  const window = criteria.range ? `-r ${criteria.range}` : `--from ${criteria.from} --to ${criteria.to}`;
+  return { window, level: criteria.level ? ` -l ${criteria.level}` : '' };
+}
 
 async function logsStats(context, { options, positionals }) {
   const criteria = logCriteria(options, positionals);
   if (options.sort === 'change' && !options.compare) {
     throw new UsageError('--sort change needs --compare.', { hint: 'Example: sentinelctl logs stats -l error -r 24h --by message --compare --sort change' });
   }
-  const body = await context.client.get('/api/logs/stats', {
-    ...criteria, by: options.by, top: options.top, compare: options.compare ? 1 : undefined, sort: options.sort,
+  const body = await getComplete(context, '/api/logs/stats', {
+    ...criteria, by: options.by, top: options.top, compare: options.compare ? 1 : undefined, sort: options.sort, timeline: options.timeline ? 1 : undefined,
   });
-  const label = { source: 'SOURCE', agent: 'DEVICE', level: 'LEVEL', message: 'MESSAGE PATTERN' }[body.criteria.by];
+  const label = { source: 'SOURCE', agent: 'DEVICE', level: 'LEVEL', message: 'MESSAGE PATTERN', service: 'SERVICE' }[body.criteria.by];
   const signed = (value) => (value === null || value === undefined ? '?' : value > 0 ? `+${value}` : String(value));
   emit(context, context.global.output === 'ndjson' ? body.groups : body, () => renderTable(body.groups, [
     { header: 'COUNT', value: (group) => String(group.count) },
@@ -176,10 +252,13 @@ async function logsStats(context, { options, positionals }) {
       { header: 'CHANGE', value: (group) => (group.isNew ? 'new' : signed(group.change)) },
     ] : [{ header: 'SHARE', value: (group) => (body.total ? `${((group.count / body.total) * 100).toFixed(1)}%` : '-') }]),
     { header: 'DEVICES', value: (group) => String(group.devices) },
+    ...(body.timelineStepSeconds ? [{ header: 'TREND', value: (group) => sparkline(group.timeline, body.window, body.timelineStepSeconds), max: 48 }] : []),
+    ...(body.timelineStepSeconds ? [{ header: 'LAST', value: (group) => (group.last ? localTime(group.last).slice(5, 16) : '-'), max: 11 }] : []),
     ...(body.criteria.by === 'agent' ? [{ header: 'AGENT ID', value: (group) => group.key, max: 36 }] : []),
     { header: label, value: (group) => (body.criteria.by === 'agent' ? group.displayName ?? group.host ?? group.key : group.key) },
   ], { width: context.width, wide: options.wide }));
   context.note(`${body.total} logs from ${body.devices} devices · ${localTime(body.window.start)} → ${localTime(body.window.end)}`);
+  if (body.timelineStepSeconds) context.note(`TREND: one column per ${Math.round(body.timelineStepSeconds / 60)} min; first/last (bucket precision) are in -o json.`);
   if (body.previous) {
     context.note(`Previous window ${localTime(body.previous.window.start)} → ${localTime(body.previous.window.end)}: ${body.previous.total} logs (${signed(body.total - body.previous.total)}) · ${body.newGroupCount} new groups`);
     if (body.falling?.length && body.criteria.sort !== 'change') {
@@ -190,13 +269,45 @@ async function logsStats(context, { options, positionals }) {
   if (body.truncated) context.hint(`Showing the top ${body.groups.length} of ${body.groupCount} groups. Raise --top (max 100) or add filters.`);
   if (body.approximate) context.hint('Message patterns are approximate on this server (collapse_nums is unavailable); narrow the window or filters for exact counts.');
   if (!body.total) context.hint(emptyLogHint(criteria));
-  const sample = body.groups.find((group) => group.searchText);
+  const sample = body.groups.find((group) => group.count > 0);
   if (body.criteria.by === 'message' && sample) {
-    context.hint(`See matching logs: sentinelctl logs search -q "${sample.searchText.replaceAll('"', '\\"')}" -r ${criteria.range ?? '24h'}${criteria.level ? ` -l ${criteria.level}` : ''} (searchText in -o json)`);
+    const { window, level } = narrowingHint(criteria);
+    context.hint(`Next: who and where → sentinelctl logs groups -p ${quoteArgument(sample.key)} ${window}${level}; raw lines → sentinelctl logs search -p ${quoteArgument(sample.key)} ${window} -n 50`);
+  }
+}
+
+const groupColumns = [
+  { header: 'COUNT', value: (group) => String(group.count) },
+  { header: 'LAST', value: (group) => (group.last ? localTime(group.last).slice(5, 16) : '-'), max: 11 },
+  { header: 'DEVICE', value: (group) => group.displayName ?? group.agentId, max: 26 },
+  { header: 'SOURCE', value: (group) => group.source, max: 22 },
+  { header: 'PATTERN', value: (group) => group.pattern },
+];
+
+async function logsGroups(context, { options, positionals }) {
+  const criteria = logCriteria(options, positionals);
+  const body = await getComplete(context, '/api/logs/groups', { ...criteria, top: options.top ?? options.limit });
+  emit(context, context.global.output === 'ndjson' ? body.groups : body, () => renderTable(body.groups, groupColumns, { width: context.width, wide: options.wide }));
+  context.note(`${body.total} logs in ${body.groupCount} groups (${body.patternCount} patterns, ${body.devices} devices) · ${localTime(body.window.start)} → ${localTime(body.window.end)}`);
+  if (body.truncated) context.hint(`Showing ${body.groups.length} of ${body.groupCount} groups, largest first. Raise --top (max 500) or add filters.`);
+  if (body.approximate) context.hint('Patterns are approximate on this server (collapse_nums is unavailable).');
+  if (!body.total) context.hint(emptyLogHint(criteria));
+  const first = body.groups[0];
+  if (first) {
+    const { window } = narrowingHint(criteria);
+    context.hint(`Each group keeps its latest line in -o json (latest.message). Raw lines of one group: sentinelctl logs search -a ${first.agentId} -p ${quoteArgument(first.pattern)} ${window} -n 50`);
   }
 }
 
-const metricNames = ['cpu', 'memory', 'disk', 'swap', 'load', 'temperature', 'processes', 'tcpConnections', 'networkRx', 'networkTx', 'reboots'];
+async function logsFacets(context, { options, positionals }) {
+  const criteria = logCriteria({ ...options, range: options.range ?? (options.from ? undefined : '1h') }, positionals);
+  const body = await context.client.get('/api/logs/facets', { ...criteria, top: options.top });
+  emit(context, body, () => Object.entries(body.facets).map(([field, values]) => `${field}:\n${values
+    .map((item) => `  ${String(item.count).padStart(9)}  ${item.displayName && item.displayName !== item.value ? `${item.displayName} (${item.value})` : item.value}`).join('\n')}`).join('\n'));
+  context.note(`${localTime(body.window.start)} → ${localTime(body.window.end)}`);
+}
+
+const metricNames = ['cpu', 'memory', 'disk', 'swap', 'load', 'temperature', 'processes', 'tcpConnections', 'networkRx', 'networkTx', 'reboots', 'diskDaysLeft'];
 const metricWindows = ['1h', '6h', '24h', '7d', '30d'];
 const deviceSeries = ['cpu', 'memory', 'disk', 'load', 'temperature', 'processes', 'tcpConnections', 'networkRx', 'networkTx', 'swap', 'uptime'];
 
@@ -247,7 +358,7 @@ async function metricsTop(context, { options, positionals }) {
 }
 
 async function fleetSummary(context, { options }) {
-  const body = await context.client.get('/api/fleet/summary', { range: options.range, logs: options.logs === false ? 0 : undefined });
+  const body = await context.client.get('/api/fleet/summary', { range: options.range, logs: options.logs === false ? 0 : undefined, trends: options.trends === false ? 0 : undefined });
   emit(context, body, () => {
     const counts = (record) => Object.entries(record ?? {}).sort((left, right) => right[1] - left[1]).map(([key, value]) => `${key} ${value}`).join(', ');
     const names = (devices) => devices.map((device) => device.displayName).join(', ');
@@ -264,9 +375,23 @@ async function fleetSummary(context, { options }) {
         { header: 'EXAMPLES', value: (entry) => names(entry.devices) },
       ], { width: context.width })
       : '  none');
+    const offline = body.devices.offline;
+    if (offline) {
+      sections.push('', `Offline: ${offline.within24h.length} within 24h${offline.within24h.length ? ` (${offline.within24h.map((device) => `${device.displayName} ${device.offlineHours}h`).join(', ')})` : ''} · ${offline.otherCount} for 1-7 days · ${offline.over7dCount} over 7 days`);
+    }
     sections.push('', 'Highest now (online, active):', renderKeyValues(Object.entries(body.resources).map(([field, devices]) => [
       field, devices.map((device) => `${device.displayName} ${field === 'temperature' ? `${device.value}°C` : `${device.value}%`}`).join(', ') || '-',
     ])));
+    if (body.trends?.error) sections.push('', `Trends: ${body.trends.error}`);
+    else if (body.trends) {
+      const list = (devices, format) => devices.map((device) => `${device.displayName} ${format(device)}`).join(', ') || 'none';
+      sections.push('', 'Trends:', renderKeyValues([
+        ['disk full <14d', list(body.trends.diskFullWithin14Days, (device) => `${device.value}d`)],
+        ['temperature +15°C', list(body.trends.temperatureRise, (device) => `${device.previous}→${device.value}°C`)],
+        [`reboots >${body.trends.rebootOutliers.threshold}/7d`, `${list(body.trends.rebootOutliers.devices, (device) => `${device.value}`)} (typical ${body.trends.rebootOutliers.typicalPerWeek}/7d)`],
+      ]));
+    }
+    if (body.logs?.complete === false) sections.push('', 'Logs: partial (some days still aggregating); run again to complete.');
     if (body.logs?.error) sections.push('', `Logs: ${body.logs.error}`);
     else if (body.logs) {
       const errors = body.logs.errors;
@@ -287,7 +412,7 @@ async function fleetSummary(context, { options }) {
     }
     return sections.join('\n');
   });
-  context.hint('Drill down: devices list --focus attention · logs stats -l error --by message --compare · metrics top memory -r 24h');
+  context.hint('Drill down: logs groups -l error -r 24h · logs stats -l error --by message --compare --timeline · metrics top diskDaysLeft · devices overview <id>');
 }
 
 async function logsContext(context, { options }) {
@@ -305,7 +430,7 @@ async function logsContext(context, { options }) {
 }
 
 async function logsHistogram(context, { options, positionals }) {
-  const body = await context.client.get('/api/logs/histogram', logCriteria(options, positionals));
+  const body = await getComplete(context, '/api/logs/histogram', logCriteria(options, positionals));
   emit(context, context.global.output === 'ndjson' ? body.buckets : body, () => {
     const peak = Math.max(1, ...body.buckets.map((bucket) => bucket.total));
     return renderTable(body.buckets, [
@@ -324,7 +449,7 @@ async function logsSources(context, { options }) {
     { header: 'SOURCE', value: (source) => source.value },
     { header: 'COUNT', value: (source) => String(source.count) },
   ], { width: context.width }));
-  context.note(`Top sources in a sample of the latest ${body.sampleSize} logs.`);
+  context.note(`Top ${body.sources.length} sources by exact count over ${body.range} (${body.total} logs).`);
 }
 
 const deviceColumns = [
@@ -696,8 +821,8 @@ async function rawApi(context, { options, positionals }) {
 const commands = [
   {
     group: 'logs', name: 'search', run: logsSearch, role: 'viewer', mutates: false,
-    summary: 'Search logs by window, device, severity, source and text; or follow live',
-    description: 'Newest first. Default 15m window and 100 logs. When fetching several pages, 250 logs are fetched per request, up to 10,000 in total. To count or rank logs, use `logs stats` instead.',
+    summary: 'Raw log lines by window, device, severity, source, text or exact pattern; --collapse for groups',
+    description: 'Newest first. Default 15m window and 100 logs. When fetching several pages, 250 logs are fetched per request, up to 10,000 in total, which on busy fleets covers only minutes. Use it for a handful of example lines after narrowing with `logs stats`/`logs groups` and -p. --collapse returns every matching group instead of lines (same as `logs groups`).',
     args: [{ name: 'text', optional: true, description: 'Same as -q (words are joined with spaces).' }],
     options: {
       ...logFilterOptions,
@@ -705,6 +830,10 @@ const commands = [
       pages: { type: 'integer', placeholder: 'N', description: `Follow the cursor for up to N pages (1-${maxPages}).` },
       all: { type: 'boolean', description: 'Fetch every page of the window, stopping at 10,000 logs.' },
       fields: { type: 'boolean', description: 'Include raw fields (admin only, audited, 50 logs per page).' },
+      collapse: { type: 'boolean', description: 'Group by device, source and pattern over the whole window with counts, first/last time and the latest line (see logs groups).' },
+      top: { type: 'integer', placeholder: 'N', description: 'With --collapse: groups to return (1-500, default 50).' },
+      dedupe: { type: 'boolean', description: 'Fold lines repeated within the same second on the same device and source into one with a repeat count.' },
+      since: { type: 'string', placeholder: 'ISO', description: 'Only logs after this timestamp (exclusive) up to now; pass the printed nextSince to poll without re-reading.' },
       follow: { alias: 'f', type: 'boolean', description: 'Keep printing new logs (for humans; never ends, agents should not use it).' },
       interval: { type: 'integer', default: 5, placeholder: 'seconds', description: 'Polling interval for --follow (min 5).' },
       wide: { alias: 'w', type: 'boolean', description: 'Do not truncate messages in table output.' },
@@ -713,22 +842,26 @@ const commands = [
     roleNote: '--fields requires admin',
     examples: [
       'sentinelctl logs search -l error -r 1h',
+      'sentinelctl logs search -p "tegra-xusb <N>.usb: not all ports suspended: -<N>" -r 24h -n 50 -o json',
+      'sentinelctl logs search -l error -r 24h --collapse -o json',
+      'sentinelctl logs search -l error --since 2026-09-30T01:02:03.456789Z -o ndjson',
       'sentinelctl logs search "connection reset" -r 24h -o ndjson',
       'sentinelctl logs search -a <agent-id> -s ssh.service -r 6h --all -o json',
       'sentinelctl logs search --from 2026-09-30T09:00:00+09:00 --to 2026-09-30T10:00:00+09:00 -l warning',
     ],
-    output: 'logs[]: timestamp (UTC ISO), severity, priority, agentId, host, displayName, source, message (+fields). -o json wraps them as {criteria, window, count, hasMore, logs}.',
+    output: 'logs[]: timestamp (UTC ISO), severity, priority, agentId, host, displayName, source, message (+fields, +repeat with --dedupe). -o json wraps them as {criteria, window, count, hasMore, nextSince?, logs}.',
   },
   {
     group: 'logs', name: 'stats', run: logsStats, role: 'viewer', mutates: false,
-    summary: 'Count and rank logs by source, device, severity or message pattern (one server-side query)',
-    description: 'Aggregates in VictoriaLogs instead of downloading logs, so counts are exact over the whole window. message groups by pattern with numbers collapsed to <N>. Cached for 60s on the server.',
+    summary: 'Count and rank logs by source, device, severity or message pattern; compare windows; trends',
+    description: 'Aggregates in VictoriaLogs instead of downloading logs, so counts are exact over the whole window. message groups by pattern with numbers collapsed to <N>; pass a pattern to -p for exact follow-ups. Windows over one day are computed per day and finished days are cached, so repeating a long query is cheap; long requests continue automatically.',
     args: [{ name: 'text', optional: true, description: 'Same as -q.' }],
     options: {
       ...logFilterOptions,
-      by: { alias: 'b', type: 'string', choices: statsDimensions, default: 'source', description: 'Group by source, agent (device), level or message pattern.' },
+      by: { alias: 'b', type: 'string', choices: statsDimensions, default: 'source', description: 'Group by source, agent (device), level, message pattern or service label.' },
       top: { type: 'integer', default: 20, placeholder: 'N', description: 'Groups to return (1-100).' },
       compare: { type: 'boolean', description: 'Also count the previous window of the same length: previousCount, change and isNew per group.' },
+      timeline: { type: 'boolean', description: 'Add per-group counts over time (about 48 buckets): when a pattern started, stopped or spiked.' },
       sort: { type: 'string', choices: ['count', 'change'], description: 'Rank by count (default) or by increase over the previous window (needs --compare).' },
       wide: { alias: 'w', type: 'boolean', description: 'Do not truncate long keys in table output.' },
     },
@@ -736,10 +869,40 @@ const commands = [
       'sentinelctl logs stats -l error -r 1h --by source',
       'sentinelctl logs stats -l error -r 24h --by agent --top 10 -o json',
       'sentinelctl logs stats -l error -r 24h --by message --compare --sort change',
+      'sentinelctl logs stats -l error -r 7d --by message --timeline --top 10',
       'sentinelctl logs stats -a <agent-id> -r 7d --by message',
       'sentinelctl logs stats "connection refused" -r 24h --by agent',
     ],
-    output: '{criteria, window, total, devices, groupCount, truncated, approximate, groups[]: {key, count, devices (distinct agents), host?, displayName?, searchText? (message: text to pass to logs search -q), previousCount?, change?, isNew?}, previous?: {window, total, devices, exhaustive}, newGroupCount?, rising?[≤5], falling?[≤5] (largest increases and decreases across all groups, not only the returned top)}.',
+    output: '{criteria, window, total, devices, groupCount, truncated, approximate, complete, coverage{chunks, done}, groups[]: {key, count, devices (distinct agents), first?, last? (with --timeline, bucket precision), host?, displayName?, searchText?, previousCount?, change?, isNew?, timeline?: [[epochSeconds, count]]}, timelineStepSeconds?, previous?: {window, total, devices, exhaustive}, newGroupCount?, rising?[≤5], falling?[≤5] (largest increases and decreases across all groups, not only the returned top)}.',
+  },
+  {
+    group: 'logs', name: 'groups', run: logsGroups, role: 'viewer', mutates: false,
+    summary: 'Every distinct (device, source, message pattern) in the window with counts, first/last time and the latest line',
+    description: 'The lossless way to see what happened: each group is counted over the whole window and keeps its latest raw line, so nothing is skipped the way a 10,000-line download skips. Typically 50-100x smaller than the raw logs. Windows over one day are computed per day and cached.',
+    args: [{ name: 'text', optional: true, description: 'Same as -q.' }],
+    options: {
+      ...logFilterOptions,
+      top: { type: 'integer', default: 50, placeholder: 'N', description: 'Groups to return, largest first (1-500).' },
+      wide: { alias: 'w', type: 'boolean', description: 'Do not truncate patterns in table output.' },
+    },
+    examples: [
+      'sentinelctl logs groups -l error -r 1h',
+      'sentinelctl logs groups -a <agent-id> -r 7d -o json',
+      'sentinelctl logs groups -p "Failed to start <N>.service" -r 24h',
+    ],
+    output: '{criteria, window, complete, coverage, total, devices, groupCount, patternCount, truncated, approximate, groups[]: {agentId, displayName, source, pattern, count, first, last, latest{timestamp, severity, message}}}.',
+  },
+  {
+    group: 'logs', name: 'facets', run: logsFacets, role: 'viewer', mutates: false,
+    summary: 'Top values of device, host, severity, source, agent version and profile in one query',
+    description: 'Orientation for an unfamiliar window: which devices, sources and severities dominate. Fleet-wide up to 24h; up to 7 days with -a, -s or -p.',
+    options: {
+      ...logFilterOptions,
+      range: { alias: 'r', type: 'string', choices: logRanges, description: 'Window (default 1h).' },
+      top: { type: 'integer', default: 10, placeholder: 'N', description: 'Values per field (1-50).' },
+    },
+    examples: ['sentinelctl logs facets -l error -r 24h', 'sentinelctl logs facets -a <agent-id> -r 7d -o json'],
+    output: '{criteria, window, facets{agent_id[{value, count, displayName}], host[], severity[], _SYSTEMD_UNIT[], …}}.',
   },
   {
     group: 'logs', name: 'context', run: logsContext, role: 'viewer', mutates: false,
@@ -761,15 +924,15 @@ const commands = [
     summary: 'Log volume and error counts over time',
     args: [{ name: 'text', optional: true, description: 'Same as -q.' }],
     options: logFilterOptions,
-    examples: ['sentinelctl logs histogram -r 24h', 'sentinelctl logs histogram -a <agent-id> -r 7d -o json'],
-    output: '{start, end, stepSeconds, total, buckets[]: {at (epoch seconds), total, errors}}. Cached for 60s on the server.',
+    examples: ['sentinelctl logs histogram -r 24h', 'sentinelctl logs histogram -a <agent-id> -r 30d -o json'],
+    output: '{start, end, complete, coverage, stepSeconds, total, buckets[]: {at (epoch seconds), total, errors}}. Finished days are cached.',
   },
   {
     group: 'logs', name: 'sources', run: logsSources, role: 'viewer', mutates: false,
-    summary: 'Log sources (systemd units, containers) seen recently',
+    summary: 'Log sources (systemd units, containers) ranked by exact count',
     options: { range: { alias: 'r', type: 'string', choices: logRanges, default: '24h', description: 'Sample window.' } },
     examples: ['sentinelctl logs sources -r 1h'],
-    output: '{range, sampleSize, sources[]: {value, count}}: top 50 in a sample of the latest 250 logs.',
+    output: '{range, total, complete, sources[]: {value, count}}: top 50 by exact count.',
   },
   {
     group: 'devices', name: 'list', run: devicesList, role: 'viewer', mutates: false,
@@ -828,7 +991,7 @@ const commands = [
   {
     group: 'metrics', name: 'top', run: metricsTop, role: 'viewer', mutates: false,
     summary: 'Rank devices by a metric aggregated over a window, with the fleet-wide distribution',
-    description: 'One server-side query over every device: avg/max/min/p95/last of the metric per device, ranked, plus min/p50/p90/p95/max across devices. --compare adds the previous window of the same length. reboots counts uptime resets. Device filters work like devices list (retired devices are excluded by default). Cached for 60s.',
+    description: 'One server-side query over every device: avg/max/min/p95/last of the metric per device, ranked, plus min/p50/p90/p95/max across devices. --compare adds the previous window of the same length. reboots counts uptime resets (many devices reboot daily on schedule; compare with the fleet p50). diskDaysLeft forecasts days until the root disk is full from the growth over the window (only growing disks; soonest first). Device filters work like devices list (retired devices are excluded by default). Cached for 60s.',
     args: [{ name: 'metric', description: metricNames.join(', ') }],
     options: {
       agg: { type: 'string', choices: ['avg', 'max', 'min', 'p95', 'last'], default: 'avg', description: 'Per-device aggregation over the window (ignored for reboots).' },
@@ -836,7 +999,7 @@ const commands = [
       from: { type: 'string', placeholder: 'ISO', description: 'Absolute window start (with --to, at most 30 days).' },
       to: { type: 'string', placeholder: 'ISO', description: 'Absolute window end.' },
       top: { type: 'integer', default: 10, placeholder: 'N', description: 'Devices to return (1-100).' },
-      order: { type: 'string', choices: ['desc', 'asc'], default: 'desc', description: 'desc = highest first, asc = lowest first.' },
+      order: { type: 'string', choices: ['desc', 'asc'], description: 'desc = highest first, asc = lowest first (default desc; asc for diskDaysLeft).' },
       compare: { type: 'boolean', description: 'Also aggregate the previous window: previous and change per device.' },
       sort: { type: 'string', choices: ['value', 'change'], description: 'Rank by value (default) or by change (needs --compare).' },
       above: { type: 'number', placeholder: 'N', description: 'Only devices whose value is above N (e.g. --above 90).' },
@@ -856,20 +1019,22 @@ const commands = [
       'sentinelctl metrics top disk --agg last --above 85 --top 50',
       'sentinelctl metrics top cpu --agg p95 -r 24h --compare --sort change',
       'sentinelctl metrics top reboots -r 7d --above 0',
+      'sentinelctl metrics top diskDaysLeft -r 24h --below 14',
       'sentinelctl metrics top networkTx -r 24h --label fleet=edge -o json',
     ],
     output: '{metric, aggregation, unit, window{start, end, seconds, range}, criteria, scope{devices, withData, withoutData, matched}, distribution{count, min, avg, p50, p90, p95, max}, previousDistribution?, devices[]: {agentId, displayName, host, status, value, previous?, change?}}.',
   },
   {
     group: 'fleet', name: 'summary', run: fleetSummary, role: 'viewer', mutates: false,
-    summary: 'One-call fleet briefing: status counts, attention reasons, busiest devices, log levels and error patterns',
-    description: 'Start here to answer "how is the infrastructure?". Combines the device inventory, attention reasons with example devices, the highest current CPU/memory/disk/temperature, and log statistics for the window: totals by level, top and rising error patterns (compared with the previous window) and the devices with the most errors. About 5KB as JSON.',
+    summary: 'One-call fleet briefing: status, attention, busiest devices, metric trends, log levels and error patterns',
+    description: 'Start here to answer "how is the infrastructure?". Combines the device inventory, attention reasons with example devices, the highest current CPU/memory/disk/temperature, offline devices split by how long, metric trends (disks full within 14 days, temperature rises of 15°C or more, reboot counts well above the fleet norm) and log statistics for the window: totals by level, top, rising, falling and new error patterns (compared with the previous window) and the devices with the most errors. About 10KB as JSON.',
     options: {
       range: { alias: 'r', type: 'string', choices: logRanges, default: '24h', description: 'Log window.' },
       logs: { type: 'boolean', description: 'Include log statistics (default on; --no-logs skips them).' },
+      trends: { type: 'boolean', description: 'Include metric trends: disks full within 14 days, temperature rises, unusual reboot counts (default on; --no-trends skips them).' },
     },
     examples: ['sentinelctl fleet summary', 'sentinelctl fleet summary -r 1h -o json', 'sentinelctl fleet summary --no-logs'],
-    output: '{range, stale, devices{total, status, lifecycle, metrics, logs}, attention[]: {reason, count, devices[≤5]}, resources{cpu, memory, disk, temperature: [{agentId, displayName, value}]}, logs{window, total, devices, levels, errors{total, previousTotal, devices, newPatterns}, topErrorPatterns[], risingErrorPatterns[], fallingErrorPatterns[] (shrunk or gone since the previous window), topErrorDevices[]}}.',
+    output: '{range, stale, devices{total, status, lifecycle, metrics, logs, offline{within24h[], over7dCount, otherCount}}, attention[]: {reason, count, devices[≤5]}, resources{cpu, memory, disk, temperature: [{agentId, displayName, value}]}, trends{diskFullWithin14Days[], temperatureRise[], rebootOutliers{typicalPerWeek, threshold, devices[]}}, logs{complete, window, total, devices, levels, errors{total, previousTotal, devices, newPatterns}, topErrorPatterns[], risingErrorPatterns[], fallingErrorPatterns[] (shrunk or gone since the previous window), topErrorDevices[]}}.',
   },
   {
     group: 'metadata', name: 'get', run: metadataGet, role: 'viewer', mutates: false,
@@ -1007,7 +1172,7 @@ const commands = [
 export const catalog = commands;
 export const groups = {
   fleet: 'One-call briefing of the whole fleet',
-  logs: 'Search, count and rank logs; compare windows; context around a log; volume trends; sources',
+  logs: 'Count, group and compare logs on the server; exact pattern search; context; volume; sources',
   metrics: 'Rank devices by an aggregated metric across the fleet',
   devices: 'Device list, details, lookup, reception overview',
   metadata: 'Device metadata: read, edit (editor), import/export (admin)',

package/src/main.mjs

@@ -121,50 +121,67 @@ function overviewHelp() {
 
 const agentGuide = `Using sentinelctl from AI agents and scripts
 
-1. A human logs in once: \`sentinelctl login\` and approve in the browser. The token lasts 30 days;
-   every later command is non-interactive. The token carries the logged-in user's role, so log in
-   with a viewer account for agents that should only read. Elsewhere, pass SENTINEL_TOKEN and
-   SENTINEL_URL as environment variables.
-2. Use -o json (one document) or -o ndjson (one item per line). stdout carries data only; summaries,
-   hints and errors go to stderr. --quiet drops summaries and hints. With json/ndjson output an error
-   is one stderr line: {"error":{"status","code","message","hint","exitCode"}}.
-3. Exit codes: 0 success · 1 server/network error · 2 usage error (fix the arguments and retry)
-   · 3 not logged in or insufficient role (do not retry). GET requests that hit 429 are retried
-   automatically after Retry-After.
-4. \`sentinelctl commands --json\` returns every command with its options, choices, required role,
-   whether it changes data, examples and output fields; \`commands logs --json\` or
-   \`commands "logs stats" --json\` returns only that part.
-5. Typical investigation (start broad, then narrow)
-   - overview:       fleet summary -o json                      (~5KB: status, attention, busiest devices,
-                                                                 log levels, top/rising/new error patterns)
-   - what changed:   logs stats -l error -r 24h --by message --compare --sort change -o json
-   - fleet metrics:  metrics top memory -r 7d -o json           (ranked devices + fleet distribution)
-                     metrics top disk --agg last --above 85 · metrics top cpu --agg p95 --compare
-                     metrics top reboots -r 7d --above 0
-   - find a device:  devices lookup <part of name>            -> agentId
-   - status:         devices list --focus attention --all -o json --select agentId,displayName,status,attentionReasons
-                     devices overview <id> -o json
-   - metrics:        devices show <id> -r 24h -o json --series cpu,memory   (series are [epochSeconds, value])
-   - logs:           logs search -q "<searchText from logs stats>" -r 24h -o ndjson
-                     logs search -a <id> -l error -r 24h -o ndjson
-                     then logs context -a <id> -s <source> -t <timestamp> for surrounding lines
-   - counts/ranking: logs stats -l error -r 24h --by agent|source|message -o json
-   - trend:          logs histogram -a <id> -r 7d -o json
-6. Efficiency
-   - Prefer aggregates: fleet summary, logs stats and metrics top answer "how many", "which the most"
-     and "what changed" in one server-side query. Do not download thousands of logs or call
-     devices show per device to compute fleet-wide numbers.
-   - Keep JSON small: --select <fields> keeps only the fields you need (devices list --all -o json is
-     ~700KB without it, ~20KB with a few fields); devices show --series none drops time series.
-   - Narrow the window and filters (-a, -s, -l, -q) first. Use --pages N (max 40) to take only what you
-     need; --all stops at 10,000 logs.
-   - Each user can run 2 log queries and 2 metric aggregations at a time (web and CLI combined); run
-     commands one after another.
-   - Do not use --follow (it never ends). Repeat a -r 15m search instead.
-   - The server caches identical log searches for 5s; stats, histograms, sources and metrics top for 60s.
-7. For commands that change data (metadata set/bulk/rollback/import, admin ...), run with --dry-run
-   first when available, check the result, then apply.
-8. Server error messages are in Korean (shared with the web console); the "hint" field is English.`;
+SETUP
+- A human logs in once (\`sentinelctl login\`, approve in the browser; 30 days). Later commands are
+  non-interactive. Elsewhere pass SENTINEL_TOKEN and SENTINEL_URL.
+- Use -o json or -o ndjson. stdout is data only; notes, hints and errors go to stderr (--quiet drops
+  notes and hints). A json/ndjson error is one stderr line {"error":{status,code,message,hint,exitCode}}.
+- Exit codes: 0 ok · 1 server/network (retry later) · 2 usage (fix the arguments) · 3 login/role (do
+  not retry). \`sentinelctl commands <group> --json\` describes options and output fields.
+
+HOW TO INVESTIGATE: broad to narrow, count before you read
+1. Overview          fleet summary -o json
+                     status, attention reasons, busiest devices, trends (disk full soon, temperature
+                     rises, unusual reboots), error levels, top/rising/falling/new error patterns.
+2. What and where    logs stats -l error -r 24h --by message --compare -o json      (patterns, change)
+                     logs groups -l error -r 24h -o json        (every device x source x pattern with
+                                                                 count, first/last time, latest line)
+3. When              logs stats -p "<pattern>" --by agent --timeline -r 7d -o json
+                     logs histogram -a <id> -r 7d -o json
+4. Read a few lines  logs search -p "<pattern>" -a <id> -r 24h -n 50 -o json
+                     logs context -a <id> -s <source> -t <timestamp>   (lines around one log)
+5. Device state      devices overview <id> -r 7d -o json · devices show <id> --series cpu,memory -o json
+6. Fleet metrics     metrics top memory -r 7d · metrics top disk --agg last --above 85
+                     metrics top diskDaysLeft --below 14 · metrics top cpu --agg p95 --compare
+                     metrics top reboots -r 7d (most devices reboot daily; compare with p50)
+
+LOG SEARCH RULES (complete answers at low cost)
+- Never download logs to count, rank or summarize them. logs search returns at most 10,000 lines,
+  which on this fleet covers minutes, not hours: the rest of the window is silently missing.
+  logs stats and logs groups count every log in the window on the server.
+- Prefer logs groups over logs search: errors in one hour are typically ~10,000 lines but ~150
+  groups, each with its exact count and latest line.
+- Copy patterns exactly. Patterns from logs stats --by message / logs groups (numbers shown as <N>)
+  go to -p/--pattern, which matches that pattern exactly and quickly. -q matches words
+  case-insensitively and can also hit unrelated messages.
+- Narrow before widening: device (-a), source (-s), service (--service), severity (-l), pattern (-p).
+  Start with 1h-24h; widen to 7d or 30d only when you need history.
+- Fleet-wide windows over 7 days need a narrowing filter (-a, -s, --service, -p or -q with
+  --case-sensitive); otherwise the server answers 422 log_query_too_broad.
+  Single-device, single-service and single-pattern queries can use the whole retention period
+  (31 days by default).
+- Windows over one day are aggregated per day and finished days are cached: repeating a long query
+  is cheap, and the CLI continues automatically if the server needs several requests. A partial
+  result says so in complete/coverage.
+- To watch for new logs, poll with --since <nextSince from the previous run>, not --follow.
+- --dedupe folds lines repeated within the same second (the agent writes some messages twice).
+- Levels: emergency/critical/error include more severe levels; warning, info and debug are exact.
+- Applications are found by source (-s: systemd unit, container) on one device, or by the service
+  label (--service, --env) across devices when the collector sets it; logs stats --by service ranks them.
+
+LIMITS (shared with the web console, per user)
+- 2 log queries and 2 metric aggregations at a time; 60 seconds of log query time per minute.
+  429 responses are retried automatically with backoff; run commands one after another.
+- Retrying a 422 or 400 without changing the arguments will fail again; read the hint.
+
+KEEP OUTPUT SMALL
+- --select a,b.c keeps only those fields (devices list --all -o json: ~700KB -> ~20KB).
+- devices show --series none drops time series; logs groups/stats return compact rows.
+
+CHANGES
+- metadata set/bulk/rollback/import and admin commands change data: run with --dry-run first when
+  available, check the result, then apply. Viewer tokens cannot change anything.
+- Server error messages are Korean (shared with the web console); hints are English.`;
 
 function commandsCatalog(selected = null) {
   return {
@@ -282,6 +299,15 @@ function apiHint(error, context, command) {
       ? 'Check the agent ID with `sentinelctl devices lookup <part of the name>`.' : 'Not found; check the ID.';
   }
   if (error.status === 409) return 'Someone saved a change first. Run the command again to apply on top of the latest revision.';
+  if (error.code === 'log_query_too_broad') {
+    return 'Narrow it: add -a <agent-id>, -s <source>, --service <name>, -p "<pattern>" (from `logs stats --by message`) or --case-sensitive with -q, or use a window of 7 days or less.';
+  }
+  if (error.code === 'log_budget_exceeded') {
+    return `Your log query time for this minute is used up${error.retryAfterSeconds ? `; wait ${error.retryAfterSeconds}s` : ''}. Prefer logs stats/groups with filters over repeated wide searches.`;
+  }
+  if (error.status === 504) {
+    return 'The log/metric backend timed out. Narrow the window or add -a/-s/-l, use -p "<pattern>" instead of -q, or add --case-sensitive. Long aggregations resume from cache if you rerun.';
+  }
   if (error.status === 429) return `Too many requests${error.retryAfterSeconds ? `; wait ${error.retryAfterSeconds}s` : ''} and retry. Narrower filters reduce load.`;
   if ([400, 413, 415, 422].includes(error.status)) {
     return command ? `Check the input: sentinelctl ${commandPath(command)} --help` : 'Check the input.';