@slack/radar-mcp 1.8.0 → 1.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -5
- package/dist/mcp/index.js +131 -9
- package/dist/mcp/session-backfill.d.ts +26 -0
- package/dist/mcp/session-backfill.js +156 -0
- package/dist/mcp/session-detail.d.ts +36 -0
- package/dist/mcp/session-detail.js +109 -0
- package/dist/mcp/session-query.d.ts +57 -0
- package/dist/mcp/session-query.js +327 -0
- package/dist/mcp/tools.js +22 -0
- package/dist/shared/constants.d.ts +8 -0
- package/dist/shared/constants.js +8 -0
- package/dist/shared/event-spool.d.ts +75 -0
- package/dist/shared/event-spool.js +366 -0
- package/dist/shared/screen.d.ts +48 -0
- package/dist/shared/screen.js +195 -6
- package/dist/shared/stream.d.ts +5 -1
- package/dist/shared/stream.js +39 -0
- package/dist/web/log-session.js +6 -0
- package/dist/web/server.js +15 -1
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -87,13 +87,21 @@ Radar starts **disabled** each session. All tools except `ping` return an error
|
|
|
87
87
|
| `get_rtm_event_detail` | Full RTM event JSON payload for a specific event. |
|
|
88
88
|
| `get_recent_clogs` | Analytics event summaries: name and timestamp. Filterable by event name substring. Persists from app start. |
|
|
89
89
|
| `get_clog_detail` | Full clog JSON payload for a specific event. |
|
|
90
|
+
| `get_recent_traces` | Performance trace/span summaries: trace name, span name, duration. Filterable by name substring. |
|
|
91
|
+
| `get_trace_detail` | Full trace/span detail for a specific trace, including tags. |
|
|
90
92
|
| `wait_for_events` | Block until matching events arrive via SSE stream. Idle-timeout based, not polling. |
|
|
91
|
-
| `search` | Full-text search across network, RTM, and
|
|
93
|
+
| `search` | Full-text search across network, RTM, clog, and trace buffers. |
|
|
92
94
|
| `get_app_state` | Current Activity/Fragment state via `dumpsys`. Shows visible screen, parameters, tab state. |
|
|
93
95
|
| `get_recent_logs` | Recent device logs, optionally filtered by tag (omit `tag` for all lines). Reads the app's own debug log buffer first, falling back to logcat; the result's `source` field (`app_buffer` / `logcat_capture` / `logcat_ring`) names which one served it. For Timber debug logs (`FKDebug`, `SlackRadar`, etc.). |
|
|
96
|
+
| `list_profiles` | List the Android users and profiles on the device (personal, work, and so on). |
|
|
97
|
+
| `activate_for_user` | Activate radar for a specific Android user or profile. |
|
|
98
|
+
| `capture_snapshot` | Bundle the current network, RTM, clog, and trace buffers, plus an optional screenshot, into one saved snapshot for a bug report or repro. |
|
|
94
99
|
| `screenshot` | Capture device screen as PNG. 720px default, `full_res=true` for full resolution. |
|
|
95
100
|
| `record_screen` | Record short screen video (MP4). Default 5s, max 30s. |
|
|
96
|
-
| `
|
|
101
|
+
| `list_databases` | List the app's on-device SQLite databases. Read-only. |
|
|
102
|
+
| `list_tables` | List the tables in a given on-device database, with row counts. Read-only. |
|
|
103
|
+
| `query_database` | Run a read-only query (SELECT, WITH, EXPLAIN, VALUES, PRAGMA) against a copy of an on-device database pulled to the host. |
|
|
104
|
+
| `clear_buffers` | Clear all captured buffers (network, RTM, clog, trace) on the device and locally. |
|
|
97
105
|
| `open_radar_dashboard` | Author or refine a custom live dashboard tab from a spec (source, filters, widgets, alerts). Suggested when a request is not covered by the pre-built tabs; not auto-invoked. Returns the dashboard URL; debug builds only, local only. |
|
|
98
106
|
| `_radar_session_enable` | Enable radar for the session. Called by `/radar-enable` skill. |
|
|
99
107
|
| `_radar_session_disable` | Disable radar for the session. Called by `/radar-disable` skill. |
|
|
@@ -129,9 +137,10 @@ All device-specific logic lives behind the `DeviceTransport` interface (`src/sha
|
|
|
129
137
|
|
|
130
138
|
What gets captured depends on the in-app server implementation. The MCP server is agnostic to the data source. The Android implementation captures:
|
|
131
139
|
|
|
132
|
-
* **Network calls**: OkHttp interceptor. Ring buffer:
|
|
133
|
-
* **RTM events**: WebSocket listener. Ring buffer:
|
|
134
|
-
* **Clogs**: Analytics interceptor. Ring buffer:
|
|
140
|
+
* **Network calls**: OkHttp interceptor. Ring buffer: 400 calls.
|
|
141
|
+
* **RTM events**: WebSocket listener. Ring buffer: 2000 events.
|
|
142
|
+
* **Clogs**: Analytics interceptor. Ring buffer: 1000 events. Persists across radar activation cycles.
|
|
143
|
+
* **Traces**: Performance span observer. Ring buffer: 1000 spans. Persists across radar activation cycles.
|
|
135
144
|
* **SQLite database** (Database tab, web dashboard only): when you open a database, a read-only **plaintext** copy of that on-device store (message text, request/response bodies, user/channel/team IDs) is pulled to a temp directory **on your own machine** (`os.tmpdir()/slack-radar-db-<port>`, files mode 0600). Auto-deleted when the dashboard process stops.
|
|
136
145
|
|
|
137
146
|
The network, RTM, and clog captures are in-memory ring buffers. The Database tab is the exception: it writes a plaintext copy of the selected on-device database to a temp file on your host for the duration of the session. Review before sharing anything from it.
|
package/dist/mcp/index.js
CHANGED
|
@@ -8,11 +8,15 @@ import http from "http";
|
|
|
8
8
|
import path from "path";
|
|
9
9
|
import { fileURLToPath } from "url";
|
|
10
10
|
import { AndroidTransport } from "../shared/android.js";
|
|
11
|
-
import { RADAR_HOST, RADAR_PORT, WEB_PORT_DEFAULT } from "../shared/constants.js";
|
|
11
|
+
import { RADAR_HOST, RADAR_PORT, WEB_PORT_DEFAULT, MAX_RADAR_RESPONSE_BYTES } from "../shared/constants.js";
|
|
12
12
|
import { stopCapture } from "../shared/logcat-capture.js";
|
|
13
|
-
import {
|
|
13
|
+
import { getSpoolFilePath, epochFromSpoolPath, initializeSpoolDir, readSpool, startSpool, stopSpool, writeFrame, registerCleanup as registerSpoolCleanup, } from "../shared/event-spool.js";
|
|
14
|
+
import { connectStream, destroyStream, waitForEvents, clearLocalBuffers, isStreamConnected, localNetworkBuffer, localRtmBuffer, setTransport, setOnReconnect, } from "../shared/stream.js";
|
|
14
15
|
import { buildBufferPath, createSnapshot } from "./snapshot.js";
|
|
15
16
|
import { clampLogLimit, shapeLogsResult } from "./logs-result.js";
|
|
17
|
+
import { querySession, clampByteCap, toNum } from "./session-query.js";
|
|
18
|
+
import { resolveDetailById } from "./session-detail.js";
|
|
19
|
+
import { backfillSpool, BACKFILL_SOURCES, makeSerializedRunner, } from "./session-backfill.js";
|
|
16
20
|
import { handleListDatabases, handleListTables, handleQueryDatabase, } from "./db-tools.js";
|
|
17
21
|
import { cleanupPulledDatabases, initializeDatabasePath, resetDbState, } from "../shared/db.js";
|
|
18
22
|
import { TOOL_DEFINITIONS, SERVER_INSTRUCTIONS, sessionGate } from "./tools.js";
|
|
@@ -200,14 +204,78 @@ async function runMcpServer() {
|
|
|
200
204
|
for (const sig of ["SIGINT", "SIGTERM", "exit"]) {
|
|
201
205
|
process.once(sig, cleanupPulledDatabases);
|
|
202
206
|
}
|
|
207
|
+
// Scope the event spool dir to THIS process by pid, same reason as the DB dir:
|
|
208
|
+
// every MCP shares one RADAR_PORT, so a fixed dir would let two concurrent
|
|
209
|
+
// servers collide on the same events-<now>.jsonl path and gc each other's
|
|
210
|
+
// live files.
|
|
211
|
+
initializeSpoolDir(process.pid);
|
|
212
|
+
// Close the session event spool on process exit (the file is kept for the GC to
|
|
213
|
+
// reclaim). Enable opens it, disable and exit close it; the file survives so a
|
|
214
|
+
// just-ended session stays readable.
|
|
215
|
+
registerSpoolCleanup();
|
|
203
216
|
let sessionEnabled = false;
|
|
217
|
+
// The envelope key each device list route wraps its rows in: /api/network -> {calls}, /api/rtm ->
|
|
218
|
+
// {events}, /api/clogs -> {clogs}, /api/traces -> {traces}. Read once against the same list routes
|
|
219
|
+
// buildApiPath pins. Defined here (the impure layer) since it is the device-response shape, not
|
|
220
|
+
// something the pure backfill orchestrator should know.
|
|
221
|
+
const BACKFILL_ENVELOPE_KEY = {
|
|
222
|
+
network: "calls",
|
|
223
|
+
rtm: "events",
|
|
224
|
+
clog: "clogs",
|
|
225
|
+
trace: "traces",
|
|
226
|
+
};
|
|
227
|
+
// Build the injected seams backfillSpool needs, wiring the real callRadar (through the pinned
|
|
228
|
+
// buildApiPath list route), readSpool, and writeFrame. fetchList pulls one source's device ring
|
|
229
|
+
// list and unwraps its envelope array; a device error or a missing or malformed envelope degrades
|
|
230
|
+
// to an empty list, so a blip on one source simply skips it (backfillSpool swallows the rest).
|
|
231
|
+
// Per-source backfill limit, matching each device ring cap so a full ring is covered (network 400,
|
|
232
|
+
// rtm 2000, clog/trace 1000). These mirror DebugSlackRadarServer's *_BUFFER_CAPACITY. Note the device
|
|
233
|
+
// still caps its response at the 1MB budget, so a very full rtm ring can come back truncated (empty);
|
|
234
|
+
// that is a bounded gap the read-time dedup and the next reconnect backfill tolerate, not a crash.
|
|
235
|
+
const BACKFILL_LIMIT = {
|
|
236
|
+
network: 400,
|
|
237
|
+
rtm: 2000,
|
|
238
|
+
clog: 1000,
|
|
239
|
+
trace: 1000,
|
|
240
|
+
};
|
|
241
|
+
function backfillDeps() {
|
|
242
|
+
return {
|
|
243
|
+
fetchList: async (source) => {
|
|
244
|
+
const tool = BACKFILL_SOURCES.get(source);
|
|
245
|
+
if (tool === undefined)
|
|
246
|
+
return [];
|
|
247
|
+
// Limit per source's ring cap so a full ring is covered; the device still caps its response
|
|
248
|
+
// at the byte budget, so an overfull ring may return fewer (bounded gap, not a failure).
|
|
249
|
+
const apiPath = buildApiPath(tool, { limit: BACKFILL_LIMIT[source] ?? 1000 });
|
|
250
|
+
if (apiPath === null)
|
|
251
|
+
return [];
|
|
252
|
+
const res = await callRadar(apiPath);
|
|
253
|
+
if (res.error)
|
|
254
|
+
return [];
|
|
255
|
+
const key = BACKFILL_ENVELOPE_KEY[source];
|
|
256
|
+
const arr = key ? res[key] : undefined;
|
|
257
|
+
return Array.isArray(arr) ? arr : [];
|
|
258
|
+
},
|
|
259
|
+
readSpool,
|
|
260
|
+
write: writeFrame,
|
|
261
|
+
};
|
|
262
|
+
}
|
|
263
|
+
// One serialized backfill runner for the whole process. Both the enable path and the reconnect hook
|
|
264
|
+
// call THIS, never backfillSpool directly, so two backfills never overlap and race their `have`
|
|
265
|
+
// snapshots into a double-write. A reconnect burst coalesces to one run now plus one fresh run after.
|
|
266
|
+
const runBackfillSerialized = makeSerializedRunner(() => backfillSpool(backfillDeps()).then(() => { }));
|
|
267
|
+
// Backfill on a stream reconnect so the gap between the drop and the re-established stream is
|
|
268
|
+
// filled from the device rings. Set once at startup; the enable path backfills the first connect
|
|
269
|
+
// itself, and noteStreamConnected only fires this on a true reconnect. The runner never throws.
|
|
270
|
+
setOnReconnect(() => {
|
|
271
|
+
void runBackfillSerialized();
|
|
272
|
+
});
|
|
204
273
|
// --- HTTP requests to the on-device radar server ---
|
|
205
|
-
//
|
|
206
|
-
//
|
|
207
|
-
//
|
|
208
|
-
//
|
|
209
|
-
//
|
|
210
|
-
const MAX_RADAR_RESPONSE_BYTES = 1024 * 1024;
|
|
274
|
+
// radarRequest forwards the device body straight into the model context, so an unbounded
|
|
275
|
+
// response could flood it. When the cap trips we stop reading, tear the socket down, and return
|
|
276
|
+
// a truncation marker rather than a partial JSON blob the parser would choke on. Bulk exports go
|
|
277
|
+
// through capture_snapshot, not this path. The ceiling now lives in constants so the device path
|
|
278
|
+
// and the session-query byte cap share one value.
|
|
211
279
|
function radarRequest(requestPath) {
|
|
212
280
|
return new Promise((resolve, reject) => {
|
|
213
281
|
// settled-guard: req.destroy() on the cap path fires "error" (ECONNRESET) AFTER we have
|
|
@@ -325,7 +393,20 @@ async function runMcpServer() {
|
|
|
325
393
|
}, true);
|
|
326
394
|
}
|
|
327
395
|
device.activate(timeout);
|
|
396
|
+
// Open the session event spool, then backfill the device rings into it. Note the live SSE push can
|
|
397
|
+
// start DURING the backfill: the backfill pulls the ring lists via callRadar, which opens the
|
|
398
|
+
// stream on its first fetch, so the trailing connectStream() below is usually a no-op (it only
|
|
399
|
+
// opens the stream when the backfill wrote nothing and never called callRadar, e.g. empty rings).
|
|
400
|
+
// Correctness does NOT rely on connect ordering. It relies on two things: the (source, id) have-set
|
|
401
|
+
// that skips a row already in the spool, and the unconditional read-time collapse (every read folds
|
|
402
|
+
// a duplicated id to one frame), so a frame the live push and the backfill both write surfaces once.
|
|
403
|
+
// Serialized so it cannot overlap a reconnect backfill. Never throws; a device blip just skips it.
|
|
404
|
+
startSpool();
|
|
405
|
+
await runBackfillSerialized();
|
|
328
406
|
connectStream();
|
|
407
|
+
// Open the session event spool so every streamed frame is captured to a host
|
|
408
|
+
// file for the life of this enabled session.
|
|
409
|
+
startSpool();
|
|
329
410
|
await new Promise((r) => setTimeout(r, 500));
|
|
330
411
|
try {
|
|
331
412
|
const status = await radarRequest("/api/ping");
|
|
@@ -355,6 +436,11 @@ async function runMcpServer() {
|
|
|
355
436
|
// deliberate user action saying "I am done", which is surprising and
|
|
356
437
|
// wastes disk + an adb logcat child.
|
|
357
438
|
stopCapture();
|
|
439
|
+
// Explicit disable closes the session event spool but keeps the file so it
|
|
440
|
+
// stays readable until the GC reclaims it. Only explicit disable and process
|
|
441
|
+
// exit close it; a transient device error in callRadar must NOT, or a
|
|
442
|
+
// momentary drop would cut the session record short.
|
|
443
|
+
stopSpool();
|
|
358
444
|
return textResult({
|
|
359
445
|
enabled: false,
|
|
360
446
|
note: "Radar disabled. Use /radar-enable to reconnect.",
|
|
@@ -475,6 +561,39 @@ async function runMcpServer() {
|
|
|
475
561
|
return textResult({ error: `Log query failed: ${e.message}`, lines: [] }, true);
|
|
476
562
|
}
|
|
477
563
|
}
|
|
564
|
+
// Derive the paging epoch from the current spool filename via the shared helper (single source of
|
|
565
|
+
// the events-<ts>.jsonl affix stripping). The <ts> changes on every rotation and re-enable, so it is
|
|
566
|
+
// a stable identity of the CURRENT file; a cursor minted here is detectably stale after a rotation.
|
|
567
|
+
function spoolEpoch() {
|
|
568
|
+
return epochFromSpoolPath(getSpoolFilePath());
|
|
569
|
+
}
|
|
570
|
+
async function handleQuerySession(filter) {
|
|
571
|
+
// Read the whole session spool. No device call on the common path: the spool is a host file.
|
|
572
|
+
// readSpool degrades to [] on a missing or damaged spool.
|
|
573
|
+
const frames = readSpool();
|
|
574
|
+
// Coerce id once here: the SDK does not enforce the input schema, so it can arrive as a digit
|
|
575
|
+
// string. selectById strict-compares against a numeric frame id, so a raw "5" would match nothing
|
|
576
|
+
// for a record that exists. querySession's pure path already coerces internally; the live id path
|
|
577
|
+
// routes here first, so it must coerce too. An unparseable id becomes undefined and falls through
|
|
578
|
+
// to the list read.
|
|
579
|
+
const id = toNum(filter.id);
|
|
580
|
+
// id read: lazy pull-on-open lives in session-detail.ts, with the device fetch and the spool
|
|
581
|
+
// write injected here (the real callRadar + writeFrame). It prefers a stored detail, pulls a
|
|
582
|
+
// body once for a summary-only pull-eligible source, caches it, and degrades to the summary on a
|
|
583
|
+
// pull failure. A single record via the id path is exempt from the newest-first byte budget.
|
|
584
|
+
if (id !== undefined) {
|
|
585
|
+
// Pass the caller's byte cap (hard-bounded) so the id path drops an oversized frame with the
|
|
586
|
+
// same exceeds-cap note the pure id-bypass uses, honoring a caller-lowered cap here too.
|
|
587
|
+
const result = await resolveDetailById(frames, id, filter.source, { pull: (apiPath) => callRadar(apiPath), write: writeFrame }, clampByteCap(filter.byte_cap));
|
|
588
|
+
// Emit COMPACT so emitted bytes equal what the cap guard measured; textResult pretty-print
|
|
589
|
+
// would add whitespace the cap never counted.
|
|
590
|
+
return textResult(JSON.stringify(result));
|
|
591
|
+
}
|
|
592
|
+
// No id: the pure newest-first shaper. querySession is pure, so the epoch (spool-file identity for
|
|
593
|
+
// the cursor) is derived here and passed in. [] turns into an empty result with a note, so this
|
|
594
|
+
// never throws a radar-not-enabled query. Emit COMPACT so emitted bytes match the measured cap.
|
|
595
|
+
return textResult(JSON.stringify(querySession(frames, filter, spoolEpoch())));
|
|
596
|
+
}
|
|
478
597
|
function handleScreenshot(args) {
|
|
479
598
|
try {
|
|
480
599
|
const fullRes = args?.full_res === true;
|
|
@@ -549,7 +668,7 @@ async function runMcpServer() {
|
|
|
549
668
|
}
|
|
550
669
|
}
|
|
551
670
|
// --- MCP Server ---
|
|
552
|
-
const server = new Server({ name: "slack-radar", version: "1.
|
|
671
|
+
const server = new Server({ name: "slack-radar", version: "1.9.1" }, { capabilities: { tools: {} }, instructions: SERVER_INSTRUCTIONS });
|
|
553
672
|
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
554
673
|
tools: TOOL_DEFINITIONS,
|
|
555
674
|
}));
|
|
@@ -618,6 +737,9 @@ async function runMcpServer() {
|
|
|
618
737
|
const { data, isError } = await handleQueryDatabase(device, db ?? "", sql ?? "", refresh === true);
|
|
619
738
|
return textResult(data, isError);
|
|
620
739
|
}
|
|
740
|
+
if (name === "query_session") {
|
|
741
|
+
return await handleQuerySession(toolArgs);
|
|
742
|
+
}
|
|
621
743
|
const apiPath = buildApiPath(name, toolArgs);
|
|
622
744
|
if (apiPath === null) {
|
|
623
745
|
return textResult(`Unknown tool: ${name}`, true);
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
import type { ParsedSSE, RadarEvent } from "../shared/stream.js";
|
|
2
|
+
export declare const BACKFILL_SOURCES: Map<string, string>;
|
|
3
|
+
export interface BackfillDeps {
|
|
4
|
+
fetchList: (source: string) => Promise<RadarEvent[]>;
|
|
5
|
+
readSpool: () => ParsedSSE[];
|
|
6
|
+
write: (frame: ParsedSSE) => void;
|
|
7
|
+
}
|
|
8
|
+
export interface BackfillSummary {
|
|
9
|
+
written: Record<string, number>;
|
|
10
|
+
}
|
|
11
|
+
/**
|
|
12
|
+
* Backfill the device rings into the spool.
|
|
13
|
+
*
|
|
14
|
+
* For each backfillable source it fetches the device ring list, reads what the spool already holds,
|
|
15
|
+
* and writes only the rows whose (base source, id) is NOT already present (as a summary OR as a
|
|
16
|
+
* cached detail). Frames are written in the SAME shape the SSE push produces ({type: source,
|
|
17
|
+
* event: row}) and oldest-first (ascending id) so the spool stays append-ordered like the live push.
|
|
18
|
+
*
|
|
19
|
+
* Never throws: a failed list fetch for one source is swallowed (that source is simply not
|
|
20
|
+
* backfilled this round) so a device blip cannot break enable or the stream. A row with no numeric
|
|
21
|
+
* id is written every round, since it cannot be deduped against the spool.
|
|
22
|
+
*
|
|
23
|
+
* Returns a per-source count of rows written, for logging and tests. Not forwarded to the model.
|
|
24
|
+
*/
|
|
25
|
+
export declare function backfillSpool(deps: BackfillDeps): Promise<BackfillSummary>;
|
|
26
|
+
export declare function makeSerializedRunner(run: () => Promise<void>): () => Promise<void>;
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
import { baseSourceOf } from "./session-query.js";
|
|
2
|
+
// The sources with a device ring and a list endpoint, each paired with the buildApiPath list-tool
|
|
3
|
+
// name so the fetch reuses the pinned device-route table instead of hand-building a path. This is
|
|
4
|
+
// the summary LIST set; keep it separate from session-detail's PULL_ELIGIBLE, which is about
|
|
5
|
+
// pulling DETAIL bodies. All four rings are backfillable here. The frame type (clog, singular)
|
|
6
|
+
// differs from the list tool name (get_recent_clogs); the frame type is the map key.
|
|
7
|
+
export const BACKFILL_SOURCES = new Map([
|
|
8
|
+
["network", "get_recent_network_calls"],
|
|
9
|
+
["rtm", "get_recent_rtm_events"],
|
|
10
|
+
["clog", "get_recent_clogs"],
|
|
11
|
+
["trace", "get_recent_traces"],
|
|
12
|
+
]);
|
|
13
|
+
// A read frame's numeric id, or undefined when it carries none. Only rows with a numeric id can be
|
|
14
|
+
// deduped; a row with no id is written every round (it cannot be matched against the spool).
|
|
15
|
+
function frameId(frame) {
|
|
16
|
+
const event = frame?.event;
|
|
17
|
+
if (typeof event !== "object" || event === null)
|
|
18
|
+
return undefined;
|
|
19
|
+
const raw = event.id;
|
|
20
|
+
return typeof raw === "number" ? raw : undefined;
|
|
21
|
+
}
|
|
22
|
+
// The base source a spooled frame belongs to: a detail tag folds to its base, any other tag is
|
|
23
|
+
// itself. So a network summary and a network_detail share base network, and a cached detail
|
|
24
|
+
// already counts as "have this id". Dedup by base via baseSourceOf, never a source-name switch, so
|
|
25
|
+
// a future source dedups for free.
|
|
26
|
+
function baseSourceOfFrame(frame) {
|
|
27
|
+
return baseSourceOf(frame.type) ?? frame.type;
|
|
28
|
+
}
|
|
29
|
+
// The dedup key for a (source, id) pair. NUL-separated so an id that contains a source substring
|
|
30
|
+
// cannot collide with a different (source, id) pair.
|
|
31
|
+
// Read a row's numeric id, or undefined. The device rows are plain objects of unknown shape.
|
|
32
|
+
function numId(row) {
|
|
33
|
+
if (typeof row !== "object" || row === null)
|
|
34
|
+
return undefined;
|
|
35
|
+
const raw = row.id;
|
|
36
|
+
return typeof raw === "number" ? raw : undefined;
|
|
37
|
+
}
|
|
38
|
+
function dedupKey(source, id) {
|
|
39
|
+
return `${source}\u0000${id}`;
|
|
40
|
+
}
|
|
41
|
+
/**
|
|
42
|
+
* Backfill the device rings into the spool.
|
|
43
|
+
*
|
|
44
|
+
* For each backfillable source it fetches the device ring list, reads what the spool already holds,
|
|
45
|
+
* and writes only the rows whose (base source, id) is NOT already present (as a summary OR as a
|
|
46
|
+
* cached detail). Frames are written in the SAME shape the SSE push produces ({type: source,
|
|
47
|
+
* event: row}) and oldest-first (ascending id) so the spool stays append-ordered like the live push.
|
|
48
|
+
*
|
|
49
|
+
* Never throws: a failed list fetch for one source is swallowed (that source is simply not
|
|
50
|
+
* backfilled this round) so a device blip cannot break enable or the stream. A row with no numeric
|
|
51
|
+
* id is written every round, since it cannot be deduped against the spool.
|
|
52
|
+
*
|
|
53
|
+
* Returns a per-source count of rows written, for logging and tests. Not forwarded to the model.
|
|
54
|
+
*/
|
|
55
|
+
export async function backfillSpool(deps) {
|
|
56
|
+
const written = {};
|
|
57
|
+
// Read the spool once and index every id it already holds by base source. A cached detail counts
|
|
58
|
+
// as having the id, so a later re-enable does not re-write the summary a detail already covers.
|
|
59
|
+
const have = new Set();
|
|
60
|
+
for (const frame of deps.readSpool()) {
|
|
61
|
+
const id = frameId(frame);
|
|
62
|
+
if (id === undefined)
|
|
63
|
+
continue;
|
|
64
|
+
have.add(dedupKey(baseSourceOfFrame(frame), id));
|
|
65
|
+
}
|
|
66
|
+
// Fetch all sources in parallel: each is an independent device pull, so running them concurrently
|
|
67
|
+
// caps the wall time at one fetch, not the sum of four (each callRadar has its own timeout). A
|
|
68
|
+
// per-source rejection is contained by allSettled so one blip never fails the others or enable.
|
|
69
|
+
const sources = [...BACKFILL_SOURCES.keys()];
|
|
70
|
+
const results = await Promise.allSettled(sources.map((source) => deps.fetchList(source)));
|
|
71
|
+
// Write sequentially over the resolved lists so the shared `have` dedup set is updated race-free.
|
|
72
|
+
for (let i = 0; i < sources.length; i++) {
|
|
73
|
+
const source = sources[i];
|
|
74
|
+
written[source] = 0;
|
|
75
|
+
const settled = results[i];
|
|
76
|
+
if (settled.status !== "fulfilled")
|
|
77
|
+
continue; // a failed source fetch just skips that source
|
|
78
|
+
const rows = settled.value;
|
|
79
|
+
if (!Array.isArray(rows))
|
|
80
|
+
continue;
|
|
81
|
+
// Append oldest-first so the spool stays append=ascending-id, which the newest-first read walk
|
|
82
|
+
// and the before_id pager depend on. The device already returns oldest-first, but sort by id
|
|
83
|
+
// ascending here rather than trust the order, matching the web server's defensive sort; a row
|
|
84
|
+
// with no numeric id keeps its relative position. This also makes the fetch order irrelevant.
|
|
85
|
+
const ordered = [...rows]
|
|
86
|
+
.map((row, idx) => ({ row, idx }))
|
|
87
|
+
.sort((a, b) => {
|
|
88
|
+
const ai = numId(a.row);
|
|
89
|
+
const bi = numId(b.row);
|
|
90
|
+
if (ai === undefined || bi === undefined)
|
|
91
|
+
return a.idx - b.idx;
|
|
92
|
+
return ai - bi;
|
|
93
|
+
})
|
|
94
|
+
.map((x) => x.row);
|
|
95
|
+
for (const row of ordered) {
|
|
96
|
+
if (typeof row !== "object" || row === null)
|
|
97
|
+
continue;
|
|
98
|
+
const id = numId(row);
|
|
99
|
+
if (id !== undefined) {
|
|
100
|
+
const key = dedupKey(source, id);
|
|
101
|
+
if (have.has(key))
|
|
102
|
+
continue;
|
|
103
|
+
// Mark it held so a duplicate id within THIS same list is not written twice either.
|
|
104
|
+
have.add(key);
|
|
105
|
+
}
|
|
106
|
+
deps.write({ type: source, event: row });
|
|
107
|
+
written[source] += 1;
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
return { written };
|
|
111
|
+
}
|
|
112
|
+
// Serialize an async run so two triggers never overlap. The wiring layer (index.ts) calls the real
|
|
113
|
+
// backfill from two places: the enable path and the reconnect hook. If a reconnect fires while a run
|
|
114
|
+
// is still awaiting its device fetches, a second run would snapshot the same point-in-time `have` set
|
|
115
|
+
// and could double-write the same id. This wrapper guarantees at most one run at a time.
|
|
116
|
+
//
|
|
117
|
+
// Coalesce-to-one-follow-up: while a run is in flight, the FIRST new trigger schedules a single fresh
|
|
118
|
+
// run to start after the current one settles; further triggers during the same run return that same
|
|
119
|
+
// pending follow-up instead of stacking N runs. So a burst of reconnects yields one run now and one
|
|
120
|
+
// fresh run after, never a run per reconnect. The wrapped run is treated as never-throwing (backfill
|
|
121
|
+
// swallows its own errors); a rejection is contained so it cannot become an unhandled rejection or
|
|
122
|
+
// wedge the chain.
|
|
123
|
+
//
|
|
124
|
+
// Pure and injectable: it wraps an arbitrary async `run`, holds no fs/device/index reference, and
|
|
125
|
+
// keeps its in-flight state in a closure. index.ts owns the single instance; the guard state lives
|
|
126
|
+
// with the wiring layer, the guard logic is testable here.
|
|
127
|
+
export function makeSerializedRunner(run) {
|
|
128
|
+
let inFlight = null;
|
|
129
|
+
let queued = null;
|
|
130
|
+
const start = () => {
|
|
131
|
+
const p = Promise.resolve()
|
|
132
|
+
.then(run)
|
|
133
|
+
.catch(() => {
|
|
134
|
+
// The run owns its errors; contain a stray rejection so the chain never wedges.
|
|
135
|
+
})
|
|
136
|
+
.finally(() => {
|
|
137
|
+
if (inFlight === p)
|
|
138
|
+
inFlight = null;
|
|
139
|
+
});
|
|
140
|
+
inFlight = p;
|
|
141
|
+
return p;
|
|
142
|
+
};
|
|
143
|
+
return () => {
|
|
144
|
+
if (inFlight === null)
|
|
145
|
+
return start();
|
|
146
|
+
if (queued === null) {
|
|
147
|
+
// Schedule exactly one fresh run after the current one settles. Clear the slot the moment it
|
|
148
|
+
// begins so a later trigger can queue the next follow-up.
|
|
149
|
+
queued = inFlight.then(() => {
|
|
150
|
+
queued = null;
|
|
151
|
+
return start();
|
|
152
|
+
});
|
|
153
|
+
}
|
|
154
|
+
return queued;
|
|
155
|
+
};
|
|
156
|
+
}
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
import type { ParsedSSE } from "../shared/stream.js";
|
|
2
|
+
import { type SessionQueryResult } from "./session-query.js";
|
|
3
|
+
export declare const PULL_ELIGIBLE: Map<string, string>;
|
|
4
|
+
export interface PulledBody {
|
|
5
|
+
error?: string;
|
|
6
|
+
_truncated?: boolean;
|
|
7
|
+
[key: string]: unknown;
|
|
8
|
+
}
|
|
9
|
+
export interface DetailDeps {
|
|
10
|
+
pull: (apiPath: string) => Promise<PulledBody>;
|
|
11
|
+
write: (frame: ParsedSSE) => void;
|
|
12
|
+
}
|
|
13
|
+
/**
|
|
14
|
+
* Resolve an id read against the spool with lazy pull-on-open.
|
|
15
|
+
*
|
|
16
|
+
* frames are the spool contents (already read; this layer does not read the file). The flow:
|
|
17
|
+
* - no frame carries the id in scope -> a not-found result with a note
|
|
18
|
+
* - a stored detail frame is present -> return it (served from file, no pull); this is the
|
|
19
|
+
* second-read path and the clog path (the device already pushed clog detail)
|
|
20
|
+
* - only a summary is present and the source is pull-eligible -> pull the body once, write it
|
|
21
|
+
* back, return the detail
|
|
22
|
+
* - the pull is unreachable, or the source is not pull-eligible -> return the summary; a
|
|
23
|
+
* pull-eligible source that could not be fetched carries an offline/evicted note
|
|
24
|
+
* - the pull is reachable but over budget -> return the summary with a size note pointing at
|
|
25
|
+
* capture_snapshot, not a retry
|
|
26
|
+
*
|
|
27
|
+
* Every branch that returns a frame is byte-capped by `cap` (the caller's byte_cap, hard-bounded by
|
|
28
|
+
* clampByteCap). A frame that alone exceeds the cap is dropped for the same exceeds-cap note the
|
|
29
|
+
* pure id-bypass uses, so a large clog served from file or a device-pushed detail cannot flood the
|
|
30
|
+
* model context. `cap` defaults to the shared response budget when the handler does not pass one.
|
|
31
|
+
*
|
|
32
|
+
* Returns the same SessionQueryResult shape the pure shaper returns, so the handler wraps it the
|
|
33
|
+
* same way. A single record via the id path is exempt from the newest-first byte packing (PR2), so
|
|
34
|
+
* this never sets truncated on a found record.
|
|
35
|
+
*/
|
|
36
|
+
export declare function resolveDetailById(frames: ParsedSSE[], id: number, source: string | undefined, deps: DetailDeps, cap?: number): Promise<SessionQueryResult>;
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
// Lazy pull-on-open for an id read of query_session, split out of the index.ts handler (like
|
|
2
|
+
// session-query.ts and api-paths.ts) so the orchestration is unit-testable without importing
|
|
3
|
+
// index.ts (index connects stdio at load, so a test import would hang). The device fetch and the
|
|
4
|
+
// spool write are INJECTED, mirroring db-tools.ts taking a transport: the pull-eligibility set and
|
|
5
|
+
// the readSpool -> select -> pull-if-needed -> writeFrame -> return flow live here, the real
|
|
6
|
+
// callRadar-backed pull and writeFrame are wired in index.ts.
|
|
7
|
+
//
|
|
8
|
+
// Selection stays in the pure shaper (session-query.selectById); this layer owns only the impure
|
|
9
|
+
// step: when the spool holds a summary and no detail for a pull-eligible source, fetch the body
|
|
10
|
+
// once, cache it, and return it. A pull failure degrades to the summary. It never throws.
|
|
11
|
+
import { MAX_RADAR_RESPONSE_BYTES } from "../shared/constants.js";
|
|
12
|
+
import { buildApiPath } from "./api-paths.js";
|
|
13
|
+
import { detailTypeFor, selectById } from "./session-query.js";
|
|
14
|
+
// Sources whose spool frame is a SUMMARY, so an id read must pull the body from the device once and
|
|
15
|
+
// write it back. The value is the device-detail tool name buildApiPath already knows, so the pull
|
|
16
|
+
// reuses the pinned device-route table instead of hand-building a path. clog is intentionally absent:
|
|
17
|
+
// the device pushes clog detail, so a clog spool frame is already the full record and needs no pull.
|
|
18
|
+
// This is the one place a source list is unavoidable; it lives here, not in the pure shaper.
|
|
19
|
+
export const PULL_ELIGIBLE = new Map([
|
|
20
|
+
["network", "get_network_call_detail"],
|
|
21
|
+
["rtm", "get_rtm_event_detail"],
|
|
22
|
+
["trace", "get_trace_detail"],
|
|
23
|
+
]);
|
|
24
|
+
// Pull one record's full body for a pull-eligible source and shape it as a detail frame carrying
|
|
25
|
+
// the requested id. Returns a reason instead of a bare null so the caller can tell the two failure
|
|
26
|
+
// modes apart: `missing` is unreachable (source not pull-eligible, a device blip, or an evicted
|
|
27
|
+
// record); `too_large` is reachable but over budget (a wire-cap trip, or a parsed body past the
|
|
28
|
+
// shared response budget). Never throws. The size backstop stops an oversized body from being
|
|
29
|
+
// persisted and re-served forever.
|
|
30
|
+
async function pullDetailFrame(source, id, deps) {
|
|
31
|
+
const detailTool = PULL_ELIGIBLE.get(source);
|
|
32
|
+
if (detailTool === undefined)
|
|
33
|
+
return "missing";
|
|
34
|
+
const apiPath = buildApiPath(detailTool, { id });
|
|
35
|
+
if (apiPath === null)
|
|
36
|
+
return "missing";
|
|
37
|
+
const res = await deps.pull(apiPath);
|
|
38
|
+
// A wire-cap trip means the body was too big for the device response, not that it is gone.
|
|
39
|
+
if (res._truncated)
|
|
40
|
+
return "too_large";
|
|
41
|
+
// A device blip: the record could not be fetched at all.
|
|
42
|
+
if (res.error)
|
|
43
|
+
return "missing";
|
|
44
|
+
// Backstop: the parsed body must still fit the shared response budget once serialized.
|
|
45
|
+
if (Buffer.byteLength(JSON.stringify(res)) > MAX_RADAR_RESPONSE_BYTES)
|
|
46
|
+
return "too_large";
|
|
47
|
+
// The stored detail MUST carry the requested id so a later id read selects it over the summary.
|
|
48
|
+
return { type: detailTypeFor(source), event: { ...res, id } };
|
|
49
|
+
}
|
|
50
|
+
/**
|
|
51
|
+
* Resolve an id read against the spool with lazy pull-on-open.
|
|
52
|
+
*
|
|
53
|
+
* frames are the spool contents (already read; this layer does not read the file). The flow:
|
|
54
|
+
* - no frame carries the id in scope -> a not-found result with a note
|
|
55
|
+
* - a stored detail frame is present -> return it (served from file, no pull); this is the
|
|
56
|
+
* second-read path and the clog path (the device already pushed clog detail)
|
|
57
|
+
* - only a summary is present and the source is pull-eligible -> pull the body once, write it
|
|
58
|
+
* back, return the detail
|
|
59
|
+
* - the pull is unreachable, or the source is not pull-eligible -> return the summary; a
|
|
60
|
+
* pull-eligible source that could not be fetched carries an offline/evicted note
|
|
61
|
+
* - the pull is reachable but over budget -> return the summary with a size note pointing at
|
|
62
|
+
* capture_snapshot, not a retry
|
|
63
|
+
*
|
|
64
|
+
* Every branch that returns a frame is byte-capped by `cap` (the caller's byte_cap, hard-bounded by
|
|
65
|
+
* clampByteCap). A frame that alone exceeds the cap is dropped for the same exceeds-cap note the
|
|
66
|
+
* pure id-bypass uses, so a large clog served from file or a device-pushed detail cannot flood the
|
|
67
|
+
* model context. `cap` defaults to the shared response budget when the handler does not pass one.
|
|
68
|
+
*
|
|
69
|
+
* Returns the same SessionQueryResult shape the pure shaper returns, so the handler wraps it the
|
|
70
|
+
* same way. A single record via the id path is exempt from the newest-first byte packing (PR2), so
|
|
71
|
+
* this never sets truncated on a found record.
|
|
72
|
+
*/
|
|
73
|
+
export async function resolveDetailById(frames, id, source, deps, cap = MAX_RADAR_RESPONSE_BYTES) {
|
|
74
|
+
const hit = selectById(frames, id, source);
|
|
75
|
+
if (hit === null) {
|
|
76
|
+
return { frames: [], count: 0, truncated: false, note: "no frame with that id in the spool" };
|
|
77
|
+
}
|
|
78
|
+
if (hit.isDetail) {
|
|
79
|
+
return cappedFrameResult(hit.frame, cap);
|
|
80
|
+
}
|
|
81
|
+
const outcome = await pullDetailFrame(hit.frame.type, id, deps);
|
|
82
|
+
if (outcome === "missing") {
|
|
83
|
+
const eligible = PULL_ELIGIBLE.has(hit.frame.type);
|
|
84
|
+
return cappedFrameResult(hit.frame, cap, eligible
|
|
85
|
+
? "the body could not be pulled from the device (offline or evicted); returning the summary"
|
|
86
|
+
: undefined);
|
|
87
|
+
}
|
|
88
|
+
if (outcome === "too_large") {
|
|
89
|
+
return cappedFrameResult(hit.frame, cap, "the full body exceeds the response budget and cannot be inlined; returning the summary. Use capture_snapshot for a bulk export.");
|
|
90
|
+
}
|
|
91
|
+
deps.write(outcome);
|
|
92
|
+
return cappedFrameResult(outcome, cap);
|
|
93
|
+
}
|
|
94
|
+
// Return a found frame as a one-record result, or drop it for the exceeds-cap note when the frame
|
|
95
|
+
// alone is over budget. This is the same guard the pure id-bypass uses (session-query.ts), applied
|
|
96
|
+
// on every id-path branch that returns a frame so an oversized clog or device-pushed detail served
|
|
97
|
+
// from file cannot flood the model context. An optional summary note rides along when the frame
|
|
98
|
+
// still fits; if the frame itself is too big, the exceeds-cap note replaces it.
|
|
99
|
+
function cappedFrameResult(frame, cap, note) {
|
|
100
|
+
if (Buffer.byteLength(JSON.stringify(frame)) > cap) {
|
|
101
|
+
return {
|
|
102
|
+
frames: [],
|
|
103
|
+
count: 0,
|
|
104
|
+
truncated: true,
|
|
105
|
+
note: "the frame with that id alone exceeds the byte cap",
|
|
106
|
+
};
|
|
107
|
+
}
|
|
108
|
+
return { frames: [frame], count: 1, truncated: false, ...(note !== undefined ? { note } : {}) };
|
|
109
|
+
}
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
import type { ParsedSSE } from "../shared/stream.js";
|
|
2
|
+
export interface SessionQueryFilter {
|
|
3
|
+
source?: string;
|
|
4
|
+
id?: number;
|
|
5
|
+
cursor?: string;
|
|
6
|
+
url_filter?: string;
|
|
7
|
+
status_code?: number;
|
|
8
|
+
event_type?: string;
|
|
9
|
+
channel?: string;
|
|
10
|
+
name?: string;
|
|
11
|
+
text?: string;
|
|
12
|
+
limit?: number;
|
|
13
|
+
byte_cap?: number;
|
|
14
|
+
}
|
|
15
|
+
export interface SessionQueryResult {
|
|
16
|
+
frames: ParsedSSE[];
|
|
17
|
+
count: number;
|
|
18
|
+
truncated: boolean;
|
|
19
|
+
cursor?: string;
|
|
20
|
+
note?: string;
|
|
21
|
+
}
|
|
22
|
+
export declare function clampByteCap(byteCap: number | undefined): number;
|
|
23
|
+
export declare function toNum(v: unknown): number | undefined;
|
|
24
|
+
export declare function detailTypeFor(source: string): `${string}_detail`;
|
|
25
|
+
export declare function baseSourceOf(type: string): string | null;
|
|
26
|
+
export interface DetailSelection {
|
|
27
|
+
frame: ParsedSSE;
|
|
28
|
+
isDetail: boolean;
|
|
29
|
+
}
|
|
30
|
+
export declare function selectById(frames: ParsedSSE[], id: number, source?: string): DetailSelection | null;
|
|
31
|
+
/**
|
|
32
|
+
* Shape parsed spool frames into a byte-capped result.
|
|
33
|
+
*
|
|
34
|
+
* frames arrive oldest-first (spool append order). Two paths:
|
|
35
|
+
*
|
|
36
|
+
* 1. id-bypass: when `id` is set (optionally scoped by `source`), return the single whole matching
|
|
37
|
+
* record regardless of age or byte budget. This is the detail-by-id path; a caller asking for a
|
|
38
|
+
* specific record wants that record, not the newest N.
|
|
39
|
+
*
|
|
40
|
+
* 2. newest-first packing: filter, walk newest-first, and accumulate serialized bytes; stop BEFORE
|
|
41
|
+
* a frame would push the running size over the cap. `truncated` is true when the cap or `limit`
|
|
42
|
+
* left older matches unreturned; only then is `cursor` (the "<epoch>:<seq>" token for the oldest
|
|
43
|
+
* frame returned) set, so a caller pages older by passing it back as `cursor`. Returned frames
|
|
44
|
+
* stay newest-first. Paging keys off append position, not device id: each source has its own id
|
|
45
|
+
* namespace so ids overlap and are not globally ordered, but append order is one monotonic key
|
|
46
|
+
* across every source, so a cursor pages the whole session cleanly.
|
|
47
|
+
*
|
|
48
|
+
* `epoch` identifies the spool file the caller read. It is baked into every emitted cursor and
|
|
49
|
+
* checked on every incoming one: a cursor whose epoch does not match came from a rotated or
|
|
50
|
+
* re-enabled spool (whose seq indexes a different file), so it is rejected with a note rather than
|
|
51
|
+
* applied to the wrong file. querySession stays PURE: the caller derives epoch from the spool path
|
|
52
|
+
* and passes it in; this function reads no fs.
|
|
53
|
+
*
|
|
54
|
+
* A caller cannot exceed the hard cap: clampByteCap bounds byte_cap to the shared response budget.
|
|
55
|
+
* An empty frame list returns an empty result with a note so a missing spool reads clearly.
|
|
56
|
+
*/
|
|
57
|
+
export declare function querySession(frames: ParsedSSE[], rawFilter: SessionQueryFilter, epoch: string): SessionQueryResult;
|