browser-flow-tracker 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -77,8 +77,11 @@ Here's a taste of what ends up in your `.md` file:
77
77
  …plus the full request and response details for each call. 🎉
78
78
 
79
79
  It's smart, too: it **throws away the boring noise** (images, fonts, styling,
80
- tracking/analytics pixels) and keeps only the meaningful stuff. And it **hides your
81
- passwords and login tokens** by default so you can share the doc safely.
80
+ tracking/analytics pixels) and keeps only the meaningful stuff. It **hides your
81
+ secrets** by default auth headers and cookies, *plus* password/token-style fields
82
+ inside request & response bodies and URLs — so you can share the doc safely. And if
83
+ your flow **opens a popup or a new tab** (an OAuth login, a payment window), that
84
+ tab's API calls are captured too, in the same recording.
82
85
 
83
86
  ---
84
87
 
@@ -229,10 +232,39 @@ node bin/bft.js record --launch --browser brave \
229
232
 
230
233
  ## 🤖 Connecting to Claude / Cursor
231
234
 
232
- You give Claude/Cursor a tiny **settings snippet** that tells them how to run the tool.
233
- There's nothing to download — it runs straight from npm with `npx`.
235
+ ### Recommended: one-line install (no Node, works on macOS, Windows & Linux)
234
236
 
235
- ### The one snippet you need
237
+ The most reliable way — **no Node.js required** — and it avoids the `spawn npx ENOENT`
238
+ PATH problems that trip up GUI apps. It installs a standalone binary and prints the exact
239
+ MCP config to paste.
240
+
241
+ **macOS & Linux** — in Terminal:
242
+
243
+ ```bash
244
+ curl -fsSL https://apiflowtracker.com/install.sh | sh
245
+ ```
246
+ Then your MCP config command is (same path on every Mac/Linux machine):
247
+ ```json
248
+ { "mcpServers": { "browser-flow-tracker": { "command": "/usr/local/bin/browser-flow-tracker" } } }
249
+ ```
250
+
251
+ **Windows** — in PowerShell:
252
+
253
+ ```powershell
254
+ irm https://apiflowtracker.com/install.ps1 | iex
255
+ ```
256
+ Then your MCP config command is the path the installer prints, e.g.:
257
+ ```json
258
+ { "mcpServers": { "browser-flow-tracker": { "command": "C:\\Users\\you\\AppData\\Local\\browser-flow-tracker\\browser-flow-tracker.exe" } } }
259
+ ```
260
+
261
+ > The installer downloads the right build for your CPU automatically (Apple Silicon or
262
+ > Intel Mac, x64 or arm64 Linux, x64 Windows). Restart your AI app after adding the config.
263
+
264
+ ### Alternative: via npx (only if you already have Node.js)
265
+
266
+ If Node 18+ is installed *and* on your app's PATH, you can skip the install and run it
267
+ straight from npm:
236
268
 
237
269
  ```json
238
270
  {
@@ -245,30 +277,21 @@ There's nothing to download — it runs straight from npm with `npx`.
245
277
  }
246
278
  ```
247
279
 
248
- That's it `npx` fetches and runs the tool automatically. It uses the browsers already
249
- installed on your computer; nothing extra to install.
250
-
251
- ### Where to put it
252
-
253
- - **Claude Code (easiest):** run this one command and it's added everywhere:
280
+ Claude Code shortcut: `claude mcp add browser-flow-tracker -- npx -y browser-flow-tracker@latest`
254
281
 
255
- ```bash
256
- claude mcp add browser-flow-tracker -- npx -y browser-flow-tracker@latest
257
- ```
282
+ > If you hit `spawn npx ENOENT`, your app can't see `npx` on its PATH — use the one-line
283
+ > installer above instead; it sidesteps the whole issue.
258
284
 
259
- (Or paste the snippet into a project's `.mcp.json`.)
285
+ ### Where the config file lives
260
286
 
261
- - **Cursor:** open your MCP settings file and add the `browser-flow-tracker` block, then
262
- restart Cursor and enable it in **Settings → MCP**. Your config lives at
263
- `.cursor/mcp.json` (per project) or `~/.cursor/mcp.json` (global, all projects).
287
+ - **Claude Code:** `.mcp.json` in a project, or your user config (managed by `claude mcp`).
288
+ - **Cursor:** `.cursor/mcp.json` (per project) or `~/.cursor/mcp.json` (global).
264
289
 
265
290
  > ⚠️ **Already using other MCP tools?** Don't replace your file — **add** the
266
291
  > `browser-flow-tracker` block *inside* your existing `mcpServers`, with a **comma** after
267
- > your previous tool. Valid JSON = matching `{ }` and commas between entries but not after
268
- > the last one. The `claude mcp add` command above does this merge for you automatically.
292
+ > your previous tool. Keep it valid JSON (commas between entries, none after the last).
269
293
 
270
- Ready-made copies of the snippet are in this repo as `.mcp.json.example` and
271
- `.cursor/mcp.json.example`.
294
+ Ready-made copies are in this repo as `.mcp.json.example` and `.cursor/mcp.json.example`.
272
295
 
273
296
  ### Prefer to run from source? (for the CLI or development)
274
297
 
@@ -279,6 +302,7 @@ want to hack on the code:
279
302
  git clone https://github.com/devggaurav/web-Api-scrapper.git
280
303
  cd web-Api-scrapper
281
304
  npm install
305
+ npm test # unit tests (redaction, filtering, exporters)
282
306
  # MCP: point your config's command at "node" with args ["<full-path>/mcp/server.js"]
283
307
  # CLI: node bin/bft.js record --launch --browser brave --url https://example.com
284
308
  ```
@@ -365,8 +389,19 @@ Not yet 😔 — Safari and Firefox work differently under the hood. For now use
365
389
  Chrome, Arc, or Edge. (Safari support is on the roadmap.)
366
390
 
367
391
  **"Is it safe to share the document?"**
368
- By default, **yes** passwords, cookies, and login tokens are automatically hidden.
369
- Only if you used `--no-redact` should you be careful.
392
+ By default, **yes**. The tool redacts secrets in all three places they show up:
393
+ auth/cookie headers, token-style URL parameters (`?token=…`, `?api_key=…`), and
394
+ password/token-style fields inside JSON & form request/response bodies
395
+ (`password`, `access_token`, `client_secret`, `otp`, card numbers, …). Field names
396
+ it doesn't recognize as secrets are kept — so if an app uses an unusual name for a
397
+ secret, skim the doc once before sharing. Only if you used `--no-redact` should you
398
+ be careful.
399
+
400
+ **"My flow opens a popup / new tab — is that captured?"**
401
+ Yes. OAuth windows, payment popups, and "open in new tab" links are attached to
402
+ automatically (they're even paused for a split second at birth so their very first
403
+ request isn't missed). Their calls appear in the same document, marked with the tab
404
+ they came from.
370
405
 
371
406
  ---
372
407
 
@@ -383,7 +418,6 @@ can actually read. That's it. 🎩
383
418
  ## 🗺️ Roadmap (coming later)
384
419
 
385
420
  - 🧭 Safari & Firefox support (via a different listening method)
386
- - 🪟 Watch multiple browser tabs at once
387
421
  - 🔀 "Diff" mode — compare two recordings to see what changed
388
422
  - 🧩 Auto-group calls into logical steps in the doc
389
423
 
package/mcp/server.js CHANGED
@@ -12,6 +12,7 @@ import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'
12
12
  import { z } from 'zod';
13
13
  import { TrackingSession } from '../src/session.js';
14
14
  import { detectInstalled } from '../src/browsers.js';
15
+ import { VERSION } from '../src/version.js';
15
16
 
16
17
  let current = null; // the single active TrackingSession
17
18
 
@@ -23,7 +24,7 @@ function err(message) {
23
24
  }
24
25
 
25
26
  const server = new McpServer(
26
- { name: 'browser-flow-tracker', version: '0.1.0' },
27
+ { name: 'browser-flow-tracker', version: VERSION },
27
28
  {
28
29
  instructions: [
29
30
  'Use these tools whenever the user wants to record, analyze, or document the',
@@ -124,7 +125,17 @@ server.registerTool(
124
125
  }
125
126
  if (!current?.active) return err('No active tracking session. Call start_tracking first.');
126
127
  const snap = current.snapshot();
127
- if (full) return json(snap);
128
+ if (full) {
129
+ // Cap bodies so a long recording can't blow past MCP result limits —
130
+ // the complete detail always lands in the .flow.json on stop.
131
+ const MAX_INLINE_BODY = 4000;
132
+ const clip = (b) => {
133
+ const s = typeof b === 'string' ? b : b == null ? b : JSON.stringify(b);
134
+ return s && s.length > MAX_INLINE_BODY ? s.slice(0, MAX_INLINE_BODY) + '… [truncated]' : b;
135
+ };
136
+ snap.flow = snap.flow.map((e) => ({ ...e, requestBody: clip(e.requestBody), responseBody: clip(e.responseBody) }));
137
+ return json(snap);
138
+ }
128
139
  const summary = snap.flow
129
140
  .filter((e) => e.category !== 'document')
130
141
  .map((e) => ({ i: e.index, method: e.method, url: `${e.host}${e.path}`, status: e.status, ms: e.durationMs }));
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "browser-flow-tracker",
3
- "version": "0.1.0",
3
+ "version": "0.2.0",
4
4
  "description": "Record and document the API flow of any Chromium browser page (Brave, Chrome, Arc, Edge) via the Chrome DevTools Protocol. Ships as a local MCP server for Claude/Cursor and a CLI. Emits structured JSON + HAR + a Markdown flow-doc.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -16,7 +16,14 @@
16
16
  ],
17
17
  "scripts": {
18
18
  "start": "node bin/bft.js",
19
- "mcp": "node mcp/server.js"
19
+ "mcp": "node mcp/server.js",
20
+ "test": "node --test",
21
+ "build:darwin-arm64": "bun build ./mcp/server.js --compile --target=bun-darwin-arm64 --outfile dist/browser-flow-tracker-darwin-arm64",
22
+ "build:darwin-x64": "bun build ./mcp/server.js --compile --target=bun-darwin-x64 --outfile dist/browser-flow-tracker-darwin-x64",
23
+ "build:linux-x64": "bun build ./mcp/server.js --compile --target=bun-linux-x64 --outfile dist/browser-flow-tracker-linux-x64",
24
+ "build:linux-arm64": "bun build ./mcp/server.js --compile --target=bun-linux-arm64 --outfile dist/browser-flow-tracker-linux-arm64",
25
+ "build:windows-x64": "bun build ./mcp/server.js --compile --target=bun-windows-x64 --outfile dist/browser-flow-tracker-windows-x64.exe",
26
+ "build:all": "npm run build:darwin-arm64 && npm run build:darwin-x64 && npm run build:linux-x64 && npm run build:linux-arm64 && npm run build:windows-x64"
20
27
  },
21
28
  "engines": {
22
29
  "node": ">=18"
@@ -41,14 +48,17 @@
41
48
  "bugs": {
42
49
  "url": "https://github.com/devggaurav/web-Api-scrapper/issues"
43
50
  },
44
- "homepage": "https://github.com/devggaurav/web-Api-scrapper#readme",
51
+ "homepage": "https://apiflowtracker.com/",
45
52
  "publishConfig": {
46
53
  "access": "public"
47
54
  },
48
55
  "dependencies": {
49
- "chrome-remote-interface": "^0.33.3",
50
56
  "@modelcontextprotocol/sdk": "^1.0.4",
57
+ "chrome-remote-interface": "^0.33.3",
51
58
  "zod": "^4.4.3"
52
59
  },
53
- "license": "MIT"
60
+ "license": "MIT",
61
+ "devDependencies": {
62
+ "bun": "^1.3.14"
63
+ }
54
64
  }
package/src/browsers.js CHANGED
@@ -30,16 +30,29 @@ const BINARIES = {
30
30
  opera: ['/usr/bin/opera'],
31
31
  chromium: ['/usr/bin/chromium', '/usr/bin/chromium-browser'],
32
32
  },
33
- win32: {
34
- brave: ['C:\\Program Files\\BraveSoftware\\Brave-Browser\\Application\\brave.exe'],
35
- chrome: [
36
- 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
37
- 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
38
- ],
39
- edge: ['C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe'],
40
- },
33
+ win32: winPaths(),
41
34
  };
42
35
 
36
+ // Windows installs land in three different roots depending on the installer:
37
+ // per-machine (Program Files), 32-bit legacy (Program Files (x86)), or
38
+ // per-user (%LOCALAPPDATA%) — the per-user one is the most common for Chrome.
39
+ function winPaths() {
40
+ const roots = [
41
+ process.env.PROGRAMFILES || 'C:\\Program Files',
42
+ process.env['PROGRAMFILES(X86)'] || 'C:\\Program Files (x86)',
43
+ process.env.LOCALAPPDATA || '',
44
+ ].filter(Boolean);
45
+ const under = (suffix) => roots.map((r) => `${r}\\${suffix}`);
46
+ return {
47
+ brave: under('BraveSoftware\\Brave-Browser\\Application\\brave.exe'),
48
+ chrome: under('Google\\Chrome\\Application\\chrome.exe'),
49
+ edge: under('Microsoft\\Edge\\Application\\msedge.exe'),
50
+ vivaldi: under('Vivaldi\\Application\\vivaldi.exe'),
51
+ opera: under('Opera\\opera.exe').concat(under('Programs\\Opera\\opera.exe')),
52
+ chromium: under('Chromium\\Application\\chrome.exe'),
53
+ };
54
+ }
55
+
43
56
  // Browsers that are Chromium-based but not covered here (Safari/WebKit,
44
57
  // Firefox/Gecko) do not expose CDP and need the proxy engine instead.
45
58
  export const UNSUPPORTED_CDP = {
@@ -1,10 +1,18 @@
1
- // Core capture engine. Attaches to a running Chromium browser over the
2
- // Chrome DevTools Protocol and records the network flow of a page target in
3
- // order, with request/response metadata and (bounded) bodies.
1
+ // Core capture engine. Connects to a Chromium browser over the Chrome
2
+ // DevTools Protocol and records the network flow of page targets in order,
3
+ // with request/response metadata and (bounded) bodies.
4
+ //
5
+ // Multi-tab aware, race-free: a single browser-level CDP connection uses
6
+ // Target.setAutoAttach with waitForDebuggerOnStart, so every new tab/popup
7
+ // (OAuth windows, payment redirects, "open in new tab") is PAUSED at creation
8
+ // until Network capture is enabled — no requests are missed, and the whole
9
+ // flow lands in one recording. Tabs are multiplexed over one socket via flat
10
+ // CDP sessions.
4
11
 
5
12
  import CDP from 'chrome-remote-interface';
6
13
 
7
14
  const DEFAULT_MAX_BODY = 512 * 1024; // 512 KB per body, to stay sane
15
+ const STOP_BODY_GRACE_MS = 1500; // max wait for in-flight body fetches on stop
8
16
 
9
17
  // Wait until the CDP endpoint is reachable (a freshly launched browser needs
10
18
  // a moment to open its debugging port).
@@ -26,74 +34,163 @@ async function waitForEndpoint(port, host, timeoutMs = 15000) {
26
34
  );
27
35
  }
28
36
 
37
+ function isRecordablePage(info) {
38
+ const url = info.url || '';
39
+ return info.type === 'page' && !url.startsWith('devtools://');
40
+ }
41
+
29
42
  export class CdpRecorder {
30
- constructor({ port = 9222, host = 'localhost', maxBodyBytes = DEFAULT_MAX_BODY, targetFilter } = {}) {
43
+ /**
44
+ * @param {object} opts
45
+ * port, host - CDP endpoint
46
+ * maxBodyBytes - per-body capture cap
47
+ * targetFilter - optional (target) => boolean to pick initial tab(s)
48
+ * attachAll - record ALL page targets, incl. any new tab that
49
+ * appears (launch mode: the whole window is ours).
50
+ * When false (attach mode), only the filtered/first tab
51
+ * plus popups OPENED BY a recorded tab are captured.
52
+ */
53
+ constructor({ port = 9222, host = 'localhost', maxBodyBytes = DEFAULT_MAX_BODY, targetFilter, attachAll = false } = {}) {
31
54
  this.port = port;
32
55
  this.host = host;
33
56
  this.maxBodyBytes = maxBodyBytes;
34
- this.targetFilter = targetFilter; // optional (target) => boolean to pick a tab
35
- this.client = null;
57
+ this.targetFilter = targetFilter;
58
+ this.attachAll = attachAll;
59
+ this.client = null; // single browser-level connection
60
+ this.sessions = new Map(); // sessionId -> tab {index, targetId, url}
61
+ this.tabTargetIds = new Set(); // targetIds we record (for openerId checks)
62
+ this.wantedTargetIds = new Set(); // initial tabs chosen at start()
63
+ this.tabSeq = 0;
36
64
  this.seq = 0;
37
- this.startWall = null;
38
65
  this.startedAt = null;
39
- // requestId -> record
40
- this.records = new Map();
41
- // preserve emission order
66
+ this.records = new Map(); // `${sessionId}:${requestId}` -> record
42
67
  this.order = [];
43
- this.target = null;
68
+ this.pendingBodies = new Set(); // in-flight getResponseBody promises
69
+ this.target = null; // main (first) target info, for the doc header
70
+ this.mainSessionId = null;
71
+ this.stopping = false;
72
+ this.everAttached = false;
44
73
  }
45
74
 
46
75
  async start() {
47
76
  const targets = await waitForEndpoint(this.port, this.host);
48
- const pages = targets.filter((t) => t.type === 'page' && !t.url.startsWith('devtools://'));
49
- let chosen = pages;
50
- if (this.targetFilter) chosen = pages.filter(this.targetFilter);
51
- const target = chosen[0] || pages[0];
52
- if (!target) throw new Error('No suitable page target to attach to.');
53
- this.target = target;
54
-
55
- this.client = await CDP({ port: this.port, host: this.host, target });
56
- const { Network, Page } = this.client;
57
-
58
- await Network.enable({ maxTotalBufferSize: 100_000_000, maxResourceBufferSize: 20_000_000 });
59
- try { await Page.enable(); } catch { /* Page domain optional */ }
77
+ const pages = targets.filter((t) => isRecordablePage({ type: t.type, url: t.url }));
78
+ let chosen = this.targetFilter ? pages.filter(this.targetFilter) : pages;
79
+ if (!this.attachAll && !this.targetFilter) chosen = pages.slice(0, 1);
80
+ if (!chosen.length) chosen = pages.slice(0, 1);
81
+ if (!chosen.length) throw new Error('No suitable page target to attach to.');
82
+ for (const t of chosen) this.wantedTargetIds.add(t.id);
60
83
 
84
+ const version = await CDP.Version({ port: this.port, host: this.host });
85
+ if (!version.webSocketDebuggerUrl) {
86
+ throw new Error('Browser exposes no webSocketDebuggerUrl; a Chromium >= 63 is required.');
87
+ }
88
+ const client = await CDP({ target: version.webSocketDebuggerUrl });
89
+ this.client = client;
61
90
  this.startedAt = new Date().toISOString();
62
91
 
63
- Network.requestWillBeSent((p) => this._onRequest(p));
64
- Network.responseReceived((p) => this._onResponse(p));
65
- Network.loadingFinished((p) => this._onFinished(p));
66
- Network.loadingFailed((p) => this._onFailed(p));
67
- Network.webSocketCreated((p) => this._onWebSocket(p));
68
-
69
- // Fires when the browser/tab is closed by the user (the CDP socket drops).
70
- // Lets the session auto-finalize and write files on browser close.
71
- this.client.on('disconnect', () => {
92
+ client.on('disconnect', () => {
72
93
  if (!this.stopping) this.onDisconnect?.();
73
94
  });
74
95
 
75
- return { target: { id: target.id, url: target.url, title: target.title } };
96
+ // Route network events to the owning tab via the flat-session id.
97
+ client.on('Network.requestWillBeSent', (p, sid) => this._onRequest(sid, p));
98
+ client.on('Network.responseReceived', (p, sid) => this._onResponse(sid, p));
99
+ client.on('Network.loadingFinished', (p, sid) => this._onFinished(sid, p));
100
+ client.on('Network.loadingFailed', (p, sid) => this._onFailed(sid, p));
101
+ client.on('Network.webSocketCreated', (p, sid) => this._onWebSocket(sid, p));
102
+
103
+ client.on('Target.attachedToTarget', (p) => { this._onAttached(p).catch(() => {}); });
104
+ client.on('Target.detachedFromTarget', ({ sessionId }) => this._onDetached(sessionId));
105
+
106
+ // Pause every NEW target at creation until we've enabled Network on it.
107
+ await client.Target.setAutoAttach({ autoAttach: true, waitForDebuggerOnStart: true, flatten: true });
108
+
109
+ // Existing tabs aren't covered by autoAttach — attach to the chosen ones.
110
+ for (const t of chosen) {
111
+ await client.Target.attachToTarget({ targetId: t.id, flatten: true });
112
+ }
113
+
114
+ // attachedToTarget events have been handled synchronously above.
115
+ return { target: this.target, tabs: this.sessions.size };
76
116
  }
77
117
 
78
- // Navigate the attached target. Used by launch mode so the recorder is
79
- // listening before the page fires its first request (avoids missing early calls).
118
+ async _onAttached({ sessionId, targetInfo, waitingForDebugger }) {
119
+ const { targetId } = targetInfo;
120
+ const wanted = this.wantedTargetIds.has(targetId);
121
+ const openedByUs = targetInfo.openerId && this.tabTargetIds.has(targetInfo.openerId);
122
+ const record =
123
+ !this.stopping &&
124
+ isRecordablePage(targetInfo) &&
125
+ (wanted || this.attachAll || openedByUs);
126
+
127
+ if (!record) {
128
+ // Not ours: resume it (it may be paused) and let it go.
129
+ try { await this.client.Runtime.runIfWaitingForDebugger(sessionId); } catch { /* ignore */ }
130
+ try { await this.client.Target.detachFromTarget({ sessionId }); } catch { /* ignore */ }
131
+ return;
132
+ }
133
+
134
+ const tab = { index: this.tabSeq++, targetId, url: targetInfo.url || '' };
135
+ this.sessions.set(sessionId, tab);
136
+ this.tabTargetIds.add(targetId);
137
+ this.everAttached = true;
138
+ if (this.mainSessionId == null) {
139
+ this.mainSessionId = sessionId;
140
+ this.target = { id: targetId, url: targetInfo.url, title: targetInfo.title };
141
+ }
142
+
143
+ try {
144
+ await this.client.Network.enable(
145
+ { maxTotalBufferSize: 100_000_000, maxResourceBufferSize: 20_000_000 },
146
+ sessionId,
147
+ );
148
+ try { await this.client.Page.enable(sessionId); } catch { /* Page domain optional */ }
149
+ } finally {
150
+ // Only resume the page once capture is on — this is what makes new
151
+ // tabs/popups lose zero requests.
152
+ if (waitingForDebugger) {
153
+ try { await this.client.Runtime.runIfWaitingForDebugger(sessionId); } catch { /* ignore */ }
154
+ }
155
+ }
156
+ }
157
+
158
+ // A recorded tab went away (closed or crashed). When the LAST one goes,
159
+ // treat it like the user closing the browser: auto-finalize. (On macOS the
160
+ // browser process can outlive its last window, so the browser-level
161
+ // disconnect alone isn't enough.)
162
+ _onDetached(sessionId) {
163
+ const tab = this.sessions.get(sessionId);
164
+ if (!tab) return;
165
+ this.sessions.delete(sessionId);
166
+ this.tabTargetIds.delete(tab.targetId);
167
+ if (!this.stopping && this.everAttached && this.sessions.size === 0) {
168
+ this.onDisconnect?.();
169
+ }
170
+ }
171
+
172
+ // Navigate the main tab. Used by launch mode so the recorder is listening
173
+ // before the page fires its first request (avoids missing early calls).
80
174
  async navigate(url) {
81
- if (!this.client) throw new Error('Recorder not started.');
82
- await this.client.Page.navigate({ url });
175
+ if (!this.client || this.mainSessionId == null) throw new Error('Recorder not started.');
176
+ await this.client.Page.navigate({ url }, this.mainSessionId);
83
177
  }
84
178
 
85
- _rec(requestId) {
86
- let rec = this.records.get(requestId);
179
+ _rec(sessionId, requestId) {
180
+ const key = `${sessionId}:${requestId}`;
181
+ let rec = this.records.get(key);
87
182
  if (!rec) {
88
183
  rec = { requestId, index: this.seq++ };
89
- this.records.set(requestId, rec);
90
- this.order.push(requestId);
184
+ this.records.set(key, rec);
185
+ this.order.push(key);
91
186
  }
92
187
  return rec;
93
188
  }
94
189
 
95
- _onRequest(p) {
96
- const rec = this._rec(p.requestId);
190
+ _onRequest(sessionId, p) {
191
+ const tab = this.sessions.get(sessionId);
192
+ if (!tab) return;
193
+ const rec = this._rec(sessionId, p.requestId);
97
194
  const r = p.request;
98
195
  // A redirect reuses the requestId; keep the redirect chain instead of losing it.
99
196
  if (rec.method && p.redirectResponse) {
@@ -109,11 +206,15 @@ export class CdpRecorder {
109
206
  rec.wallTime = p.wallTime;
110
207
  rec.timestamp = p.timestamp;
111
208
  rec.initiator = p.initiator ? { type: p.initiator.type } : undefined;
112
- if (this.startWall == null && p.wallTime) this.startWall = p.wallTime;
209
+ rec.tab = tab.index;
210
+ // A tab's first document request is its real URL (popups start blank).
211
+ if (p.type === 'Document') tab.url = r.url;
212
+ rec.tabUrl = tab.url;
113
213
  }
114
214
 
115
- _onResponse(p) {
116
- const rec = this._rec(p.requestId);
215
+ _onResponse(sessionId, p) {
216
+ if (!this.sessions.has(sessionId)) return;
217
+ const rec = this._rec(sessionId, p.requestId);
117
218
  const res = p.response;
118
219
  rec.status = res.status;
119
220
  rec.statusText = res.statusText;
@@ -128,18 +229,25 @@ export class CdpRecorder {
128
229
  }
129
230
  }
130
231
 
131
- async _onFinished(p) {
132
- const rec = this._rec(p.requestId);
232
+ _onFinished(sessionId, p) {
233
+ if (!this.sessions.has(sessionId)) return;
234
+ const rec = this._rec(sessionId, p.requestId);
133
235
  rec.encodedDataLength = p.encodedDataLength;
134
236
  rec.finishedTs = p.timestamp;
135
237
  if (rec.timestamp != null && p.timestamp != null) {
136
238
  rec.durationMs = Math.max(0, (p.timestamp - rec.timestamp) * 1000);
137
239
  }
138
- await this._captureBody(rec);
240
+ // Track the fetch so stop() can wait for in-flight bodies instead of
241
+ // closing the socket under them (which silently dropped tail-end bodies).
242
+ const promise = this._captureBody(sessionId, rec).finally(() => {
243
+ this.pendingBodies.delete(promise);
244
+ });
245
+ this.pendingBodies.add(promise);
139
246
  }
140
247
 
141
- _onFailed(p) {
142
- const rec = this._rec(p.requestId);
248
+ _onFailed(sessionId, p) {
249
+ if (!this.sessions.has(sessionId)) return;
250
+ const rec = this._rec(sessionId, p.requestId);
143
251
  rec.failed = true;
144
252
  rec.errorText = p.errorText;
145
253
  rec.canceled = p.canceled;
@@ -149,19 +257,24 @@ export class CdpRecorder {
149
257
  }
150
258
  }
151
259
 
152
- _onWebSocket(p) {
153
- const rec = this._rec(p.requestId);
260
+ _onWebSocket(sessionId, p) {
261
+ const tab = this.sessions.get(sessionId);
262
+ if (!tab) return;
263
+ const rec = this._rec(sessionId, p.requestId);
154
264
  rec.method = 'WS';
155
265
  rec.url = p.url;
156
266
  rec.resourceType = 'WebSocket';
267
+ rec.tab = tab.index;
268
+ rec.tabUrl = tab.url;
157
269
  }
158
270
 
159
- async _captureBody(rec) {
271
+ async _captureBody(sessionId, rec) {
160
272
  if (rec.responseBody !== undefined) return;
161
273
  try {
162
- const { body, base64Encoded } = await this.client.Network.getResponseBody({
163
- requestId: rec.requestId,
164
- });
274
+ const { body, base64Encoded } = await this.client.Network.getResponseBody(
275
+ { requestId: rec.requestId },
276
+ sessionId,
277
+ );
165
278
  if (base64Encoded) {
166
279
  rec.responseBodyBase64 = true;
167
280
  rec.responseBody = body.length > this.maxBodyBytes ? '[binary body omitted]' : body;
@@ -172,29 +285,45 @@ export class CdpRecorder {
172
285
  rec.responseBody = body;
173
286
  }
174
287
  } catch {
175
- // Body may be unavailable (evicted, 204, websocket, etc.) — that's fine.
288
+ // Body may be unavailable (evicted, 204, websocket, closed tab) — fine.
176
289
  }
177
290
  }
178
291
 
179
292
  // Snapshot of everything captured so far, in emission order.
180
293
  getRecords() {
181
- return this.order.map((id) => this.records.get(id)).filter(Boolean);
294
+ return this.order.map((key) => this.records.get(key)).filter(Boolean);
182
295
  }
183
296
 
184
- async stop() {
297
+ async stop({ closeBrowser = false } = {}) {
185
298
  this.stopping = true; // suppress the disconnect->auto-finalize path
186
- // Give any in-flight loadingFinished handlers a beat to grab bodies.
187
- await new Promise((r) => setTimeout(r, 200));
299
+ // Let in-flight loadingFinished body fetches land (bounded wait).
300
+ if (this.pendingBodies.size) {
301
+ await Promise.race([
302
+ Promise.allSettled([...this.pendingBodies]),
303
+ new Promise((r) => setTimeout(r, STOP_BODY_GRACE_MS)),
304
+ ]);
305
+ }
188
306
  const records = this.getRecords();
307
+ // Quit the browser via CDP while the socket is still open. Works even for
308
+ // a reused persistent-profile window we didn't spawn.
309
+ let browserClosed = false;
310
+ if (closeBrowser && this.client) {
311
+ try {
312
+ await this.client.Browser.close();
313
+ browserClosed = true;
314
+ } catch { /* fall back to killing the child process, if any */ }
315
+ }
189
316
  if (this.client) {
190
317
  try { await this.client.close(); } catch { /* ignore */ }
191
318
  this.client = null;
192
319
  }
320
+ this.sessions.clear();
193
321
  return {
194
322
  startedAt: this.startedAt,
195
323
  stoppedAt: new Date().toISOString(),
196
- target: this.target ? { id: this.target.id, url: this.target.url, title: this.target.title } : null,
324
+ target: this.target,
197
325
  records,
326
+ browserClosed,
198
327
  };
199
328
  }
200
329
  }
@@ -1,6 +1,8 @@
1
1
  // Minimal HAR 1.2 exporter so recordings can be opened in Chrome DevTools,
2
2
  // Charles, Insomnia, Postman, etc.
3
3
 
4
+ import { VERSION } from '../version.js';
5
+
4
6
  function headerArray(headers) {
5
7
  return Object.entries(headers || {}).map(([name, value]) => ({ name, value: String(value) }));
6
8
  }
@@ -16,7 +18,9 @@ export function toHar(records, meta = {}) {
16
18
  const reqBody = r.requestBody;
17
19
  const resBody = r.responseBody;
18
20
  return {
19
- startedDateTime: meta.startedAt || new Date(0).toISOString(),
21
+ // Per-request wall time so DevTools/Charles render a real waterfall
22
+ // instead of stacking everything at the session start.
23
+ startedDateTime: r.startedAt || meta.startedAt || new Date(0).toISOString(),
20
24
  time: r.durationMs || 0,
21
25
  request: {
22
26
  method: r.method || 'GET',
@@ -54,7 +58,7 @@ export function toHar(records, meta = {}) {
54
58
  return {
55
59
  log: {
56
60
  version: '1.2',
57
- creator: { name: 'browser-flow-tracker', version: '0.1.0' },
61
+ creator: { name: 'browser-flow-tracker', version: VERSION },
58
62
  pages: [],
59
63
  entries,
60
64
  },
@@ -79,6 +79,7 @@ export function toMarkdown(session, options = {}) {
79
79
  lines.push(`### ${i + 1}. ${e.method} ${e.path}`);
80
80
  lines.push('');
81
81
  lines.push(`- **URL:** \`${e.url}\``);
82
+ if (e.tab > 0) lines.push(`- **Tab:** popup/new tab #${e.tab}${e.tabUrl ? ` (\`${e.tabUrl}\`)` : ''}`);
82
83
  lines.push(`- **Status:** ${e.failed ? `✗ ${e.errorText}` : `${e.status || ''} ${e.statusText || ''}`}`);
83
84
  if (e.durationMs != null) lines.push(`- **Duration:** ${e.durationMs}ms`);
84
85
  if (e.initiator) lines.push(`- **Initiated by:** ${e.initiator}`);
package/src/normalize.js CHANGED
@@ -1,20 +1,7 @@
1
1
  // Turns raw CDP records into a clean, ordered, API-focused flow.
2
2
 
3
3
  import { classify } from './filter.js';
4
-
5
- const SENSITIVE_HEADERS = new Set([
6
- 'authorization', 'cookie', 'set-cookie', 'x-api-key', 'x-auth-token',
7
- 'proxy-authorization', 'x-csrf-token',
8
- ]);
9
-
10
- function redactHeaders(headers, redact) {
11
- if (!headers) return {};
12
- const out = {};
13
- for (const [k, v] of Object.entries(headers)) {
14
- out[k] = redact && SENSITIVE_HEADERS.has(k.toLowerCase()) ? '[redacted]' : v;
15
- }
16
- return out;
17
- }
4
+ import { redactHeaders, redactQuery, redactBody } from './redact.js';
18
5
 
19
6
  function tryParseJson(body) {
20
7
  if (typeof body !== 'string') return undefined;
@@ -54,10 +41,11 @@ export function normalize(records, { includeNoise = false, redact = true } = {})
54
41
  url: rec.url,
55
42
  host,
56
43
  path,
57
- query,
44
+ query: query ? redactQuery(query, redact) : undefined,
58
45
  resourceType: rec.resourceType,
59
46
  status: rec.status,
60
47
  statusText: rec.statusText,
48
+ startedAt: rec.wallTime ? new Date(rec.wallTime * 1000).toISOString() : undefined,
61
49
  durationMs: rec.durationMs != null ? Math.round(rec.durationMs) : undefined,
62
50
  sizeBytes: rec.encodedDataLength,
63
51
  fromCache: rec.fromCache || undefined,
@@ -65,17 +53,19 @@ export function normalize(records, { includeNoise = false, redact = true } = {})
65
53
  errorText: rec.errorText,
66
54
  initiator: rec.initiator?.type,
67
55
  redirects: rec.redirects,
56
+ tab: rec.tab, // 0-based tab index; >0 means a popup / new tab in the flow
57
+ tabUrl: rec.tab > 0 ? rec.tabUrl : undefined,
68
58
  requestHeaders: redactHeaders(rec.requestHeaders, redact),
69
59
  responseHeaders: redactHeaders(rec.responseHeaders, redact),
70
60
  };
71
61
 
72
62
  const reqJson = tryParseJson(rec.requestBody);
73
- entry.requestBody = reqJson !== undefined ? reqJson : rec.requestBody;
63
+ entry.requestBody = redactBody(reqJson !== undefined ? reqJson : rec.requestBody, redact);
74
64
  entry.requestBodyIsJson = reqJson !== undefined;
75
65
 
76
66
  if (!rec.responseBodyBase64) {
77
67
  const resJson = tryParseJson(rec.responseBody);
78
- entry.responseBody = resJson !== undefined ? resJson : rec.responseBody;
68
+ entry.responseBody = redactBody(resJson !== undefined ? resJson : rec.responseBody, redact);
79
69
  entry.responseBodyIsJson = resJson !== undefined;
80
70
  entry.responseBodyTruncated = rec.responseBodyTruncated || undefined;
81
71
  } else {
package/src/redact.js ADDED
@@ -0,0 +1,101 @@
1
+ // Redaction of secrets so recordings are safe to share by default.
2
+ // Covers three places secrets actually show up:
3
+ // 1. headers (Authorization, Cookie, ...)
4
+ // 2. URL query params (?token=..., ?api_key=...)
5
+ // 3. JSON / form-encoded request & response bodies ({"password": ...},
6
+ // login responses returning {"access_token": ...}, etc.)
7
+
8
+ export const SENSITIVE_HEADERS = new Set([
9
+ 'authorization', 'cookie', 'set-cookie', 'x-api-key', 'x-auth-token',
10
+ 'proxy-authorization', 'x-csrf-token', 'x-xsrf-token', 'x-session-token',
11
+ 'x-access-token', 'x-refresh-token', 'api-key', 'apikey',
12
+ ]);
13
+
14
+ // Key names (matched case-insensitively, ignoring -_ separators) whose VALUES
15
+ // are secrets wherever they appear: query params, JSON bodies, form bodies.
16
+ const SENSITIVE_KEY_RE = new RegExp(
17
+ '^(' + [
18
+ 'password', 'passwd', 'pwd', 'passphrase', 'currentpassword', 'newpassword',
19
+ 'oldpassword', 'confirmpassword', 'secret', 'clientsecret', 'appsecret',
20
+ 'token', 'accesstoken', 'refreshtoken', 'idtoken', 'authtoken', 'apitoken',
21
+ 'sessiontoken', 'bearertoken', 'csrftoken', 'xsrftoken',
22
+ 'apikey', 'apisecret', 'privatekey', 'secretkey', 'encryptionkey',
23
+ 'auth', 'authorization', 'credential', 'credentials',
24
+ 'sessionid', 'sid', 'jsessionid', 'phpsessid',
25
+ 'otp', 'totp', 'mfacode', 'pin', 'securitycode', 'verificationcode',
26
+ 'cardnumber', 'ccnumber', 'cvv', 'cvc', 'ssn',
27
+ ].join('|') + ')$',
28
+ 'i',
29
+ );
30
+
31
+ const REDACTED = '[redacted]';
32
+ const MAX_DEPTH = 12;
33
+
34
+ function isSensitiveKey(key) {
35
+ return SENSITIVE_KEY_RE.test(String(key).replace(/[-_.]/g, ''));
36
+ }
37
+
38
+ export function redactHeaders(headers, redact = true) {
39
+ if (!headers) return {};
40
+ const out = {};
41
+ for (const [k, v] of Object.entries(headers)) {
42
+ out[k] = redact && SENSITIVE_HEADERS.has(k.toLowerCase()) ? REDACTED : v;
43
+ }
44
+ return out;
45
+ }
46
+
47
+ // Flat object of query params ({token: 'abc'} -> {token: '[redacted]'}).
48
+ export function redactQuery(query, redact = true) {
49
+ if (!query || !redact) return query;
50
+ const out = {};
51
+ for (const [k, v] of Object.entries(query)) {
52
+ out[k] = isSensitiveKey(k) ? REDACTED : v;
53
+ }
54
+ return out;
55
+ }
56
+
57
+ // Deep-walk parsed JSON, replacing values of sensitive keys.
58
+ export function redactJson(value, depth = 0) {
59
+ if (value == null || depth > MAX_DEPTH) return value;
60
+ if (Array.isArray(value)) return value.map((v) => redactJson(v, depth + 1));
61
+ if (typeof value === 'object') {
62
+ const out = {};
63
+ for (const [k, v] of Object.entries(value)) {
64
+ out[k] = isSensitiveKey(k) ? REDACTED : redactJson(v, depth + 1);
65
+ }
66
+ return out;
67
+ }
68
+ return value;
69
+ }
70
+
71
+ // application/x-www-form-urlencoded bodies ("user=a&password=b").
72
+ function looksLikeFormBody(s) {
73
+ return /^[^=\s&]+=[^&\n]*(&[^=\s&]+=[^&\n]*)*$/.test(s) && s.includes('=');
74
+ }
75
+
76
+ function redactFormBody(s) {
77
+ return s
78
+ .split('&')
79
+ .map((pair) => {
80
+ const eq = pair.indexOf('=');
81
+ if (eq === -1) return pair;
82
+ const key = pair.slice(0, eq);
83
+ let decoded = key;
84
+ try { decoded = decodeURIComponent(key); } catch { /* keep raw */ }
85
+ return isSensitiveKey(decoded) ? `${key}=${encodeURIComponent(REDACTED)}` : pair;
86
+ })
87
+ .join('&');
88
+ }
89
+
90
+ /**
91
+ * Redact a body. `body` may be a parsed JSON value (object/array) or a raw
92
+ * string (form-encoded or anything else). Returns the same shape.
93
+ */
94
+ export function redactBody(body, redact = true) {
95
+ if (!redact || body == null) return body;
96
+ if (typeof body === 'object') return redactJson(body);
97
+ if (typeof body === 'string' && looksLikeFormBody(body.trim())) {
98
+ return redactFormBody(body.trim());
99
+ }
100
+ return body;
101
+ }
package/src/session.js CHANGED
@@ -11,6 +11,7 @@ import { launchBrowser } from './browsers.js';
11
11
  import { normalize } from './normalize.js';
12
12
  import { toHar } from './exporters/har.js';
13
13
  import { toMarkdown } from './exporters/markdown.js';
14
+ import { VERSION } from './version.js';
14
15
 
15
16
  function stamp() {
16
17
  // Filesystem-safe timestamp without relying on locale.
@@ -93,7 +94,10 @@ export class TrackingSession {
93
94
  ? (t) => t.url && t.url.includes(urlMatch)
94
95
  : undefined;
95
96
 
96
- this.recorder = new CdpRecorder({ port, host, targetFilter });
97
+ // Launch mode: the whole window is ours, so record every tab in it
98
+ // (popups, OAuth windows, new tabs). Attach mode records the chosen tab
99
+ // plus any popups that tab opens.
100
+ this.recorder = new CdpRecorder({ port, host, targetFilter, attachAll: Boolean(launch) });
97
101
  // If the user closes the browser/tab, auto-finalize and write the files.
98
102
  this.recorder.onDisconnect = () => { this._onBrowserClosed(); };
99
103
  const info = await this.recorder.start();
@@ -141,7 +145,7 @@ export class TrackingSession {
141
145
  // return that result instead of erroring on a later explicit stop.
142
146
  if (this.finalized) return this.finalized;
143
147
  if (!this.recorder) throw new Error('Session not started.');
144
- const raw = await this.recorder.stop();
148
+ const raw = await this.recorder.stop({ closeBrowser });
145
149
  this.active = false;
146
150
 
147
151
  const norm = normalize(raw.records, {
@@ -151,7 +155,7 @@ export class TrackingSession {
151
155
 
152
156
  const session = {
153
157
  tool: 'browser-flow-tracker',
154
- version: '0.1.0',
158
+ version: VERSION,
155
159
  startedAt: raw.startedAt,
156
160
  stoppedAt: raw.stoppedAt,
157
161
  target: raw.target,
@@ -162,9 +166,11 @@ export class TrackingSession {
162
166
 
163
167
  let files = null;
164
168
  if (write) {
165
- const dir = outDir || join(process.cwd(), 'recordings');
169
+ // Fall back to the defaults given at start_tracking time, so an
170
+ // explicit stop honors them the same way auto-finalize does.
171
+ const dir = outDir || this.opts.outDir || join(process.cwd(), 'recordings');
166
172
  mkdirSync(dir, { recursive: true });
167
- const base = name || `flow-${stamp()}`;
173
+ const base = name || this.opts.name || `flow-${stamp()}`;
168
174
  const jsonPath = join(dir, `${base}.flow.json`);
169
175
  const harPath = join(dir, `${base}.har`);
170
176
  const mdPath = join(dir, `${base}.md`);
@@ -174,7 +180,9 @@ export class TrackingSession {
174
180
  files = { json: jsonPath, har: harPath, markdown: mdPath };
175
181
  }
176
182
 
177
- if (closeBrowser && this.launched?.child) {
183
+ // Browser.close via CDP is preferred (works for reused windows too);
184
+ // killing the child we spawned is the fallback.
185
+ if (closeBrowser && !raw.browserClosed && this.launched?.child) {
178
186
  try { this.launched.child.kill(); } catch { /* ignore */ }
179
187
  }
180
188
 
package/src/version.js ADDED
@@ -0,0 +1,4 @@
1
+ // Single source of truth for the tool version (keep in sync with package.json).
2
+ // A plain constant (not a package.json read) so the bun-compiled standalone
3
+ // binaries don't need the file on disk at runtime.
4
+ export const VERSION = '0.2.0';