pw-repl 0.5.0 → 0.6.0

package/AGENTS.md

@@ -8,15 +8,25 @@ folder this file is in) when it is not installed globally.
 
 If you hit a limitation or write a workaround, consider adding the capability to the REPL instead.
 
-- A command is a function in the `commands` object in `lib/commands.js`, plus an entry in `lib/help.js`
-  under one topic.
+- A command is a function in the `commands` object of the module for what it works on, plus an entry
+  in `lib/help.js` under one topic; `lib/commands.js` gathers every module's commands. A command that
+  needs no tab, only reads, or takes `--all` is also listed in `lib/runner.js`'s `NO_TAB_NEEDED`,
+  `READ_ONLY` or `INSPECTION`, and one that takes fixed words in `complete` (`lib/commands.js`).
+- The modules, one per facility, each with its commands: `lib/tabs.js` (tabs, navigating),
+  `lib/elements.js` (choosing a match; the interact and mouse commands), `lib/inspect.js` (reading the
+  page, screenshots, eval, CDP, waits), `lib/watch.js`, `lib/requestlog.js` (requests, bodies, console),
+  `lib/routes.js`, `lib/network.js`, `lib/emulation.js`, `lib/capture.js`, `lib/dialogs.js` and
+  `lib/modes.js`. Beneath them, `lib/tabstate.js` is the one record kept per tab, `lib/cdp.js` the CDP
+  session kept per tab, and `lib/util.js` small shared helpers. No facility requires `lib/modes.js` or
+  `lib/commands.js`, and a unit test fails on a require cycle, which CommonJS would not report.
 - `bin/pw-repl.js` is the command line; `lib/start.js` connects and runs the prompt; `lib/runner.js`
   runs commands one at a time (`quit` and `dialog` skip its queue); `lib/server.js` is the command
   server; `lib/send.js` and `lib/client.js` are `send` and `where`; `lib/background.js` is
   `serve --background`, `attach` and `stop`; `lib/launch.js` is `--launch`; `lib/syntax.js` is how a
   command line's words are read, shared with `send`; `lib/cli-names.js` maps playwright-cli's command
   names to these; `lib/state.js` holds the session state, with each client's selected tab;
-  `lib/output.js` routes all output so the server can return it.
+  `lib/output.js` routes all output so the server can return it; `lib/skill.js` stamps the skill
+  `pw-repl skill` prints, and gives `where` its hash.
 - Match playwright-cli (Microsoft's CLI for agents) where it is reasonable: agents are likelier to be
   trained on it. A new command, option or output that does what one of its does takes its name and form,
   or accepts them too (`lib/cli-names.js`, `help playwright-cli`). Differ where sharing a browser with a
@@ -26,15 +36,27 @@ If you hit a limitation or write a workaround, consider adding the capability to
   (`test/harness.js`), and the real REPL with its server. Nothing is mocked, and the shared browser is
   never touched. It finds Chromium through `PW_TEST_CHROME`, or where `--launch` looks (Playwright's
   browsers, then the `PATH`), and skips the browser tests if there is none. Add a test with each new
-  command or behaviour.
+  command or behaviour. In `test/repl.test.js` each test starts in a fresh tab of its own, and every mode
+  is turned off after it; a test whose REPL quits, times out or is killed goes in
+  `test/repl-alone.test.js`, with a Chromium of its own.
+- Help entries (`lib/help.js`) are plain paragraphs: `wrap()` lays each out at 104 columns, and keeps as
+  written a paragraph with a bullet, a table or an indented example. `test/fixtures` holds the rendered
+  help; after an intended change to it, save it again (the command is in `test/units.test.js`).
 - Time each full run, and put the result in the body of the commit it tests (`npm test: 151 tests, 79s`).
   Compare with the last commit that has one (`git log --grep='npm test:'`): a run that grew by more than
   a few seconds means a slow test crept in, usually one waiting out a real timeout (5s for a click).
   Find it (each test prints its time) and make it faster, or say why it can't be, before committing.
+- Don't race the clock in a test: a page that changes on a timer, or a command timed against a budget,
+  fails on a busy machine. Have the page change when the test asks (eval), or measure the thing itself
+  (frames drawn, not clicks timed). A click waits for its element to hold still for two frames.
+- A test that times out in a run much slower than usual (the machine is busy) may be load, not a bug:
+  run it again, alone. If it fails alone too, find out why before going on.
 - For anything the tests can't reach, exercise the change in a running REPL of your own: on a socket of
   your own, with `--launch` unless the change needs the user's browser, and then in a tab of your own.
 - `skill/SKILL.md` is how agents learn to use the REPL: keep it in step with a change to how it is
-  used, and leave its Custom rules section empty.
+  used, and leave its Custom rules section empty. Keep its `<!-- pw-repl skill stamp -->` line: `pw-repl
+  skill` replaces it with the version and a hash of the skill's text, which `where` reports, so agents
+  can tell a saved copy is out of date.
 
 ## Waves
 

package/bin/pw-repl.js

@@ -84,6 +84,16 @@ function parseSendArgs(args, allowCommand) {
       }
     } else if (arg === '--') { i++; break; } else break;
   }
+  // An address that is refused (not loopback) is a usage error, said once, not a stack trace.
+  const endpoint = options.endpoint || process.env.PW_ENDPOINT;
+  if (endpoint) {
+    try {
+      require('../lib/client').parseEndpoint(endpoint);
+    } catch (error) {
+      console.error(`pw-repl: ${error.message}`);
+      process.exit(64);
+    }
+  }
   const words = args.slice(i);
   if (!allowCommand && words.length) usage();
   // For a command that reads a quoted selector (fill "text=Your name" Ada), a

package/lib/capture.js

@@ -0,0 +1,139 @@
+// Capturing a tab's requests and console messages together, from now until capture off.
+const { state, onShutdown } = require('./state');
+const out = require('./output');
+const { printOutput } = out;
+const { numbers, MAX_EVENT_TEXT, NO_TAB, clock, hints } = require('./util');
+const { consoleText, requestEntries } = require('./requestlog');
+
+const MAX_CAPTURE_EVENTS = 10000;
+const MAX_CAPTURE_TEXT = 400000;
+
+let cap = null;
+let lastCapture = null;
+
+// The capture running now, or null, for the modes it counts as.
+function currentCapture() {
+  return cap;
+}
+
+// Records the selected tab's requests and console messages together, in time
+// order, until capture off, or for a set number of seconds.
+async function startCapture(tokens) {
+  if (cap) { out.log('Already capturing; capture off stops it.'); return; }
+  if (!state.page || state.page.isClosed()) throw new Error(NO_TAB);
+  const usage = 'Usage: capture on [requests|console] [seconds] (seconds from 1 to 3600)';
+  const kinds = tokens.filter(t => t === 'requests' || t === 'console');
+  const numbers = tokens.filter(t => /^\d+$/.test(t));
+  if (kinds.length > 1 || numbers.length > 1 || kinds.length + numbers.length !== tokens.length) throw new Error(usage);
+  const seconds = numbers.length ? Number(numbers[0]) : null;
+  if (seconds !== null && (seconds < 1 || seconds > 3600)) throw new Error(usage);
+  const label = kinds[0] || 'requests and console';
+  const startedAt = Date.now();
+  const events = [];
+  const handlers = [];
+  let dropped = 0;
+  let shortened = 0;
+  let bytes = 0;
+  const record = event => {
+    let text = String(event.text);
+    if (text.length > MAX_EVENT_TEXT) { text = `${text.slice(0, MAX_EVENT_TEXT)}…`; shortened += 1; }
+    const size = text.length;
+    if (events.length >= MAX_CAPTURE_EVENTS || bytes + size > MAX_CAPTURE_TEXT) { dropped += 1; return; }
+    bytes += size;
+    events.push({ ...event, text });
+  };
+  const page = state.page;
+  const listen = (event, handler) => { page.on(event, handler); handlers.push([event, handler]); };
+  if (label !== 'console') listen('request', req => record({ t: Date.now(), tag: 'request', req, text: `${req.method()} ${req.url()}` }));
+  if (label !== 'requests') {
+    listen('console', msg => record({ t: Date.now(), tag: msg.type(), text: consoleText(msg) }));
+    listen('pageerror', error => record({ t: Date.now(), tag: 'pageerror', text: error.stack || error.message }));
+  }
+  // A capture must not outlive its tab: nothing could see or stop it.
+  listen('close', () => {
+    if (cap?.page !== page) return;
+    endCapture();
+    out.notice('The captured tab closed, which ended the capture; capture shows what it recorded.', page);
+  });
+  const capture = {
+    page, label, startedAt, events, handlers, wake: null, owner: state.client,
+    get dropped() { return dropped; },
+    get shortened() { return shortened; },
+  };
+  cap = capture;
+  if (seconds) {
+    out.log(`Capturing ${label} for ${seconds}s...`);
+    // Also ends early if the capture does, e.g. because its tab closed.
+    await new Promise(resolve => {
+      const timer = setTimeout(resolve, seconds * 1000);
+      capture.wake = () => { clearTimeout(timer); resolve(); };
+    });
+    if (cap === capture) stopCapture();
+    else printCapture(lastCapture);
+  } else {
+    out.log(`Capturing ${label}; capture off stops it and prints what it recorded.`);
+  }
+}
+
+function endCapture() {
+  cap.handlers.forEach(([event, h]) => cap.page.off(event, h));
+  const { label, startedAt, events, dropped, shortened, wake } = cap;
+  cap = null;
+  lastCapture = { label, startedAt, events, dropped, shortened };
+  if (wake) wake();
+}
+
+function stopCapture(all = false) {
+  if (!cap) { out.log('Not capturing.'); return; }
+  endCapture();
+  printCapture(lastCapture, all);
+}
+
+// One line per event, from the start of the capture: a request as requests
+// shows it (its status is looked up now, so it is known if it has finished),
+// a console message or page error as console shows it.
+function printCapture(capture, all = false) {
+  const lines = capture.events.map(event => {
+    const at = `+${((event.t - capture.startedAt) / 1000).toFixed(3)}s`;
+    if (event.tag !== 'request') return `${at} [${event.tag}] ${event.text}`;
+    const entry = requestEntries.get(event.req);
+    return entry ? `${at} #${entry.id} ${entry.method} ${entry.status} ${entry.url}` : `${at} ${event.text}`;
+  });
+  if (lines.length) printOutput(lines.join('\n'), all);
+  else out.log('Nothing captured');
+  const notes = [];
+  if (capture.dropped) notes.push(`${capture.dropped} event(s) omitted`);
+  if (capture.shortened) notes.push(`${capture.shortened} message(s) shortened`);
+  if (notes.length) out.log(`[capture limits: ${notes.join('; ')}]`);
+}
+
+function discardCapture() {
+  if (!cap) return;
+  cap.handlers.forEach(([event, handler]) => cap.page.off(event, handler));
+  cap = null;
+}
+
+onShutdown(discardCapture);
+
+const commands = {
+  async capture(args, all) {
+    const [sub, ...rest] = (args || '').trim().split(/\s+/).filter(Boolean);
+    if (sub === 'on') return startCapture(rest);
+    if (sub === 'off' && !rest.length) return stopCapture(all);
+    if (sub) throw new Error('Usage: capture on [requests|console] [seconds] | capture off');
+    if (cap) {
+      out.log(`Capturing ${cap.label} on ${cap.page.url()} since ${clock(cap.startedAt)} (${cap.events.length} so far)`);
+      out.log(hints([['capture off', 'stop and print what it recorded']]));
+      return;
+    }
+    if (lastCapture) {
+      out.log(`Not capturing. The last capture, of ${lastCapture.label} from ${clock(lastCapture.startedAt)}:`);
+      printCapture(lastCapture, all);
+    } else {
+      out.log('Not capturing.');
+    }
+    out.log(hints([['capture on [requests|console] [seconds]', 'record the selected tab\'s requests and console messages in time order']]));
+  },
+};
+
+module.exports = { commands, currentCapture, endCapture };

package/lib/cdp.js

@@ -0,0 +1,31 @@
+// The CDP session kept for each tab, and keeping a background tab drawing.
+const { tabState } = require('./tabstate');
+
+// Per page: a CDP session's emulation applies to the page it is attached to
+// and resets, in part, when it detaches, so one is kept for each tab.
+function keptSession(p) {
+  if (!('session' in tabState(p))) tabState(p).session = p.context().newCDPSession(p).catch(error => { delete tabState(p).session; throw error; });
+  return tabState(p).session;
+}
+
+// Chrome all but stops drawing a tab that is not in front once it has had
+// input (about a frame a second), and Playwright waits for frames to see an
+// element hold still before acting on it, so each click in a tab opened in the
+// background took 1-2s. A screencast keeps the tab drawing without bringing it
+// to the front, for as long as the REPL runs (the throttle is back within
+// seconds of stopping it). Its frames are tiny and rare; the cost is that the
+// tab renders at full rate, as if in front, an animation or a video included.
+async function keepDrawing(p) {
+  if ('drawing' in tabState(p)) return;
+  tabState(p).drawing = true;
+  try {
+    const session = await keptSession(p);
+    session.on('Page.screencastFrame', frame => session.send('Page.screencastFrameAck', { sessionId: frame.sessionId }).catch(() => {}));
+    await session.send('Page.startScreencast', { format: 'jpeg', quality: 1, maxWidth: 8, maxHeight: 8, everyNthFrame: 60 });
+  } catch {
+    // Only slower without it.
+    delete tabState(p).drawing;
+  }
+}
+
+module.exports = { keptSession, keepDrawing };