@humanjs/playwright 0.5.0 → 0.7.0

package/README.md

@@ -1,5 +1,14 @@
 # @humanjs/playwright
 
+<p>
+  <a href="https://www.npmjs.com/package/@humanjs/playwright"><img alt="npm" src="https://img.shields.io/npm/v/@humanjs/playwright"></a>
+  <a href="https://www.npmjs.com/package/@humanjs/playwright"><img alt="downloads" src="https://img.shields.io/npm/dt/@humanjs/playwright"></a>
+  <a href="https://github.com/totigm/humanjs"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-totigm%2Fhumanjs-181717?logo=github"></a>
+  <a href="https://github.com/totigm/humanjs/actions/workflows/ci.yml"><img alt="CI" src="https://github.com/totigm/humanjs/actions/workflows/ci.yml/badge.svg"></a>
+  <a href="https://github.com/totigm/humanjs/blob/main/LICENSE"><img alt="license" src="https://img.shields.io/npm/l/@humanjs/playwright"></a>
+  <a href="https://humanjs.dev"><img alt="docs" src="https://img.shields.io/badge/docs-humanjs.dev-emerald"></a>
+</p>
+
 Humanize Playwright sessions for AI agents, QA tests, and demos. Drop-in adapter for an existing Playwright `Page`.
 
 ## Install
@@ -332,7 +341,32 @@ await rec.toTimeline('session.json');   // → JSON on disk
 const timeline = rec.timeline;          // → in-memory object
 ```
 
-The shape (`Timeline` with `personality`, `seed`, `speed`, `durationMs`, and an `events` array of `{ type, params, tMs, durationMs }`) is intended for observability pipelines, replay infrastructure, analytics, and debugger UIs. `toTimeline()` doesn't touch the browser context — call it before or after `toVideo()`, multiple times, in any order.
+The shape (`Timeline` with `personality`, `seed`, `speed`, `durationMs`, and an `events` array of `{ type, params, tMs, durationMs }`, plus `inputValue` on captured `type`/`paste` events) is intended for observability pipelines, replay infrastructure, analytics, and debugger UIs. `toTimeline()` doesn't touch the browser context — call it before or after `toVideo()`, multiple times, in any order.
+
+**Code export** — turn the same recording into runnable code:
+
+```ts
+await rec.toHumanJS('session.ts');         // standalone HumanJS script
+await rec.toPlaywright('session.spec.ts'); // @playwright/test spec (humanized)
+```
+
+`toHumanJS()` emits a standalone script (`createHuman` + `human.*`); `toPlaywright()` emits a `@playwright/test` spec that drives the page through HumanJS, so the generated test runs humanized too. String selectors round-trip verbatim. Both work on timeline-only recordings and are unaffected by `dispose()`.
+
+`toPlaywright()` also derives the assertions it safely can from the recording — a `read` implies its target was visible (`toBeVisible`), a captured input implies its value (`toHaveValue`) — and leaves a `TODO` for outcome assertions (URL changed, text appeared) that can't be inferred from actions alone. It never fabricates assertions that might fail on a correct run.
+
+Generated tests are built to *be tests*: they run `speed: process.env.CI ? 'instant' : '<recorded>'` (instant in CI, recorded feel locally) and drop recorded `sleep()` pauses (timing fidelity belongs in a demo, not a test — pass `toPlaywright(path, { keepSleeps: true })` to keep them). The test title comes from the recording's name (`human.record({ name })`) and is overridable with `{ title }`.
+
+Two more options: `{ steps: true }` groups the actions into `test.step(...)` blocks (a new step per navigation) for collapsible sections in the HTML report and trace; `{ baseUrl: true }` rewrites same-origin `goto`s to relative paths and adds a note to set `use.baseURL` in your `playwright.config.ts` — so the same test runs against local / staging / prod.
+
+By default the actual typed/pasted text is captured into the timeline (and the exported code). Values typed into `input[type="password"]` are always masked; set `captureInputs: false` to record none — exports then emit empty-string placeholders:
+
+```ts
+await human.record({ captureInputs: false }, fn);
+```
+
+> Captured input values land in the timeline JSON and any exported code — treat those artifacts with the same care as the values themselves.
+
+Two limits, by design: a target passed as a `Locator` or a raw `point(x, y)` doesn't round-trip to a clean selector (points are emitted verbatim with a flag comment — locator/point → selector synthesis is a planned follow-up), and reads driven by word-count or raw text emit a note instead of code.
 
 **Quality presets** trade off file size, encoding time, and visual fidelity. Defaults to `'high'`:
 
@@ -364,6 +398,39 @@ await rec.toTimeline('session.json');   // works
 
 Every recording is a regular plugin action — `beforeAction` and `afterAction` observe `{ type: 'record' }` exactly like `'click'` or `'scroll'`.
 
+## Using your own browser or a persistent profile
+
+`createHuman(page)` wraps **any** Playwright `Page` — so reusing a saved login, your installed Chrome, or an already-running browser is just a matter of how you create that page. HumanJS adds nothing special here; these are standard Playwright entry points, collected so you don't have to hunt for them.
+
+**Persistent profile** — keep cookies, local storage, and logins across runs. The first run signs in; later runs are already authenticated:
+
+```ts
+import { chromium, createHuman } from '@humanjs/playwright';
+
+const context = await chromium.launchPersistentContext('./.humanjs-profile', {
+  headless: false,
+  channel: 'chrome', // optional: use installed Google Chrome instead of bundled Chromium
+});
+const page = context.pages()[0] ?? (await context.newPage());
+
+const human = await createHuman(page, { personality: 'careful' });
+// …drive the page; state persists in ./.humanjs-profile for next time
+```
+
+**Attach to an already-running browser** — drive a Chrome you started yourself, with all its existing tabs, extensions, and sessions. Launch Chrome with a debugging port first (`chrome --remote-debugging-port=9222`), then:
+
+```ts
+import { chromium, createHuman } from '@humanjs/playwright';
+
+const browser = await chromium.connectOverCDP('http://localhost:9222');
+const context = browser.contexts()[0];
+const page = context.pages()[0] ?? (await context.newPage());
+
+const human = await createHuman(page, { personality: 'careful' });
+```
+
+> **Heads up:** a persistent profile or a connected real browser carries whatever you're signed into. Driving it means the automation can act with those sessions' privileges — keep that in mind for anything sensitive, and be wary of pages that try to manipulate an agent into actions while logged in.
+
 ## License
 
 MIT

package/dist/index.cjs

@@ -325,15 +325,22 @@ function clamp(value, min, max) {
 // src/mouse/index.ts
 async function executeClick(target, ctx, options = {}) {
   const button = options.button ?? "left";
-  const locator = typeof target === "string" ? ctx.page.locator(target) : target;
   if (ctx.speed === "instant") {
-    const box = await locator.boundingBox();
+    if (isPoint(target)) {
+      await ctx.page.mouse.click(target.x, target.y, { button });
+      ctx.setMousePosition(target);
+      return { target };
+    }
+    const locator = typeof target === "string" ? ctx.page.locator(target) : target;
+    const box2 = await locator.boundingBox();
     await locator.click({ button });
-    const center = box ? { x: box.x + box.width / 2, y: box.y + box.height / 2 } : ctx.getMousePosition();
+    const center = box2 ? { x: box2.x + box2.width / 2, y: box2.y + box2.height / 2 } : ctx.getMousePosition();
     ctx.setMousePosition(center);
     return { target: center };
   }
-  const targetPoint = await moveToTarget(target, ctx, "click");
+  const { point: targetPoint, box } = await resolveTargetPointAndBox(target, ctx, "click");
+  await maybeMisclickBeat(ctx, box, targetPoint);
+  await walkBezierTo(targetPoint, ctx);
   const preClickMs = computeDwellTime(
     ctx.personality.dwell.preClickMs,
     ctx.personality.dwell.preClickJitter,
@@ -362,7 +369,7 @@ async function executeHover(target, ctx) {
     ctx.setMousePosition(center);
     return { target: center };
   }
-  const targetPoint = await moveToTarget(target, ctx, "hover");
+  const targetPoint = await moveToTarget(target, ctx);
   const dwellMs = computeDwellTime(
     ctx.personality.dwell.preClickMs,
     ctx.personality.dwell.preClickJitter,
@@ -445,10 +452,9 @@ async function executeMove(target, ctx) {
   ctx.setMousePosition(point);
   return { target: point };
 }
-async function moveToTarget(target, ctx, action) {
-  const box = await readBoxWithAutoScroll(target, ctx, action);
+async function moveToTarget(target, ctx) {
+  const box = await readBoxWithAutoScroll(target, ctx, "hover");
   const targetPoint = pickClickPoint(box, ctx.rng, ctx.personality.mouse.clickSpread);
-  if (action === "click") await maybeMisclickBeat(ctx, box, targetPoint);
   await walkBezierTo(targetPoint, ctx);
   return targetPoint;
 }
@@ -711,6 +717,205 @@ async function detectKindFromTag(locator) {
   if (tag === "pre" || tag === "code") return "code";
   return void 0;
 }
+
+// src/recording/codegen.ts
+var POINT_RE = /^point\((-?\d+(?:\.\d+)?),\s*(-?\d+(?:\.\d+)?)\)$/;
+var POINT_COMMENT = " // raw coordinate \u2014 replace with a locator for a stable selector";
+var UNCAPTURED_COMMENT = " // input not captured (masked or captureInputs disabled) \u2014 fill in (e.g. process.env.X)";
+function q(value) {
+  return `'${String(value ?? "").replace(/\\/g, "\\\\").replace(/'/g, "\\'").replace(/\n/g, "\\n").replace(/\r/g, "\\r")}'`;
+}
+function targetArg(desc) {
+  const s = String(desc ?? "");
+  const m = s.match(POINT_RE);
+  if (m) return { code: `{ x: ${m[1]}, y: ${m[2]} }`, isPoint: true };
+  return { code: q(s), isPoint: false };
+}
+function createHumanOptions(timeline, ciSpeed = false) {
+  const parts = [`    personality: ${q(timeline.personality)},`];
+  if (timeline.seed !== null) parts.push(`    seed: ${q(timeline.seed)},`);
+  parts.push(
+    ciSpeed ? `    speed: process.env.CI ? 'instant' : ${q(timeline.speed)},` : `    speed: ${q(timeline.speed)},`
+  );
+  return `{
+${parts.join("\n")}
+  }`;
+}
+function emitScroll(target) {
+  const s = String(target ?? "natural");
+  if (s === "natural") return "  await human.scroll('natural');";
+  const by = s.match(/^by:(-?\d+(?:\.\d+)?)$/);
+  if (by) return `  await human.scroll({ by: ${by[1]} });`;
+  const to = s.match(/^to:(-?\d+(?:\.\d+)?)$/);
+  if (to) return `  await human.scroll({ to: ${to[1]} });`;
+  return `  await human.scroll(${q(s)});`;
+}
+function emitAction(e, opts = {}) {
+  const p = e.params;
+  switch (e.type) {
+    case "goto": {
+      const url = String(p.url ?? "");
+      if (opts.baseOrigin && url.startsWith(opts.baseOrigin)) {
+        return `  await human.goto(${q(url.slice(opts.baseOrigin.length) || "/")});`;
+      }
+      return `  await human.goto(${q(url)});`;
+    }
+    case "click":
+    case "rightClick":
+    case "hover":
+    case "move": {
+      const { code, isPoint: isPoint2 } = targetArg(p.target);
+      return `  await human.${e.type}(${code});${isPoint2 ? POINT_COMMENT : ""}`;
+    }
+    case "drag": {
+      const from = targetArg(p.from);
+      const to = targetArg(p.to);
+      const comment = from.isPoint || to.isPoint ? POINT_COMMENT : "";
+      return `  await human.drag(${from.code}, ${to.code});${comment}`;
+    }
+    case "type":
+    case "paste": {
+      const { code, isPoint: isPoint2 } = targetArg(p.target);
+      if (e.inputValue === void 0) {
+        return `  await human.${e.type}(${code}, '');${UNCAPTURED_COMMENT}`;
+      }
+      const call = `  await human.${e.type}(${code}, ${q(e.inputValue)});`;
+      if (opts.asserts && !isPoint2) {
+        return `${call}
+  await expect(page.locator(${code})).toHaveValue(${q(e.inputValue)});`;
+      }
+      return call;
+    }
+    case "press":
+      return `  await human.press(${q(p.key)});`;
+    case "scroll":
+      return emitScroll(p.target);
+    case "read": {
+      const desc = String(p.target ?? "");
+      if (/^\d+ words$/.test(desc) || /^text:\d+ chars$/.test(desc)) {
+        return `  // human.read(...) \u2014 ${desc}; original target not captured`;
+      }
+      const call = `  await human.read(${q(desc)});`;
+      if (opts.asserts) return `${call}
+  await expect(page.locator(${q(desc)})).toBeVisible();`;
+      return call;
+    }
+    case "sleep":
+      return `  await sleep(${Number(p.ms) || 0});`;
+    case "reload":
+      return "  await human.reload();";
+    case "goBack":
+      return "  await human.goBack();";
+    case "goForward":
+      return "  await human.goForward();";
+    default:
+      return `  // unsupported action: ${e.type}`;
+  }
+}
+function needsSleepImport(timeline) {
+  return timeline.events.some((e) => e.type === "sleep");
+}
+function generateHumanJS(timeline) {
+  const imports = needsSleepImport(timeline) ? "import { chromium, createHuman, sleep } from '@humanjs/playwright';" : "import { chromium, createHuman } from '@humanjs/playwright';";
+  const body = timeline.events.map((e) => emitAction(e)).join("\n");
+  return `${imports}
+
+async function main() {
+  const browser = await chromium.launch({ headless: false });
+  const page = await browser.newPage();
+  const human = await createHuman(page, ${createHumanOptions(timeline)});
+
+${body}
+
+  await browser.close();
+}
+
+main();
+`;
+}
+var NAV_TYPES = /* @__PURE__ */ new Set(["goto", "reload", "goBack", "goForward"]);
+function sharedGotoOrigin(events) {
+  let origin;
+  for (const e of events) {
+    if (e.type !== "goto") continue;
+    try {
+      const o = new URL(String(e.params.url ?? "")).origin;
+      if (origin === void 0) origin = o;
+      else if (origin !== o) return void 0;
+    } catch {
+      return void 0;
+    }
+  }
+  return origin;
+}
+function indentLines(block, pad) {
+  return block.split("\n").map((line) => line.length > 0 ? pad + line : line).join("\n");
+}
+function stepLabel(event, index, baseOrigin) {
+  switch (event.type) {
+    case "goto": {
+      const url = String(event.params.url ?? "");
+      const path = baseOrigin && url.startsWith(baseOrigin) ? url.slice(baseOrigin.length) || "/" : url;
+      return `go to ${path}`;
+    }
+    case "reload":
+      return "reload";
+    case "goBack":
+      return "go back";
+    case "goForward":
+      return "go forward";
+    default:
+      return `step ${index + 1}`;
+  }
+}
+function emitSteps(events, opts) {
+  const groups = [];
+  for (const e of events) {
+    const last = groups[groups.length - 1];
+    if (last === void 0 || NAV_TYPES.has(e.type)) groups.push([e]);
+    else last.push(e);
+  }
+  return groups.map((group, i) => {
+    const [first] = group;
+    const label = first ? stepLabel(first, i, opts.baseOrigin) : `step ${i + 1}`;
+    const inner = group.map((e) => indentLines(emitAction(e, opts), "  ")).join("\n");
+    return `  await test.step(${q(label)}, async () => {
+${inner}
+  });`;
+  }).join("\n\n");
+}
+function generatePlaywrightTest(timeline, options = {}) {
+  const events = options.keepSleeps ? timeline.events : timeline.events.filter((e) => e.type !== "sleep");
+  const baseOrigin = options.baseUrl ? sharedGotoOrigin(events) : void 0;
+  const emitOpts = { asserts: true, baseOrigin };
+  const body = options.steps ? emitSteps(events, emitOpts) : events.map((e) => emitAction(e, emitOpts)).join("\n");
+  const needsSleep = events.some((e) => e.type === "sleep");
+  const hasAsserts = body.includes("await expect(");
+  const testImport = hasAsserts ? "import { expect, test } from '@playwright/test';" : "import { test } from '@playwright/test';";
+  const humanImport = needsSleep ? "import { createHuman, sleep } from '@humanjs/playwright';" : "import { createHuman } from '@humanjs/playwright';";
+  const title = options.title ?? timeline.name ?? "recorded session";
+  const baseUrlNote = baseOrigin ? `  // Set use.baseURL = ${q(baseOrigin)} in playwright.config.ts for these relative paths.
+
+` : "";
+  const todo = [
+    hasAsserts ? "  // TODO: add assertions for the outcome of this flow, e.g.:" : "  // TODO: assert the outcome \u2014 import { expect } from '@playwright/test', e.g.:",
+    "  //   await expect(page).toHaveURL(/dashboard/);",
+    "  //   await expect(page.getByText('Welcome back')).toBeVisible();"
+  ].join("\n");
+  return `${testImport}
+${humanImport}
+
+test(${q(title)}, async ({ page }) => {
+  const human = await createHuman(page, ${createHumanOptions(timeline, true)});
+
+${baseUrlNote}${body}
+
+${todo}
+});
+`;
+}
+
+// src/recording/index.ts
 var pendingFrameCleanups = /* @__PURE__ */ new Set();
 var exitHandlerInstalled = false;
 function ensureExitHandler() {
@@ -806,6 +1011,7 @@ var Recording = class {
   get timeline() {
     return {
       version: 1,
+      ...this.#timelineSource.name !== void 0 ? { name: this.#timelineSource.name } : {},
       personality: this.#timelineSource.personality,
       seed: this.#timelineSource.seed,
       speed: this.#timelineSource.speed,
@@ -956,6 +1162,38 @@ var Recording = class {
 `, "utf8");
     return outputPath;
   }
+  /**
+   * Generates a standalone, runnable HumanJS script from the timeline and
+   * writes it to `outputPath`. String selectors round-trip verbatim; typed
+   * values are included when `captureInputs` was on (passwords masked).
+   *
+   * Independent of frame capture — works on timeline-only recordings and is
+   * unaffected by `dispose()`.
+   *
+   * @returns the resolved output path.
+   */
+  async toHumanJS(outputPath) {
+    await promises.mkdir(path.dirname(outputPath), { recursive: true });
+    await promises.writeFile(outputPath, generateHumanJS(this.timeline), "utf8");
+    return outputPath;
+  }
+  /**
+   * Generates a `@playwright/test` spec from the timeline — a humanized test
+   * (uses `createHuman` + `human.*`), not raw Playwright — and writes it to
+   * `outputPath`. Runs instant in CI / recorded speed locally, drops timing
+   * `sleep()`s (pass `{ keepSleeps: true }` to keep them), and derives the
+   * assertions it safely can.
+   *
+   * Independent of frame capture — works on timeline-only recordings and is
+   * unaffected by `dispose()`.
+   *
+   * @returns the resolved output path.
+   */
+  async toPlaywright(outputPath, options) {
+    await promises.mkdir(path.dirname(outputPath), { recursive: true });
+    await promises.writeFile(outputPath, generatePlaywrightTest(this.timeline, options), "utf8");
+    return outputPath;
+  }
   /**
    * Releases the captured-frames temp directory. After this call, `toVideo()`
    * and `toGif()` throw — but `toTimeline()` and the in-memory `timeline`
@@ -1227,7 +1465,8 @@ async function createHuman(page, options = {}) {
   let hasRecorded = false;
   let activeRecordingEvents = null;
   let activeRecordingStartMs = 0;
-  async function performAction(action, actionFn) {
+  let activeRecordingCaptureInputs = false;
+  async function performAction(action, actionFn, recordMeta) {
     for (const plugin of plugins) {
       await plugin.beforeAction?.(action);
     }
@@ -1241,7 +1480,8 @@ async function createHuman(page, options = {}) {
           type: action.type,
           params: action.params ?? {},
           tMs: startedAt - activeRecordingStartMs,
-          durationMs
+          durationMs,
+          ...recordMeta?.inputValue !== void 0 ? { inputValue: recordMeta.inputValue } : {}
         });
       }
       for (const plugin of plugins) {
@@ -1255,7 +1495,8 @@ async function createHuman(page, options = {}) {
           params: action.params ?? {},
           tMs: startedAt - activeRecordingStartMs,
           durationMs: Date.now() - startedAt,
-          error: error instanceof Error ? error.message : String(error)
+          error: error instanceof Error ? error.message : String(error),
+          ...recordMeta?.inputValue !== void 0 ? { inputValue: recordMeta.inputValue } : {}
         });
       }
       for (const plugin of plugins) {
@@ -1282,6 +1523,21 @@ async function createHuman(page, options = {}) {
     }
     return target.toString?.() ?? "locator";
   };
+  const isPointTarget = (target) => typeof target !== "string" && "x" in target && "y" in target && typeof target.x === "number";
+  const captureInputValue = async (target, value) => {
+    if (activeRecordingEvents === null || !activeRecordingCaptureInputs) return void 0;
+    const locator = typeof target === "string" ? page.locator(target) : isPointTarget(target) ? null : target;
+    if (locator !== null) {
+      try {
+        const fieldType = await locator.first().getAttribute("type", { timeout: 1e3 });
+        if (fieldType?.toLowerCase() === "password") {
+          return void 0;
+        }
+      } catch {
+      }
+    }
+    return value;
+  };
   return {
     personality,
     speed,
@@ -1323,6 +1579,7 @@ async function createHuman(page, options = {}) {
     },
     async type(target, value) {
       const description = describeMouseTarget(target);
+      const inputValue = await captureInputValue(target, value);
       await performAction(
         { type: "type", params: { target: description, length: value.length } },
         async () => {
@@ -1330,11 +1587,13 @@ async function createHuman(page, options = {}) {
             await executeClick(target, mouseCtx());
           }
           await executeType(target, value, { page, personality, rng, speed });
-        }
+        },
+        inputValue !== void 0 ? { inputValue } : void 0
       );
     },
     async paste(target, value) {
       const description = describeMouseTarget(target);
+      const inputValue = await captureInputValue(target, value);
       await performAction(
         { type: "paste", params: { target: description, length: value.length } },
         async () => {
@@ -1342,7 +1601,8 @@ async function createHuman(page, options = {}) {
             await executeClick(target, mouseCtx());
           }
           await executePaste(target, value, { page, personality, rng, speed });
-        }
+        },
+        inputValue !== void 0 ? { inputValue } : void 0
       );
     },
     async press(key) {
@@ -1411,6 +1671,7 @@ async function createHuman(page, options = {}) {
       const windowStartMs = Date.now();
       activeRecordingEvents = events;
       activeRecordingStartMs = windowStartMs;
+      activeRecordingCaptureInputs = recordOptions.captureInputs !== false;
       let windowEndMs = windowStartMs;
       try {
         await performAction({ type: "record", params: {} }, async () => {
@@ -1425,14 +1686,62 @@ async function createHuman(page, options = {}) {
         throw error;
       } finally {
         activeRecordingEvents = null;
+        activeRecordingCaptureInputs = false;
       }
       const captureResult = captureSession ? await captureSession.stop() : null;
       return new Recording(captureResult, windowStartMs, windowEndMs, {
+        name: recordOptions.name,
         personality: personality.name,
         seed: options.seed === void 0 ? null : String(options.seed),
         speed,
         events
       });
+    },
+    // ────────────────────────────────────────────────────────────────────
+    // Thin re-exports of common Playwright `Page` methods. See the `Human`
+    // interface for the rationale; implementations forward unchanged.
+    // ────────────────────────────────────────────────────────────────────
+    screenshot(opts) {
+      return page.screenshot(opts);
+    },
+    pageText() {
+      return page.innerText("body");
+    },
+    content() {
+      return page.content();
+    },
+    url() {
+      return page.url();
+    },
+    title() {
+      return page.title();
+    },
+    async reload(opts) {
+      await performAction({ type: "reload", params: {} }, async () => {
+        await page.reload(opts);
+      });
+    },
+    async goBack(opts) {
+      await performAction({ type: "goBack", params: {} }, async () => {
+        await page.goBack(opts);
+      });
+    },
+    async goForward(opts) {
+      await performAction({ type: "goForward", params: {} }, async () => {
+        await page.goForward(opts);
+      });
+    },
+    waitForLoadState(state, opts) {
+      return page.waitForLoadState(state, opts);
+    },
+    waitForURL(url, opts) {
+      return page.waitForURL(url, opts);
+    },
+    setViewportSize(size) {
+      return page.setViewportSize(size);
+    },
+    pdf(opts) {
+      return page.pdf(opts);
     }
   };
 }