libretto 0.5.0 → 0.5.2

package/README.md

@@ -1,70 +1,134 @@
 # Libretto
 
-Libretto gives your coding agent superpowers for building, debugging, and maintaining browser RPA integrations.
+[![npm version](https://img.shields.io/npm/v/libretto)](https://www.npmjs.com/package/libretto)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+[![GitHub Discussions](https://img.shields.io/github/discussions/saffron-health/libretto)](https://github.com/saffron-health/libretto/discussions)
 
-It is designed for engineering teams that automate workflows in web apps and want to move from brittle browser-only scripts to faster, more reliable network-first integrations.
+Libretto is a toolkit for building robust web integrations. It gives your coding agent a live browser and a token-efficient CLI to:
 
-## Installation
+- Inspect live pages with minimal context overhead
+- Capture network traffic to reverse-engineer site APIs
+- Record user actions and replay them as automation scripts
+- Debug broken workflows interactively against the real site
 
-```bash
-npm install --save-dev libretto
-```
+We at [Saffron Health](https://saffron.health) built Libretto to help us maintain our browser integrations to common healthcare software. We're open-sourcing it so other teams have an easier time doing the same thing.
 
-Chromium is downloaded automatically via a `postinstall` script. If postinstall scripts are disabled (e.g. `--ignore-scripts`, common in monorepos), run init manually:
+https://github.com/user-attachments/assets/9b9a0ab3-5133-4b20-b3be-459943349d18
+
+## Installation
 
 ```bash
+npm install libretto
+
+# Install skill, download Chromium if not already installed, configure snapshot analysis
 npx libretto init
-```
 
-This installs the Chromium browser binary and optionally configures an AI subagent (Gemini, Claude, or Codex) that can analyze page snapshots without consuming the coding agent's context window.
+# Configure or change the snapshot analysis model (see Configuration section below). `npx libretto init` sets this up the first time.
+npx libretto ai configure <openai | anthropic | gemini | vertex>
+```
 
-## Usage
+## Use cases
 
-Libretto is usually used through prompts with the Libretto skill.
+Libretto is designed to be used as a skill through your coding agent. Here are some example prompts:
 
 ### One-shot script generation
 
-```text
-Use the Libretto skill. Go on LinkedIn and scrape the first 10 posts for content, who posted it, the number of reactions, the first 25 comments, and the first 25 reposts.
-```
+> Use the Libretto skill. Go on LinkedIn and scrape the first 10 posts for content, who posted it, the number of reactions, the first 25 comments, and the first 25 reposts.
+
+Your coding agent will open a window for you to log into LinkedIn, and then automatically start exploring.
 
 ### Interactive script building
 
-```text
-Use the Libretto skill. Let's interactively build a script to scrape scheduling info from the eClinicalWorks EHR.
-```
+> I'm gonna show you a workflow in the eclinicalworks EHR to get a patient's primary insurance ID. Use libretto skill to turn it into a playwright script that takes patient name and dob as input to get back the insurance ID. URL is ...
+
+Libretto can read your actions you perform in the browser, so you can perform a workflow, then ask it to use your actions to rebuild the workflow.
 
 ### Convert browser automation to network requests
 
-```text
-We have a browser script at ./integration.ts that automates going to Hacker News and getting the first 10 posts. Convert it to direct network scripts instead. Use the Libretto skill.
-```
+> We have a browser script at ./integration.ts that automates going to Hacker News and getting the first 10 posts. Convert it to direct network scripts instead. Use the Libretto skill.
+
+Libretto can read network requests from the browser, which it can use to reverse engineer the API and create a script that directly calls those requests. Directly making API calls is faster, and more reliable, than UI automation. You can also ask Libretto to conduct a security analysis which analyzes the requests for common security cookies, so you can understand whether a network request approach will be safe.
 
 ### Fix broken integrations
 
-```text
-We have a browser script at ./integration.ts that is supposed to go to Availity and perform an eligibility check for a patient. But I'm getting a broken selector error when I run it. Fix it. Use the Libretto skill.
+> We have a browser script at ./integration.ts that is supposed to go to Availity and perform an eligibility check for a patient. But I'm getting a broken selector error when I run it. Fix it. Use the Libretto skill.
+
+Agents can use Libretto to reproduce the failure, pause the workflow at any point, inspect the live page, and fix issues, all autonomously.
+
+### CLI usage
+
+You can also use Libretto directly from the command line. All commands accept `--session <name>` to target a specific session.
+
+```bash
+npx libretto init                          # interactive; run yourself, not through an agent
+npx libretto open <url>                    # launch browser and open a URL (headed by default)
+npx libretto snapshot --objective "..." --context "..."  # capture PNG + HTML and analyze with an LLM
+npx libretto exec "<code>"                 # execute Playwright TypeScript against the open page
+npx libretto run <file> <export>           # run an exported workflow from a file
+npx libretto resume                        # resume a paused workflow
+npx libretto network                       # view captured network requests
+npx libretto actions                       # view captured user/agent actions
+npx libretto pages                         # list open pages in the session
+npx libretto save <domain>                 # save browser session (cookies, localStorage) for reuse
+npx libretto close                         # close the browser
+npx libretto ai configure <provider>       # configure snapshot analysis model
 ```
 
-You can also run workflows directly from the CLI:
+## Configuration
+
+All Libretto state lives in a `.libretto/` directory at your project root. Configuration is stored in `.libretto/config.json`.
+
+### Config file
+
+`.libretto/config.json` controls snapshot analysis and viewport settings:
+
+```json
+{
+  "version": 1,
+  "ai": {
+    "model": "openai/gpt-5.4",
+    "updatedAt": "2026-01-01T00:00:00.000Z"
+  },
+  "viewport": { "width": 1280, "height": 800 }
+}
+```
+
+The `ai` field configures which model Libretto uses for snapshot analysis — extracting selectors, identifying interactive elements, or diagnosing why a step failed. This keeps heavy visual context out of your coding agent's context window. Snapshot analysis is required.
+
+The easiest way to set the model is through the CLI:
 
 ```bash
-npx libretto help
-npx libretto run ./integration.ts main
+npx libretto ai configure <openai | anthropic | gemini | vertex>
 ```
 
-## The `.libretto/` directory
+Provider credentials are read from environment variables or a `.env` file at your project root: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY`, or `GOOGLE_CLOUD_PROJECT` for Vertex.
+
+The `viewport` field sets the default browser viewport size. Both fields are optional.
+
+### Sessions
 
-Libretto stores local runtime state in a `.libretto/` directory at your project root. Sensitive directories (`sessions/` and `profiles/`) are automatically git-ignored via `.libretto/.gitignore`.
+Each Libretto session gets its own directory under `.libretto/sessions/<name>/` containing runtime state. Sessions are git-ignored.
 
-- **`profiles/<domain>.json`** — Saved browser sessions (cookies, localStorage) for authenticated sites. Created via `npx libretto save <domain>`. Machine-local and never committed.
-- **`sessions/<name>/`** — Per-session runtime state:
-  - `state.json` — Session metadata (debug port, PID, status)
-  - `logs.jsonl` — Structured session logs
-  - `network.jsonl` — Captured network requests (URLs, methods, headers, response status)
-  - `actions.jsonl` — Recorded user actions (clicks, fills, navigations)
-  - `snapshots/` — Screenshot PNGs and HTML snapshots captured via `npx libretto snapshot`
-- **`ai.json`** — AI runtime configuration set via `npx libretto ai configure`.
+- `state.json` — session metadata (debug port, PID, status)
+- `logs.jsonl` — structured session logs
+- `network.jsonl` — captured network requests
+- `actions.jsonl` — recorded user actions
+- `snapshots/` — screenshot PNGs and HTML snapshots
+
+### Profiles
+
+Profiles save browser sessions (cookies, localStorage) so you can reuse authenticated state across runs. They are stored in `.libretto/profiles/<domain>.json`, created via `npx libretto save <domain>`. Profiles are machine-local and git-ignored.
+
+## Community
+
+Have a question, idea, or want to share what you've built? Join the conversation on [GitHub Discussions](https://github.com/saffron-health/libretto/discussions).
+
+- **[Q&A](https://github.com/saffron-health/libretto/discussions/categories/q-a)** — Ask questions and get help
+- **[Ideas](https://github.com/saffron-health/libretto/discussions/categories/ideas)** — Suggest new features or improvements
+- **[Show and tell](https://github.com/saffron-health/libretto/discussions/categories/show-and-tell)** — Share your workflows and automations
+- **[General](https://github.com/saffron-health/libretto/discussions/categories/general)** — Chat about anything Libretto-related
+
+Found a bug? Please [open an issue](https://github.com/saffron-health/libretto/issues/new).
 
 ## Authors
 
@@ -80,3 +144,13 @@ pnpm build
 pnpm type-check
 pnpm test
 ```
+
+Source layout:
+
+- `src/cli/` — CLI commands
+- `src/runtime/` — browser runtime (network, recovery, downloads, extraction)
+- `src/shared/` — shared utilities (config, LLM client, logging, state)
+- `test/` — test files (`*.spec.ts`)
+- `skills/libretto/` — source of truth for the Libretto skill; mirrors are synced on `pnpm i`
+
+To check that skill mirrors are in sync without fixing them, run `pnpm check:skills`. To release, run `pnpm prepare-release`.

package/dist/cli/cli.js

@@ -1,19 +1,10 @@
-import {
-  closeLogger,
-  createLoggerForSession,
-  ensureLibrettoSetup
-} from "./core/context.js";
-import {
-  SESSION_DEFAULT,
-  validateSessionName
-} from "./core/session.js";
+import { ensureLibrettoSetup } from "./core/context.js";
 import { createCLIApp } from "./router.js";
 function renderUsage(app) {
   return `${app.renderHelp()}
 
 Options:
-  --session <name>        Use a named session (default: "default")
-                          Built-in sessions: default, dev-server, browser-agent
+  --session <name>        Use a named session (auto-generated for open/run if omitted)
 
 Examples:
   libretto open https://linkedin.com
@@ -31,7 +22,7 @@ Examples:
   libretto ai configure openai/gpt-4o
   libretto snapshot
   libretto snapshot --objective "Find the submit button" --context "Submitting a referral form, already filled in patient details"
-  libretto resume --session default
+  libretto resume --session my-session
   libretto close
   libretto close --all
   libretto close --all --force
@@ -56,67 +47,6 @@ Sessions:
   Each session runs an isolated browser instance on a dynamic port.
 `;
 }
-function readSessionArgBeforePassthrough(rawArgs) {
-  for (let index = 0; index < rawArgs.length; index += 1) {
-    const token = rawArgs[index];
-    if (token === "--") return void 0;
-    if (token === "--session") {
-      const value2 = rawArgs[index + 1];
-      if (!value2 || value2 === "--" || value2.startsWith("--")) {
-        return null;
-      }
-      return value2;
-    }
-    if (!token.startsWith("--session=")) continue;
-    const value = token.slice("--session=".length);
-    if (value.length === 0 || value === "--" || value.startsWith("--")) {
-      return null;
-    }
-    return value;
-  }
-  return void 0;
-}
-function parseSessionForLog(rawArgs) {
-  const value = readSessionArgBeforePassthrough(rawArgs);
-  if (value === void 0 || value === null) {
-    return SESSION_DEFAULT;
-  }
-  try {
-    validateSessionName(value);
-    return value;
-  } catch {
-    return SESSION_DEFAULT;
-  }
-}
-function validateLegacySessionArg(rawArgs) {
-  const value = readSessionArgBeforePassthrough(rawArgs);
-  if (value === void 0) return;
-  if (value === null) {
-    throw new Error(
-      `Usage: libretto <command> [--session <name>]
-Missing or invalid --session value.`
-    );
-  }
-  validateSessionName(value);
-}
-function initializeLogger(rawArgs) {
-  const sessionForLog = parseSessionForLog(rawArgs);
-  const logger = createLoggerForSession(sessionForLog);
-  logger.info("cli-start", {
-    args: rawArgs,
-    cwd: process.cwd(),
-    session: sessionForLog
-  });
-  return logger;
-}
-async function withCliLogger(rawArgs, run) {
-  const logger = initializeLogger(rawArgs);
-  try {
-    return await run(logger);
-  } finally {
-    await closeLogger(logger);
-  }
-}
 function isRootHelpRequest(rawArgs) {
   if (rawArgs.length === 0) return true;
   if (rawArgs[0] === "--help" || rawArgs[0] === "-h") return true;
@@ -126,32 +56,27 @@ async function runLibrettoCLI() {
   const rawArgs = process.argv.slice(2);
   let exitCode = 0;
   ensureLibrettoSetup();
-  await withCliLogger(rawArgs, async (logger) => {
-    const app = createCLIApp(logger);
-    try {
-      validateLegacySessionArg(rawArgs);
-      if (isRootHelpRequest(rawArgs)) {
-        console.log(renderUsage(app));
-        return;
-      }
-      logger.info("cli-command", { args: rawArgs });
-      const result = await app.run(rawArgs);
-      if (typeof result === "string") {
-        console.log(result);
-      }
-    } catch (err) {
-      logger.error("cli-error", { error: err, args: rawArgs });
-      const message = err instanceof Error ? err.message : String(err);
-      if (message.startsWith("Unknown command: ")) {
-        console.error(`${message}
+  const app = createCLIApp();
+  try {
+    if (isRootHelpRequest(rawArgs)) {
+      console.log(renderUsage(app));
+      return;
+    }
+    const result = await app.run(rawArgs);
+    if (typeof result === "string") {
+      console.log(result);
+    }
+  } catch (err) {
+    const message = err instanceof Error ? err.message : String(err);
+    if (message.startsWith("Unknown command: ")) {
+      console.error(`${message}
 `);
-        console.log(renderUsage(app));
-      } else {
-        console.error(message);
-      }
-      exitCode = 1;
+      console.log(renderUsage(app));
+    } else {
+      console.error(message);
     }
-  });
+    exitCode = 1;
+  }
   process.exit(exitCode);
 }
 export {

package/dist/cli/commands/browser.js

@@ -2,17 +2,21 @@ import { z } from "zod";
 import {
   runClose as runCloseWithLogger,
   runCloseAll as runCloseAllWithLogger,
+  runConnect as runConnectWithLogger,
   runOpen,
   runPages,
   runSave
 } from "../core/browser.js";
-import { withSessionLogger } from "../core/context.js";
-import { assertSessionAvailableForStart } from "../core/session.js";
+import { createLoggerForSession, withSessionLogger } from "../core/context.js";
+import {
+  assertSessionAvailableForStart,
+  validateSessionName
+} from "../core/session.js";
 import { SimpleCLI } from "../framework/simple-cli.js";
 import {
-  loadSessionStateMiddleware,
-  resolveSessionMiddleware,
-  sessionOption
+  sessionOption,
+  withAutoSession,
+  withRequiredSession
 } from "./shared.js";
 function parseViewportArg(viewportArg) {
   if (!viewportArg) return void 0;
@@ -52,16 +56,32 @@ const openInput = SimpleCLI.input({
   (input) => !(input.headed && input.headless),
   "Cannot pass both --headed and --headless."
 );
-function createOpenCommand(logger) {
-  return SimpleCLI.command({
-    description: "Launch browser and open URL (headed by default)"
-  }).input(openInput).use(resolveSessionMiddleware).handle(async ({ input, ctx }) => {
-    assertSessionAvailableForStart(ctx.session, logger);
-    const headed = input.headed || !input.headless;
-    const viewport = parseViewportArg(input.viewport);
-    await runOpen(input.url, headed, ctx.session, logger, { viewport });
-  });
-}
+const openCommand = SimpleCLI.command({
+  description: "Launch browser and open URL (headed by default)"
+}).input(openInput).use(withAutoSession()).handle(async ({ input, ctx }) => {
+  assertSessionAvailableForStart(ctx.session, ctx.logger);
+  const headed = input.headed || !input.headless;
+  const viewport = parseViewportArg(input.viewport);
+  await runOpen(input.url, headed, ctx.session, ctx.logger, { viewport });
+});
+const connectInput = SimpleCLI.input({
+  positionals: [
+    SimpleCLI.positional("cdpUrl", z.string().optional(), {
+      help: "CDP endpoint URL (e.g. http://127.0.0.1:9222)"
+    })
+  ],
+  named: {
+    session: sessionOption()
+  }
+}).refine(
+  (input) => Boolean(input.cdpUrl),
+  `Usage: libretto connect <cdp-url> --session <name>`
+);
+const connectCommand = SimpleCLI.command({
+  description: "Connect to an existing Chrome DevTools Protocol (CDP) endpoint"
+}).input(connectInput).use(withAutoSession()).handle(async ({ input, ctx }) => {
+  await runConnectWithLogger(input.cdpUrl, ctx.session, ctx.logger);
+});
 const saveInput = SimpleCLI.input({
   positionals: [
     SimpleCLI.positional("urlOrDomain", z.string().optional(), {
@@ -73,72 +93,79 @@ const saveInput = SimpleCLI.input({
   }
 }).refine(
   (input) => Boolean(input.urlOrDomain),
-  `Usage: libretto save <url|domain> [--session <name>]`
+  `Usage: libretto save <url|domain> --session <name>`
 );
-function createSaveCommand(logger) {
-  return SimpleCLI.command({
-    description: "Save current browser session"
-  }).input(saveInput).use(resolveSessionMiddleware).use(loadSessionStateMiddleware).handle(async ({ input, ctx }) => {
-    await runSave(input.urlOrDomain, ctx.session, logger);
-  });
-}
+const saveCommand = SimpleCLI.command({
+  description: "Save current browser session"
+}).input(saveInput).use(withRequiredSession()).handle(async ({ input, ctx }) => {
+  await runSave(input.urlOrDomain, ctx.session, ctx.logger);
+});
 const pagesInput = SimpleCLI.input({
   positionals: [],
   named: {
     session: sessionOption()
   }
 });
-function createPagesCommand(logger) {
-  return SimpleCLI.command({
-    description: "List open pages in the session"
-  }).input(pagesInput).use(resolveSessionMiddleware).use(loadSessionStateMiddleware).handle(async ({ ctx }) => {
-    await runPages(ctx.session, logger);
-  });
-}
+const pagesCommand = SimpleCLI.command({
+  description: "List open pages in the session"
+}).input(pagesInput).use(withRequiredSession()).handle(async ({ ctx }) => {
+  await runPages(ctx.session, ctx.logger);
+});
 const closeInput = SimpleCLI.input({
   positionals: [],
   named: {
     session: sessionOption(),
-    all: SimpleCLI.flag({ help: "Close all tracked sessions in this workspace" }),
-    force: SimpleCLI.flag({ help: "Force kill sessions that ignore SIGTERM (requires --all)" })
+    all: SimpleCLI.flag({
+      help: "Close all tracked sessions in this workspace"
+    }),
+    force: SimpleCLI.flag({
+      help: "Force kill sessions that ignore SIGTERM (requires --all)"
+    })
+  }
+}).refine(
+  (input) => input.all || input.session,
+  `Usage: libretto close --session <name>
+Usage: libretto close --all [--force]`
+);
+const closeCommand = SimpleCLI.command({
+  description: "Close the browser"
+}).input(closeInput).handle(async ({ input }) => {
+  if (input.force && !input.all) {
+    throw new Error(`Usage: libretto close --all [--force]`);
   }
+  if (input.all) {
+    const logger2 = createLoggerForSession("cli");
+    await runCloseAllWithLogger(logger2, { force: input.force });
+    return;
+  }
+  validateSessionName(input.session);
+  const logger = createLoggerForSession(input.session);
+  await runCloseWithLogger(input.session, logger);
 });
-function createCloseCommand(logger) {
-  return SimpleCLI.command({
-    description: "Close the browser"
-  }).input(closeInput).use(resolveSessionMiddleware).handle(async ({ input, ctx }) => {
-    if (input.force && !input.all) {
-      throw new Error(`Usage: libretto close --all [--force]`);
-    }
-    if (input.all) {
-      await runCloseAllWithLogger(logger, { force: input.force });
-      return;
-    }
-    await runCloseWithLogger(ctx.session, logger);
-  });
-}
-function createBrowserCommands(logger) {
-  return {
-    open: createOpenCommand(logger),
-    save: createSaveCommand(logger),
-    pages: createPagesCommand(logger),
-    close: createCloseCommand(logger)
-  };
-}
+const browserCommands = {
+  open: openCommand,
+  connect: connectCommand,
+  save: saveCommand,
+  pages: pagesCommand,
+  close: closeCommand
+};
 async function runClose(session) {
   await withSessionLogger(session, async (logger) => {
     await runCloseWithLogger(session, logger);
   });
 }
 export {
+  browserCommands,
+  closeCommand,
   closeInput,
-  createBrowserCommands,
-  createCloseCommand,
-  createOpenCommand,
-  createPagesCommand,
-  createSaveCommand,
+  connectCommand,
+  connectInput,
+  openCommand,
   openInput,
+  pagesCommand,
   pagesInput,
+  parseViewportArg,
   runClose,
+  saveCommand,
   saveInput
 };