libretto 0.5.0 → 0.5.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (122) hide show
  1. package/README.md +109 -35
  2. package/dist/cli/cli.js +22 -97
  3. package/dist/cli/commands/browser.js +86 -59
  4. package/dist/cli/commands/execution.js +199 -86
  5. package/dist/cli/commands/init.js +34 -29
  6. package/dist/cli/commands/logs.js +4 -5
  7. package/dist/cli/commands/shared.js +30 -29
  8. package/dist/cli/commands/snapshot.js +26 -39
  9. package/dist/cli/core/ai-config.js +21 -4
  10. package/dist/cli/core/api-snapshot-analyzer.js +15 -5
  11. package/dist/cli/core/browser.js +207 -37
  12. package/dist/cli/core/context.js +4 -1
  13. package/dist/cli/core/session-telemetry.js +434 -174
  14. package/dist/cli/core/session.js +21 -8
  15. package/dist/cli/core/snapshot-analyzer.js +14 -31
  16. package/dist/cli/core/snapshot-api-config.js +2 -6
  17. package/dist/cli/core/telemetry.js +20 -4
  18. package/dist/cli/framework/simple-cli.js +45 -25
  19. package/dist/cli/router.js +14 -21
  20. package/dist/cli/workers/run-integration-runtime.js +24 -5
  21. package/dist/cli/workers/run-integration-worker-protocol.js +3 -1
  22. package/dist/cli/workers/run-integration-worker.js +1 -4
  23. package/dist/index.d.ts +1 -2
  24. package/dist/index.js +7 -10
  25. package/dist/runtime/download/download.js +5 -1
  26. package/dist/runtime/extract/extract.js +11 -2
  27. package/dist/runtime/network/network.js +8 -1
  28. package/dist/runtime/recovery/agent.js +6 -2
  29. package/dist/runtime/recovery/errors.js +3 -1
  30. package/dist/runtime/recovery/recovery.js +3 -1
  31. package/dist/shared/condense-dom/condense-dom.js +17 -69
  32. package/dist/shared/config/config.d.ts +1 -9
  33. package/dist/shared/config/config.js +0 -18
  34. package/dist/shared/config/index.d.ts +2 -1
  35. package/dist/shared/config/index.js +0 -10
  36. package/dist/shared/debug/pause.js +9 -3
  37. package/dist/shared/dom-semantics.d.ts +8 -0
  38. package/dist/shared/dom-semantics.js +69 -0
  39. package/dist/shared/instrumentation/instrument.js +101 -5
  40. package/dist/shared/llm/ai-sdk-adapter.js +3 -1
  41. package/dist/shared/llm/client.js +3 -1
  42. package/dist/shared/logger/index.js +4 -1
  43. package/dist/shared/run/api.js +3 -1
  44. package/dist/shared/run/browser.js +47 -3
  45. package/dist/shared/state/session-state.d.ts +2 -1
  46. package/dist/shared/state/session-state.js +5 -2
  47. package/dist/shared/visualization/ghost-cursor.js +36 -14
  48. package/dist/shared/visualization/highlight.js +9 -6
  49. package/dist/shared/workflow/workflow.d.ts +4 -5
  50. package/dist/shared/workflow/workflow.js +3 -5
  51. package/package.json +6 -2
  52. package/scripts/check-skills-sync.mjs +25 -0
  53. package/scripts/compare-eval-summary.mjs +47 -0
  54. package/scripts/postinstall.mjs +15 -15
  55. package/scripts/prepare-release.sh +97 -0
  56. package/scripts/skills-libretto.mjs +103 -0
  57. package/scripts/summarize-evals.mjs +135 -0
  58. package/scripts/sync-skills.mjs +12 -0
  59. package/skills/libretto/SKILL.md +132 -54
  60. package/skills/libretto/references/action-logs.md +101 -0
  61. package/skills/libretto/references/auth-profiles.md +1 -2
  62. package/skills/libretto/references/code-generation-rules.md +210 -0
  63. package/skills/libretto/references/configuration-file-reference.md +53 -0
  64. package/skills/libretto/references/pages-and-page-targeting.md +1 -1
  65. package/skills/libretto/references/site-security-review.md +143 -0
  66. package/src/cli/cli.ts +23 -110
  67. package/src/cli/commands/browser.ts +94 -70
  68. package/src/cli/commands/execution.ts +233 -102
  69. package/src/cli/commands/init.ts +37 -33
  70. package/src/cli/commands/logs.ts +7 -7
  71. package/src/cli/commands/shared.ts +36 -37
  72. package/src/cli/commands/snapshot.ts +44 -59
  73. package/src/cli/core/ai-config.ts +24 -4
  74. package/src/cli/core/api-snapshot-analyzer.ts +17 -6
  75. package/src/cli/core/browser.ts +260 -49
  76. package/src/cli/core/context.ts +7 -2
  77. package/src/cli/core/session-telemetry.ts +449 -197
  78. package/src/cli/core/session.ts +21 -7
  79. package/src/cli/core/snapshot-analyzer.ts +26 -46
  80. package/src/cli/core/snapshot-api-config.ts +170 -175
  81. package/src/cli/core/telemetry.ts +39 -4
  82. package/src/cli/framework/simple-cli.ts +144 -77
  83. package/src/cli/router.ts +13 -21
  84. package/src/cli/workers/run-integration-runtime.ts +36 -9
  85. package/src/cli/workers/run-integration-worker-protocol.ts +2 -0
  86. package/src/cli/workers/run-integration-worker.ts +1 -4
  87. package/src/index.ts +73 -66
  88. package/src/runtime/download/download.ts +62 -58
  89. package/src/runtime/download/index.ts +5 -5
  90. package/src/runtime/extract/extract.ts +71 -61
  91. package/src/runtime/network/index.ts +3 -3
  92. package/src/runtime/network/network.ts +99 -93
  93. package/src/runtime/recovery/agent.ts +217 -212
  94. package/src/runtime/recovery/errors.ts +107 -104
  95. package/src/runtime/recovery/index.ts +3 -3
  96. package/src/runtime/recovery/recovery.ts +38 -35
  97. package/src/shared/condense-dom/condense-dom.ts +27 -82
  98. package/src/shared/config/config.ts +0 -19
  99. package/src/shared/config/index.ts +0 -5
  100. package/src/shared/debug/pause.ts +57 -51
  101. package/src/shared/dom-semantics.ts +68 -0
  102. package/src/shared/instrumentation/errors.ts +64 -62
  103. package/src/shared/instrumentation/index.ts +5 -5
  104. package/src/shared/instrumentation/instrument.ts +339 -209
  105. package/src/shared/llm/ai-sdk-adapter.ts +58 -55
  106. package/src/shared/llm/client.ts +181 -174
  107. package/src/shared/llm/types.ts +39 -39
  108. package/src/shared/logger/index.ts +11 -4
  109. package/src/shared/logger/logger.ts +312 -306
  110. package/src/shared/logger/sinks.ts +118 -114
  111. package/src/shared/paths/paths.ts +50 -49
  112. package/src/shared/paths/repo-root.ts +17 -17
  113. package/src/shared/run/api.ts +5 -1
  114. package/src/shared/run/browser.ts +65 -3
  115. package/src/shared/state/index.ts +9 -9
  116. package/src/shared/state/session-state.ts +46 -43
  117. package/src/shared/visualization/ghost-cursor.ts +180 -149
  118. package/src/shared/visualization/highlight.ts +89 -86
  119. package/src/shared/visualization/index.ts +13 -13
  120. package/src/shared/workflow/workflow.ts +19 -25
  121. package/skills/libretto/references/reverse-engineering-network-requests.md +0 -39
  122. package/skills/libretto/references/user-action-log.md +0 -31
@@ -1,19 +1,19 @@
1
1
  import type { Page } from "playwright";
2
2
 
3
3
  export type HighlightOptions = {
4
- color?: string;
5
- zIndex?: number;
4
+ color?: string;
5
+ zIndex?: number;
6
6
  };
7
7
 
8
8
  const HIGHLIGHT_DEFAULTS = {
9
- color: "rgba(59, 130, 246, 0.25)",
10
- zIndex: 2147483645,
9
+ color: "rgba(59, 130, 246, 0.25)",
10
+ zIndex: 2147483645,
11
11
  };
12
12
 
13
13
  const LAYER_ID = "__libretto_highlight_layer__";
14
14
 
15
15
  function buildHighlightInitScript(opts: { zIndex: number }): string {
16
- return `
16
+ return `
17
17
  (function() {
18
18
  if (document.getElementById("${LAYER_ID}")) return;
19
19
  var el = document.createElement("div");
@@ -27,59 +27,59 @@ function buildHighlightInitScript(opts: { zIndex: number }): string {
27
27
  const installedPages = new WeakSet<Page>();
28
28
 
29
29
  export async function ensureHighlightLayer(
30
- page: Page,
31
- options?: HighlightOptions,
30
+ page: Page,
31
+ options?: HighlightOptions,
32
32
  ): Promise<void> {
33
- const existingOpts = (page as any).__librettoHighlightOpts as
34
- | { color: string; zIndex: number }
35
- | undefined;
36
- const zIndex =
37
- options?.zIndex ?? existingOpts?.zIndex ?? HIGHLIGHT_DEFAULTS.zIndex;
38
- const initScript = buildHighlightInitScript({ zIndex });
39
-
40
- if (!installedPages.has(page)) {
41
- installedPages.add(page);
42
- await page.addInitScript({ content: initScript });
43
- }
44
-
45
- // Store/refresh options for later.
46
- (page as any).__librettoHighlightOpts = {
47
- color: options?.color ?? existingOpts?.color ?? HIGHLIGHT_DEFAULTS.color,
48
- zIndex,
49
- };
50
-
51
- // Re-run in-page installer so overlays recover after page.setContent() or DOM resets.
52
- try {
53
- await page.evaluate(new Function(initScript) as () => void);
54
- } catch {
55
- // Page may not be ready
56
- }
33
+ const existingOpts = (page as any).__librettoHighlightOpts as
34
+ | { color: string; zIndex: number }
35
+ | undefined;
36
+ const zIndex =
37
+ options?.zIndex ?? existingOpts?.zIndex ?? HIGHLIGHT_DEFAULTS.zIndex;
38
+ const initScript = buildHighlightInitScript({ zIndex });
39
+
40
+ if (!installedPages.has(page)) {
41
+ installedPages.add(page);
42
+ await page.addInitScript({ content: initScript });
43
+ }
44
+
45
+ // Store/refresh options for later.
46
+ (page as any).__librettoHighlightOpts = {
47
+ color: options?.color ?? existingOpts?.color ?? HIGHLIGHT_DEFAULTS.color,
48
+ zIndex,
49
+ };
50
+
51
+ // Re-run in-page installer so overlays recover after page.setContent() or DOM resets.
52
+ try {
53
+ await page.evaluate(new Function(initScript) as () => void);
54
+ } catch {
55
+ // Page may not be ready
56
+ }
57
57
  }
58
58
 
59
59
  export type ShowHighlightParams = {
60
- box: { x: number; y: number; width: number; height: number };
61
- label?: string;
62
- color?: string;
63
- durationMs?: number;
60
+ box: { x: number; y: number; width: number; height: number };
61
+ label?: string;
62
+ color?: string;
63
+ durationMs?: number;
64
64
  };
65
65
 
66
66
  export async function showHighlight(
67
- page: Page,
68
- params: ShowHighlightParams,
67
+ page: Page,
68
+ params: ShowHighlightParams,
69
69
  ): Promise<void> {
70
- const opts = (page as any).__librettoHighlightOpts ?? HIGHLIGHT_DEFAULTS;
71
- const color = params.color ?? opts.color;
72
- const durationMs = params.durationMs ?? 350;
73
-
74
- try {
75
- await page.evaluate(
76
- ({ layerId, box, color, label, durationMs }) => {
77
- const layer = document.getElementById(layerId);
78
- if (!layer) return;
79
-
80
- const rect = document.createElement("div");
81
- rect.className = "__libretto_highlight_rect__";
82
- rect.style.cssText = `
70
+ const opts = (page as any).__librettoHighlightOpts ?? HIGHLIGHT_DEFAULTS;
71
+ const color = params.color ?? opts.color;
72
+ const durationMs = params.durationMs ?? 350;
73
+
74
+ try {
75
+ await page.evaluate(
76
+ ({ layerId, box, color, label, durationMs }) => {
77
+ const layer = document.getElementById(layerId);
78
+ if (!layer) return;
79
+
80
+ const rect = document.createElement("div");
81
+ rect.className = "__libretto_highlight_rect__";
82
+ rect.style.cssText = `
83
83
  position:absolute;
84
84
  left:${box.x}px;
85
85
  top:${box.y}px;
@@ -93,10 +93,10 @@ export async function showHighlight(
93
93
  opacity:1;
94
94
  `;
95
95
 
96
- if (label) {
97
- const labelEl = document.createElement("div");
98
- labelEl.textContent = label;
99
- labelEl.style.cssText = `
96
+ if (label) {
97
+ const labelEl = document.createElement("div");
98
+ labelEl.textContent = label;
99
+ labelEl.style.cssText = `
100
100
  position:absolute;
101
101
  top:-22px;
102
102
  left:0;
@@ -108,39 +108,42 @@ export async function showHighlight(
108
108
  white-space:nowrap;
109
109
  pointer-events:none;
110
110
  `;
111
- rect.appendChild(labelEl);
112
- }
113
-
114
- layer.appendChild(rect);
115
-
116
- // Auto-fade after duration
117
- setTimeout(() => {
118
- rect.style.opacity = "0";
119
- setTimeout(() => rect.remove(), 250);
120
- }, durationMs);
121
- },
122
- {
123
- layerId: LAYER_ID,
124
- box: params.box,
125
- color,
126
- label: params.label,
127
- durationMs,
128
- },
129
- );
130
- } catch {
131
- // Best-effort
132
- }
111
+ rect.appendChild(labelEl);
112
+ }
113
+
114
+ layer.appendChild(rect);
115
+
116
+ // Auto-fade after duration
117
+ setTimeout(() => {
118
+ rect.style.opacity = "0";
119
+ setTimeout(() => rect.remove(), 250);
120
+ }, durationMs);
121
+ },
122
+ {
123
+ layerId: LAYER_ID,
124
+ box: params.box,
125
+ color,
126
+ label: params.label,
127
+ durationMs,
128
+ },
129
+ );
130
+ } catch {
131
+ // Best-effort
132
+ }
133
133
  }
134
134
 
135
135
  export async function clearHighlights(page: Page): Promise<void> {
136
- try {
137
- await page.evaluate(({ layerId }) => {
138
- const layer = document.getElementById(layerId);
139
- if (!layer) return;
140
- const rects = layer.querySelectorAll(".__libretto_highlight_rect__");
141
- rects.forEach((r) => r.remove());
142
- }, { layerId: LAYER_ID });
143
- } catch {
144
- // Best-effort
145
- }
136
+ try {
137
+ await page.evaluate(
138
+ ({ layerId }) => {
139
+ const layer = document.getElementById(layerId);
140
+ if (!layer) return;
141
+ const rects = layer.querySelectorAll(".__libretto_highlight_rect__");
142
+ rects.forEach((r) => r.remove());
143
+ },
144
+ { layerId: LAYER_ID },
145
+ );
146
+ } catch {
147
+ // Best-effort
148
+ }
146
149
  }
@@ -1,18 +1,18 @@
1
1
  export {
2
- ensureGhostCursor,
3
- moveGhostCursor,
4
- moveGhostCursorWithDistance,
5
- ghostClick,
6
- hideGhostCursor,
7
- getGhostCursorPosition,
8
- type GhostCursorOptions,
9
- type GhostCursorStyle,
2
+ ensureGhostCursor,
3
+ moveGhostCursor,
4
+ moveGhostCursorWithDistance,
5
+ ghostClick,
6
+ hideGhostCursor,
7
+ getGhostCursorPosition,
8
+ type GhostCursorOptions,
9
+ type GhostCursorStyle,
10
10
  } from "./ghost-cursor.js";
11
11
 
12
12
  export {
13
- ensureHighlightLayer,
14
- showHighlight,
15
- clearHighlights,
16
- type HighlightOptions,
17
- type ShowHighlightParams,
13
+ ensureHighlightLayer,
14
+ showHighlight,
15
+ clearHighlights,
16
+ type HighlightOptions,
17
+ type ShowHighlightParams,
18
18
  } from "./highlight.js";
@@ -3,40 +3,34 @@ import type { MinimalLogger } from "../logger/logger.js";
3
3
 
4
4
  export const LIBRETTO_WORKFLOW_BRAND = Symbol.for("libretto.workflow");
5
5
 
6
- export type LibrettoWorkflowMetadata = {};
7
-
8
6
  export type LibrettoWorkflowContext<S = {}> = {
9
- page: Page;
10
- logger: MinimalLogger;
11
- services: S;
7
+ session: string;
8
+ page: Page;
9
+ logger: MinimalLogger;
10
+ services: S;
12
11
  };
13
12
 
14
- export type LibrettoWorkflowHandler<Input = unknown, Output = unknown, S = {}> = (
15
- ctx: LibrettoWorkflowContext<S>,
16
- input: Input,
17
- ) => Promise<Output>;
13
+ export type LibrettoWorkflowHandler<
14
+ Input = unknown,
15
+ Output = unknown,
16
+ S = {},
17
+ > = (ctx: LibrettoWorkflowContext<S>, input: Input) => Promise<Output>;
18
18
 
19
19
  export class LibrettoWorkflow<Input = unknown, Output = unknown, S = {}> {
20
- public readonly [LIBRETTO_WORKFLOW_BRAND] = true;
21
- public readonly metadata: LibrettoWorkflowMetadata;
22
- private readonly handler: LibrettoWorkflowHandler<Input, Output, S>;
20
+ public readonly [LIBRETTO_WORKFLOW_BRAND] = true;
21
+ private readonly handler: LibrettoWorkflowHandler<Input, Output, S>;
23
22
 
24
- constructor(
25
- metadata: LibrettoWorkflowMetadata,
26
- handler: LibrettoWorkflowHandler<Input, Output, S>,
27
- ) {
28
- this.metadata = metadata;
29
- this.handler = handler;
30
- }
23
+ constructor(handler: LibrettoWorkflowHandler<Input, Output, S>) {
24
+ this.handler = handler;
25
+ }
31
26
 
32
- async run(ctx: LibrettoWorkflowContext<S>, input: Input): Promise<Output> {
33
- return this.handler(ctx, input);
34
- }
27
+ async run(ctx: LibrettoWorkflowContext<S>, input: Input): Promise<Output> {
28
+ return this.handler(ctx, input);
29
+ }
35
30
  }
36
31
 
37
32
  export function workflow<Input = unknown, Output = unknown, S = {}>(
38
- metadata: LibrettoWorkflowMetadata,
39
- handler: LibrettoWorkflowHandler<Input, Output, S>,
33
+ handler: LibrettoWorkflowHandler<Input, Output, S>,
40
34
  ): LibrettoWorkflow<Input, Output, S> {
41
- return new LibrettoWorkflow(metadata, handler);
35
+ return new LibrettoWorkflow(handler);
42
36
  }
@@ -1,39 +0,0 @@
1
- # Reverse Engineering Network Requests
2
-
3
- Use this reference when the user wants to turn a browser workflow into direct network requests.
4
-
5
- This is the default approach for new integrations when the site exposes a clear and stable HTTP request path.
6
-
7
- ## When to Use This
8
-
9
- - The page clearly loads or submits data through HTTP requests.
10
- - The user can perform the workflow manually in a headed browser.
11
- - Replaying the request is likely faster or more stable than reproducing every UI action.
12
- - Fall back to browser automation when the request path is unclear, too dynamic, or blocked by anti-bot systems.
13
-
14
- ## Workflow
15
-
16
- - Open the page in headed mode.
17
- - Let the user perform the relevant workflow manually.
18
- - Read the network log after the relevant step.
19
- - Identify the smallest set of requests that actually carries the data or performs the action.
20
- - Confirm with the user before replaying any request that could mutate data.
21
- - Recreate the request in code outside Libretto.
22
- - Verify the resulting workflow with `npx libretto run ...`.
23
-
24
- ## Commands
25
-
26
- ```bash
27
- npx libretto open https://target.example.com --headed
28
- npx libretto network --last 20
29
- npx libretto network --method POST --last 20
30
- npx libretto network --filter 'referral|patient|search'
31
- npx libretto exec "return await networkLog({ method: 'POST', last: 10 })"
32
- ```
33
-
34
- ## Notes
35
-
36
- - Start with the request that returns the data you need, not every request on the page.
37
- - Prefer captured requests over guessing payload shape.
38
- - If the request format is opaque, highly dynamic, or heavily defended, fall back to UI automation for that part.
39
- - Treat all replayed requests as potentially side-effectful until proven otherwise.
@@ -1,31 +0,0 @@
1
- # User Action Log
2
-
3
- Use this reference when the user performs steps manually in a headed Libretto session and you want to incorporate those steps into a workflow.
4
-
5
- ## When to Use This
6
-
7
- - The user demonstrates the workflow in the browser window.
8
- - You want to know what they clicked, typed, or selected.
9
- - You need to reconcile manual user actions with captured network requests.
10
-
11
- ## Workflow
12
-
13
- - Open the session in headed mode.
14
- - Ask the user to perform the workflow.
15
- - Read the user action log after they finish.
16
- - Use the log to identify the important transitions in the workflow.
17
- - Combine the action log with `snapshot` or `network` when the log alone is not enough.
18
-
19
- ## Commands
20
-
21
- ```bash
22
- npx libretto actions --source user --last 20
23
- npx libretto actions --source user --filter 'button|input|select'
24
- npx libretto exec "return await actionLog({ source: 'user', last: 10 })"
25
- ```
26
-
27
- ## Notes
28
-
29
- - The action log is most useful for reconstructing the sequence of a workflow, not for discovering every selector you need.
30
- - If the user performed relevant manual steps, read the action log before writing or revising the workflow code.
31
- - Use the action log to anchor your understanding, then inspect the current page state with `snapshot` or `exec`.