libretto 0.5.0 → 0.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +109 -35
- package/dist/cli/cli.js +22 -97
- package/dist/cli/commands/browser.js +86 -59
- package/dist/cli/commands/execution.js +199 -86
- package/dist/cli/commands/init.js +34 -29
- package/dist/cli/commands/logs.js +4 -5
- package/dist/cli/commands/shared.js +30 -29
- package/dist/cli/commands/snapshot.js +26 -39
- package/dist/cli/core/ai-config.js +21 -4
- package/dist/cli/core/api-snapshot-analyzer.js +15 -5
- package/dist/cli/core/browser.js +207 -37
- package/dist/cli/core/context.js +4 -1
- package/dist/cli/core/session-telemetry.js +434 -174
- package/dist/cli/core/session.js +21 -8
- package/dist/cli/core/snapshot-analyzer.js +14 -31
- package/dist/cli/core/snapshot-api-config.js +2 -6
- package/dist/cli/core/telemetry.js +20 -4
- package/dist/cli/framework/simple-cli.js +45 -25
- package/dist/cli/router.js +14 -21
- package/dist/cli/workers/run-integration-runtime.js +24 -5
- package/dist/cli/workers/run-integration-worker-protocol.js +3 -1
- package/dist/cli/workers/run-integration-worker.js +1 -4
- package/dist/index.d.ts +1 -2
- package/dist/index.js +7 -10
- package/dist/runtime/download/download.js +5 -1
- package/dist/runtime/extract/extract.js +11 -2
- package/dist/runtime/network/network.js +8 -1
- package/dist/runtime/recovery/agent.js +6 -2
- package/dist/runtime/recovery/errors.js +3 -1
- package/dist/runtime/recovery/recovery.js +3 -1
- package/dist/shared/condense-dom/condense-dom.js +17 -69
- package/dist/shared/config/config.d.ts +1 -9
- package/dist/shared/config/config.js +0 -18
- package/dist/shared/config/index.d.ts +2 -1
- package/dist/shared/config/index.js +0 -10
- package/dist/shared/debug/pause.js +9 -3
- package/dist/shared/dom-semantics.d.ts +8 -0
- package/dist/shared/dom-semantics.js +69 -0
- package/dist/shared/instrumentation/instrument.js +101 -5
- package/dist/shared/llm/ai-sdk-adapter.js +3 -1
- package/dist/shared/llm/client.js +3 -1
- package/dist/shared/logger/index.js +4 -1
- package/dist/shared/run/api.js +3 -1
- package/dist/shared/run/browser.js +47 -3
- package/dist/shared/state/session-state.d.ts +2 -1
- package/dist/shared/state/session-state.js +5 -2
- package/dist/shared/visualization/ghost-cursor.js +36 -14
- package/dist/shared/visualization/highlight.js +9 -6
- package/dist/shared/workflow/workflow.d.ts +4 -5
- package/dist/shared/workflow/workflow.js +3 -5
- package/package.json +6 -2
- package/scripts/check-skills-sync.mjs +25 -0
- package/scripts/compare-eval-summary.mjs +47 -0
- package/scripts/postinstall.mjs +15 -15
- package/scripts/prepare-release.sh +97 -0
- package/scripts/skills-libretto.mjs +103 -0
- package/scripts/summarize-evals.mjs +135 -0
- package/scripts/sync-skills.mjs +12 -0
- package/skills/libretto/SKILL.md +132 -54
- package/skills/libretto/references/action-logs.md +101 -0
- package/skills/libretto/references/auth-profiles.md +1 -2
- package/skills/libretto/references/code-generation-rules.md +210 -0
- package/skills/libretto/references/configuration-file-reference.md +53 -0
- package/skills/libretto/references/pages-and-page-targeting.md +1 -1
- package/skills/libretto/references/site-security-review.md +143 -0
- package/src/cli/cli.ts +23 -110
- package/src/cli/commands/browser.ts +94 -70
- package/src/cli/commands/execution.ts +233 -102
- package/src/cli/commands/init.ts +37 -33
- package/src/cli/commands/logs.ts +7 -7
- package/src/cli/commands/shared.ts +36 -37
- package/src/cli/commands/snapshot.ts +44 -59
- package/src/cli/core/ai-config.ts +24 -4
- package/src/cli/core/api-snapshot-analyzer.ts +17 -6
- package/src/cli/core/browser.ts +260 -49
- package/src/cli/core/context.ts +7 -2
- package/src/cli/core/session-telemetry.ts +449 -197
- package/src/cli/core/session.ts +21 -7
- package/src/cli/core/snapshot-analyzer.ts +26 -46
- package/src/cli/core/snapshot-api-config.ts +170 -175
- package/src/cli/core/telemetry.ts +39 -4
- package/src/cli/framework/simple-cli.ts +144 -77
- package/src/cli/router.ts +13 -21
- package/src/cli/workers/run-integration-runtime.ts +36 -9
- package/src/cli/workers/run-integration-worker-protocol.ts +2 -0
- package/src/cli/workers/run-integration-worker.ts +1 -4
- package/src/index.ts +73 -66
- package/src/runtime/download/download.ts +62 -58
- package/src/runtime/download/index.ts +5 -5
- package/src/runtime/extract/extract.ts +71 -61
- package/src/runtime/network/index.ts +3 -3
- package/src/runtime/network/network.ts +99 -93
- package/src/runtime/recovery/agent.ts +217 -212
- package/src/runtime/recovery/errors.ts +107 -104
- package/src/runtime/recovery/index.ts +3 -3
- package/src/runtime/recovery/recovery.ts +38 -35
- package/src/shared/condense-dom/condense-dom.ts +27 -82
- package/src/shared/config/config.ts +0 -19
- package/src/shared/config/index.ts +0 -5
- package/src/shared/debug/pause.ts +57 -51
- package/src/shared/dom-semantics.ts +68 -0
- package/src/shared/instrumentation/errors.ts +64 -62
- package/src/shared/instrumentation/index.ts +5 -5
- package/src/shared/instrumentation/instrument.ts +339 -209
- package/src/shared/llm/ai-sdk-adapter.ts +58 -55
- package/src/shared/llm/client.ts +181 -174
- package/src/shared/llm/types.ts +39 -39
- package/src/shared/logger/index.ts +11 -4
- package/src/shared/logger/logger.ts +312 -306
- package/src/shared/logger/sinks.ts +118 -114
- package/src/shared/paths/paths.ts +50 -49
- package/src/shared/paths/repo-root.ts +17 -17
- package/src/shared/run/api.ts +5 -1
- package/src/shared/run/browser.ts +65 -3
- package/src/shared/state/index.ts +9 -9
- package/src/shared/state/session-state.ts +46 -43
- package/src/shared/visualization/ghost-cursor.ts +180 -149
- package/src/shared/visualization/highlight.ts +89 -86
- package/src/shared/visualization/index.ts +13 -13
- package/src/shared/workflow/workflow.ts +19 -25
- package/skills/libretto/references/reverse-engineering-network-requests.md +0 -39
- package/skills/libretto/references/user-action-log.md +0 -31
|
@@ -1,19 +1,19 @@
|
|
|
1
1
|
import type { Page } from "playwright";
|
|
2
2
|
|
|
3
3
|
export type HighlightOptions = {
|
|
4
|
-
|
|
5
|
-
|
|
4
|
+
color?: string;
|
|
5
|
+
zIndex?: number;
|
|
6
6
|
};
|
|
7
7
|
|
|
8
8
|
const HIGHLIGHT_DEFAULTS = {
|
|
9
|
-
|
|
10
|
-
|
|
9
|
+
color: "rgba(59, 130, 246, 0.25)",
|
|
10
|
+
zIndex: 2147483645,
|
|
11
11
|
};
|
|
12
12
|
|
|
13
13
|
const LAYER_ID = "__libretto_highlight_layer__";
|
|
14
14
|
|
|
15
15
|
function buildHighlightInitScript(opts: { zIndex: number }): string {
|
|
16
|
-
|
|
16
|
+
return `
|
|
17
17
|
(function() {
|
|
18
18
|
if (document.getElementById("${LAYER_ID}")) return;
|
|
19
19
|
var el = document.createElement("div");
|
|
@@ -27,59 +27,59 @@ function buildHighlightInitScript(opts: { zIndex: number }): string {
|
|
|
27
27
|
const installedPages = new WeakSet<Page>();
|
|
28
28
|
|
|
29
29
|
export async function ensureHighlightLayer(
|
|
30
|
-
|
|
31
|
-
|
|
30
|
+
page: Page,
|
|
31
|
+
options?: HighlightOptions,
|
|
32
32
|
): Promise<void> {
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
33
|
+
const existingOpts = (page as any).__librettoHighlightOpts as
|
|
34
|
+
| { color: string; zIndex: number }
|
|
35
|
+
| undefined;
|
|
36
|
+
const zIndex =
|
|
37
|
+
options?.zIndex ?? existingOpts?.zIndex ?? HIGHLIGHT_DEFAULTS.zIndex;
|
|
38
|
+
const initScript = buildHighlightInitScript({ zIndex });
|
|
39
|
+
|
|
40
|
+
if (!installedPages.has(page)) {
|
|
41
|
+
installedPages.add(page);
|
|
42
|
+
await page.addInitScript({ content: initScript });
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
// Store/refresh options for later.
|
|
46
|
+
(page as any).__librettoHighlightOpts = {
|
|
47
|
+
color: options?.color ?? existingOpts?.color ?? HIGHLIGHT_DEFAULTS.color,
|
|
48
|
+
zIndex,
|
|
49
|
+
};
|
|
50
|
+
|
|
51
|
+
// Re-run in-page installer so overlays recover after page.setContent() or DOM resets.
|
|
52
|
+
try {
|
|
53
|
+
await page.evaluate(new Function(initScript) as () => void);
|
|
54
|
+
} catch {
|
|
55
|
+
// Page may not be ready
|
|
56
|
+
}
|
|
57
57
|
}
|
|
58
58
|
|
|
59
59
|
export type ShowHighlightParams = {
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
60
|
+
box: { x: number; y: number; width: number; height: number };
|
|
61
|
+
label?: string;
|
|
62
|
+
color?: string;
|
|
63
|
+
durationMs?: number;
|
|
64
64
|
};
|
|
65
65
|
|
|
66
66
|
export async function showHighlight(
|
|
67
|
-
|
|
68
|
-
|
|
67
|
+
page: Page,
|
|
68
|
+
params: ShowHighlightParams,
|
|
69
69
|
): Promise<void> {
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
70
|
+
const opts = (page as any).__librettoHighlightOpts ?? HIGHLIGHT_DEFAULTS;
|
|
71
|
+
const color = params.color ?? opts.color;
|
|
72
|
+
const durationMs = params.durationMs ?? 350;
|
|
73
|
+
|
|
74
|
+
try {
|
|
75
|
+
await page.evaluate(
|
|
76
|
+
({ layerId, box, color, label, durationMs }) => {
|
|
77
|
+
const layer = document.getElementById(layerId);
|
|
78
|
+
if (!layer) return;
|
|
79
|
+
|
|
80
|
+
const rect = document.createElement("div");
|
|
81
|
+
rect.className = "__libretto_highlight_rect__";
|
|
82
|
+
rect.style.cssText = `
|
|
83
83
|
position:absolute;
|
|
84
84
|
left:${box.x}px;
|
|
85
85
|
top:${box.y}px;
|
|
@@ -93,10 +93,10 @@ export async function showHighlight(
|
|
|
93
93
|
opacity:1;
|
|
94
94
|
`;
|
|
95
95
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
96
|
+
if (label) {
|
|
97
|
+
const labelEl = document.createElement("div");
|
|
98
|
+
labelEl.textContent = label;
|
|
99
|
+
labelEl.style.cssText = `
|
|
100
100
|
position:absolute;
|
|
101
101
|
top:-22px;
|
|
102
102
|
left:0;
|
|
@@ -108,39 +108,42 @@ export async function showHighlight(
|
|
|
108
108
|
white-space:nowrap;
|
|
109
109
|
pointer-events:none;
|
|
110
110
|
`;
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
111
|
+
rect.appendChild(labelEl);
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
layer.appendChild(rect);
|
|
115
|
+
|
|
116
|
+
// Auto-fade after duration
|
|
117
|
+
setTimeout(() => {
|
|
118
|
+
rect.style.opacity = "0";
|
|
119
|
+
setTimeout(() => rect.remove(), 250);
|
|
120
|
+
}, durationMs);
|
|
121
|
+
},
|
|
122
|
+
{
|
|
123
|
+
layerId: LAYER_ID,
|
|
124
|
+
box: params.box,
|
|
125
|
+
color,
|
|
126
|
+
label: params.label,
|
|
127
|
+
durationMs,
|
|
128
|
+
},
|
|
129
|
+
);
|
|
130
|
+
} catch {
|
|
131
|
+
// Best-effort
|
|
132
|
+
}
|
|
133
133
|
}
|
|
134
134
|
|
|
135
135
|
export async function clearHighlights(page: Page): Promise<void> {
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
136
|
+
try {
|
|
137
|
+
await page.evaluate(
|
|
138
|
+
({ layerId }) => {
|
|
139
|
+
const layer = document.getElementById(layerId);
|
|
140
|
+
if (!layer) return;
|
|
141
|
+
const rects = layer.querySelectorAll(".__libretto_highlight_rect__");
|
|
142
|
+
rects.forEach((r) => r.remove());
|
|
143
|
+
},
|
|
144
|
+
{ layerId: LAYER_ID },
|
|
145
|
+
);
|
|
146
|
+
} catch {
|
|
147
|
+
// Best-effort
|
|
148
|
+
}
|
|
146
149
|
}
|
|
@@ -1,18 +1,18 @@
|
|
|
1
1
|
export {
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
2
|
+
ensureGhostCursor,
|
|
3
|
+
moveGhostCursor,
|
|
4
|
+
moveGhostCursorWithDistance,
|
|
5
|
+
ghostClick,
|
|
6
|
+
hideGhostCursor,
|
|
7
|
+
getGhostCursorPosition,
|
|
8
|
+
type GhostCursorOptions,
|
|
9
|
+
type GhostCursorStyle,
|
|
10
10
|
} from "./ghost-cursor.js";
|
|
11
11
|
|
|
12
12
|
export {
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
13
|
+
ensureHighlightLayer,
|
|
14
|
+
showHighlight,
|
|
15
|
+
clearHighlights,
|
|
16
|
+
type HighlightOptions,
|
|
17
|
+
type ShowHighlightParams,
|
|
18
18
|
} from "./highlight.js";
|
|
@@ -3,40 +3,34 @@ import type { MinimalLogger } from "../logger/logger.js";
|
|
|
3
3
|
|
|
4
4
|
export const LIBRETTO_WORKFLOW_BRAND = Symbol.for("libretto.workflow");
|
|
5
5
|
|
|
6
|
-
export type LibrettoWorkflowMetadata = {};
|
|
7
|
-
|
|
8
6
|
export type LibrettoWorkflowContext<S = {}> = {
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
7
|
+
session: string;
|
|
8
|
+
page: Page;
|
|
9
|
+
logger: MinimalLogger;
|
|
10
|
+
services: S;
|
|
12
11
|
};
|
|
13
12
|
|
|
14
|
-
export type LibrettoWorkflowHandler<
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
13
|
+
export type LibrettoWorkflowHandler<
|
|
14
|
+
Input = unknown,
|
|
15
|
+
Output = unknown,
|
|
16
|
+
S = {},
|
|
17
|
+
> = (ctx: LibrettoWorkflowContext<S>, input: Input) => Promise<Output>;
|
|
18
18
|
|
|
19
19
|
export class LibrettoWorkflow<Input = unknown, Output = unknown, S = {}> {
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
private readonly handler: LibrettoWorkflowHandler<Input, Output, S>;
|
|
20
|
+
public readonly [LIBRETTO_WORKFLOW_BRAND] = true;
|
|
21
|
+
private readonly handler: LibrettoWorkflowHandler<Input, Output, S>;
|
|
23
22
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
) {
|
|
28
|
-
this.metadata = metadata;
|
|
29
|
-
this.handler = handler;
|
|
30
|
-
}
|
|
23
|
+
constructor(handler: LibrettoWorkflowHandler<Input, Output, S>) {
|
|
24
|
+
this.handler = handler;
|
|
25
|
+
}
|
|
31
26
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
27
|
+
async run(ctx: LibrettoWorkflowContext<S>, input: Input): Promise<Output> {
|
|
28
|
+
return this.handler(ctx, input);
|
|
29
|
+
}
|
|
35
30
|
}
|
|
36
31
|
|
|
37
32
|
export function workflow<Input = unknown, Output = unknown, S = {}>(
|
|
38
|
-
|
|
39
|
-
handler: LibrettoWorkflowHandler<Input, Output, S>,
|
|
33
|
+
handler: LibrettoWorkflowHandler<Input, Output, S>,
|
|
40
34
|
): LibrettoWorkflow<Input, Output, S> {
|
|
41
|
-
|
|
35
|
+
return new LibrettoWorkflow(handler);
|
|
42
36
|
}
|
|
@@ -1,39 +0,0 @@
|
|
|
1
|
-
# Reverse Engineering Network Requests
|
|
2
|
-
|
|
3
|
-
Use this reference when the user wants to turn a browser workflow into direct network requests.
|
|
4
|
-
|
|
5
|
-
This is the default approach for new integrations when the site exposes a clear and stable HTTP request path.
|
|
6
|
-
|
|
7
|
-
## When to Use This
|
|
8
|
-
|
|
9
|
-
- The page clearly loads or submits data through HTTP requests.
|
|
10
|
-
- The user can perform the workflow manually in a headed browser.
|
|
11
|
-
- Replaying the request is likely faster or more stable than reproducing every UI action.
|
|
12
|
-
- Fall back to browser automation when the request path is unclear, too dynamic, or blocked by anti-bot systems.
|
|
13
|
-
|
|
14
|
-
## Workflow
|
|
15
|
-
|
|
16
|
-
- Open the page in headed mode.
|
|
17
|
-
- Let the user perform the relevant workflow manually.
|
|
18
|
-
- Read the network log after the relevant step.
|
|
19
|
-
- Identify the smallest set of requests that actually carries the data or performs the action.
|
|
20
|
-
- Confirm with the user before replaying any request that could mutate data.
|
|
21
|
-
- Recreate the request in code outside Libretto.
|
|
22
|
-
- Verify the resulting workflow with `npx libretto run ...`.
|
|
23
|
-
|
|
24
|
-
## Commands
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
npx libretto open https://target.example.com --headed
|
|
28
|
-
npx libretto network --last 20
|
|
29
|
-
npx libretto network --method POST --last 20
|
|
30
|
-
npx libretto network --filter 'referral|patient|search'
|
|
31
|
-
npx libretto exec "return await networkLog({ method: 'POST', last: 10 })"
|
|
32
|
-
```
|
|
33
|
-
|
|
34
|
-
## Notes
|
|
35
|
-
|
|
36
|
-
- Start with the request that returns the data you need, not every request on the page.
|
|
37
|
-
- Prefer captured requests over guessing payload shape.
|
|
38
|
-
- If the request format is opaque, highly dynamic, or heavily defended, fall back to UI automation for that part.
|
|
39
|
-
- Treat all replayed requests as potentially side-effectful until proven otherwise.
|
|
@@ -1,31 +0,0 @@
|
|
|
1
|
-
# User Action Log
|
|
2
|
-
|
|
3
|
-
Use this reference when the user performs steps manually in a headed Libretto session and you want to incorporate those steps into a workflow.
|
|
4
|
-
|
|
5
|
-
## When to Use This
|
|
6
|
-
|
|
7
|
-
- The user demonstrates the workflow in the browser window.
|
|
8
|
-
- You want to know what they clicked, typed, or selected.
|
|
9
|
-
- You need to reconcile manual user actions with captured network requests.
|
|
10
|
-
|
|
11
|
-
## Workflow
|
|
12
|
-
|
|
13
|
-
- Open the session in headed mode.
|
|
14
|
-
- Ask the user to perform the workflow.
|
|
15
|
-
- Read the user action log after they finish.
|
|
16
|
-
- Use the log to identify the important transitions in the workflow.
|
|
17
|
-
- Combine the action log with `snapshot` or `network` when the log alone is not enough.
|
|
18
|
-
|
|
19
|
-
## Commands
|
|
20
|
-
|
|
21
|
-
```bash
|
|
22
|
-
npx libretto actions --source user --last 20
|
|
23
|
-
npx libretto actions --source user --filter 'button|input|select'
|
|
24
|
-
npx libretto exec "return await actionLog({ source: 'user', last: 10 })"
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
## Notes
|
|
28
|
-
|
|
29
|
-
- The action log is most useful for reconstructing the sequence of a workflow, not for discovering every selector you need.
|
|
30
|
-
- If the user performed relevant manual steps, read the action log before writing or revising the workflow code.
|
|
31
|
-
- Use the action log to anchor your understanding, then inspect the current page state with `snapshot` or `exec`.
|