imprint-mcp 0.4.6 → 0.4.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. package/README.md +4 -4
  2. package/examples/google-flights/README.md +2 -0
  3. package/examples/google-flights/_shared/flights_request.ts +10 -4
  4. package/examples/google-flights/get_flight_booking_details/index.ts +5 -2
  5. package/examples/google-flights/get_flight_booking_details/parser.ts +8 -0
  6. package/examples/google-flights/get_flight_booking_details/workflow.json +5 -2
  7. package/examples/google-flights/get_flight_calendar_prices/index.ts +5 -2
  8. package/examples/google-flights/get_flight_calendar_prices/parser.ts +8 -4
  9. package/examples/google-flights/get_flight_calendar_prices/workflow.json +5 -2
  10. package/examples/google-flights/lookup_airport/index.ts +3 -0
  11. package/examples/google-flights/lookup_airport/parser.ts +8 -1
  12. package/examples/google-flights/lookup_airport/workflow.json +3 -0
  13. package/examples/google-flights/search_flights/index.ts +63 -8
  14. package/examples/google-flights/search_flights/parser.ts +10 -0
  15. package/examples/google-flights/search_flights/request-transform.ts +45 -0
  16. package/examples/google-flights/search_flights/workflow.json +63 -8
  17. package/package.json +1 -1
  18. package/prompts/build-planning.md +1 -1
  19. package/prompts/compile-agent.md +5 -3
  20. package/prompts/prereq-builder.md +2 -1
  21. package/src/imprint/backend-ladder.ts +436 -43
  22. package/src/imprint/cdp-browser-fetch.ts +176 -6
  23. package/src/imprint/cdp-jar-cache.ts +105 -10
  24. package/src/imprint/compile-tools.ts +2 -2
  25. package/src/imprint/mcp-server.ts +152 -65
  26. package/src/imprint/probe-backends.ts +41 -10
  27. package/src/imprint/runtime.ts +24 -12
  28. package/src/imprint/stealth-fetch.ts +71 -0
  29. package/src/imprint/stealth-token-cache.ts +38 -1
  30. package/src/imprint/types.ts +45 -0
package/README.md CHANGED
@@ -169,8 +169,8 @@ When an API call gets blocked, Imprint doesn't jump to DOM replay. It escalates
169
169
  │ + API
170
170
 
171
171
  cdp-replay ~2-35s API calls run inside a live, trusted Chrome —
172
- a protected POST refreshes its anti-bot token
173
- between calls (multi-step state-changing flows)
172
+ reuses browser-observed request state and refreshes
173
+ anti-bot tokens between protected POSTs
174
174
 
175
175
  stealth-fetch ~1-12s Defeats Akamai, Cloudflare, DataDome
176
176
 
@@ -178,9 +178,9 @@ When an API call gets blocked, Imprint doesn't jump to DOM replay. It escalates
178
178
  playbook ~9s Full DOM replay — universal fallback
179
179
  ```
180
180
 
181
- The full order is `fetch → fetch-bootstrap → cdp-replay → stealth-fetch → playbook`; `auto` mode walks it and stops at the first backend that works.
181
+ The full order is `fetch → fetch-bootstrap → cdp-replay → stealth-fetch → playbook`; `auto` mode walks it and stops at the first backend that works. Workflows that declare browser-observed request captures can start at `cdp-replay`, so MCP sessions reuse the same Chrome instead of paying a cold bootstrap on each route/date.
182
182
 
183
- For bot-protected sites, `imprint probe-backends <site> --tool <toolName>` writes a `backends.json` preference cache so cron and MCP start from the known-good backend instead of rediscovering blocked rungs. Use `imprint probe-backends <site> --all` to refresh every tool in a multi-tool site; `imprint mcp status` reports stale or invalid backend caches before they quietly fall back to the default ladder. CDP replay records both cold and warm timings when it succeeds: a timeout-safe cold start may rank by its fast warm runtime, but a cold start above the preferred threshold stays behind cold-safe backends in durable cache order.
183
+ For bot-protected sites, `imprint probe-backends <site> --tool <toolName>` writes a `backends.json` preference cache so cron and MCP start from the known-good backend instead of rediscovering blocked rungs. Use `imprint probe-backends <site> --all` to refresh every tool in a multi-tool site; `imprint mcp status` reports stale or invalid backend caches before they quietly fall back to the default ladder. Execution-only workflow changes keep the cached backend order when the backend capability hash still matches. CDP replay records both cold and warm timings when it succeeds: a timeout-safe cold start may rank by its fast warm runtime, but a cold start above the preferred threshold stays behind cold-safe backends in durable cache order.
184
184
 
185
185
  Every recording compiles to *both* `workflow.json` and `playbook.yaml`, so the ladder always has a DOM fallback.
186
186
 
@@ -17,6 +17,8 @@ A 4-tool MCP server for Google Flights, compiled from a recording of a normal fl
17
17
 
18
18
  - **Protocol**: Google's `/_/FlightsFrontendUi` **`batchexecute`** endpoint returns a nested-array (protobuf-ish) payload. The compiler reverse-engineered the encoding into `_shared/batchexecute.ts` (shared decoder) + per-tool `parser.ts`, and the `f.req` request shape into `_shared/flights_request.ts` + per-tool `request-transform.ts`.
19
19
  - **Anti-bot**: the per-page `f.sid` / `bl` tokens are bootstrapped at runtime (`${state.f_sid}` placeholders), and calls run on the **cdp-replay** rung (requests issued inside a live, trusted Chrome) with a **stealth-fetch** fallback.
20
+ - **MCP pacing**: `search_flights` declares `execution.minCallSpacingMs: 2000` because Google Flights can return fast empty result sets when warm CDP searches are fired back-to-back with no breathing room.
21
+ - **Bounded fallback**: these tools declare `execution.skipPlaybookFallback` so MCP calls fail fast after the API/browser-backed rungs are exhausted instead of spending the rest of the agent timeout in an unstructured DOM replay.
20
22
  - **Artifacts per tool**: `workflow.json` (API replay), `playbook.yaml` (DOM fallback), `index.ts` (MCP tool), `parser.ts` + `request-transform.ts` (codecs).
21
23
 
22
24
  ## Install
@@ -8,9 +8,11 @@
8
8
  // with the date range living in the outer wrapper; Shopping/Booking use the full
9
9
  // 15-slot leg with DATE at [6]. Verified by decoding seq 97 vs seq 111.
10
10
 
11
- // Fresh searches emit wrapper `...,0,0,0,1]` and leg[14]=3 (proven seq 111/140).
12
- // In-page-refined searches use `...,0,1,0,1]` with return-leg[14]=1 (seq 194/425)
13
- // a UI freshness flag, not a user param; we always emit the fresh form for shopping.
11
+ // Fresh searches emit wrapper `...,0,0,0,1]` and use leg[14]=3 for normal
12
+ // shopping legs. Return leg [14]=1 appears in booking / selected-leg flows, not
13
+ // in the initial search request used by this tool.
14
+ // In-page-refined searches use `...,0,1,0,1]` — a UI freshness flag, not a user
15
+ // param; we always emit the fresh form for shopping.
14
16
  // Booking outbound legs use [14]=3, return legs [14]=1 (seq 764/811).
15
17
 
16
18
  function buildLeg(leg: any): any[] {
@@ -74,7 +76,11 @@ export function transform(
74
76
  let payload: any;
75
77
 
76
78
  if (rpc === 'GetShoppingResults') {
77
- payload = [[], sp, 0, 0, 0, 1];
79
+ const searchContext =
80
+ typeof p.searchContextToken === 'string' && p.searchContextToken
81
+ ? [null, null, null, p.searchContextToken]
82
+ : [];
83
+ payload = [searchContext, sp, 0, 0, 0, 1];
78
84
  } else if (rpc === 'GetCalendarPicker') {
79
85
  const legs = sp[13];
80
86
  if (Array.isArray(legs)) sp[13] = legs.map((l: any) => (Array.isArray(l) ? l.slice(0, 4) : l));
@@ -94,7 +94,7 @@ const WORKFLOW: Workflow = {
94
94
  "captures": [
95
95
  {
96
96
  "name": "f_sid",
97
- "required": false,
97
+ "required": true,
98
98
  "capability": "browser_bootstrap",
99
99
  "source": "html_regex",
100
100
  "pattern": "\"FdrFJe\":\"([^\"]+)\"",
@@ -102,7 +102,7 @@ const WORKFLOW: Workflow = {
102
102
  },
103
103
  {
104
104
  "name": "bl",
105
- "required": false,
105
+ "required": true,
106
106
  "capability": "browser_bootstrap",
107
107
  "source": "html_regex",
108
108
  "pattern": "\"cfb2h\":\"([^\"]+)\"",
@@ -112,6 +112,9 @@ const WORKFLOW: Workflow = {
112
112
  },
113
113
  "parserModule": "./parser.ts",
114
114
  "requestTransformModule": "./request-transform.ts",
115
+ "execution": {
116
+ "skipPlaybookFallback": true
117
+ },
115
118
  "liveVerified": true
116
119
  };
117
120
 
@@ -142,6 +142,9 @@ export function extract(
142
142
  let frames: Array<{ rpcid: string | null; payload: any }> = [];
143
143
  if (typeof rawResponse === 'string') {
144
144
  frames = decodeBatchExecute(rawResponse);
145
+ if (frames.length === 0) {
146
+ throw new Error('Google Flights GetBookingResults response did not contain a batchexecute payload');
147
+ }
145
148
  } else if (rawResponse != null) {
146
149
  frames = [{ rpcid: null, payload: rawResponse }];
147
150
  }
@@ -170,6 +173,11 @@ export function extract(
170
173
 
171
174
  const segments = [...segMap.values()];
172
175
  const fareOptions = [...fareMap.values()];
176
+ if (segments.length === 0 && fareOptions.length === 0) {
177
+ throw new Error(
178
+ 'Google Flights GetBookingResults payload did not contain recognizable booking details',
179
+ );
180
+ }
173
181
  const prices = fareOptions.map((f) => f.priceUSD);
174
182
 
175
183
  return {
@@ -65,7 +65,7 @@
65
65
  "name": "f_sid",
66
66
  "pattern": "\"FdrFJe\":\"([^\"]+)\"",
67
67
  "group": 1,
68
- "required": false,
68
+ "required": true,
69
69
  "capability": "browser_bootstrap"
70
70
  },
71
71
  {
@@ -73,7 +73,7 @@
73
73
  "name": "bl",
74
74
  "pattern": "\"cfb2h\":\"([^\"]+)\"",
75
75
  "group": 1,
76
- "required": false,
76
+ "required": true,
77
77
  "capability": "browser_bootstrap"
78
78
  }
79
79
  ]
@@ -94,5 +94,8 @@
94
94
  ],
95
95
  "requestTransformModule": "./request-transform.ts",
96
96
  "parserModule": "./parser.ts",
97
+ "execution": {
98
+ "skipPlaybookFallback": true
99
+ },
97
100
  "liveVerified": true
98
101
  }
@@ -74,7 +74,7 @@ const WORKFLOW: Workflow = {
74
74
  "captures": [
75
75
  {
76
76
  "name": "f_sid",
77
- "required": false,
77
+ "required": true,
78
78
  "capability": "browser_bootstrap",
79
79
  "source": "html_regex",
80
80
  "pattern": "\"FdrFJe\":\"([^\"]+)\"",
@@ -82,7 +82,7 @@ const WORKFLOW: Workflow = {
82
82
  },
83
83
  {
84
84
  "name": "bl",
85
- "required": false,
85
+ "required": true,
86
86
  "capability": "browser_bootstrap",
87
87
  "source": "html_regex",
88
88
  "pattern": "\"cfb2h\":\"([^\"]+)\"",
@@ -92,6 +92,9 @@ const WORKFLOW: Workflow = {
92
92
  },
93
93
  "parserModule": "./parser.ts",
94
94
  "requestTransformModule": "./request-transform.ts",
95
+ "execution": {
96
+ "skipPlaybookFallback": true
97
+ },
95
98
  "liveVerified": true
96
99
  };
97
100
 
@@ -54,6 +54,9 @@ export function extract(
54
54
  ): unknown {
55
55
  const raw = typeof rawResponse === 'string' ? rawResponse : JSON.stringify(rawResponse ?? '');
56
56
  const frames = decodeBatchExecute(raw);
57
+ if (frames.length === 0) {
58
+ throw new Error('Google Flights GetCalendarPicker response did not contain a batchexecute payload');
59
+ }
57
60
 
58
61
  let payload: unknown = null;
59
62
  for (const f of frames) {
@@ -63,13 +66,14 @@ export function extract(
63
66
  break;
64
67
  }
65
68
  }
66
- // If no frame produced entries, still attempt the first frame's payload so an
67
- // empty (zero-result) response yields an empty calendar rather than throwing.
68
- if (payload == null && frames.length > 0) payload = frames[0]?.payload ?? null;
69
-
70
69
  const entries = collectEntries(payload).sort((a, b) =>
71
70
  a.departureDate < b.departureDate ? -1 : a.departureDate > b.departureDate ? 1 : 0,
72
71
  );
72
+ if (entries.length === 0) {
73
+ throw new Error(
74
+ 'Google Flights GetCalendarPicker payload did not contain recognizable calendar prices',
75
+ );
76
+ }
73
77
 
74
78
  const prices: Record<string, number> = {};
75
79
  for (const e of entries) prices[e.departureDate] = e.lowestPriceUSD;
@@ -44,7 +44,7 @@
44
44
  "name": "f_sid",
45
45
  "pattern": "\"FdrFJe\":\"([^\"]+)\"",
46
46
  "group": 1,
47
- "required": false,
47
+ "required": true,
48
48
  "capability": "browser_bootstrap"
49
49
  },
50
50
  {
@@ -52,7 +52,7 @@
52
52
  "name": "bl",
53
53
  "pattern": "\"cfb2h\":\"([^\"]+)\"",
54
54
  "group": 1,
55
- "required": false,
55
+ "required": true,
56
56
  "capability": "browser_bootstrap"
57
57
  }
58
58
  ]
@@ -74,5 +74,8 @@
74
74
  ],
75
75
  "requestTransformModule": "./request-transform.ts",
76
76
  "parserModule": "./parser.ts",
77
+ "execution": {
78
+ "skipPlaybookFallback": true
79
+ },
77
80
  "liveVerified": true
78
81
  }
@@ -71,6 +71,9 @@ const WORKFLOW: Workflow = {
71
71
  },
72
72
  "parserModule": "./parser.ts",
73
73
  "requestTransformModule": "./request-transform.ts",
74
+ "execution": {
75
+ "skipPlaybookFallback": true
76
+ },
74
77
  "liveVerified": true
75
78
  };
76
79
 
@@ -45,8 +45,15 @@ export function extract(
45
45
  ): unknown {
46
46
  const raw = typeof rawResponse === 'string' ? rawResponse : JSON.stringify(rawResponse);
47
47
  const payload = extractRpcPayload(raw, 'tDoGIe');
48
+ if (payload == null) {
49
+ throw new Error('Google Flights tDoGIe response did not contain a batchexecute payload');
50
+ }
48
51
 
49
- const matchesRaw = Array.isArray(payload) && Array.isArray(payload[1]) ? payload[1] : [];
52
+ if (!Array.isArray(payload) || !Array.isArray(payload[1])) {
53
+ throw new Error('Google Flights tDoGIe payload did not contain a recognizable match list');
54
+ }
55
+
56
+ const matchesRaw = payload[1];
50
57
  const matches = matchesRaw
51
58
  .map(parseItem)
52
59
  .filter((m): m is AirportMatch => m !== null);
@@ -53,5 +53,8 @@
53
53
  ],
54
54
  "parserModule": "./parser.ts",
55
55
  "requestTransformModule": "./request-transform.ts",
56
+ "execution": {
57
+ "skipPlaybookFallback": true
58
+ },
56
59
  "liveVerified": true
57
60
  }
@@ -120,11 +120,13 @@ const WORKFLOW: Workflow = {
120
120
  "requests": [
121
121
  {
122
122
  "method": "POST",
123
- "url": "https://www.google.com/_/FlightsFrontendUi/data/travel.frontend.flights.FlightsFrontendService/GetShoppingResults?f.sid=${state.f_sid}&bl=${state.bl}&hl=en-US&soc-app=162&soc-platform=1&soc-device=1&_reqid=1708023&rt=c",
123
+ "url": "https://www.google.com/_/FlightsFrontendUi/data/travel.frontend.flights.FlightsFrontendService/GetShoppingResults?f.sid=${state.f_sid}&bl=${state.bl}&hl=en-US&soc-app=162&soc-platform=1&soc-device=1&_reqid=${state.reqid}&rt=c",
124
124
  "headers": {
125
125
  "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
126
126
  "X-Same-Domain": "1",
127
- "Referer": "https://www.google.com/travel/flights",
127
+ "Referer": "https://www.google.com/travel/flights?q=${param.bootstrap_query}&curr=USD",
128
+ "Accept-Language": "en-US,en;q=0.9",
129
+ "X-Goog-BatchExecute-Bgr": "${state.bgr}",
128
130
  "x-goog-ext-259736195-jspb": "[\"en-US\",\"US\",\"USD\",2,null,[420],null,null,7,[]]"
129
131
  },
130
132
  "body": "f.req=${param.origin}|${param.destination}|${param.departure_date}|${param.return_date}|${param.trip_type}|${param.max_stops}|${param.airlines}|${param.max_price}|${param.outbound_times}|${param.return_times}|${param.max_duration}|${param.carry_on_bags}&",
@@ -133,30 +135,83 @@ const WORKFLOW: Workflow = {
133
135
  ],
134
136
  "site": "google-flights",
135
137
  "bootstrap": {
136
- "url": "https://www.google.com/travel/flights",
138
+ "url": "https://www.google.com/travel/flights?q=${param.bootstrap_query}&curr=USD",
137
139
  "waitUntil": "domcontentloaded",
138
140
  "timeoutMs": 30000,
139
141
  "captures": [
140
142
  {
141
143
  "name": "f_sid",
142
- "required": false,
144
+ "required": true,
143
145
  "capability": "browser_bootstrap",
144
- "source": "html_regex",
145
- "pattern": "\"FdrFJe\":\"([^\"]+)\"",
146
+ "source": "request_url_regex",
147
+ "pattern": "[?&]f\\.sid=([^&]+)",
148
+ "method": "POST",
149
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
150
+ "mode": "last",
146
151
  "group": 1
147
152
  },
148
153
  {
149
154
  "name": "bl",
155
+ "required": true,
156
+ "capability": "browser_bootstrap",
157
+ "source": "request_url_regex",
158
+ "pattern": "[?&]bl=([^&]+)",
159
+ "method": "POST",
160
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
161
+ "mode": "last",
162
+ "group": 1
163
+ },
164
+ {
165
+ "name": "bgr",
166
+ "required": true,
167
+ "capability": "browser_bootstrap",
168
+ "source": "request_header",
169
+ "header": "X-Goog-BatchExecute-Bgr",
170
+ "method": "POST",
171
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
172
+ "mode": "last"
173
+ },
174
+ {
175
+ "name": "reqid",
176
+ "required": true,
177
+ "capability": "browser_bootstrap",
178
+ "source": "request_url_regex",
179
+ "pattern": "[?&]_reqid=([^&]+)",
180
+ "method": "POST",
181
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
182
+ "mode": "last",
183
+ "group": 1
184
+ },
185
+ {
186
+ "name": "search_context_token",
150
187
  "required": false,
151
188
  "capability": "browser_bootstrap",
152
- "source": "html_regex",
153
- "pattern": "\"cfb2h\":\"([^\"]+)\"",
189
+ "source": "request_body_regex",
190
+ "pattern": "%5B%5Bnull%2Cnull%2Cnull%2C%5C%22(.+?)%5C%22%5D",
191
+ "method": "POST",
192
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
193
+ "mode": "last",
194
+ "group": 1
195
+ },
196
+ {
197
+ "name": "observed_search_body",
198
+ "required": false,
199
+ "capability": "browser_bootstrap",
200
+ "source": "request_body_regex",
201
+ "pattern": "^(f\\.req=.*)$",
202
+ "method": "POST",
203
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
204
+ "mode": "last",
154
205
  "group": 1
155
206
  }
156
207
  ]
157
208
  },
158
209
  "parserModule": "./parser.ts",
159
210
  "requestTransformModule": "./request-transform.ts",
211
+ "execution": {
212
+ "minCallSpacingMs": 2000,
213
+ "skipPlaybookFallback": true
214
+ },
160
215
  "liveVerified": true
161
216
  };
162
217
 
@@ -147,6 +147,11 @@ export function extract(
147
147
  const frames = decodeBatchExecute(rawResponse);
148
148
  payload = frames[0]?.payload;
149
149
  }
150
+ if (payload == null) {
151
+ throw new Error(
152
+ 'Google Flights GetShoppingResults response did not contain a batchexecute payload',
153
+ );
154
+ }
150
155
  } else {
151
156
  payload = rawResponse;
152
157
  }
@@ -162,6 +167,11 @@ export function extract(
162
167
  }
163
168
 
164
169
  const itineraries = [...byToken.values()];
170
+ if (itineraries.length === 0) {
171
+ throw new Error(
172
+ 'Google Flights GetShoppingResults payload did not contain recognizable itineraries',
173
+ );
174
+ }
165
175
  return {
166
176
  count: itineraries.length,
167
177
  itineraries,
@@ -62,13 +62,56 @@ function num(v: unknown): number | undefined {
62
62
  return Number.isFinite(n) && n > 0 ? n : undefined;
63
63
  }
64
64
 
65
+ function buildBootstrapQuery(params: Params): string {
66
+ const origin = params.origin != null ? String(params.origin) : '';
67
+ const destination = params.destination != null ? String(params.destination) : '';
68
+ const departureDate = params.departure_date != null ? String(params.departure_date) : '';
69
+ const tripType = String(params.trip_type ?? 'round_trip').toLowerCase();
70
+ if (tripType === 'one_way' || tripType === 'oneway' || tripType === '2') {
71
+ return `One way flights from ${origin} to ${destination} on ${departureDate}`;
72
+ }
73
+ if (params.return_date) {
74
+ return `Round trip flights from ${origin} to ${destination} departing ${departureDate} returning ${params.return_date}`;
75
+ }
76
+ return `Flights from ${origin} to ${destination} on ${departureDate}`;
77
+ }
78
+
79
+ export function prepareParams(params?: Params): Params {
80
+ const p: Params = params ?? {};
81
+ return {
82
+ ...p,
83
+ bootstrap_query: buildBootstrapQuery(p),
84
+ };
85
+ }
86
+
87
+ function hasNonDefaultFilters(params: Params): boolean {
88
+ if (params.max_stops != null && params.max_stops !== '' && Number(params.max_stops) !== 3) {
89
+ return true;
90
+ }
91
+ return Boolean(
92
+ params.airlines ||
93
+ num(params.max_price) ||
94
+ params.outbound_times ||
95
+ params.return_times ||
96
+ num(params.max_duration) ||
97
+ num(params.carry_on_bags),
98
+ );
99
+ }
100
+
65
101
  export function transform(
66
102
  method: string,
67
103
  url: string,
68
104
  responses: Record<string, any>,
69
105
  params?: Params,
106
+ state?: Record<string, unknown>,
70
107
  ): { url: string; body: string } {
71
108
  const p: Params = params ?? {};
109
+ const observedSearchBody =
110
+ typeof state?.observed_search_body === 'string' ? state.observed_search_body : undefined;
111
+ if (observedSearchBody && !hasNonDefaultFilters(p)) {
112
+ return { url, body: observedSearchBody };
113
+ }
114
+
72
115
  const tripType = mapTripType(p.trip_type);
73
116
  const stops = p.max_stops != null && p.max_stops !== '' ? mapStops(p.max_stops) : 0;
74
117
  const { alliances, carriers } = parseAirlines(p.airlines);
@@ -113,6 +156,8 @@ export function transform(
113
156
  // CONFIG[10] wire form is [1, <carry-on count>]; shared builder emits
114
157
  // [carryOn, checked], so map count -> checked slot, constant 1 -> first.
115
158
  bags: carryOn != null ? { carryOn: 1, checked: carryOn } : undefined,
159
+ searchContextToken:
160
+ typeof state?.search_context_token === 'string' ? state.search_context_token : undefined,
116
161
  };
117
162
 
118
163
  return sharedTransform(method, url, responses, mapped);
@@ -101,23 +101,72 @@
101
101
  }
102
102
  ],
103
103
  "bootstrap": {
104
- "url": "https://www.google.com/travel/flights",
104
+ "url": "https://www.google.com/travel/flights?q=${param.bootstrap_query}&curr=USD",
105
105
  "waitUntil": "domcontentloaded",
106
106
  "timeoutMs": 30000,
107
107
  "captures": [
108
108
  {
109
- "source": "html_regex",
109
+ "source": "request_url_regex",
110
110
  "name": "f_sid",
111
- "pattern": "\"FdrFJe\":\"([^\"]+)\"",
111
+ "pattern": "[?&]f\\.sid=([^&]+)",
112
112
  "group": 1,
113
- "required": false,
113
+ "method": "POST",
114
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
115
+ "mode": "last",
116
+ "required": true,
114
117
  "capability": "browser_bootstrap"
115
118
  },
116
119
  {
117
- "source": "html_regex",
120
+ "source": "request_url_regex",
118
121
  "name": "bl",
119
- "pattern": "\"cfb2h\":\"([^\"]+)\"",
122
+ "pattern": "[?&]bl=([^&]+)",
123
+ "group": 1,
124
+ "method": "POST",
125
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
126
+ "mode": "last",
127
+ "required": true,
128
+ "capability": "browser_bootstrap"
129
+ },
130
+ {
131
+ "source": "request_header",
132
+ "name": "bgr",
133
+ "header": "X-Goog-BatchExecute-Bgr",
134
+ "method": "POST",
135
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
136
+ "mode": "last",
137
+ "required": true,
138
+ "capability": "browser_bootstrap"
139
+ },
140
+ {
141
+ "source": "request_url_regex",
142
+ "name": "reqid",
143
+ "pattern": "[?&]_reqid=([^&]+)",
144
+ "group": 1,
145
+ "method": "POST",
146
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
147
+ "mode": "last",
148
+ "required": true,
149
+ "capability": "browser_bootstrap"
150
+ },
151
+ {
152
+ "source": "request_body_regex",
153
+ "name": "search_context_token",
154
+ "pattern": "%5B%5Bnull%2Cnull%2Cnull%2C%5C%22(.+?)%5C%22%5D",
120
155
  "group": 1,
156
+ "method": "POST",
157
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
158
+ "mode": "last",
159
+ "required": false,
160
+ "capability": "browser_bootstrap"
161
+ },
162
+ {
163
+ "source": "request_body_regex",
164
+ "name": "observed_search_body",
165
+ "pattern": "^(f\\.req=.*)$",
166
+ "group": 1,
167
+ "method": "POST",
168
+ "urlPattern": "FlightsFrontendService/GetShoppingResults",
169
+ "mode": "last",
121
170
  "required": false,
122
171
  "capability": "browser_bootstrap"
123
172
  }
@@ -126,11 +175,13 @@
126
175
  "requests": [
127
176
  {
128
177
  "method": "POST",
129
- "url": "https://www.google.com/_/FlightsFrontendUi/data/travel.frontend.flights.FlightsFrontendService/GetShoppingResults?f.sid=${state.f_sid}&bl=${state.bl}&hl=en-US&soc-app=162&soc-platform=1&soc-device=1&_reqid=1708023&rt=c",
178
+ "url": "https://www.google.com/_/FlightsFrontendUi/data/travel.frontend.flights.FlightsFrontendService/GetShoppingResults?f.sid=${state.f_sid}&bl=${state.bl}&hl=en-US&soc-app=162&soc-platform=1&soc-device=1&_reqid=${state.reqid}&rt=c",
130
179
  "headers": {
131
180
  "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8",
132
181
  "X-Same-Domain": "1",
133
- "Referer": "https://www.google.com/travel/flights",
182
+ "Referer": "https://www.google.com/travel/flights?q=${param.bootstrap_query}&curr=USD",
183
+ "Accept-Language": "en-US,en;q=0.9",
184
+ "X-Goog-BatchExecute-Bgr": "${state.bgr}",
134
185
  "x-goog-ext-259736195-jspb": "[\"en-US\",\"US\",\"USD\",2,null,[420],null,null,7,[]]"
135
186
  },
136
187
  "body": "f.req=${param.origin}|${param.destination}|${param.departure_date}|${param.return_date}|${param.trip_type}|${param.max_stops}|${param.airlines}|${param.max_price}|${param.outbound_times}|${param.return_times}|${param.max_duration}|${param.carry_on_bags}&",
@@ -139,5 +190,9 @@
139
190
  ],
140
191
  "requestTransformModule": "./request-transform.ts",
141
192
  "parserModule": "./parser.ts",
193
+ "execution": {
194
+ "minCallSpacingMs": 2000,
195
+ "skipPlaybookFallback": true
196
+ },
142
197
  "liveVerified": true
143
198
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "imprint-mcp",
3
- "version": "0.4.6",
3
+ "version": "0.4.7",
4
4
  "description": "Teach an AI agent how to use any website. Once. Records a real browser session + narration; generates a deterministic MCP tool plus a DOM-replay playbook fallback.",
5
5
  "type": "module",
6
6
  "exports": {
@@ -59,7 +59,7 @@ You receive:
59
59
 
60
60
  1. **Emit exactly one `perTool` entry per `selectedTools` entry**, using the same `toolName`. Do not invent or drop tools.
61
61
  2. **Only hoist a shared module when ≥2 selected tools genuinely share it.** Single-use logic stays inside that tool's own parser.ts / request-transform.ts — do NOT create a `_shared/` module for it.
62
- 3. **`request-transform`** — URL signing or body construction shared across tools. Wire-up: the consuming tool sets `requestTransformModule: "../_shared/<name>.ts"`. Ground it in `ephemeralValues` (browser_minted, high-entropy query param) and `sourceSeqs`. The exported `transform(method, url, responses, params?)` returns the signed URL (or `{ url, body? }`).
62
+ 3. **`request-transform`** — URL signing, body construction, or bootstrap-param preparation shared across tools. Wire-up: the consuming tool sets `requestTransformModule: "../_shared/<name>.ts"`. Ground it in `ephemeralValues` (browser_minted, high-entropy query param) and `sourceSeqs`. The exported `transform(method, url, responses, params?, state?)` returns the signed URL (or `{ url, body?, headers? }`); an optional `prepareParams(params)` can return primitive params used before bootstrap URL substitution.
63
63
  4. **`parser-helper`** — a decoder/normalizer ≥2 tools' parsers call (e.g. a shared JSPB walker, a shared field mapper). The consuming tool's parser.ts does `import { ... } from '../_shared/<name>.ts'`. Ground it in a captured response body (`sourceSeqs`).
64
64
  5. **`types`** — shared TypeScript interfaces used by ≥2 parsers. Type-only; no runtime behavior.
65
65
  6. **Auth is NEVER a shared module.** Login is request data, and the runtime cannot run a shared sub-workflow. Put the exact recipe in each tool's `authRecipe` (login seqs, credential names, captures with `${state.X}` wiring) and set `required: false` with empty arrays when a tool needs no login. Every authed tool replicates the same recipe inline.