@upcrawl/sdk 1.4.0 → 1.4.1

package/README.md

@@ -314,71 +314,220 @@ const result: ScrapeResponse = await Upcrawl.scrape(options);
 | `Upcrawl.generatePdfFromUrl(options)` | Generate PDF from a URL |
 | `Upcrawl.executeCode(options)` | Execute code in an isolated sandbox |
 
-### Scrape Options
-
-| Option | Type | Description |
-|--------|------|-------------|
-| `url` | `string` | URL to scrape (required) |
-| `type` | `'markdown' \| 'html'` | Output format |
-| `onlyMainContent` | `boolean` | Remove nav, ads, footers |
-| `extractMetadata` | `boolean` | Extract page metadata |
-| `timeoutMs` | `number` | Request timeout (1000-120000) |
-| `waitUntil` | `string` | Page load strategy |
-| `summary` | `{ query: string }` | LLM summarization |
-
-### Search Options
-
-| Option | Type | Description |
-|--------|------|-------------|
-| `queries` | `string[]` | Search queries (1-20) |
-| `limit` | `number` | Results per query (1-100) |
-| `location` | `string` | Search location (e.g., 'US') |
-| `includeDomains` | `string[]` | Only include these domains |
-| `excludeDomains` | `string[]` | Exclude these domains |
-
-### Generate PDF Options (from HTML)
-
-| Option | Type | Description |
-|--------|------|-------------|
-| `html` | `string` | HTML content to convert (required) |
-| `title` | `string` | Filename for the PDF |
-| `pageSize` | `'A4' \| 'Letter' \| 'Legal'` | Page size (default: A4) |
-| `landscape` | `boolean` | Landscape orientation |
-| `margin` | `{ top?, right?, bottom?, left? }` | Page margins |
-| `printBackground` | `boolean` | Print background graphics |
-| `skipChartWait` | `boolean` | Skip chart rendering wait |
-| `timeoutMs` | `number` | Timeout (5000-120000) |
-
-### Generate PDF Options (from URL)
-
-| Option | Type | Description |
-|--------|------|-------------|
-| `url` | `string` | URL to convert (required) |
-| `title` | `string` | Filename for the PDF |
-| `pageSize` | `'A4' \| 'Letter' \| 'Legal'` | Page size (default: A4) |
-| `landscape` | `boolean` | Landscape orientation |
-| `margin` | `{ top?, right?, bottom?, left? }` | Page margins |
-| `printBackground` | `boolean` | Print background graphics |
-| `timeoutMs` | `number` | Timeout (5000-120000) |
-
-### Execute Code Options
-
-| Option | Type | Description |
-|--------|------|-------------|
-| `code` | `string` | Code to execute (required) |
-| `language` | `'python'` | Language runtime (default: python) |
-
-### Execute Code Response
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `stdout` | `string` | Standard output |
-| `stderr` | `string` | Standard error |
-| `exitCode` | `number` | Exit code (0 = success, 124 = timeout) |
-| `executionTimeMs` | `number` | Execution time in milliseconds |
-| `timedOut` | `boolean` | Whether execution timed out |
-| `memoryUsageMb` | `number` | Peak memory usage in MB |
-| `cost` | `number` | Cost in USD |
+<!-- TYPES-START -->
+<!-- Auto-generated by `pnpm generate:docs` — do not edit manually -->
+
+### UpcrawlConfig
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `apiKey` | string | No | Your Upcrawl API key |
+| `baseUrl` | string | No | Custom API base URL |
+| `timeout` | number | No | Request timeout in milliseconds |
+
+### SummaryQuery
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `query` | string | Yes | Query/instruction for content summarization |
+
+### ScrapeOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `url` | string | Yes | URL to scrape (required) |
+| `type` | `"html"` \| `"markdown"` | No | Output format: html or markdown. Defaults to "html" |
+| `onlyMainContent` | boolean | No | Extract only main content (removes nav, ads, footers). Defaults to true |
+| `extractMetadata` | boolean | No | Whether to extract page metadata |
+| `summary` | object | No | Summary query for LLM summarization |
+| `timeoutMs` | number | No | Custom timeout in milliseconds (1000-120000) |
+| `waitUntil` | `"load"` \| `"domcontentloaded"` \| `"networkidle"` | No | Wait strategy for page load |
+
+### ScrapeMetadata
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `title` | string | No |  |
+| `description` | string | No |  |
+| `canonicalUrl` | string | No |  |
+| `finalUrl` | string | No |  |
+| `contentType` | string | No |  |
+| `contentLength` | number | No |  |
+
+### ScrapeResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `url` | string | Yes | Original URL that was scraped |
+| `html` | string \| null | No | Rendered HTML content (when type is html) |
+| `markdown` | string \| null | No | Content converted to Markdown (when type is markdown) |
+| `statusCode` | number \| null | Yes | HTTP status code |
+| `success` | boolean | Yes | Whether scraping was successful |
+| `error` | string | No | Error message if scraping failed |
+| `timestamp` | string | Yes | ISO timestamp when scraping completed |
+| `loadTimeMs` | number | Yes | Time taken to load and render the page in milliseconds |
+| `metadata` | object | No | Additional page metadata |
+| `retryCount` | number | Yes | Number of retry attempts made |
+| `cost` | number | No | Cost in USD for this scrape operation |
+| `content` | string \| null | No | Content after summarization (when summary query provided) |
+
+### BatchScrapeOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `urls` | string \| object[] | Yes | Array of URLs to scrape (strings or detailed request objects) |
+| `type` | `"html"` \| `"markdown"` | No | Output format: html or markdown |
+| `onlyMainContent` | boolean | No | Extract only main content (removes nav, ads, footers) |
+| `summary` | object | No | Summary query for LLM summarization |
+| `batchTimeoutMs` | number | No | Global timeout for entire batch operation in milliseconds (10000-600000) |
+| `failFast` | boolean | No | Whether to stop on first error |
+
+### BatchScrapeResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `results` | object[] | Yes | Array of scrape results |
+| `total` | number | Yes | Total number of URLs processed |
+| `successful` | number | Yes | Number of successful scrapes |
+| `failed` | number | Yes | Number of failed scrapes |
+| `totalTimeMs` | number | Yes | Total time taken for batch operation in milliseconds |
+| `timestamp` | string | Yes | Timestamp when batch operation completed |
+| `cost` | number | No | Total cost in USD for all scrape operations |
+
+### SearchOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `queries` | string[] | Yes | Array of search queries to execute (1-20) |
+| `limit` | number | No | Number of results per query (1-100). Defaults to 10 |
+| `location` | string | No | Location for search (e.g., "IN", "US") |
+| `includeDomains` | string[] | No | Domains to include (will add site: to query) |
+| `excludeDomains` | string[] | No | Domains to exclude (will add -site: to query) |
+
+### SearchResultWeb
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `url` | string | Yes | URL of the search result |
+| `title` | string | Yes | Title of the search result |
+| `description` | string | Yes | Description/snippet of the search result |
+
+### SearchResultItem
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `query` | string | Yes | The search query |
+| `success` | boolean | Yes | Whether the search was successful |
+| `results` | object[] | Yes | Parsed search result links |
+| `error` | string | No | Error message if failed |
+| `loadTimeMs` | number | No | Time taken in milliseconds |
+| `cost` | number | No | Cost in USD for this query |
+
+### SearchResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `results` | object[] | Yes | Array of search results per query |
+| `total` | number | Yes | Total number of queries |
+| `successful` | number | Yes | Number of successful searches |
+| `failed` | number | Yes | Number of failed searches |
+| `totalTimeMs` | number | Yes | Total time in milliseconds |
+| `timestamp` | string | Yes | ISO timestamp |
+| `cost` | number | No | Total cost in USD |
+
+### PdfMargin
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `top` | string | No |  |
+| `right` | string | No |  |
+| `bottom` | string | No |  |
+| `left` | string | No |  |
+
+### GeneratePdfOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `html` | string | Yes | Complete HTML content to convert to PDF (required) |
+| `title` | string | No | Title used for the exported filename |
+| `pageSize` | `"A4"` \| `"Letter"` \| `"Legal"` | No | Page size. Defaults to "A4" |
+| `landscape` | boolean | No | Landscape orientation. Defaults to false |
+| `margin` | object | No | Page margins (e.g., { top: "20mm", right: "20mm", bottom: "20mm", left: "20mm" }) |
+| `printBackground` | boolean | No | Print background graphics and colors. Defaults to true |
+| `skipChartWait` | boolean | No | Skip waiting for chart rendering signal. Defaults to false |
+| `timeoutMs` | number | No | Timeout in milliseconds (5000-120000). Defaults to 30000 |
+
+### GeneratePdfFromUrlOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `url` | string | Yes | URL to navigate to and convert to PDF (required) |
+| `title` | string | No | Title used for the exported filename |
+| `pageSize` | `"A4"` \| `"Letter"` \| `"Legal"` | No | Page size. Defaults to "A4" |
+| `landscape` | boolean | No | Landscape orientation. Defaults to false |
+| `margin` | object | No | Page margins |
+| `printBackground` | boolean | No | Print background graphics and colors. Defaults to true |
+| `timeoutMs` | number | No | Timeout in milliseconds (5000-120000). Defaults to 30000 |
+
+### PdfResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `success` | boolean | Yes | Whether PDF generation succeeded |
+| `url` | string | No | Public URL of the generated PDF |
+| `filename` | string | No | Generated filename |
+| `blobName` | string | No | Blob storage path |
+| `error` | string | No | Error message on failure |
+| `durationMs` | number | Yes | Total time taken in milliseconds |
+
+### ExecuteCodeOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `code` | string | Yes | Code to execute (required) |
+| `language` | `"python"` | No | Language runtime. Defaults to "python" |
+
+### ExecuteCodeResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `stdout` | string | Yes | Standard output from the executed code |
+| `stderr` | string | Yes | Standard error from the executed code |
+| `exitCode` | number | Yes | Process exit code (0 = success, 124 = timeout) |
+| `executionTimeMs` | number | Yes | Execution time in milliseconds |
+| `timedOut` | boolean | Yes | Whether execution was killed due to timeout |
+| `memoryUsageMb` | number | No | Peak memory usage in megabytes |
+| `error` | string | No | Error message if execution infrastructure failed |
+| `cost` | number | No | Cost in USD for this execution |
+
+### UpcrawlErrorResponse
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `error` | object | Yes |  |
+| `statusCode` | number | No |  |
+
+### CreateBrowserSessionOptions
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `width` | number | No | Browser viewport width (800-3840). Defaults to 1280 |
+| `height` | number | No | Browser viewport height (600-2160). Defaults to 720 |
+| `headless` | boolean | No | Run browser in headless mode. Defaults to true |
+
+### BrowserSession
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `sessionId` | string | Yes | Unique session identifier |
+| `wsEndpoint` | string | Yes | WebSocket URL for connecting with Playwright/Puppeteer |
+| `vncUrl` | string \| null | Yes | VNC URL for viewing the browser (if available) |
+| `affinityCookie` | string | No | Affinity cookie for sticky session routing (format: SCRAPER_AFFINITY=xxx) - extracted from response headers |
+| `createdAt` | Date | Yes | Session creation timestamp |
+| `width` | number | Yes | Browser viewport width |
+| `height` | number | Yes | Browser viewport height |
+
+<!-- TYPES-END -->
 
 ## License