n8n-nodes-crawlbrulee 0.2.0 → 0.3.1

package/README.md

@@ -47,7 +47,7 @@ pick a resource, then an operation:
 | Scrape | Scrape URL | fetch one url, get back the content you asked for |
 | Scrape | Scrape URL (Async) | start a background job, get back its job id |
 | Scrape | Get Scrape Status | pending, running, done or failed |
-| Scrape | Get Scrape Result | the content of a finished job |
+| Scrape | Get Scrape Result | the content of a finished job, for 24 hours after it was submitted |
 | Map | Map Website | a site's urls, from its sitemap and its homepage links |
 | Account | Get Credit Usage | credits used and left this billing cycle, plus your concurrency limit |
 | Account | Get Account Info | the organization and token behind the api key |
@@ -62,7 +62,7 @@ every field has a hint in n8n. three things the hints don't tell you:
 
 ### screenshots as files
 
-turn on **Download Screenshot** — it's on **Scrape URL** and on **Get Scrape Result** — and the node attaches the image as binary data under `screenshot`, plus one key per tile (`screenshot_slice_0`, `screenshot_slice_1`, …) when you set Slice Height. the urls stay in the json too. a download can fail, since a signed url can expire; the item still comes through with its page content and the reason lands on the json as `screenshot_download_error`.
+turn on **Download Screenshot** — it's on **Scrape URL** and on **Get Scrape Result** — and the node attaches the image as binary data under `screenshot`, plus one key per tile (`screenshot_slice_0`, `screenshot_slice_1`, …) when you set Slice Height. the urls stay in the json too, but they are signed links that expire 24 hours after the scrape (for **Scrape URL (Async)**, 24 hours after the job was submitted), so turn this on when you need the image later in the workflow. a download can fail, since a signed url can expire; the item still comes through with its page content and the reason lands on the json as `screenshot_download_error`.
 
 ### what comes back
 
@@ -76,6 +76,10 @@ set the node's "On Error" setting to continue and a failed item comes back as `{
 
 a page the site really served is a normal item, whatever its status. a 404, 410 or 503 page comes back with its content, like any other page. it is not a node error, so "On Error" never sees it. when the status matters to your workflow, check it before you use the content — for example an **If** node on `{{ $json.page_status_code }}` is less than 400.
 
+### zero data retention
+
+turn on **Zero Data Retention** under **Options** of **Scrape URL**, **Scrape URL (Async)** or **Map Website** to keep the result out of the shared cache. anything stored to deliver it is kept for 24 hours, then deleted. it adds 1 credit and must be enabled for your organization. if it is not, the call fails with a clear message and costs no credits. see [zero data retention](https://crawlbrulee.com/docs/zero-data-retention).
+
 ### what a call cost
 
 scrape, map and finished-job responses carry `response_meta.usage`, so the cost of each call is in front of you:
@@ -86,11 +90,10 @@ scrape, map and finished-job responses carry `response_meta.usage`, so the cost
 | `engine_credit_cost` | the engine's base price: 1 for `http`, 3 for `browser`, 5 for `screenshot`, 0 for `cache` |
 | `proxy_multiplier` | 1 for the Basic proxy tier, 5 for Advanced |
 | `screenshot_slicing_credit_cost` | 1 when the screenshot was cut into slices on this call, else 0 (reusing slices from the cache costs 0). not on **Map Website** |
+| `zero_data_retention_credit_cost` | 1 when **Zero Data Retention** added its credit to this call, else 0 |
 | `engine`, `proxy` | the engine and the proxy tier that did the work |
 
-`total_credit_cost` is always `engine_credit_cost × proxy_multiplier + screenshot_slicing_credit_cost`. a page we don't bill, such as one the site answered with a 5xx status, has 0 in every `*_credit_cost` field. the [credits and pricing](https://crawlbrulee.com/docs/credits-and-pricing) page lists what is billed.
-
-`credits` and `screenshot_slices` are still there, with the same values as `total_credit_cost` and `screenshot_slicing_credit_cost`. they are deprecated and will be removed in a future version, so read the new names in new workflows.
+`total_credit_cost` is always `engine_credit_cost × proxy_multiplier + screenshot_slicing_credit_cost + zero_data_retention_credit_cost` (map has no slicing part). a page we don't bill, such as one the site answered with a 5xx status, has 0 in every `*_credit_cost` field. the [credits and pricing](https://crawlbrulee.com/docs/credits-and-pricing) page lists what is billed.
 
 ## the crawlbrulee Trigger
 
@@ -106,7 +109,7 @@ with the secret set, the trigger checks the HMAC-SHA256 signature on every deliv
 
 ## errors
 
-the node turns api errors into messages you can act on: a rate limit says how long to wait, out of credits and concurrency limit are each named, an anti-bot block says so with no retry hint, a scrape error points you at the Advanced proxy tier and at Require JS, and a site we could not reach at all (for example it timed out, or its certificate is bad) tells you to check the url and try again later. the raw response stays on the error for the n8n error view, and every error name is in the [error reference](https://crawlbrulee.com/docs/errors).
+the node turns api errors into messages you can act on: a rate limit says how long to wait, a Zero Data Retention request your organization cannot make yet says to turn the option off or ask us to enable it, out of credits and concurrency limit are each named, an anti-bot block says so with no retry hint, a scrape error points you at the Advanced proxy tier and at Require JS, and a site we could not reach at all (for example it timed out, or its certificate is bad) tells you to check the url and try again later. the raw response stays on the error for the n8n error view, and every error name is in the [error reference](https://crawlbrulee.com/docs/errors).
 
 ## development
 

package/dist/nodes/Crawlbrulee/builders/map.js

@@ -8,6 +8,8 @@ function buildMapBody(_node, _itemIndex, p) {
     const o = (_a = p.options) !== null && _a !== void 0 ? _a : {};
     if (o.sitemapOnly === true)
         body.sitemap_only = true;
+    if (o.zeroDataRetention === true)
+        body.zero_data_retention = true;
     const types = {};
     if (typeof o.internal === 'boolean')
         types.internal = o.internal;

package/dist/nodes/Crawlbrulee/builders/scrape.js

@@ -88,6 +88,8 @@ function buildScrapeBody(node, itemIndex, p) {
         body.proxy = o.proxy;
     if (o.requireJs === true)
         body.require_js = true;
+    if (o.zeroDataRetention === true)
+        body.zero_data_retention = true;
     const cleanup = {};
     if (typeof o.removeAdsAndPopups === 'boolean')
         cleanup.ads_and_popups = o.removeAdsAndPopups;

package/dist/nodes/Crawlbrulee/descriptions/map.js

@@ -92,6 +92,7 @@ exports.mapFields = [
                 default: false,
                 description: 'Whether to read only sitemap.xml and skip homepage link extraction',
             },
+            shared_1.zeroDataRetentionOption,
         ],
     },
 ];

package/dist/nodes/Crawlbrulee/descriptions/scrape.js

@@ -17,13 +17,13 @@ exports.scrapeOperations = {
             name: 'Get Scrape Result',
             value: 'getScrapeResult',
             action: 'Get scrape result',
-            description: "Fetch the content of a finished async scrape. The site's own HTTP status is in page_status_code.",
+            description: "Fetch the content of a finished async scrape, up to 24 hours after it was submitted. The site's own HTTP status is in page_status_code.",
         },
         {
             name: 'Get Scrape Status',
             value: 'getScrapeStatus',
             action: 'Get scrape status',
-            description: 'Check whether an async scrape is pending, running, done or failed',
+            description: 'Check whether an async scrape is pending, running, done or failed. Available for 24 hours after it was submitted.',
         },
         {
             name: 'Scrape URL',
@@ -200,7 +200,7 @@ exports.scrapeFields = [
         name: 'downloadScreenshot',
         type: 'boolean',
         default: false,
-        description: 'Whether to download the screenshot (and its slices) into binary data on the item',
+        description: 'Whether to download the screenshot (and its slices) into binary data on the item. Screenshot links are signed and expire 24 hours after the scrape.',
         displayOptions: {
             show: {
                 resource: ['scrape'],
@@ -244,6 +244,7 @@ exports.scrapeFields = [
                 default: false,
                 description: 'Whether to render JavaScript in a headless browser first. Adds latency and credits.',
             },
+            shared_1.zeroDataRetentionOption,
         ],
     },
     {
@@ -268,7 +269,7 @@ exports.scrapeFields = [
         name: 'downloadScreenshot',
         type: 'boolean',
         default: false,
-        description: 'Whether to download the screenshot (and its slices) into binary data when the result has one',
+        description: 'Whether to download the screenshot (and its slices) into binary data when the result has one. Screenshot links expire 24 hours after the job was submitted.',
         displayOptions: { show: { resource: ['scrape'], operation: ['getScrapeResult'] } },
     },
 ];

package/dist/nodes/Crawlbrulee/descriptions/shared.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.localeOption = exports.countryOption = exports.cacheMaxAgeOption = exports.proxyOption = void 0;
+exports.localeOption = exports.countryOption = exports.zeroDataRetentionOption = exports.cacheMaxAgeOption = exports.proxyOption = void 0;
 exports.proxyOption = {
     displayName: 'Proxy',
     name: 'proxy',
@@ -30,6 +30,13 @@ const cacheMaxAgeOption = (defaultText) => ({
     description: `Seconds, or an ISO-8601 cutoff. Cached results older than this are skipped. Default ${defaultText}.`,
 });
 exports.cacheMaxAgeOption = cacheMaxAgeOption;
+exports.zeroDataRetentionOption = {
+    displayName: 'Zero Data Retention',
+    name: 'zeroDataRetention',
+    type: 'boolean',
+    default: false,
+    description: 'Whether to keep the result out of the shared cache. Anything stored to deliver it is kept for 24 hours, then deleted. Adds 1 credit. Must be enabled for your organization.',
+};
 exports.countryOption = {
     displayName: 'Country',
     name: 'country',

package/dist/nodes/Crawlbrulee/transport/errors.js

@@ -5,6 +5,7 @@ exports.toNodeApiError = toNodeApiError;
 const n8n_workflow_1 = require("n8n-workflow");
 const KEY_REJECTED = 'crawlbrulee rejected the API key. Check the key on the credential.';
 const UNREACHABLE = 'crawlbrulee could not reach the target site. Check the URL. If it is right, the site may be down or slow, so try again later. This costs no credits.';
+const ZDR_NOT_ENABLED = 'Zero data retention is not enabled for your organization. Turn the Zero Data Retention option off, or contact sales@crawlbrulee.com to have it turned on. This costs no credits.';
 const SCRAPE_HINT = 'The Advanced proxy tier has a higher success rate; enable Require JS for JavaScript-rendered content.';
 function isApiErrorBody(body) {
     return (typeof body === 'object' &&
@@ -42,6 +43,8 @@ function describeApiError(status, body) {
             return { message: body.message, description };
         case 'target_unreachable':
             return { message: UNREACHABLE, description };
+        case 'zero_data_retention_not_enabled':
+            return { message: ZDR_NOT_ENABLED, description };
         case 'scrape_error':
             return { message: `${body.message.replace(/[.\s]*$/, '')}. ${SCRAPE_HINT}`, description };
         default:

package/dist/nodes/Crawlbrulee/transport/request.js

@@ -3,7 +3,7 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.USER_AGENT = void 0;
 exports.crawlbruleeRequest = crawlbruleeRequest;
 const errors_1 = require("./errors");
-exports.USER_AGENT = 'n8n-nodes-crawlbrulee/0.2.0';
+exports.USER_AGENT = 'n8n-nodes-crawlbrulee/0.3.1';
 function stripTrailingSlash(url) {
     return url.replace(/\/+$/, '');
 }

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "n8n-nodes-crawlbrulee",
-	"version": "0.2.0",
+	"version": "0.3.1",
 	"description": "crawlbrulee for n8n — EU-native web scraping for AI agents & developers. scrape any url to markdown, html, links, images, metadata or a screenshot, map a site's urls, and react to finished background scrapes.",
 	"license": "MIT",
 	"homepage": "https://crawlbrulee.com",
@@ -57,7 +57,7 @@
 		]
 	},
 	"devDependencies": {
-		"@crawlbrulee/sdk": "^1.1.0",
+		"@crawlbrulee/sdk": "^1.2.1",
 		"@n8n/node-cli": "0.48.6",
 		"@types/node": "^22.0.0",
 		"eslint": "9.32.0",