n8n-nodes-scraperapi 0.2.4 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@ Bring web data into self-hosted n8n agents and workflows with Geonode’s commun
6
6
 
7
7
  **[Try Geonode for free](https://app.geonode.com/sign-up)**
8
8
 
9
- [Integrations page](https://geonode.com/integrations) · [Geonode API docs](https://docs.geonode.com/docs/scraper-api) · [n8n setup guide](https://docs.geonode.com/docs/scraper-api/guides/sdks/n8n) · [View package on npm](https://www.npmjs.com/package/n8n-nodes-scraperapi) · [View source on GitHub](https://github.com/geonodecom/integration-n8n) · [Complete guide](docs/guide.md)
9
+ [Integrations page](https://geonode.com/integrations/integration-n8n) · [Geonode API docs](https://docs.geonode.com/docs/scraper-api) · [n8n setup guide](https://docs.geonode.com/docs/scraper-api/guides/sdks/n8n) · [View package on npm](https://www.npmjs.com/package/n8n-nodes-scraperapi) · [View source on GitHub](https://github.com/geonodecom/integration-n8n) · [Complete guide](docs/guide.md)
10
10
 
11
11
  **For self-hosted n8n.** This is a community node, not a verified n8n Cloud node. The npm package name remains `n8n-nodes-scraperapi`.
12
12
 
@@ -18,13 +18,13 @@ Bring web data into self-hosted n8n agents and workflows with Geonode’s commun
18
18
 
19
19
  - **Feed a knowledge base.** Extract source pages as Markdown, then connect your own chunking, embedding and storage nodes.
20
20
  - **Refresh a website dataset.** Start a bounded crawl and process the returned content after the job completes.
21
- - **React to completed extraction.** Use Scraper API Trigger to start follow-up processing for extraction or crawl completion.
21
+ - **React to completed extraction.** Use Geonode Scraper API Trigger to start follow-up processing for extraction or crawl completion.
22
22
 
23
23
  ## Get started
24
24
 
25
25
  1. In n8n, open **Settings → Community Nodes → Install** and enter `n8n-nodes-scraperapi`.
26
- 2. Create a **Scraper API** credential. Paste your Geonode API key and keep the base URL `https://scraper.geonode.io`.
27
- 3. Add a **Scraper API** node. Choose **Extract → Run Extraction**, enter a URL, and choose **Markdown** as the output format.
26
+ 2. Create a **Geonode Scraper API** credential. Paste your Geonode API key and keep the base URL `https://scraper.geonode.io`.
27
+ 3. Add a **Geonode Scraper API** node. Choose **Extract → Run Extraction**, enter a URL, and choose **Markdown** as the output format.
28
28
  4. Execute the step. The result contains the page content in `data.markdown` and its cost in `tokens_charged`.
29
29
 
30
30
  You can obtain a key from **API Keys** in the [Geonode dashboard](https://app.geonode.com/). The credential test checks authenticated usage statistics.
@@ -38,7 +38,7 @@ You can obtain a key from **API Keys** in the [Geonode dashboard](https://app.ge
38
38
  | Collect a site | Crawl from a seed URL with depth and page limits. |
39
39
  | Discover URLs | Map a site's links before choosing pages to extract. |
40
40
  | Search the web | Run search jobs with engine and locale controls. |
41
- | React to completed jobs | Use **Scraper API Trigger** for extraction or crawl completion. |
41
+ | React to completed jobs | Use **Geonode Scraper API Trigger** for extraction or crawl completion. |
42
42
  | Monitor usage | Read statistics and manage webhook subscriptions. |
43
43
 
44
44
  For long jobs, use asynchronous extraction or a start/status sequence. The [full guide](docs/guide.md) covers polling, webhook signature checks, errors and retries.
@@ -46,8 +46,8 @@ For long jobs, use asynchronous extraction or a start/status sequence. The [full
46
46
  <!-- BEGIN SHARED WORKFLOW GUIDE -->
47
47
  ## Example: a URL to a spreadsheet row
48
48
 
49
- 1. Supply a URL from a trigger or an Edit Fields node, then add Scraper API → Extract → Run Extraction.
50
- 2. Select your Scraper API credential, map the incoming URL, choose Markdown and keep synchronous processing for the first test.
49
+ 1. Supply a URL from a trigger or an Edit Fields node, then add Geonode Scraper API → Extract → Run Extraction.
50
+ 2. Select your Geonode Scraper API credential, map the incoming URL, choose Markdown and keep synchronous processing for the first test.
51
51
  3. Execute the step and inspect data.markdown, metadata.url and tokens_charged in the returned item's JSON.
52
52
  4. Connect a separate Google Sheets node and map those fields into Content, Source URL and Usage columns using n8n's expression editor.
53
53
  5. Test the destination write before activating the workflow. Add a separate AI node if you want a summary rather than the raw Markdown.
@@ -58,7 +58,7 @@ For long jobs, use asynchronous extraction or a start/status sequence. The [full
58
58
 
59
59
  Each successful input item produces an output item whose JSON contains the API response. A queued extraction returns job information, while completed extraction returns content. Do not map a job ID as if it were Markdown.
60
60
 
61
- Scraper API Trigger supports extraction and crawl completion. Activation registers a webhook and deactivation removes it. Keep signature verification enabled and use a callback URL reachable by the API.
61
+ Geonode Scraper API Trigger supports extraction and crawl completion. Activation registers a webhook and deactivation removes it. Keep signature verification enabled and use a callback URL reachable by the API.
62
62
 
63
63
  If Continue On Fail is enabled, a failed item can contain an error string. Route those items separately instead of sending them to your normal content destination.
64
64
 
@@ -2,6 +2,7 @@ import { IAuthenticateGeneric, ICredentialTestRequest, ICredentialType, INodePro
2
2
  export declare class ScraperApiApi implements ICredentialType {
3
3
  name: string;
4
4
  displayName: string;
5
+ icon: "file:scraper-geonode.svg";
5
6
  documentationUrl: string;
6
7
  properties: INodeProperties[];
7
8
  authenticate: IAuthenticateGeneric;
@@ -4,7 +4,8 @@ exports.ScraperApiApi = void 0;
4
4
  class ScraperApiApi {
5
5
  constructor() {
6
6
  this.name = 'scraperApiApi';
7
- this.displayName = 'Scraper API';
7
+ this.displayName = 'Geonode Scraper API';
8
+ this.icon = 'file:scraper-geonode.svg';
8
9
  this.documentationUrl = 'https://docs.geonode.com/docs/scraper-api/v1/reference';
9
10
  this.properties = [
10
11
  {
@@ -1 +1 @@
1
- {"version":3,"file":"ScraperApiApi.credentials.js","sourceRoot":"","sources":["../../credentials/ScraperApiApi.credentials.ts"],"names":[],"mappings":";;;AAOA,MAAa,aAAa;IAA1B;QACE,SAAI,GAAG,eAAe,CAAC;QACvB,gBAAW,GAAG,aAAa,CAAC;QAC5B,qBAAgB,GAAG,wDAAwD,CAAC;QAE5E,eAAU,GAAsB;YAC9B;gBACE,WAAW,EAAE,UAAU;gBACvB,IAAI,EAAE,SAAS;gBACf,IAAI,EAAE,QAAQ;gBACd,OAAO,EAAE,4BAA4B;gBACrC,WAAW,EAAE,4BAA4B;gBACzC,WAAW,EAAE,4DAA4D;gBACzE,QAAQ,EAAE,IAAI;aACf;YACD;gBACE,WAAW,EAAE,SAAS;gBACtB,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,QAAQ;gBACd,WAAW,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE;gBAC/B,OAAO,EAAE,EAAE;gBACX,QAAQ,EAAE,IAAI;gBACd,WAAW,EACT,+FAA+F;aAClG;SACF,CAAC;QAEF,iBAAY,GAAyB;YACnC,IAAI,EAAE,SAAS;YACf,UAAU,EAAE;gBACV,OAAO,EAAE;oBACP,WAAW,EAAE,0BAA0B;iBACxC;aACF;SACF,CAAC;QAIF,SAAI,GAA2B;YAC7B,OAAO,EAAE;gBACP,OAAO,EAAE,2BAA2B;gBACpC,GAAG,EAAE,gBAAgB;gBACrB,MAAM,EAAE,KAAK;aACd;SACF,CAAC;IACJ,CAAC;CAAA;AA7CD,sCA6CC"}
1
+ {"version":3,"file":"ScraperApiApi.credentials.js","sourceRoot":"","sources":["../../credentials/ScraperApiApi.credentials.ts"],"names":[],"mappings":";;;AAOA,MAAa,aAAa;IAA1B;QACE,SAAI,GAAG,eAAe,CAAC;QACvB,gBAAW,GAAG,qBAAqB,CAAC;QACpC,SAAI,GAAG,0BAAmC,CAAC;QAC3C,qBAAgB,GAAG,wDAAwD,CAAC;QAE5E,eAAU,GAAsB;YAC9B;gBACE,WAAW,EAAE,UAAU;gBACvB,IAAI,EAAE,SAAS;gBACf,IAAI,EAAE,QAAQ;gBACd,OAAO,EAAE,4BAA4B;gBACrC,WAAW,EAAE,4BAA4B;gBACzC,WAAW,EAAE,4DAA4D;gBACzE,QAAQ,EAAE,IAAI;aACf;YACD;gBACE,WAAW,EAAE,SAAS;gBACtB,IAAI,EAAE,QAAQ;gBACd,IAAI,EAAE,QAAQ;gBACd,WAAW,EAAE,EAAE,QAAQ,EAAE,IAAI,EAAE;gBAC/B,OAAO,EAAE,EAAE;gBACX,QAAQ,EAAE,IAAI;gBACd,WAAW,EACT,+FAA+F;aAClG;SACF,CAAC;QAEF,iBAAY,GAAyB;YACnC,IAAI,EAAE,SAAS;YACf,UAAU,EAAE;gBACV,OAAO,EAAE;oBACP,WAAW,EAAE,0BAA0B;iBACxC;aACF;SACF,CAAC;QAIF,SAAI,GAA2B;YAC7B,OAAO,EAAE;gBACP,OAAO,EAAE,2BAA2B;gBACpC,GAAG,EAAE,gBAAgB;gBACrB,MAAM,EAAE,KAAK;aACd;SACF,CAAC;IACJ,CAAC;CAAA;AA9CD,sCA8CC"}