@brightdata/brightdata-plugin 1.0.0 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +241 -155
  2. package/assets/brightdata-logo.png +0 -0
  3. package/assets/openclaw-logo.png +0 -0
  4. package/package.json +1 -1
package/README.md CHANGED
@@ -1,6 +1,34 @@
1
- # Bright Data Plugin for OpenClaw
1
+ <p align="center">
2
+ <img src="assets/brightdata-logo.png" height="48" alt="Bright Data" />&nbsp;&nbsp;&nbsp;&nbsp;
3
+ <img src="assets/openclaw-logo.png" height="108" alt="OpenClaw" />
4
+ </p>
2
5
 
3
- Web search, scraping, browser automation, and 50+ structured data tools — powered by [Bright Data](https://brightdata.com).
6
+ <h1 align="center">Bright Data Plugin for OpenClaw</h1>
7
+
8
+ <p align="center">
9
+ <strong>Web search · Scraping · Browser automation · 50+ structured data tools</strong><br/>
10
+ Powered by <a href="https://brightdata.com">Bright Data</a> — the world's leading web data platform
11
+ </p>
12
+
13
+ <p align="center">
14
+ <a href="https://www.npmjs.com/package/@brightdata/brightdata-plugin"><img src="https://img.shields.io/npm/v/@brightdata/brightdata-plugin?color=0066CC&label=npm" alt="npm version" /></a>
15
+ <a href="https://www.npmjs.com/package/@brightdata/brightdata-plugin"><img src="https://img.shields.io/npm/dm/@brightdata/brightdata-plugin?color=0066CC" alt="npm downloads" /></a>
16
+ <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT license" /></a>
17
+ <a href="https://brightdata.com"><img src="https://img.shields.io/badge/Bright%20Data-Powered-orange" alt="Bright Data Powered" /></a>
18
+ </p>
19
+
20
+ ---
21
+
22
+ ## What This Plugin Does
23
+
24
+ This plugin brings the full power of Bright Data's infrastructure directly into your OpenClaw agent — no manual API wiring, no proxies to configure. Install once, and your agent gains:
25
+
26
+ - **Real-time web search** via Google, Bing, and Yandex with geo-targeting
27
+ - **Bot-bypass scraping** through Bright Data Web Unlocker — handles CAPTCHAs, JS rendering, and rate limits automatically
28
+ - **Full browser automation** via a real Chromium instance routed through Bright Data's residential proxy network
29
+ - **50+ structured data tools** for Amazon, LinkedIn, Instagram, TikTok, YouTube, Reddit, and more — no parsing required
30
+
31
+ ---
4
32
 
5
33
  ## Install
6
34
 
@@ -8,184 +36,228 @@ Web search, scraping, browser automation, and 50+ structured data tools — powe
8
36
  openclaw plugins install @brightdata/brightdata-plugin
9
37
  ```
10
38
 
39
+ ---
40
+
11
41
  ## Configure
12
42
 
13
- Set your Bright Data API token (get one at [brightdata.com](https://brightdata.com)):
43
+ Get your API token at [brightdata.com](https://brightdata.com) → Account → API Token.
14
44
 
15
45
  ```bash
16
- # Option A: environment variable
17
- export BRIGHTDATA_API_TOKEN=your_token
46
+ # Option A — environment variable (recommended for local dev)
47
+ export BRIGHTDATA_API_TOKEN=your_token_here
18
48
 
19
- # Option B: OpenClaw config
20
- openclaw config set plugins.entries.brightdata.config.webSearch.apiKey your_token
49
+ # Option B — OpenClaw config (recommended for persistent setups)
50
+ openclaw config set plugins.entries.brightdata.config.webSearch.apiKey your_token_here
21
51
  ```
22
52
 
23
- The plugin auto-creates two proxy zones on first use: `mcp_unlocker` (Web Unlocker) and `mcp_browser` (Browser API). To use existing zones:
53
+ > The plugin automatically creates two proxy zones on first use: `mcp_unlocker` (Web Unlocker) and `mcp_browser` (Browser API). No manual zone setup required.
54
+
55
+ To use existing zones instead:
24
56
 
25
57
  ```bash
26
- export BRIGHTDATA_UNLOCKER_ZONE=my_unlocker_zone
27
- export BRIGHTDATA_BROWSER_ZONE=my_browser_zone
58
+ export BRIGHTDATA_UNLOCKER_ZONE=my_existing_zone
59
+ export BRIGHTDATA_BROWSER_ZONE=my_existing_browser_zone
28
60
  ```
29
61
 
30
- ## What's Included
62
+ ---
63
+
64
+ ## Tools Overview
31
65
 
32
66
  **66 tools** across five categories:
33
67
 
34
- | Category | Tools | Description |
35
- |----------|-------|-------------|
36
- | [Search](#search) | 2 | Web search via Google, Bing, Yandex with geo-targeting |
37
- | [Scrape](#scrape) | 1 | Page extraction as markdown, text, or HTML — handles bot protection |
38
- | [Batch](#batch) | 2 | Parallel search and scrape (up to 5 items) |
39
- | [Browser](#browser-automation) | 14 | Full browser automation via Playwright CDP |
40
- | [Web Data](#web-data) | 47 | Structured data from Amazon, LinkedIn, Instagram, TikTok, and more |
68
+ | Category | Count | What it does |
69
+ |----------|------:|-------------|
70
+ | [Search](#-search) | 2 | Real-time SERP results from Google, Bing, Yandex |
71
+ | [Scrape](#-scrape) | 1 | Full-page extraction with bot bypass |
72
+ | [Batch](#-batch-operations) | 2 | Parallel search and scrape up to 5 at a time |
73
+ | [Browser](#-browser-automation) | 14 | Full browser control via residential proxies |
74
+ | [Web Data](#-web-data--50-platforms) | 47 | Structured data from 47 platforms |
41
75
 
42
76
  ---
43
77
 
44
- ### Search
78
+ ## Search
45
79
 
46
- The plugin registers as an OpenClaw **web search provider** — it appears in provider selection automatically.
80
+ The plugin registers automatically as an OpenClaw **web search provider** — it appears alongside other providers in the search provider selection UI.
47
81
 
48
- | Tool | Description |
49
- |------|-------------|
50
- | `brightdata_search` | Search Google, Bing, or Yandex with pagination, geo-targeting, and result count control |
82
+ ### `brightdata_search`
51
83
 
52
- ```
53
- Parameters: query (required), engine ("google"|"bing"|"yandex"), count (1-10),
54
- cursor, geo_location (2-letter ISO), timeoutSeconds
55
- ```
84
+ Search Google, Bing, or Yandex and get structured results back.
56
85
 
57
- ### Scrape
86
+ | Parameter | Type | Description |
87
+ |-----------|------|-------------|
88
+ | `query` | `string` | **Required.** Search query |
89
+ | `engine` | `"google" \| "bing" \| "yandex"` | Search engine (default: `google`) |
90
+ | `count` | `number` | Results to return, 1–10 |
91
+ | `cursor` | `string` | Pagination cursor for next page |
92
+ | `geo_location` | `string` | 2-letter ISO country code (e.g. `"us"`, `"de"`) |
93
+ | `timeoutSeconds` | `number` | Request timeout override |
58
94
 
59
- | Tool | Description |
60
- |------|-------------|
61
- | `brightdata_scrape` | Fetch and extract a page through Bright Data Web Unlocker, including bot-protected pages |
95
+ ### `brightdata_search_batch`
62
96
 
63
- ```
64
- Parameters: url (required), extractMode ("markdown"|"text"|"html"), maxChars (≥100), timeoutSeconds
65
- ```
97
+ Run up to 5 search queries in parallel. Partial failures are returned inline without failing the whole batch.
66
98
 
67
- ### Batch
99
+ ---
68
100
 
69
- | Tool | Description |
70
- |------|-------------|
71
- | `brightdata_search_batch` | Run up to 5 search queries in parallel |
72
- | `brightdata_scrape_batch` | Scrape up to 5 URLs in parallel |
101
+ ## Scrape
102
+
103
+ ### `brightdata_scrape`
104
+
105
+ Fetch any page through Bright Data Web Unlocker. Works on bot-protected sites, JavaScript-rendered pages, and geo-restricted content.
73
106
 
74
- ### Browser Automation
107
+ | Parameter | Type | Description |
108
+ |-----------|------|-------------|
109
+ | `url` | `string` | **Required.** HTTP/HTTPS URL to scrape |
110
+ | `extractMode` | `"markdown" \| "text" \| "html"` | Output format (default: `markdown`) |
111
+ | `maxChars` | `number` | Maximum characters to return (min: 100) |
112
+ | `timeoutSeconds` | `number` | Request timeout override |
75
113
 
76
- Full browser control via Bright Data's residential proxy network. Sessions are scoped per user context and idle-timeout after 10 minutes.
114
+ ### `brightdata_scrape_batch`
115
+
116
+ Scrape up to 5 URLs in parallel with the same extraction options.
117
+
118
+ ---
119
+
120
+ ## Browser Automation
121
+
122
+ Full Chromium browser control routed through Bright Data's residential proxy network. Sessions are automatically scoped per user context and idle-timeout after 10 minutes.
77
123
 
78
124
  | Tool | Description |
79
125
  |------|-------------|
80
126
  | `brightdata_browser_navigate` | Navigate to a URL (optional country routing) |
81
- | `brightdata_browser_go_back` | Go back |
82
- | `brightdata_browser_go_forward` | Go forward |
83
- | `brightdata_browser_snapshot` | Capture ARIA snapshot with interactive element refs |
84
- | `brightdata_browser_click` | Click an element by ref |
85
- | `brightdata_browser_type` | Type into an element by ref (optional submit) |
127
+ | `brightdata_browser_snapshot` | Capture an ARIA snapshot with interactive element refs |
128
+ | `brightdata_browser_click` | Click an element by its snapshot ref |
129
+ | `brightdata_browser_type` | Type into a field by ref (optional Enter to submit) |
130
+ | `brightdata_browser_fill_form` | Fill multiple form fields in a single operation |
86
131
  | `brightdata_browser_screenshot` | Take a screenshot (viewport or full page) |
87
- | `brightdata_browser_get_html` | Get page HTML |
88
- | `brightdata_browser_get_text` | Get page text content |
89
- | `brightdata_browser_scroll` | Scroll to bottom |
132
+ | `brightdata_browser_get_html` | Get current page HTML |
133
+ | `brightdata_browser_get_text` | Get current page text content |
134
+ | `brightdata_browser_scroll` | Scroll to bottom of page |
90
135
  | `brightdata_browser_scroll_to` | Scroll to a specific element by ref |
91
- | `brightdata_browser_wait_for` | Wait for an element to be visible |
136
+ | `brightdata_browser_wait_for` | Wait for an element to become visible |
92
137
  | `brightdata_browser_network_requests` | List network requests since page load |
93
- | `brightdata_browser_fill_form` | Fill multiple form fields in one operation |
138
+ | `brightdata_browser_go_back` | Navigate back |
139
+ | `brightdata_browser_go_forward` | Navigate forward |
140
+
141
+ ---
94
142
 
95
- ### Web Data
143
+ ## Web Data — 50+ Platforms
96
144
 
97
- Structured data extraction from 47 platforms via Bright Data datasets. Each tool accepts a `url` or `keyword` input and returns structured JSON.
145
+ Structured data from real pages via Bright Data datasets. Each tool accepts a `url` or `keyword` and returns clean, typed JSON — no scraping, no parsing.
146
+
147
+ <details>
148
+ <summary><strong>E-commerce (10 tools)</strong></summary>
149
+
150
+ | Tool | Data |
151
+ |------|------|
152
+ | `brightdata_amazon_product` | Product details, pricing, specs |
153
+ | `brightdata_amazon_product_reviews` | Customer reviews and ratings |
154
+ | `brightdata_amazon_product_search` | Search results with rankings |
155
+ | `brightdata_walmart_product` | Product details and availability |
156
+ | `brightdata_walmart_seller` | Seller profile and metrics |
157
+ | `brightdata_ebay_product` | Listing details and bids |
158
+ | `brightdata_homedepot_products` | Product catalog and pricing |
159
+ | `brightdata_zara_products` | Fashion catalog data |
160
+ | `brightdata_etsy_products` | Handmade and vintage listings |
161
+ | `brightdata_bestbuy_products` | Electronics catalog and deals |
162
+
163
+ </details>
98
164
 
99
165
  <details>
100
- <summary>All 47 dataset tools</summary>
101
-
102
- **E-commerce**
103
- | Tool | Platform |
104
- |------|----------|
105
- | `brightdata_amazon_product` | Amazon product details |
106
- | `brightdata_amazon_product_reviews` | Amazon reviews |
107
- | `brightdata_amazon_product_search` | Amazon search results |
108
- | `brightdata_walmart_product` | Walmart product details |
109
- | `brightdata_walmart_seller` | Walmart seller info |
110
- | `brightdata_ebay_product` | eBay product details |
111
- | `brightdata_homedepot_products` | Home Depot products |
112
- | `brightdata_zara_products` | Zara products |
113
- | `brightdata_etsy_products` | Etsy products |
114
- | `brightdata_bestbuy_products` | Best Buy products |
115
-
116
- **Professional Networks**
117
- | Tool | Platform |
118
- |------|----------|
119
- | `brightdata_linkedin_person_profile` | LinkedIn person profile |
120
- | `brightdata_linkedin_company_profile` | LinkedIn company profile |
121
- | `brightdata_linkedin_job_listings` | LinkedIn jobs |
122
- | `brightdata_linkedin_posts` | LinkedIn posts |
123
- | `brightdata_linkedin_people_search` | LinkedIn people search |
124
- | `brightdata_crunchbase_company` | Crunchbase company data |
125
- | `brightdata_zoominfo_company_profile` | ZoomInfo company profile |
126
-
127
- **Social Media — Instagram**
128
- | Tool | Platform |
129
- |------|----------|
130
- | `brightdata_instagram_profiles` | Instagram profiles |
131
- | `brightdata_instagram_posts` | Instagram posts |
132
- | `brightdata_instagram_reels` | Instagram reels |
133
- | `brightdata_instagram_comments` | Instagram comments |
134
-
135
- **Social Media — Facebook**
136
- | Tool | Platform |
137
- |------|----------|
138
- | `brightdata_facebook_posts` | Facebook posts |
139
- | `brightdata_facebook_marketplace_listings` | Facebook Marketplace |
140
- | `brightdata_facebook_company_reviews` | Facebook company reviews |
141
- | `brightdata_facebook_events` | Facebook events |
142
-
143
- **Social Media — TikTok**
144
- | Tool | Platform |
145
- |------|----------|
146
- | `brightdata_tiktok_profiles` | TikTok profiles |
147
- | `brightdata_tiktok_posts` | TikTok posts |
148
- | `brightdata_tiktok_shop` | TikTok Shop |
149
- | `brightdata_tiktok_comments` | TikTok comments |
150
-
151
- **Social Media — X (Twitter)**
152
- | Tool | Platform |
153
- |------|----------|
154
- | `brightdata_x_posts` | X posts |
155
- | `brightdata_x_profile_posts` | X profile posts |
156
-
157
- **Social Media — YouTube & Reddit**
158
- | Tool | Platform |
159
- |------|----------|
160
- | `brightdata_youtube_profiles` | YouTube profiles |
161
- | `brightdata_youtube_videos` | YouTube videos |
162
- | `brightdata_youtube_comments` | YouTube comments |
163
- | `brightdata_reddit_posts` | Reddit posts |
164
-
165
- **Maps, Shopping & Apps**
166
- | Tool | Platform |
167
- |------|----------|
168
- | `brightdata_google_maps_reviews` | Google Maps reviews |
169
- | `brightdata_google_shopping` | Google Shopping |
170
- | `brightdata_google_play_store` | Google Play Store |
171
- | `brightdata_apple_app_store` | Apple App Store |
172
-
173
- **Finance, News & Code**
174
- | Tool | Platform |
175
- |------|----------|
176
- | `brightdata_reuter_news` | Reuters news |
177
- | `brightdata_yahoo_finance_business` | Yahoo Finance |
178
- | `brightdata_github_repository_file` | GitHub repository files |
179
-
180
- **Real Estate & Travel**
181
- | Tool | Platform |
182
- |------|----------|
183
- | `brightdata_zillow_properties_listing` | Zillow listings |
184
- | `brightdata_booking_hotel_listings` | Booking.com hotels |
185
-
186
- **AI Insights**
187
- | Tool | Platform |
188
- |------|----------|
166
+ <summary><strong>Professional Networks (7 tools)</strong></summary>
167
+
168
+ | Tool | Data |
169
+ |------|------|
170
+ | `brightdata_linkedin_person_profile` | Full person profile |
171
+ | `brightdata_linkedin_company_profile` | Company overview and stats |
172
+ | `brightdata_linkedin_job_listings` | Open positions with details |
173
+ | `brightdata_linkedin_posts` | Post content and engagement |
174
+ | `brightdata_linkedin_people_search` | People search results |
175
+ | `brightdata_crunchbase_company` | Funding, investors, founders |
176
+ | `brightdata_zoominfo_company_profile` | Company intelligence data |
177
+
178
+ </details>
179
+
180
+ <details>
181
+ <summary><strong>Social Media (17 tools)</strong></summary>
182
+
183
+ **Instagram**
184
+ | Tool | Data |
185
+ |------|------|
186
+ | `brightdata_instagram_profiles` | Profile stats and bio |
187
+ | `brightdata_instagram_posts` | Post content and engagement |
188
+ | `brightdata_instagram_reels` | Reel metadata and views |
189
+ | `brightdata_instagram_comments` | Comment threads |
190
+
191
+ **Facebook**
192
+ | Tool | Data |
193
+ |------|------|
194
+ | `brightdata_facebook_posts` | Post content and reactions |
195
+ | `brightdata_facebook_marketplace_listings` | Marketplace items |
196
+ | `brightdata_facebook_company_reviews` | Page reviews and ratings |
197
+ | `brightdata_facebook_events` | Event details and attendance |
198
+
199
+ **TikTok**
200
+ | Tool | Data |
201
+ |------|------|
202
+ | `brightdata_tiktok_profiles` | Creator profile and stats |
203
+ | `brightdata_tiktok_posts` | Video content and metrics |
204
+ | `brightdata_tiktok_shop` | TikTok Shop product data |
205
+ | `brightdata_tiktok_comments` | Comment threads |
206
+
207
+ **X (Twitter)**
208
+ | Tool | Data |
209
+ |------|------|
210
+ | `brightdata_x_posts` | Post content and metrics |
211
+ | `brightdata_x_profile_posts` | Profile post history |
212
+
213
+ **YouTube & Reddit**
214
+ | Tool | Data |
215
+ |------|------|
216
+ | `brightdata_youtube_profiles` | Channel stats and info |
217
+ | `brightdata_youtube_videos` | Video details and metrics |
218
+ | `brightdata_youtube_comments` | Comment threads |
219
+ | `brightdata_reddit_posts` | Post content and scores |
220
+
221
+ </details>
222
+
223
+ <details>
224
+ <summary><strong>Maps, Shopping & Apps (4 tools)</strong></summary>
225
+
226
+ | Tool | Data |
227
+ |------|------|
228
+ | `brightdata_google_maps_reviews` | Location reviews and ratings |
229
+ | `brightdata_google_shopping` | Shopping results and prices |
230
+ | `brightdata_google_play_store` | App details and reviews |
231
+ | `brightdata_apple_app_store` | App details and reviews |
232
+
233
+ </details>
234
+
235
+ <details>
236
+ <summary><strong>Finance, News & Code (3 tools)</strong></summary>
237
+
238
+ | Tool | Data |
239
+ |------|------|
240
+ | `brightdata_reuter_news` | Reuters news articles |
241
+ | `brightdata_yahoo_finance_business` | Company financials and news |
242
+ | `brightdata_github_repository_file` | Repository file contents |
243
+
244
+ </details>
245
+
246
+ <details>
247
+ <summary><strong>Real Estate & Travel (2 tools)</strong></summary>
248
+
249
+ | Tool | Data |
250
+ |------|------|
251
+ | `brightdata_zillow_properties_listing` | Property listings and estimates |
252
+ | `brightdata_booking_hotel_listings` | Hotel listings and pricing |
253
+
254
+ </details>
255
+
256
+ <details>
257
+ <summary><strong>AI Insights (3 tools)</strong></summary>
258
+
259
+ | Tool | Data |
260
+ |------|------|
189
261
  | `brightdata_chatgpt_ai_insights` | ChatGPT responses |
190
262
  | `brightdata_grok_ai_insights` | Grok responses |
191
263
  | `brightdata_perplexity_ai_insights` | Perplexity responses |
@@ -196,18 +268,20 @@ Structured data extraction from 47 platforms via Bright Data datasets. Each tool
196
268
 
197
269
  ## Configuration Reference
198
270
 
199
- | Setting | Env Var | Config Path | Default |
200
- |---------|---------|-------------|---------|
201
- | API Token | `BRIGHTDATA_API_TOKEN` | `...webSearch.apiKey` | *required* |
271
+ All settings can be provided via environment variable or OpenClaw config. Environment variables take priority.
272
+
273
+ | Setting | Environment Variable | Config Path | Default |
274
+ |---------|---------------------|-------------|---------|
275
+ | API Token | `BRIGHTDATA_API_TOKEN` | `...webSearch.apiKey` | **required** |
202
276
  | Base URL | `BRIGHTDATA_BASE_URL` | `...webSearch.baseUrl` | `https://api.brightdata.com` |
203
277
  | Unlocker Zone | `BRIGHTDATA_UNLOCKER_ZONE` | `...webSearch.unlockerZone` | `mcp_unlocker` |
204
278
  | Browser Zone | `BRIGHTDATA_BROWSER_ZONE` | `...webSearch.browserZone` | `mcp_browser` |
205
- | Timeout | — | `...webSearch.timeoutSeconds` | `30` (search) / `60` (scrape) |
206
- | Polling Timeout | — | `...webSearch.pollingTimeoutSeconds` | `600` |
279
+ | Request Timeout | — | `...webSearch.timeoutSeconds` | `30s` search / `60s` scrape |
280
+ | Polling Timeout | — | `...webSearch.pollingTimeoutSeconds` | `600s` |
207
281
 
208
- Config paths are prefixed with `plugins.entries.brightdata.config`.
282
+ > Config paths are prefixed with `plugins.entries.brightdata.config`.
209
283
 
210
- Environment variables take priority over config file values.
284
+ ---
211
285
 
212
286
  ## Plugin Management
213
287
 
@@ -215,20 +289,32 @@ Environment variables take priority over config file values.
215
289
  # Install
216
290
  openclaw plugins install @brightdata/brightdata-plugin
217
291
 
218
- # Verify
292
+ # Verify installation and loaded tools
219
293
  openclaw plugins inspect brightdata
220
294
 
221
- # Disable / re-enable
295
+ # Update to latest version
296
+ openclaw plugins update brightdata
297
+
298
+ # Temporarily disable
222
299
  openclaw plugins disable brightdata
223
- openclaw plugins enable brightdata
224
300
 
225
- # Update
226
- openclaw plugins update brightdata
301
+ # Re-enable
302
+ openclaw plugins enable brightdata
227
303
 
228
304
  # Uninstall
229
305
  openclaw plugins uninstall brightdata
230
306
  ```
231
307
 
308
+ ---
309
+
310
+ ## Requirements
311
+
312
+ - OpenClaw installed and running
313
+ - A [Bright Data account](https://brightdata.com) with API token
314
+ - Node.js 22+ (managed by OpenClaw)
315
+
316
+ ---
317
+
232
318
  ## License
233
319
 
234
- MIT
320
+ MIT © [Bright Data](https://brightdata.com)
package/assets/brightdata-logo.png ADDED
Binary file
package/assets/openclaw-logo.png ADDED
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@brightdata/brightdata-plugin",
3
- "version": "1.0.0",
3
+ "version": "1.0.1",
4
4
  "description": "OpenClaw Bright Data plugin",
5
5
  "type": "module",
6
6
  "main": "index.ts",