scrapeloop-mcp 0.38.0 → 0.39.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scrapeloop-mcp",
-  "version": "0.38.0",
+  "version": "0.39.0",
   "description": "Scrapeloop MCP server  -  set up and run Scrapeloop end-to-end (integrations, scraping, lead database, verification, cleaners, enrichment, strategies, Instantly campaigns) from any MCP client (Claude Desktop, Cursor, ChatGPT, …).",
   "type": "module",
   "scripts": {

package/src/index.js

@@ -601,14 +601,14 @@ const TOOLS = {
     run: (a) => api('GET', `/plans/${enc(a.plan_id)}/budget`),
   },
   set_scrape_plan_budget: {
-    def: { description: 'Save reviewed credit limits and Table ownership for a paused complete-search plan with a ready-email goal. All four credit answers are required. A number is a limit; explicit null means no plan limit for that one (then only the workspace balance stops the plan), and omission is rejected. Zero blocks spending. No-cap checks still require allowed tools, bounded jobs and available workspace credits. Never infer no cap without approval. Include Raw, Email ready and every check Table. One Table belongs to one budget. Preserve expected_revision from the current budget; omit it only for the first save. A lost reply requires readback before retry. This starts no work and does not enable checks. website_language permits the free bounded Website languages preset with a saved attempt. owner_website explicitly permits the free bounded system Owner name finder; it still needs a saved job and attempt. review_history permits only the system Google review stats preset with source explicitly outscraper and a separate review_budget in USD, default zero. Select an existing connection from get_scrape_plan_budget.review_connections. Omitted review_budget preserves saved settings. Each row reserves 0.147 USD; recorded units use the published ceiling, not a confirmed account invoice. Unknown requests stay held and never retry. Google-first, custom recipes and other paid vendors remain blocked. Use review_complete_scrape_plan to check readiness before requesting exact launch approval.', inputSchema: obj({
+    def: { description: 'Save reviewed credit limits and Table ownership for a paused complete-search plan with a ready-email goal. All four credit answers are required. A number is a limit; explicit null means no plan limit for that one (then only the workspace balance stops the plan), and omission is rejected. Zero blocks spending. No-cap checks still require allowed tools, bounded jobs and available workspace credits. Never infer no cap without approval. Include Raw, Email ready and every check Table. One Table belongs to one budget. Preserve expected_revision from the current budget; omit it only for the first save. A lost reply requires readback before retry. This starts no work and does not enable checks. website_language permits the free bounded Website languages preset with a saved attempt. owner_website explicitly permits the free bounded system Owner name finder; it still needs a saved job and attempt. review_history permits only the system Google review stats preset with source explicitly outscraper and a separate review_budget in USD, default zero. Select an existing connection from get_scrape_plan_budget.review_connections. Omitted review_budget preserves saved settings. Each row reserves 0.147 USD; recorded units use the published ceiling, not a confirmed account invoice. Unknown requests stay held and never retry. business_email_finder permits a Business email finder column that was made with confirmed_only on (create_table_native_column options). Such a column keeps only an address the email check called real, holds 3 verify credits per row and charges 1 per check that got an answer. A Business email finder column without confirmed_only stays blocked, because it can keep an address nobody confirmed. Google-first, custom recipes and other paid vendors remain blocked. Use review_complete_scrape_plan to check readiness before requesting exact launch approval.', inputSchema: obj({
       plan_id: { ...S, format: 'uuid' }, expected_revision: { ...S, format: 'uuid' },
       limits: obj({
         monthly_lead_credits: { type: ['integer', 'null'], minimum: 0, maximum: 1000000000 },
         lifetime_lead_credits: { type: ['integer', 'null'], minimum: 0, maximum: 1000000000 },
         monthly_verify_credits: { type: ['integer', 'null'], minimum: 0, maximum: 1000000000 },
         lifetime_verify_credits: { type: ['integer', 'null'], minimum: 0, maximum: 1000000000 },
-        allowed_checks: { type: 'array', items: { ...S, enum: ['verify_email', 'verify_catchall', 'verify_phone', 'owner_website', 'review_history', 'website_language'] }, maxItems: 6 },
+        allowed_checks: { type: 'array', items: { ...S, enum: ['verify_email', 'verify_catchall', 'verify_phone', 'owner_website', 'review_history', 'website_language', 'business_email_finder'] }, maxItems: 7 },
         review_budget: obj({ monthly_usd: { type: 'number', minimum: 0, maximum: 1000000, multipleOf: 0.001 }, lifetime_usd: { type: 'number', minimum: 0, maximum: 1000000, multipleOf: 0.001 }, credential_id: { type: ['string', 'null'], format: 'uuid' } }, ['monthly_usd', 'lifetime_usd', 'credential_id']),
       }, ['monthly_lead_credits', 'lifetime_lead_credits', 'monthly_verify_credits', 'lifetime_verify_credits', 'allowed_checks']),
       list_ids: { type: 'array', items: { ...S, format: 'uuid' }, minItems: 1, maxItems: 20 },
@@ -1740,7 +1740,7 @@ const TOOLS = {
   create_table_native_column: {
     def: {
       description:
-        `Attach one native Scrapeloop tool as a real Smart Table column: ${NATIVE_TABLE_TOOLS.join(', ')}. These are Scrapeloop's first-party tools, several of which superseded legacy presets, so they are addressed by tool key here rather than by preset slug through create_table_preset_column (which refuses a superseded slug for a new column). Each bills MANAGED VERIFICATION CREDITS per row, never dollars and never the customer's own vendor keys: verify_email reserves 1 credit per selected row, verify_phone reserves 5, each email finder reserves 3, linkedin_finder reserves 5, linkedin_activity reserves 3, and google_profile reserves 1. Settlement charges only fresh vendor checks and returns the unused reservation; missing inputs, cached rows, and skipped rows are free. linkedin_finder charges its 5 credits only for a row where it found the owner's LinkedIn profile; a row with no owner found is free. linkedin_activity reads each row's personal LinkedIn profile and newest post and labels it Active, Maybe or Weak (cell text), so LinkedIn invites go to people likely to answer; it charges 3 credits only for a profile it read, and rows with no personal profile, cached rows and profiles that no longer exist are free. google_profile finds a business on Google Maps for a lead that did not come from Google Maps (a Yelp lead, an imported list): it searches the business name in its town, accepts a result only when the phone, the website or the exact name in the same town matches, and saves place_id, gbp_url, rating and reviews_count on the cell (status found, not_found or unclear) and the place ID on a lead that had none; it charges 1 credit only for a profile found, and a business not on Google, more than one matching profile and cached rows are free. Filter on col.<key>.reviews_count to keep small businesses; a business with no reviews yet is saved as 0, and an unknown count has no number and never passes a numeric rule. Creating a column is always free. auto_run defaults false and queues nothing; when auto_run would backfill existing rows the API returns a 409 estimate whose cost.total_credits is the real charge, and you confirm it with confirm_auto_run plus confirmed_credits (CREDITS, not dollars - estimated_cost_usd is our vendor cost and is far smaller than what the workspace is billed). An auto_run that exceeds the workspace verify-credit balance returns 402 with {credit_kind, required, available} before anything is created. The Table master auto-run switch and workspace budget caps stay fail-closed. options are per tool and only the documented names are accepted - call get_table_native_tools for the current list. Owner email finder needs an owner first name plus the company website; point options.owner_name_column at an Owner name finder column to source the name, which auto_run requires. Business email finder is a SEPARATE column - create it with its own call and set options.owner_email_column to run it behind an owner finder. The Catch-all verifier is NOT creatable here: it needs a Verify email column that has already RUN and returned Catch All or Unknown, so use verify_catchall per address, or add the column from the Table's add-column menu.`,
+        `Attach one native Scrapeloop tool as a real Smart Table column: ${NATIVE_TABLE_TOOLS.join(', ')}. These are Scrapeloop's first-party tools, several of which superseded legacy presets, so they are addressed by tool key here rather than by preset slug through create_table_preset_column (which refuses a superseded slug for a new column). Each bills MANAGED VERIFICATION CREDITS per row, never dollars and never the customer's own vendor keys: verify_email reserves 1 credit per selected row, verify_phone reserves 5, each email finder reserves 3, linkedin_finder reserves 5, linkedin_activity reserves 3, and google_profile reserves 1. Settlement charges only fresh vendor checks and returns the unused reservation; missing inputs, cached rows, and skipped rows are free. linkedin_finder charges its 5 credits only for a row where it found the owner's LinkedIn profile; a row with no owner found is free. linkedin_activity reads each row's personal LinkedIn profile and newest post and labels it Active, Maybe or Weak (cell text), so LinkedIn invites go to people likely to answer; it charges 3 credits only for a profile it read, and rows with no personal profile, cached rows and profiles that no longer exist are free. google_profile finds a business on Google Maps for a lead that did not come from Google Maps (a Yelp lead, an imported list): it searches the business name in its town, accepts a result only when the phone, the website or the exact name in the same town matches, and saves place_id, gbp_url, rating and reviews_count on the cell (status found, not_found or unclear) and the place ID on a lead that had none; it charges 1 credit only for a profile found, and a business not on Google, more than one matching profile and cached rows are free. Filter on col.<key>.reviews_count to keep small businesses; a business with no reviews yet is saved as 0, and an unknown count has no number and never passes a numeric rule. Creating a column is always free. auto_run defaults false and queues nothing; when auto_run would backfill existing rows the API returns a 409 estimate whose cost.total_credits is the real charge, and you confirm it with confirm_auto_run plus confirmed_credits (CREDITS, not dollars - estimated_cost_usd is our vendor cost and is far smaller than what the workspace is billed). An auto_run that exceeds the workspace verify-credit balance returns 402 with {credit_kind, required, available} before anything is created. The Table master auto-run switch and workspace budget caps stay fail-closed. options are per tool and only the documented names are accepted - call get_table_native_tools for the current list. Owner email finder needs an owner first name plus the company website; point options.owner_name_column at an Owner name finder column to source the name, which auto_run requires. Business email finder is a SEPARATE column - create it with its own call and set options.owner_email_column to run it behind an owner finder. Business email finder also has careful options for a Table that runs by itself. confirmed_only keeps an address only when the email check says it is real (verified, or an exact Role answer), tries the next name only when this mailbox does not exist, never keeps a catch all, unknown or full mailbox answer, and saves each answer the way a Verify email column does, so that column gets it for free afterwards. require_mail_server skips a website with no mail server at no charge. max_checks keeps a row to 1, 2 or 3 names. skip_shared_website skips, at no charge, a website that is a shared page or that another lead of the workspace also has. write_to_lead_email writes a confirmed address on a lead that has no email yet, and never an address another lead already has. run_if runs a row only when a condition is true. A Table under a search plan's limits only takes this finder with confirmed_only on. new_rows_only (with auto_run) runs only rows added from now on: rows already in the Table are not run, nothing is queued or held, and no confirmation is needed. The Catch-all verifier is NOT creatable here: it needs a Verify email column that has already RUN and returned Catch All or Unknown, so use verify_catchall per address, or add the column from the Table's add-column menu.`,
       inputSchema: obj(
         {
           table_id: S,
@@ -1754,10 +1754,15 @@ const TOOLS = {
             description:
               'The 409 estimate\'s cost.total_credits, echoed back exactly. Verification credits, not dollars.',
           },
+          new_rows_only: {
+            ...B,
+            description:
+              'With auto_run true: run only rows that are added from now on. Rows already in the Table are not run now, so nothing is queued or held and there is nothing to confirm. Leave confirm_auto_run and confirmed_credits out. Use this on a Table that already has many rows, such as the first Table of a search plan. Good to know: a column that reads another column, in an input or in run_if, still follows that column. When that other column finishes again for an old row, this column runs for that row once.',
+          },
           options: {
             ...O,
             description:
-              'Tool-specific options. verify_phone: phone_col, default_region. email_finder: owner_name_column. business_email_finder: owner_email_column. linkedin_finder: business_col. linkedin_activity: linkedin_col, business_col. google_profile: business_col. An undocumented name is rejected, never ignored.',
+              'Tool-specific options. verify_phone: phone_col, default_region. email_finder: owner_name_column. business_email_finder: owner_email_column, confirmed_only, require_mail_server, max_checks, skip_shared_website, shared_website_max_leads, write_to_lead_email, run_if. linkedin_finder: business_col. linkedin_activity: linkedin_col, business_col. google_profile: business_col. An undocumented name is rejected, never ignored.',
           },
           operation_id: {
             ...S,
@@ -1794,6 +1799,7 @@ const TOOLS = {
         ...(a.confirmed_credits !== undefined
           ? { confirmed_credits: a.confirmed_credits }
           : {}),
+        ...(a.new_rows_only !== undefined ? { new_rows_only: a.new_rows_only } : {}),
         ...(a.options !== undefined ? { options: a.options } : {}),
       };
       return api('POST', `/tables/${enc(a.table_id)}/native-columns`, requestBody, {
@@ -1863,7 +1869,7 @@ const TOOLS = {
   import_leads: {
     def: {
       description:
-        'Bring your own leads: atomically import externally-sourced records into a campaign-bindable Scrapeloop Table without scraping or lead credits. A generic record needs at least one of: email, linkedin_url, external_id (your own id for the record) or an absolute source_url (the page the record came from), so Scrapeloop can tell which lead it is. Email may be null. Records from a public source list cannot be imported here, only previewed. The rules for such records (sourcing_contract) switch on when you pass sourcing_contract:true, and also by themselves when one record carries source, source_state and niche together. A real import is then refused with 409 and nothing is imported: no lead, no Table. The 409 has an error sentence written for the user (it says what was seen, such as how many records carry all three, and what to change: show it as it is written), destination "agency_local_businesses" and dry_run_available:true. Use preview_import to see what such records would do. That preview has a warning that says the same. A reason of a skipped record there ends with: fixing this will not get it imported. Follow the warning, do not try to make such records pass. In that preview each record needs a controlled source, source_state and niche and a valid phone or email, a local business with only a phone gets an id made from its source and its phone, and source and niche tags are worked out. If these are the user\'s own leads, say what happened and ask before you change them: without source_state the same records import as ordinary leads. Matching is workspace-scoped and deterministic: normalized email first, then local-business phone, canonical LinkedIn, then external_id or source_url. The legacy dedupe_by="external_id" mode matches by external_id or source_url first. A later record with a real email merges into the earlier lead that only had a LinkedIn link, an external_id or a source_url, instead of duplicating it. No placeholder emails are created. verification_status and custom_fields are preserved, and custom_fields flow to Instantly as custom variables. Target list_id or pass list_name to create-or-get a Table. Batch up to 1000 records. One operation_id is reused across HTTP retries, so an uncertain response can be recovered without duplicating leads. You may pass a prior operation_id; reusing it with different inputs returns 409. Each record: {source?, source_state?, email?, linkedin_url?, source_url?, external_id?, first_name, last_name, business_name, phone, website, address, city, state_region, country, niche, rating, reviews_count, gbp_link?, facebook?, instagram?, verification_status ("verified"|"catchall_verified"|"unverified"|"risky"|"invalid"), strategy_tags:[], custom_fields:{}}. Returns {operation_id, list_id, inserted, updated, deduped, invalid, invalid_reasons}. Each entry of invalid_reasons has a reason written for the user: show it as it is written.',
+        'Bring your own leads: atomically import externally-sourced records into a campaign-bindable Scrapeloop Table without scraping or lead credits. A generic record needs at least one of: email, linkedin_url, external_id (your own id for the record) or an absolute source_url (the page the record came from), so Scrapeloop can tell which lead it is. Email may be null. Records from a public source list cannot be imported here, only previewed. The rules for such records (sourcing_contract) switch on when you pass sourcing_contract:true, and also by themselves when one record carries source, source_state and niche together. A real import is then refused with 409 and nothing is imported: no lead, no Table. The 409 has an error sentence written for the user (it says what was seen, such as how many records carry all three, and what to change: show it as it is written), destination "agency_local_businesses" and dry_run_available:true. Use preview_import to see what such records would do. That preview has a warning that says the same. A reason of a skipped record there ends with: fixing this will not get it imported. Follow the warning, do not try to make such records pass. In that preview each record needs a controlled source, source_state and niche and a valid phone or email, a local business with only a phone gets an id made from its source and its phone, and source and niche tags are worked out. If these are the user\'s own leads, say what happened and ask before you change them: without source_state the same records import as ordinary leads. Matching is workspace-scoped and deterministic: normalized email first, then local-business phone, canonical LinkedIn, then external_id or source_url. The legacy dedupe_by="external_id" mode matches by external_id or source_url first. A later record with a real email merges into the earlier lead that only had a LinkedIn link, an external_id or a source_url, instead of duplicating it. No placeholder emails are created. verification_status and custom_fields are preserved, and custom_fields flow to Instantly as custom variables. Target list_id or pass list_name to create-or-get a Table. Batch up to 1000 records. One operation_id is reused across HTTP retries, so an uncertain response can be recovered without duplicating leads. You may pass a prior operation_id; reusing it with different inputs returns 409. Each record: {source?, source_state?, email?, linkedin_url?, source_url?, external_id?, first_name, last_name, business_name, phone, website, address, city, state_region, country, niche, rating, reviews_count, gbp_link?, facebook?, instagram?, verification_status ("verified"|"catchall_verified"|"unverified"|"risky"|"invalid"), strategy_tags:[], custom_fields:{}}. Returns {operation_id, list_id, inserted, updated, deduped, invalid, invalid_reasons}. Each entry of invalid_reasons has a reason written for the user: show it as it is written. A lead keeps the marks no_resale, workspace_only and legal_basis gdpr_li (extra fields that say the lead needs more care) once it has them: a later import cannot turn them off. When the answer has marks_kept_note, show it as it is written.',
       inputSchema: obj(
         {
           list_id: S,
@@ -2061,7 +2067,7 @@ const TOOLS = {
   import_leads_csv: {
     def: {
       description:
-        "Bring your own leads from CSV/TSV into a campaign-bindable Table. Each row needs at least one of: email, LinkedIn URL, external ID or an absolute source URL, so Scrapeloop can tell which lead it is. A row with no email needs no placeholder email. The parser auto-detects the delimiter, maps common headers (including linkedin_url and source_url), preserves unmapped columns as custom_fields, and normalizes verifier verdicts. A CSV from a public source list cannot be imported here, only previewed. A file with source, source_state and niche columns switches on the rules for such lists (sourcing_contract) by itself, and so does sourcing_contract:true. A real import of it is refused with 409 and nothing is imported: no lead, no Table. The 409 has an error sentence written for the user (it says what was seen, such as the names of the three columns, and what to change: show it as it is written), destination \"agency_local_businesses\" and dry_run_available:true. With dry_run:true the same file is previewed (the answer then has a warning that says the real import will be refused, and what to change): every loaded row must also have a valid phone or email, source IDs are namespaced, a local business with only a phone gets an id made from its source and its phone, and source/niche tags are worked out. If this is the user's own file, say what happened and ask before you change it: without the source_state column the same rows import as ordinary leads. With a name of its own, such as Lead state, that column is kept as an extra field. Matching is workspace-scoped: email first, then local-business phone, LinkedIn, then external ID or source URL. Set dry_run:true first to preview writes and column mapping. Mutating CSV imports are atomic and reuse one operation_id across HTTP retries. You may pass a prior operation_id; reusing it with changed CSV inputs returns 409. No scraping, lead credits, or re-verification. Target list_id or pass list_name. Up to 1000 rows per call. Skipped rows come back in invalid_reasons. Each entry has a reason written for the user: show it as it is written.",
+        "Bring your own leads from CSV/TSV into a campaign-bindable Table. Each row needs at least one of: email, LinkedIn URL, external ID or an absolute source URL, so Scrapeloop can tell which lead it is. A row with no email needs no placeholder email. The parser auto-detects the delimiter, maps common headers (including linkedin_url and source_url), preserves unmapped columns as custom_fields, and normalizes verifier verdicts. A CSV from a public source list cannot be imported here, only previewed. A file with source, source_state and niche columns switches on the rules for such lists (sourcing_contract) by itself, and so does sourcing_contract:true. A real import of it is refused with 409 and nothing is imported: no lead, no Table. The 409 has an error sentence written for the user (it says what was seen, such as the names of the three columns, and what to change: show it as it is written), destination \"agency_local_businesses\" and dry_run_available:true. With dry_run:true the same file is previewed (the answer then has a warning that says the real import will be refused, and what to change): every loaded row must also have a valid phone or email, source IDs are namespaced, a local business with only a phone gets an id made from its source and its phone, and source/niche tags are worked out. If this is the user's own file, say what happened and ask before you change it: without the source_state column the same rows import as ordinary leads. With a name of its own, such as Lead state, that column is kept as an extra field. Matching is workspace-scoped: email first, then local-business phone, LinkedIn, then external ID or source URL. Set dry_run:true first to preview writes and column mapping. Mutating CSV imports are atomic and reuse one operation_id across HTTP retries. You may pass a prior operation_id; reusing it with changed CSV inputs returns 409. No scraping, lead credits, or re-verification. Target list_id or pass list_name. Up to 1000 rows per call. Skipped rows come back in invalid_reasons. Each entry has a reason written for the user: show it as it is written. A lead keeps the marks no_resale, workspace_only and legal_basis gdpr_li (extra fields that say the lead needs more care) once it has them: a later import cannot turn them off. When the answer has marks_kept_note, show it as it is written.",
       inputSchema: obj(
         {
           csv: S,
@@ -2117,7 +2123,7 @@ const TOOLS = {
   preview_import: {
     def: {
       description:
-        'Dry-run for import_leads: report exactly what an import WOULD do ({would_insert, would_update, deduped, invalid, invalid_reasons, list_exists}) WITHOUT writing anything (no leads, no list, no tags created). Same inputs as import_leads. Call this first for a bring-your-own batch, show the user the plan (e.g. "adds 340 new, updates 12, skips 3 bad rows"), then confirm before import_leads. One kind of batch can only be previewed: records under the rules for public source lists (sourcing_contract:true, or a record with source, source_state and niche together). The preview works for them, and a real import is refused with 409. The preview then has a warning that says so and what to change: show it as it is written, and follow it. Fixing the skipped records of such a batch does not get them imported. Each entry of invalid_reasons has a reason written for the user: show it as it is written.',
+        'Dry-run for import_leads: report exactly what an import WOULD do ({would_insert, would_update, deduped, invalid, invalid_reasons, list_exists}) WITHOUT writing anything (no leads, no list, no tags created). Same inputs as import_leads. Call this first for a bring-your-own batch, show the user the plan (e.g. "adds 340 new, updates 12, skips 3 bad rows"), then confirm before import_leads. One kind of batch can only be previewed: records under the rules for public source lists (sourcing_contract:true, or a record with source, source_state and niche together). The preview works for them, and a real import is refused with 409. The preview then has a warning that says so and what to change: show it as it is written, and follow it. Fixing the skipped records of such a batch does not get them imported. Each entry of invalid_reasons has a reason written for the user: show it as it is written. A lead keeps the marks no_resale, workspace_only and legal_basis gdpr_li (extra fields that say the lead needs more care) once it has them: a later import cannot turn them off. When the answer has marks_kept_note, show it as it is written.',
       inputSchema: obj(
         {
           list_id: S,