mcp-scraper 0.21.0 → 0.21.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -2
- package/dist/bin/api-server.cjs +368 -224
- package/dist/bin/api-server.cjs.map +1 -1
- package/dist/bin/api-server.js +2 -2
- package/dist/bin/mcp-scraper-cli.cjs +1 -1
- package/dist/bin/mcp-scraper-cli.cjs.map +1 -1
- package/dist/bin/mcp-scraper-cli.js +1 -1
- package/dist/bin/mcp-scraper-install.cjs +2 -2
- package/dist/bin/mcp-scraper-install.cjs.map +1 -1
- package/dist/bin/mcp-scraper-install.js +2 -2
- package/dist/bin/mcp-stdio-server.cjs +12 -8
- package/dist/bin/mcp-stdio-server.cjs.map +1 -1
- package/dist/bin/mcp-stdio-server.js +4 -4
- package/dist/{chunk-HXGZMMW5.js → chunk-2FF4GBQ4.js} +1 -5
- package/dist/{chunk-HXGZMMW5.js.map → chunk-2FF4GBQ4.js.map} +1 -1
- package/dist/chunk-NBF4F4UT.js +7 -0
- package/dist/chunk-NBF4F4UT.js.map +1 -0
- package/dist/{chunk-TUSLATZY.js → chunk-X3NIXHHG.js} +2 -2
- package/dist/{chunk-Q44DN6T2.js → chunk-YNHPZU6B.js} +2 -2
- package/dist/{chunk-Q44DN6T2.js.map → chunk-YNHPZU6B.js.map} +1 -1
- package/dist/{chunk-XHJJQRNQ.js → chunk-ZNNYPE2U.js} +13 -9
- package/dist/{chunk-XHJJQRNQ.js.map → chunk-ZNNYPE2U.js.map} +1 -1
- package/dist/{server-PH3VJ74X.js → server-CHTX6YQA.js} +336 -207
- package/dist/server-CHTX6YQA.js.map +1 -0
- package/dist/{site-extract-repository-YNPXEZCB.js → site-extract-repository-5V5EEQ72.js} +3 -3
- package/dist/{worker-45WEWCMQ.js → worker-GE5JMHE6.js} +2 -2
- package/docs/mcp-tool-manifest.generated.json +28 -10
- package/docs/specs/google-drive-bulk-access-and-mcp-schema-passthrough-spec.md +16 -7
- package/docs/specs/unified-credit-and-scheduled-execution-billing-spec.md +96 -117
- package/package.json +2 -2
- package/dist/chunk-L4LMSXLO.js +0 -7
- package/dist/chunk-L4LMSXLO.js.map +0 -1
- package/dist/server-PH3VJ74X.js.map +0 -1
- /package/dist/{chunk-TUSLATZY.js.map → chunk-X3NIXHHG.js.map} +0 -0
- /package/dist/{site-extract-repository-YNPXEZCB.js.map → site-extract-repository-5V5EEQ72.js.map} +0 -0
- /package/dist/{worker-45WEWCMQ.js.map → worker-GE5JMHE6.js.map} +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Unified Credits, Integrations, and Scheduled Execution Billing
|
|
2
2
|
|
|
3
|
-
- Status:
|
|
4
|
-
- Spec date: 2026-07-
|
|
3
|
+
- Status: Released v1 contract plus explicitly deferred hardening roadmap
|
|
4
|
+
- Spec date: 2026-07-14
|
|
5
5
|
- Primary repos: mcp-scraper, mcp-scraper-scheduler, mcpscraper-web, mcpscraper-sdk
|
|
6
6
|
- Commercial decision: Scheduled Actions is a product capability, not a separate subscription
|
|
7
7
|
- Accounting decision: MCP Scraper's main credit ledger is the only debit authority
|
|
@@ -10,21 +10,29 @@
|
|
|
10
10
|
|
|
11
11
|
MCP Scraper will use one visible usage currency: Credits.
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
Scheduled Actions uses the same Credit wallet as the rest of MCP Scraper. It is no longer sold as a separate $10-per-month product with a 1,000-run quota. Integrations themselves are included with paid plans: connecting, reconnecting, direct reads, enabled actions, supported exports, and connection-to-Memory snapshots do not currently create a separate integration-operation debit.
|
|
14
14
|
|
|
15
15
|
The replacement contract is:
|
|
16
16
|
|
|
17
17
|
1. Every paid MCP Scraper plan includes access to integrations and scheduling.
|
|
18
|
-
2.
|
|
19
|
-
3. Each scheduled occurrence that
|
|
20
|
-
4.
|
|
21
|
-
5.
|
|
22
|
-
6.
|
|
23
|
-
7.
|
|
18
|
+
2. OAuth connect and reconnect cost 0 Credits and there is no per-connection subscription or active-connection minimum.
|
|
19
|
+
3. Each scheduled occurrence that passes the balance check and begins execution costs 75 Credits exactly once.
|
|
20
|
+
4. A deterministic `connection_sync` occurrence has no model charge beyond that 75-Credit base.
|
|
21
|
+
5. An agent-mode occurrence additionally settles OpenRouter's actual reported model cost, multiplied by 1.5, then converted to Credits.
|
|
22
|
+
6. Token prices may be shown as current per-million-token estimates in Credits, but OpenRouter's reported cost is the settlement truth.
|
|
23
|
+
7. Separately metered scraper, browser, transcription, video-analysis, and similar tools keep their published rate-card prices when a schedule calls them.
|
|
24
|
+
8. The proposed 5/10/15/25-Credit connected-operation tariff, per-record/byte tariff, and 20,000-Credit active-connection minimum are deferred and are not enabled.
|
|
24
25
|
|
|
25
|
-
This design encourages usage: paid users can immediately connect and
|
|
26
|
+
This design encourages usage: paid users can immediately connect and use accounts without per-operation friction, while recurring automation and model work draw from the shared Credit balance. There is no artificial 1,000-run cliff or add-on product to understand.
|
|
26
27
|
|
|
27
|
-
|
|
28
|
+
### Released v1 implementation boundary
|
|
29
|
+
|
|
30
|
+
- Authorization currently performs an idempotent wallet balance check, not a hard balance reservation.
|
|
31
|
+
- The server-side default maximum is 5,000 Credits per run and is environment-configurable; it is not yet editable in the customer UI.
|
|
32
|
+
- A missing authoritative model cost moves the schedule to `cost_pending` and pauses it. Automated recovery/reconciliation is still a hardening item.
|
|
33
|
+
- Connected-operation receipts, user-editable monthly caps, auto-refill, and active-connection minimums remain future work unless a later commercial decision explicitly enables them.
|
|
34
|
+
|
|
35
|
+
## What was found before the v1 cutover
|
|
28
36
|
|
|
29
37
|
The originally intended scheduler economics are recognizable in the code, but the implementation is not safe enough to keep.
|
|
30
38
|
|
|
@@ -38,9 +46,9 @@ The originally intended scheduler economics are recognizable in the code, but th
|
|
|
38
46
|
|
|
39
47
|
So the answer to “was $10 meant to open the feature and then AI cost was marked up 50%?” is: substantially yes. The entitlement and the 50% markup both exist.
|
|
40
48
|
|
|
41
|
-
###
|
|
49
|
+
### Pre-cutover billing defect
|
|
42
50
|
|
|
43
|
-
The
|
|
51
|
+
The pre-cutover path did not reliably implement that intended price:
|
|
44
52
|
|
|
45
53
|
1. schedule-execute invokes both meterLlmUsage and billPassThroughCost for the generated result.
|
|
46
54
|
2. Both paths can call recordCost.
|
|
@@ -50,7 +58,7 @@ The effective path does not reliably implement that intended price:
|
|
|
50
58
|
6. The current metering client is fail-open, so missing configuration or a failed internal request can instead produce no debit.
|
|
51
59
|
7. The current call uses top-level usage and provider metadata. For a multi-step agent result, this describes the last step, not necessarily the complete run. Earlier model steps can be omitted.
|
|
52
60
|
|
|
53
|
-
The
|
|
61
|
+
The scheduler environment had the internal MCP Scraper URL, internal debit secret, and OpenRouter key configured, so this was a real production risk rather than dead local code. The v1 cutover removed the duplicate/fail-open path and now aggregates model-step costs before one settlement.
|
|
54
62
|
|
|
55
63
|
### Required immediate interpretation
|
|
56
64
|
|
|
@@ -58,25 +66,25 @@ Do not assume existing scheduled-agent charges equal a clean 1.5 markup. Before
|
|
|
58
66
|
|
|
59
67
|
## Canonical Build Spec
|
|
60
68
|
|
|
61
|
-
- **Request:** Replace separate Scheduled Actions billing with
|
|
69
|
+
- **Request:** Replace separate Scheduled Actions billing with the shared Credit wallet; charge one 75-Credit occurrence base and correctly settle all-step OpenRouter cost at 1.5; keep direct integration use included.
|
|
62
70
|
- **Lane:** Mastra Agentic Development plus MCP/API product billing.
|
|
63
71
|
- **Capability:** Unified Metered Execution.
|
|
64
72
|
- **Audience:** MCP Scraper customers, scheduled-agent users, MCP clients, SDK users, operators, finance, and support.
|
|
65
|
-
- **User-Facing Outcome:** A user connects a service,
|
|
73
|
+
- **User-Facing Outcome:** A user connects a service, uses it directly without a separate operation fee, optionally moves a bounded result into Memory, and pays 75 Credits plus applicable model/tool usage when recurring work executes.
|
|
66
74
|
- **Target Runtime:** Main MCP Scraper API and credit ledger; Vercel-hosted Mastra scheduler; Inngest workers; Nango connections and Proxy/functions; private Blob artifacts; public MCP; npm/SDK clients.
|
|
67
75
|
- **Inputs:** Authenticated identity, connection ID, tool or sync request, scheduled occurrence, model selection, per-run and monthly Credit limits, actual provider results, OpenRouter usage metadata, bytes and records delivered.
|
|
68
76
|
- **Agents:** Existing Scheduled Action Agent for interpretive work. Deterministic connection_sync workers remain non-agentic. Both use the same billing context.
|
|
69
77
|
- **Workflows:** Connect; direct read/write; bulk export; memory import; schedule estimate; occurrence claim; Credit authorization; execution; incremental settlement; final settlement; receipt; retry/reconciliation.
|
|
70
78
|
- **Tools:** Existing generic connected-service tools, provider-native tools, schedule tools, bulk export/import tools, credits_info, a new schedule estimator, and a shared internal billing adapter.
|
|
71
79
|
- **MCP:** Credit behavior is transport-independent. Tool descriptions state charge categories and estimates, results include receipt summaries, and exact provider tools remain protected by tenant ownership, action switches, and grants.
|
|
72
|
-
- **Signals:** Billing authorization created; low balance; budget threshold; cost pending; settlement complete; insufficient Credits
|
|
80
|
+
- **Signals:** Billing authorization created; low balance; budget threshold; cost pending; settlement complete; and insufficient Credits. Auto-refill and connection-minimum signals belong only to the deferred roadmap.
|
|
73
81
|
- **Background Tasks:** Usage aggregation, connection-minimum settlement, pending-cost reconciliation, low-balance notifications, automatic refill, legacy subscription migration, receipt retention, and rate-snapshot refresh.
|
|
74
82
|
- **Guardrails:** Integer microcredit arithmetic, idempotent charges, one debit authority, no model-visible billing credentials, fail closed before externally costly work, action approval independent of price, no prompt or provider credential in billing events.
|
|
75
83
|
- **Observability:** Itemized receipt per run; raw vendor cost; multiplier; model/provider; tokens; bytes; changed records; connection and schedule IDs; idempotency keys; authorization, settlement, and refund state.
|
|
76
84
|
- **Enforcement:** A call cannot bypass the billing adapter by using Mastra, the generic MCP bridge, direct REST, SDK, Nango, or a background job. Duplicate occurrence or HTTP retry cannot debit twice.
|
|
77
85
|
- **Persistence:** The main MCP Scraper/Turso credit ledger owns balances and debits. Scheduler Postgres owns schedule state, occurrence state, and a mirrored receipt reference, not the authoritative wallet.
|
|
78
86
|
- **Deployment:** Main API/ledger first, then scheduler and connection adapters, then public MCP, frontend, SDK/npm/MCPB, generated docs, Stripe migration, and Vercel production.
|
|
79
|
-
- **Done Contract:** A production scheduled run
|
|
87
|
+
- **Done Contract:** A production scheduled run performs one balance authorization, charges its 75-Credit base once, aggregates every OpenRouter model step once, and exposes the same pricing contract through the UI, MCP, API, and SDK. Hard reservations and full receipt UX remain the next hardening phase.
|
|
80
88
|
- **Receipt Requirements:** Code and deployment SHAs; rate-table revision; Stripe migration result; test output; production Vercel IDs; npm/MCPB versions; sample agent and deterministic-sync billing receipts; reconciliation report; public pricing and tool-description parity.
|
|
81
89
|
|
|
82
90
|
## Product contract in plain language
|
|
@@ -85,7 +93,7 @@ Do not assume existing scheduled-agent charges equal a clean 1.5 markup. Before
|
|
|
85
93
|
|
|
86
94
|
Scheduled Actions remains a first-class page and capability. It is not a separate product, subscription, or quota bundle.
|
|
87
95
|
|
|
88
|
-
Users with an active paid plan can create schedules immediately.
|
|
96
|
+
Users with an active paid plan can create schedules immediately. A future pay-as-you-go activation may use a minimum $10 Credit purchase, but that path is not part of the released v1 contract. If released, the $10 must be a wallet purchase rather than an expiring scheduler access fee.
|
|
89
97
|
|
|
90
98
|
Existing scheduled-action subscriptions are migrated; customers do not keep paying a separate scheduler subscription.
|
|
91
99
|
|
|
@@ -93,11 +101,12 @@ Existing scheduled-action subscriptions are migrated; customers do not keep payi
|
|
|
93
101
|
|
|
94
102
|
A scheduled occurrence pays:
|
|
95
103
|
|
|
96
|
-
|
|
97
|
-
+
|
|
98
|
-
+
|
|
99
|
-
|
|
100
|
-
|
|
104
|
+
75-Credit execution charge
|
|
105
|
+
+ 1.5 × actual OpenRouter cost when agent mode uses a model
|
|
106
|
+
+ any separately metered scraper, browser, transcription,
|
|
107
|
+
video-analysis, or other published rate-card tools it calls
|
|
108
|
+
|
|
109
|
+
Direct connected-service reads, enabled actions, supported exports, and bounded Memory snapshots do not add a separate integration-operation line item today.
|
|
101
110
|
|
|
102
111
|
Internal pagination, retries, leases, and orchestration steps are not separate “runs.” A retry of the same occurrence cannot pay the fixed charge again.
|
|
103
112
|
|
|
@@ -162,36 +171,34 @@ Auto-refill is off by default. The customer chooses a threshold and a monthly ma
|
|
|
162
171
|
|
|
163
172
|
This is intentionally less generous than subscription grants. Plans remain the best value; pay-as-you-go exists for activation and overflow.
|
|
164
173
|
|
|
165
|
-
##
|
|
174
|
+
## Released v1 rate table
|
|
166
175
|
|
|
167
176
|
All rates live in one versioned server-side registry. Frontends, MCP descriptions, SDKs, and docs consume generated projections from that registry.
|
|
168
177
|
|
|
169
178
|
### Connection operations
|
|
170
179
|
|
|
171
|
-
|
|
180
|
+
Released v1 rates:
|
|
172
181
|
|
|
173
182
|
| Billing class | Charge |
|
|
174
183
|
|---|---:|
|
|
175
184
|
| OAuth connect, reconnect, or consent callback | Free |
|
|
176
|
-
|
|
|
177
|
-
|
|
|
178
|
-
|
|
|
179
|
-
|
|
|
180
|
-
|
|
|
181
|
-
| Delivered or changed record | 2 Credits each |
|
|
182
|
-
| Transferred content | 10 Credits per started MiB |
|
|
183
|
-
| Provider or MCP Scraper validation failure before provider work | Free |
|
|
184
|
-
| Duplicate page, unchanged delta record, or idempotent replay | Free |
|
|
185
|
+
| Direct approved read | Included |
|
|
186
|
+
| Direct enabled action | Included |
|
|
187
|
+
| Supported connected-service export | Included |
|
|
188
|
+
| Bounded connection-to-Memory snapshot | Included |
|
|
189
|
+
| Active connection minimum | None |
|
|
185
190
|
|
|
186
|
-
|
|
191
|
+
Action safety remains independent of price: ownership, live inventory, source-controlled policy, the account action switch, and exact scheduled grants still apply.
|
|
187
192
|
|
|
188
|
-
|
|
193
|
+
### Deferred connected-operation pricing options (not enabled)
|
|
189
194
|
|
|
190
|
-
|
|
195
|
+
Earlier design work proposed 5/10/15/25-Credit read/write classes, a 25-Credit bulk base, per-record charges, and per-MiB charges. Those values are retained only as research for a later commercial decision. They are not present in the production registry and must not appear as current pricing in UI, MCP descriptions, SDKs, or support answers.
|
|
191
196
|
|
|
192
|
-
###
|
|
197
|
+
### Deferred active connection minimum (not enabled)
|
|
193
198
|
|
|
194
|
-
|
|
199
|
+
An earlier proposal described a 20,000-Credit minimum per active connection per 30-day billing period. The released product has no active-connection minimum, no first-connection waiver clock, and no connection billing-pause flow. Enabling any such minimum requires a new explicit commercial decision, customer notice, and a separate release.
|
|
200
|
+
|
|
201
|
+
The remainder of this subsection records the deferred proposal for future evaluation only:
|
|
195
202
|
|
|
196
203
|
One tenant-owned credential/account row is one connection. Two Google accounts, two Slack workspaces, or two WordPress sites are separate connections. Disconnected and billing-paused credentials do not accrue a new minimum; active and reauthorization-grace credentials accrue only for the active seconds in the period.
|
|
197
204
|
|
|
@@ -249,20 +256,14 @@ It also allows a light schedule to remain inexpensive. A schedule running at 8 A
|
|
|
249
256
|
An agent occurrence settles:
|
|
250
257
|
|
|
251
258
|
75-Credit occurrence base
|
|
252
|
-
+ provider tool charges
|
|
253
|
-
+ connected data records/bytes
|
|
254
259
|
+ OpenRouter actual cost times 1.5
|
|
255
|
-
+
|
|
260
|
+
+ any separately metered published tools invoked by the run
|
|
256
261
|
|
|
257
262
|
#### Deterministic connection sync
|
|
258
263
|
|
|
259
264
|
A connection_sync occurrence settles:
|
|
260
265
|
|
|
261
266
|
75-Credit occurrence base
|
|
262
|
-
+ 25-Credit bulk slice base when a slice starts
|
|
263
|
-
+ changed records
|
|
264
|
-
+ transferred bytes
|
|
265
|
-
+ memory embedding/ingestion processing
|
|
266
267
|
|
|
267
268
|
There is no model charge unless that deterministic pipeline actually invokes a model.
|
|
268
269
|
|
|
@@ -298,24 +299,22 @@ Assume one MiniMax M3 run uses 100,000 input tokens, 20,000 output tokens, and o
|
|
|
298
299
|
| Line item | Credits |
|
|
299
300
|
|---|---:|
|
|
300
301
|
| Scheduled occurrence | 75 |
|
|
301
|
-
| Connected-service read |
|
|
302
|
+
| Connected-service read | Included |
|
|
302
303
|
| Estimated model input | 300 |
|
|
303
304
|
| Estimated model output | 240 |
|
|
304
|
-
| Estimated total |
|
|
305
|
+
| Estimated total | 615 |
|
|
305
306
|
|
|
306
|
-
The model lines settle from the actual reported OpenRouter cost, so
|
|
307
|
+
The model lines settle from the actual reported OpenRouter cost, so 615 is an estimate until the receipt closes.
|
|
307
308
|
|
|
308
309
|
#### Deterministic bulk sync
|
|
309
310
|
|
|
310
|
-
Assume a sync delivers 500 changed records and 50 MiB:
|
|
311
|
+
Assume a deterministic connection sync delivers 500 changed records and 50 MiB without calling a separately metered tool:
|
|
311
312
|
|
|
312
313
|
| Line item | Credits |
|
|
313
314
|
|---|---:|
|
|
314
315
|
| Scheduled occurrence | 75 |
|
|
315
|
-
|
|
|
316
|
-
|
|
|
317
|
-
| 50 MiB transferred | 500 |
|
|
318
|
-
| Total before memory processing | 1,600 |
|
|
316
|
+
| Connected records and transfer | Included |
|
|
317
|
+
| Total | 75 |
|
|
319
318
|
|
|
320
319
|
### Existing non-scheduler AI classes
|
|
321
320
|
|
|
@@ -347,24 +346,16 @@ The UI may place raw dollar rates in a secondary tooltip or expandable explanati
|
|
|
347
346
|
|
|
348
347
|
## Per-run and monthly budgets
|
|
349
348
|
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
- maxCreditsPerRun, default 5,000 Credits;
|
|
353
|
-
- maxCreditsPerMonth, default 50,000 Credits;
|
|
354
|
-
- lowBalanceBehavior, initially pause;
|
|
355
|
-
- optional model allowlist;
|
|
356
|
-
- optional maximum model steps and output tokens;
|
|
357
|
-
- an estimate produced before activation.
|
|
358
|
-
|
|
359
|
-
The user can edit these limits without recreating the schedule.
|
|
349
|
+
Released v1 has a server-side `maxCreditsPerRun` default of 5,000 Credits and pauses on insufficient balance or unresolved model cost. The cap is environment-configurable and is not yet user-editable. User-defined monthly caps, model allowlists, auto-refill behavior, and a pre-activation estimator remain future work.
|
|
360
350
|
|
|
361
351
|
Before each occurrence:
|
|
362
352
|
|
|
363
|
-
1.
|
|
364
|
-
2. If the
|
|
365
|
-
3.
|
|
366
|
-
4.
|
|
367
|
-
|
|
353
|
+
1. Perform an idempotent balance authorization before model execution.
|
|
354
|
+
2. If the wallet cannot cover the maximum, skip without a run-base charge.
|
|
355
|
+
3. Settle the 75-Credit base once plus all authoritative model-step cost.
|
|
356
|
+
4. If authoritative model cost is missing, move to `cost_pending` and pause later occurrences.
|
|
357
|
+
|
|
358
|
+
Hard reservations, incremental budget stopping, monthly caps, and automatic release of unused reserved balance remain hardening work.
|
|
368
359
|
|
|
369
360
|
The current very large max-output-token ceiling cannot function as the spending guard. Budget enforcement must occur during execution, including after each model step.
|
|
370
361
|
|
|
@@ -374,28 +365,26 @@ The current very large max-output-token ceiling cannot function as the spending
|
|
|
374
365
|
|
|
375
366
|
Inngest or scheduler trigger
|
|
376
367
|
-> idempotently claim identity + schedule + scheduledFor
|
|
377
|
-
-> ask main billing service for
|
|
378
|
-
-> receive opaque billing context
|
|
368
|
+
-> ask main billing service for an idempotent balance authorization
|
|
369
|
+
-> receive an opaque billing context
|
|
379
370
|
-> execute exact schedule grants
|
|
380
|
-
->
|
|
381
|
-
->
|
|
382
|
-
->
|
|
383
|
-
-> release unused reservation
|
|
384
|
-
-> persist itemized receipt reference
|
|
385
|
-
-> expose the same receipt on API, MCP, SDK, and UI
|
|
371
|
+
-> aggregate every OpenRouter model-step cost
|
|
372
|
+
-> settle the 75-Credit base plus model amount once
|
|
373
|
+
-> persist billing event references
|
|
386
374
|
|
|
387
375
|
The billing context travels in an internal authenticated transport header or server-side execution context. It is never put in the model prompt, tool arguments, provider payload, or public result.
|
|
388
376
|
|
|
377
|
+
Hard reservations, incremental child-call line items, and full customer-facing itemized receipts are the next hardening phase rather than released-v1 claims.
|
|
378
|
+
|
|
389
379
|
### Direct non-scheduled flow
|
|
390
380
|
|
|
391
|
-
Direct MCP, REST, or SDK work follows the same
|
|
381
|
+
Direct connected-service MCP, REST, or SDK work is included and still follows the same tenant ownership and permission checks. If that work invokes a separately metered public tool, the existing tool-specific billing path applies:
|
|
392
382
|
|
|
393
|
-
|
|
394
|
-
-> execute
|
|
395
|
-
->
|
|
396
|
-
-> return result plus receipt summary
|
|
383
|
+
validate tenant-owned exact tool
|
|
384
|
+
-> execute the connected-service operation without an integration debit
|
|
385
|
+
-> separately meter only published scraper/media/browser classes used
|
|
397
386
|
|
|
398
|
-
The generic MCP bridge and a provider-native MCP tool must produce
|
|
387
|
+
The generic MCP bridge and a provider-native MCP tool must not silently produce different integration pricing for the same operation.
|
|
399
388
|
|
|
400
389
|
### Multi-step model accounting
|
|
401
390
|
|
|
@@ -430,9 +419,9 @@ The main MCP Scraper billing service is the only component allowed to mutate the
|
|
|
430
419
|
|
|
431
420
|
The existing internal memory AI-debit endpoint becomes a compatibility adapter during migration, then routes through the unified billing service. It may not apply an extra 3.0 multiplier to a scheduled_agent_llm event.
|
|
432
421
|
|
|
433
|
-
## Billing data model
|
|
422
|
+
## Billing data model and hardening roadmap
|
|
434
423
|
|
|
435
|
-
Names are illustrative;
|
|
424
|
+
The released v1 authorization and billing-event tables are smaller than the target below. Names are illustrative; the following reservation, receipt, and connection-period structures describe future hardening and must not be treated as already released.
|
|
436
425
|
|
|
437
426
|
### billing_events
|
|
438
427
|
|
|
@@ -515,7 +504,7 @@ The canonical occurrence key is:
|
|
|
515
504
|
|
|
516
505
|
schedule:{identityId}:{scheduleId}:{scheduledForIso}
|
|
517
506
|
|
|
518
|
-
It owns at most one occurrence-base event.
|
|
507
|
+
It owns at most one occurrence-base event. In released v1 the base event is created only after the claim obtains its balance authorization and execution begins. Future hard-reservation work will retain the same key. Worker retries, leases, and resumed slices reference that event.
|
|
519
508
|
|
|
520
509
|
### Line-item keys
|
|
521
510
|
|
|
@@ -534,7 +523,7 @@ The main ledger rejects a second settlement for the same key and returns the exi
|
|
|
534
523
|
|
|
535
524
|
- Work completed before a provider or user error remains chargeable.
|
|
536
525
|
- No provider-work line is charged for validation failure before the provider call.
|
|
537
|
-
-
|
|
526
|
+
- Included connected-service records and bytes do not create their own charge in released v1.
|
|
538
527
|
- Unused reservation is always released.
|
|
539
528
|
- The 75-Credit platform base is refunded when MCP Scraper itself fails before useful execution.
|
|
540
529
|
- The base is not refunded for invalid customer configuration, expired OAuth after a valid start, provider outage after dispatch, or user-authored agent failure unless support policy grants a courtesy refund.
|
|
@@ -597,6 +586,8 @@ Refunds append compensating ledger events. Settled rows are never silently edite
|
|
|
597
586
|
|
|
598
587
|
## Public API, MCP, and SDK contract
|
|
599
588
|
|
|
589
|
+
Released v1 publishes the 75-Credit occurrence base and 1.5-times actual OpenRouter rule across the hosted MCP, npm packages, generated SDK contracts, marketing pages, and scheduler status. The richer fields below are the follow-on receipt and budget roadmap.
|
|
590
|
+
|
|
600
591
|
### Existing tools to extend
|
|
601
592
|
|
|
602
593
|
credits_info adds:
|
|
@@ -706,10 +697,8 @@ The deterministic worker:
|
|
|
706
697
|
|
|
707
698
|
- claims the same occurrence key;
|
|
708
699
|
- authorizes its budget;
|
|
709
|
-
-
|
|
710
|
-
-
|
|
711
|
-
- reports content bytes once per stored artifact;
|
|
712
|
-
- reports memory processing through existing explicit classes;
|
|
700
|
+
- charges the 75-Credit occurrence base once;
|
|
701
|
+
- does not add record, byte, or connected-operation charges in released v1;
|
|
713
702
|
- checkpoints before settlement so retry behavior is deterministic.
|
|
714
703
|
|
|
715
704
|
The worker never invokes a model merely to paginate or transfer content.
|
|
@@ -724,14 +713,7 @@ Under every paid plan:
|
|
|
724
713
|
|
|
725
714
|
> Integrations and scheduled work included. Usage draws from your shared Credit balance.
|
|
726
715
|
|
|
727
|
-
|
|
728
|
-
|
|
729
|
-
- active connection minimum;
|
|
730
|
-
- read/write/bulk rates;
|
|
731
|
-
- 75-Credit scheduled occurrence;
|
|
732
|
-
- current model estimates in Credits;
|
|
733
|
-
- “actual OpenRouter cost plus 50%” disclosure;
|
|
734
|
-
- expiry, refill, and budget behavior.
|
|
716
|
+
Show the 75-Credit scheduled occurrence, the “actual OpenRouter cost plus 50%” disclosure, and the fact that direct integration use is included. Do not show the deferred active-connection minimum or per-operation tariff as current pricing.
|
|
735
717
|
|
|
736
718
|
### Integrations page
|
|
737
719
|
|
|
@@ -740,10 +722,6 @@ Each card shows:
|
|
|
740
722
|
- connected account count;
|
|
741
723
|
- real callable read/action inventory;
|
|
742
724
|
- action switch;
|
|
743
|
-
- current billing-period usage;
|
|
744
|
-
- minimum progress;
|
|
745
|
-
- next settlement;
|
|
746
|
-
- billing state;
|
|
747
725
|
- a link to import/schedule.
|
|
748
726
|
|
|
749
727
|
OAuth authorization itself remains free.
|
|
@@ -782,6 +760,10 @@ Users can filter by connection, schedule, model, provider, and date.
|
|
|
782
760
|
|
|
783
761
|
## Scheduled Actions subscription migration
|
|
784
762
|
|
|
763
|
+
### Live migration result
|
|
764
|
+
|
|
765
|
+
The legacy live-mode Stripe price `price_1ToXcHS8aAcsk3TGf3cW7zHx` still exists as an active $10 monthly price, but a read-only all-status audit on 2026-07-14 found zero subscriptions. The old checkout route now returns `410 Gone`, so no customer cancellation, proration, or compensating migration Credit is currently required. The price may be archived separately as Stripe catalog cleanup; archiving it is not required for runtime cutover and was not performed automatically.
|
|
766
|
+
|
|
785
767
|
### Customer treatment
|
|
786
768
|
|
|
787
769
|
1. Stop new scheduler-subscription checkout after the unified billing path is proven.
|
|
@@ -854,11 +836,11 @@ Issue compensating Credits for defensible overcharges. Do not retroactively debi
|
|
|
854
836
|
- Enable per-run/month caps and cost-pending reconciliation.
|
|
855
837
|
- Expand to production cohorts after receipt review.
|
|
856
838
|
|
|
857
|
-
### Gate 5 — deterministic sync and direct connection billing
|
|
839
|
+
### Gate 5 — deterministic sync and direct connection billing (partially deferred)
|
|
858
840
|
|
|
859
|
-
- Route connection_sync
|
|
860
|
-
-
|
|
861
|
-
-
|
|
841
|
+
- Route `connection_sync` through the shared 75-Credit occurrence context.
|
|
842
|
+
- Keep generic MCP, REST, SDK, exports, and snapshots included in released v1.
|
|
843
|
+
- Reconsider connected-operation tariffs or active-connection minimums only after a new explicit commercial decision, shadow period, and customer notice.
|
|
862
844
|
|
|
863
845
|
### Gate 6 — commercial migration
|
|
864
846
|
|
|
@@ -908,13 +890,10 @@ Issue compensating Credits for defensible overcharges. Do not retroactively debi
|
|
|
908
890
|
### Connection tests
|
|
909
891
|
|
|
910
892
|
- OAuth callback is free.
|
|
911
|
-
-
|
|
893
|
+
- Direct reads, enabled actions, exports, and bounded Memory snapshots do not create an integration-operation debit.
|
|
912
894
|
- Internal pagination is not separately billed.
|
|
913
|
-
-
|
|
914
|
-
-
|
|
915
|
-
- Direct MCP, REST, SDK, Mastra, and background calls produce parity.
|
|
916
|
-
- Connection minimum offsets eligible usage and prorates correctly.
|
|
917
|
-
- First-connection waiver and billing-pause grace work.
|
|
895
|
+
- Direct MCP, REST, SDK, Mastra, and background calls publish the same included-use rule.
|
|
896
|
+
- Deferred operation tariffs and active-connection minimums are absent from the production registry and public copy.
|
|
918
897
|
|
|
919
898
|
### Ledger tests
|
|
920
899
|
|
|
@@ -961,7 +940,7 @@ Receipts must be sanitized. Do not include prompts, connection tokens, API keys,
|
|
|
961
940
|
- Separate scheduling entitlement or 1,000-run quota still enforced on any surface.
|
|
962
941
|
- Frontend, hosted MCP, SDK/npm, MCPB, or public docs showing different rates.
|
|
963
942
|
- A fixed token rate described as authoritative rather than an estimate.
|
|
964
|
-
-
|
|
943
|
+
- Any connected-operation tariff or active-connection minimum exposed without a new approved commercial release.
|
|
965
944
|
- Commercial migration without a reconciliation and compensating-credit policy.
|
|
966
945
|
|
|
967
946
|
## Decisions locked by this spec
|
|
@@ -972,10 +951,10 @@ Receipts must be sanitized. Do not include prompts, connection tokens, API keys,
|
|
|
972
951
|
- scheduled_agent_llm uses a 1.5 multiplier.
|
|
973
952
|
- Model token tables are estimates.
|
|
974
953
|
- The fixed scheduled-occurrence charge is 75 Credits.
|
|
975
|
-
-
|
|
954
|
+
- Direct integration use is included; the proposed 5/10/15/25 tariffs and 20,000-Credit active-connection minimum are not enabled.
|
|
976
955
|
- OAuth is free.
|
|
977
956
|
- Main MCP Scraper owns the only mutable Credit ledger.
|
|
978
|
-
- Every product and execution surface
|
|
957
|
+
- Every product and execution surface publishes the same price contract; shared itemized receipt UX remains hardening work.
|
|
979
958
|
- Action authorization remains separate from billing.
|
|
980
959
|
|
|
981
960
|
## Decisions that may change only through a rate revision
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "mcp-scraper",
|
|
3
|
-
"version": "0.21.
|
|
3
|
+
"version": "0.21.2",
|
|
4
4
|
"description": "MCP server for MCP Scraper web intelligence tools",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.cjs",
|
|
@@ -76,7 +76,7 @@
|
|
|
76
76
|
"hono": "^4.12.18",
|
|
77
77
|
"inngest": "^4.4.0",
|
|
78
78
|
"jose": "^6.2.3",
|
|
79
|
-
"mcpscraper-memory-tools": "^1.5.
|
|
79
|
+
"mcpscraper-memory-tools": "^1.5.1",
|
|
80
80
|
"p-limit": "^7.3.0",
|
|
81
81
|
"papaparse": "^5.4.0",
|
|
82
82
|
"playwright": "^1.44.0",
|
package/dist/chunk-L4LMSXLO.js
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"sources":["../src/version.ts"],"sourcesContent":["export const PACKAGE_VERSION = '0.21.0'\n"],"mappings":";AAAO,IAAM,kBAAkB;","names":[]}
|