mcp-scraper 0.19.0 → 0.20.0

package/docs/specs/unified-credit-and-scheduled-execution-billing-spec.md

@@ -0,0 +1,1007 @@
+# Unified Credits, Integrations, and Scheduled Execution Billing
+
+- Status: Proposed canonical implementation spec
+- Spec date: 2026-07-13
+- Primary repos: mcp-scraper, mcp-scraper-scheduler, mcpscraper-web, mcpscraper-sdk
+- Commercial decision: Scheduled Actions is a product capability, not a separate subscription
+- Accounting decision: MCP Scraper's main credit ledger is the only debit authority
+
+## Executive decision
+
+MCP Scraper will use one visible usage currency: Credits.
+
+Integrations, bulk connected-data movement, scheduled execution, model inference, and memory ingestion will all consume the same wallet. Scheduled Actions will no longer be sold as a separate $10-per-month product with a 1,000-run quota.
+
+The replacement contract is:
+
+1. Every paid MCP Scraper plan includes access to integrations and scheduling.
+2. A customer without a recurring plan may activate metered features with a minimum $10 credit purchase once pay-as-you-go activation exists.
+3. Each scheduled occurrence that successfully reserves Credits and begins execution costs a small fixed execution charge plus the exact work performed.
+4. OpenRouter-backed AI is settled from OpenRouter's actual reported cost, multiplied by 1.5, then converted to Credits.
+5. Token prices may be shown as current per-million-token estimates in Credits, but the reported OpenRouter cost is the invoice truth.
+6. Active connections have a monthly Credit minimum that is offset by their actual eligible connection usage. This pays for OAuth, refresh, provider infrastructure, and idle connection overhead without charging a confusing separate integration subscription.
+7. Every execution path—the public MCP, REST API, SDK, Mastra, background sync, and frontend—uses the same billing adapter and produces the same itemized receipt.
+
+This design encourages usage: paid users can immediately connect and schedule work, higher-usage customers naturally consume more Credits, and there is no artificial 1,000-run cliff or add-on product to understand.
+
+## What exists today
+
+The originally intended scheduler economics are recognizable in the code, but the implementation is not safe enough to keep.
+
+### Confirmed historical behavior
+
+- Stripe has a $10-per-month Scheduled Actions checkout.
+- The webhook grants a scheduling entitlement with a 1,000-run period quota.
+- The scheduler contains a 1.5 OpenRouter markup constant.
+- The scheduler reads OpenRouter's provider-reported usage cost.
+- Scheduled agent execution attempts to record model usage and a pass-through cost.
+
+So the answer to “was $10 meant to open the feature and then AI cost was marked up 50%?” is: substantially yes. The entitlement and the 50% markup both exist.
+
+### Current billing defect
+
+The effective path does not reliably implement that intended price:
+
+1. schedule-execute invokes both meterLlmUsage and billPassThroughCost for the generated result.
+2. Both paths can call recordCost.
+3. recordCost can forward each raw AI cost to the main MCP Scraper AI-debit endpoint.
+4. That endpoint applies the existing 3.0 memory-AI multiplier.
+5. If both HTTP debits succeed, the final-step model cost can therefore produce two 3.0 debits—up to 6.0 times that raw cost—while the scheduler's local usage record describes a 1.5 billable amount.
+6. The current metering client is fail-open, so missing configuration or a failed internal request can instead produce no debit.
+7. The current call uses top-level usage and provider metadata. For a multi-step agent result, this describes the last step, not necessarily the complete run. Earlier model steps can be omitted.
+
+The live scheduler environment has the internal MCP Scraper URL, internal debit secret, and OpenRouter key configured. This means the path is wired in production configuration; it is not merely dead local code.
+
+### Required immediate interpretation
+
+Do not assume existing scheduled-agent charges equal a clean 1.5 markup. Before changing customers or migrating subscriptions, reconcile scheduler usage records against the main credit ledger. Overcharges receive compensating Credits. Historical fail-open undercharges are not collected retroactively.
+
+## Canonical Build Spec
+
+- **Request:** Replace separate Scheduled Actions billing with a unified Credit wallet; correctly settle connected-service operations, deterministic syncs, and all-step OpenRouter costs; expose clear per-run budgets and receipts on every product surface.
+- **Lane:** Mastra Agentic Development plus MCP/API product billing.
+- **Capability:** Unified Metered Execution.
+- **Audience:** MCP Scraper customers, scheduled-agent users, MCP clients, SDK users, operators, finance, and support.
+- **User-Facing Outcome:** A user connects a service, optionally moves its data into memory, creates a schedule, sees an estimate and spending limit, and pays one itemized Credit charge for each completed or partially completed unit of work.
+- **Target Runtime:** Main MCP Scraper API and credit ledger; Vercel-hosted Mastra scheduler; Inngest workers; Nango connections and Proxy/functions; private Blob artifacts; public MCP; npm/SDK clients.
+- **Inputs:** Authenticated identity, connection ID, tool or sync request, scheduled occurrence, model selection, per-run and monthly Credit limits, actual provider results, OpenRouter usage metadata, bytes and records delivered.
+- **Agents:** Existing Scheduled Action Agent for interpretive work. Deterministic connection_sync workers remain non-agentic. Both use the same billing context.
+- **Workflows:** Connect; direct read/write; bulk export; memory import; schedule estimate; occurrence claim; Credit authorization; execution; incremental settlement; final settlement; receipt; retry/reconciliation.
+- **Tools:** Existing generic connected-service tools, provider-native tools, schedule tools, bulk export/import tools, credits_info, a new schedule estimator, and a shared internal billing adapter.
+- **MCP:** Credit behavior is transport-independent. Tool descriptions state charge categories and estimates, results include receipt summaries, and exact provider tools remain protected by tenant ownership, action switches, and grants.
+- **Signals:** Billing authorization created; low balance; budget threshold; cost pending; settlement complete; insufficient Credits; automatic refill complete/failed; connection minimum due; connection billing paused.
+- **Background Tasks:** Usage aggregation, connection-minimum settlement, pending-cost reconciliation, low-balance notifications, automatic refill, legacy subscription migration, receipt retention, and rate-snapshot refresh.
+- **Guardrails:** Integer microcredit arithmetic, idempotent charges, one debit authority, no model-visible billing credentials, fail closed before externally costly work, action approval independent of price, no prompt or provider credential in billing events.
+- **Observability:** Itemized receipt per run; raw vendor cost; multiplier; model/provider; tokens; bytes; changed records; connection and schedule IDs; idempotency keys; authorization, settlement, and refund state.
+- **Enforcement:** A call cannot bypass the billing adapter by using Mastra, the generic MCP bridge, direct REST, SDK, Nango, or a background job. Duplicate occurrence or HTTP retry cannot debit twice.
+- **Persistence:** The main MCP Scraper/Turso credit ledger owns balances and debits. Scheduler Postgres owns schedule state, occurrence state, and a mirrored receipt reference, not the authoritative wallet.
+- **Deployment:** Main API/ledger first, then scheduler and connection adapters, then public MCP, frontend, SDK/npm/MCPB, generated docs, Stripe migration, and Vercel production.
+- **Done Contract:** A production scheduled run authorizes a budget, records all model steps and connected-service work once, settles one exact Credit receipt, releases unused Credits, and presents the same receipt through the UI, MCP, API, and SDK.
+- **Receipt Requirements:** Code and deployment SHAs; rate-table revision; Stripe migration result; test output; production Vercel IDs; npm/MCPB versions; sample agent and deterministic-sync billing receipts; reconciliation report; public pricing and tool-description parity.
+
+## Product contract in plain language
+
+### Scheduling is included
+
+Scheduled Actions remains a first-class page and capability. It is not a separate product, subscription, or quota bundle.
+
+Users with an active paid plan can create schedules immediately. Pay-as-you-go accounts can create schedules after a minimum $10 wallet activation is released. The $10 is a Credit purchase, not an expiring scheduler access fee.
+
+Existing scheduled-action subscriptions are migrated; customers do not keep paying a separate scheduler subscription.
+
+### Customers pay for a run, not internal loops
+
+A scheduled occurrence pays:
+
+    fixed execution charge
+    + connected-service work
+    + bulk records and bytes delivered
+    + model usage
+    + memory processing
+
+Internal pagination, retries, leases, and orchestration steps are not separate “runs.” A retry of the same occurrence cannot pay the fixed charge again.
+
+### OpenRouter AI costs are transparent
+
+The customer-facing rule is:
+
+> AI model use consumes Credits equal to 1.5 times OpenRouter's actual reported cost.
+
+MCP Scraper may show a model's current input, output, cached-input, image, and audio prices as estimated Credits per unit. Actual settlement always uses OpenRouter's returned usage.cost across every successful model step. This correctly captures routing, cache discounts, reasoning tokens, media, and provider-specific pricing.
+
+### Action safety is not for sale
+
+Charging more for a write does not authorize it. Mutations still require:
+
+- a tenant-owned active connection;
+- a live upstream tool that intersects source-controlled policy;
+- the connection's action switch;
+- the exact scheduled grant when scheduled;
+- any required confirmation or destructive-action rule.
+
+## Credit unit and accounting rules
+
+### Canonical units
+
+- Internal accounting unit: mc, an integer microcredit.
+- Public accounting unit: Credit.
+- Conversion: 100 mc = 1 Credit.
+- Billing conversion: 2,000,000 mc per $3.00 of vendor cost, represented as an exact rational rather than a floating-point value.
+- Fractional public Credits may be displayed to two decimal places, but all ledger math uses integer mc.
+- Credit lots continue to expire according to the current three-month policy unless a later commercial decision changes it.
+- Debits consume eligible lots using the existing ledger order.
+
+The public skill currently says 1 Credit equals 1,000 mc while the source rate table says 1 Credit equals 100 mc. This 10-times documentation discrepancy is a release blocker. All public rates and examples must be generated from the canonical source table.
+
+### Current plan grants
+
+The current source rate table implies:
+
+| Plan | Monthly price | Credits | Approximate Credits per dollar |
+|---|---:|---:|---:|
+| Starter | $12 | 80,000 | 6,666.7 |
+| Growth | $40 | 266,667 | 6,666.7 |
+| Scale | $100 | 800,000 | 8,000 |
+
+The Scale grant is a deliberate volume discount. A fixed Credit charge therefore has a lower cash-equivalent markup for a Scale customer. Product copy may say the billing formula is 1.5 times raw OpenRouter cost; it must not promise that every plan produces a 50% cash gross margin after volume discounts.
+
+The billing conversion must become an explicit rate-registry constant. It must no longer be derived from a Stripe plan's price or Credit grant. Otherwise an innocent plan-price edit would silently reprice AI work.
+
+### One-time activation and refill
+
+The old one-time pack mapping must not be reactivated because its $10 pack would grant 110,000 Credits and undercut recurring plans.
+
+Recommended pay-as-you-go contract:
+
+| Purchase | Credits | Purpose |
+|---|---:|---|
+| $10 minimum activation/refill | 50,000 | Enables metered integrations and schedules for a non-plan account |
+| Customer-selected auto-refill | Same 5,000 Credits per dollar | Prevents interrupted schedules, subject to a monthly dollar cap |
+
+Auto-refill is off by default. The customer chooses a threshold and a monthly maximum, and receives a receipt for each Stripe charge.
+
+This is intentionally less generous than subscription grants. Plans remain the best value; pay-as-you-go exists for activation and overflow.
+
+## Canonical rate table
+
+All rates live in one versioned server-side registry. Frontends, MCP descriptions, SDKs, and docs consume generated projections from that registry.
+
+### Connection operations
+
+Initial launch rates:
+
+| Billing class | Charge |
+|---|---:|
+| OAuth connect, reconnect, or consent callback | Free |
+| Successful bounded read | 5 Credits |
+| Successful low-complexity write | 10 Credits |
+| Successful medium-complexity write | 15 Credits |
+| Successful high-risk or high-complexity approved write | 25 Credits |
+| Start a bulk export or sync slice | 25 Credits |
+| Delivered or changed record | 2 Credits each |
+| Transferred content | 10 Credits per started MiB |
+| Provider or MCP Scraper validation failure before provider work | Free |
+| Duplicate page, unchanged delta record, or idempotent replay | Free |
+
+Tool manifests classify each action; a caller cannot choose a cheaper class. The provider-policy review owns read/write/risk classification.
+
+Charge for the outcome, not raw upstream requests. For example, a 500-item export that requires ten provider pages is one bulk base plus 500 delivered records, not ten separate read charges.
+
+A bounded read is limited to one logical object or at most 100 summary records and 1 MiB of structured output. Requests above either threshold use the bulk contract. A provider tool cannot evade bulk rates merely by accepting a very large page size.
+
+### Active connection minimum
+
+Each active connection has a 20,000-Credit minimum per 30-day billing period, prorated to the time it is active.
+
+One tenant-owned credential/account row is one connection. Two Google accounts, two Slack workspaces, or two WordPress sites are separate connections. Disconnected and billing-paused credentials do not accrue a new minimum; active and reauthorization-grace credentials accrue only for the active seconds in the period.
+
+Eligible connected-service usage offsets this minimum:
+
+- bounded reads and writes;
+- bulk-export base;
+- delivered/changed records;
+- transferred bytes;
+- explicit provider-operation surcharges when later added.
+
+The following do not offset the connection minimum:
+
+- OpenRouter model inference;
+- scheduled-occurrence base;
+- generic web scraping;
+- memory storage capacity;
+- memory embeddings or other AI processing.
+
+The first connection's first 30 days are waived. This creates a real activation path before an idle-connection charge appears.
+
+At settlement, only the shortfall is charged:
+
+    connection minimum debit
+    = max(0, prorated minimum - eligible usage already charged)
+
+If a wallet cannot cover the minimum:
+
+1. Mark the connection billing-paused.
+2. Allow a seven-day read-only recovery window for refill or plan activation.
+3. Block new scheduled use and mutations immediately.
+4. After the grace period, remove or disconnect the stored upstream credential if necessary to stop ongoing vendor charges.
+5. Keep non-secret connection history and explain that reconnection is required.
+
+This behavior must be disclosed before connection and in low-balance notices.
+
+### Scheduled execution
+
+| Billing class | Charge |
+|---|---:|
+| Authorized scheduled occurrence that begins execution | 75 Credits |
+| Duplicate/replayed occurrence | Free |
+| Skipped before execution for insufficient Credits | Free |
+| Work inside the occurrence | Normal applicable rates |
+
+The 75-Credit base preserves the rough economics of the former $10-for-1,000-runs product:
+
+- 1,000 runs use 75,000 Credits.
+- That is approximately $9.38 of Scale plan Credits or $11.25 of Starter/Growth plan Credits.
+
+It also allows a light schedule to remain inexpensive. A schedule running at 8 AM, noon, and 4 PM Mountain executes about 90 times in a 30-day month, using 6,750 Credits in fixed run charges before its data and model work.
+
+#### Agent schedule
+
+An agent occurrence settles:
+
+    75-Credit occurrence base
+    + provider tool charges
+    + connected data records/bytes
+    + OpenRouter actual cost times 1.5
+    + memory ingestion or artifact processing
+
+#### Deterministic connection sync
+
+A connection_sync occurrence settles:
+
+    75-Credit occurrence base
+    + 25-Credit bulk slice base when a slice starts
+    + changed records
+    + transferred bytes
+    + memory embedding/ingestion processing
+
+There is no model charge unless that deterministic pipeline actually invokes a model.
+
+### OpenRouter model inference
+
+The settlement formula is:
+
+    raw_cost_usd = sum(OpenRouter usage.cost for every successful model step)
+    billable_mc = ceil(raw_cost_usd × 1.5 × canonical_mc_per_usd)
+    billable_credits = billable_mc / 100
+
+At the current canonical conversion derived from Starter/Growth grants:
+
+    canonical_mc_per_usd = 2,000,000 / 3
+    $1.00 raw OpenRouter cost = 10,000 billed Credits
+
+As of this spec, OpenRouter lists MiniMax M3 at $0.30 per million input tokens, $1.20 per million output tokens, and $0.06 per million cached-input tokens for standard providers. The customer-facing estimate is therefore:
+
+| MiniMax M3 usage | Estimated Credits |
+|---|---:|
+| 1 million input tokens | 3,000 Credits |
+| 1 million output tokens | 12,000 Credits |
+| 1 million cached-input tokens | 600 Credits |
+
+These are display estimates, not a settlement substitute. The rate UI must show the source timestamp and refresh when OpenRouter pricing changes.
+
+### Worked examples
+
+#### Light scheduled agent
+
+Assume one MiniMax M3 run uses 100,000 input tokens, 20,000 output tokens, and one bounded connected-service read:
+
+| Line item | Credits |
+|---|---:|
+| Scheduled occurrence | 75 |
+| Connected-service read | 5 |
+| Estimated model input | 300 |
+| Estimated model output | 240 |
+| Estimated total | 620 |
+
+The model lines settle from the actual reported OpenRouter cost, so 620 is an estimate until the receipt closes.
+
+#### Deterministic bulk sync
+
+Assume a sync delivers 500 changed records and 50 MiB:
+
+| Line item | Credits |
+|---|---:|
+| Scheduled occurrence | 75 |
+| Bulk slice | 25 |
+| 500 changed records | 1,000 |
+| 50 MiB transferred | 500 |
+| Total before memory processing | 1,600 |
+
+### Existing non-scheduler AI classes
+
+The unified registry keeps existing product-specific multipliers explicit:
+
+| Billing class | Initial multiplier |
+|---|---:|
+| scheduled_agent_llm | 1.5 |
+| memory_ai | 3.0 |
+| site_audit_ai | 1.5 |
+| video_analysis_ai | 1.5 |
+| media_transcription_ai | 3.0 |
+
+The caller supplies the billing class, not the multiplier. The main billing server resolves the multiplier from its versioned registry. This prevents a compromised or outdated worker from choosing its own price.
+
+## Price display policy
+
+The primary display is Credits.
+
+Recommended copy:
+
+> Integrations and scheduling are included. Each run uses Credits for the work it performs.
+
+> AI model use is billed at OpenRouter's actual reported cost plus 50%, converted to Credits.
+
+> Estimated MiniMax M3 rate: 3,000 Credits per million input tokens and 12,000 Credits per million output tokens. Exact usage may vary by provider, caching, reasoning, media, and routing.
+
+The UI may place raw dollar rates in a secondary tooltip or expandable explanation. A single fixed dollar-per-million claim must not replace actual-cost settlement.
+
+## Per-run and monthly budgets
+
+Every schedule has:
+
+- maxCreditsPerRun, default 5,000 Credits;
+- maxCreditsPerMonth, default 50,000 Credits;
+- lowBalanceBehavior, initially pause;
+- optional model allowlist;
+- optional maximum model steps and output tokens;
+- an estimate produced before activation.
+
+The user can edit these limits without recreating the schedule.
+
+Before each occurrence:
+
+1. Calculate a conservative reservation from the per-run cap and available balance.
+2. If the monthly cap has insufficient headroom, skip with no run-base charge.
+3. If the wallet cannot reserve the required amount, skip with no run-base charge and notify the owner.
+4. Stop further model steps and provider mutations before exceeding the reserved cap.
+5. Release unused reserved Credits after settlement.
+
+The current very large max-output-token ceiling cannot function as the spending guard. Budget enforcement must occur during execution, including after each model step.
+
+## End-to-end execution contract
+
+### Occurrence flow
+
+    Inngest or scheduler trigger
+      -> idempotently claim identity + schedule + scheduledFor
+      -> ask main billing service for a Credit reservation
+      -> receive opaque billing context and reservation
+      -> execute exact schedule grants
+      -> attach billing context to internal child calls
+      -> accumulate provider, record, byte, model, and memory line items
+      -> settle actual integer mc once
+      -> release unused reservation
+      -> persist itemized receipt reference
+      -> expose the same receipt on API, MCP, SDK, and UI
+
+The billing context travels in an internal authenticated transport header or server-side execution context. It is never put in the model prompt, tool arguments, provider payload, or public result.
+
+### Direct non-scheduled flow
+
+Direct MCP, REST, or SDK work follows the same abbreviated path:
+
+    authorize maximum charge
+      -> execute tenant-owned exact tool
+      -> settle outcome
+      -> return result plus receipt summary
+
+The generic MCP bridge and a provider-native MCP tool must produce the same billing class for the same operation.
+
+### Multi-step model accounting
+
+For Mastra/AI SDK generate results:
+
+- top-level usage represents the last step;
+- totalUsage aggregates tokens but does not by itself provide a trustworthy total dollar cost;
+- providerMetadata on each step contains that step's OpenRouter usage.cost in the current runtime.
+
+Therefore the scheduler must iterate all result.steps and sum each providerMetadata.openrouter.usage.cost. It must not debit from both a generic usage meter and a pass-through-cost function.
+
+When a step has no usage.cost:
+
+1. Save the OpenRouter generation ID when available.
+2. Query OpenRouter's generation endpoint for the authoritative cost.
+3. Keep the receipt in cost_pending state until reconciled.
+4. Use a current rate snapshot only for authorization and estimates, never silent final settlement.
+5. Pause later occurrences if pending costs cannot be reconciled within the configured window.
+
+## Single debit authority
+
+The main MCP Scraper billing service is the only component allowed to mutate the customer's Credit balance.
+
+| Component | May do | Must not do |
+|---|---|---|
+| Main billing service | Authorize, reserve, settle, refund, expire, and report Credits | Execute schedules or hold provider credentials |
+| Scheduler | Request authorization, emit line items, mirror receipt status | Directly debit a wallet or apply a multiplier |
+| Mastra agent | Use granted tools within a budget | Receive wallet secrets or price itself |
+| Connected-service adapter | Report classified outcomes, records, and bytes | Create plan-dependent rates |
+| Nango | Authenticate provider calls and return results | Own MCP Scraper pricing or Credits |
+| Frontend/SDK/MCP | Display estimates and receipts from server | Calculate authoritative charges locally |
+
+The existing internal memory AI-debit endpoint becomes a compatibility adapter during migration, then routes through the unified billing service. It may not apply an extra 3.0 multiplier to a scheduled_agent_llm event.
+
+## Billing data model
+
+Names are illustrative; final migrations follow existing repository naming.
+
+### billing_events
+
+Required fields:
+
+- id;
+- identity_id;
+- idempotency_key, globally unique;
+- billing_class;
+- source_surface;
+- schedule_id, nullable;
+- occurrence_id, nullable;
+- connection_id, nullable;
+- tool_name, nullable;
+- model and provider, nullable;
+- generation_id, nullable;
+- raw_cost_usd_nanos or an exact decimal representation, nullable;
+- multiplier_basis_points, nullable;
+- input_tokens, output_tokens, cached_input_tokens, reasoning_tokens, nullable;
+- record_count, nullable;
+- byte_count, nullable;
+- amount_mc;
+- rate_revision;
+- status: estimated, authorized, pending, settled, refunded, void, or cost_pending;
+- safe metadata JSON;
+- created_at and settled_at.
+
+Do not store prompts, model responses, provider tokens, OAuth tokens, email bodies, file content, or raw tool arguments.
+
+### credit_reservations
+
+Required fields:
+
+- id;
+- identity_id;
+- idempotency_key, unique;
+- max_amount_mc;
+- consumed_amount_mc;
+- released_amount_mc;
+- status: active, settled, released, expired;
+- expires_at;
+- occurrence_id or request_id;
+- created_at and updated_at.
+
+The balance available for new work excludes active reservations.
+
+### billing_receipts
+
+A receipt groups line items for one direct request, bulk job slice, or scheduled occurrence:
+
+- receipt ID;
+- identity;
+- source and run IDs;
+- rate revision;
+- total mc and public Credits;
+- line-item summaries;
+- reserved, settled, and released amounts;
+- status;
+- timestamps;
+- safe failure/refund reason.
+
+### connection_billing_periods
+
+Required fields:
+
+- identity and connection ID;
+- period start/end;
+- active seconds and prorated minimum;
+- eligible usage mc;
+- shortfall mc;
+- settlement event ID;
+- status;
+- grace deadline.
+
+## Idempotency and retry rules
+
+### Scheduled occurrence key
+
+The canonical occurrence key is:
+
+    schedule:{identityId}:{scheduleId}:{scheduledForIso}
+
+It owns at most one occurrence-base event. The base event is created only after the claim obtains a Credit reservation and execution begins. Worker retries, leases, and resumed slices reference that event.
+
+### Line-item keys
+
+Examples:
+
+    occurrenceKey:base
+    occurrenceKey:llm:{stepId}
+    occurrenceKey:connection:{callId}
+    occurrenceKey:records:{dataset}:{checkpoint}
+    occurrenceKey:bytes:{artifactId}
+    request:{requestId}:tool:{toolCallId}
+
+The main ledger rejects a second settlement for the same key and returns the existing receipt.
+
+### Partial success
+
+- Work completed before a provider or user error remains chargeable.
+- No provider-work line is charged for validation failure before the provider call.
+- Changed records already delivered remain charged if a later page fails.
+- Unused reservation is always released.
+- The 75-Credit platform base is refunded when MCP Scraper itself fails before useful execution.
+- The base is not refunded for invalid customer configuration, expired OAuth after a valid start, provider outage after dispatch, or user-authored agent failure unless support policy grants a courtesy refund.
+- Actual OpenRouter cost remains chargeable whenever the provider incurred it, even if the final workflow outcome fails.
+
+Refund reasons are structured and support-visible.
+
+## Internal billing API
+
+The exact route naming may follow current conventions, but the contract must include:
+
+### Authorize
+
+POST /api/internal/billing/authorizations
+
+Input:
+
+- identity;
+- idempotency key;
+- billing intent;
+- maximum mc;
+- schedule, occurrence, connection, and request references;
+- rate revision requested or latest.
+
+Output:
+
+- authorization ID;
+- opaque billing-context token;
+- reserved mc;
+- rate revision;
+- expiry;
+- remaining wallet and monthly-budget summaries.
+
+### Add line items
+
+POST /api/internal/billing/authorizations/:id/line-items
+
+Input is a batch of classified, idempotent measurements. The server applies rates and multipliers. The worker never sends an authoritative final Credit amount for a class whose price is server-owned.
+
+### Settle
+
+POST /api/internal/billing/authorizations/:id/settle
+
+The server atomically debits actual mc, releases the remainder, and returns the final receipt.
+
+### Void or refund
+
+POST /api/internal/billing/authorizations/:id/void
+
+POST /api/internal/billing/receipts/:id/refunds
+
+Refunds append compensating ledger events. Settled rows are never silently edited.
+
+### Availability policy
+
+- If authorization is unavailable, fail closed before externally costly work.
+- If line-item reporting is temporarily unavailable after work starts, persist a durable outbox record.
+- If settlement is pending, mark cost_pending and pause future work that could increase exposure.
+- Never discard a charge measurement because an HTTP request failed.
+
+## Public API, MCP, and SDK contract
+
+### Existing tools to extend
+
+credits_info adds:
+
+- canonical Credit conversion;
+- rate revision;
+- connection rates;
+- connection minimum policy;
+- schedule base;
+- current model estimate table;
+- actual-cost language;
+- credit expiry and auto-refill summary.
+
+create_scheduled_action adds:
+
+- maxCreditsPerRun;
+- maxCreditsPerMonth;
+- estimatedCreditsPerRun;
+- estimate assumptions and warning range.
+
+get_schedule_status adds:
+
+- month-to-date Credit spend;
+- remaining monthly budget;
+- last authorization and receipt IDs;
+- last itemized Credit summary;
+- paused-for-billing state;
+- pending-cost state.
+
+Connection list/status adds:
+
+- current billing period;
+- eligible usage toward minimum;
+- projected shortfall;
+- billing-paused/grace state;
+- next settlement date.
+
+All connected-service execution tools return a compact billing summary:
+
+- receipt ID;
+- Credits charged;
+- line-item category;
+- pending or settled status.
+
+### New estimator
+
+Add estimate_scheduled_action_cost to the root MCP and corresponding REST/SDK methods.
+
+Inputs:
+
+- schedule mode;
+- frequency and timezone;
+- selected connections and tools;
+- expected records/bytes when known;
+- model and estimated input/output tokens when agentic;
+- memory destination;
+- proposed budgets.
+
+Outputs:
+
+- fixed Credits per occurrence and per month;
+- variable estimate range;
+- model rate snapshot;
+- assumptions;
+- recommended maxCreditsPerRun and maxCreditsPerMonth;
+- warning that actual OpenRouter-reported cost controls settlement.
+
+Estimation is free and creates no reservation.
+
+### Tool-description language
+
+Every billable tool schema includes:
+
+- its billing class;
+- fixed charge if any;
+- variable units;
+- whether failure can incur provider/model cost;
+- link or pointer to credits_info;
+- whether it participates in a connection minimum.
+
+Tool descriptions do not hardcode copied rates by hand. They are generated or injected from the canonical registry.
+
+## Mastra integration
+
+### Scheduled Action Agent
+
+The agent receives:
+
+- exact tenant-owned connection toolsets;
+- a server-side Credit budget;
+- a per-step guard;
+- remaining-budget context as trusted runtime metadata, not user prompt content;
+- no access to billing credentials or wallet mutation tools.
+
+Use onStepFinish or the current equivalent to:
+
+1. capture each model step's OpenRouter usage cost;
+2. append one idempotent line item;
+3. check remaining reserved budget;
+4. stop before the next expensive step when the cap is reached.
+
+The final result uses total tool outcomes plus all step records. Remove the dual meterLlmUsage and billPassThroughCost debit behavior.
+
+### Connection sync worker
+
+The deterministic worker:
+
+- claims the same occurrence key;
+- authorizes its budget;
+- reports one bulk-slice base;
+- reports only changed/delivered records;
+- reports content bytes once per stored artifact;
+- reports memory processing through existing explicit classes;
+- checkpoints before settlement so retry behavior is deterministic.
+
+The worker never invokes a model merely to paginate or transfer content.
+
+## User experience
+
+### Pricing page
+
+Remove the Scheduled Actions add-on card.
+
+Under every paid plan:
+
+> Integrations and scheduled work included. Usage draws from your shared Credit balance.
+
+Add a concise rate drawer containing:
+
+- active connection minimum;
+- read/write/bulk rates;
+- 75-Credit scheduled occurrence;
+- current model estimates in Credits;
+- “actual OpenRouter cost plus 50%” disclosure;
+- expiry, refill, and budget behavior.
+
+### Integrations page
+
+Each card shows:
+
+- connected account count;
+- real callable read/action inventory;
+- action switch;
+- current billing-period usage;
+- minimum progress;
+- next settlement;
+- billing state;
+- a link to import/schedule.
+
+OAuth authorization itself remains free.
+
+### Scheduling page
+
+Keep Scheduling as a prominent feature and navigation destination.
+
+The creation flow shows:
+
+1. frequency and timezone;
+2. selected connection/data/action;
+3. destination or intended result;
+4. estimated fixed and variable Credits;
+5. per-run and monthly caps;
+6. low-balance behavior;
+7. action confirmation/grants;
+8. activate.
+
+The schedule detail view presents an itemized history, not just “one run.”
+
+### Usage page
+
+Group receipts by:
+
+- direct tools;
+- connected-service operations;
+- bulk/import;
+- scheduled occurrences;
+- OpenRouter AI;
+- memory processing;
+- connection minimums;
+- refunds.
+
+Users can filter by connection, schedule, model, provider, and date.
+
+## Scheduled Actions subscription migration
+
+### Customer treatment
+
+1. Stop new scheduler-subscription checkout after the unified billing path is proven.
+2. Identify active, trialing, past-due, and cancelled-at-period-end scheduler subscriptions.
+3. Grant a migration Credit amount equal to the unused prepaid portion, with a customer-friendly floor.
+4. Cancel the separate Stripe scheduler subscription at the correct boundary without interrupting saved schedules.
+5. Remove entitlement and 1,000-run quota gates.
+6. Preserve schedule definitions, timezone, encrypted service key, grants, and history.
+7. Assign default per-run and monthly budgets.
+8. Notify customers before the first Credit-billed occurrence.
+
+Recommended migration grant:
+
+    max(prorated unused $10 converted at plan rate, 75,000 Credits for a fully unused current period)
+
+The exact grant must be tested against Stripe period data and approved before execution.
+
+### Historical billing reconciliation
+
+Build a one-time report joining:
+
+- scheduler mem_usage_ledger scheduled entries;
+- schedule occurrence/run IDs;
+- raw OpenRouter costs and step metadata where retained;
+- main credit-ledger debit events;
+- Stripe scheduler subscription state.
+
+Classify each occurrence:
+
+- correctly debited once;
+- potentially duplicated;
+- wrong multiplier;
+- missing earlier model steps;
+- fail-open/no debit;
+- no usable evidence.
+
+Issue compensating Credits for defensible overcharges. Do not retroactively debit undercharges. Retain an operator receipt and customer-support note.
+
+## Rollout sequence
+
+### Gate 0 — freeze unsafe expansion
+
+- Do not add more callers to the current scheduled AI debit functions.
+- Add telemetry that identifies both debit paths without changing customer balances.
+- Preserve current schedule service while the replacement is built.
+
+### Gate 1 — canonical registry and generated public rates
+
+- Move Credit conversion, billing classes, multipliers, and display estimates into one versioned registry.
+- Generate the MCP skill, public rate API, frontend copy data, and SDK constants.
+- Fix the 100-versus-1,000 mc public discrepancy.
+
+### Gate 2 — unified ledger and reservation API
+
+- Add billing events, reservations, receipts, idempotency constraints, and durable outbox.
+- Wrap existing direct Credit debits behind the service.
+- Keep old endpoints as compatibility adapters.
+
+### Gate 3 — shadow scheduler billing
+
+- Authorize no real reservations yet.
+- Calculate the proposed 75-Credit base and all-step OpenRouter amount in shadow.
+- Compare against current debits and production receipts.
+- Prove multi-step aggregation and single-debit behavior.
+
+### Gate 4 — scheduler cutover
+
+- Enable reservations and settlement for internal/test tenants.
+- Remove the duplicate model debit path.
+- Enable per-run/month caps and cost-pending reconciliation.
+- Expand to production cohorts after receipt review.
+
+### Gate 5 — deterministic sync and direct connection billing
+
+- Route connection_sync and bulk export through the same billing context.
+- Then route generic MCP, REST, and SDK calls.
+- Start active-connection minimums only after a full shadow period and customer notice.
+
+### Gate 6 — commercial migration
+
+- Stop new schedule add-on sales.
+- Grant migration Credits and cancel old Stripe subscriptions.
+- Remove entitlement/quota enforcement.
+- Activate $10 pay-as-you-go refill only after its Stripe and ledger path is complete.
+
+### Gate 7 — full release
+
+- Update frontend, pricing, scheduler copy, MCP schemas, npm package, SDKs, MCPB, terms, support docs, and Vercel production.
+- Run production smoke receipts.
+- Publish a customer explanation and migration FAQ.
+
+## Required tests
+
+### Rate and arithmetic tests
+
+- 100 mc equals 1 public Credit everywhere.
+- All charges use integers and defined ceiling rules.
+- Vendor-cost conversion uses exact rational or decimal arithmetic, never binary floating-point ledger math.
+- OpenRouter raw cost times 1.5 yields the exact expected mc.
+- Plan grants do not alter the number of Credits charged for an identical action.
+- MiniMax display estimates are generated from a timestamped rate snapshot.
+
+### Model accounting tests
+
+- One-step result settles once.
+- Multi-step result sums every step's usage.cost.
+- Top-level final-step metadata alone cannot settle the run.
+- meterLlmUsage and pass-through code cannot each debit the same result.
+- Missing cost becomes cost_pending and reconciles by generation ID.
+- Model failure after incurring cost retains that provider-cost line.
+
+### Scheduling tests
+
+- A new schedule does not require a scheduler subscription.
+- One occurrence receives one 75-Credit base.
+- Retry and lease replay do not duplicate the base.
+- Insufficient Credits before authorization produces a free skip.
+- Per-run cap stops later model/provider work.
+- Monthly cap pauses future occurrences.
+- Unused reservation is released.
+- agent and connection_sync modes both produce receipts.
+- Existing saved schedules survive entitlement migration.
+
+### Connection tests
+
+- OAuth callback is free.
+- Bounded reads and approved writes receive correct server-owned classes.
+- Internal pagination is not separately billed.
+- Unchanged incremental records are not charged.
+- Bytes are charged once per artifact.
+- Direct MCP, REST, SDK, Mastra, and background calls produce parity.
+- Connection minimum offsets eligible usage and prorates correctly.
+- First-connection waiver and billing-pause grace work.
+
+### Ledger tests
+
+- Globally unique idempotency keys prevent double debit.
+- Reservation and settlement are atomic.
+- Refunds append compensating events.
+- Active reservations reduce available balance.
+- Durable outbox survives billing-service interruption.
+- No prompt, result body, OAuth token, or billing secret enters billing metadata.
+
+### Product-surface tests
+
+- Pricing page no longer sells Scheduled Actions separately.
+- Integrations and Scheduling are in desktop and mobile navigation.
+- credits_info, estimator, schedule status, receipts, UI, and SDK agree.
+- Public skill and source rate table have generated parity.
+- npm, MCPB, hosted MCP, REST, and Vercel production expose the same schemas.
+
+## Production proof
+
+Before marking complete, capture:
+
+1. A tiny direct connected-service read and receipt.
+2. A safe approved connected-service write and receipt.
+3. A two-step scheduled agent run proving both OpenRouter step costs settle once.
+4. A deterministic sync with unchanged and changed records proving delta billing.
+5. An insufficient-balance occurrence proving a free skip.
+6. A retry proving no duplicate base or line item.
+7. A failed MCP Scraper execution proving the correct base refund.
+8. A current model-rate endpoint and matching frontend/MCP display.
+9. A migrated scheduler subscription with saved schedule continuity.
+10. Vercel deployment, npm version, MCPB manifest, SDK version, and generated-doc parity.
+
+Receipts must be sanitized. Do not include prompts, connection tokens, API keys, customer content, raw provider arguments, or signed artifact URLs.
+
+## Release blockers
+
+- Any remaining path that directly mutates Credits outside the main billing service.
+- Any duplicate scheduled model debit.
+- Settlement based only on final-step metadata.
+- Fail-open execution that can incur provider cost without a durable pending charge.
+- Missing idempotency across occurrence, tool call, record batch, or artifact.
+- The public 1 Credit equals 1,000 mc mismatch.
+- Separate scheduling entitlement or 1,000-run quota still enforced on any surface.
+- Frontend, hosted MCP, SDK/npm, MCPB, or public docs showing different rates.
+- A fixed token rate described as authoritative rather than an estimate.
+- Connection minimum enabled without customer notice, waiver, and billing-pause behavior.
+- Commercial migration without a reconciliation and compensating-credit policy.
+
+## Decisions locked by this spec
+
+- Scheduled Actions is included and metered, not a separate subscription.
+- Credits are the primary customer-facing unit.
+- OpenRouter actual reported cost is the AI settlement source.
+- scheduled_agent_llm uses a 1.5 multiplier.
+- Model token tables are estimates.
+- The fixed scheduled-occurrence charge is 75 Credits.
+- The launch active-connection minimum is 20,000 Credits per 30 days, offset by eligible connection usage.
+- OAuth is free.
+- Main MCP Scraper owns the only mutable Credit ledger.
+- Every product and execution surface produces a shared receipt.
+- Action authorization remains separate from billing.
+
+## Decisions that may change only through a rate revision
+
+- Exact per-read, per-write, per-record, and per-MiB rates.
+- Connection-minimum amount and waiver duration.
+- Pay-as-you-go Credit grant per dollar.
+- Default per-run and monthly caps.
+- Existing non-scheduler AI multipliers.
+
+Changing one of these does not require a protocol redesign, but it requires a new rate revision, generated-surface parity, tests, and customer notice where applicable.
+
+## Source references
+
+- OpenRouter usage accounting: https://openrouter.ai/docs/cookbook/administration/usage-accounting
+- OpenRouter API usage object: https://openrouter.ai/docs/api/reference/overview
+- Current MiniMax M3 pricing: https://openrouter.ai/minimax/minimax-m3/pricing
+- Current MCP Scraper rate source: src/api/rates.ts
+- Current main AI debit route: src/api/server.ts
+- Current scheduler billing logic: ../mcp-scraper-scheduler/src/mastra/lib/schedule-billing.ts
+- Current scheduled execution path: ../mcp-scraper-scheduler/src/mastra/steps/schedule-execute.ts
+- Current schedule entitlement re-export: ../mcp-scraper-scheduler/src/mastra/db/schedule-entitlements.ts
+- Current scheduler usage re-export: ../mcp-scraper-scheduler/src/mastra/db/usage.ts
+- Installed memory DB entitlement implementation: ../mcp-scraper-scheduler/node_modules/mcpscraper-memory-db/dist/db/schedule-entitlements.js
+- Installed memory DB usage and debit implementation: ../mcp-scraper-scheduler/node_modules/mcpscraper-memory-db/dist/db/usage.js
+
+## Final acceptance statement
+
+The implementation is complete only when a customer can connect a service, schedule or directly invoke work, move data into memory, and receive one understandable Credit receipt whose amount is identical across the live ledger, frontend, MCP, REST, Mastra, and SDK. A local code path, pricing-page change, or scheduler-only implementation does not satisfy this spec.