stripe-experiment-sync 0.0.5 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +113 -42
- package/dist/index.cjs +2615 -1705
- package/dist/index.d.cts +476 -117
- package/dist/index.d.ts +476 -117
- package/dist/index.js +2612 -1704
- package/dist/migrations/0042_convert_to_jsonb_generated_columns.sql +1821 -0
- package/dist/migrations/0043_add_account_id.sql +49 -0
- package/dist/migrations/0044_make_account_id_required.sql +54 -0
- package/dist/migrations/0045_sync_status.sql +18 -0
- package/dist/migrations/0046_sync_status_per_account.sql +91 -0
- package/dist/migrations/0047_api_key_hashes.sql +12 -0
- package/dist/migrations/0048_rename_reserved_columns.sql +1253 -0
- package/dist/migrations/0049_remove_redundant_underscores_from_metadata_tables.sql +68 -0
- package/dist/migrations/0050_rename_id_to_match_stripe_api.sql +239 -0
- package/dist/migrations/0051_remove_webhook_uuid.sql +7 -0
- package/dist/migrations/0052_webhook_url_uniqueness.sql +7 -0
- package/dist/migrations/0053_sync_observability.sql +104 -0
- package/dist/migrations/0054_drop_sync_status.sql +5 -0
- package/package.json +10 -6
package/README.md
CHANGED
|
@@ -7,11 +7,8 @@ A TypeScript library to synchronize Stripe data into a PostgreSQL database, desi
|
|
|
7
7
|
|
|
8
8
|
## Features
|
|
9
9
|
|
|
10
|
-
-
|
|
10
|
+
- Programmatic management of Stripe webhooks for real-time updates
|
|
11
11
|
- Sync Stripe objects (customers, invoices, products, etc.) to your PostgreSQL database
|
|
12
|
-
- Automatic database migrations
|
|
13
|
-
- Express middleware integration with automatic body parsing
|
|
14
|
-
- UUID-based webhook routing for security
|
|
15
12
|
|
|
16
13
|
## Installation
|
|
17
14
|
|
|
@@ -23,45 +20,25 @@ pnpm add stripe-experiment-sync stripe
|
|
|
23
20
|
yarn add stripe-experiment-sync stripe
|
|
24
21
|
```
|
|
25
22
|
|
|
26
|
-
##
|
|
23
|
+
## Usage
|
|
27
24
|
|
|
28
|
-
|
|
25
|
+
```ts
|
|
26
|
+
import { StripeSync } from 'stripe-experiment-sync'
|
|
29
27
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
}
|
|
39
|
-
// Or dynamically determine from request, ngrok, etc.
|
|
40
|
-
return `https://${process.env.REPLIT_DOMAINS?.split(',')[0]}`
|
|
41
|
-
}
|
|
42
|
-
|
|
43
|
-
const stripeAutoSync = new StripeAutoSync({
|
|
44
|
-
databaseUrl: process.env.DATABASE_URL,
|
|
45
|
-
stripeApiKey: process.env.STRIPE_SECRET_KEY,
|
|
46
|
-
baseUrl: getPublicUrl,
|
|
28
|
+
const sync = new StripeSync({
|
|
29
|
+
poolConfig: {
|
|
30
|
+
connectionString: 'postgres://user:pass@host:port/db',
|
|
31
|
+
max: 10, // Maximum number of connections
|
|
32
|
+
},
|
|
33
|
+
stripeSecretKey: 'sk_test_...',
|
|
34
|
+
stripeWebhookSecret: 'whsec_...',
|
|
35
|
+
// logger: <a pino logger>
|
|
47
36
|
})
|
|
48
37
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
await stripeAutoSync.stop() // Cleanup
|
|
38
|
+
// Example: process a Stripe webhook
|
|
39
|
+
await sync.processWebhook(payload, signature)
|
|
52
40
|
```
|
|
53
41
|
|
|
54
|
-
### Configuration Options
|
|
55
|
-
|
|
56
|
-
| Option | Required | Default | Description |
|
|
57
|
-
|--------|----------|---------|-------------|
|
|
58
|
-
| `databaseUrl` | Yes | - | PostgreSQL connection string |
|
|
59
|
-
| `stripeApiKey` | Yes | - | Stripe secret key (sk_...) |
|
|
60
|
-
| `baseUrl` | Yes | - | Function returning your public URL |
|
|
61
|
-
| `webhookPath` | No | `/stripe-webhooks` | Path where webhook handler is mounted |
|
|
62
|
-
| `schema` | No | `stripe` | Database schema name |
|
|
63
|
-
| `stripeApiVersion` | No | `2020-08-27` | Stripe API version |
|
|
64
|
-
|
|
65
42
|
## Low-Level API (Advanced)
|
|
66
43
|
|
|
67
44
|
For more control, you can use the `StripeSync` class directly:
|
|
@@ -83,12 +60,50 @@ const sync = new StripeSync({
|
|
|
83
60
|
await sync.processWebhook(payload, signature)
|
|
84
61
|
```
|
|
85
62
|
|
|
63
|
+
### Processing Webhooks
|
|
64
|
+
|
|
65
|
+
The `processWebhook` method validates and processes Stripe webhook events:
|
|
66
|
+
|
|
67
|
+
```typescript
|
|
68
|
+
// Process a webhook event with signature validation
|
|
69
|
+
await sync.processWebhook(payload, signature)
|
|
70
|
+
|
|
71
|
+
// Or process an event directly (no signature validation):
|
|
72
|
+
await sync.processEvent(event)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Managed Webhook Endpoints
|
|
76
|
+
|
|
77
|
+
The library provides methods to create and manage webhook endpoints:
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
// Create or reuse an existing webhook endpoint for a URL
|
|
81
|
+
const webhook = await sync.findOrCreateManagedWebhook('https://example.com/stripe-webhooks', {
|
|
82
|
+
enabled_events: ['*'], // or specific events like ['customer.created', 'invoice.paid']
|
|
83
|
+
description: 'My app webhook',
|
|
84
|
+
})
|
|
85
|
+
|
|
86
|
+
// Create a new webhook endpoint (always creates new)
|
|
87
|
+
const webhook = await sync.createManagedWebhook('https://example.com/stripe-webhooks', {
|
|
88
|
+
enabled_events: ['customer.created', 'customer.updated'],
|
|
89
|
+
})
|
|
90
|
+
|
|
91
|
+
// Get a managed webhook by ID
|
|
92
|
+
const webhook = await sync.getManagedWebhook('we_xxx')
|
|
93
|
+
|
|
94
|
+
// Delete a managed webhook
|
|
95
|
+
await sync.deleteManagedWebhook('we_xxx')
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
**Note:** The library automatically manages webhook endpoints for you. When you call `findOrCreateManagedWebhook()` with a URL, it will reuse an existing webhook if one is found in the database, or create a new one if needed. Old or orphaned webhooks from this package are automatically cleaned up.
|
|
99
|
+
|
|
100
|
+
**Race Condition Protection:** The library uses PostgreSQL advisory locks to prevent race conditions when multiple instances call `findOrCreateManagedWebhook()` concurrently for the same URL. A unique constraint on `(url, account_id)` provides an additional safety net at the database level.
|
|
101
|
+
|
|
86
102
|
## Configuration
|
|
87
103
|
|
|
88
104
|
| Option | Type | Description |
|
|
89
105
|
| ------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
90
106
|
| `databaseUrl` | string | **Deprecated:** Use `poolConfig` with a connection string instead. |
|
|
91
|
-
| `schema` | string | Database schema name (default: `stripe`) |
|
|
92
107
|
| `stripeSecretKey` | string | Stripe secret key |
|
|
93
108
|
| `stripeWebhookSecret` | string | Stripe webhook signing secret |
|
|
94
109
|
| `stripeApiVersion` | string | Stripe API version (default: `2020-08-27`) |
|
|
@@ -103,16 +118,70 @@ await sync.processWebhook(payload, signature)
|
|
|
103
118
|
|
|
104
119
|
The library will create and manage a `stripe` schema in your PostgreSQL database, with tables for all supported Stripe objects (products, customers, invoices, etc.).
|
|
105
120
|
|
|
121
|
+
> **Important:** The library uses a fixed schema name of `stripe`. This cannot be configured as the SQL migrations hardcode this schema name.
|
|
122
|
+
|
|
123
|
+
> **Note:** Fields and tables prefixed with an underscore (`_`) are reserved for internal metadata managed by the sync engine and should not be modified directly. These include fields like `_account`, `_cursor`, `_synced_at`, and tables like `_migrations`, `_accounts`, `_sync_run`, and `_sync_obj_run`.
|
|
124
|
+
|
|
106
125
|
### Migrations
|
|
107
126
|
|
|
108
127
|
Migrations are included in the `db/migrations` directory. You can run them using the provided `runMigrations` function:
|
|
109
128
|
|
|
110
129
|
```ts
|
|
111
|
-
import { runMigrations } from '
|
|
130
|
+
import { runMigrations } from 'stripe-experiment-sync'
|
|
112
131
|
|
|
113
132
|
await runMigrations({ databaseUrl: 'postgres://...' })
|
|
114
133
|
```
|
|
115
134
|
|
|
135
|
+
## Account Management
|
|
136
|
+
|
|
137
|
+
### Getting Current Account
|
|
138
|
+
|
|
139
|
+
Retrieve the currently authenticated Stripe account:
|
|
140
|
+
|
|
141
|
+
```ts
|
|
142
|
+
const account = await sync.getCurrentAccount()
|
|
143
|
+
console.log(account.id) // e.g., "acct_xxx"
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
### Listing Synced Accounts
|
|
147
|
+
|
|
148
|
+
Get all Stripe accounts that have been synced to the database:
|
|
149
|
+
|
|
150
|
+
```ts
|
|
151
|
+
const accounts = await sync.getAllSyncedAccounts()
|
|
152
|
+
// Returns array of Stripe account objects from database
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
### Deleting Synced Account Data
|
|
156
|
+
|
|
157
|
+
**⚠️ DANGEROUS:** Delete all synced data for a specific Stripe account from the database. This operation cannot be undone!
|
|
158
|
+
|
|
159
|
+
```ts
|
|
160
|
+
// Preview what will be deleted (dry-run mode)
|
|
161
|
+
const preview = await sync.dangerouslyDeleteSyncedAccountData('acct_xxx', {
|
|
162
|
+
dryRun: true,
|
|
163
|
+
useTransaction: true,
|
|
164
|
+
})
|
|
165
|
+
console.log(preview.deletedRecordCounts) // Shows count per table
|
|
166
|
+
|
|
167
|
+
// Actually delete the data
|
|
168
|
+
const result = await sync.dangerouslyDeleteSyncedAccountData('acct_xxx', {
|
|
169
|
+
dryRun: false, // default
|
|
170
|
+
useTransaction: true, // default - wraps deletion in transaction
|
|
171
|
+
})
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Options:
|
|
175
|
+
|
|
176
|
+
- `dryRun` (default: `false`): If true, only counts records without deleting
|
|
177
|
+
- `useTransaction` (default: `true`): If true, wraps all deletions in a database transaction for atomicity
|
|
178
|
+
|
|
179
|
+
The method returns:
|
|
180
|
+
|
|
181
|
+
- `deletedAccountId`: The account ID that was deleted
|
|
182
|
+
- `deletedRecordCounts`: Object mapping table names to number of records deleted
|
|
183
|
+
- `warnings`: Array of warning messages (e.g., if you're deleting your cached account)
|
|
184
|
+
|
|
116
185
|
## Backfilling and Syncing Data
|
|
117
186
|
|
|
118
187
|
### Syncing a Single Entity
|
|
@@ -125,12 +194,12 @@ await sync.syncSingleEntity('cus_12345')
|
|
|
125
194
|
|
|
126
195
|
The entity type is detected automatically based on the Stripe ID prefix (e.g., `cus_` for customer, `prod_` for product). `ent_` is not supported at the moment.
|
|
127
196
|
|
|
128
|
-
###
|
|
197
|
+
### Syncing Data
|
|
129
198
|
|
|
130
|
-
To
|
|
199
|
+
To sync Stripe data (e.g., all products created after a certain date), use the `processUntilDone` method:
|
|
131
200
|
|
|
132
201
|
```ts
|
|
133
|
-
await sync.
|
|
202
|
+
await sync.processUntilDone({
|
|
134
203
|
object: 'product',
|
|
135
204
|
created: { gte: 1643872333 }, // Unix timestamp
|
|
136
205
|
})
|
|
@@ -139,5 +208,7 @@ await sync.syncBackfill({
|
|
|
139
208
|
- `object` can be one of: `all`, `charge`, `customer`, `dispute`, `invoice`, `payment_method`, `payment_intent`, `plan`, `price`, `product`, `setup_intent`, `subscription`.
|
|
140
209
|
- `created` is a Stripe RangeQueryParam and supports `gt`, `gte`, `lt`, `lte`.
|
|
141
210
|
|
|
211
|
+
The sync engine automatically tracks per-account cursors in the `_sync_run` and `_sync_obj_run` tables. When you call sync methods without an explicit `created` filter, they will automatically resume from the last synced position for that account and resource. This enables incremental syncing that can resume after interruptions.
|
|
212
|
+
|
|
142
213
|
> **Note:**
|
|
143
214
|
> For large Stripe accounts (more than 10,000 objects), it is recommended to write a script that loops through each day and sets the `created` date filters to the start and end of day. This avoids timeouts and memory issues when syncing large datasets.
|