@rdlabo/workers-hono-kit 0.9.6 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -14
- package/dist/aws/secrets-manager.d.ts +2 -2
- package/dist/aws/secrets-manager.js +1 -1
- package/dist/db/connection.d.ts +8 -9
- package/dist/db/connection.js +14 -22
- package/dist/db/database.d.ts +14 -13
- package/dist/db/database.js +10 -11
- package/dist/firebase/identity-toolkit.d.ts +2 -2
- package/dist/firebase/identity-toolkit.js +8 -2
- package/dist/index.d.ts +2 -2
- package/dist/index.js +1 -1
- package/dist/queue/consumer.d.ts +27 -7
- package/dist/queue/consumer.js +24 -8
- package/dist/queue/error-handler.d.ts +4 -2
- package/dist/queue/error-handler.js +7 -2
- package/dist/queue/send.d.ts +7 -5
- package/dist/queue/send.js +7 -5
- package/package.json +1 -1
- package/scripts/check-subrequest-fanout.mjs +2 -1
package/README.md
CHANGED
|
@@ -2,10 +2,10 @@
|
|
|
2
2
|
|
|
3
3
|
Infrastructure toolkit for building APIs on [Hono](https://hono.dev) + [Cloudflare Workers](https://workers.cloudflare.com).
|
|
4
4
|
|
|
5
|
-
It provides
|
|
5
|
+
It provides Workers-oriented building blocks for a NestJS-style API, plus middleware for common HTTP response concerns:
|
|
6
6
|
|
|
7
7
|
- **Firebase ID-token verification** on Workers via [`jose`](https://github.com/panva/jose) (RS256 against Google's securetoken JWKS), with optional Identity Toolkit REST for `getUser` / `deleteUser`.
|
|
8
|
-
- **AWS Secrets Manager / STS AssumeRole / CloudFront signed URLs** via SigV4-signed `fetch` ([`aws4fetch`](https://github.com/mhart/aws4fetch)) or Web Crypto
|
|
8
|
+
- **AWS Secrets Manager / STS AssumeRole / CloudFront signed URLs** via focused SigV4-signed `fetch` ([`aws4fetch`](https://github.com/mhart/aws4fetch)) or Web Crypto, avoiding a broad SDK dependency when only a few AWS APIs are needed.
|
|
9
9
|
- **Middleware**: `finalizeResponse` (weak ETag via `hono/etag`), `validate` (NestJS `ValidationPipe`-shaped 400), and zod number-coercion helpers.
|
|
10
10
|
- **Standard API errors**: `createHttpErrorHandler` / `notFoundHandler` / `HttpStatus`.
|
|
11
11
|
- **Deadlock retry** (`ER_LOCK_DEADLOCK` exponential backoff) and an optional **MySQL data layer** (`@rdlabo/workers-hono-kit/db`) for Hyperdrive + Drizzle.
|
|
@@ -104,9 +104,19 @@ npm install ai ai-gateway-provider # createAiGatewayProvider
|
|
|
104
104
|
| `verifyAppleReceipt(receipt, opts)` / `classifyAppleRenewal(verify, now)` / `AppleRenewalClassification` / `AppleRenewalState` / `AppleVerifyReceiptResponse` / `ApplePendingRenewalInfo` / `AppleLatestReceiptInfo` | Verify an App Store receipt (production → sandbox fallback; inject `password` / `fetchImpl`) and classify it into `billing_retry` / `lapsed` / `active` / `unknown` plus the raw fields used (`statusCode` / `billingRetryStatus` / `autoRenewStatus`, latest `original_transaction_id` / `expires_date_ms`). |
|
|
105
105
|
| `googleAccessToken(creds, fetch?)` / `getGoogleSubscription(opts)` / `classifyGoogleSubscription(purchase, now)` / `GoogleSubscriptionClassification` / `GoogleSubscriptionState` / `GoogleSubscriptionPurchase` / `GoogleOAuthCredentials` | Exchange a refresh token for an Android Publisher access token (throws on `invalid_grant`), fetch a subscription purchase, and classify it into `canceled` / `gone` / `active` / `unknown` plus raw `statusCode` / `cancelReason`. |
|
|
106
106
|
| `sendInChunks(queue, messages, options?)` / `QueueLike` / `QueueSendMessage` | Send queue messages in bounded chunks to stay under the Workers subrequest cap per invocation. `options.chunkSize` sets the per-batch size (defaults to and is capped at 100). |
|
|
107
|
-
| `processBatch(batch, handler, options?)` / `MessageBatchLike` / `QueueMessageLike` / `ProcessBatchOptions` / `ProcessBatchResult` | Process a queue batch with bounded concurrency
|
|
108
|
-
| `createQueueErrorHandler(options)` / `CreateQueueErrorHandlerOptions` | Factory for `processBatch`'s `onError`: logs every failure; optional Sentry capture with queue/message context; optional `maxRetries` gate (report only on final attempt). |
|
|
109
|
-
| `ExecutionContextLike` | Minimal `waitUntil`-only Workers execution context shape
|
|
107
|
+
| `processBatch(batch, handler, options?)` / `isNonRetryableQueueError(error)` / `NonRetryableQueueErrorLike` / `MessageBatchLike` / `QueueMessageLike` / `ProcessBatchOptions` / `ProcessBatchResult` | Process a queue batch with bounded concurrency. Errors explicitly tagged with `queueDisposition: 'discard'` are reported and acked as permanent failures; all other errors are retried. |
|
|
108
|
+
| `createQueueErrorHandler(options)` / `CreateQueueErrorHandlerOptions` | Factory for `processBatch`'s `onError`: logs every failure; optional Sentry capture with queue/message context; optional `maxRetries` gate (report only on final attempt, except permanent failures which are reported immediately). |
|
|
109
|
+
| `ExecutionContextLike` | Minimal `waitUntil`-only Workers execution context shape used by lifecycle-compatible APIs and deferred work helpers. |
|
|
110
|
+
|
|
111
|
+
Permanent Queue failures must opt in with the Queue-specific marker; unrelated `retryable` fields are ignored:
|
|
112
|
+
|
|
113
|
+
```ts
|
|
114
|
+
import type { NonRetryableQueueErrorLike } from '@rdlabo/workers-hono-kit';
|
|
115
|
+
|
|
116
|
+
class CustomerLinkMissingError extends Error implements NonRetryableQueueErrorLike {
|
|
117
|
+
readonly queueDisposition = 'discard' as const;
|
|
118
|
+
}
|
|
119
|
+
```
|
|
110
120
|
|
|
111
121
|
### Data layer — `@rdlabo/workers-hono-kit/db`
|
|
112
122
|
|
|
@@ -114,12 +124,12 @@ Requires the `drizzle-orm` and `mysql2` peers. Reads run against a replica via r
|
|
|
114
124
|
|
|
115
125
|
| Export | Description |
|
|
116
126
|
| --- | --- |
|
|
117
|
-
| `createHyperdriveDatabase(options)` | `DisposableDatabase` that lazily opens primary/replica connections from Hyperdrive bindings per request; `dispose()`
|
|
127
|
+
| `createHyperdriveDatabase(options)` | `DisposableDatabase` that lazily opens primary/replica connections from Hyperdrive bindings per request. Workers cleans them up at invocation end; the legacy `dispose()` is a no-op. |
|
|
118
128
|
| `createMysqlDatabase(options)` | Assemble a `Database` from an already-connected Drizzle ORM + replica `QueryRunner`. |
|
|
119
129
|
| `databaseFrom(orm, replica)` | Build a `Database` from an existing Drizzle instance + replica handle. |
|
|
120
130
|
| `Database` / `DisposableDatabase` / `QueryRunner` / `TxOf` | The `read` / `write` / `transaction` API and its supporting types. |
|
|
121
|
-
| `hyperdriveConnectionOptions(hyperdrive, overrides?)` / `HyperdriveLike` / `ExecutionContextLike` | Build mysql2 `createConnection` options from a Hyperdrive binding (`disableEval`, `decimalNumbers`, `timezone '+09:00'` by default). `
|
|
122
|
-
| `withMysqlConnections(...)` | Open primary/replica connections
|
|
131
|
+
| `hyperdriveConnectionOptions(hyperdrive, overrides?)` / `HyperdriveLike` / `ExecutionContextLike` | Build mysql2 `createConnection` options from a Hyperdrive binding (`disableEval`, `decimalNumbers`, `timezone '+09:00'` by default). `timezone` controls mysql2's JavaScript `Date` conversion; it does not change the MySQL session timezone. |
|
|
132
|
+
| `withMysqlConnections(...)` | Open primary/replica connections in parallel and run a function. Workers cleans them up at invocation end. |
|
|
123
133
|
| `retryWhenDeadlock(fn, retries?, delay?)` | Same deadlock-retry helper as the root export. |
|
|
124
134
|
| `insertIdOf` / `affectedRowsOf` / `insertedIdsOf` / `DzWriteResult` | Extract `insertId` / `affectedRows` (and derive contiguous bulk-insert ids) from a mysql2 write result. |
|
|
125
135
|
| `toJstDate` / `jstTimestampParams` / `jstDatetimeParams` / `jstDateParams` | JST date/time normalization params (advanced use). |
|
|
@@ -588,12 +598,8 @@ const db = createHyperdriveDatabase({
|
|
|
588
598
|
createOrm: (conn) => drizzle(conn, { ...DRIZZLE_ORM_OPTIONS, schema }),
|
|
589
599
|
});
|
|
590
600
|
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
await db.write((dz) => dz.insert(users).values({ name })); // primary, deadlock-retried
|
|
594
|
-
} finally {
|
|
595
|
-
await db.dispose();
|
|
596
|
-
}
|
|
601
|
+
const rows = await db.read('SELECT * FROM users WHERE id = ?', [id]); // replica, raw SQL
|
|
602
|
+
await db.write((dz) => dz.insert(users).values({ name })); // primary, deadlock-retried
|
|
597
603
|
```
|
|
598
604
|
|
|
599
605
|
### KV cache
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
* AWS credentials used to sign Secrets Manager requests.
|
|
3
3
|
*
|
|
4
4
|
* @remarks
|
|
5
|
-
* Cloudflare Workers
|
|
5
|
+
* Cloudflare Workers do not provide the AWS default credential chain, so AWS credentials are supplied
|
|
6
6
|
* as Workers secrets and used to produce a SigV4 signature.
|
|
7
7
|
*/
|
|
8
8
|
export interface AwsSecretsOptions {
|
|
@@ -19,7 +19,7 @@ export interface AwsSecretsOptions {
|
|
|
19
19
|
* Fetch and parse a secret from AWS Secrets Manager, caching the result per isolate.
|
|
20
20
|
*
|
|
21
21
|
* Issues a `GetSecretValue` call to Secrets Manager via a SigV4-signed `fetch` (using aws4fetch), with no
|
|
22
|
-
* AWS SDK involved. The parsed `SecretString` is cached per isolate keyed by region, access key ID, and
|
|
22
|
+
* broad AWS SDK dependency involved. The parsed `SecretString` is cached per isolate keyed by region, access key ID, and
|
|
23
23
|
* secret ID; concurrent first-time callers share one in-flight request, and a rejected fetch clears the
|
|
24
24
|
* cache entry so the next call retries.
|
|
25
25
|
*
|
|
@@ -15,7 +15,7 @@ let cache = null;
|
|
|
15
15
|
* Fetch and parse a secret from AWS Secrets Manager, caching the result per isolate.
|
|
16
16
|
*
|
|
17
17
|
* Issues a `GetSecretValue` call to Secrets Manager via a SigV4-signed `fetch` (using aws4fetch), with no
|
|
18
|
-
* AWS SDK involved. The parsed `SecretString` is cached per isolate keyed by region, access key ID, and
|
|
18
|
+
* broad AWS SDK dependency involved. The parsed `SecretString` is cached per isolate keyed by region, access key ID, and
|
|
19
19
|
* secret ID; concurrent first-time callers share one in-flight request, and a rejected fetch clears the
|
|
20
20
|
* cache entry so the next call retries.
|
|
21
21
|
*
|
package/dist/db/connection.d.ts
CHANGED
|
@@ -31,11 +31,9 @@ export interface HyperdriveLike {
|
|
|
31
31
|
* - `decimalNumbers: true` — return `DECIMAL`/`NEWDECIMAL` columns as JS `number` rather than
|
|
32
32
|
* strings, so raw-SQL reads and Drizzle's inferred types align on a single numeric domain type.
|
|
33
33
|
* This assumes no column's precision exceeds the JS safe-integer range.
|
|
34
|
-
* - `timezone: '+09:00'` —
|
|
35
|
-
*
|
|
36
|
-
*
|
|
37
|
-
* value differs, which is invisible to the application). Non-JST deployments can override this
|
|
38
|
-
* via `extra: { timezone: '...' }`.
|
|
34
|
+
* - `timezone: '+09:00'` — interpret and serialize JavaScript `Date` values as JST. This is a
|
|
35
|
+
* mysql2 client-side conversion option; it does not issue `SET time_zone` or change the MySQL
|
|
36
|
+
* session timezone. Non-JST deployments can override it via `extra: { timezone: '...' }`.
|
|
39
37
|
*
|
|
40
38
|
* @param hyperdrive - the Hyperdrive binding to derive connection fields from.
|
|
41
39
|
* @param extra - additional mysql2 options merged last, overriding the defaults above.
|
|
@@ -43,14 +41,15 @@ export interface HyperdriveLike {
|
|
|
43
41
|
*/
|
|
44
42
|
export declare function hyperdriveConnectionOptions(hyperdrive: HyperdriveLike, extra?: Record<string, unknown>): Record<string, unknown>;
|
|
45
43
|
/**
|
|
46
|
-
* Open primary and replica connections
|
|
44
|
+
* Open primary and replica connections in parallel and run `fn` with them.
|
|
47
45
|
*
|
|
48
|
-
*
|
|
49
|
-
* `
|
|
46
|
+
* Cloudflare Workers automatically cleans up connections created during an invocation. Calling
|
|
47
|
+
* `Connection.end()` is unnecessary and can race work registered with `waitUntil`. The `ctx`
|
|
48
|
+
* parameter remains for API compatibility and is intentionally not used for connection teardown.
|
|
50
49
|
*
|
|
51
50
|
* @typeParam T - resolved value produced by `fn`.
|
|
52
51
|
* @param hyperdrives - the primary and replica Hyperdrive bindings to connect to.
|
|
53
|
-
* @param ctx - the execution context
|
|
52
|
+
* @param ctx - the request execution context, retained for API compatibility.
|
|
54
53
|
* @param fn - callback invoked with the open `primary` and `replica` connections.
|
|
55
54
|
* @param connectionOptions - extra mysql2 options forwarded to {@link hyperdriveConnectionOptions}.
|
|
56
55
|
* @returns the value resolved by `fn`.
|
package/dist/db/connection.js
CHANGED
|
@@ -11,11 +11,9 @@ import { MYSQL_TIMEZONE } from './jst.js';
|
|
|
11
11
|
* - `decimalNumbers: true` — return `DECIMAL`/`NEWDECIMAL` columns as JS `number` rather than
|
|
12
12
|
* strings, so raw-SQL reads and Drizzle's inferred types align on a single numeric domain type.
|
|
13
13
|
* This assumes no column's precision exceeds the JS safe-integer range.
|
|
14
|
-
* - `timezone: '+09:00'` —
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
* value differs, which is invisible to the application). Non-JST deployments can override this
|
|
18
|
-
* via `extra: { timezone: '...' }`.
|
|
14
|
+
* - `timezone: '+09:00'` — interpret and serialize JavaScript `Date` values as JST. This is a
|
|
15
|
+
* mysql2 client-side conversion option; it does not issue `SET time_zone` or change the MySQL
|
|
16
|
+
* session timezone. Non-JST deployments can override it via `extra: { timezone: '...' }`.
|
|
19
17
|
*
|
|
20
18
|
* @param hyperdrive - the Hyperdrive binding to derive connection fields from.
|
|
21
19
|
* @param extra - additional mysql2 options merged last, overriding the defaults above.
|
|
@@ -35,14 +33,15 @@ export function hyperdriveConnectionOptions(hyperdrive, extra) {
|
|
|
35
33
|
};
|
|
36
34
|
}
|
|
37
35
|
/**
|
|
38
|
-
* Open primary and replica connections
|
|
36
|
+
* Open primary and replica connections in parallel and run `fn` with them.
|
|
39
37
|
*
|
|
40
|
-
*
|
|
41
|
-
* `
|
|
38
|
+
* Cloudflare Workers automatically cleans up connections created during an invocation. Calling
|
|
39
|
+
* `Connection.end()` is unnecessary and can race work registered with `waitUntil`. The `ctx`
|
|
40
|
+
* parameter remains for API compatibility and is intentionally not used for connection teardown.
|
|
42
41
|
*
|
|
43
42
|
* @typeParam T - resolved value produced by `fn`.
|
|
44
43
|
* @param hyperdrives - the primary and replica Hyperdrive bindings to connect to.
|
|
45
|
-
* @param ctx - the execution context
|
|
44
|
+
* @param ctx - the request execution context, retained for API compatibility.
|
|
46
45
|
* @param fn - callback invoked with the open `primary` and `replica` connections.
|
|
47
46
|
* @param connectionOptions - extra mysql2 options forwarded to {@link hyperdriveConnectionOptions}.
|
|
48
47
|
* @returns the value resolved by `fn`.
|
|
@@ -59,17 +58,10 @@ export function hyperdriveConnectionOptions(hyperdrive, extra) {
|
|
|
59
58
|
* ```
|
|
60
59
|
*/
|
|
61
60
|
export async function withMysqlConnections(hyperdrives, ctx, fn, connectionOptions) {
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
}
|
|
69
|
-
finally {
|
|
70
|
-
const closing = [primary, replica].filter((c) => c !== undefined).map((c) => c.end());
|
|
71
|
-
if (closing.length > 0) {
|
|
72
|
-
ctx.waitUntil(Promise.allSettled(closing));
|
|
73
|
-
}
|
|
74
|
-
}
|
|
61
|
+
void ctx;
|
|
62
|
+
const [primary, replica] = await Promise.all([
|
|
63
|
+
createConnection(hyperdriveConnectionOptions(hyperdrives.primary, connectionOptions)),
|
|
64
|
+
createConnection(hyperdriveConnectionOptions(hyperdrives.replica, connectionOptions)),
|
|
65
|
+
]);
|
|
66
|
+
return fn({ primary, replica });
|
|
75
67
|
}
|
package/dist/db/database.d.ts
CHANGED
|
@@ -79,19 +79,22 @@ export interface Database<TDrizzle, TTx = TxOf<TDrizzle>> {
|
|
|
79
79
|
transaction<T>(fn: (tx: TTx) => Promise<T>): Promise<T>;
|
|
80
80
|
}
|
|
81
81
|
/**
|
|
82
|
-
* A {@link Database} that
|
|
82
|
+
* A {@link Database} that opens its own connections.
|
|
83
83
|
*
|
|
84
84
|
* @remarks
|
|
85
|
-
* Used by
|
|
85
|
+
* Used by variants that open connections internally. Lifecycle behavior depends on the backing
|
|
86
|
+
* implementation: pool-backed databases close their pool, while Hyperdrive-backed databases leave
|
|
87
|
+
* invocation-scoped connection cleanup to the Workers runtime.
|
|
86
88
|
*
|
|
87
89
|
* @typeParam TDrizzle - the consumer's Drizzle ORM type.
|
|
88
90
|
* @typeParam TTx - the transaction-handle type, inferred from `TDrizzle` by default.
|
|
89
91
|
*/
|
|
90
92
|
export interface DisposableDatabase<TDrizzle, TTx = TxOf<TDrizzle>> extends Database<TDrizzle, TTx> {
|
|
91
93
|
/**
|
|
92
|
-
*
|
|
94
|
+
* Release resources owned by the implementation. Hyperdrive-backed databases keep this method as
|
|
95
|
+
* a compatibility no-op; pool-backed databases use it to close their pool.
|
|
93
96
|
*
|
|
94
|
-
* @returns a promise that
|
|
97
|
+
* @returns a promise that resolves after implementation-specific cleanup.
|
|
95
98
|
*/
|
|
96
99
|
dispose(): Promise<void>;
|
|
97
100
|
}
|
|
@@ -155,13 +158,15 @@ export interface CreateHyperdriveDatabaseOptions<TDrizzle> {
|
|
|
155
158
|
* Create a {@link DisposableDatabase} that lazily opens its connections from Hyperdrive bindings.
|
|
156
159
|
*
|
|
157
160
|
* @remarks
|
|
158
|
-
* Construct one per request and
|
|
159
|
-
*
|
|
160
|
-
*
|
|
161
|
+
* Construct one per request. Connections and the ORM are created on first use and reused for the
|
|
162
|
+
* lifetime of the instance; the read/write/transaction surface is identical to
|
|
163
|
+
* {@link createMysqlDatabase}. Workers automatically cleans up connections at the end of the
|
|
164
|
+
* invocation, so callers do not need to close them manually.
|
|
161
165
|
*
|
|
162
166
|
* @typeParam TDrizzle - the consumer's Drizzle ORM type.
|
|
163
167
|
* @param options - the primary/replica Hyperdrive bindings, the ORM factory, and connection options.
|
|
164
|
-
* @returns a {@link DisposableDatabase}
|
|
168
|
+
* @returns a {@link DisposableDatabase} whose compatibility `dispose()` method is a no-op; Workers
|
|
169
|
+
* cleans up its invocation-scoped connections automatically.
|
|
165
170
|
* @example
|
|
166
171
|
* ```ts
|
|
167
172
|
* const db = createHyperdriveDatabase({
|
|
@@ -169,11 +174,7 @@ export interface CreateHyperdriveDatabaseOptions<TDrizzle> {
|
|
|
169
174
|
* replicaHyperdrive: env.REPLICA,
|
|
170
175
|
* createOrm: (primary) => drizzle(primary, { schema, ...DRIZZLE_ORM_OPTIONS }),
|
|
171
176
|
* });
|
|
172
|
-
*
|
|
173
|
-
* await db.write((dz) => dz.insert(users).values(user));
|
|
174
|
-
* } finally {
|
|
175
|
-
* await db.dispose();
|
|
176
|
-
* }
|
|
177
|
+
* await db.write((dz) => dz.insert(users).values(user));
|
|
177
178
|
* ```
|
|
178
179
|
*/
|
|
179
180
|
export declare function createHyperdriveDatabase<TDrizzle>(options: CreateHyperdriveDatabaseOptions<TDrizzle>): DisposableDatabase<TDrizzle>;
|
package/dist/db/database.js
CHANGED
|
@@ -27,13 +27,15 @@ export function createMysqlDatabase(options) {
|
|
|
27
27
|
* Create a {@link DisposableDatabase} that lazily opens its connections from Hyperdrive bindings.
|
|
28
28
|
*
|
|
29
29
|
* @remarks
|
|
30
|
-
* Construct one per request and
|
|
31
|
-
*
|
|
32
|
-
*
|
|
30
|
+
* Construct one per request. Connections and the ORM are created on first use and reused for the
|
|
31
|
+
* lifetime of the instance; the read/write/transaction surface is identical to
|
|
32
|
+
* {@link createMysqlDatabase}. Workers automatically cleans up connections at the end of the
|
|
33
|
+
* invocation, so callers do not need to close them manually.
|
|
33
34
|
*
|
|
34
35
|
* @typeParam TDrizzle - the consumer's Drizzle ORM type.
|
|
35
36
|
* @param options - the primary/replica Hyperdrive bindings, the ORM factory, and connection options.
|
|
36
|
-
* @returns a {@link DisposableDatabase}
|
|
37
|
+
* @returns a {@link DisposableDatabase} whose compatibility `dispose()` method is a no-op; Workers
|
|
38
|
+
* cleans up its invocation-scoped connections automatically.
|
|
37
39
|
* @example
|
|
38
40
|
* ```ts
|
|
39
41
|
* const db = createHyperdriveDatabase({
|
|
@@ -41,11 +43,7 @@ export function createMysqlDatabase(options) {
|
|
|
41
43
|
* replicaHyperdrive: env.REPLICA,
|
|
42
44
|
* createOrm: (primary) => drizzle(primary, { schema, ...DRIZZLE_ORM_OPTIONS }),
|
|
43
45
|
* });
|
|
44
|
-
*
|
|
45
|
-
* await db.write((dz) => dz.insert(users).values(user));
|
|
46
|
-
* } finally {
|
|
47
|
-
* await db.dispose();
|
|
48
|
-
* }
|
|
46
|
+
* await db.write((dz) => dz.insert(users).values(user));
|
|
49
47
|
* ```
|
|
50
48
|
*/
|
|
51
49
|
export function createHyperdriveDatabase(options) {
|
|
@@ -71,8 +69,9 @@ export function createHyperdriveDatabase(options) {
|
|
|
71
69
|
const dz = (await ormFor());
|
|
72
70
|
return retryWhenDeadlock(() => dz.transaction(fn));
|
|
73
71
|
},
|
|
74
|
-
|
|
75
|
-
|
|
72
|
+
/** @deprecated Workers cleans up invocation-scoped connections automatically. */
|
|
73
|
+
dispose() {
|
|
74
|
+
return Promise.resolve();
|
|
76
75
|
},
|
|
77
76
|
};
|
|
78
77
|
}
|
|
@@ -16,8 +16,8 @@ export interface ServiceAccount {
|
|
|
16
16
|
* Minimal Google Identity Toolkit REST client for the user-management operations that token
|
|
17
17
|
* verification does not cover: `accounts:lookup` (getUser) and `accounts:delete` (deleteUser).
|
|
18
18
|
*
|
|
19
|
-
* This
|
|
20
|
-
*
|
|
19
|
+
* This is a focused REST alternative for projects that only need a small subset of Firebase Auth
|
|
20
|
+
* administration and do not want to bundle the broader `firebase-admin` SDK.
|
|
21
21
|
*
|
|
22
22
|
* @remarks
|
|
23
23
|
* Authentication follows the JWT-bearer flow: a JWT assertion is signed with the service
|
|
@@ -11,8 +11,8 @@ const LOOKUP_CHUNK_SIZE = 100;
|
|
|
11
11
|
* Minimal Google Identity Toolkit REST client for the user-management operations that token
|
|
12
12
|
* verification does not cover: `accounts:lookup` (getUser) and `accounts:delete` (deleteUser).
|
|
13
13
|
*
|
|
14
|
-
* This
|
|
15
|
-
*
|
|
14
|
+
* This is a focused REST alternative for projects that only need a small subset of Firebase Auth
|
|
15
|
+
* administration and do not want to bundle the broader `firebase-admin` SDK.
|
|
16
16
|
*
|
|
17
17
|
* @remarks
|
|
18
18
|
* Authentication follows the JWT-bearer flow: a JWT assertion is signed with the service
|
|
@@ -152,6 +152,12 @@ export class IdentityToolkit {
|
|
|
152
152
|
body: JSON.stringify({ localId: uid }),
|
|
153
153
|
});
|
|
154
154
|
if (!res.ok) {
|
|
155
|
+
const body = (await res.json().catch(() => undefined));
|
|
156
|
+
// Account deletion is an idempotent operation for callers. A retry after a lost response may
|
|
157
|
+
// legitimately find that the preceding request already removed the Firebase user.
|
|
158
|
+
if (body?.error?.message === 'USER_NOT_FOUND') {
|
|
159
|
+
return;
|
|
160
|
+
}
|
|
155
161
|
throw new Error(`Identity Toolkit delete failed: ${res.status}`);
|
|
156
162
|
}
|
|
157
163
|
}
|
package/dist/index.d.ts
CHANGED
|
@@ -78,8 +78,8 @@ export type { GoogleSubscriptionClassification, GoogleSubscriptionState, GoogleS
|
|
|
78
78
|
export { retryWhenDeadlock } from './db/retry.js';
|
|
79
79
|
export { sendInChunks } from './queue/send.js';
|
|
80
80
|
export type { QueueLike, QueueSendMessage } from './queue/send.js';
|
|
81
|
-
export { processBatch } from './queue/consumer.js';
|
|
82
|
-
export type { QueueMessageLike, MessageBatchLike, ProcessBatchOptions, ProcessBatchResult } from './queue/consumer.js';
|
|
81
|
+
export { isNonRetryableQueueError, processBatch } from './queue/consumer.js';
|
|
82
|
+
export type { QueueMessageLike, MessageBatchLike, NonRetryableQueueErrorLike, ProcessBatchOptions, ProcessBatchResult, } from './queue/consumer.js';
|
|
83
83
|
export { createQueueErrorHandler } from './queue/error-handler.js';
|
|
84
84
|
export type { CreateQueueErrorHandlerOptions } from './queue/error-handler.js';
|
|
85
85
|
export { createAiGatewayProvider } from './ai/gateway.js';
|
package/dist/index.js
CHANGED
|
@@ -59,7 +59,7 @@ export { classifyGoogleSubscription, getGoogleSubscription, googleAccessToken }
|
|
|
59
59
|
export { retryWhenDeadlock } from './db/retry.js';
|
|
60
60
|
// queue
|
|
61
61
|
export { sendInChunks } from './queue/send.js';
|
|
62
|
-
export { processBatch } from './queue/consumer.js';
|
|
62
|
+
export { isNonRetryableQueueError, processBatch } from './queue/consumer.js';
|
|
63
63
|
export { createQueueErrorHandler } from './queue/error-handler.js';
|
|
64
64
|
// ai
|
|
65
65
|
export { createAiGatewayProvider } from './ai/gateway.js';
|
package/dist/queue/consumer.d.ts
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Consumer-side helper for processing a Cloudflare Queues `MessageBatch` with per-message success and
|
|
3
|
-
* failure handling.
|
|
3
|
+
* failure handling, including explicit acknowledgement of permanent failures.
|
|
4
4
|
*
|
|
5
5
|
* A queue consumer invocation receives at most `max_batch_size` messages (configured in
|
|
6
6
|
* `wrangler.toml`), which is precisely the mechanism that bounds its subrequest budget: with a small
|
|
7
7
|
* `max_batch_size`, each invocation performs a fixed, small number of external calls no matter how
|
|
8
8
|
* many messages are backed up in the queue. {@link processBatch} applies the standard
|
|
9
9
|
* ack-on-success / retry-on-failure discipline so one poison message does not fail its whole batch.
|
|
10
|
+
* Errors explicitly tagged with `queueDisposition: 'discard'` are reported and acknowledged because
|
|
11
|
+
* delivering the same payload again cannot make them converge.
|
|
10
12
|
*
|
|
11
13
|
* Messages are processed sequentially. This keeps the number of *simultaneously open* subrequests at
|
|
12
14
|
* one, staying well clear of the Workers concurrent-connection ceiling, and makes the per-invocation
|
|
@@ -66,8 +68,10 @@ export interface MessageBatchLike<Body = unknown> {
|
|
|
66
68
|
*/
|
|
67
69
|
export interface ProcessBatchOptions<Body = unknown> {
|
|
68
70
|
/**
|
|
69
|
-
* Invoked when `handler` throws for a message,
|
|
70
|
-
* Use it to log or report; it must not throw. Defaults to `console.error`.
|
|
71
|
+
* Invoked when `handler` throws for a message, before the message is retried or acknowledged as a
|
|
72
|
+
* permanent failure. Use it to log or report; it must not throw. Defaults to `console.error`.
|
|
73
|
+
* If a custom hook does throw, `processBatch` emits a console fallback and still applies the
|
|
74
|
+
* original error's disposition so a telemetry outage cannot turn a poison message into retries.
|
|
71
75
|
*/
|
|
72
76
|
onError?: (error: unknown, message: QueueMessageLike<Body>) => void;
|
|
73
77
|
/**
|
|
@@ -82,14 +86,30 @@ export interface ProcessBatchOptions<Body = unknown> {
|
|
|
82
86
|
export interface ProcessBatchResult {
|
|
83
87
|
/** Messages whose handler completed successfully and were acked. */
|
|
84
88
|
processed: number;
|
|
89
|
+
/** Messages whose handler failed permanently and were acked instead of retried. */
|
|
90
|
+
discarded: number;
|
|
85
91
|
/** Messages whose handler threw and were marked for retry. */
|
|
86
92
|
failed: number;
|
|
87
93
|
}
|
|
88
94
|
/**
|
|
89
|
-
*
|
|
95
|
+
* Error contract for failures that cannot converge by redelivering the same queue message.
|
|
90
96
|
*
|
|
91
|
-
*
|
|
92
|
-
*
|
|
97
|
+
* Domain packages should use a named error class with this flag. {@link processBatch} owns the
|
|
98
|
+
* transport decision: it reports the failure through `onError`, acknowledges the message, and does
|
|
99
|
+
* not spend retries or dead-letter capacity on it.
|
|
100
|
+
*/
|
|
101
|
+
export interface NonRetryableQueueErrorLike {
|
|
102
|
+
readonly queueDisposition: 'discard';
|
|
103
|
+
}
|
|
104
|
+
/** Return whether an unknown thrown value explicitly opts out of queue retry. */
|
|
105
|
+
export declare function isNonRetryableQueueError(error: unknown): error is NonRetryableQueueErrorLike;
|
|
106
|
+
/**
|
|
107
|
+
* Process every message in `batch` sequentially, acking successes and permanent failures while
|
|
108
|
+
* retrying transient or unclassified failures.
|
|
109
|
+
*
|
|
110
|
+
* Each message is passed to `handler`; if it resolves the message is acked. A thrown error is routed
|
|
111
|
+
* to {@link ProcessBatchOptions.onError}; errors tagged with `queueDisposition: 'discard'` are then
|
|
112
|
+
* acked and counted as discarded, while all other errors are marked for retry (honoring
|
|
93
113
|
* {@link ProcessBatchOptions.retryDelaySeconds}). One failing message never affects the others, and
|
|
94
114
|
* the returned counts let tests assert that the per-invocation workload — and therefore the
|
|
95
115
|
* subrequest count — stayed bounded by the batch size.
|
|
@@ -99,7 +119,7 @@ export interface ProcessBatchResult {
|
|
|
99
119
|
* @param handler - Async work for a single message; performs the bounded external call(s). Receives
|
|
100
120
|
* the decoded `body` and the raw message (for `attempts`, `id`, etc.).
|
|
101
121
|
* @param options - Error reporting and retry tuning; see {@link ProcessBatchOptions}.
|
|
102
|
-
* @returns The number of processed and failed messages.
|
|
122
|
+
* @returns The number of processed, discarded, and retryable-failed messages.
|
|
103
123
|
* @example
|
|
104
124
|
* ```ts
|
|
105
125
|
* const { processed, failed } = await processBatch(
|
package/dist/queue/consumer.js
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Consumer-side helper for processing a Cloudflare Queues `MessageBatch` with per-message success and
|
|
3
|
-
* failure handling.
|
|
3
|
+
* failure handling, including explicit acknowledgement of permanent failures.
|
|
4
4
|
*
|
|
5
5
|
* A queue consumer invocation receives at most `max_batch_size` messages (configured in
|
|
6
6
|
* `wrangler.toml`), which is precisely the mechanism that bounds its subrequest budget: with a small
|
|
7
7
|
* `max_batch_size`, each invocation performs a fixed, small number of external calls no matter how
|
|
8
8
|
* many messages are backed up in the queue. {@link processBatch} applies the standard
|
|
9
9
|
* ack-on-success / retry-on-failure discipline so one poison message does not fail its whole batch.
|
|
10
|
+
* Errors explicitly tagged with `queueDisposition: 'discard'` are reported and acknowledged because
|
|
11
|
+
* delivering the same payload again cannot make them converge.
|
|
10
12
|
*
|
|
11
13
|
* Messages are processed sequentially. This keeps the number of *simultaneously open* subrequests at
|
|
12
14
|
* one, staying well clear of the Workers concurrent-connection ceiling, and makes the per-invocation
|
|
@@ -27,11 +29,17 @@
|
|
|
27
29
|
*
|
|
28
30
|
* @packageDocumentation
|
|
29
31
|
*/
|
|
32
|
+
/** Return whether an unknown thrown value explicitly opts out of queue retry. */
|
|
33
|
+
export function isNonRetryableQueueError(error) {
|
|
34
|
+
return (typeof error === 'object' && error !== null && 'queueDisposition' in error && error.queueDisposition === 'discard');
|
|
35
|
+
}
|
|
30
36
|
/**
|
|
31
|
-
* Process every message in `batch` sequentially, acking
|
|
37
|
+
* Process every message in `batch` sequentially, acking successes and permanent failures while
|
|
38
|
+
* retrying transient or unclassified failures.
|
|
32
39
|
*
|
|
33
|
-
* Each message is passed to `handler`; if it resolves the message is acked
|
|
34
|
-
*
|
|
40
|
+
* Each message is passed to `handler`; if it resolves the message is acked. A thrown error is routed
|
|
41
|
+
* to {@link ProcessBatchOptions.onError}; errors tagged with `queueDisposition: 'discard'` are then
|
|
42
|
+
* acked and counted as discarded, while all other errors are marked for retry (honoring
|
|
35
43
|
* {@link ProcessBatchOptions.retryDelaySeconds}). One failing message never affects the others, and
|
|
36
44
|
* the returned counts let tests assert that the per-invocation workload — and therefore the
|
|
37
45
|
* subrequest count — stayed bounded by the batch size.
|
|
@@ -41,7 +49,7 @@
|
|
|
41
49
|
* @param handler - Async work for a single message; performs the bounded external call(s). Receives
|
|
42
50
|
* the decoded `body` and the raw message (for `attempts`, `id`, etc.).
|
|
43
51
|
* @param options - Error reporting and retry tuning; see {@link ProcessBatchOptions}.
|
|
44
|
-
* @returns The number of processed and failed messages.
|
|
52
|
+
* @returns The number of processed, discarded, and retryable-failed messages.
|
|
45
53
|
* @example
|
|
46
54
|
* ```ts
|
|
47
55
|
* const { processed, failed } = await processBatch(
|
|
@@ -58,6 +66,7 @@ export async function processBatch(batch, handler, options) {
|
|
|
58
66
|
});
|
|
59
67
|
const retryOptions = options?.retryDelaySeconds === undefined ? undefined : { delaySeconds: options.retryDelaySeconds };
|
|
60
68
|
let processed = 0;
|
|
69
|
+
let discarded = 0;
|
|
61
70
|
let failed = 0;
|
|
62
71
|
for (const message of batch.messages) {
|
|
63
72
|
try {
|
|
@@ -69,12 +78,19 @@ export async function processBatch(batch, handler, options) {
|
|
|
69
78
|
try {
|
|
70
79
|
onError(error, message);
|
|
71
80
|
}
|
|
72
|
-
catch {
|
|
73
|
-
//
|
|
81
|
+
catch (reportingError) {
|
|
82
|
+
// Reporting is best-effort. Preserve the domain error's disposition, but never let a broken
|
|
83
|
+
// custom reporter make a permanent failure disappear without any local trace.
|
|
84
|
+
console.error(`[queue:${batch.queue}] onError failed for message ${message.id}`, reportingError, 'original error:', error);
|
|
85
|
+
}
|
|
86
|
+
if (isNonRetryableQueueError(error)) {
|
|
87
|
+
message.ack();
|
|
88
|
+
discarded++;
|
|
89
|
+
continue;
|
|
74
90
|
}
|
|
75
91
|
message.retry(retryOptions);
|
|
76
92
|
failed++;
|
|
77
93
|
}
|
|
78
94
|
}
|
|
79
|
-
return { processed, failed };
|
|
95
|
+
return { processed, discarded, failed };
|
|
80
96
|
}
|
|
@@ -8,8 +8,10 @@ export interface CreateQueueErrorHandlerOptions {
|
|
|
8
8
|
queue: string;
|
|
9
9
|
/**
|
|
10
10
|
* When set, `captureException` is called only after the final delivery attempt
|
|
11
|
-
* (`message.attempts > maxRetries`).
|
|
12
|
-
*
|
|
11
|
+
* (`message.attempts > maxRetries`). Errors tagged with `queueDisposition: 'discard'` are captured
|
|
12
|
+
* on their first delivery because {@link processBatch} acknowledges them immediately. Cloudflare
|
|
13
|
+
* Queues uses 1-based `attempts`; the last delivery before the dead-letter queue has
|
|
14
|
+
* `attempts === maxRetries + 1`.
|
|
13
15
|
*/
|
|
14
16
|
maxRetries?: number;
|
|
15
17
|
/** Optional Sentry client. Omit for console-only reporting (e.g. airlec). */
|
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { isNonRetryableQueueError } from './consumer.js';
|
|
1
2
|
/**
|
|
2
3
|
* Factory for {@link processBatch}'s `onError` hook: logs every failure and optionally reports to
|
|
3
4
|
* Sentry (or another sink) with queue / message id / attempts / body context.
|
|
@@ -10,11 +11,15 @@ export function createQueueErrorHandler(options) {
|
|
|
10
11
|
if (!capture) {
|
|
11
12
|
return;
|
|
12
13
|
}
|
|
13
|
-
if (maxRetries !== undefined && message.attempts <= maxRetries) {
|
|
14
|
+
if (!isNonRetryableQueueError(error) && maxRetries !== undefined && message.attempts <= maxRetries) {
|
|
14
15
|
return;
|
|
15
16
|
}
|
|
16
17
|
capture(error, {
|
|
17
|
-
tags: {
|
|
18
|
+
tags: {
|
|
19
|
+
queue,
|
|
20
|
+
queue_message_id: message.id,
|
|
21
|
+
queue_disposition: isNonRetryableQueueError(error) ? 'discard' : 'retry',
|
|
22
|
+
},
|
|
18
23
|
extra: { attempts: message.attempts, body: message.body },
|
|
19
24
|
});
|
|
20
25
|
};
|
package/dist/queue/send.d.ts
CHANGED
|
@@ -2,11 +2,13 @@
|
|
|
2
2
|
* Producer-side helper for fanning a large list of items into a Cloudflare Queue without letting the
|
|
3
3
|
* producer's own subrequest count scale linearly with the list.
|
|
4
4
|
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
5
|
+
* Queue bindings call a Cloudflare internal service. The internal-service subrequest budget is
|
|
6
|
+
* 1,000 on Workers Free and matches the configured subrequest limit on Paid (10,000 by default,
|
|
7
|
+
* configurable up to 10 million). Each {@link QueueLike.send} counts as one subrequest. Enqueuing
|
|
8
|
+
* `N` items with per-item `send()` calls therefore reintroduces the very unbounded fan-out that
|
|
9
|
+
* queues exist to remove. {@link sendInChunks} instead groups items into batches and issues one
|
|
10
|
+
* {@link QueueLike.sendBatch} per batch, so the producer spends `ceil(N / chunkSize)` subrequests
|
|
11
|
+
* regardless of how large `N` grows.
|
|
10
12
|
*
|
|
11
13
|
* The heavy per-item work (external API calls, etc.) is expected to run in the queue *consumer*,
|
|
12
14
|
* where each invocation processes only `max_batch_size` messages and thus enjoys its own bounded
|
package/dist/queue/send.js
CHANGED
|
@@ -2,11 +2,13 @@
|
|
|
2
2
|
* Producer-side helper for fanning a large list of items into a Cloudflare Queue without letting the
|
|
3
3
|
* producer's own subrequest count scale linearly with the list.
|
|
4
4
|
*
|
|
5
|
-
*
|
|
6
|
-
*
|
|
7
|
-
*
|
|
8
|
-
*
|
|
9
|
-
*
|
|
5
|
+
* Queue bindings call a Cloudflare internal service. The internal-service subrequest budget is
|
|
6
|
+
* 1,000 on Workers Free and matches the configured subrequest limit on Paid (10,000 by default,
|
|
7
|
+
* configurable up to 10 million). Each {@link QueueLike.send} counts as one subrequest. Enqueuing
|
|
8
|
+
* `N` items with per-item `send()` calls therefore reintroduces the very unbounded fan-out that
|
|
9
|
+
* queues exist to remove. {@link sendInChunks} instead groups items into batches and issues one
|
|
10
|
+
* {@link QueueLike.sendBatch} per batch, so the producer spends `ceil(N / chunkSize)` subrequests
|
|
11
|
+
* regardless of how large `N` grows.
|
|
10
12
|
*
|
|
11
13
|
* The heavy per-item work (external API calls, etc.) is expected to run in the queue *consumer*,
|
|
12
14
|
* where each invocation processes only `max_batch_size` messages and thus enjoys its own bounded
|
package/package.json
CHANGED
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
/**
|
|
3
3
|
* check-subrequest-fanout — flag per-item external-call fan-outs that scale with data size.
|
|
4
4
|
*
|
|
5
|
-
* Cloudflare Workers cap subrequests per invocation (50
|
|
5
|
+
* Cloudflare Workers cap external subrequests per invocation (50 on Free; 10,000 by default on
|
|
6
|
+
* Paid, configurable up to 10 million). Looping an external call
|
|
6
7
|
* (fetch / AI / Stripe / push / ES) once per row reintroduces an unbounded fan-out that eventually
|
|
7
8
|
* exceeds the cap as the userbase/data grows. This gate greps for the concurrency-loop markers that
|
|
8
9
|
* usually wrap such fan-outs and fails CI unless the site is explicitly annotated as safe.
|