@quickback-dev/cli 0.10.15 → 0.10.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/commands/compile.d.ts.map +1 -1
- package/dist/commands/compile.js +40 -2
- package/dist/commands/compile.js.map +1 -1
- package/dist/docs/content.js +6 -6
- package/dist/docs/content.js.map +1 -1
- package/dist/index.js +1 -1
- package/dist/skill/SKILL.md +142 -1051
- package/dist/skill/agents/quickback-specialist/AGENT.md +36 -0
- package/dist/skill/docs/changelog.md +1192 -0
- package/dist/skill/docs/compiler/claude-code-skill.md +312 -0
- package/dist/skill/docs/compiler/cloud-compiler/authentication.md +63 -0
- package/dist/skill/docs/compiler/cloud-compiler/cli.md +307 -0
- package/dist/skill/docs/compiler/cloud-compiler/endpoints.md +78 -0
- package/dist/skill/docs/compiler/cloud-compiler/index.md +65 -0
- package/dist/skill/docs/compiler/cloud-compiler/local-compiler.md +115 -0
- package/dist/skill/docs/compiler/cloud-compiler/troubleshooting.md +76 -0
- package/dist/skill/docs/compiler/config/agents.md +144 -0
- package/dist/skill/docs/compiler/config/auth.md +109 -0
- package/dist/skill/docs/compiler/config/bindings.md +176 -0
- package/dist/skill/docs/compiler/config/domains.md +149 -0
- package/dist/skill/docs/compiler/config/index.md +419 -0
- package/dist/skill/docs/compiler/config/output.md +338 -0
- package/dist/skill/docs/compiler/config/providers.md +248 -0
- package/dist/skill/docs/compiler/config/security.md +255 -0
- package/dist/skill/docs/compiler/config/single-tenant.md +128 -0
- package/dist/skill/docs/compiler/config/variables.md +157 -0
- package/dist/skill/docs/compiler/definitions/access.md +445 -0
- package/dist/skill/docs/compiler/definitions/actions.md +884 -0
- package/dist/skill/docs/compiler/definitions/concepts.md +52 -0
- package/dist/skill/docs/compiler/definitions/feature.md +164 -0
- package/dist/skill/docs/compiler/definitions/firewall.md +285 -0
- package/dist/skill/docs/compiler/definitions/guards.md +241 -0
- package/dist/skill/docs/compiler/definitions/index.md +219 -0
- package/dist/skill/docs/compiler/definitions/masking.md +241 -0
- package/dist/skill/docs/compiler/definitions/read.md +220 -0
- package/dist/skill/docs/compiler/definitions/schema.md +697 -0
- package/dist/skill/docs/compiler/definitions/validation.md +35 -0
- package/dist/skill/docs/compiler/definitions/views.md +292 -0
- package/dist/skill/docs/compiler/getting-started/claude-code.md +126 -0
- package/dist/skill/docs/compiler/getting-started/full-example.md +431 -0
- package/dist/skill/docs/compiler/getting-started/hand-crafted.md +178 -0
- package/dist/skill/docs/compiler/getting-started/index.md +192 -0
- package/dist/skill/docs/compiler/getting-started/patterns.md +225 -0
- package/dist/skill/docs/compiler/getting-started/template-blog.md +280 -0
- package/dist/skill/docs/compiler/getting-started/template-cloudflare.md +260 -0
- package/dist/skill/docs/compiler/getting-started/template-empty.md +112 -0
- package/dist/skill/docs/compiler/getting-started/templates.md +111 -0
- package/dist/skill/docs/compiler/index.md +85 -0
- package/dist/skill/docs/compiler/targets/hono-api/cloudflare.md +94 -0
- package/dist/skill/docs/compiler/targets/hono-api/database/d1.md +246 -0
- package/dist/skill/docs/compiler/targets/hono-api/database/index.md +16 -0
- package/dist/skill/docs/compiler/targets/hono-api/database/neon.md +144 -0
- package/dist/skill/docs/compiler/targets/hono-api/database/using-d1.md +39 -0
- package/dist/skill/docs/compiler/targets/hono-api/index.md +37 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/actions-api.md +323 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/batch-operations.md +357 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/caching.md +105 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/crud.md +652 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/errors.md +224 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/index.md +37 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/openapi.md +203 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/query-params.md +199 -0
- package/dist/skill/docs/compiler/targets/hono-api/using-the-api/views-api.md +161 -0
- package/dist/skill/docs/compiler/targets/index.md +47 -0
- package/dist/skill/docs/compiler/targets/supabase-rls/index.md +55 -0
- package/dist/skill/docs/compiler/targets/supabase-rls/limitations.md +48 -0
- package/dist/skill/docs/compiler/targets/supabase-rls/mappings.md +99 -0
- package/dist/skill/docs/compiler/targets/supabase-rls/setup.md +88 -0
- package/dist/skill/docs/compiler/troubleshooting.md +120 -0
- package/dist/skill/docs/index.md +77 -0
- package/dist/skill/docs/stack/account-ui/customization.md +378 -0
- package/dist/skill/docs/stack/account-ui/environment-variables.md +325 -0
- package/dist/skill/docs/stack/account-ui/features/admin.md +76 -0
- package/dist/skill/docs/stack/account-ui/features/api-keys.md +22 -0
- package/dist/skill/docs/stack/account-ui/features/avatars.md +37 -0
- package/dist/skill/docs/stack/account-ui/features/cli-authorize.md +21 -0
- package/dist/skill/docs/stack/account-ui/features/index.md +291 -0
- package/dist/skill/docs/stack/account-ui/features/organizations.md +29 -0
- package/dist/skill/docs/stack/account-ui/features/passkeys.md +31 -0
- package/dist/skill/docs/stack/account-ui/features/passwordless.md +28 -0
- package/dist/skill/docs/stack/account-ui/features/sessions.md +20 -0
- package/dist/skill/docs/stack/account-ui/hooks.md +209 -0
- package/dist/skill/docs/stack/account-ui/index.md +147 -0
- package/dist/skill/docs/stack/account-ui/library-usage.md +257 -0
- package/dist/skill/docs/stack/account-ui/standalone.md +149 -0
- package/dist/skill/docs/stack/account-ui/with-quickback.md +160 -0
- package/dist/skill/docs/stack/account-ui/worker.md +362 -0
- package/dist/skill/docs/stack/auth/api-keys.md +15 -0
- package/dist/skill/docs/stack/auth/device-auth.md +14 -0
- package/dist/skill/docs/stack/auth/index.md +64 -0
- package/dist/skill/docs/stack/auth/jwt-optimization.md +245 -0
- package/dist/skill/docs/stack/auth/plugins/aws-ses.md +127 -0
- package/dist/skill/docs/stack/auth/plugins/combo-auth.md +74 -0
- package/dist/skill/docs/stack/auth/plugins/index.md +228 -0
- package/dist/skill/docs/stack/auth/plugins/upgrade-anonymous.md +155 -0
- package/dist/skill/docs/stack/auth/security.md +455 -0
- package/dist/skill/docs/stack/auth/using-auth.md +39 -0
- package/dist/skill/docs/stack/cms/actions.md +199 -0
- package/dist/skill/docs/stack/cms/components.md +479 -0
- package/dist/skill/docs/stack/cms/connecting.md +199 -0
- package/dist/skill/docs/stack/cms/dashboard.md +39 -0
- package/dist/skill/docs/stack/cms/index.md +158 -0
- package/dist/skill/docs/stack/cms/inline-editing.md +136 -0
- package/dist/skill/docs/stack/cms/pages.md +275 -0
- package/dist/skill/docs/stack/cms/record-layouts.md +151 -0
- package/dist/skill/docs/stack/cms/schema-format.md +437 -0
- package/dist/skill/docs/stack/cms/schema-registry.md +200 -0
- package/dist/skill/docs/stack/cms/security.md +203 -0
- package/dist/skill/docs/stack/cms/table-views.md +142 -0
- package/dist/skill/docs/stack/index.md +65 -0
- package/dist/skill/docs/stack/queues/handlers.md +317 -0
- package/dist/skill/docs/stack/queues/index.md +56 -0
- package/dist/skill/docs/stack/queues/using-queues.md +192 -0
- package/dist/skill/docs/stack/realtime/durable-objects.md +488 -0
- package/dist/skill/docs/stack/realtime/index.md +67 -0
- package/dist/skill/docs/stack/realtime/using-realtime.md +211 -0
- package/dist/skill/docs/stack/storage/index.md +17 -0
- package/dist/skill/docs/stack/storage/kv.md +204 -0
- package/dist/skill/docs/stack/storage/r2.md +232 -0
- package/dist/skill/docs/stack/storage/using-kv.md +70 -0
- package/dist/skill/docs/stack/storage/using-r2.md +55 -0
- package/dist/skill/docs/stack/vector/embeddings.md +554 -0
- package/dist/skill/docs/stack/vector/index.md +44 -0
- package/dist/skill/docs/stack/vector/using-embeddings.md +186 -0
- package/dist/skill/docs/stack/webhooks/inbound.md +186 -0
- package/dist/skill/docs/stack/webhooks/index.md +36 -0
- package/dist/skill/docs/stack/webhooks/outbound.md +281 -0
- package/package.json +1 -1
- package/src/skill/SKILL.md +142 -1051
- package/src/skill/agents/quickback-specialist/AGENT.md +36 -0
- package/src/skill/docs/changelog.md +1192 -0
- package/src/skill/docs/compiler/claude-code-skill.md +312 -0
- package/src/skill/docs/compiler/cloud-compiler/authentication.md +63 -0
- package/src/skill/docs/compiler/cloud-compiler/cli.md +307 -0
- package/src/skill/docs/compiler/cloud-compiler/endpoints.md +78 -0
- package/src/skill/docs/compiler/cloud-compiler/index.md +65 -0
- package/src/skill/docs/compiler/cloud-compiler/local-compiler.md +115 -0
- package/src/skill/docs/compiler/cloud-compiler/troubleshooting.md +76 -0
- package/src/skill/docs/compiler/config/agents.md +144 -0
- package/src/skill/docs/compiler/config/auth.md +109 -0
- package/src/skill/docs/compiler/config/bindings.md +176 -0
- package/src/skill/docs/compiler/config/domains.md +149 -0
- package/src/skill/docs/compiler/config/index.md +419 -0
- package/src/skill/docs/compiler/config/output.md +338 -0
- package/src/skill/docs/compiler/config/providers.md +248 -0
- package/src/skill/docs/compiler/config/security.md +255 -0
- package/src/skill/docs/compiler/config/single-tenant.md +128 -0
- package/src/skill/docs/compiler/config/variables.md +157 -0
- package/src/skill/docs/compiler/definitions/access.md +445 -0
- package/src/skill/docs/compiler/definitions/actions.md +884 -0
- package/src/skill/docs/compiler/definitions/concepts.md +52 -0
- package/src/skill/docs/compiler/definitions/feature.md +164 -0
- package/src/skill/docs/compiler/definitions/firewall.md +285 -0
- package/src/skill/docs/compiler/definitions/guards.md +241 -0
- package/src/skill/docs/compiler/definitions/index.md +219 -0
- package/src/skill/docs/compiler/definitions/masking.md +241 -0
- package/src/skill/docs/compiler/definitions/read.md +220 -0
- package/src/skill/docs/compiler/definitions/schema.md +697 -0
- package/src/skill/docs/compiler/definitions/validation.md +35 -0
- package/src/skill/docs/compiler/definitions/views.md +292 -0
- package/src/skill/docs/compiler/getting-started/claude-code.md +126 -0
- package/src/skill/docs/compiler/getting-started/full-example.md +431 -0
- package/src/skill/docs/compiler/getting-started/hand-crafted.md +178 -0
- package/src/skill/docs/compiler/getting-started/index.md +192 -0
- package/src/skill/docs/compiler/getting-started/patterns.md +225 -0
- package/src/skill/docs/compiler/getting-started/template-blog.md +280 -0
- package/src/skill/docs/compiler/getting-started/template-cloudflare.md +260 -0
- package/src/skill/docs/compiler/getting-started/template-empty.md +112 -0
- package/src/skill/docs/compiler/getting-started/templates.md +111 -0
- package/src/skill/docs/compiler/index.md +85 -0
- package/src/skill/docs/compiler/targets/hono-api/cloudflare.md +94 -0
- package/src/skill/docs/compiler/targets/hono-api/database/d1.md +246 -0
- package/src/skill/docs/compiler/targets/hono-api/database/index.md +16 -0
- package/src/skill/docs/compiler/targets/hono-api/database/neon.md +144 -0
- package/src/skill/docs/compiler/targets/hono-api/database/using-d1.md +39 -0
- package/src/skill/docs/compiler/targets/hono-api/index.md +37 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/actions-api.md +323 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/batch-operations.md +357 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/caching.md +105 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/crud.md +652 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/errors.md +224 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/index.md +37 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/openapi.md +203 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/query-params.md +199 -0
- package/src/skill/docs/compiler/targets/hono-api/using-the-api/views-api.md +161 -0
- package/src/skill/docs/compiler/targets/index.md +47 -0
- package/src/skill/docs/compiler/targets/supabase-rls/index.md +55 -0
- package/src/skill/docs/compiler/targets/supabase-rls/limitations.md +48 -0
- package/src/skill/docs/compiler/targets/supabase-rls/mappings.md +99 -0
- package/src/skill/docs/compiler/targets/supabase-rls/setup.md +88 -0
- package/src/skill/docs/compiler/troubleshooting.md +120 -0
- package/src/skill/docs/index.md +77 -0
- package/src/skill/docs/stack/account-ui/customization.md +378 -0
- package/src/skill/docs/stack/account-ui/environment-variables.md +325 -0
- package/src/skill/docs/stack/account-ui/features/admin.md +76 -0
- package/src/skill/docs/stack/account-ui/features/api-keys.md +22 -0
- package/src/skill/docs/stack/account-ui/features/avatars.md +37 -0
- package/src/skill/docs/stack/account-ui/features/cli-authorize.md +21 -0
- package/src/skill/docs/stack/account-ui/features/index.md +291 -0
- package/src/skill/docs/stack/account-ui/features/organizations.md +29 -0
- package/src/skill/docs/stack/account-ui/features/passkeys.md +31 -0
- package/src/skill/docs/stack/account-ui/features/passwordless.md +28 -0
- package/src/skill/docs/stack/account-ui/features/sessions.md +20 -0
- package/src/skill/docs/stack/account-ui/hooks.md +209 -0
- package/src/skill/docs/stack/account-ui/index.md +147 -0
- package/src/skill/docs/stack/account-ui/library-usage.md +257 -0
- package/src/skill/docs/stack/account-ui/standalone.md +149 -0
- package/src/skill/docs/stack/account-ui/with-quickback.md +160 -0
- package/src/skill/docs/stack/account-ui/worker.md +362 -0
- package/src/skill/docs/stack/auth/api-keys.md +15 -0
- package/src/skill/docs/stack/auth/device-auth.md +14 -0
- package/src/skill/docs/stack/auth/index.md +64 -0
- package/src/skill/docs/stack/auth/jwt-optimization.md +245 -0
- package/src/skill/docs/stack/auth/plugins/aws-ses.md +127 -0
- package/src/skill/docs/stack/auth/plugins/combo-auth.md +74 -0
- package/src/skill/docs/stack/auth/plugins/index.md +228 -0
- package/src/skill/docs/stack/auth/plugins/upgrade-anonymous.md +155 -0
- package/src/skill/docs/stack/auth/security.md +455 -0
- package/src/skill/docs/stack/auth/using-auth.md +39 -0
- package/src/skill/docs/stack/cms/actions.md +199 -0
- package/src/skill/docs/stack/cms/components.md +479 -0
- package/src/skill/docs/stack/cms/connecting.md +199 -0
- package/src/skill/docs/stack/cms/dashboard.md +39 -0
- package/src/skill/docs/stack/cms/index.md +158 -0
- package/src/skill/docs/stack/cms/inline-editing.md +136 -0
- package/src/skill/docs/stack/cms/pages.md +275 -0
- package/src/skill/docs/stack/cms/record-layouts.md +151 -0
- package/src/skill/docs/stack/cms/schema-format.md +437 -0
- package/src/skill/docs/stack/cms/schema-registry.md +200 -0
- package/src/skill/docs/stack/cms/security.md +203 -0
- package/src/skill/docs/stack/cms/table-views.md +142 -0
- package/src/skill/docs/stack/index.md +65 -0
- package/src/skill/docs/stack/queues/handlers.md +317 -0
- package/src/skill/docs/stack/queues/index.md +56 -0
- package/src/skill/docs/stack/queues/using-queues.md +192 -0
- package/src/skill/docs/stack/realtime/durable-objects.md +488 -0
- package/src/skill/docs/stack/realtime/index.md +67 -0
- package/src/skill/docs/stack/realtime/using-realtime.md +211 -0
- package/src/skill/docs/stack/storage/index.md +17 -0
- package/src/skill/docs/stack/storage/kv.md +204 -0
- package/src/skill/docs/stack/storage/r2.md +232 -0
- package/src/skill/docs/stack/storage/using-kv.md +70 -0
- package/src/skill/docs/stack/storage/using-r2.md +55 -0
- package/src/skill/docs/stack/vector/embeddings.md +554 -0
- package/src/skill/docs/stack/vector/index.md +44 -0
- package/src/skill/docs/stack/vector/using-embeddings.md +186 -0
- package/src/skill/docs/stack/webhooks/inbound.md +186 -0
- package/src/skill/docs/stack/webhooks/index.md +36 -0
- package/src/skill/docs/stack/webhooks/outbound.md +281 -0
|
@@ -0,0 +1,317 @@
|
|
|
1
|
+
# Custom Queue Handlers
|
|
2
|
+
|
|
3
|
+
Quickback lets you define custom queue handlers that integrate seamlessly with the generated queue consumer. This is useful for background processing like material extraction pipelines, batch jobs, and async workflows.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
Custom queue handlers are defined in the `services/queues/` directory using the `defineQueue` helper. The compiler extracts your handler logic and generates a unified queue consumer that dispatches messages based on their type.
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
quickback/
|
|
11
|
+
├── features/ # Data layer (CRUD, actions, security)
|
|
12
|
+
│ ├── materials/
|
|
13
|
+
│ └── claims/
|
|
14
|
+
└── services/ # Infrastructure layer
|
|
15
|
+
└── queues/
|
|
16
|
+
├── ingest.ts # Material processing
|
|
17
|
+
└── claim-batches.ts # Batch processing
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Defining a Queue Handler
|
|
21
|
+
|
|
22
|
+
Use `defineQueue` to create a handler:
|
|
23
|
+
|
|
24
|
+
```typescript
|
|
25
|
+
// services/queues/ingest.ts
|
|
26
|
+
import { defineQueue } from 'quickback';
|
|
27
|
+
import { eq } from 'drizzle-orm';
|
|
28
|
+
|
|
29
|
+
interface ProcessMaterialMessage {
|
|
30
|
+
type: 'process_material';
|
|
31
|
+
materialId: string;
|
|
32
|
+
organizationId: string;
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
export default defineQueue<ProcessMaterialMessage>({
|
|
36
|
+
name: 'ingest',
|
|
37
|
+
messageType: 'process_material',
|
|
38
|
+
description: 'Process materials through extraction pipeline',
|
|
39
|
+
|
|
40
|
+
execute: async ({ message, db, env, services, ack, retry }) => {
|
|
41
|
+
const { materialId, organizationId } = message;
|
|
42
|
+
|
|
43
|
+
// Dynamic import of schema
|
|
44
|
+
const { materials } = await import('./features/materials/schema');
|
|
45
|
+
|
|
46
|
+
// Get material from database
|
|
47
|
+
const [material] = await db
|
|
48
|
+
.select()
|
|
49
|
+
.from(materials)
|
|
50
|
+
.where(eq(materials.id, materialId))
|
|
51
|
+
.limit(1);
|
|
52
|
+
|
|
53
|
+
if (!material) {
|
|
54
|
+
console.error('[IngestQueue] Material not found:', materialId);
|
|
55
|
+
return { success: false, error: 'Material not found' };
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
// Process the material...
|
|
59
|
+
await db
|
|
60
|
+
.update(materials)
|
|
61
|
+
.set({ extractionStatus: 'processing' })
|
|
62
|
+
.where(eq(materials.id, materialId));
|
|
63
|
+
|
|
64
|
+
// Your processing logic here
|
|
65
|
+
|
|
66
|
+
return { success: true };
|
|
67
|
+
},
|
|
68
|
+
});
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Configuration Options
|
|
72
|
+
|
|
73
|
+
| Option | Type | Required | Description |
|
|
74
|
+
|--------|------|----------|-------------|
|
|
75
|
+
| `name` | `string` | Yes | Handler identifier (used in logs) |
|
|
76
|
+
| `messageType` | `string` | Yes | Message type to match (e.g., `'process_material'`) |
|
|
77
|
+
| `description` | `string` | No | Human-readable description |
|
|
78
|
+
| `execute` | `function` | Yes | Handler function |
|
|
79
|
+
|
|
80
|
+
## Execute Function
|
|
81
|
+
|
|
82
|
+
The `execute` function receives a context object:
|
|
83
|
+
|
|
84
|
+
```typescript
|
|
85
|
+
execute: async ({ message, db, env, services, ack, retry }) => {
|
|
86
|
+
// message - The message payload (typed via generic)
|
|
87
|
+
// db - Drizzle database instance
|
|
88
|
+
// env - Cloudflare bindings (AI, queues, etc.)
|
|
89
|
+
// services - Generated services (ai, etc.)
|
|
90
|
+
// ack() - Acknowledge message (auto-called on success)
|
|
91
|
+
// retry() - Retry message (auto-called on failure)
|
|
92
|
+
|
|
93
|
+
return { success: true }; // or { success: false, error: 'reason' }
|
|
94
|
+
}
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Constants and Imports
|
|
98
|
+
|
|
99
|
+
The compiler extracts top-level constants and imports from your handler file and includes them in the generated queue consumer:
|
|
100
|
+
|
|
101
|
+
```typescript
|
|
102
|
+
// services/queues/ingest.ts
|
|
103
|
+
import { eq } from 'drizzle-orm';
|
|
104
|
+
import { generateId } from '@news-assist/shared';
|
|
105
|
+
|
|
106
|
+
const TEXT_GENERATION_MODEL = '@cf/meta/llama-3.1-8b-instruct';
|
|
107
|
+
|
|
108
|
+
const EXTRACTION_PROMPT = `You are a news analyst. Extract factual claims from this article.
|
|
109
|
+
|
|
110
|
+
For each claim, output a JSON object with:
|
|
111
|
+
- content: The claim as a complete, standalone sentence
|
|
112
|
+
- claimType: One of NEWS, QUOTE, OFFICIAL, ALLEGATION, STATISTIC, FINDING
|
|
113
|
+
- urgency: 1-5 (5 = most urgent/breaking)
|
|
114
|
+
|
|
115
|
+
Output ONLY a JSON array of claims, no other text.`;
|
|
116
|
+
|
|
117
|
+
export default defineQueue({
|
|
118
|
+
// ... handler definition
|
|
119
|
+
});
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
The compiler:
|
|
123
|
+
1. Extracts all `import` statements
|
|
124
|
+
2. Extracts top-level `const` declarations (including multi-line template literals)
|
|
125
|
+
3. Inlines them in the generated queue consumer
|
|
126
|
+
|
|
127
|
+
## Dynamic Schema Imports
|
|
128
|
+
|
|
129
|
+
Since your schema files are generated by Quickback, use dynamic imports to reference them:
|
|
130
|
+
|
|
131
|
+
```typescript
|
|
132
|
+
execute: async ({ message, db }) => {
|
|
133
|
+
// Dynamic import - path is relative to generated src/ directory
|
|
134
|
+
const { materials, claimExtractionBatches } = await import('./features/materials/schema');
|
|
135
|
+
|
|
136
|
+
const [material] = await db
|
|
137
|
+
.select()
|
|
138
|
+
.from(materials)
|
|
139
|
+
.where(eq(materials.id, message.materialId));
|
|
140
|
+
|
|
141
|
+
// ...
|
|
142
|
+
}
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
## Sending to Other Queues
|
|
146
|
+
|
|
147
|
+
Chain queue handlers by sending messages to other queues:
|
|
148
|
+
|
|
149
|
+
```typescript
|
|
150
|
+
execute: async ({ message, db, env }) => {
|
|
151
|
+
const { materialId } = message;
|
|
152
|
+
|
|
153
|
+
// Process and create batch...
|
|
154
|
+
const batchId = generateId('batch');
|
|
155
|
+
|
|
156
|
+
// Send to next stage
|
|
157
|
+
if (env.CLAIM_BATCHES_QUEUE) {
|
|
158
|
+
await env.CLAIM_BATCHES_QUEUE.send({
|
|
159
|
+
type: 'claim_batch_stage',
|
|
160
|
+
batch_id: batchId,
|
|
161
|
+
material_id: materialId,
|
|
162
|
+
stage: 'subeditor',
|
|
163
|
+
});
|
|
164
|
+
}
|
|
165
|
+
|
|
166
|
+
return { success: true };
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
## Generated Output
|
|
171
|
+
|
|
172
|
+
When queue handlers are defined, the compiler generates a unified `queue-consumer.ts`:
|
|
173
|
+
|
|
174
|
+
```typescript
|
|
175
|
+
// src/queue-consumer.ts (generated)
|
|
176
|
+
import { drizzle } from 'drizzle-orm/d1';
|
|
177
|
+
import { eq } from 'drizzle-orm';
|
|
178
|
+
import { generateId } from '@news-assist/shared';
|
|
179
|
+
import { createServices } from './lib/services';
|
|
180
|
+
|
|
181
|
+
const TEXT_GENERATION_MODEL = '@cf/meta/llama-3.1-8b-instruct';
|
|
182
|
+
const EXTRACTION_PROMPT = `...`;
|
|
183
|
+
|
|
184
|
+
// Embedding handler (if embeddings configured)
|
|
185
|
+
const embeddingQueueHandler = async (batch, env) => { ... };
|
|
186
|
+
|
|
187
|
+
// Custom handler: ingest
|
|
188
|
+
const ingestQueueHandler = async (batch, env) => {
|
|
189
|
+
const db = drizzle(env.DB);
|
|
190
|
+
const services = createServices(env);
|
|
191
|
+
|
|
192
|
+
for (const message of batch.messages) {
|
|
193
|
+
try {
|
|
194
|
+
const result = await executeHandler({
|
|
195
|
+
message: message.body,
|
|
196
|
+
env, db, services,
|
|
197
|
+
ack: () => message.ack(),
|
|
198
|
+
retry: () => message.retry(),
|
|
199
|
+
});
|
|
200
|
+
|
|
201
|
+
if (result.success) message.ack();
|
|
202
|
+
else message.retry();
|
|
203
|
+
} catch (error) {
|
|
204
|
+
message.retry();
|
|
205
|
+
}
|
|
206
|
+
}
|
|
207
|
+
};
|
|
208
|
+
|
|
209
|
+
// Unified dispatcher
|
|
210
|
+
export const queue = async (batch, env) => {
|
|
211
|
+
const messageType = batch.messages[0]?.body?.type;
|
|
212
|
+
|
|
213
|
+
switch (messageType) {
|
|
214
|
+
case 'embedding':
|
|
215
|
+
await embeddingQueueHandler(batch, env);
|
|
216
|
+
break;
|
|
217
|
+
case 'process_material':
|
|
218
|
+
await ingestQueueHandler(batch, env);
|
|
219
|
+
break;
|
|
220
|
+
default:
|
|
221
|
+
console.warn('[Queue] Unknown message type:', messageType);
|
|
222
|
+
for (const msg of batch.messages) msg.ack();
|
|
223
|
+
}
|
|
224
|
+
};
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
## wrangler.toml Configuration
|
|
228
|
+
|
|
229
|
+
Configure your queues in wrangler.toml:
|
|
230
|
+
|
|
231
|
+
```toml
|
|
232
|
+
# Queue producers (send to queues)
|
|
233
|
+
[[queues.producers]]
|
|
234
|
+
queue = "my-app-ingest-queue"
|
|
235
|
+
binding = "INGEST_QUEUE"
|
|
236
|
+
|
|
237
|
+
[[queues.producers]]
|
|
238
|
+
queue = "my-app-claim-batches-queue"
|
|
239
|
+
binding = "CLAIM_BATCHES_QUEUE"
|
|
240
|
+
|
|
241
|
+
# Queue consumers (receive from queues)
|
|
242
|
+
[[queues.consumers]]
|
|
243
|
+
queue = "my-app-ingest-queue"
|
|
244
|
+
max_batch_size = 10
|
|
245
|
+
max_batch_timeout = 30
|
|
246
|
+
max_retries = 3
|
|
247
|
+
|
|
248
|
+
[[queues.consumers]]
|
|
249
|
+
queue = "my-app-claim-batches-queue"
|
|
250
|
+
max_batch_size = 10
|
|
251
|
+
max_batch_timeout = 30
|
|
252
|
+
max_retries = 3
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
## Deployment
|
|
256
|
+
|
|
257
|
+
After defining queue handlers:
|
|
258
|
+
|
|
259
|
+
1. **Create the queues:**
|
|
260
|
+
```bash
|
|
261
|
+
wrangler queues create my-app-ingest-queue
|
|
262
|
+
wrangler queues create my-app-claim-batches-queue
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
2. **Compile:**
|
|
266
|
+
```bash
|
|
267
|
+
quickback compile
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
3. **Deploy:**
|
|
271
|
+
```bash
|
|
272
|
+
wrangler deploy
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
## Error Handling
|
|
276
|
+
|
|
277
|
+
The queue consumer handles errors automatically:
|
|
278
|
+
|
|
279
|
+
- **Return `{ success: true }`** - Message is acknowledged
|
|
280
|
+
- **Return `{ success: false, error: '...' }`** - Message is retried
|
|
281
|
+
- **Throw an exception** - Message is retried
|
|
282
|
+
|
|
283
|
+
After `max_retries` (default 3), failed messages are dropped or sent to a dead-letter queue if configured.
|
|
284
|
+
|
|
285
|
+
```typescript
|
|
286
|
+
execute: async ({ message, db }) => {
|
|
287
|
+
try {
|
|
288
|
+
// Your processing logic
|
|
289
|
+
return { success: true };
|
|
290
|
+
} catch (error) {
|
|
291
|
+
console.error('[Handler] Failed:', error);
|
|
292
|
+
return {
|
|
293
|
+
success: false,
|
|
294
|
+
error: error instanceof Error ? error.message : 'Unknown error'
|
|
295
|
+
};
|
|
296
|
+
}
|
|
297
|
+
}
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
## Combining with Embeddings
|
|
301
|
+
|
|
302
|
+
Custom queue handlers work alongside automatic embeddings. The compiler generates a single queue consumer that handles both:
|
|
303
|
+
|
|
304
|
+
```typescript
|
|
305
|
+
switch (messageType) {
|
|
306
|
+
case 'embedding':
|
|
307
|
+
// Auto-generated embedding handler
|
|
308
|
+
await embeddingQueueHandler(batch, env);
|
|
309
|
+
break;
|
|
310
|
+
case 'process_material':
|
|
311
|
+
// Your custom handler
|
|
312
|
+
await ingestQueueHandler(batch, env);
|
|
313
|
+
break;
|
|
314
|
+
}
|
|
315
|
+
```
|
|
316
|
+
|
|
317
|
+
See [Automatic Embeddings](/stack/vector/embeddings) for configuring auto-embedding on CRUD operations.
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Queues
|
|
2
|
+
|
|
3
|
+
Cloudflare Queues provide reliable, at-least-once message delivery for background processing. Quickback uses queues for embedding generation, webhook delivery, and custom background jobs.
|
|
4
|
+
|
|
5
|
+
## Built-in Queues
|
|
6
|
+
|
|
7
|
+
The compiler auto-configures queues when certain features are enabled:
|
|
8
|
+
|
|
9
|
+
| Feature | Queue Binding | Purpose |
|
|
10
|
+
|---------|--------------|---------|
|
|
11
|
+
| Embeddings | `EMBEDDINGS_QUEUE` | Async embedding generation via Workers AI |
|
|
12
|
+
| Webhooks | `WEBHOOKS_QUEUE` | Async webhook delivery with retry |
|
|
13
|
+
| Custom | User-defined | Your own background processing |
|
|
14
|
+
|
|
15
|
+
## Queue Defaults
|
|
16
|
+
|
|
17
|
+
| Setting | Default |
|
|
18
|
+
|---------|---------|
|
|
19
|
+
| Max batch size | 10 messages |
|
|
20
|
+
| Max batch timeout | 30 seconds |
|
|
21
|
+
| Max retries | 3 |
|
|
22
|
+
|
|
23
|
+
## Custom Queues
|
|
24
|
+
|
|
25
|
+
Define custom queue handlers for background processing like data pipelines, batch jobs, and async workflows. Custom handlers integrate into the same queue consumer as built-in queues.
|
|
26
|
+
|
|
27
|
+
```typescript
|
|
28
|
+
database: defineDatabase("cloudflare-d1", {
|
|
29
|
+
binding: "DB",
|
|
30
|
+
additionalQueues: [
|
|
31
|
+
{
|
|
32
|
+
name: "my-app-processing-queue",
|
|
33
|
+
binding: "PROCESSING_QUEUE",
|
|
34
|
+
maxBatchSize: 5,
|
|
35
|
+
maxBatchTimeout: 60,
|
|
36
|
+
maxRetries: 5,
|
|
37
|
+
},
|
|
38
|
+
],
|
|
39
|
+
})
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
## How It Works
|
|
43
|
+
|
|
44
|
+
All queues share a single Cloudflare Workers queue consumer. The consumer inspects each message's `type` field to dispatch to the correct handler:
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
Message arrives → Check type field
|
|
48
|
+
├─ "embedding" → Embedding handler
|
|
49
|
+
├─ "inbound"/"outbound" → Webhook handler
|
|
50
|
+
└─ "process_material" → Custom handler
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Pages
|
|
54
|
+
|
|
55
|
+
- **[Custom Queue Handlers](/stack/queues/handlers)** — `defineQueue()`, handler context, and generated output
|
|
56
|
+
- **[Using Queues](/stack/queues/using-queues)** — Publishing messages, wrangler config, and deployment
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# Using Queues
|
|
2
|
+
|
|
3
|
+
This page covers the practical aspects of working with Cloudflare Queues in a Quickback project — publishing messages, configuring wrangler, and deploying.
|
|
4
|
+
|
|
5
|
+
## Publishing Messages
|
|
6
|
+
|
|
7
|
+
Send messages to a queue using the Cloudflare Queue binding in your action or route code:
|
|
8
|
+
|
|
9
|
+
```typescript
|
|
10
|
+
// In an action handler
|
|
11
|
+
export const execute: ActionExecutor = async ({ ctx, input, db }) => {
|
|
12
|
+
await ctx.env.PROCESSING_QUEUE.send({
|
|
13
|
+
type: "process_material",
|
|
14
|
+
materialId: input.materialId,
|
|
15
|
+
organizationId: ctx.activeOrgId,
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
return { queued: true };
|
|
19
|
+
};
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
The `type` field is required — the queue consumer uses it to dispatch messages to the correct handler.
|
|
23
|
+
|
|
24
|
+
### Sending to Built-in Queues
|
|
25
|
+
|
|
26
|
+
**Embeddings** are auto-enqueued by the compiler after CRUD operations. You can also manually enqueue:
|
|
27
|
+
|
|
28
|
+
```typescript
|
|
29
|
+
await ctx.env.EMBEDDINGS_QUEUE.send({
|
|
30
|
+
type: "embedding",
|
|
31
|
+
table: "claims",
|
|
32
|
+
id: record.id,
|
|
33
|
+
content: `${record.title} ${record.content}`,
|
|
34
|
+
model: "@cf/baai/bge-base-en-v1.5",
|
|
35
|
+
embeddingColumn: "embedding",
|
|
36
|
+
metadata: { organizationId: ctx.activeOrgId },
|
|
37
|
+
organizationId: ctx.activeOrgId,
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
**Webhooks** are enqueued via `emitWebhookEvent()` — see [Outbound Webhooks](/stack/webhooks/outbound).
|
|
42
|
+
|
|
43
|
+
## Chaining Queues
|
|
44
|
+
|
|
45
|
+
Queue handlers can publish to other queues for multi-stage pipelines:
|
|
46
|
+
|
|
47
|
+
```typescript
|
|
48
|
+
export default defineQueue({
|
|
49
|
+
name: "ingest",
|
|
50
|
+
messageType: "process_material",
|
|
51
|
+
|
|
52
|
+
execute: async ({ message, db, env }) => {
|
|
53
|
+
// Stage 1: Extract data
|
|
54
|
+
const claims = await extractClaims(message.materialId, db);
|
|
55
|
+
|
|
56
|
+
// Stage 2: Send each claim to next queue
|
|
57
|
+
for (const claim of claims) {
|
|
58
|
+
await env.ANALYSIS_QUEUE.send({
|
|
59
|
+
type: "analyze_claim",
|
|
60
|
+
claimId: claim.id,
|
|
61
|
+
});
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
return { success: true };
|
|
65
|
+
},
|
|
66
|
+
});
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Wrangler Configuration
|
|
70
|
+
|
|
71
|
+
Queue bindings are auto-generated in `wrangler.toml`. Here's what the compiler produces:
|
|
72
|
+
|
|
73
|
+
```toml
|
|
74
|
+
# Embeddings queue (auto-configured when embeddings enabled)
|
|
75
|
+
[[queues.producers]]
|
|
76
|
+
queue = "my-app-embeddings-queue"
|
|
77
|
+
binding = "EMBEDDINGS_QUEUE"
|
|
78
|
+
|
|
79
|
+
[[queues.consumers]]
|
|
80
|
+
queue = "my-app-embeddings-queue"
|
|
81
|
+
max_batch_size = 10
|
|
82
|
+
max_batch_timeout = 30
|
|
83
|
+
max_retries = 3
|
|
84
|
+
|
|
85
|
+
# Webhooks queue (auto-configured when webhooks enabled)
|
|
86
|
+
[[queues.producers]]
|
|
87
|
+
queue = "my-app-webhooks-queue"
|
|
88
|
+
binding = "WEBHOOKS_QUEUE"
|
|
89
|
+
|
|
90
|
+
[[queues.consumers]]
|
|
91
|
+
queue = "my-app-webhooks-queue"
|
|
92
|
+
max_batch_size = 10
|
|
93
|
+
max_batch_timeout = 30
|
|
94
|
+
max_retries = 3
|
|
95
|
+
dead_letter_queue = "my-app-webhooks-dlq"
|
|
96
|
+
|
|
97
|
+
# Custom queue (from additionalQueues config)
|
|
98
|
+
[[queues.producers]]
|
|
99
|
+
queue = "my-app-processing-queue"
|
|
100
|
+
binding = "PROCESSING_QUEUE"
|
|
101
|
+
|
|
102
|
+
[[queues.consumers]]
|
|
103
|
+
queue = "my-app-processing-queue"
|
|
104
|
+
max_batch_size = 5
|
|
105
|
+
max_batch_timeout = 60
|
|
106
|
+
max_retries = 5
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Generated Queue Consumer
|
|
110
|
+
|
|
111
|
+
The compiler generates a unified `src/queue-consumer.ts` that handles all message types:
|
|
112
|
+
|
|
113
|
+
```typescript
|
|
114
|
+
// src/queue-consumer.ts (generated)
|
|
115
|
+
export const queue = async (
|
|
116
|
+
batch: MessageBatch<any>,
|
|
117
|
+
env: CloudflareBindings
|
|
118
|
+
): Promise<void> => {
|
|
119
|
+
const messageType = batch.messages[0]?.body?.type;
|
|
120
|
+
|
|
121
|
+
switch (messageType) {
|
|
122
|
+
case "embedding":
|
|
123
|
+
await embeddingQueueHandler(batch, env);
|
|
124
|
+
break;
|
|
125
|
+
case "process_material":
|
|
126
|
+
await ingestQueueHandler(batch, env);
|
|
127
|
+
break;
|
|
128
|
+
default:
|
|
129
|
+
console.warn("[Queue] Unknown message type:", messageType);
|
|
130
|
+
for (const msg of batch.messages) msg.ack();
|
|
131
|
+
}
|
|
132
|
+
};
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
This consumer is exported alongside the `fetch` handler in the Workers entry point.
|
|
136
|
+
|
|
137
|
+
## Deployment
|
|
138
|
+
|
|
139
|
+
### 1. Create Queues
|
|
140
|
+
|
|
141
|
+
Before deploying, create the queues in Cloudflare:
|
|
142
|
+
|
|
143
|
+
```bash
|
|
144
|
+
wrangler queues create my-app-embeddings-queue
|
|
145
|
+
wrangler queues create my-app-processing-queue
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### 2. Compile and Deploy
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
quickback compile
|
|
152
|
+
wrangler deploy
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
The single worker handles both HTTP requests and queue consumption.
|
|
156
|
+
|
|
157
|
+
### 3. Verify
|
|
158
|
+
|
|
159
|
+
Check queue status:
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
wrangler queues list
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
## Error Handling
|
|
166
|
+
|
|
167
|
+
The queue consumer handles errors per-message:
|
|
168
|
+
|
|
169
|
+
- **Return `{ success: true }`** — Message is acknowledged
|
|
170
|
+
- **Return `{ success: false, error: "..." }`** — Message is retried
|
|
171
|
+
- **Throw an exception** — Message is retried
|
|
172
|
+
|
|
173
|
+
After `max_retries` (default 3), failed messages are dropped or sent to a dead-letter queue if configured.
|
|
174
|
+
|
|
175
|
+
## Monitoring
|
|
176
|
+
|
|
177
|
+
Queue metrics are available in the Cloudflare dashboard:
|
|
178
|
+
|
|
179
|
+
- Messages enqueued / processed
|
|
180
|
+
- Retry count
|
|
181
|
+
- Consumer lag
|
|
182
|
+
- Dead-letter queue depth
|
|
183
|
+
|
|
184
|
+
## Cloudflare Only
|
|
185
|
+
|
|
186
|
+
Queues require the Cloudflare runtime. They are not available with the Bun or Node runtimes.
|
|
187
|
+
|
|
188
|
+
## See Also
|
|
189
|
+
|
|
190
|
+
- [Custom Queue Handlers](/stack/queues/handlers) — `defineQueue()` API and handler context
|
|
191
|
+
- [Automatic Embeddings](/stack/vector/embeddings) — How embedding jobs are enqueued
|
|
192
|
+
- [Outbound Webhooks](/stack/webhooks/outbound) — How webhook deliveries are enqueued
|