experimental-a2 0.20.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +55 -0
- package/README.md +1 -1
- package/dist/{actor-C7b-1arn.d.ts → actor-CqibLgjQ.d.ts} +15 -7
- package/dist/actor-CqibLgjQ.d.ts.map +1 -0
- package/dist/actor-client.d.ts +1 -1
- package/dist/actor-client.js +2 -2
- package/dist/actor-react.d.ts +3 -3
- package/dist/actor-react.js +5 -5
- package/dist/{actor-shared-t7w8xIKP.js → actor-shared-DFET5q16.js} +3 -3
- package/dist/actor-shared-DFET5q16.js.map +1 -0
- package/dist/actor.d.ts +1 -1
- package/dist/actor.js +54 -43
- package/dist/actor.js.map +1 -1
- package/dist/{ai-Cz3hqVhk.d.ts → ai-DAu77CDr.d.ts} +8 -8
- package/dist/{ai-Cz3hqVhk.d.ts.map → ai-DAu77CDr.d.ts.map} +1 -1
- package/dist/{ai-mMiWul1T.js → ai-EEfKyc-z.js} +9 -9
- package/dist/ai-EEfKyc-z.js.map +1 -0
- package/dist/ai-client.d.ts +2 -2
- package/dist/ai-client.js +2 -2
- package/dist/ai-fx.d.ts +4 -2
- package/dist/ai-fx.d.ts.map +1 -1
- package/dist/ai-fx.js +214 -67
- package/dist/ai-fx.js.map +1 -1
- package/dist/{ai-harness-CV6rXbB3.d.ts → ai-harness-DTjkAr2T.d.ts} +2 -2
- package/dist/{ai-harness-CV6rXbB3.d.ts.map → ai-harness-DTjkAr2T.d.ts.map} +1 -1
- package/dist/{ai-harness-Clbg5ei7.js → ai-harness-dQeJaR1t.js} +7 -3
- package/dist/ai-harness-dQeJaR1t.js.map +1 -0
- package/dist/ai-pi.d.ts +1 -1
- package/dist/ai-pi.js +3 -3
- package/dist/ai-pi.js.map +1 -1
- package/dist/ai-react.d.ts +5 -5
- package/dist/ai-react.js +2 -2
- package/dist/ai-server.d.ts +14 -17
- package/dist/ai-server.d.ts.map +1 -1
- package/dist/ai-server.js +86 -73
- package/dist/ai-server.js.map +1 -1
- package/dist/ai.d.ts +2 -2
- package/dist/ai.js +2 -2
- package/dist/{client-BhHLoNKF.js → client-BQS4i_V9.js} +29 -6
- package/dist/client-BQS4i_V9.js.map +1 -0
- package/dist/{client-h_DKFAjj.d.ts → client-CUMbS_38.d.ts} +3 -3
- package/dist/{client-h_DKFAjj.d.ts.map → client-CUMbS_38.d.ts.map} +1 -1
- package/dist/client.d.ts +3 -3
- package/dist/client.js +1 -1
- package/dist/{contract-BYKASlCV.js → contract-CWukt01d.js} +2 -2
- package/dist/contract-CWukt01d.js.map +1 -0
- package/dist/index.d.ts +5 -5
- package/dist/index.js +3 -3
- package/dist/{initial-snapshot-Bx-c0pDP.d.ts → initial-snapshot-D3Lve43o.d.ts} +4 -4
- package/dist/initial-snapshot-D3Lve43o.d.ts.map +1 -0
- package/dist/{internal-CscSY7K_.js → internal-CvsrprnU.js} +40 -16
- package/dist/internal-CvsrprnU.js.map +1 -0
- package/dist/otel.d.ts +1 -1
- package/dist/otel.js +1 -1
- package/dist/otel.js.map +1 -1
- package/dist/{react-BBXgHWFc.d.ts → react-BoHvC3v9.d.ts} +4 -4
- package/dist/{react-BBXgHWFc.d.ts.map → react-BoHvC3v9.d.ts.map} +1 -1
- package/dist/react.d.ts +1 -1
- package/dist/react.js +1 -1
- package/dist/{reducer-CqA59Q6U.d.ts → reducer-CYPHsSmd.d.ts} +2 -2
- package/dist/{reducer-CqA59Q6U.d.ts.map → reducer-CYPHsSmd.d.ts.map} +1 -1
- package/dist/{reducer-BAXc8JyF.js → reducer-CjmvZSAb.js} +2 -2
- package/dist/{reducer-BAXc8JyF.js.map → reducer-CjmvZSAb.js.map} +1 -1
- package/dist/runtime-consumer-CIeL33wj.js +12 -0
- package/dist/runtime-consumer-CIeL33wj.js.map +1 -0
- package/dist/scheduler-qstash.d.ts +2 -2
- package/dist/scheduler-qstash.js +3 -3
- package/dist/scheduler-task-X1SToDjP.js.map +1 -1
- package/dist/scheduler-vercel.d.ts +2 -2
- package/dist/scheduler-vercel.js +3 -3
- package/dist/scheduler-vercel.js.map +1 -1
- package/dist/{server-qyb5aJJG.js → server-DtFrWxi3.js} +412 -116
- package/dist/server-DtFrWxi3.js.map +1 -0
- package/dist/{server-DYAATFuT.d.ts → server-qgez0Xny.d.ts} +108 -41
- package/dist/server-qgez0Xny.d.ts.map +1 -0
- package/dist/server.browser.js +1 -1
- package/dist/server.browser.js.map +1 -1
- package/dist/server.d.ts +4 -4
- package/dist/server.js +2 -2
- package/dist/{session-clear-C-QwTXV1.d.ts → session-clear-D1PFe7xl.d.ts} +2 -2
- package/dist/session-clear-D1PFe7xl.d.ts.map +1 -0
- package/dist/{session-clear-DGI4jTQ5.js → session-clear-wHd5GCpZ.js} +5 -4
- package/dist/session-clear-wHd5GCpZ.js.map +1 -0
- package/dist/store-D0MhBvg6.d.ts +532 -0
- package/dist/store-D0MhBvg6.d.ts.map +1 -0
- package/dist/store-N8PXxDAS.js.map +1 -1
- package/dist/store-codec-DwrWlgQx.js +65 -0
- package/dist/store-codec-DwrWlgQx.js.map +1 -0
- package/dist/store-memory.d.ts +1 -1
- package/dist/store-memory.d.ts.map +1 -1
- package/dist/store-memory.js +406 -225
- package/dist/store-memory.js.map +1 -1
- package/dist/{store-polling-7WJV0YWP.js → store-polling-BZt2Rf3D.js} +2 -2
- package/dist/{store-polling-7WJV0YWP.js.map → store-polling-BZt2Rf3D.js.map} +1 -1
- package/dist/store-postgres.d.ts +1 -1
- package/dist/store-postgres.d.ts.map +1 -1
- package/dist/store-postgres.js +1598 -587
- package/dist/store-postgres.js.map +1 -1
- package/dist/store-presence-polling-JuAy2Ck2.js +111 -0
- package/dist/store-presence-polling-JuAy2Ck2.js.map +1 -0
- package/dist/store-redis-http.d.ts +1 -1
- package/dist/store-redis-http.js +4 -4
- package/dist/store-redis-notify-CsJw_hfD.js +3060 -0
- package/dist/store-redis-notify-CsJw_hfD.js.map +1 -0
- package/dist/store-redis.d.ts +1 -1
- package/dist/store-redis.d.ts.map +1 -1
- package/dist/store-redis.js +4 -5
- package/dist/store-redis.js.map +1 -1
- package/dist/store-sqlite.d.ts +1 -1
- package/dist/store-sqlite.d.ts.map +1 -1
- package/dist/store-sqlite.js +527 -299
- package/dist/store-sqlite.js.map +1 -1
- package/dist/stream-buffer-DrTyiIuo.js +57 -0
- package/dist/stream-buffer-DrTyiIuo.js.map +1 -0
- package/dist/{telemetry-Cyfqo6Qt.d.ts → telemetry-CWwsh-Gx.d.ts} +3 -3
- package/dist/telemetry-CWwsh-Gx.d.ts.map +1 -0
- package/dist/{wire-BKxH1vbP.js → wire-CV-Z7wz-.js} +6 -5
- package/dist/wire-CV-Z7wz-.js.map +1 -0
- package/docs/01-quickstart.mdx +21 -15
- package/docs/02-examples.mdx +1 -1
- package/docs/actors/01-introduction.mdx +13 -2
- package/docs/actors/02-concurrency.mdx +1 -1
- package/docs/agents/01-introduction.mdx +1 -1
- package/docs/agents/02-tools.mdx +12 -11
- package/docs/agents/03-conversations.mdx +36 -25
- package/docs/agents/04-harnesses.mdx +14 -7
- package/docs/client/03-presence.mdx +18 -14
- package/docs/contract/01-events.mdx +2 -2
- package/docs/contract/02-state.mdx +18 -14
- package/docs/contract/03-sessions.mdx +32 -25
- package/docs/index.mdx +29 -24
- package/docs/reference/01-core.mdx +7 -6
- package/docs/reference/02-server.mdx +250 -191
- package/docs/reference/04-ai.mdx +33 -23
- package/docs/reference/05-telemetry.mdx +27 -13
- package/docs/reference/06-errors.mdx +25 -19
- package/docs/reference/meta.ts +1 -1
- package/docs/schedulers/01-overview.mdx +8 -8
- package/docs/schedulers/02-vercel-queues.mdx +2 -2
- package/docs/schedulers/03-qstash.mdx +4 -4
- package/docs/server/01-consumers.mdx +418 -0
- package/docs/server/02-timers.mdx +31 -21
- package/docs/server/03-cancellation.mdx +52 -44
- package/docs/server/04-routes.mdx +4 -1
- package/docs/server/05-sql-projections.mdx +391 -0
- package/docs/server/06-guarantees.mdx +61 -51
- package/docs/stores/01-overview.mdx +3 -3
- package/docs/stores/02-redis.mdx +24 -10
- package/docs/stores/03-redis-http.mdx +8 -6
- package/docs/stores/04-postgres.mdx +65 -14
- package/docs/stores/05-sqlite.mdx +5 -3
- package/docs/upgrades/01-overview.mdx +31 -0
- package/docs/upgrades/78-0-22.mdx +115 -0
- package/docs/upgrades/79-0-21.mdx +165 -0
- package/docs/upgrades/80-0-20.mdx +43 -0
- package/docs/upgrades/81-0-19.mdx +33 -0
- package/docs/upgrades/82-0-18.mdx +33 -0
- package/docs/upgrades/83-0-17.mdx +69 -0
- package/docs/upgrades/meta.ts +3 -0
- package/examples/playground/DEPLOY.md +39 -7
- package/examples/playground/app/a0/chat.test.tsx +2 -2
- package/examples/playground/app/a0/sandbox-log.ts +17 -15
- package/examples/playground/app/a0/sandbox.ts +58 -13
- package/examples/playground/app/a0/server.ts +4 -4
- package/examples/playground/app/a2slack/a2slack.css +1067 -0
- package/examples/playground/app/a2slack/c/[channel]/page.tsx +20 -0
- package/examples/playground/app/a2slack/consumers/agent.ts +63 -0
- package/examples/playground/app/a2slack/consumers/feed.ts +164 -0
- package/examples/playground/app/a2slack/consumers/notify.ts +116 -0
- package/examples/playground/app/a2slack/consumers/search.ts +37 -0
- package/examples/playground/app/a2slack/consumers/triage.ts +31 -0
- package/examples/playground/app/a2slack/consumers/welcome.ts +25 -0
- package/examples/playground/app/a2slack/db/a2.ts +209 -0
- package/examples/playground/app/a2slack/db/index.ts +73 -0
- package/examples/playground/app/a2slack/db/migrations/0000_extensions.sql +4 -0
- package/examples/playground/app/a2slack/db/migrations/0001_workspace.sql +74 -0
- package/examples/playground/app/a2slack/db/migrations/0002_a2.sql +215 -0
- package/examples/playground/app/a2slack/db/migrations/0003_a2_batch.sql +67 -0
- package/examples/playground/app/a2slack/db/migrations/0004_a2_session_event_ids.sql +2 -0
- package/examples/playground/app/a2slack/db/migrations/meta/0000_snapshot.json +18 -0
- package/examples/playground/app/a2slack/db/migrations/meta/0001_snapshot.json +500 -0
- package/examples/playground/app/a2slack/db/migrations/meta/0002_snapshot.json +1159 -0
- package/examples/playground/app/a2slack/db/migrations/meta/0003_snapshot.json +1159 -0
- package/examples/playground/app/a2slack/db/migrations/meta/0004_snapshot.json +1165 -0
- package/examples/playground/app/a2slack/db/migrations/meta/_journal.json +41 -0
- package/examples/playground/app/a2slack/db/queries.ts +263 -0
- package/examples/playground/app/a2slack/db/schema.ts +145 -0
- package/examples/playground/app/a2slack/drills/a2slack.drill.ts +708 -0
- package/examples/playground/app/a2slack/drills/harness.ts +244 -0
- package/examples/playground/app/a2slack/events/route.ts +36 -0
- package/examples/playground/app/a2slack/identity.ts +44 -0
- package/examples/playground/app/a2slack/inbox/route.ts +8 -0
- package/examples/playground/app/a2slack/lag/data/route.ts +8 -0
- package/examples/playground/app/a2slack/lag/page.tsx +101 -0
- package/examples/playground/app/a2slack/lag.ts +73 -0
- package/examples/playground/app/a2slack/layout.tsx +43 -0
- package/examples/playground/app/a2slack/messages/route.ts +17 -0
- package/examples/playground/app/a2slack/model.test.ts +208 -0
- package/examples/playground/app/a2slack/model.ts +369 -0
- package/examples/playground/app/a2slack/models.test.ts +72 -0
- package/examples/playground/app/a2slack/page.tsx +10 -0
- package/examples/playground/app/a2slack/scheduler/route.ts +5 -0
- package/examples/playground/app/a2slack/search/route.ts +40 -0
- package/examples/playground/app/a2slack/search-index.ts +108 -0
- package/examples/playground/app/a2slack/server.test.ts +315 -0
- package/examples/playground/app/a2slack/server.ts +55 -0
- package/examples/playground/app/a2slack/sessions.ts +50 -0
- package/examples/playground/app/a2slack/thread/route.ts +8 -0
- package/examples/playground/app/a2slack/ui/ChannelView.tsx +87 -0
- package/examples/playground/app/a2slack/ui/Composer.tsx +117 -0
- package/examples/playground/app/a2slack/ui/DotsShadow.tsx +42 -0
- package/examples/playground/app/a2slack/ui/Feed.tsx +157 -0
- package/examples/playground/app/a2slack/ui/InboxContext.tsx +63 -0
- package/examples/playground/app/a2slack/ui/InboxView.tsx +82 -0
- package/examples/playground/app/a2slack/ui/Menu.tsx +128 -0
- package/examples/playground/app/a2slack/ui/MessageRow.tsx +314 -0
- package/examples/playground/app/a2slack/ui/NameGate.tsx +35 -0
- package/examples/playground/app/a2slack/ui/SearchContext.tsx +146 -0
- package/examples/playground/app/a2slack/ui/SearchResults.tsx +97 -0
- package/examples/playground/app/a2slack/ui/Shell.tsx +34 -0
- package/examples/playground/app/a2slack/ui/Sidebar.tsx +199 -0
- package/examples/playground/app/a2slack/ui/ThreadPanel.tsx +134 -0
- package/examples/playground/app/a2slack/ui/useInbox.ts +71 -0
- package/examples/playground/app/agent/server.ts +3 -3
- package/examples/playground/app/chat/[chatId]/chat-client.tsx +1 -1
- package/examples/playground/app/chat/server.ts +45 -35
- package/examples/playground/app/components/demo-icons.tsx +7 -0
- package/examples/playground/app/components/demos.tsx +7 -0
- package/examples/playground/app/orders/[orderId]/order-client.tsx +2 -2
- package/examples/playground/app/orders/page.tsx +1 -1
- package/examples/playground/app/orders/server.ts +49 -30
- package/examples/playground/app/recovery/[recoveryId]/recovery-client.tsx +14 -15
- package/examples/playground/app/recovery/page.tsx +4 -4
- package/examples/playground/app/recovery/server.ts +23 -32
- package/examples/playground/drizzle.config.ts +14 -0
- package/examples/playground/lib/store.ts +1 -1
- package/examples/playground/next.config.ts +6 -1
- package/examples/playground/package.json +12 -1
- package/examples/playground/scripts/migrate.mjs +31 -0
- package/examples/playground/vercel.json +9 -0
- package/examples/playground/vitest.drills.config.ts +32 -0
- package/package.json +5 -2
- package/sql/postgres/drizzle-schema.ts +209 -0
- package/sql/postgres/schema.sql +227 -67
- package/src/actor-shared.ts +10 -2
- package/src/actor.ts +101 -71
- package/src/ai-agent.ts +162 -111
- package/src/ai-control-server.ts +8 -8
- package/src/ai-fx.ts +324 -82
- package/src/ai-harness.ts +17 -2
- package/src/ai-pi.ts +2 -2
- package/src/ai-server.ts +2 -2
- package/src/ai.ts +7 -7
- package/src/client.ts +19 -2
- package/src/contract.ts +1 -1
- package/src/deterministic-id.ts +27 -4
- package/src/errors.ts +8 -5
- package/src/idempotent-replay.ts +0 -1
- package/src/index.ts +1 -1
- package/src/internal.ts +57 -16
- package/src/otel.ts +1 -1
- package/src/postgres-schema.ts +98 -57
- package/src/presence-recovery.ts +18 -11
- package/src/presence-tombstones.ts +52 -0
- package/src/push-envelope.ts +20 -0
- package/src/runtime-consumer.ts +10 -0
- package/src/scheduler-task.ts +2 -2
- package/src/scheduler-vercel.ts +1 -1
- package/src/server.browser.ts +1 -1
- package/src/server.ts +819 -218
- package/src/session-socket.ts +1 -1
- package/src/store-codec.ts +66 -0
- package/src/store-memory.ts +604 -286
- package/src/store-postgres.ts +2006 -857
- package/src/store-presence-polling.ts +59 -34
- package/src/store-redis-core.ts +1961 -1333
- package/src/store-redis-notify.ts +76 -26
- package/src/store-redis.ts +2 -3
- package/src/store-sqlite.ts +915 -516
- package/src/store.ts +241 -102
- package/src/telemetry.ts +2 -1
- package/src/wire.ts +1 -0
- package/dist/actor-C7b-1arn.d.ts.map +0 -1
- package/dist/actor-shared-t7w8xIKP.js.map +0 -1
- package/dist/ai-harness-Clbg5ei7.js.map +0 -1
- package/dist/ai-mMiWul1T.js.map +0 -1
- package/dist/client-BhHLoNKF.js.map +0 -1
- package/dist/contract-BYKASlCV.js.map +0 -1
- package/dist/idempotent-replay-JdgMQRRm.js +0 -21
- package/dist/idempotent-replay-JdgMQRRm.js.map +0 -1
- package/dist/initial-snapshot-Bx-c0pDP.d.ts.map +0 -1
- package/dist/internal-CscSY7K_.js.map +0 -1
- package/dist/server-DYAATFuT.d.ts.map +0 -1
- package/dist/server-qyb5aJJG.js.map +0 -1
- package/dist/session-clear-C-QwTXV1.d.ts.map +0 -1
- package/dist/session-clear-DGI4jTQ5.js.map +0 -1
- package/dist/store-CyLUy86v.d.ts +0 -404
- package/dist/store-CyLUy86v.d.ts.map +0 -1
- package/dist/store-codec-DTG0Ftek.js +0 -10
- package/dist/store-codec-DTG0Ftek.js.map +0 -1
- package/dist/store-presence-polling-B1ixeqIS.js +0 -94
- package/dist/store-presence-polling-B1ixeqIS.js.map +0 -1
- package/dist/store-redis-notify-CM0AfLY0.js +0 -2544
- package/dist/store-redis-notify-CM0AfLY0.js.map +0 -1
- package/dist/stream-buffer-Cx7AAPGq.js +0 -23
- package/dist/stream-buffer-Cx7AAPGq.js.map +0 -1
- package/dist/telemetry-Cyfqo6Qt.d.ts.map +0 -1
- package/dist/wire-BKxH1vbP.js.map +0 -1
- package/docs/server/01-handlers.mdx +0 -227
|
@@ -0,0 +1,391 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: SQL projections
|
|
3
|
+
description: Keep tables in your own database from A2's logs with a consumer, so lists, filters, and joins are SQL queries.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
## The tables
|
|
7
|
+
|
|
8
|
+
```ts db/schema.ts
|
|
9
|
+
import { bigint, pgTable, primaryKey, text } from 'drizzle-orm/pg-core'
|
|
10
|
+
|
|
11
|
+
/** The order list: one row per order. */
|
|
12
|
+
export const orderList = pgTable('order_list', {
|
|
13
|
+
orderId: text('order_id').primaryKey(),
|
|
14
|
+
shopId: text('shop_id').notNull(),
|
|
15
|
+
status: text('status').notNull(),
|
|
16
|
+
})
|
|
17
|
+
|
|
18
|
+
/** The last event each consumer applied from each session's log. */
|
|
19
|
+
export const positions = pgTable(
|
|
20
|
+
'positions',
|
|
21
|
+
{
|
|
22
|
+
consumer: text('consumer').notNull(),
|
|
23
|
+
sessionId: text('session_id').notNull(),
|
|
24
|
+
logIndex: bigint('log_index', { mode: 'number' }).notNull(),
|
|
25
|
+
},
|
|
26
|
+
(table) => [primaryKey({ columns: [table.consumer, table.sessionId] })],
|
|
27
|
+
)
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
```ts db/client.ts
|
|
31
|
+
import { drizzle } from 'drizzle-orm/node-postgres'
|
|
32
|
+
import * as schema from '@/db/schema'
|
|
33
|
+
|
|
34
|
+
export const db = drizzle({
|
|
35
|
+
connection: { connectionString: process.env.DATABASE_URL },
|
|
36
|
+
schema,
|
|
37
|
+
})
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
A2 keeps each session's log. When people list, filter, and join across
|
|
41
|
+
sessions, read tables in your database instead, kept from the logs by a
|
|
42
|
+
[consumer](/server/consumers): a projection. This page keeps an order list
|
|
43
|
+
with Drizzle on Postgres, as an example. The pattern is the same with any SQL
|
|
44
|
+
database and client.
|
|
45
|
+
|
|
46
|
+
The order list has a row per order. `positions` has a row per consumer and
|
|
47
|
+
session: the index of the last event that consumer applied to your tables.
|
|
48
|
+
Your migrations create both, like any table. A2's own tables can live in the
|
|
49
|
+
same database; see [Postgres](/stores/postgres#with-drizzle).
|
|
50
|
+
|
|
51
|
+
## Why a position row
|
|
52
|
+
|
|
53
|
+
A2 keeps each consumer's cursor, its place in a session's log, in A2's store,
|
|
54
|
+
and moves it after `run` returns. It fences its own writes by attempt: an
|
|
55
|
+
append, a returned batch, or a cursor move from an attempt that lost its claim
|
|
56
|
+
is refused. Your SQL commits in your own transaction, which A2 can't fence or
|
|
57
|
+
make atomic with its cursor. Two things follow:
|
|
58
|
+
|
|
59
|
+
- **A crash between your commit and the cursor move retries the event.** Your
|
|
60
|
+
rows are in, the cursor isn't, and A2 runs the event again.
|
|
61
|
+
- **An attempt that lost its claim can still commit.** Its claim lapsed while
|
|
62
|
+
it ran, a newer attempt applied the event and the ones after it, and then
|
|
63
|
+
the old attempt commits an older event over newer rows.
|
|
64
|
+
|
|
65
|
+
The position row fences both. It commits in the same transaction as your rows,
|
|
66
|
+
and a transaction writes only when it moves the position forward, so each
|
|
67
|
+
event lands in your tables once, in log order.
|
|
68
|
+
|
|
69
|
+
## Claim the event first
|
|
70
|
+
|
|
71
|
+
```ts server/order-list.ts
|
|
72
|
+
import { eq, lt } from 'drizzle-orm'
|
|
73
|
+
import { consumer } from 'experimental-a2/server'
|
|
74
|
+
import { orders } from '@/contracts'
|
|
75
|
+
import { db } from '@/db/client'
|
|
76
|
+
import { orderList, positions } from '@/db/schema'
|
|
77
|
+
|
|
78
|
+
const name = 'order-list-v1'
|
|
79
|
+
|
|
80
|
+
export const orderListTable = consumer({
|
|
81
|
+
contract: orders,
|
|
82
|
+
name,
|
|
83
|
+
lane: 'order-list',
|
|
84
|
+
backfill: true,
|
|
85
|
+
}).run(async ({ event }) => {
|
|
86
|
+
await db.transaction(async (tx) => {
|
|
87
|
+
const claimed = await tx
|
|
88
|
+
.insert(positions)
|
|
89
|
+
.values({ consumer: name, sessionId: event.sessionId, logIndex: event.index })
|
|
90
|
+
.onConflictDoUpdate({
|
|
91
|
+
target: [positions.consumer, positions.sessionId],
|
|
92
|
+
set: { logIndex: event.index },
|
|
93
|
+
setWhere: lt(positions.logIndex, event.index),
|
|
94
|
+
})
|
|
95
|
+
.returning({ logIndex: positions.logIndex })
|
|
96
|
+
// this event, or a later one, is already in:
|
|
97
|
+
if (claimed.length === 0) return
|
|
98
|
+
|
|
99
|
+
if (event.type === 'created') {
|
|
100
|
+
await tx.insert(orderList).values({
|
|
101
|
+
orderId: event.sessionId,
|
|
102
|
+
shopId: event.payload.shopId,
|
|
103
|
+
status: 'created',
|
|
104
|
+
})
|
|
105
|
+
} else {
|
|
106
|
+
await tx
|
|
107
|
+
.update(orderList)
|
|
108
|
+
.set({ status: event.type })
|
|
109
|
+
.where(eq(orderList.orderId, event.sessionId))
|
|
110
|
+
}
|
|
111
|
+
})
|
|
112
|
+
})
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The transaction's first statement claims the event. It inserts the session's
|
|
116
|
+
position, or moves it to this event's index if it's behind, and returns the
|
|
117
|
+
row when it did. No row back means this event, or a later one, is already in
|
|
118
|
+
your tables: return, and the transaction changes nothing. Otherwise write your
|
|
119
|
+
rows. They commit with the position, or none of it does.
|
|
120
|
+
|
|
121
|
+
In plain SQL, it's the same statement on Postgres and SQLite:
|
|
122
|
+
|
|
123
|
+
```sql
|
|
124
|
+
insert into positions (consumer, session_id, log_index)
|
|
125
|
+
values ($1, $2, $3)
|
|
126
|
+
on conflict (consumer, session_id) do update
|
|
127
|
+
set log_index = excluded.log_index
|
|
128
|
+
where positions.log_index < excluded.log_index
|
|
129
|
+
returning 1
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
It comes first so a duplicate waits before it does any work. On Postgres, at
|
|
133
|
+
the default read committed level, two transactions that claim the same event
|
|
134
|
+
can't both get the row back: the second one's insert waits for the first to
|
|
135
|
+
commit, then checks the condition again against the committed position, finds
|
|
136
|
+
it caught up, and returns nothing. If the first rolls back, the second claims
|
|
137
|
+
the event. Reading the position first with `select ... for update` doesn't
|
|
138
|
+
cover a session's first event: there's no row to lock yet, so both
|
|
139
|
+
transactions read nothing and both apply it.
|
|
140
|
+
|
|
141
|
+
## The consumer
|
|
142
|
+
|
|
143
|
+
```ts server/orders.ts
|
|
144
|
+
import { createServer } from 'experimental-a2/server'
|
|
145
|
+
import { postgres } from 'experimental-a2/store-postgres'
|
|
146
|
+
import { orders } from '@/contracts'
|
|
147
|
+
import { orderListTable } from '@/server/order-list'
|
|
148
|
+
|
|
149
|
+
export const ordersServer = createServer({
|
|
150
|
+
contract: orders,
|
|
151
|
+
store: postgres({ connectionString: process.env.DATABASE_URL }),
|
|
152
|
+
consumers: [orderListTable],
|
|
153
|
+
})
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
- **A string lane runs a session's events in order.** One at a time, in log
|
|
157
|
+
order, one transaction each. Sessions run independently.
|
|
158
|
+
- **`backfill` fills it from history.** It reads each session's log from its
|
|
159
|
+
first event, not only events appended after it exists. A session starts on
|
|
160
|
+
its next append, or when you [consume](#read-your-writes) it. Nothing
|
|
161
|
+
enumerates sessions, so to fill the table for orders that never change
|
|
162
|
+
again, consume each one.
|
|
163
|
+
- **A rebuild is a new name.** See [Rebuild it](#rebuild-it).
|
|
164
|
+
|
|
165
|
+
## Read it
|
|
166
|
+
|
|
167
|
+
```ts server/order-list.ts
|
|
168
|
+
export function shopOrders(shopId: string) {
|
|
169
|
+
return db.select().from(orderList).where(eq(orderList.shopId, shopId))
|
|
170
|
+
}
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
Lists, filters, and joins are queries against your tables.
|
|
174
|
+
|
|
175
|
+
## Read your writes
|
|
176
|
+
|
|
177
|
+
```ts server/place-order.ts
|
|
178
|
+
import { orderListTable, shopOrders } from '@/server/order-list'
|
|
179
|
+
import { ordersServer } from '@/server/orders'
|
|
180
|
+
|
|
181
|
+
export async function placeOrder(orderId: string, shopId: string) {
|
|
182
|
+
const order = ordersServer.session(orderId)
|
|
183
|
+
const [created] = await order.append({
|
|
184
|
+
type: 'created',
|
|
185
|
+
payload: { shopId, items: [] },
|
|
186
|
+
})
|
|
187
|
+
await order.consume(orderListTable, { through: created.index })
|
|
188
|
+
// the new order is in the list:
|
|
189
|
+
return shopOrders(shopId)
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
Your tables lag the log a little. Before a read that has to include a write
|
|
194
|
+
you just made, [consume](/server/consumers#run-a-consumer-now-with-consume)
|
|
195
|
+
the projection through the index the append returned. It returns once your
|
|
196
|
+
tables have that event.
|
|
197
|
+
|
|
198
|
+
A consumer that reads your tables waits the same way:
|
|
199
|
+
|
|
200
|
+
```ts server/receipts.ts
|
|
201
|
+
import { consumer } from 'experimental-a2/server'
|
|
202
|
+
import { orders } from '@/contracts'
|
|
203
|
+
import { orderListTable } from '@/server/order-list'
|
|
204
|
+
|
|
205
|
+
export const receipts = consumer({
|
|
206
|
+
contract: orders,
|
|
207
|
+
name: 'receipts',
|
|
208
|
+
types: ['shop.notified'],
|
|
209
|
+
}).run(async ({ event, session }) => {
|
|
210
|
+
await session.consume(orderListTable, { through: event.index })
|
|
211
|
+
// the order list has this event; read it:
|
|
212
|
+
// const [order] = await db.select().from(orderList).where(eq(orderList.orderId, event.sessionId))
|
|
213
|
+
})
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
Pass `event.index - 1` instead when the consumer applies the triggering event
|
|
217
|
+
itself and needs your tables only up to it. A consumer can't consume itself.
|
|
218
|
+
|
|
219
|
+
`consume` takes the session's lock in A2's store on every call, even when your
|
|
220
|
+
tables are caught up, and while another invocation holds the next event, it
|
|
221
|
+
polls until that one finishes. On a hot path, read your position row first,
|
|
222
|
+
and consume only when it's behind the index you need:
|
|
223
|
+
|
|
224
|
+
```ts server/read-through.ts
|
|
225
|
+
import { and, eq } from 'drizzle-orm'
|
|
226
|
+
import { db } from '@/db/client'
|
|
227
|
+
import { positions } from '@/db/schema'
|
|
228
|
+
import { orderListTable } from '@/server/order-list'
|
|
229
|
+
import { ordersServer } from '@/server/orders'
|
|
230
|
+
|
|
231
|
+
export async function readThrough(orderId: string, index: number) {
|
|
232
|
+
const [position] = await db
|
|
233
|
+
.select({ logIndex: positions.logIndex })
|
|
234
|
+
.from(positions)
|
|
235
|
+
.where(
|
|
236
|
+
and(
|
|
237
|
+
eq(positions.consumer, orderListTable.name),
|
|
238
|
+
eq(positions.sessionId, orderId),
|
|
239
|
+
),
|
|
240
|
+
)
|
|
241
|
+
if ((position?.logIndex ?? 0) < index) {
|
|
242
|
+
await ordersServer.session(orderId).consume(orderListTable, { through: index })
|
|
243
|
+
}
|
|
244
|
+
}
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
## Live views from the table
|
|
248
|
+
|
|
249
|
+
```ts server/order-view.ts
|
|
250
|
+
import { and, eq } from 'drizzle-orm'
|
|
251
|
+
import { db } from '@/db/client'
|
|
252
|
+
import { orderList, positions } from '@/db/schema'
|
|
253
|
+
import { orderListTable } from '@/server/order-list'
|
|
254
|
+
|
|
255
|
+
/** One order's row, and the log index it reflects. */
|
|
256
|
+
export function orderView(orderId: string) {
|
|
257
|
+
return db.transaction(
|
|
258
|
+
async (tx) => {
|
|
259
|
+
const [position] = await tx
|
|
260
|
+
.select({ logIndex: positions.logIndex })
|
|
261
|
+
.from(positions)
|
|
262
|
+
.where(
|
|
263
|
+
and(
|
|
264
|
+
eq(positions.consumer, orderListTable.name),
|
|
265
|
+
eq(positions.sessionId, orderId),
|
|
266
|
+
),
|
|
267
|
+
)
|
|
268
|
+
const [order] = await tx
|
|
269
|
+
.select()
|
|
270
|
+
.from(orderList)
|
|
271
|
+
.where(eq(orderList.orderId, orderId))
|
|
272
|
+
return { order, through: position?.logIndex ?? 0 }
|
|
273
|
+
},
|
|
274
|
+
{ isolationLevel: 'repeatable read' },
|
|
275
|
+
)
|
|
276
|
+
}
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
A view of one session can load its rows from your tables and stay live with
|
|
280
|
+
the session's [stream](/client/react). Read the rows and the position in one
|
|
281
|
+
snapshot, a repeatable read transaction on Postgres, and return them together.
|
|
282
|
+
The client follows the stream and applies only the events after `through`, so
|
|
283
|
+
it neither misses an event nor applies one twice.
|
|
284
|
+
|
|
285
|
+
## SQLite
|
|
286
|
+
|
|
287
|
+
```ts server/order-list-sqlite.ts
|
|
288
|
+
import { DatabaseSync } from 'node:sqlite'
|
|
289
|
+
import { consumer } from 'experimental-a2/server'
|
|
290
|
+
import { orders } from '@/contracts'
|
|
291
|
+
|
|
292
|
+
// your migrations create order_list and positions in this file:
|
|
293
|
+
const db = new DatabaseSync('.a2/app.db')
|
|
294
|
+
const name = 'order-list-v1'
|
|
295
|
+
|
|
296
|
+
const claim = db.prepare(`
|
|
297
|
+
insert into positions (consumer, session_id, log_index)
|
|
298
|
+
values (?, ?, ?)
|
|
299
|
+
on conflict (consumer, session_id) do update
|
|
300
|
+
set log_index = excluded.log_index
|
|
301
|
+
where positions.log_index < excluded.log_index
|
|
302
|
+
returning 1
|
|
303
|
+
`)
|
|
304
|
+
const insertOrder = db.prepare(
|
|
305
|
+
'insert into order_list (order_id, shop_id, status) values (?, ?, ?)',
|
|
306
|
+
)
|
|
307
|
+
const setStatus = db.prepare(
|
|
308
|
+
'update order_list set status = ? where order_id = ?',
|
|
309
|
+
)
|
|
310
|
+
|
|
311
|
+
export const orderListTable = consumer({
|
|
312
|
+
contract: orders,
|
|
313
|
+
name,
|
|
314
|
+
lane: 'order-list',
|
|
315
|
+
backfill: true,
|
|
316
|
+
}).run(async ({ event }) => {
|
|
317
|
+
// synchronous from begin to commit, with nothing awaited in between:
|
|
318
|
+
db.exec('begin immediate')
|
|
319
|
+
try {
|
|
320
|
+
if (claim.get(name, event.sessionId, event.index) !== undefined) {
|
|
321
|
+
if (event.type === 'created') {
|
|
322
|
+
insertOrder.run(event.sessionId, event.payload.shopId, 'created')
|
|
323
|
+
} else {
|
|
324
|
+
setStatus.run(event.type, event.sessionId)
|
|
325
|
+
}
|
|
326
|
+
}
|
|
327
|
+
db.exec('commit')
|
|
328
|
+
} catch (error) {
|
|
329
|
+
db.exec('rollback')
|
|
330
|
+
throw error
|
|
331
|
+
}
|
|
332
|
+
})
|
|
333
|
+
```
|
|
334
|
+
|
|
335
|
+
The claim is the same statement. Run the transaction synchronously, from
|
|
336
|
+
`begin immediate` to `commit`, with nothing awaited in between:
|
|
337
|
+
`node:sqlite`'s `DatabaseSync` runs each statement on the calling thread, so
|
|
338
|
+
nothing else runs in the middle, and two of your own transactions never
|
|
339
|
+
interleave on one handle.
|
|
340
|
+
|
|
341
|
+
It's required when your tables share a file with A2's
|
|
342
|
+
[SQLite store](/stores/sqlite). A2 writes on the same thread, through its own
|
|
343
|
+
handle, and SQLite lets one writer hold a file at a time. If your transaction
|
|
344
|
+
awaited while it held the file, an A2 write to it would block the thread on
|
|
345
|
+
A2's five-second busy timeout, and your transaction couldn't reach `commit`
|
|
346
|
+
until that write gave up.
|
|
347
|
+
|
|
348
|
+
## Throughput and lag
|
|
349
|
+
|
|
350
|
+
A string lane runs a session's events one at a time, one transaction each, so
|
|
351
|
+
a projection keeps up with a session as long as its transactions outpace that
|
|
352
|
+
session's appends. In the playground's a2slack drills, with Redis and Postgres
|
|
353
|
+
on the same machine, the feed projection, which keeps a whole workspace in one
|
|
354
|
+
session, ran an event in about 0.4 ms and rebuilt at about 1,800 events a
|
|
355
|
+
second. Posting 10,000 messages in under a second put it up to about six
|
|
356
|
+
seconds behind. A remote database adds at least a round trip to every event.
|
|
357
|
+
|
|
358
|
+
Watch it with each `a2.event` span's `a2.event.lag`, how many events the
|
|
359
|
+
session had appended after the one it claimed, and `a2.event.age_ms`, how old
|
|
360
|
+
that event was when its run started, grouped by `a2.event.consumer`. See
|
|
361
|
+
[Telemetry](/reference/telemetry#the-span-catalogue). When a session's appends
|
|
362
|
+
keep outrunning its projection, keep each transaction to the claim and your
|
|
363
|
+
writes, and do slower work, like calling an API, in a consumer of its own.
|
|
364
|
+
|
|
365
|
+
## Rebuild it
|
|
366
|
+
|
|
367
|
+
To change what a projection writes, or to fill it again, give its consumer a
|
|
368
|
+
new name, such as `order-list-v2`. A new name is a new consumer: it has no
|
|
369
|
+
cursors in A2 and no rows in `positions`, and `backfill` fills it from each
|
|
370
|
+
session's first event, as each session next runs.
|
|
371
|
+
|
|
372
|
+
Point it at new tables, created by the migration that ships it, and switch
|
|
373
|
+
your reads to them once it has caught up. A deployment still running
|
|
374
|
+
`order-list-v1` keeps writing to the old ones until it's gone. To reuse the
|
|
375
|
+
old tables instead, truncate them, along with the old name's rows in
|
|
376
|
+
`positions`, in a migration that runs once no deployment runs the old name:
|
|
377
|
+
an earlier deployment that still does keeps writing to tables you emptied.
|
|
378
|
+
|
|
379
|
+
## Point in time
|
|
380
|
+
|
|
381
|
+
The order list holds each order as of its latest event. To read one as of an
|
|
382
|
+
earlier index, such as next to an old event, keep a versions table too: a row
|
|
383
|
+
per event, keyed by session and log index, written in the same transaction as
|
|
384
|
+
the rest. The row as of index N is the newest at or below it:
|
|
385
|
+
|
|
386
|
+
```sql
|
|
387
|
+
select status from order_versions
|
|
388
|
+
where order_id = $1 and log_index <= $2
|
|
389
|
+
order by log_index desc
|
|
390
|
+
limit 1
|
|
391
|
+
```
|
|
@@ -1,61 +1,68 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Guarantees
|
|
3
|
-
description: Appends are effectively-once,
|
|
3
|
+
description: Appends are effectively-once, consumers are at-least-once, and retries keep event identity across crashes.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
## In short
|
|
7
7
|
|
|
8
|
-
- **Appends are effectively-once.** Event ids deduplicate writes
|
|
9
|
-
before its
|
|
10
|
-
|
|
11
|
-
|
|
8
|
+
- **Appends are effectively-once.** Event ids deduplicate writes within a
|
|
9
|
+
session. A batch commits before its consumers start; they may start before
|
|
10
|
+
`append` resolves.
|
|
11
|
+
- **Consumers are at-least-once.** They may run again after a crash, so
|
|
12
|
+
[side effects use idempotency keys](/server/consumers#consumers-can-run-twice).
|
|
12
13
|
- **The log stays ordered.** Every session has one immutable append order.
|
|
13
|
-
- **
|
|
14
|
-
Only events in the same lane run one at a time in log order.
|
|
14
|
+
- **Consumers are concurrent by default.** Log order does not serialize them.
|
|
15
|
+
Only events in the same lane of one consumer run one at a time in log order.
|
|
16
|
+
- **Every consumer keeps its own place.** One consumer's failures, retries, and
|
|
17
|
+
dead letters never hold up another.
|
|
15
18
|
|
|
16
19
|
## One event, many attempts
|
|
17
20
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
+
A consumer keeps a durable cursor in every session: where it has gotten to,
|
|
22
|
+
and the event it runs next, its head. An event is pending for a consumer until
|
|
23
|
+
that consumer's cursor moves past it. Every wakeup (an append, a scheduler
|
|
24
|
+
watchdog, or `server.drain()`) claims the session's eligible heads, and the
|
|
25
|
+
cursors decide what remains. Reads never dispatch consumers.
|
|
21
26
|
|
|
22
27
|
Each claim expires. If its holder dies, the claim lapses and the next wakeup
|
|
23
|
-
claims the
|
|
28
|
+
claims the head again with a larger `ctx.attempt`. The retry gets the same
|
|
24
29
|
event id, index, type, and payload. A kill after the claim can use up an
|
|
25
|
-
attempt before your code runs, and that attempt counts toward the
|
|
26
|
-
[failure budget](#when-a-
|
|
30
|
+
attempt before your code runs, and that attempt counts toward the head's
|
|
31
|
+
[failure budget](#when-a-consumer-keeps-failing). An older attempt can never
|
|
27
32
|
overwrite a newer one's completion or failure.
|
|
28
33
|
|
|
29
34
|
:::note[Recovery is not an event]
|
|
30
35
|
A2 does not append `recovered` or `continued`. Recovery retries the same event.
|
|
31
|
-
Your log contains only application facts.
|
|
36
|
+
Your log contains only application facts; cursors live beside it.
|
|
32
37
|
:::
|
|
33
38
|
|
|
34
39
|
## Immediate and dependent events
|
|
35
40
|
|
|
36
|
-
[`ctx.session.append(name, ...events)`](/server/
|
|
41
|
+
[`ctx.session.append(name, ...events)`](/server/consumers#append-what-should-happen-now)
|
|
37
42
|
commits immediately: its events stay in the log if the parent later crashes,
|
|
38
43
|
and run concurrently with it unless both share a lane.
|
|
39
|
-
[Returned events](/server/
|
|
40
|
-
with the
|
|
44
|
+
[Returned events](/server/consumers#return-what-happens-after-success) commit
|
|
45
|
+
with the cursor's advance past the parent, in one transaction.
|
|
41
46
|
|
|
42
|
-
If
|
|
47
|
+
If `run` throws or dies before that transaction, neither the completion
|
|
43
48
|
nor the returned events exist, and the retry produces the same deterministic
|
|
44
49
|
ids. If the transaction commits but its acknowledgement is lost, the retry gets
|
|
45
50
|
the committed rows back. A partial or changed batch is rejected.
|
|
46
51
|
|
|
47
52
|
The transaction covers only A2's log. An HTTP call, database write, or other
|
|
48
53
|
external effect can't join it, so a crash after the effect but before
|
|
49
|
-
completion repeats it. Use `event.id` as an idempotency key
|
|
50
|
-
|
|
54
|
+
completion repeats it. Use `event.id` as an idempotency key, with the session
|
|
55
|
+
when ids can repeat across sessions (see
|
|
56
|
+
[Consumers can run twice](/server/consumers#consumers-can-run-twice)), or add
|
|
57
|
+
fencing in the external system.
|
|
51
58
|
|
|
52
59
|
## Lanes
|
|
53
60
|
|
|
54
|
-
[Lanes](/server/
|
|
55
|
-
lowest-index unfinished event in a lane is eligible, so a
|
|
56
|
-
events in that lane, not the whole session
|
|
57
|
-
|
|
58
|
-
affects only new events.
|
|
61
|
+
[Lanes](/server/consumers#lanes) are serial groups of one consumer in one
|
|
62
|
+
session. Only the lowest-index unfinished event in a lane is eligible, so a
|
|
63
|
+
failure blocks later events in that lane, not the whole session and not other
|
|
64
|
+
consumers. Unlaned events and different lanes run concurrently. A2 stores each
|
|
65
|
+
consumer's lane at append, so a changed lane resolver affects only new events.
|
|
59
66
|
|
|
60
67
|
A lane controls A2 dispatch, not the outside world. A2 claims every eligible
|
|
61
68
|
event, with no global concurrency limit or priority queue. Put capacity limits
|
|
@@ -64,55 +71,57 @@ runtime.
|
|
|
64
71
|
|
|
65
72
|
## Append and scheduling
|
|
66
73
|
|
|
67
|
-
A top-level append commits before any wakeup. A2 then starts the
|
|
68
|
-
inline and, with a scheduler, arms a watchdog in parallel.
|
|
69
|
-
for the queue, and
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
74
|
+
A top-level append commits before any wakeup. A2 then starts the consumers
|
|
75
|
+
inline and, with a scheduler, arms a watchdog in parallel. Consumers never wait
|
|
76
|
+
for the queue, and one may start before `append` returns. `append` waits up to
|
|
77
|
+
two seconds for the watchdog send, never for consumers. A failed arm records
|
|
78
|
+
`a2.append.armed = false` on the trace. Consumer appends and returned events
|
|
79
|
+
join the running drain and add no arm.
|
|
73
80
|
|
|
74
81
|
[`append.dispatch`](/schedulers/overview#dispatch-in-a-fresh-invocation) skips
|
|
75
82
|
the inline drain and hands the work to the scheduler instead.
|
|
76
83
|
|
|
77
|
-
An event
|
|
78
|
-
|
|
84
|
+
An event no consumer takes starts no work. It starts no drain or arm unless
|
|
85
|
+
older work in the session is still pending. A dead letter is not pending: it
|
|
86
|
+
waits for a redrive, which wakes the session itself.
|
|
79
87
|
|
|
80
88
|
## Causal trees are durable
|
|
81
89
|
|
|
82
|
-
Every event a
|
|
83
|
-
event's index and
|
|
84
|
-
of one
|
|
85
|
-
keeps its original cause.
|
|
90
|
+
Every event a consumer appends or returns stores a `cause`: the consumer, its
|
|
91
|
+
lane, the triggering event's index, and the attempt. Top-level appends store
|
|
92
|
+
`null`. Children of one run are siblings, nested runs add levels, and a
|
|
93
|
+
retried child keeps its original cause.
|
|
86
94
|
|
|
87
95
|
## A chain that crosses the function timeout
|
|
88
96
|
|
|
89
|
-
A function killed mid-
|
|
90
|
-
fresh invocation: same event, new attempt. Events already
|
|
91
|
-
|
|
97
|
+
A function killed mid-run restarts that consumer's `run` from its entry point
|
|
98
|
+
in a fresh invocation: same event, new attempt. Events a cursor already moved
|
|
99
|
+
past stay done, so a chain of events can outlast one invocation, but each run
|
|
92
100
|
must fit in a fresh one: an attempt that runs out of time is a failed attempt,
|
|
93
|
-
and a
|
|
101
|
+
and a run that never fits dead-letters after ten. Split longer work into
|
|
94
102
|
smaller events or increase the function duration. On Vercel, A2 caps the claim window
|
|
95
103
|
at the function deadline when the platform exposes it, so a retry starts
|
|
96
104
|
promptly after a timeout. Once the deadline passes, A2 claims nothing more in
|
|
97
105
|
that invocation, so a timeout costs the event one attempt.
|
|
98
106
|
|
|
99
|
-
## When a
|
|
107
|
+
## When a consumer keeps failing
|
|
100
108
|
|
|
101
|
-
A caught failure counts against that
|
|
102
|
-
other eligible
|
|
109
|
+
A caught failure counts against that consumer's head, releases its claim, and
|
|
110
|
+
leaves other eligible heads running. A2 holds it back for a delay that starts at
|
|
103
111
|
one second and doubles with each failed attempt up to 30 seconds, then retries
|
|
104
112
|
it on the session's next wake. With a scheduler, that's a few seconds at first,
|
|
105
113
|
then the delay itself. An attempt that ends before it finishes, because its
|
|
106
114
|
process timed out, crashed, or lost its claim, counts too, once the next
|
|
107
|
-
attempt takes its claim over. Ten failed attempts dead-letter the event
|
|
108
|
-
events in its lane stay blocked, while unlaned events
|
|
115
|
+
attempt takes its claim over. Ten failed attempts dead-letter the event for
|
|
116
|
+
that consumer: later events in its lane stay blocked, while unlaned events,
|
|
117
|
+
other lanes, and other consumers continue.
|
|
109
118
|
A crash costs every event it interrupts one attempt, so a session whose work
|
|
110
119
|
keeps outrunning its function dead-letters: bound its fan-out, or put work that
|
|
111
120
|
waits on the same resource in one lane. A `NonRetriableError`, or a returned
|
|
112
121
|
batch the store refuses because it reuses an event id, dead-letters the event
|
|
113
122
|
on that first attempt instead: a retry would fail the same way. The event
|
|
114
123
|
waits until you
|
|
115
|
-
[retry or skip it](/server/
|
|
124
|
+
[retry or skip it](/server/consumers#resolve-a-dead-letter) with
|
|
116
125
|
`session.redrive()`.
|
|
117
126
|
|
|
118
127
|
## Limits
|
|
@@ -123,13 +132,14 @@ waits until you
|
|
|
123
132
|
| No scheduler configured | A later top-level append or explicit `drain()` wakes the session. Reads never do. |
|
|
124
133
|
| Scheduler invocation dies after claiming but before arming | Its unacknowledged queue delivery is the slower fallback. |
|
|
125
134
|
| Store backend unavailable | Safe progress stops until it returns. |
|
|
126
|
-
| External effect succeeds before A2 completion | The
|
|
135
|
+
| External effect succeeds before A2 completion | The consumer may repeat it. Use idempotency or external fencing. |
|
|
127
136
|
|
|
128
137
|
## What append never throws for
|
|
129
138
|
|
|
130
|
-
A failing
|
|
131
|
-
if persistent, [dead-letter the event](#when-a-
|
|
139
|
+
A failing consumer never makes `append` throw. Its failures are retried and,
|
|
140
|
+
if persistent, [dead-letter the event](#when-a-consumer-keeps-failing) for
|
|
141
|
+
that consumer.
|
|
132
142
|
|
|
133
|
-
Append success means the event is durable, not that its
|
|
143
|
+
Append success means the event is durable, not that its consumers finished. An
|
|
134
144
|
identical batch retry returns rows with the same ids. A mixed batch throws
|
|
135
145
|
[`PARTIAL_DUPLICATE_BATCH`](/reference/errors).
|
|
@@ -11,9 +11,9 @@ import { orders } from '@/contracts'
|
|
|
11
11
|
export const ordersServer = createServer({
|
|
12
12
|
contract: orders,
|
|
13
13
|
store: redis({ url: process.env.REDIS_URL }),
|
|
14
|
-
|
|
14
|
+
consumers: [
|
|
15
15
|
/* ... */
|
|
16
|
-
|
|
16
|
+
],
|
|
17
17
|
})
|
|
18
18
|
```
|
|
19
19
|
|
|
@@ -34,4 +34,4 @@ A store holds A2's session histories. It doesn't connect A2 to your
|
|
|
34
34
|
application tables; see
|
|
35
35
|
[Sessions and your database](/contract/sessions#sessions-and-your-database).
|
|
36
36
|
|
|
37
|
-
Servers with
|
|
37
|
+
Servers with consumers or timers also need a [scheduler](/schedulers/overview).
|
package/docs/stores/02-redis.mdx
CHANGED
|
@@ -11,9 +11,9 @@ import { orders } from '@/contracts'
|
|
|
11
11
|
export const ordersServer = createServer({
|
|
12
12
|
contract: orders,
|
|
13
13
|
store: redis({ url: process.env.REDIS_URL }),
|
|
14
|
-
|
|
14
|
+
consumers: [
|
|
15
15
|
/* ... */
|
|
16
|
-
|
|
16
|
+
],
|
|
17
17
|
})
|
|
18
18
|
```
|
|
19
19
|
|
|
@@ -21,9 +21,9 @@ export const ordersServer = createServer({
|
|
|
21
21
|
npm i ioredis
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
Works on a single Redis instance and on non-cluster providers such as
|
|
25
|
-
which is durable by default (use its TCP `rediss://` URL). There's
|
|
26
|
-
set up: A2 creates its keys as it writes.
|
|
24
|
+
Works on a single Redis or Valkey instance and on non-cluster providers such as
|
|
25
|
+
Upstash, which is durable by default (use its TCP `rediss://` URL). There's
|
|
26
|
+
nothing to set up: A2 creates its keys as it writes.
|
|
27
27
|
|
|
28
28
|
## Options
|
|
29
29
|
|
|
@@ -31,19 +31,33 @@ set up: A2 creates its keys as it writes.
|
|
|
31
31
|
| --- | --- |
|
|
32
32
|
| `url` | creates an `ioredis` client |
|
|
33
33
|
| `client` | your own `ioredis` client instead |
|
|
34
|
-
| `keyPrefix` | isolates several apps on one Redis. Default `'a2'
|
|
34
|
+
| `keyPrefix` | isolates several apps on one Redis. Default `'a2'`; no braces |
|
|
35
35
|
| `clock`, `ids` | test seams for timestamps and generated event ids |
|
|
36
36
|
|
|
37
|
+
## Upgrade
|
|
38
|
+
|
|
39
|
+
The store records its key layout in `<keyPrefix>:schema`, and refuses a prefix
|
|
40
|
+
marked with another one with `STORE_NOT_CONFIGURED`. A release that changes
|
|
41
|
+
the layout says what to do on its [upgrade page](/upgrades/overview). This
|
|
42
|
+
release names every key differently from earlier ones, so give the store a
|
|
43
|
+
fresh `keyPrefix`.
|
|
44
|
+
|
|
37
45
|
## How it works
|
|
38
46
|
|
|
39
|
-
- Each session is a Redis Stream
|
|
40
|
-
|
|
41
|
-
|
|
47
|
+
- Each session is a Redis Stream, and every key of a session shares one hash
|
|
48
|
+
tag.
|
|
49
|
+
- Every write is one atomic script over its session's keys. On Upstash, which
|
|
50
|
+
runs a script under a database-wide lock by default, A2 marks each script to
|
|
51
|
+
lock only its session's keys. Redis and Valkey run one command at a time, so
|
|
52
|
+
A2 keeps scripts short instead.
|
|
53
|
+
- Every write publishes a wake-up after its script, so live streams are
|
|
54
|
+
push-driven, and connections scale with your processes, not with your
|
|
55
|
+
viewers.
|
|
42
56
|
- A wake-up is disposable: a safety re-read recovers a lost one within ten
|
|
43
57
|
seconds.
|
|
44
58
|
|
|
45
59
|
## Cost per command
|
|
46
60
|
|
|
47
61
|
On providers that bill per command, such as Upstash, appending an event with a
|
|
48
|
-
|
|
62
|
+
consumer and running it takes 19 commands, and an idle live stream takes 6 a
|
|
49
63
|
minute.
|