@ductape/mcp 0.1.37 → 0.1.39

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/dist/index.js +285 -73
  2. package/package.json +1 -1
  3. package/src/index.ts +285 -73
package/dist/index.js CHANGED
@@ -276,19 +276,24 @@ SPECIALIZED MODULES (for injecting handles directly without @InjectContext):
276
276
  ALL params are passed as a JSON array in positional order matching the SDK signature.
277
277
 
278
278
  ━━━ MODULE: product ━━━
279
+ IMPORTANT: ALL product.* methods require the access key and will return 403 with a publishable key.
280
+ Use ductape_cli for ALL product operations — never ductape_execute:
281
+ ductape_cli("products get --tag <tag> --json") ← fetch product + full inventory
282
+ ductape_cli("products create --name <name> --tag <tag>")
283
+ ductape_cli("products environments list <tag> --json")
284
+ ductape_cli("products environments get <tag> <slug> --json")
285
+ ductape_cli("products apps list --product <id> --json")
286
+
287
+ SDK method signatures (for reference, admin key only):
279
288
  product.create [data: { name, description, tag?, envs?: [{slug, name}] }]
280
289
  product.fetch [product_tag: string]
281
290
  product.update [product_tag: string, data: { name?: string, description?: string }]
282
- product.init [product_tag: string]
283
- product.environments.create [product_tag, data: { slug: string, env_name: string, description: string, active?: boolean }]
284
- product.environments.update [product_tag, slug: string, data: { env_name?: string, description?: string, active?: boolean }]
291
+ product.environments.create [product_tag, data: { slug, env_name, description, active? }]
285
292
  product.environments.list [product_tag]
286
293
  product.environments.fetch [product_tag, slug]
287
- product.apps.connect [product_tag, app_tag]
288
- product.apps.add [product_tag, app: { access_tag: string, envs: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
294
+ product.apps.add [product_tag, app: { access_tag, envs: [{ app_env_slug, product_env_slug, variables?, auth? }] }]
289
295
  product.apps.list [product_tag]
290
296
  product.apps.fetch [product_tag, access_tag]
291
- product.apps.update [product_tag, access_tag: string, data: { version?: string, envs?: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
292
297
 
293
298
  ━━━ MODULE: app ━━━
294
299
  app.create [data: { app_name: string, description: string, unique?: boolean }]
@@ -1722,6 +1727,31 @@ Important:
1722
1727
  apps: `
1723
1728
  DUCTAPE APPS
1724
1729
 
1730
+ WHAT AN APP IS:
1731
+ A Ductape App is a pre-configured, versioned API integration definition. It is NOT a generic HTTP
1732
+ client, NOT a job scheduler, and NOT anything you can call without registering first.
1733
+
1734
+ An App must be fully set up in Ductape before any code can use it:
1735
+ 1. The App record must be created (name, tag, description)
1736
+ 2. Environments must be added (each environment slug → base URL for that stage)
1737
+ 3. Auth scheme must be configured (how outbound requests authenticate: apikey, bearer, OAuth2, etc.)
1738
+ 4. Action endpoints must be defined (each action = one HTTP endpoint spec: method, path, body/query/header shape, response shape)
1739
+ 5. The App must be connected to the product (product.apps.add) and its envs mapped
1740
+
1741
+ ONLY after all five steps can any code call:
1742
+ ctx.api.run({ app: '<app_tag>', event: '<action_tag>', input: { ... } }) ← in a feature handler
1743
+ actions.run([{ product, env, app: '<app_tag>', action: '<action_tag>', input }]) ← at runtime
1744
+
1745
+ ctx.api (alias for ctx.action) is NOT a generic HTTP call. It ONLY invokes a pre-registered
1746
+ Ductape App Action. If the App or action tag does not exist in the product, the call will fail.
1747
+ actions.dispatch is the same as actions.run but scheduled as a background job — it still requires
1748
+ a registered App. There is no way to dispatch a job to an arbitrary URL via ctx.api or actions.dispatch.
1749
+
1750
+ If a feature step needs to call an external service and no App is registered for it yet:
1751
+ → Flag it as "App to create" in your plan (STEP 4 of the feature design workflow)
1752
+ → Create the App and all its actions first (see below)
1753
+ → Only then write the ctx.api.run call
1754
+
1725
1755
  An app is a versioned API integration definition. It contains environments (base URLs), actions
1726
1756
  (individual endpoint specs), auth schemes, webhooks, variables, and constants.
1727
1757
 
@@ -1737,6 +1767,21 @@ Manage environments (base URLs per stage):
1737
1767
  ductape_execute("app.environments.create", [app_tag, { slug, env_name, base_url }])
1738
1768
  ductape_execute("app.environments.list", [app_tag])
1739
1769
 
1770
+ Discover apps in a product and their actions (ALWAYS do this before writing any ctx.api.run call):
1771
+ Step 1 — list apps connected to the product:
1772
+ ductape_cli("products get --tag <product_tag> --json") → full product document; check apps[]
1773
+ ductape_cli("products apps list --product <product_id> --json") → apps[] with access_tag, envs
1774
+ Step 2 — list actions in an app:
1775
+ ductape_execute("actions.list", [app_tag]) → returns all action tags + names
1776
+ Step 3 — fetch the input schema for an action:
1777
+ ductape_execute("actions.fetch", [app_tag, action_tag])
1778
+ → returns { body: {fieldName: {type, required}}, params: {}, query: {}, headers: {} }
1779
+ OR: call ductape_generate_payload (operation_family="action", method="run",
1780
+ targets={app: "app_tag", action: "action_tag"}) to get the exact resolved payload shape
1781
+ Step 4 — call ductape_schema({ module: "app" }) if you need the JSON schema for creating/updating
1782
+ app resources (not for runtime input — use actions.fetch or ductape_generate_payload for that)
1783
+ NEVER assume action input field names. Always fetch the action definition first.
1784
+
1740
1785
  Manage actions (individual API endpoints):
1741
1786
  ductape_execute("actions.create", [app_tag, {
1742
1787
  tag, name, resource: "/users/{id}", method: "GET"|"POST"|"PUT"|"PATCH"|"DELETE",
@@ -1750,6 +1795,16 @@ Manage actions (individual API endpoints):
1750
1795
  ductape_execute("actions.list", [app_tag])
1751
1796
  ductape_execute("actions.fetch", [app_tag, action_tag])
1752
1797
 
1798
+ Action input — flat input format:
1799
+ Fields are resolved to the correct location (body/params/query/headers) by matching the action schema.
1800
+ For ambiguous keys, use explicit prefixes:
1801
+ input: { amount: 1000 } → auto-resolved (body.amount if body field exists)
1802
+ input: { "body:amount": 1000 } → explicit body
1803
+ input: { "params:id": "user_123" } → route parameter
1804
+ input: { "query:limit": 10 } → query string
1805
+ input: { "headers:X-Idempotency-Key": "..." } → request header
1806
+ Always use ductape_generate_payload or actions.fetch to know the exact field names — never guess.
1807
+
1753
1808
  Run an action at runtime:
1754
1809
  → CALL ductape_generate_payload FIRST (operation_family="action", method="run", targets={app, action})
1755
1810
  ductape_execute("actions.run", [{ product, env, app, action, input: { "body:field": value } }])
@@ -1769,14 +1824,21 @@ Variables (per-env mutable values) and Constants (fixed values):
1769
1824
  ductape_execute("app.constants.create", [app_tag, { key, value }])
1770
1825
 
1771
1826
  Connecting an app to a product (after creation):
1772
- ductape_execute("product.apps.add", [product_tag, {
1773
- access_tag: "app_access_tag",
1774
- envs: [{ app_env_slug: "production", product_env_slug: "prd",
1775
- variables: [{ key: "BASE_URL", value: "https://api.example.com" }],
1776
- auth: { auth_tag: "api-key-auth", data: "$Secret{API_KEY}" } }]
1777
- }])
1778
- ductape_execute("product.apps.list", [product_tag])
1779
- ductape_execute("product.apps.fetch", [product_tag, access_tag])
1827
+ NOTE: All product.* module methods require the access key and CANNOT use ductape_execute.
1828
+ Use ductape_cli for all product-level operations.
1829
+
1830
+ FULL FLOW to make an app callable from a product:
1831
+ 1. Create the app: ductape_cli("apps create --name \\"Service Name\\" --description \\"...\\"")
1832
+ 2. Add environments: ductape_execute("app.environments.create", [app_tag, { slug: "prd", env_name: "Production", base_url: "https://api.example.com" }])
1833
+ 3. Configure auth: ductape_execute("auths.create", [app_tag, { tag, name, setup_type: "apikey"|"bearer"|"basic"|"oauth2", expiry, period }])
1834
+ 4. Define actions: ductape_execute("actions.create", [app_tag, { tag, name, resource, method, body?, params?, query?, headers?, response? }])
1835
+ OR import: ductape_cli("apps import <file.json> -t postman|openapi")
1836
+ 5. Connect to product: Requires the product_id (from ductape_cli("products get --tag <tag> --json")).
1837
+ There is no CLI command for this step — the SDK product.apps.add method requires
1838
+ an access key which only the backend can provide. Use ductape_execute via an
1839
+ admin-authenticated context, or connect via the Workbench UI.
1840
+ 6. Verify: ductape_cli("products get --tag <product_tag> --json") → check apps[] contains the app
1841
+ ductape_execute("actions.list", [app_tag]) → verify actions are registered
1780
1842
  `.trim(),
1781
1843
  products: `
1782
1844
  DUCTAPE PRODUCTS
@@ -1785,20 +1847,20 @@ A product is the top-level namespace for all Ductape infrastructure: apps, datab
1785
1847
  vectors, storage, brokers, sessions, caches, notifications, resilience, features, jobs, and envs.
1786
1848
  Every SDK service call resolves within a product context.
1787
1849
 
1850
+ IMPORTANT: The product module requires the access key. ALL product operations must use ductape_cli,
1851
+ not ductape_execute (which only accepts the publishable key and will return 403 for product.*).
1852
+
1788
1853
  Create a product:
1789
1854
  ductape_cli("products create --name \\"My App\\" --tag my-app")
1790
- ductape_execute("product.create", [{ name: "My App", tag: "my-app",
1791
- envs: [{ slug: "dev", name: "Development" }, { slug: "prd", name: "Production" }] }])
1792
1855
 
1793
1856
  Environments — every resource's envs array MUST cover all product env slugs:
1794
- ductape_execute("product.environments.create", [product_tag, { slug, env_name, description, active? }])
1795
- ductape_execute("product.environments.list", [product_tag])
1796
- ductape_execute("product.environments.fetch", [product_tag, slug])
1797
- BEFORE registering any resource, always run environments.list and collect all slugs.
1857
+ ductape_cli("products environments list <product_tag> --json")
1858
+ ductape_cli("products environments get <product_tag> <slug> --json")
1859
+ BEFORE registering any resource, always run environments list and collect all slugs.
1798
1860
 
1799
1861
  Fetch / update:
1800
- ductape_execute("product.fetch", [product_tag])
1801
- ductape_execute("product.update", [product_tag, { name?, description? }])
1862
+ ductape_cli("products get --tag <product_tag> --json")
1863
+ ductape_cli("products get --id <product_id> --json")
1802
1864
 
1803
1865
  Connect apps to a product:
1804
1866
  See ductape_docs({ topic: "apps" }) for product.apps.add / product.apps.list.
@@ -2081,21 +2143,32 @@ STEP 1 — UNDERSTAND the goal
2081
2143
  understand: what the feature does, what it returns, what can fail and how failures should behave.
2082
2144
 
2083
2145
  STEP 2 — INVENTORY existing Ductape components
2084
- Call ductape_execute("products.fetch", [product_tag]) to read the product.
2146
+ Call ductape_cli("products get --tag <product_tag> --json") to read the full product document.
2147
+ (product.* requires the access key — never use ductape_execute for product reads, it will return 403)
2085
2148
  Note what already exists:
2086
2149
  - databases[] → available for ctx.database.insert/query/update/delete steps
2087
- - apps[] → available for ctx.api.run steps (check app.events[] for event tags)
2150
+ - apps[] → available for ctx.api.run steps ONLY if the App is fully registered:
2151
+ (a) App record exists, (b) environments defined with base URLs,
2152
+ (c) auth scheme configured, (d) action endpoints defined,
2153
+ (e) connected to the product via product.apps.add.
2154
+ Check app.events[] for action tags. If no app exists for a service the
2155
+ feature needs to call, flag it as "App to create" in the plan — do NOT
2156
+ assume ctx.api can call any URL or schedule any job without a registered App.
2088
2157
  - notifications[] → available for ctx.notification.email/sms/push steps
2089
2158
  - storage[] → available for ctx.storage.upload/download steps
2090
- - messageBrokers[] → available for ctx.messaging.produce steps
2159
+ - messageBrokers[] → available for ctx.events.produce steps
2091
2160
  - graphs[] → available for ctx.graph steps
2092
- - features[] → can be called as child features via ctx.feature()
2161
+ - features[] → can be called as child features via ctx.feature.execute()
2093
2162
  - caches[], sessions[]
2094
2163
  Do NOT assume a component or event tag exists — verify from the product before using it.
2164
+ Do NOT treat ctx.api as a generic HTTP call or job scheduler. It requires a registered App.
2095
2165
 
2096
2166
  STEP 3 — PLAN each step
2097
2167
  For every logical step:
2098
- a. Identify which existing component handles it, or flag it as needing creation
2168
+ a. Identify which existing component handles it, or flag it as needing creation.
2169
+ If a step calls an external service, it MUST go through a registered Ductape App.
2170
+ If no App for that service exists in the product → mark it "App to create: <service name>".
2171
+ DO NOT plan a raw HTTP call, a direct dispatch to a URL, or any workaround in place of a missing App.
2099
2172
  b. Note how inputs flow: ctx.input fields, or return values from earlier steps (plain JS variables — no special notation needed)
2100
2173
  c. Decide if a rollback handler is needed (e.g. charge → refund on later failure)
2101
2174
  d. Decide allow_fail: true for non-critical steps (email, analytics, audit logs)
@@ -2139,7 +2212,8 @@ STEP 7 — HANDLE conditionals, loops, and branching (when applicable)
2139
2212
  → Only the scenario whose input matches runs at execution time
2140
2213
 
2141
2214
  STEP 8 — SET rollbacks for reversible steps
2142
- Any step that allocates a resource should undo it if a later step fails:
2215
+ Any step that allocates a resource should undo it if a later step fails.
2216
+ ctx.api.run requires a pre-registered Ductape App — 'stripe' below is the tag of a registered App:
2143
2217
  const charge = await ctx.step(
2144
2218
  'charge',
2145
2219
  async () => ctx.api.run({ app: 'stripe', event: 'create-charge', input: { amount: ctx.input.amount } }),
@@ -2199,6 +2273,52 @@ Signals and queries (for long-running features):
2199
2273
 
2200
2274
  Rollback strategies: reverse_all | reverse_critical | compensate | none
2201
2275
  Feature statuses: pending | running | completed | failed | rolled_back | rolling_back | paused
2276
+
2277
+ ━━━ FEATURE RECORDING SEMANTICS ━━━
2278
+
2279
+ When you call features.define({ handler }), the handler runs TWICE:
2280
+
2281
+ 1. RECORDING PHASE (at define time) — handler is called with a RecordingContext.
2282
+ All ctx.step() calls return lightweight proxy objects, not real data.
2283
+ This phase captures the step graph: which steps exist, their types, tags, and declared
2284
+ inputs/outputs. No real API calls, DB queries, or side effects occur.
2285
+ Arbitrary JS code OUTSIDE ctx.step() ALSO runs during recording — with proxy values.
2286
+ For loops: use recordInput so the handler sees sample data and all iterations are recorded.
2287
+ For branches: use branchOverrides so each path is captured.
2288
+
2289
+ 2. EXECUTION PHASE (at runtime) — handler is called with a real ExecutionContext.
2290
+ ctx.step() actually executes. All real Ductape component calls happen.
2291
+ Arbitrary JS logic (math, string ops, conditionals on step results) runs for real.
2292
+
2293
+ Implication: put all meaningful business logic INSIDE ctx.step() handlers, not in the
2294
+ outer handler body. Code in the outer body runs during recording with proxy values and
2295
+ may behave unexpectedly (e.g. typeof proxy === 'object' is true but .someField is a proxy).
2296
+
2297
+ Features do NOT execute arbitrary NestJS or server code directly. A feature handler can only
2298
+ call Ductape component primitives (ctx.api, ctx.database, ctx.notification, etc.) as steps.
2299
+ To invoke internal application business logic, produce a broker event from a feature step
2300
+ (ctx.messaging.produce) and consume it in your NestJS service — that is the correct pattern.
2301
+
2302
+ ━━━ ORCHESTRATION DECISION RULE ━━━
2303
+
2304
+ One component operation at a future time:
2305
+ → use that component's own dispatch method
2306
+ e.g. ductape.events.dispatch({ ..., schedule: { start_at: ... } })
2307
+ e.g. ductape.api.dispatch({ ..., schedule: { start_at: ... } })
2308
+ e.g. ductape.database.dispatch({ ..., schedule: { start_at: ... } })
2309
+
2310
+ Several durable Ductape component operations in sequence (with rollback / retry / state):
2311
+ → define a Feature, then features.dispatch to schedule it
2312
+
2313
+ Invoke internal application business logic (your own NestJS/backend service code):
2314
+ → produce a broker event (ctx.messaging.produce or ductape.events.produce)
2315
+ → consume it in your NestJS service with events.consume in onModuleInit
2316
+ → your service method runs with full access to DI, DB transactions, etc.
2317
+ Do NOT create an App Action just to call your own service over HTTP.
2318
+
2319
+ Invoke an external/public HTTP service:
2320
+ → create a Ductape App (register base URL, auth, action endpoints) then use ctx.api.run
2321
+ → requires the App to be fully registered and connected to the product first
2202
2322
  `.trim(),
2203
2323
  events: `
2204
2324
  DUCTAPE EVENTS (MESSAGE BROKERS)
@@ -2296,81 +2416,173 @@ Import (register an EXISTING cloud resource):
2296
2416
  redis: { host: "...", port: 6379, password?: "..." }
2297
2417
  nats: { servers: ["nats://host:4222"], token?: "...", user?: "...", pass?: "...", tls?: true }
2298
2418
 
2299
- ━━━ STEP 2: ADD TOPIC DEFINITIONS ━━━
2300
-
2301
- For most providers (GCP Pub/Sub, Kafka, RabbitMQ, Redis, NATS, Azure Service Bus):
2302
- SKIP this step. Topics are auto-created when you create a producer (step 3).
2303
- You do NOT need to call messageBrokers.topics.create before creating a producer.
2419
+ ━━━ STEP 2: DEFINE TOPICS ━━━
2304
2420
 
2305
- Only call topics.create explicitly when:
2306
- - Using AWS SQS (must supply queueUrls per envauto-creation cannot know the queue URL)
2307
- - You want to pre-set sample data or idempotency config on the topic
2421
+ Topics MUST be defined before any consumer can subscribe to them.
2422
+ Producing to a topic also calls ensureTopicRegistered in the background but DO NOT rely on
2423
+ auto-registration for consume paths. Always create topics explicitly.
2308
2424
 
2309
- If you do need it (SQS or explicit config):
2310
2425
  ductape_execute("messageBrokers.topics.create", [product_tag, {
2311
- tag: "player-joined",
2312
- name: "Player Joined",
2313
- broker: "notifications-broker", // broker component tag
2426
+ tag: "order-created", // topic tag (just the topic part, NOT "broker:topic")
2427
+ name: "Order Created",
2428
+ broker: "order-events", // broker component tag
2314
2429
  description?: string,
2315
- sample: { playerId: "string", username: "string" }, // example message shape
2316
- idempotent?: boolean,
2317
- // SQS only — map per-env queue URLs (each topic can be a different queue):
2430
+ sample: { orderId: "string", total: 0 }, // documents expected message shape
2431
+ idempotent?: boolean, // if true, Ductape deduplicates by idempotency_key
2432
+ // AWS SQS only — must supply per-env queue URL:
2318
2433
  queueUrls?: [
2319
- { env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123456/my-product-player-joined-snd" },
2320
- { env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123456/my-product-player-joined-prd" }
2434
+ { env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-snd" },
2435
+ { env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-prd" }
2321
2436
  ]
2322
2437
  }])
2323
2438
 
2324
- List topics on a broker:
2439
+ List / fetch topics:
2325
2440
  ductape_execute("messageBrokers.topics.list", [product_tag, "broker-tag"])
2441
+ ductape_execute("messageBrokers.fetch", [product_tag, "broker-tag"]) → includes topics[]
2326
2442
 
2327
- ━━━ STEP 3: PRODUCE AND CONSUME — WRITTEN IN APPLICATION CODE ━━━
2443
+ ━━━ STEP 3: PRODUCE — WRITTEN IN APPLICATION CODE ━━━
2328
2444
 
2329
- There is NO admin command or file to declare producers/consumers.
2330
- There is NO "create producer" step before writing code.
2331
- Producers and consumers are registered automatically by the SDK the first time your code calls
2332
- produce/consume you do not pre-declare them.
2445
+ There is NO admin command to declare a producer. Producers are auto-registered by the SDK on
2446
+ the first produce call you do not pre-declare them.
2447
+ Do NOT call ductape_generate_payload for messaging. The producer owns the schema.
2448
+ Infer the message shape from context, present it to the user for approval, then implement.
2333
2449
 
2334
- The entire producer/consumer contract is the code you write in your controllers or services:
2335
-
2336
- Produce (publish a message) write in your service/controller:
2337
- Do NOT call ductape_generate_payload for messaging. Events have no pre-existing backend schema
2338
- to discover — the producer defines the schema. Instead, infer the message shape from context
2339
- (event name, existing data models, user input), present it to the user for approval, then implement.
2450
+ GENERAL BACKEND (TypeScript/Node.js not NestJS):
2451
+ import Ductape from '@ductape/sdk';
2452
+ const ductape = new Ductape({ accessKey: 'your-access-key' });
2340
2453
  await ductape.events.produce({
2341
2454
  product: "my-product",
2342
2455
  env: "prd",
2343
- event: "broker_tag:topic_tag", // "broker_tag:topic_tag" — always colon-separated
2344
- message: { key: value }, // shape inferred from context, approved by user
2456
+ event: "broker-tag:topic-tag", // always colon-separated
2457
+ message: { key: value },
2458
+ session?: "session-tag:jwt", // optional — traces message to a user session
2459
+ });
2460
+ // Idempotent publish (deduplicates — prevents double-processing on retries):
2461
+ await ductape.events.publishIdempotent({
2462
+ product, env, event, message,
2463
+ idempotencyKey: "order-123-charge", // stable key unique to this logical operation
2464
+ idempotencyTtl?: 86400, // seconds; default 86400 (24h)
2465
+ });
2466
+
2467
+ NESTJS — method decorator:
2468
+ import { Messaging } from '@ductape/nestjs';
2469
+ @Injectable() export class OrdersService {
2470
+ @Messaging.Produce({ event: 'order-events:order-created' })
2471
+ emitOrderCreated(payload: { orderId: string; total: number }) { return payload; }
2472
+
2473
+ // Scheduled dispatch — fire-and-forget with optional schedule:
2474
+ @Messaging.Dispatch({ broker: 'order-events', event: 'order-events:order-created',
2475
+ schedule?: { start_at?, cron?, every?, limit?, tz? } })
2476
+ scheduleOrderNotification(payload: Record<string, unknown>) { return payload; }
2477
+ }
2478
+
2479
+ CLIENT-SIDE (browser — publishable key):
2480
+ Clients CAN produce messages using a publishable key + session token.
2481
+ Only produce to topics whose schema is safe for client authorship.
2482
+ NEVER allow clients to produce to topics that trigger privileged server-side operations
2483
+ (payments, admin actions, state mutations) — those must go through a backend endpoint first.
2484
+ import Ductape from '@ductape/sdk';
2485
+ const ductape = new Ductape({ publishableKey: 'pk_...', env: 'prd', product: 'my-product' });
2486
+ await ductape.events.produce({
2487
+ event: "user-events:user-action",
2488
+ message: { action: "button-click", screen: "dashboard" },
2489
+ session: "user-session:eyJ...", // REQUIRED for client-side produce
2345
2490
  });
2346
- Idempotent publish (deduplicates by key):
2347
- await ductape.events.publishIdempotent({ product, env, event, message, idempotencyKey, idempotencyTtl? })
2348
2491
 
2349
- Consume (subscribe) — write in your service/controller:
2492
+ SCHEDULED DISPATCH (background job):
2493
+ ductape_execute("messageBrokers.dispatch", [{
2494
+ product, env,
2495
+ broker: "order-events", // broker tag
2496
+ event: "order-events:reminder-due", // "broker:topic"
2497
+ input: { message: { orderId: "123" } },
2498
+ retries?: 3,
2499
+ session?: "session-tag:jwt",
2500
+ schedule?: {
2501
+ start_at?: 1735689600000, // Unix ms or ISO string
2502
+ cron?: "0 9 * * *", // recurring cron
2503
+ every?: 86400000, // recurring interval ms
2504
+ limit?: 10, // max repetitions
2505
+ endDate?: "2026-12-31",
2506
+ tz?: "America/New_York",
2507
+ },
2508
+ }])
2509
+ Returns: { job_id, status: "scheduled"|"queued", scheduled_at, recurring, next_run_at? }
2510
+
2511
+ ━━━ STEP 4: CONSUME — WRITTEN IN APPLICATION CODE ━━━
2512
+
2513
+ Consumers are auto-registered by the SDK on first consume call.
2514
+ Consumer registration options (all optional — used for tracking in Workbench):
2515
+ consumer?: { tag?: string, name?: string, description?: string }
2516
+ If tag is omitted, Ductape generates one: "consumer-<brokerTag>-<topicTag>".
2517
+
2518
+ ACK BEHAVIOR (automatic):
2519
+ - Callback returns successfully → message is acknowledged (ack)
2520
+ - Callback throws → message is tracked as failed; broker nacks/retries per provider behavior
2521
+ - After max retries → message moves to dead-letter queue (DLQ)
2522
+ There is no manual ack API. Acknowledgement is implicit from callback outcome.
2523
+
2524
+ CONSUMER GROUPS (Kafka-specific):
2525
+ Consumer groups are set in the broker's envs[].config.groupId (at broker registration time).
2526
+ All service instances sharing the same groupId form a consumer group and share partition load.
2527
+ To configure: set groupId in the kafka config when creating/updating the broker.
2528
+
2529
+ CONCURRENCY:
2530
+ Ductape has no per-consumer concurrency setting. Concurrency is determined by:
2531
+ - Number of running service instances (horizontal scale)
2532
+ - Broker-level partition count (Kafka) or visibility timeout (SQS)
2533
+ Run multiple instances of your service to scale consumption.
2534
+
2535
+ GENERAL BACKEND (TypeScript/Node.js — not NestJS):
2536
+ Start consuming in your module init or service startup:
2350
2537
  await ductape.events.consume({
2351
2538
  product: "my-product",
2352
2539
  env: "prd",
2353
- event: "broker_tag:topic_tag",
2354
- callback: async (message) => { /* handle message */ },
2540
+ event: "order-events:order-created",
2541
+ callback: async (message) => {
2542
+ // All real processing logic goes here.
2543
+ // Throw to nack. Return to ack.
2544
+ await processOrder(message as { orderId: string; total: number });
2545
+ },
2546
+ consumer?: { tag: "order-processor", name: "Order Processor" },
2355
2547
  });
2356
- Callback errors are re-thrown so the broker can nack/retry.
2357
2548
 
2358
- Background dispatch with scheduling write in your service/controller:
2359
- await ductape.events.dispatch({ product, env, broker, event, input: { message },
2360
- schedule?: { start_at?, cron?, every?, limit?, endDate?, tz? } })
2549
+ NESTJS use SDK in onModuleInit (no @Messaging.Consume decorator exists yet):
2550
+ @Injectable()
2551
+ export class OrderConsumerService implements OnModuleInit {
2552
+ constructor(private readonly ductape: Ductape) {}
2553
+ async onModuleInit() {
2554
+ await this.ductape.events.consume({
2555
+ product: "my-product",
2556
+ env: process.env.DUCTAPE_ENV || 'prd',
2557
+ event: "order-events:order-created",
2558
+ callback: async (message) => { await this.handle(message); },
2559
+ consumer: { tag: "order-consumer", name: "Order Consumer" },
2560
+ });
2561
+ }
2562
+ private async handle(message: unknown) { /* business logic */ }
2563
+ }
2361
2564
 
2362
- For the four standard producer declarations (match-state, match-report, projection-updated,
2363
- notification), write these produce calls in the relevant application service methods there is
2364
- no separate configuration file or CLI step. The SDK creates the producer metadata on first call.
2565
+ CLIENT-SIDE: Clients CANNOT consume. Event consumption is always server-side only.
2566
+ This is the key distinction between server topics (produce + consume) and client-observable
2567
+ topics (produce from client, consume on server). Never set up a consumer in browser code.
2365
2568
 
2366
- Message payload is AES-encrypted before the tracking API call — tracking never sees plaintext.
2569
+ DEAD-LETTER QUEUE (DLQ):
2570
+ Messages whose callbacks consistently throw are automatically moved to the DLQ.
2571
+ Query: ductape_execute("messageBrokers.messages.getDeadLetters",
2572
+ [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }])
2573
+ Reprocess: ductape_execute("messageBrokers.reprocessDLQ",
2574
+ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }])
2575
+ Replay: ductape_execute("messageBrokers.replayEvent",
2576
+ [{ product, env, eventId, force? }])
2367
2577
 
2368
2578
  ━━━ OBSERVABILITY ━━━
2369
2579
 
2370
2580
  messageBrokers.messages.query [{ product, env, brokerTag, topicTag?, status?, page?, limit? }]
2371
2581
  messageBrokers.messages.getStats [{ product, env, brokerTag }]
2372
2582
  messageBrokers.messages.getDashboard [{ product, env, brokerTag }]
2373
- messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, limit? }]
2583
+ messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }]
2584
+ messageBrokers.messages.getProducers [{ product, env, brokerTag, topicTag?, page?, limit? }]
2585
+ messageBrokers.messages.getConsumers [{ product, env, brokerTag, topicTag?, page?, limit? }]
2374
2586
  messageBrokers.replayEvent [{ product, env, eventId, force? }]
2375
2587
  messageBrokers.reprocessDLQ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }]
2376
2588
  messageBrokers.checkIdempotency [{ product, env, brokerTag, idempotency_key }]
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ductape/mcp",
3
- "version": "0.1.37",
3
+ "version": "0.1.39",
4
4
  "description": "MCP server that exposes Ductape SDK operations via the backend proxy",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
package/src/index.ts CHANGED
@@ -287,19 +287,24 @@ SPECIALIZED MODULES (for injecting handles directly without @InjectContext):
287
287
  ALL params are passed as a JSON array in positional order matching the SDK signature.
288
288
 
289
289
  ━━━ MODULE: product ━━━
290
+ IMPORTANT: ALL product.* methods require the access key and will return 403 with a publishable key.
291
+ Use ductape_cli for ALL product operations — never ductape_execute:
292
+ ductape_cli("products get --tag <tag> --json") ← fetch product + full inventory
293
+ ductape_cli("products create --name <name> --tag <tag>")
294
+ ductape_cli("products environments list <tag> --json")
295
+ ductape_cli("products environments get <tag> <slug> --json")
296
+ ductape_cli("products apps list --product <id> --json")
297
+
298
+ SDK method signatures (for reference, admin key only):
290
299
  product.create [data: { name, description, tag?, envs?: [{slug, name}] }]
291
300
  product.fetch [product_tag: string]
292
301
  product.update [product_tag: string, data: { name?: string, description?: string }]
293
- product.init [product_tag: string]
294
- product.environments.create [product_tag, data: { slug: string, env_name: string, description: string, active?: boolean }]
295
- product.environments.update [product_tag, slug: string, data: { env_name?: string, description?: string, active?: boolean }]
302
+ product.environments.create [product_tag, data: { slug, env_name, description, active? }]
296
303
  product.environments.list [product_tag]
297
304
  product.environments.fetch [product_tag, slug]
298
- product.apps.connect [product_tag, app_tag]
299
- product.apps.add [product_tag, app: { access_tag: string, envs: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
305
+ product.apps.add [product_tag, app: { access_tag, envs: [{ app_env_slug, product_env_slug, variables?, auth? }] }]
300
306
  product.apps.list [product_tag]
301
307
  product.apps.fetch [product_tag, access_tag]
302
- product.apps.update [product_tag, access_tag: string, data: { version?: string, envs?: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
303
308
 
304
309
  ━━━ MODULE: app ━━━
305
310
  app.create [data: { app_name: string, description: string, unique?: boolean }]
@@ -1788,6 +1793,31 @@ Important:
1788
1793
  apps: `
1789
1794
  DUCTAPE APPS
1790
1795
 
1796
+ WHAT AN APP IS:
1797
+ A Ductape App is a pre-configured, versioned API integration definition. It is NOT a generic HTTP
1798
+ client, NOT a job scheduler, and NOT anything you can call without registering first.
1799
+
1800
+ An App must be fully set up in Ductape before any code can use it:
1801
+ 1. The App record must be created (name, tag, description)
1802
+ 2. Environments must be added (each environment slug → base URL for that stage)
1803
+ 3. Auth scheme must be configured (how outbound requests authenticate: apikey, bearer, OAuth2, etc.)
1804
+ 4. Action endpoints must be defined (each action = one HTTP endpoint spec: method, path, body/query/header shape, response shape)
1805
+ 5. The App must be connected to the product (product.apps.add) and its envs mapped
1806
+
1807
+ ONLY after all five steps can any code call:
1808
+ ctx.api.run({ app: '<app_tag>', event: '<action_tag>', input: { ... } }) ← in a feature handler
1809
+ actions.run([{ product, env, app: '<app_tag>', action: '<action_tag>', input }]) ← at runtime
1810
+
1811
+ ctx.api (alias for ctx.action) is NOT a generic HTTP call. It ONLY invokes a pre-registered
1812
+ Ductape App Action. If the App or action tag does not exist in the product, the call will fail.
1813
+ actions.dispatch is the same as actions.run but scheduled as a background job — it still requires
1814
+ a registered App. There is no way to dispatch a job to an arbitrary URL via ctx.api or actions.dispatch.
1815
+
1816
+ If a feature step needs to call an external service and no App is registered for it yet:
1817
+ → Flag it as "App to create" in your plan (STEP 4 of the feature design workflow)
1818
+ → Create the App and all its actions first (see below)
1819
+ → Only then write the ctx.api.run call
1820
+
1791
1821
  An app is a versioned API integration definition. It contains environments (base URLs), actions
1792
1822
  (individual endpoint specs), auth schemes, webhooks, variables, and constants.
1793
1823
 
@@ -1803,6 +1833,21 @@ Manage environments (base URLs per stage):
1803
1833
  ductape_execute("app.environments.create", [app_tag, { slug, env_name, base_url }])
1804
1834
  ductape_execute("app.environments.list", [app_tag])
1805
1835
 
1836
+ Discover apps in a product and their actions (ALWAYS do this before writing any ctx.api.run call):
1837
+ Step 1 — list apps connected to the product:
1838
+ ductape_cli("products get --tag <product_tag> --json") → full product document; check apps[]
1839
+ ductape_cli("products apps list --product <product_id> --json") → apps[] with access_tag, envs
1840
+ Step 2 — list actions in an app:
1841
+ ductape_execute("actions.list", [app_tag]) → returns all action tags + names
1842
+ Step 3 — fetch the input schema for an action:
1843
+ ductape_execute("actions.fetch", [app_tag, action_tag])
1844
+ → returns { body: {fieldName: {type, required}}, params: {}, query: {}, headers: {} }
1845
+ OR: call ductape_generate_payload (operation_family="action", method="run",
1846
+ targets={app: "app_tag", action: "action_tag"}) to get the exact resolved payload shape
1847
+ Step 4 — call ductape_schema({ module: "app" }) if you need the JSON schema for creating/updating
1848
+ app resources (not for runtime input — use actions.fetch or ductape_generate_payload for that)
1849
+ NEVER assume action input field names. Always fetch the action definition first.
1850
+
1806
1851
  Manage actions (individual API endpoints):
1807
1852
  ductape_execute("actions.create", [app_tag, {
1808
1853
  tag, name, resource: "/users/{id}", method: "GET"|"POST"|"PUT"|"PATCH"|"DELETE",
@@ -1816,6 +1861,16 @@ Manage actions (individual API endpoints):
1816
1861
  ductape_execute("actions.list", [app_tag])
1817
1862
  ductape_execute("actions.fetch", [app_tag, action_tag])
1818
1863
 
1864
+ Action input — flat input format:
1865
+ Fields are resolved to the correct location (body/params/query/headers) by matching the action schema.
1866
+ For ambiguous keys, use explicit prefixes:
1867
+ input: { amount: 1000 } → auto-resolved (body.amount if body field exists)
1868
+ input: { "body:amount": 1000 } → explicit body
1869
+ input: { "params:id": "user_123" } → route parameter
1870
+ input: { "query:limit": 10 } → query string
1871
+ input: { "headers:X-Idempotency-Key": "..." } → request header
1872
+ Always use ductape_generate_payload or actions.fetch to know the exact field names — never guess.
1873
+
1819
1874
  Run an action at runtime:
1820
1875
  → CALL ductape_generate_payload FIRST (operation_family="action", method="run", targets={app, action})
1821
1876
  ductape_execute("actions.run", [{ product, env, app, action, input: { "body:field": value } }])
@@ -1835,14 +1890,21 @@ Variables (per-env mutable values) and Constants (fixed values):
1835
1890
  ductape_execute("app.constants.create", [app_tag, { key, value }])
1836
1891
 
1837
1892
  Connecting an app to a product (after creation):
1838
- ductape_execute("product.apps.add", [product_tag, {
1839
- access_tag: "app_access_tag",
1840
- envs: [{ app_env_slug: "production", product_env_slug: "prd",
1841
- variables: [{ key: "BASE_URL", value: "https://api.example.com" }],
1842
- auth: { auth_tag: "api-key-auth", data: "$Secret{API_KEY}" } }]
1843
- }])
1844
- ductape_execute("product.apps.list", [product_tag])
1845
- ductape_execute("product.apps.fetch", [product_tag, access_tag])
1893
+ NOTE: All product.* module methods require the access key and CANNOT use ductape_execute.
1894
+ Use ductape_cli for all product-level operations.
1895
+
1896
+ FULL FLOW to make an app callable from a product:
1897
+ 1. Create the app: ductape_cli("apps create --name \\"Service Name\\" --description \\"...\\"")
1898
+ 2. Add environments: ductape_execute("app.environments.create", [app_tag, { slug: "prd", env_name: "Production", base_url: "https://api.example.com" }])
1899
+ 3. Configure auth: ductape_execute("auths.create", [app_tag, { tag, name, setup_type: "apikey"|"bearer"|"basic"|"oauth2", expiry, period }])
1900
+ 4. Define actions: ductape_execute("actions.create", [app_tag, { tag, name, resource, method, body?, params?, query?, headers?, response? }])
1901
+ OR import: ductape_cli("apps import <file.json> -t postman|openapi")
1902
+ 5. Connect to product: Requires the product_id (from ductape_cli("products get --tag <tag> --json")).
1903
+ There is no CLI command for this step — the SDK product.apps.add method requires
1904
+ an access key which only the backend can provide. Use ductape_execute via an
1905
+ admin-authenticated context, or connect via the Workbench UI.
1906
+ 6. Verify: ductape_cli("products get --tag <product_tag> --json") → check apps[] contains the app
1907
+ ductape_execute("actions.list", [app_tag]) → verify actions are registered
1846
1908
  `.trim(),
1847
1909
 
1848
1910
  products: `
@@ -1852,20 +1914,20 @@ A product is the top-level namespace for all Ductape infrastructure: apps, datab
1852
1914
  vectors, storage, brokers, sessions, caches, notifications, resilience, features, jobs, and envs.
1853
1915
  Every SDK service call resolves within a product context.
1854
1916
 
1917
+ IMPORTANT: The product module requires the access key. ALL product operations must use ductape_cli,
1918
+ not ductape_execute (which only accepts the publishable key and will return 403 for product.*).
1919
+
1855
1920
  Create a product:
1856
1921
  ductape_cli("products create --name \\"My App\\" --tag my-app")
1857
- ductape_execute("product.create", [{ name: "My App", tag: "my-app",
1858
- envs: [{ slug: "dev", name: "Development" }, { slug: "prd", name: "Production" }] }])
1859
1922
 
1860
1923
  Environments — every resource's envs array MUST cover all product env slugs:
1861
- ductape_execute("product.environments.create", [product_tag, { slug, env_name, description, active? }])
1862
- ductape_execute("product.environments.list", [product_tag])
1863
- ductape_execute("product.environments.fetch", [product_tag, slug])
1864
- BEFORE registering any resource, always run environments.list and collect all slugs.
1924
+ ductape_cli("products environments list <product_tag> --json")
1925
+ ductape_cli("products environments get <product_tag> <slug> --json")
1926
+ BEFORE registering any resource, always run environments list and collect all slugs.
1865
1927
 
1866
1928
  Fetch / update:
1867
- ductape_execute("product.fetch", [product_tag])
1868
- ductape_execute("product.update", [product_tag, { name?, description? }])
1929
+ ductape_cli("products get --tag <product_tag> --json")
1930
+ ductape_cli("products get --id <product_id> --json")
1869
1931
 
1870
1932
  Connect apps to a product:
1871
1933
  See ductape_docs({ topic: "apps" }) for product.apps.add / product.apps.list.
@@ -2153,21 +2215,32 @@ STEP 1 — UNDERSTAND the goal
2153
2215
  understand: what the feature does, what it returns, what can fail and how failures should behave.
2154
2216
 
2155
2217
  STEP 2 — INVENTORY existing Ductape components
2156
- Call ductape_execute("products.fetch", [product_tag]) to read the product.
2218
+ Call ductape_cli("products get --tag <product_tag> --json") to read the full product document.
2219
+ (product.* requires the access key — never use ductape_execute for product reads, it will return 403)
2157
2220
  Note what already exists:
2158
2221
  - databases[] → available for ctx.database.insert/query/update/delete steps
2159
- - apps[] → available for ctx.api.run steps (check app.events[] for event tags)
2222
+ - apps[] → available for ctx.api.run steps ONLY if the App is fully registered:
2223
+ (a) App record exists, (b) environments defined with base URLs,
2224
+ (c) auth scheme configured, (d) action endpoints defined,
2225
+ (e) connected to the product via product.apps.add.
2226
+ Check app.events[] for action tags. If no app exists for a service the
2227
+ feature needs to call, flag it as "App to create" in the plan — do NOT
2228
+ assume ctx.api can call any URL or schedule any job without a registered App.
2160
2229
  - notifications[] → available for ctx.notification.email/sms/push steps
2161
2230
  - storage[] → available for ctx.storage.upload/download steps
2162
- - messageBrokers[] → available for ctx.messaging.produce steps
2231
+ - messageBrokers[] → available for ctx.events.produce steps
2163
2232
  - graphs[] → available for ctx.graph steps
2164
- - features[] → can be called as child features via ctx.feature()
2233
+ - features[] → can be called as child features via ctx.feature.execute()
2165
2234
  - caches[], sessions[]
2166
2235
  Do NOT assume a component or event tag exists — verify from the product before using it.
2236
+ Do NOT treat ctx.api as a generic HTTP call or job scheduler. It requires a registered App.
2167
2237
 
2168
2238
  STEP 3 — PLAN each step
2169
2239
  For every logical step:
2170
- a. Identify which existing component handles it, or flag it as needing creation
2240
+ a. Identify which existing component handles it, or flag it as needing creation.
2241
+ If a step calls an external service, it MUST go through a registered Ductape App.
2242
+ If no App for that service exists in the product → mark it "App to create: <service name>".
2243
+ DO NOT plan a raw HTTP call, a direct dispatch to a URL, or any workaround in place of a missing App.
2171
2244
  b. Note how inputs flow: ctx.input fields, or return values from earlier steps (plain JS variables — no special notation needed)
2172
2245
  c. Decide if a rollback handler is needed (e.g. charge → refund on later failure)
2173
2246
  d. Decide allow_fail: true for non-critical steps (email, analytics, audit logs)
@@ -2211,7 +2284,8 @@ STEP 7 — HANDLE conditionals, loops, and branching (when applicable)
2211
2284
  → Only the scenario whose input matches runs at execution time
2212
2285
 
2213
2286
  STEP 8 — SET rollbacks for reversible steps
2214
- Any step that allocates a resource should undo it if a later step fails:
2287
+ Any step that allocates a resource should undo it if a later step fails.
2288
+ ctx.api.run requires a pre-registered Ductape App — 'stripe' below is the tag of a registered App:
2215
2289
  const charge = await ctx.step(
2216
2290
  'charge',
2217
2291
  async () => ctx.api.run({ app: 'stripe', event: 'create-charge', input: { amount: ctx.input.amount } }),
@@ -2271,6 +2345,52 @@ Signals and queries (for long-running features):
2271
2345
 
2272
2346
  Rollback strategies: reverse_all | reverse_critical | compensate | none
2273
2347
  Feature statuses: pending | running | completed | failed | rolled_back | rolling_back | paused
2348
+
2349
+ ━━━ FEATURE RECORDING SEMANTICS ━━━
2350
+
2351
+ When you call features.define({ handler }), the handler runs TWICE:
2352
+
2353
+ 1. RECORDING PHASE (at define time) — handler is called with a RecordingContext.
2354
+ All ctx.step() calls return lightweight proxy objects, not real data.
2355
+ This phase captures the step graph: which steps exist, their types, tags, and declared
2356
+ inputs/outputs. No real API calls, DB queries, or side effects occur.
2357
+ Arbitrary JS code OUTSIDE ctx.step() ALSO runs during recording — with proxy values.
2358
+ For loops: use recordInput so the handler sees sample data and all iterations are recorded.
2359
+ For branches: use branchOverrides so each path is captured.
2360
+
2361
+ 2. EXECUTION PHASE (at runtime) — handler is called with a real ExecutionContext.
2362
+ ctx.step() actually executes. All real Ductape component calls happen.
2363
+ Arbitrary JS logic (math, string ops, conditionals on step results) runs for real.
2364
+
2365
+ Implication: put all meaningful business logic INSIDE ctx.step() handlers, not in the
2366
+ outer handler body. Code in the outer body runs during recording with proxy values and
2367
+ may behave unexpectedly (e.g. typeof proxy === 'object' is true but .someField is a proxy).
2368
+
2369
+ Features do NOT execute arbitrary NestJS or server code directly. A feature handler can only
2370
+ call Ductape component primitives (ctx.api, ctx.database, ctx.notification, etc.) as steps.
2371
+ To invoke internal application business logic, produce a broker event from a feature step
2372
+ (ctx.messaging.produce) and consume it in your NestJS service — that is the correct pattern.
2373
+
2374
+ ━━━ ORCHESTRATION DECISION RULE ━━━
2375
+
2376
+ One component operation at a future time:
2377
+ → use that component's own dispatch method
2378
+ e.g. ductape.events.dispatch({ ..., schedule: { start_at: ... } })
2379
+ e.g. ductape.api.dispatch({ ..., schedule: { start_at: ... } })
2380
+ e.g. ductape.database.dispatch({ ..., schedule: { start_at: ... } })
2381
+
2382
+ Several durable Ductape component operations in sequence (with rollback / retry / state):
2383
+ → define a Feature, then features.dispatch to schedule it
2384
+
2385
+ Invoke internal application business logic (your own NestJS/backend service code):
2386
+ → produce a broker event (ctx.messaging.produce or ductape.events.produce)
2387
+ → consume it in your NestJS service with events.consume in onModuleInit
2388
+ → your service method runs with full access to DI, DB transactions, etc.
2389
+ Do NOT create an App Action just to call your own service over HTTP.
2390
+
2391
+ Invoke an external/public HTTP service:
2392
+ → create a Ductape App (register base URL, auth, action endpoints) then use ctx.api.run
2393
+ → requires the App to be fully registered and connected to the product first
2274
2394
  `.trim(),
2275
2395
 
2276
2396
  events: `
@@ -2369,81 +2489,173 @@ Import (register an EXISTING cloud resource):
2369
2489
  redis: { host: "...", port: 6379, password?: "..." }
2370
2490
  nats: { servers: ["nats://host:4222"], token?: "...", user?: "...", pass?: "...", tls?: true }
2371
2491
 
2372
- ━━━ STEP 2: ADD TOPIC DEFINITIONS ━━━
2373
-
2374
- For most providers (GCP Pub/Sub, Kafka, RabbitMQ, Redis, NATS, Azure Service Bus):
2375
- SKIP this step. Topics are auto-created when you create a producer (step 3).
2376
- You do NOT need to call messageBrokers.topics.create before creating a producer.
2492
+ ━━━ STEP 2: DEFINE TOPICS ━━━
2377
2493
 
2378
- Only call topics.create explicitly when:
2379
- - Using AWS SQS (must supply queueUrls per envauto-creation cannot know the queue URL)
2380
- - You want to pre-set sample data or idempotency config on the topic
2494
+ Topics MUST be defined before any consumer can subscribe to them.
2495
+ Producing to a topic also calls ensureTopicRegistered in the background but DO NOT rely on
2496
+ auto-registration for consume paths. Always create topics explicitly.
2381
2497
 
2382
- If you do need it (SQS or explicit config):
2383
2498
  ductape_execute("messageBrokers.topics.create", [product_tag, {
2384
- tag: "player-joined",
2385
- name: "Player Joined",
2386
- broker: "notifications-broker", // broker component tag
2499
+ tag: "order-created", // topic tag (just the topic part, NOT "broker:topic")
2500
+ name: "Order Created",
2501
+ broker: "order-events", // broker component tag
2387
2502
  description?: string,
2388
- sample: { playerId: "string", username: "string" }, // example message shape
2389
- idempotent?: boolean,
2390
- // SQS only — map per-env queue URLs (each topic can be a different queue):
2503
+ sample: { orderId: "string", total: 0 }, // documents expected message shape
2504
+ idempotent?: boolean, // if true, Ductape deduplicates by idempotency_key
2505
+ // AWS SQS only — must supply per-env queue URL:
2391
2506
  queueUrls?: [
2392
- { env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123456/my-product-player-joined-snd" },
2393
- { env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123456/my-product-player-joined-prd" }
2507
+ { env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-snd" },
2508
+ { env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-prd" }
2394
2509
  ]
2395
2510
  }])
2396
2511
 
2397
- List topics on a broker:
2512
+ List / fetch topics:
2398
2513
  ductape_execute("messageBrokers.topics.list", [product_tag, "broker-tag"])
2514
+ ductape_execute("messageBrokers.fetch", [product_tag, "broker-tag"]) → includes topics[]
2399
2515
 
2400
- ━━━ STEP 3: PRODUCE AND CONSUME — WRITTEN IN APPLICATION CODE ━━━
2516
+ ━━━ STEP 3: PRODUCE — WRITTEN IN APPLICATION CODE ━━━
2401
2517
 
2402
- There is NO admin command or file to declare producers/consumers.
2403
- There is NO "create producer" step before writing code.
2404
- Producers and consumers are registered automatically by the SDK the first time your code calls
2405
- produce/consume you do not pre-declare them.
2518
+ There is NO admin command to declare a producer. Producers are auto-registered by the SDK on
2519
+ the first produce call you do not pre-declare them.
2520
+ Do NOT call ductape_generate_payload for messaging. The producer owns the schema.
2521
+ Infer the message shape from context, present it to the user for approval, then implement.
2406
2522
 
2407
- The entire producer/consumer contract is the code you write in your controllers or services:
2408
-
2409
- Produce (publish a message) write in your service/controller:
2410
- Do NOT call ductape_generate_payload for messaging. Events have no pre-existing backend schema
2411
- to discover — the producer defines the schema. Instead, infer the message shape from context
2412
- (event name, existing data models, user input), present it to the user for approval, then implement.
2523
+ GENERAL BACKEND (TypeScript/Node.js not NestJS):
2524
+ import Ductape from '@ductape/sdk';
2525
+ const ductape = new Ductape({ accessKey: 'your-access-key' });
2413
2526
  await ductape.events.produce({
2414
2527
  product: "my-product",
2415
2528
  env: "prd",
2416
- event: "broker_tag:topic_tag", // "broker_tag:topic_tag" — always colon-separated
2417
- message: { key: value }, // shape inferred from context, approved by user
2529
+ event: "broker-tag:topic-tag", // always colon-separated
2530
+ message: { key: value },
2531
+ session?: "session-tag:jwt", // optional — traces message to a user session
2532
+ });
2533
+ // Idempotent publish (deduplicates — prevents double-processing on retries):
2534
+ await ductape.events.publishIdempotent({
2535
+ product, env, event, message,
2536
+ idempotencyKey: "order-123-charge", // stable key unique to this logical operation
2537
+ idempotencyTtl?: 86400, // seconds; default 86400 (24h)
2418
2538
  });
2419
- Idempotent publish (deduplicates by key):
2420
- await ductape.events.publishIdempotent({ product, env, event, message, idempotencyKey, idempotencyTtl? })
2421
2539
 
2422
- Consume (subscribe) write in your service/controller:
2540
+ NESTJSmethod decorator:
2541
+ import { Messaging } from '@ductape/nestjs';
2542
+ @Injectable() export class OrdersService {
2543
+ @Messaging.Produce({ event: 'order-events:order-created' })
2544
+ emitOrderCreated(payload: { orderId: string; total: number }) { return payload; }
2545
+
2546
+ // Scheduled dispatch — fire-and-forget with optional schedule:
2547
+ @Messaging.Dispatch({ broker: 'order-events', event: 'order-events:order-created',
2548
+ schedule?: { start_at?, cron?, every?, limit?, tz? } })
2549
+ scheduleOrderNotification(payload: Record<string, unknown>) { return payload; }
2550
+ }
2551
+
2552
+ CLIENT-SIDE (browser — publishable key):
2553
+ Clients CAN produce messages using a publishable key + session token.
2554
+ Only produce to topics whose schema is safe for client authorship.
2555
+ NEVER allow clients to produce to topics that trigger privileged server-side operations
2556
+ (payments, admin actions, state mutations) — those must go through a backend endpoint first.
2557
+ import Ductape from '@ductape/sdk';
2558
+ const ductape = new Ductape({ publishableKey: 'pk_...', env: 'prd', product: 'my-product' });
2559
+ await ductape.events.produce({
2560
+ event: "user-events:user-action",
2561
+ message: { action: "button-click", screen: "dashboard" },
2562
+ session: "user-session:eyJ...", // REQUIRED for client-side produce
2563
+ });
2564
+
2565
+ SCHEDULED DISPATCH (background job):
2566
+ ductape_execute("messageBrokers.dispatch", [{
2567
+ product, env,
2568
+ broker: "order-events", // broker tag
2569
+ event: "order-events:reminder-due", // "broker:topic"
2570
+ input: { message: { orderId: "123" } },
2571
+ retries?: 3,
2572
+ session?: "session-tag:jwt",
2573
+ schedule?: {
2574
+ start_at?: 1735689600000, // Unix ms or ISO string
2575
+ cron?: "0 9 * * *", // recurring cron
2576
+ every?: 86400000, // recurring interval ms
2577
+ limit?: 10, // max repetitions
2578
+ endDate?: "2026-12-31",
2579
+ tz?: "America/New_York",
2580
+ },
2581
+ }])
2582
+ Returns: { job_id, status: "scheduled"|"queued", scheduled_at, recurring, next_run_at? }
2583
+
2584
+ ━━━ STEP 4: CONSUME — WRITTEN IN APPLICATION CODE ━━━
2585
+
2586
+ Consumers are auto-registered by the SDK on first consume call.
2587
+ Consumer registration options (all optional — used for tracking in Workbench):
2588
+ consumer?: { tag?: string, name?: string, description?: string }
2589
+ If tag is omitted, Ductape generates one: "consumer-<brokerTag>-<topicTag>".
2590
+
2591
+ ACK BEHAVIOR (automatic):
2592
+ - Callback returns successfully → message is acknowledged (ack)
2593
+ - Callback throws → message is tracked as failed; broker nacks/retries per provider behavior
2594
+ - After max retries → message moves to dead-letter queue (DLQ)
2595
+ There is no manual ack API. Acknowledgement is implicit from callback outcome.
2596
+
2597
+ CONSUMER GROUPS (Kafka-specific):
2598
+ Consumer groups are set in the broker's envs[].config.groupId (at broker registration time).
2599
+ All service instances sharing the same groupId form a consumer group and share partition load.
2600
+ To configure: set groupId in the kafka config when creating/updating the broker.
2601
+
2602
+ CONCURRENCY:
2603
+ Ductape has no per-consumer concurrency setting. Concurrency is determined by:
2604
+ - Number of running service instances (horizontal scale)
2605
+ - Broker-level partition count (Kafka) or visibility timeout (SQS)
2606
+ Run multiple instances of your service to scale consumption.
2607
+
2608
+ GENERAL BACKEND (TypeScript/Node.js — not NestJS):
2609
+ Start consuming in your module init or service startup:
2423
2610
  await ductape.events.consume({
2424
2611
  product: "my-product",
2425
2612
  env: "prd",
2426
- event: "broker_tag:topic_tag",
2427
- callback: async (message) => { /* handle message */ },
2613
+ event: "order-events:order-created",
2614
+ callback: async (message) => {
2615
+ // All real processing logic goes here.
2616
+ // Throw to nack. Return to ack.
2617
+ await processOrder(message as { orderId: string; total: number });
2618
+ },
2619
+ consumer?: { tag: "order-processor", name: "Order Processor" },
2428
2620
  });
2429
- Callback errors are re-thrown so the broker can nack/retry.
2430
2621
 
2431
- Background dispatch with scheduling write in your service/controller:
2432
- await ductape.events.dispatch({ product, env, broker, event, input: { message },
2433
- schedule?: { start_at?, cron?, every?, limit?, endDate?, tz? } })
2622
+ NESTJS use SDK in onModuleInit (no @Messaging.Consume decorator exists yet):
2623
+ @Injectable()
2624
+ export class OrderConsumerService implements OnModuleInit {
2625
+ constructor(private readonly ductape: Ductape) {}
2626
+ async onModuleInit() {
2627
+ await this.ductape.events.consume({
2628
+ product: "my-product",
2629
+ env: process.env.DUCTAPE_ENV || 'prd',
2630
+ event: "order-events:order-created",
2631
+ callback: async (message) => { await this.handle(message); },
2632
+ consumer: { tag: "order-consumer", name: "Order Consumer" },
2633
+ });
2634
+ }
2635
+ private async handle(message: unknown) { /* business logic */ }
2636
+ }
2434
2637
 
2435
- For the four standard producer declarations (match-state, match-report, projection-updated,
2436
- notification), write these produce calls in the relevant application service methods there is
2437
- no separate configuration file or CLI step. The SDK creates the producer metadata on first call.
2638
+ CLIENT-SIDE: Clients CANNOT consume. Event consumption is always server-side only.
2639
+ This is the key distinction between server topics (produce + consume) and client-observable
2640
+ topics (produce from client, consume on server). Never set up a consumer in browser code.
2438
2641
 
2439
- Message payload is AES-encrypted before the tracking API call — tracking never sees plaintext.
2642
+ DEAD-LETTER QUEUE (DLQ):
2643
+ Messages whose callbacks consistently throw are automatically moved to the DLQ.
2644
+ Query: ductape_execute("messageBrokers.messages.getDeadLetters",
2645
+ [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }])
2646
+ Reprocess: ductape_execute("messageBrokers.reprocessDLQ",
2647
+ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }])
2648
+ Replay: ductape_execute("messageBrokers.replayEvent",
2649
+ [{ product, env, eventId, force? }])
2440
2650
 
2441
2651
  ━━━ OBSERVABILITY ━━━
2442
2652
 
2443
2653
  messageBrokers.messages.query [{ product, env, brokerTag, topicTag?, status?, page?, limit? }]
2444
2654
  messageBrokers.messages.getStats [{ product, env, brokerTag }]
2445
2655
  messageBrokers.messages.getDashboard [{ product, env, brokerTag }]
2446
- messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, limit? }]
2656
+ messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }]
2657
+ messageBrokers.messages.getProducers [{ product, env, brokerTag, topicTag?, page?, limit? }]
2658
+ messageBrokers.messages.getConsumers [{ product, env, brokerTag, topicTag?, page?, limit? }]
2447
2659
  messageBrokers.replayEvent [{ product, env, eventId, force? }]
2448
2660
  messageBrokers.reprocessDLQ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }]
2449
2661
  messageBrokers.checkIdempotency [{ product, env, brokerTag, idempotency_key }]