@ductape/mcp 0.1.37 → 0.1.39
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +285 -73
- package/package.json +1 -1
- package/src/index.ts +285 -73
package/dist/index.js
CHANGED
|
@@ -276,19 +276,24 @@ SPECIALIZED MODULES (for injecting handles directly without @InjectContext):
|
|
|
276
276
|
ALL params are passed as a JSON array in positional order matching the SDK signature.
|
|
277
277
|
|
|
278
278
|
━━━ MODULE: product ━━━
|
|
279
|
+
IMPORTANT: ALL product.* methods require the access key and will return 403 with a publishable key.
|
|
280
|
+
Use ductape_cli for ALL product operations — never ductape_execute:
|
|
281
|
+
ductape_cli("products get --tag <tag> --json") ← fetch product + full inventory
|
|
282
|
+
ductape_cli("products create --name <name> --tag <tag>")
|
|
283
|
+
ductape_cli("products environments list <tag> --json")
|
|
284
|
+
ductape_cli("products environments get <tag> <slug> --json")
|
|
285
|
+
ductape_cli("products apps list --product <id> --json")
|
|
286
|
+
|
|
287
|
+
SDK method signatures (for reference, admin key only):
|
|
279
288
|
product.create [data: { name, description, tag?, envs?: [{slug, name}] }]
|
|
280
289
|
product.fetch [product_tag: string]
|
|
281
290
|
product.update [product_tag: string, data: { name?: string, description?: string }]
|
|
282
|
-
product.
|
|
283
|
-
product.environments.create [product_tag, data: { slug: string, env_name: string, description: string, active?: boolean }]
|
|
284
|
-
product.environments.update [product_tag, slug: string, data: { env_name?: string, description?: string, active?: boolean }]
|
|
291
|
+
product.environments.create [product_tag, data: { slug, env_name, description, active? }]
|
|
285
292
|
product.environments.list [product_tag]
|
|
286
293
|
product.environments.fetch [product_tag, slug]
|
|
287
|
-
product.apps.
|
|
288
|
-
product.apps.add [product_tag, app: { access_tag: string, envs: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
|
|
294
|
+
product.apps.add [product_tag, app: { access_tag, envs: [{ app_env_slug, product_env_slug, variables?, auth? }] }]
|
|
289
295
|
product.apps.list [product_tag]
|
|
290
296
|
product.apps.fetch [product_tag, access_tag]
|
|
291
|
-
product.apps.update [product_tag, access_tag: string, data: { version?: string, envs?: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
|
|
292
297
|
|
|
293
298
|
━━━ MODULE: app ━━━
|
|
294
299
|
app.create [data: { app_name: string, description: string, unique?: boolean }]
|
|
@@ -1722,6 +1727,31 @@ Important:
|
|
|
1722
1727
|
apps: `
|
|
1723
1728
|
DUCTAPE APPS
|
|
1724
1729
|
|
|
1730
|
+
WHAT AN APP IS:
|
|
1731
|
+
A Ductape App is a pre-configured, versioned API integration definition. It is NOT a generic HTTP
|
|
1732
|
+
client, NOT a job scheduler, and NOT anything you can call without registering first.
|
|
1733
|
+
|
|
1734
|
+
An App must be fully set up in Ductape before any code can use it:
|
|
1735
|
+
1. The App record must be created (name, tag, description)
|
|
1736
|
+
2. Environments must be added (each environment slug → base URL for that stage)
|
|
1737
|
+
3. Auth scheme must be configured (how outbound requests authenticate: apikey, bearer, OAuth2, etc.)
|
|
1738
|
+
4. Action endpoints must be defined (each action = one HTTP endpoint spec: method, path, body/query/header shape, response shape)
|
|
1739
|
+
5. The App must be connected to the product (product.apps.add) and its envs mapped
|
|
1740
|
+
|
|
1741
|
+
ONLY after all five steps can any code call:
|
|
1742
|
+
ctx.api.run({ app: '<app_tag>', event: '<action_tag>', input: { ... } }) ← in a feature handler
|
|
1743
|
+
actions.run([{ product, env, app: '<app_tag>', action: '<action_tag>', input }]) ← at runtime
|
|
1744
|
+
|
|
1745
|
+
ctx.api (alias for ctx.action) is NOT a generic HTTP call. It ONLY invokes a pre-registered
|
|
1746
|
+
Ductape App Action. If the App or action tag does not exist in the product, the call will fail.
|
|
1747
|
+
actions.dispatch is the same as actions.run but scheduled as a background job — it still requires
|
|
1748
|
+
a registered App. There is no way to dispatch a job to an arbitrary URL via ctx.api or actions.dispatch.
|
|
1749
|
+
|
|
1750
|
+
If a feature step needs to call an external service and no App is registered for it yet:
|
|
1751
|
+
→ Flag it as "App to create" in your plan (STEP 4 of the feature design workflow)
|
|
1752
|
+
→ Create the App and all its actions first (see below)
|
|
1753
|
+
→ Only then write the ctx.api.run call
|
|
1754
|
+
|
|
1725
1755
|
An app is a versioned API integration definition. It contains environments (base URLs), actions
|
|
1726
1756
|
(individual endpoint specs), auth schemes, webhooks, variables, and constants.
|
|
1727
1757
|
|
|
@@ -1737,6 +1767,21 @@ Manage environments (base URLs per stage):
|
|
|
1737
1767
|
ductape_execute("app.environments.create", [app_tag, { slug, env_name, base_url }])
|
|
1738
1768
|
ductape_execute("app.environments.list", [app_tag])
|
|
1739
1769
|
|
|
1770
|
+
Discover apps in a product and their actions (ALWAYS do this before writing any ctx.api.run call):
|
|
1771
|
+
Step 1 — list apps connected to the product:
|
|
1772
|
+
ductape_cli("products get --tag <product_tag> --json") → full product document; check apps[]
|
|
1773
|
+
ductape_cli("products apps list --product <product_id> --json") → apps[] with access_tag, envs
|
|
1774
|
+
Step 2 — list actions in an app:
|
|
1775
|
+
ductape_execute("actions.list", [app_tag]) → returns all action tags + names
|
|
1776
|
+
Step 3 — fetch the input schema for an action:
|
|
1777
|
+
ductape_execute("actions.fetch", [app_tag, action_tag])
|
|
1778
|
+
→ returns { body: {fieldName: {type, required}}, params: {}, query: {}, headers: {} }
|
|
1779
|
+
OR: call ductape_generate_payload (operation_family="action", method="run",
|
|
1780
|
+
targets={app: "app_tag", action: "action_tag"}) to get the exact resolved payload shape
|
|
1781
|
+
Step 4 — call ductape_schema({ module: "app" }) if you need the JSON schema for creating/updating
|
|
1782
|
+
app resources (not for runtime input — use actions.fetch or ductape_generate_payload for that)
|
|
1783
|
+
NEVER assume action input field names. Always fetch the action definition first.
|
|
1784
|
+
|
|
1740
1785
|
Manage actions (individual API endpoints):
|
|
1741
1786
|
ductape_execute("actions.create", [app_tag, {
|
|
1742
1787
|
tag, name, resource: "/users/{id}", method: "GET"|"POST"|"PUT"|"PATCH"|"DELETE",
|
|
@@ -1750,6 +1795,16 @@ Manage actions (individual API endpoints):
|
|
|
1750
1795
|
ductape_execute("actions.list", [app_tag])
|
|
1751
1796
|
ductape_execute("actions.fetch", [app_tag, action_tag])
|
|
1752
1797
|
|
|
1798
|
+
Action input — flat input format:
|
|
1799
|
+
Fields are resolved to the correct location (body/params/query/headers) by matching the action schema.
|
|
1800
|
+
For ambiguous keys, use explicit prefixes:
|
|
1801
|
+
input: { amount: 1000 } → auto-resolved (body.amount if body field exists)
|
|
1802
|
+
input: { "body:amount": 1000 } → explicit body
|
|
1803
|
+
input: { "params:id": "user_123" } → route parameter
|
|
1804
|
+
input: { "query:limit": 10 } → query string
|
|
1805
|
+
input: { "headers:X-Idempotency-Key": "..." } → request header
|
|
1806
|
+
Always use ductape_generate_payload or actions.fetch to know the exact field names — never guess.
|
|
1807
|
+
|
|
1753
1808
|
Run an action at runtime:
|
|
1754
1809
|
→ CALL ductape_generate_payload FIRST (operation_family="action", method="run", targets={app, action})
|
|
1755
1810
|
ductape_execute("actions.run", [{ product, env, app, action, input: { "body:field": value } }])
|
|
@@ -1769,14 +1824,21 @@ Variables (per-env mutable values) and Constants (fixed values):
|
|
|
1769
1824
|
ductape_execute("app.constants.create", [app_tag, { key, value }])
|
|
1770
1825
|
|
|
1771
1826
|
Connecting an app to a product (after creation):
|
|
1772
|
-
|
|
1773
|
-
|
|
1774
|
-
|
|
1775
|
-
|
|
1776
|
-
|
|
1777
|
-
}])
|
|
1778
|
-
ductape_execute("
|
|
1779
|
-
ductape_execute("
|
|
1827
|
+
NOTE: All product.* module methods require the access key and CANNOT use ductape_execute.
|
|
1828
|
+
Use ductape_cli for all product-level operations.
|
|
1829
|
+
|
|
1830
|
+
FULL FLOW to make an app callable from a product:
|
|
1831
|
+
1. Create the app: ductape_cli("apps create --name \\"Service Name\\" --description \\"...\\"")
|
|
1832
|
+
2. Add environments: ductape_execute("app.environments.create", [app_tag, { slug: "prd", env_name: "Production", base_url: "https://api.example.com" }])
|
|
1833
|
+
3. Configure auth: ductape_execute("auths.create", [app_tag, { tag, name, setup_type: "apikey"|"bearer"|"basic"|"oauth2", expiry, period }])
|
|
1834
|
+
4. Define actions: ductape_execute("actions.create", [app_tag, { tag, name, resource, method, body?, params?, query?, headers?, response? }])
|
|
1835
|
+
OR import: ductape_cli("apps import <file.json> -t postman|openapi")
|
|
1836
|
+
5. Connect to product: Requires the product_id (from ductape_cli("products get --tag <tag> --json")).
|
|
1837
|
+
There is no CLI command for this step — the SDK product.apps.add method requires
|
|
1838
|
+
an access key which only the backend can provide. Use ductape_execute via an
|
|
1839
|
+
admin-authenticated context, or connect via the Workbench UI.
|
|
1840
|
+
6. Verify: ductape_cli("products get --tag <product_tag> --json") → check apps[] contains the app
|
|
1841
|
+
ductape_execute("actions.list", [app_tag]) → verify actions are registered
|
|
1780
1842
|
`.trim(),
|
|
1781
1843
|
products: `
|
|
1782
1844
|
DUCTAPE PRODUCTS
|
|
@@ -1785,20 +1847,20 @@ A product is the top-level namespace for all Ductape infrastructure: apps, datab
|
|
|
1785
1847
|
vectors, storage, brokers, sessions, caches, notifications, resilience, features, jobs, and envs.
|
|
1786
1848
|
Every SDK service call resolves within a product context.
|
|
1787
1849
|
|
|
1850
|
+
IMPORTANT: The product module requires the access key. ALL product operations must use ductape_cli,
|
|
1851
|
+
not ductape_execute (which only accepts the publishable key and will return 403 for product.*).
|
|
1852
|
+
|
|
1788
1853
|
Create a product:
|
|
1789
1854
|
ductape_cli("products create --name \\"My App\\" --tag my-app")
|
|
1790
|
-
ductape_execute("product.create", [{ name: "My App", tag: "my-app",
|
|
1791
|
-
envs: [{ slug: "dev", name: "Development" }, { slug: "prd", name: "Production" }] }])
|
|
1792
1855
|
|
|
1793
1856
|
Environments — every resource's envs array MUST cover all product env slugs:
|
|
1794
|
-
|
|
1795
|
-
|
|
1796
|
-
|
|
1797
|
-
BEFORE registering any resource, always run environments.list and collect all slugs.
|
|
1857
|
+
ductape_cli("products environments list <product_tag> --json")
|
|
1858
|
+
ductape_cli("products environments get <product_tag> <slug> --json")
|
|
1859
|
+
BEFORE registering any resource, always run environments list and collect all slugs.
|
|
1798
1860
|
|
|
1799
1861
|
Fetch / update:
|
|
1800
|
-
|
|
1801
|
-
|
|
1862
|
+
ductape_cli("products get --tag <product_tag> --json")
|
|
1863
|
+
ductape_cli("products get --id <product_id> --json")
|
|
1802
1864
|
|
|
1803
1865
|
Connect apps to a product:
|
|
1804
1866
|
See ductape_docs({ topic: "apps" }) for product.apps.add / product.apps.list.
|
|
@@ -2081,21 +2143,32 @@ STEP 1 — UNDERSTAND the goal
|
|
|
2081
2143
|
understand: what the feature does, what it returns, what can fail and how failures should behave.
|
|
2082
2144
|
|
|
2083
2145
|
STEP 2 — INVENTORY existing Ductape components
|
|
2084
|
-
Call
|
|
2146
|
+
Call ductape_cli("products get --tag <product_tag> --json") to read the full product document.
|
|
2147
|
+
(product.* requires the access key — never use ductape_execute for product reads, it will return 403)
|
|
2085
2148
|
Note what already exists:
|
|
2086
2149
|
- databases[] → available for ctx.database.insert/query/update/delete steps
|
|
2087
|
-
- apps[] → available for ctx.api.run steps
|
|
2150
|
+
- apps[] → available for ctx.api.run steps ONLY if the App is fully registered:
|
|
2151
|
+
(a) App record exists, (b) environments defined with base URLs,
|
|
2152
|
+
(c) auth scheme configured, (d) action endpoints defined,
|
|
2153
|
+
(e) connected to the product via product.apps.add.
|
|
2154
|
+
Check app.events[] for action tags. If no app exists for a service the
|
|
2155
|
+
feature needs to call, flag it as "App to create" in the plan — do NOT
|
|
2156
|
+
assume ctx.api can call any URL or schedule any job without a registered App.
|
|
2088
2157
|
- notifications[] → available for ctx.notification.email/sms/push steps
|
|
2089
2158
|
- storage[] → available for ctx.storage.upload/download steps
|
|
2090
|
-
- messageBrokers[] → available for ctx.
|
|
2159
|
+
- messageBrokers[] → available for ctx.events.produce steps
|
|
2091
2160
|
- graphs[] → available for ctx.graph steps
|
|
2092
|
-
- features[] → can be called as child features via ctx.feature()
|
|
2161
|
+
- features[] → can be called as child features via ctx.feature.execute()
|
|
2093
2162
|
- caches[], sessions[]
|
|
2094
2163
|
Do NOT assume a component or event tag exists — verify from the product before using it.
|
|
2164
|
+
Do NOT treat ctx.api as a generic HTTP call or job scheduler. It requires a registered App.
|
|
2095
2165
|
|
|
2096
2166
|
STEP 3 — PLAN each step
|
|
2097
2167
|
For every logical step:
|
|
2098
|
-
a. Identify which existing component handles it, or flag it as needing creation
|
|
2168
|
+
a. Identify which existing component handles it, or flag it as needing creation.
|
|
2169
|
+
If a step calls an external service, it MUST go through a registered Ductape App.
|
|
2170
|
+
If no App for that service exists in the product → mark it "App to create: <service name>".
|
|
2171
|
+
DO NOT plan a raw HTTP call, a direct dispatch to a URL, or any workaround in place of a missing App.
|
|
2099
2172
|
b. Note how inputs flow: ctx.input fields, or return values from earlier steps (plain JS variables — no special notation needed)
|
|
2100
2173
|
c. Decide if a rollback handler is needed (e.g. charge → refund on later failure)
|
|
2101
2174
|
d. Decide allow_fail: true for non-critical steps (email, analytics, audit logs)
|
|
@@ -2139,7 +2212,8 @@ STEP 7 — HANDLE conditionals, loops, and branching (when applicable)
|
|
|
2139
2212
|
→ Only the scenario whose input matches runs at execution time
|
|
2140
2213
|
|
|
2141
2214
|
STEP 8 — SET rollbacks for reversible steps
|
|
2142
|
-
Any step that allocates a resource should undo it if a later step fails
|
|
2215
|
+
Any step that allocates a resource should undo it if a later step fails.
|
|
2216
|
+
ctx.api.run requires a pre-registered Ductape App — 'stripe' below is the tag of a registered App:
|
|
2143
2217
|
const charge = await ctx.step(
|
|
2144
2218
|
'charge',
|
|
2145
2219
|
async () => ctx.api.run({ app: 'stripe', event: 'create-charge', input: { amount: ctx.input.amount } }),
|
|
@@ -2199,6 +2273,52 @@ Signals and queries (for long-running features):
|
|
|
2199
2273
|
|
|
2200
2274
|
Rollback strategies: reverse_all | reverse_critical | compensate | none
|
|
2201
2275
|
Feature statuses: pending | running | completed | failed | rolled_back | rolling_back | paused
|
|
2276
|
+
|
|
2277
|
+
━━━ FEATURE RECORDING SEMANTICS ━━━
|
|
2278
|
+
|
|
2279
|
+
When you call features.define({ handler }), the handler runs TWICE:
|
|
2280
|
+
|
|
2281
|
+
1. RECORDING PHASE (at define time) — handler is called with a RecordingContext.
|
|
2282
|
+
All ctx.step() calls return lightweight proxy objects, not real data.
|
|
2283
|
+
This phase captures the step graph: which steps exist, their types, tags, and declared
|
|
2284
|
+
inputs/outputs. No real API calls, DB queries, or side effects occur.
|
|
2285
|
+
Arbitrary JS code OUTSIDE ctx.step() ALSO runs during recording — with proxy values.
|
|
2286
|
+
For loops: use recordInput so the handler sees sample data and all iterations are recorded.
|
|
2287
|
+
For branches: use branchOverrides so each path is captured.
|
|
2288
|
+
|
|
2289
|
+
2. EXECUTION PHASE (at runtime) — handler is called with a real ExecutionContext.
|
|
2290
|
+
ctx.step() actually executes. All real Ductape component calls happen.
|
|
2291
|
+
Arbitrary JS logic (math, string ops, conditionals on step results) runs for real.
|
|
2292
|
+
|
|
2293
|
+
Implication: put all meaningful business logic INSIDE ctx.step() handlers, not in the
|
|
2294
|
+
outer handler body. Code in the outer body runs during recording with proxy values and
|
|
2295
|
+
may behave unexpectedly (e.g. typeof proxy === 'object' is true but .someField is a proxy).
|
|
2296
|
+
|
|
2297
|
+
Features do NOT execute arbitrary NestJS or server code directly. A feature handler can only
|
|
2298
|
+
call Ductape component primitives (ctx.api, ctx.database, ctx.notification, etc.) as steps.
|
|
2299
|
+
To invoke internal application business logic, produce a broker event from a feature step
|
|
2300
|
+
(ctx.messaging.produce) and consume it in your NestJS service — that is the correct pattern.
|
|
2301
|
+
|
|
2302
|
+
━━━ ORCHESTRATION DECISION RULE ━━━
|
|
2303
|
+
|
|
2304
|
+
One component operation at a future time:
|
|
2305
|
+
→ use that component's own dispatch method
|
|
2306
|
+
e.g. ductape.events.dispatch({ ..., schedule: { start_at: ... } })
|
|
2307
|
+
e.g. ductape.api.dispatch({ ..., schedule: { start_at: ... } })
|
|
2308
|
+
e.g. ductape.database.dispatch({ ..., schedule: { start_at: ... } })
|
|
2309
|
+
|
|
2310
|
+
Several durable Ductape component operations in sequence (with rollback / retry / state):
|
|
2311
|
+
→ define a Feature, then features.dispatch to schedule it
|
|
2312
|
+
|
|
2313
|
+
Invoke internal application business logic (your own NestJS/backend service code):
|
|
2314
|
+
→ produce a broker event (ctx.messaging.produce or ductape.events.produce)
|
|
2315
|
+
→ consume it in your NestJS service with events.consume in onModuleInit
|
|
2316
|
+
→ your service method runs with full access to DI, DB transactions, etc.
|
|
2317
|
+
Do NOT create an App Action just to call your own service over HTTP.
|
|
2318
|
+
|
|
2319
|
+
Invoke an external/public HTTP service:
|
|
2320
|
+
→ create a Ductape App (register base URL, auth, action endpoints) then use ctx.api.run
|
|
2321
|
+
→ requires the App to be fully registered and connected to the product first
|
|
2202
2322
|
`.trim(),
|
|
2203
2323
|
events: `
|
|
2204
2324
|
DUCTAPE EVENTS (MESSAGE BROKERS)
|
|
@@ -2296,81 +2416,173 @@ Import (register an EXISTING cloud resource):
|
|
|
2296
2416
|
redis: { host: "...", port: 6379, password?: "..." }
|
|
2297
2417
|
nats: { servers: ["nats://host:4222"], token?: "...", user?: "...", pass?: "...", tls?: true }
|
|
2298
2418
|
|
|
2299
|
-
━━━ STEP 2:
|
|
2300
|
-
|
|
2301
|
-
For most providers (GCP Pub/Sub, Kafka, RabbitMQ, Redis, NATS, Azure Service Bus):
|
|
2302
|
-
SKIP this step. Topics are auto-created when you create a producer (step 3).
|
|
2303
|
-
You do NOT need to call messageBrokers.topics.create before creating a producer.
|
|
2419
|
+
━━━ STEP 2: DEFINE TOPICS ━━━
|
|
2304
2420
|
|
|
2305
|
-
|
|
2306
|
-
|
|
2307
|
-
|
|
2421
|
+
Topics MUST be defined before any consumer can subscribe to them.
|
|
2422
|
+
Producing to a topic also calls ensureTopicRegistered in the background — but DO NOT rely on
|
|
2423
|
+
auto-registration for consume paths. Always create topics explicitly.
|
|
2308
2424
|
|
|
2309
|
-
If you do need it (SQS or explicit config):
|
|
2310
2425
|
ductape_execute("messageBrokers.topics.create", [product_tag, {
|
|
2311
|
-
tag: "
|
|
2312
|
-
name: "
|
|
2313
|
-
broker: "
|
|
2426
|
+
tag: "order-created", // topic tag (just the topic part, NOT "broker:topic")
|
|
2427
|
+
name: "Order Created",
|
|
2428
|
+
broker: "order-events", // broker component tag
|
|
2314
2429
|
description?: string,
|
|
2315
|
-
sample: {
|
|
2316
|
-
idempotent?: boolean,
|
|
2317
|
-
// SQS only —
|
|
2430
|
+
sample: { orderId: "string", total: 0 }, // documents expected message shape
|
|
2431
|
+
idempotent?: boolean, // if true, Ductape deduplicates by idempotency_key
|
|
2432
|
+
// AWS SQS only — must supply per-env queue URL:
|
|
2318
2433
|
queueUrls?: [
|
|
2319
|
-
{ env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/
|
|
2320
|
-
{ env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/
|
|
2434
|
+
{ env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-snd" },
|
|
2435
|
+
{ env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-prd" }
|
|
2321
2436
|
]
|
|
2322
2437
|
}])
|
|
2323
2438
|
|
|
2324
|
-
List
|
|
2439
|
+
List / fetch topics:
|
|
2325
2440
|
ductape_execute("messageBrokers.topics.list", [product_tag, "broker-tag"])
|
|
2441
|
+
ductape_execute("messageBrokers.fetch", [product_tag, "broker-tag"]) → includes topics[]
|
|
2326
2442
|
|
|
2327
|
-
━━━ STEP 3: PRODUCE
|
|
2443
|
+
━━━ STEP 3: PRODUCE — WRITTEN IN APPLICATION CODE ━━━
|
|
2328
2444
|
|
|
2329
|
-
There is NO admin command
|
|
2330
|
-
|
|
2331
|
-
|
|
2332
|
-
|
|
2445
|
+
There is NO admin command to declare a producer. Producers are auto-registered by the SDK on
|
|
2446
|
+
the first produce call — you do not pre-declare them.
|
|
2447
|
+
Do NOT call ductape_generate_payload for messaging. The producer owns the schema.
|
|
2448
|
+
Infer the message shape from context, present it to the user for approval, then implement.
|
|
2333
2449
|
|
|
2334
|
-
|
|
2335
|
-
|
|
2336
|
-
|
|
2337
|
-
Do NOT call ductape_generate_payload for messaging. Events have no pre-existing backend schema
|
|
2338
|
-
to discover — the producer defines the schema. Instead, infer the message shape from context
|
|
2339
|
-
(event name, existing data models, user input), present it to the user for approval, then implement.
|
|
2450
|
+
GENERAL BACKEND (TypeScript/Node.js — not NestJS):
|
|
2451
|
+
import Ductape from '@ductape/sdk';
|
|
2452
|
+
const ductape = new Ductape({ accessKey: 'your-access-key' });
|
|
2340
2453
|
await ductape.events.produce({
|
|
2341
2454
|
product: "my-product",
|
|
2342
2455
|
env: "prd",
|
|
2343
|
-
event: "
|
|
2344
|
-
message: { key: value },
|
|
2456
|
+
event: "broker-tag:topic-tag", // always colon-separated
|
|
2457
|
+
message: { key: value },
|
|
2458
|
+
session?: "session-tag:jwt", // optional — traces message to a user session
|
|
2459
|
+
});
|
|
2460
|
+
// Idempotent publish (deduplicates — prevents double-processing on retries):
|
|
2461
|
+
await ductape.events.publishIdempotent({
|
|
2462
|
+
product, env, event, message,
|
|
2463
|
+
idempotencyKey: "order-123-charge", // stable key unique to this logical operation
|
|
2464
|
+
idempotencyTtl?: 86400, // seconds; default 86400 (24h)
|
|
2465
|
+
});
|
|
2466
|
+
|
|
2467
|
+
NESTJS — method decorator:
|
|
2468
|
+
import { Messaging } from '@ductape/nestjs';
|
|
2469
|
+
@Injectable() export class OrdersService {
|
|
2470
|
+
@Messaging.Produce({ event: 'order-events:order-created' })
|
|
2471
|
+
emitOrderCreated(payload: { orderId: string; total: number }) { return payload; }
|
|
2472
|
+
|
|
2473
|
+
// Scheduled dispatch — fire-and-forget with optional schedule:
|
|
2474
|
+
@Messaging.Dispatch({ broker: 'order-events', event: 'order-events:order-created',
|
|
2475
|
+
schedule?: { start_at?, cron?, every?, limit?, tz? } })
|
|
2476
|
+
scheduleOrderNotification(payload: Record<string, unknown>) { return payload; }
|
|
2477
|
+
}
|
|
2478
|
+
|
|
2479
|
+
CLIENT-SIDE (browser — publishable key):
|
|
2480
|
+
Clients CAN produce messages using a publishable key + session token.
|
|
2481
|
+
Only produce to topics whose schema is safe for client authorship.
|
|
2482
|
+
NEVER allow clients to produce to topics that trigger privileged server-side operations
|
|
2483
|
+
(payments, admin actions, state mutations) — those must go through a backend endpoint first.
|
|
2484
|
+
import Ductape from '@ductape/sdk';
|
|
2485
|
+
const ductape = new Ductape({ publishableKey: 'pk_...', env: 'prd', product: 'my-product' });
|
|
2486
|
+
await ductape.events.produce({
|
|
2487
|
+
event: "user-events:user-action",
|
|
2488
|
+
message: { action: "button-click", screen: "dashboard" },
|
|
2489
|
+
session: "user-session:eyJ...", // REQUIRED for client-side produce
|
|
2345
2490
|
});
|
|
2346
|
-
Idempotent publish (deduplicates by key):
|
|
2347
|
-
await ductape.events.publishIdempotent({ product, env, event, message, idempotencyKey, idempotencyTtl? })
|
|
2348
2491
|
|
|
2349
|
-
|
|
2492
|
+
SCHEDULED DISPATCH (background job):
|
|
2493
|
+
ductape_execute("messageBrokers.dispatch", [{
|
|
2494
|
+
product, env,
|
|
2495
|
+
broker: "order-events", // broker tag
|
|
2496
|
+
event: "order-events:reminder-due", // "broker:topic"
|
|
2497
|
+
input: { message: { orderId: "123" } },
|
|
2498
|
+
retries?: 3,
|
|
2499
|
+
session?: "session-tag:jwt",
|
|
2500
|
+
schedule?: {
|
|
2501
|
+
start_at?: 1735689600000, // Unix ms or ISO string
|
|
2502
|
+
cron?: "0 9 * * *", // recurring cron
|
|
2503
|
+
every?: 86400000, // recurring interval ms
|
|
2504
|
+
limit?: 10, // max repetitions
|
|
2505
|
+
endDate?: "2026-12-31",
|
|
2506
|
+
tz?: "America/New_York",
|
|
2507
|
+
},
|
|
2508
|
+
}])
|
|
2509
|
+
Returns: { job_id, status: "scheduled"|"queued", scheduled_at, recurring, next_run_at? }
|
|
2510
|
+
|
|
2511
|
+
━━━ STEP 4: CONSUME — WRITTEN IN APPLICATION CODE ━━━
|
|
2512
|
+
|
|
2513
|
+
Consumers are auto-registered by the SDK on first consume call.
|
|
2514
|
+
Consumer registration options (all optional — used for tracking in Workbench):
|
|
2515
|
+
consumer?: { tag?: string, name?: string, description?: string }
|
|
2516
|
+
If tag is omitted, Ductape generates one: "consumer-<brokerTag>-<topicTag>".
|
|
2517
|
+
|
|
2518
|
+
ACK BEHAVIOR (automatic):
|
|
2519
|
+
- Callback returns successfully → message is acknowledged (ack)
|
|
2520
|
+
- Callback throws → message is tracked as failed; broker nacks/retries per provider behavior
|
|
2521
|
+
- After max retries → message moves to dead-letter queue (DLQ)
|
|
2522
|
+
There is no manual ack API. Acknowledgement is implicit from callback outcome.
|
|
2523
|
+
|
|
2524
|
+
CONSUMER GROUPS (Kafka-specific):
|
|
2525
|
+
Consumer groups are set in the broker's envs[].config.groupId (at broker registration time).
|
|
2526
|
+
All service instances sharing the same groupId form a consumer group and share partition load.
|
|
2527
|
+
To configure: set groupId in the kafka config when creating/updating the broker.
|
|
2528
|
+
|
|
2529
|
+
CONCURRENCY:
|
|
2530
|
+
Ductape has no per-consumer concurrency setting. Concurrency is determined by:
|
|
2531
|
+
- Number of running service instances (horizontal scale)
|
|
2532
|
+
- Broker-level partition count (Kafka) or visibility timeout (SQS)
|
|
2533
|
+
Run multiple instances of your service to scale consumption.
|
|
2534
|
+
|
|
2535
|
+
GENERAL BACKEND (TypeScript/Node.js — not NestJS):
|
|
2536
|
+
Start consuming in your module init or service startup:
|
|
2350
2537
|
await ductape.events.consume({
|
|
2351
2538
|
product: "my-product",
|
|
2352
2539
|
env: "prd",
|
|
2353
|
-
event: "
|
|
2354
|
-
callback: async (message) => {
|
|
2540
|
+
event: "order-events:order-created",
|
|
2541
|
+
callback: async (message) => {
|
|
2542
|
+
// All real processing logic goes here.
|
|
2543
|
+
// Throw to nack. Return to ack.
|
|
2544
|
+
await processOrder(message as { orderId: string; total: number });
|
|
2545
|
+
},
|
|
2546
|
+
consumer?: { tag: "order-processor", name: "Order Processor" },
|
|
2355
2547
|
});
|
|
2356
|
-
Callback errors are re-thrown so the broker can nack/retry.
|
|
2357
2548
|
|
|
2358
|
-
|
|
2359
|
-
|
|
2360
|
-
|
|
2549
|
+
NESTJS — use SDK in onModuleInit (no @Messaging.Consume decorator exists yet):
|
|
2550
|
+
@Injectable()
|
|
2551
|
+
export class OrderConsumerService implements OnModuleInit {
|
|
2552
|
+
constructor(private readonly ductape: Ductape) {}
|
|
2553
|
+
async onModuleInit() {
|
|
2554
|
+
await this.ductape.events.consume({
|
|
2555
|
+
product: "my-product",
|
|
2556
|
+
env: process.env.DUCTAPE_ENV || 'prd',
|
|
2557
|
+
event: "order-events:order-created",
|
|
2558
|
+
callback: async (message) => { await this.handle(message); },
|
|
2559
|
+
consumer: { tag: "order-consumer", name: "Order Consumer" },
|
|
2560
|
+
});
|
|
2561
|
+
}
|
|
2562
|
+
private async handle(message: unknown) { /* business logic */ }
|
|
2563
|
+
}
|
|
2361
2564
|
|
|
2362
|
-
|
|
2363
|
-
|
|
2364
|
-
|
|
2565
|
+
CLIENT-SIDE: Clients CANNOT consume. Event consumption is always server-side only.
|
|
2566
|
+
This is the key distinction between server topics (produce + consume) and client-observable
|
|
2567
|
+
topics (produce from client, consume on server). Never set up a consumer in browser code.
|
|
2365
2568
|
|
|
2366
|
-
|
|
2569
|
+
DEAD-LETTER QUEUE (DLQ):
|
|
2570
|
+
Messages whose callbacks consistently throw are automatically moved to the DLQ.
|
|
2571
|
+
Query: ductape_execute("messageBrokers.messages.getDeadLetters",
|
|
2572
|
+
[{ product, env, brokerTag, topicTag?, consumerTag?, limit? }])
|
|
2573
|
+
Reprocess: ductape_execute("messageBrokers.reprocessDLQ",
|
|
2574
|
+
[{ product, env, brokerTag, topicTag?, messageIds?, limit? }])
|
|
2575
|
+
Replay: ductape_execute("messageBrokers.replayEvent",
|
|
2576
|
+
[{ product, env, eventId, force? }])
|
|
2367
2577
|
|
|
2368
2578
|
━━━ OBSERVABILITY ━━━
|
|
2369
2579
|
|
|
2370
2580
|
messageBrokers.messages.query [{ product, env, brokerTag, topicTag?, status?, page?, limit? }]
|
|
2371
2581
|
messageBrokers.messages.getStats [{ product, env, brokerTag }]
|
|
2372
2582
|
messageBrokers.messages.getDashboard [{ product, env, brokerTag }]
|
|
2373
|
-
messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, limit? }]
|
|
2583
|
+
messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }]
|
|
2584
|
+
messageBrokers.messages.getProducers [{ product, env, brokerTag, topicTag?, page?, limit? }]
|
|
2585
|
+
messageBrokers.messages.getConsumers [{ product, env, brokerTag, topicTag?, page?, limit? }]
|
|
2374
2586
|
messageBrokers.replayEvent [{ product, env, eventId, force? }]
|
|
2375
2587
|
messageBrokers.reprocessDLQ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }]
|
|
2376
2588
|
messageBrokers.checkIdempotency [{ product, env, brokerTag, idempotency_key }]
|
package/package.json
CHANGED
package/src/index.ts
CHANGED
|
@@ -287,19 +287,24 @@ SPECIALIZED MODULES (for injecting handles directly without @InjectContext):
|
|
|
287
287
|
ALL params are passed as a JSON array in positional order matching the SDK signature.
|
|
288
288
|
|
|
289
289
|
━━━ MODULE: product ━━━
|
|
290
|
+
IMPORTANT: ALL product.* methods require the access key and will return 403 with a publishable key.
|
|
291
|
+
Use ductape_cli for ALL product operations — never ductape_execute:
|
|
292
|
+
ductape_cli("products get --tag <tag> --json") ← fetch product + full inventory
|
|
293
|
+
ductape_cli("products create --name <name> --tag <tag>")
|
|
294
|
+
ductape_cli("products environments list <tag> --json")
|
|
295
|
+
ductape_cli("products environments get <tag> <slug> --json")
|
|
296
|
+
ductape_cli("products apps list --product <id> --json")
|
|
297
|
+
|
|
298
|
+
SDK method signatures (for reference, admin key only):
|
|
290
299
|
product.create [data: { name, description, tag?, envs?: [{slug, name}] }]
|
|
291
300
|
product.fetch [product_tag: string]
|
|
292
301
|
product.update [product_tag: string, data: { name?: string, description?: string }]
|
|
293
|
-
product.
|
|
294
|
-
product.environments.create [product_tag, data: { slug: string, env_name: string, description: string, active?: boolean }]
|
|
295
|
-
product.environments.update [product_tag, slug: string, data: { env_name?: string, description?: string, active?: boolean }]
|
|
302
|
+
product.environments.create [product_tag, data: { slug, env_name, description, active? }]
|
|
296
303
|
product.environments.list [product_tag]
|
|
297
304
|
product.environments.fetch [product_tag, slug]
|
|
298
|
-
product.apps.
|
|
299
|
-
product.apps.add [product_tag, app: { access_tag: string, envs: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
|
|
305
|
+
product.apps.add [product_tag, app: { access_tag, envs: [{ app_env_slug, product_env_slug, variables?, auth? }] }]
|
|
300
306
|
product.apps.list [product_tag]
|
|
301
307
|
product.apps.fetch [product_tag, access_tag]
|
|
302
|
-
product.apps.update [product_tag, access_tag: string, data: { version?: string, envs?: [{ app_env_slug: string, product_env_slug: string, variables?: [{key: string, value: string}], auth?: { auth_tag: string, data: string|object, expiry?: number } }] }]
|
|
303
308
|
|
|
304
309
|
━━━ MODULE: app ━━━
|
|
305
310
|
app.create [data: { app_name: string, description: string, unique?: boolean }]
|
|
@@ -1788,6 +1793,31 @@ Important:
|
|
|
1788
1793
|
apps: `
|
|
1789
1794
|
DUCTAPE APPS
|
|
1790
1795
|
|
|
1796
|
+
WHAT AN APP IS:
|
|
1797
|
+
A Ductape App is a pre-configured, versioned API integration definition. It is NOT a generic HTTP
|
|
1798
|
+
client, NOT a job scheduler, and NOT anything you can call without registering first.
|
|
1799
|
+
|
|
1800
|
+
An App must be fully set up in Ductape before any code can use it:
|
|
1801
|
+
1. The App record must be created (name, tag, description)
|
|
1802
|
+
2. Environments must be added (each environment slug → base URL for that stage)
|
|
1803
|
+
3. Auth scheme must be configured (how outbound requests authenticate: apikey, bearer, OAuth2, etc.)
|
|
1804
|
+
4. Action endpoints must be defined (each action = one HTTP endpoint spec: method, path, body/query/header shape, response shape)
|
|
1805
|
+
5. The App must be connected to the product (product.apps.add) and its envs mapped
|
|
1806
|
+
|
|
1807
|
+
ONLY after all five steps can any code call:
|
|
1808
|
+
ctx.api.run({ app: '<app_tag>', event: '<action_tag>', input: { ... } }) ← in a feature handler
|
|
1809
|
+
actions.run([{ product, env, app: '<app_tag>', action: '<action_tag>', input }]) ← at runtime
|
|
1810
|
+
|
|
1811
|
+
ctx.api (alias for ctx.action) is NOT a generic HTTP call. It ONLY invokes a pre-registered
|
|
1812
|
+
Ductape App Action. If the App or action tag does not exist in the product, the call will fail.
|
|
1813
|
+
actions.dispatch is the same as actions.run but scheduled as a background job — it still requires
|
|
1814
|
+
a registered App. There is no way to dispatch a job to an arbitrary URL via ctx.api or actions.dispatch.
|
|
1815
|
+
|
|
1816
|
+
If a feature step needs to call an external service and no App is registered for it yet:
|
|
1817
|
+
→ Flag it as "App to create" in your plan (STEP 4 of the feature design workflow)
|
|
1818
|
+
→ Create the App and all its actions first (see below)
|
|
1819
|
+
→ Only then write the ctx.api.run call
|
|
1820
|
+
|
|
1791
1821
|
An app is a versioned API integration definition. It contains environments (base URLs), actions
|
|
1792
1822
|
(individual endpoint specs), auth schemes, webhooks, variables, and constants.
|
|
1793
1823
|
|
|
@@ -1803,6 +1833,21 @@ Manage environments (base URLs per stage):
|
|
|
1803
1833
|
ductape_execute("app.environments.create", [app_tag, { slug, env_name, base_url }])
|
|
1804
1834
|
ductape_execute("app.environments.list", [app_tag])
|
|
1805
1835
|
|
|
1836
|
+
Discover apps in a product and their actions (ALWAYS do this before writing any ctx.api.run call):
|
|
1837
|
+
Step 1 — list apps connected to the product:
|
|
1838
|
+
ductape_cli("products get --tag <product_tag> --json") → full product document; check apps[]
|
|
1839
|
+
ductape_cli("products apps list --product <product_id> --json") → apps[] with access_tag, envs
|
|
1840
|
+
Step 2 — list actions in an app:
|
|
1841
|
+
ductape_execute("actions.list", [app_tag]) → returns all action tags + names
|
|
1842
|
+
Step 3 — fetch the input schema for an action:
|
|
1843
|
+
ductape_execute("actions.fetch", [app_tag, action_tag])
|
|
1844
|
+
→ returns { body: {fieldName: {type, required}}, params: {}, query: {}, headers: {} }
|
|
1845
|
+
OR: call ductape_generate_payload (operation_family="action", method="run",
|
|
1846
|
+
targets={app: "app_tag", action: "action_tag"}) to get the exact resolved payload shape
|
|
1847
|
+
Step 4 — call ductape_schema({ module: "app" }) if you need the JSON schema for creating/updating
|
|
1848
|
+
app resources (not for runtime input — use actions.fetch or ductape_generate_payload for that)
|
|
1849
|
+
NEVER assume action input field names. Always fetch the action definition first.
|
|
1850
|
+
|
|
1806
1851
|
Manage actions (individual API endpoints):
|
|
1807
1852
|
ductape_execute("actions.create", [app_tag, {
|
|
1808
1853
|
tag, name, resource: "/users/{id}", method: "GET"|"POST"|"PUT"|"PATCH"|"DELETE",
|
|
@@ -1816,6 +1861,16 @@ Manage actions (individual API endpoints):
|
|
|
1816
1861
|
ductape_execute("actions.list", [app_tag])
|
|
1817
1862
|
ductape_execute("actions.fetch", [app_tag, action_tag])
|
|
1818
1863
|
|
|
1864
|
+
Action input — flat input format:
|
|
1865
|
+
Fields are resolved to the correct location (body/params/query/headers) by matching the action schema.
|
|
1866
|
+
For ambiguous keys, use explicit prefixes:
|
|
1867
|
+
input: { amount: 1000 } → auto-resolved (body.amount if body field exists)
|
|
1868
|
+
input: { "body:amount": 1000 } → explicit body
|
|
1869
|
+
input: { "params:id": "user_123" } → route parameter
|
|
1870
|
+
input: { "query:limit": 10 } → query string
|
|
1871
|
+
input: { "headers:X-Idempotency-Key": "..." } → request header
|
|
1872
|
+
Always use ductape_generate_payload or actions.fetch to know the exact field names — never guess.
|
|
1873
|
+
|
|
1819
1874
|
Run an action at runtime:
|
|
1820
1875
|
→ CALL ductape_generate_payload FIRST (operation_family="action", method="run", targets={app, action})
|
|
1821
1876
|
ductape_execute("actions.run", [{ product, env, app, action, input: { "body:field": value } }])
|
|
@@ -1835,14 +1890,21 @@ Variables (per-env mutable values) and Constants (fixed values):
|
|
|
1835
1890
|
ductape_execute("app.constants.create", [app_tag, { key, value }])
|
|
1836
1891
|
|
|
1837
1892
|
Connecting an app to a product (after creation):
|
|
1838
|
-
|
|
1839
|
-
|
|
1840
|
-
|
|
1841
|
-
|
|
1842
|
-
|
|
1843
|
-
}])
|
|
1844
|
-
ductape_execute("
|
|
1845
|
-
ductape_execute("
|
|
1893
|
+
NOTE: All product.* module methods require the access key and CANNOT use ductape_execute.
|
|
1894
|
+
Use ductape_cli for all product-level operations.
|
|
1895
|
+
|
|
1896
|
+
FULL FLOW to make an app callable from a product:
|
|
1897
|
+
1. Create the app: ductape_cli("apps create --name \\"Service Name\\" --description \\"...\\"")
|
|
1898
|
+
2. Add environments: ductape_execute("app.environments.create", [app_tag, { slug: "prd", env_name: "Production", base_url: "https://api.example.com" }])
|
|
1899
|
+
3. Configure auth: ductape_execute("auths.create", [app_tag, { tag, name, setup_type: "apikey"|"bearer"|"basic"|"oauth2", expiry, period }])
|
|
1900
|
+
4. Define actions: ductape_execute("actions.create", [app_tag, { tag, name, resource, method, body?, params?, query?, headers?, response? }])
|
|
1901
|
+
OR import: ductape_cli("apps import <file.json> -t postman|openapi")
|
|
1902
|
+
5. Connect to product: Requires the product_id (from ductape_cli("products get --tag <tag> --json")).
|
|
1903
|
+
There is no CLI command for this step — the SDK product.apps.add method requires
|
|
1904
|
+
an access key which only the backend can provide. Use ductape_execute via an
|
|
1905
|
+
admin-authenticated context, or connect via the Workbench UI.
|
|
1906
|
+
6. Verify: ductape_cli("products get --tag <product_tag> --json") → check apps[] contains the app
|
|
1907
|
+
ductape_execute("actions.list", [app_tag]) → verify actions are registered
|
|
1846
1908
|
`.trim(),
|
|
1847
1909
|
|
|
1848
1910
|
products: `
|
|
@@ -1852,20 +1914,20 @@ A product is the top-level namespace for all Ductape infrastructure: apps, datab
|
|
|
1852
1914
|
vectors, storage, brokers, sessions, caches, notifications, resilience, features, jobs, and envs.
|
|
1853
1915
|
Every SDK service call resolves within a product context.
|
|
1854
1916
|
|
|
1917
|
+
IMPORTANT: The product module requires the access key. ALL product operations must use ductape_cli,
|
|
1918
|
+
not ductape_execute (which only accepts the publishable key and will return 403 for product.*).
|
|
1919
|
+
|
|
1855
1920
|
Create a product:
|
|
1856
1921
|
ductape_cli("products create --name \\"My App\\" --tag my-app")
|
|
1857
|
-
ductape_execute("product.create", [{ name: "My App", tag: "my-app",
|
|
1858
|
-
envs: [{ slug: "dev", name: "Development" }, { slug: "prd", name: "Production" }] }])
|
|
1859
1922
|
|
|
1860
1923
|
Environments — every resource's envs array MUST cover all product env slugs:
|
|
1861
|
-
|
|
1862
|
-
|
|
1863
|
-
|
|
1864
|
-
BEFORE registering any resource, always run environments.list and collect all slugs.
|
|
1924
|
+
ductape_cli("products environments list <product_tag> --json")
|
|
1925
|
+
ductape_cli("products environments get <product_tag> <slug> --json")
|
|
1926
|
+
BEFORE registering any resource, always run environments list and collect all slugs.
|
|
1865
1927
|
|
|
1866
1928
|
Fetch / update:
|
|
1867
|
-
|
|
1868
|
-
|
|
1929
|
+
ductape_cli("products get --tag <product_tag> --json")
|
|
1930
|
+
ductape_cli("products get --id <product_id> --json")
|
|
1869
1931
|
|
|
1870
1932
|
Connect apps to a product:
|
|
1871
1933
|
See ductape_docs({ topic: "apps" }) for product.apps.add / product.apps.list.
|
|
@@ -2153,21 +2215,32 @@ STEP 1 — UNDERSTAND the goal
|
|
|
2153
2215
|
understand: what the feature does, what it returns, what can fail and how failures should behave.
|
|
2154
2216
|
|
|
2155
2217
|
STEP 2 — INVENTORY existing Ductape components
|
|
2156
|
-
Call
|
|
2218
|
+
Call ductape_cli("products get --tag <product_tag> --json") to read the full product document.
|
|
2219
|
+
(product.* requires the access key — never use ductape_execute for product reads, it will return 403)
|
|
2157
2220
|
Note what already exists:
|
|
2158
2221
|
- databases[] → available for ctx.database.insert/query/update/delete steps
|
|
2159
|
-
- apps[] → available for ctx.api.run steps
|
|
2222
|
+
- apps[] → available for ctx.api.run steps ONLY if the App is fully registered:
|
|
2223
|
+
(a) App record exists, (b) environments defined with base URLs,
|
|
2224
|
+
(c) auth scheme configured, (d) action endpoints defined,
|
|
2225
|
+
(e) connected to the product via product.apps.add.
|
|
2226
|
+
Check app.events[] for action tags. If no app exists for a service the
|
|
2227
|
+
feature needs to call, flag it as "App to create" in the plan — do NOT
|
|
2228
|
+
assume ctx.api can call any URL or schedule any job without a registered App.
|
|
2160
2229
|
- notifications[] → available for ctx.notification.email/sms/push steps
|
|
2161
2230
|
- storage[] → available for ctx.storage.upload/download steps
|
|
2162
|
-
- messageBrokers[] → available for ctx.
|
|
2231
|
+
- messageBrokers[] → available for ctx.events.produce steps
|
|
2163
2232
|
- graphs[] → available for ctx.graph steps
|
|
2164
|
-
- features[] → can be called as child features via ctx.feature()
|
|
2233
|
+
- features[] → can be called as child features via ctx.feature.execute()
|
|
2165
2234
|
- caches[], sessions[]
|
|
2166
2235
|
Do NOT assume a component or event tag exists — verify from the product before using it.
|
|
2236
|
+
Do NOT treat ctx.api as a generic HTTP call or job scheduler. It requires a registered App.
|
|
2167
2237
|
|
|
2168
2238
|
STEP 3 — PLAN each step
|
|
2169
2239
|
For every logical step:
|
|
2170
|
-
a. Identify which existing component handles it, or flag it as needing creation
|
|
2240
|
+
a. Identify which existing component handles it, or flag it as needing creation.
|
|
2241
|
+
If a step calls an external service, it MUST go through a registered Ductape App.
|
|
2242
|
+
If no App for that service exists in the product → mark it "App to create: <service name>".
|
|
2243
|
+
DO NOT plan a raw HTTP call, a direct dispatch to a URL, or any workaround in place of a missing App.
|
|
2171
2244
|
b. Note how inputs flow: ctx.input fields, or return values from earlier steps (plain JS variables — no special notation needed)
|
|
2172
2245
|
c. Decide if a rollback handler is needed (e.g. charge → refund on later failure)
|
|
2173
2246
|
d. Decide allow_fail: true for non-critical steps (email, analytics, audit logs)
|
|
@@ -2211,7 +2284,8 @@ STEP 7 — HANDLE conditionals, loops, and branching (when applicable)
|
|
|
2211
2284
|
→ Only the scenario whose input matches runs at execution time
|
|
2212
2285
|
|
|
2213
2286
|
STEP 8 — SET rollbacks for reversible steps
|
|
2214
|
-
Any step that allocates a resource should undo it if a later step fails
|
|
2287
|
+
Any step that allocates a resource should undo it if a later step fails.
|
|
2288
|
+
ctx.api.run requires a pre-registered Ductape App — 'stripe' below is the tag of a registered App:
|
|
2215
2289
|
const charge = await ctx.step(
|
|
2216
2290
|
'charge',
|
|
2217
2291
|
async () => ctx.api.run({ app: 'stripe', event: 'create-charge', input: { amount: ctx.input.amount } }),
|
|
@@ -2271,6 +2345,52 @@ Signals and queries (for long-running features):
|
|
|
2271
2345
|
|
|
2272
2346
|
Rollback strategies: reverse_all | reverse_critical | compensate | none
|
|
2273
2347
|
Feature statuses: pending | running | completed | failed | rolled_back | rolling_back | paused
|
|
2348
|
+
|
|
2349
|
+
━━━ FEATURE RECORDING SEMANTICS ━━━
|
|
2350
|
+
|
|
2351
|
+
When you call features.define({ handler }), the handler runs TWICE:
|
|
2352
|
+
|
|
2353
|
+
1. RECORDING PHASE (at define time) — handler is called with a RecordingContext.
|
|
2354
|
+
All ctx.step() calls return lightweight proxy objects, not real data.
|
|
2355
|
+
This phase captures the step graph: which steps exist, their types, tags, and declared
|
|
2356
|
+
inputs/outputs. No real API calls, DB queries, or side effects occur.
|
|
2357
|
+
Arbitrary JS code OUTSIDE ctx.step() ALSO runs during recording — with proxy values.
|
|
2358
|
+
For loops: use recordInput so the handler sees sample data and all iterations are recorded.
|
|
2359
|
+
For branches: use branchOverrides so each path is captured.
|
|
2360
|
+
|
|
2361
|
+
2. EXECUTION PHASE (at runtime) — handler is called with a real ExecutionContext.
|
|
2362
|
+
ctx.step() actually executes. All real Ductape component calls happen.
|
|
2363
|
+
Arbitrary JS logic (math, string ops, conditionals on step results) runs for real.
|
|
2364
|
+
|
|
2365
|
+
Implication: put all meaningful business logic INSIDE ctx.step() handlers, not in the
|
|
2366
|
+
outer handler body. Code in the outer body runs during recording with proxy values and
|
|
2367
|
+
may behave unexpectedly (e.g. typeof proxy === 'object' is true but .someField is a proxy).
|
|
2368
|
+
|
|
2369
|
+
Features do NOT execute arbitrary NestJS or server code directly. A feature handler can only
|
|
2370
|
+
call Ductape component primitives (ctx.api, ctx.database, ctx.notification, etc.) as steps.
|
|
2371
|
+
To invoke internal application business logic, produce a broker event from a feature step
|
|
2372
|
+
(ctx.messaging.produce) and consume it in your NestJS service — that is the correct pattern.
|
|
2373
|
+
|
|
2374
|
+
━━━ ORCHESTRATION DECISION RULE ━━━
|
|
2375
|
+
|
|
2376
|
+
One component operation at a future time:
|
|
2377
|
+
→ use that component's own dispatch method
|
|
2378
|
+
e.g. ductape.events.dispatch({ ..., schedule: { start_at: ... } })
|
|
2379
|
+
e.g. ductape.api.dispatch({ ..., schedule: { start_at: ... } })
|
|
2380
|
+
e.g. ductape.database.dispatch({ ..., schedule: { start_at: ... } })
|
|
2381
|
+
|
|
2382
|
+
Several durable Ductape component operations in sequence (with rollback / retry / state):
|
|
2383
|
+
→ define a Feature, then features.dispatch to schedule it
|
|
2384
|
+
|
|
2385
|
+
Invoke internal application business logic (your own NestJS/backend service code):
|
|
2386
|
+
→ produce a broker event (ctx.messaging.produce or ductape.events.produce)
|
|
2387
|
+
→ consume it in your NestJS service with events.consume in onModuleInit
|
|
2388
|
+
→ your service method runs with full access to DI, DB transactions, etc.
|
|
2389
|
+
Do NOT create an App Action just to call your own service over HTTP.
|
|
2390
|
+
|
|
2391
|
+
Invoke an external/public HTTP service:
|
|
2392
|
+
→ create a Ductape App (register base URL, auth, action endpoints) then use ctx.api.run
|
|
2393
|
+
→ requires the App to be fully registered and connected to the product first
|
|
2274
2394
|
`.trim(),
|
|
2275
2395
|
|
|
2276
2396
|
events: `
|
|
@@ -2369,81 +2489,173 @@ Import (register an EXISTING cloud resource):
|
|
|
2369
2489
|
redis: { host: "...", port: 6379, password?: "..." }
|
|
2370
2490
|
nats: { servers: ["nats://host:4222"], token?: "...", user?: "...", pass?: "...", tls?: true }
|
|
2371
2491
|
|
|
2372
|
-
━━━ STEP 2:
|
|
2373
|
-
|
|
2374
|
-
For most providers (GCP Pub/Sub, Kafka, RabbitMQ, Redis, NATS, Azure Service Bus):
|
|
2375
|
-
SKIP this step. Topics are auto-created when you create a producer (step 3).
|
|
2376
|
-
You do NOT need to call messageBrokers.topics.create before creating a producer.
|
|
2492
|
+
━━━ STEP 2: DEFINE TOPICS ━━━
|
|
2377
2493
|
|
|
2378
|
-
|
|
2379
|
-
|
|
2380
|
-
|
|
2494
|
+
Topics MUST be defined before any consumer can subscribe to them.
|
|
2495
|
+
Producing to a topic also calls ensureTopicRegistered in the background — but DO NOT rely on
|
|
2496
|
+
auto-registration for consume paths. Always create topics explicitly.
|
|
2381
2497
|
|
|
2382
|
-
If you do need it (SQS or explicit config):
|
|
2383
2498
|
ductape_execute("messageBrokers.topics.create", [product_tag, {
|
|
2384
|
-
tag: "
|
|
2385
|
-
name: "
|
|
2386
|
-
broker: "
|
|
2499
|
+
tag: "order-created", // topic tag (just the topic part, NOT "broker:topic")
|
|
2500
|
+
name: "Order Created",
|
|
2501
|
+
broker: "order-events", // broker component tag
|
|
2387
2502
|
description?: string,
|
|
2388
|
-
sample: {
|
|
2389
|
-
idempotent?: boolean,
|
|
2390
|
-
// SQS only —
|
|
2503
|
+
sample: { orderId: "string", total: 0 }, // documents expected message shape
|
|
2504
|
+
idempotent?: boolean, // if true, Ductape deduplicates by idempotency_key
|
|
2505
|
+
// AWS SQS only — must supply per-env queue URL:
|
|
2391
2506
|
queueUrls?: [
|
|
2392
|
-
{ env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/
|
|
2393
|
-
{ env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/
|
|
2507
|
+
{ env_slug: "snd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-snd" },
|
|
2508
|
+
{ env_slug: "prd", url: "https://sqs.us-east-1.amazonaws.com/123/queue-prd" }
|
|
2394
2509
|
]
|
|
2395
2510
|
}])
|
|
2396
2511
|
|
|
2397
|
-
List
|
|
2512
|
+
List / fetch topics:
|
|
2398
2513
|
ductape_execute("messageBrokers.topics.list", [product_tag, "broker-tag"])
|
|
2514
|
+
ductape_execute("messageBrokers.fetch", [product_tag, "broker-tag"]) → includes topics[]
|
|
2399
2515
|
|
|
2400
|
-
━━━ STEP 3: PRODUCE
|
|
2516
|
+
━━━ STEP 3: PRODUCE — WRITTEN IN APPLICATION CODE ━━━
|
|
2401
2517
|
|
|
2402
|
-
There is NO admin command
|
|
2403
|
-
|
|
2404
|
-
|
|
2405
|
-
|
|
2518
|
+
There is NO admin command to declare a producer. Producers are auto-registered by the SDK on
|
|
2519
|
+
the first produce call — you do not pre-declare them.
|
|
2520
|
+
Do NOT call ductape_generate_payload for messaging. The producer owns the schema.
|
|
2521
|
+
Infer the message shape from context, present it to the user for approval, then implement.
|
|
2406
2522
|
|
|
2407
|
-
|
|
2408
|
-
|
|
2409
|
-
|
|
2410
|
-
Do NOT call ductape_generate_payload for messaging. Events have no pre-existing backend schema
|
|
2411
|
-
to discover — the producer defines the schema. Instead, infer the message shape from context
|
|
2412
|
-
(event name, existing data models, user input), present it to the user for approval, then implement.
|
|
2523
|
+
GENERAL BACKEND (TypeScript/Node.js — not NestJS):
|
|
2524
|
+
import Ductape from '@ductape/sdk';
|
|
2525
|
+
const ductape = new Ductape({ accessKey: 'your-access-key' });
|
|
2413
2526
|
await ductape.events.produce({
|
|
2414
2527
|
product: "my-product",
|
|
2415
2528
|
env: "prd",
|
|
2416
|
-
event: "
|
|
2417
|
-
message: { key: value },
|
|
2529
|
+
event: "broker-tag:topic-tag", // always colon-separated
|
|
2530
|
+
message: { key: value },
|
|
2531
|
+
session?: "session-tag:jwt", // optional — traces message to a user session
|
|
2532
|
+
});
|
|
2533
|
+
// Idempotent publish (deduplicates — prevents double-processing on retries):
|
|
2534
|
+
await ductape.events.publishIdempotent({
|
|
2535
|
+
product, env, event, message,
|
|
2536
|
+
idempotencyKey: "order-123-charge", // stable key unique to this logical operation
|
|
2537
|
+
idempotencyTtl?: 86400, // seconds; default 86400 (24h)
|
|
2418
2538
|
});
|
|
2419
|
-
Idempotent publish (deduplicates by key):
|
|
2420
|
-
await ductape.events.publishIdempotent({ product, env, event, message, idempotencyKey, idempotencyTtl? })
|
|
2421
2539
|
|
|
2422
|
-
|
|
2540
|
+
NESTJS — method decorator:
|
|
2541
|
+
import { Messaging } from '@ductape/nestjs';
|
|
2542
|
+
@Injectable() export class OrdersService {
|
|
2543
|
+
@Messaging.Produce({ event: 'order-events:order-created' })
|
|
2544
|
+
emitOrderCreated(payload: { orderId: string; total: number }) { return payload; }
|
|
2545
|
+
|
|
2546
|
+
// Scheduled dispatch — fire-and-forget with optional schedule:
|
|
2547
|
+
@Messaging.Dispatch({ broker: 'order-events', event: 'order-events:order-created',
|
|
2548
|
+
schedule?: { start_at?, cron?, every?, limit?, tz? } })
|
|
2549
|
+
scheduleOrderNotification(payload: Record<string, unknown>) { return payload; }
|
|
2550
|
+
}
|
|
2551
|
+
|
|
2552
|
+
CLIENT-SIDE (browser — publishable key):
|
|
2553
|
+
Clients CAN produce messages using a publishable key + session token.
|
|
2554
|
+
Only produce to topics whose schema is safe for client authorship.
|
|
2555
|
+
NEVER allow clients to produce to topics that trigger privileged server-side operations
|
|
2556
|
+
(payments, admin actions, state mutations) — those must go through a backend endpoint first.
|
|
2557
|
+
import Ductape from '@ductape/sdk';
|
|
2558
|
+
const ductape = new Ductape({ publishableKey: 'pk_...', env: 'prd', product: 'my-product' });
|
|
2559
|
+
await ductape.events.produce({
|
|
2560
|
+
event: "user-events:user-action",
|
|
2561
|
+
message: { action: "button-click", screen: "dashboard" },
|
|
2562
|
+
session: "user-session:eyJ...", // REQUIRED for client-side produce
|
|
2563
|
+
});
|
|
2564
|
+
|
|
2565
|
+
SCHEDULED DISPATCH (background job):
|
|
2566
|
+
ductape_execute("messageBrokers.dispatch", [{
|
|
2567
|
+
product, env,
|
|
2568
|
+
broker: "order-events", // broker tag
|
|
2569
|
+
event: "order-events:reminder-due", // "broker:topic"
|
|
2570
|
+
input: { message: { orderId: "123" } },
|
|
2571
|
+
retries?: 3,
|
|
2572
|
+
session?: "session-tag:jwt",
|
|
2573
|
+
schedule?: {
|
|
2574
|
+
start_at?: 1735689600000, // Unix ms or ISO string
|
|
2575
|
+
cron?: "0 9 * * *", // recurring cron
|
|
2576
|
+
every?: 86400000, // recurring interval ms
|
|
2577
|
+
limit?: 10, // max repetitions
|
|
2578
|
+
endDate?: "2026-12-31",
|
|
2579
|
+
tz?: "America/New_York",
|
|
2580
|
+
},
|
|
2581
|
+
}])
|
|
2582
|
+
Returns: { job_id, status: "scheduled"|"queued", scheduled_at, recurring, next_run_at? }
|
|
2583
|
+
|
|
2584
|
+
━━━ STEP 4: CONSUME — WRITTEN IN APPLICATION CODE ━━━
|
|
2585
|
+
|
|
2586
|
+
Consumers are auto-registered by the SDK on first consume call.
|
|
2587
|
+
Consumer registration options (all optional — used for tracking in Workbench):
|
|
2588
|
+
consumer?: { tag?: string, name?: string, description?: string }
|
|
2589
|
+
If tag is omitted, Ductape generates one: "consumer-<brokerTag>-<topicTag>".
|
|
2590
|
+
|
|
2591
|
+
ACK BEHAVIOR (automatic):
|
|
2592
|
+
- Callback returns successfully → message is acknowledged (ack)
|
|
2593
|
+
- Callback throws → message is tracked as failed; broker nacks/retries per provider behavior
|
|
2594
|
+
- After max retries → message moves to dead-letter queue (DLQ)
|
|
2595
|
+
There is no manual ack API. Acknowledgement is implicit from callback outcome.
|
|
2596
|
+
|
|
2597
|
+
CONSUMER GROUPS (Kafka-specific):
|
|
2598
|
+
Consumer groups are set in the broker's envs[].config.groupId (at broker registration time).
|
|
2599
|
+
All service instances sharing the same groupId form a consumer group and share partition load.
|
|
2600
|
+
To configure: set groupId in the kafka config when creating/updating the broker.
|
|
2601
|
+
|
|
2602
|
+
CONCURRENCY:
|
|
2603
|
+
Ductape has no per-consumer concurrency setting. Concurrency is determined by:
|
|
2604
|
+
- Number of running service instances (horizontal scale)
|
|
2605
|
+
- Broker-level partition count (Kafka) or visibility timeout (SQS)
|
|
2606
|
+
Run multiple instances of your service to scale consumption.
|
|
2607
|
+
|
|
2608
|
+
GENERAL BACKEND (TypeScript/Node.js — not NestJS):
|
|
2609
|
+
Start consuming in your module init or service startup:
|
|
2423
2610
|
await ductape.events.consume({
|
|
2424
2611
|
product: "my-product",
|
|
2425
2612
|
env: "prd",
|
|
2426
|
-
event: "
|
|
2427
|
-
callback: async (message) => {
|
|
2613
|
+
event: "order-events:order-created",
|
|
2614
|
+
callback: async (message) => {
|
|
2615
|
+
// All real processing logic goes here.
|
|
2616
|
+
// Throw to nack. Return to ack.
|
|
2617
|
+
await processOrder(message as { orderId: string; total: number });
|
|
2618
|
+
},
|
|
2619
|
+
consumer?: { tag: "order-processor", name: "Order Processor" },
|
|
2428
2620
|
});
|
|
2429
|
-
Callback errors are re-thrown so the broker can nack/retry.
|
|
2430
2621
|
|
|
2431
|
-
|
|
2432
|
-
|
|
2433
|
-
|
|
2622
|
+
NESTJS — use SDK in onModuleInit (no @Messaging.Consume decorator exists yet):
|
|
2623
|
+
@Injectable()
|
|
2624
|
+
export class OrderConsumerService implements OnModuleInit {
|
|
2625
|
+
constructor(private readonly ductape: Ductape) {}
|
|
2626
|
+
async onModuleInit() {
|
|
2627
|
+
await this.ductape.events.consume({
|
|
2628
|
+
product: "my-product",
|
|
2629
|
+
env: process.env.DUCTAPE_ENV || 'prd',
|
|
2630
|
+
event: "order-events:order-created",
|
|
2631
|
+
callback: async (message) => { await this.handle(message); },
|
|
2632
|
+
consumer: { tag: "order-consumer", name: "Order Consumer" },
|
|
2633
|
+
});
|
|
2634
|
+
}
|
|
2635
|
+
private async handle(message: unknown) { /* business logic */ }
|
|
2636
|
+
}
|
|
2434
2637
|
|
|
2435
|
-
|
|
2436
|
-
|
|
2437
|
-
|
|
2638
|
+
CLIENT-SIDE: Clients CANNOT consume. Event consumption is always server-side only.
|
|
2639
|
+
This is the key distinction between server topics (produce + consume) and client-observable
|
|
2640
|
+
topics (produce from client, consume on server). Never set up a consumer in browser code.
|
|
2438
2641
|
|
|
2439
|
-
|
|
2642
|
+
DEAD-LETTER QUEUE (DLQ):
|
|
2643
|
+
Messages whose callbacks consistently throw are automatically moved to the DLQ.
|
|
2644
|
+
Query: ductape_execute("messageBrokers.messages.getDeadLetters",
|
|
2645
|
+
[{ product, env, brokerTag, topicTag?, consumerTag?, limit? }])
|
|
2646
|
+
Reprocess: ductape_execute("messageBrokers.reprocessDLQ",
|
|
2647
|
+
[{ product, env, brokerTag, topicTag?, messageIds?, limit? }])
|
|
2648
|
+
Replay: ductape_execute("messageBrokers.replayEvent",
|
|
2649
|
+
[{ product, env, eventId, force? }])
|
|
2440
2650
|
|
|
2441
2651
|
━━━ OBSERVABILITY ━━━
|
|
2442
2652
|
|
|
2443
2653
|
messageBrokers.messages.query [{ product, env, brokerTag, topicTag?, status?, page?, limit? }]
|
|
2444
2654
|
messageBrokers.messages.getStats [{ product, env, brokerTag }]
|
|
2445
2655
|
messageBrokers.messages.getDashboard [{ product, env, brokerTag }]
|
|
2446
|
-
messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, limit? }]
|
|
2656
|
+
messageBrokers.messages.getDeadLetters [{ product, env, brokerTag, topicTag?, consumerTag?, limit? }]
|
|
2657
|
+
messageBrokers.messages.getProducers [{ product, env, brokerTag, topicTag?, page?, limit? }]
|
|
2658
|
+
messageBrokers.messages.getConsumers [{ product, env, brokerTag, topicTag?, page?, limit? }]
|
|
2447
2659
|
messageBrokers.replayEvent [{ product, env, eventId, force? }]
|
|
2448
2660
|
messageBrokers.reprocessDLQ [{ product, env, brokerTag, topicTag?, messageIds?, limit? }]
|
|
2449
2661
|
messageBrokers.checkIdempotency [{ product, env, brokerTag, idempotency_key }]
|