dirigent-examples 0.15.0__py3-none-any.whl

dirigent_examples/shelves/queues/README.md

@@ -0,0 +1,59 @@
+# Queue examples
+
+A queue sensor starts a run from a message instead of a clock or a webhook. Each poke reads
+what has arrived since the last one and succeeds the moment there is enough, so the batch
+becomes ordinary step output and the rest of the DAG never learns where it came from. A publish
+goes the other way: a step puts records on a topic and succeeds once the broker has them.
+[docs/queues.md](../../docs/queues.md) is the family's home.
+
+Every block here is **ordinary**: each reaches only the brokers its connection names, so no id
+has to be allowlisted to run one.
+
+## Bring the brokers up first
+
+No document here can run without a broker. `infra/compose.queues.yaml` is a
+single-node Redpanda (which speaks the Kafka protocol) and a RabbitMQ, both unprotected,
+for a developer's machine:
+
+```bash
+make queues-up      # docker compose -f infra/compose.queues.yaml up -d --wait
+```
+
+Then create the topic or the queue, put something on it, and create the connection each
+document names. Each document's header spells out its own three commands, and the produce
+document needs nothing put on the topic first: it publishes what it reads back.
+
+Every document **names** its connection under `requires.connections` rather than carrying one,
+which is the shape to copy: bootstrap servers, an AMQP URL and a sealed password belong to an
+environment, not to a pipeline. On an instance they are created once:
+
+```bash
+dg connection create kafka orders-topic --set bootstrap_servers='["127.0.0.1:9092"]'
+dg connection create rabbitmq shop-queue \
+  --set url=amqp://dirigent@127.0.0.1:5672/ --set password=dirigent
+```
+
+On the compose stack the brokers are the `infra/compose.brokers.yaml` overlay instead
+(`make docker-run-queues`), and the same two connections name `redpanda:9092` and
+`amqp://dirigent@rabbitmq:5672/`, which is what a container resolves.
+
+## Pipelines
+
+| File | What it teaches |
+| --- | --- |
+| [kafka-consume-then-transform.yaml](kafka-consume-then-transform.yaml) | Waiting on a topic: `min_messages` and `max_messages`, why the cursor rather than a consumer group holds the offsets, and the batch read by a downstream transform. |
+| [kafka-produce-then-consume.yaml](kafka-produce-then-consume.yaml) | Publishing: the two shapes an element of `records` may take, where a message key comes from, what `acks: all` and an idempotent producer do and do not promise, and why this one consumes from `earliest`. |
+| [report-to-kafka.yaml](report-to-kafka.yaml) | Publishing a rendered page: `report.render` hands its text on, and `kafka.produce` puts it on the topic as one keyed record. |
+| [rabbitmq-consume-ack-on-success.yaml](rabbitmq-consume-ack-on-success.yaml) | When a message is acknowledged: why `ack: on_success` is the default, what a poke that parks does with the messages it took, and what `always` gives up in exchange. |
+| [report-to-rabbitmq.yaml](report-to-rabbitmq.yaml) | Publishing one message: the default exchange, where a routing key is a queue name, and what `persistent` buys. |
+
+## The cursor, in one paragraph
+
+A sensor's poke keeps its place in a **cursor**, a JSON map the poke returns with a `NotYet`
+and the next poke receives as `ctx.cursor`. `kafka.consume` keeps the offsets it has read to in
+it; `rabbitmq.consume` keeps only a count and the last delivery tag, because the broker holds
+the state there. It replaces rather than merges, it is stored by the transaction that parks the
+attempt -- which makes advancing it at-least-once, so a poke may read the same ground twice --
+and it lives only as long as the waiting attempt, because a poke that succeeds ends the step.
+Anything a later step needs is in the output, which is why `kafka.consume` reports the offsets
+it ended at there.

dirigent_examples/shelves/queues/kafka-consume-then-transform.yaml

@@ -0,0 +1,105 @@
+# A run that starts from what is on a Kafka topic, and reshapes the batch it took.
+#
+# NEEDS A BROKER, and a topic to read:
+#   docker compose -f infra/compose.queues.yaml up -d --wait
+#   docker exec -it dirigent-queues-redpanda-1 rpk topic create orders
+#   dg connection create kafka orders-topic --set bootstrap_servers='["127.0.0.1:9092"]'
+#   dg run examples/queues/kafka-consume-then-transform.yaml
+# Then produce to the topic in another shell, whenever -- start: latest means the run takes
+# what arrives after it began rather than what was already there, and the first poke writes
+# that place down, so a message landing between two pokes is read by the next one:
+#   docker exec -it dirigent-queues-redpanda-1 \
+#     rpk topic produce orders --format '%v\n'   # then type a JSON line and press ctrl-d
+# On the compose stack the broker is the infra/compose.brokers.yaml overlay and the connection
+# names it as `redpanda:9092`; docs/queues.md is the family's home.
+#
+# Two hops, and what each one hands on:
+#   wait       kafka.consume, parked until at least one message is on the topic. Output:
+#              messages, count, and the offsets it ended at.
+#   summarise  a jq transform over that batch, which is the only thing this example does with
+#              the messages -- the point is that they are ordinary step output from here on.
+#
+# THE CONNECTION IS NAMED, NOT CARRIED. requires.connections says the instance must already
+# hold `orders-topic`, and refuses the document at apply if it does not. That is the shape to
+# copy: bootstrap servers, a security setting and a sealed password belong to an environment,
+# not to a pipeline, so pointing this at staging instead of production is an edit to one
+# connection and to no document.
+#
+# WHY min_messages: 1. A poke succeeds as soon as there is one message, which is what "a run
+# per batch, as soon as there is a batch" means. Raise it and the sensor waits for a fuller
+# batch -- and a poke that finds fewer than that parks, keeping the offsets it read, so the
+# next poke carries on rather than starting over. max_messages: 50 is the other half: it
+# bounds what one run carries, so a topic that has been quiet for a week does not hand a
+# single step ten thousand messages.
+#
+# NO group_id, DELIBERATELY. Without one the sensor tracks offsets itself, in the attempt's
+# cursor, and commits nothing back to the cluster. Nothing on the broker has to be arranged
+# for this to work, and a second pipeline over the same topic sees every message rather than
+# racing this one for it. Set a group_id when the offsets should outlive the run -- then the
+# commit happens in the poke that succeeded, and never in one that parked.
+#
+# READING THE SAME GROUND TWICE. Advancing the cursor is at-least-once: it is written by the
+# transaction that parks the attempt, so a worker that dies before that commit leaves the
+# older offsets behind and the next poke reads those messages again. A step that must not act
+# twice on a message makes itself idempotent; this one only reshapes, so it does not care.
+#
+# TO MAKE IT YOURS: point the connection at your own cluster, change the topic, and replace
+# the summarise step with whatever actually consumes the batch.
+
+format: dirigent/v1
+kind: pipeline
+code: kafka-consume-then-transform
+name: Consume a Kafka topic, then transform the batch
+description: Wait for messages on a Kafka topic and reshape the batch the sensor handed on.
+
+tags: [queues, kafka, sensor, transform, starter]
+
+requires:
+  blocks:
+    - kafka.consume
+    - transform.jq
+  connections:
+    - orders-topic
+
+steps:
+  wait:
+    block: kafka.consume
+    # A sensor's poll is a promise about how often the world is checked, and this one is
+    # checked every ten seconds. Between pokes the attempt is a row with a due time, not a
+    # worker: waiting on a quiet topic for the whole deadline costs nothing.
+    poll: 10s
+    deadline: 1h
+    # A wait that ran out is not a failure here: no messages arrived inside the hour, so the
+    # run is skipped and the next trigger tries again.
+    on_timeout: skip
+    config:
+      connection: orders-topic
+      topic: orders
+      min_messages: 1
+      max_messages: 50
+      # How long one poke waits on the broker before answering with what it has. It bounds the
+      # poke, never the wait.
+      poll_timeout: 5s
+      # latest begins at whatever arrives next, which is what a pipeline started by a message
+      # wants: earliest would replay the whole topic on the first poke of every run.
+      start: latest
+      # A value is JSON here because whatever produces to this topic writes JSON. text or
+      # base64 are the other two, and base64 is the default for a key because a key is often
+      # not text at all.
+      value_format: json
+
+  summarise:
+    block: transform.jq
+    depends_on: [wait]
+    config:
+      # The batch is ordinary step output from here on: a list of objects, each with the
+      # topic, partition, offset, key, value, timestamp and headers the broker recorded.
+      input: "${steps.wait.output.messages}"
+      program: >-
+        {
+          count: length,
+          partitions: (map(.partition) | unique),
+          first_offset: (map(.offset) | min),
+          last_offset: (map(.offset) | max),
+          values: map(.value)
+        }

dirigent_examples/shelves/queues/kafka-produce-then-consume.yaml

@@ -0,0 +1,124 @@
+# A run that puts records on a Kafka topic and then reads its own batch back off it.
+#
+# NEEDS A BROKER, and a topic to write to:
+#   docker compose -f infra/compose.queues.yaml up -d --wait
+#   docker exec -it dirigent-queues-redpanda-1 rpk topic create orders
+#   dg connection create kafka orders-topic --set bootstrap_servers='["127.0.0.1:9092"]'
+#   dg run examples/queues/kafka-produce-then-consume.yaml
+# Nothing else has to produce for this one: it publishes what it later reads, so the whole
+# round trip is in the document.
+#
+# ON THE COMPOSE STACK the broker is the infra/compose.brokers.yaml overlay
+# (`make docker-run-queues`), where the connection names it `redpanda:9092` rather than
+# `127.0.0.1:9092` -- a container resolves the service, not the host. Create the topic on that
+# stack with `docker compose ... exec redpanda rpk topic create orders`.
+# docs/queues.md is the family's home.
+#
+# Three hops, and what each one hands on:
+#   publish    kafka.produce, three records onto the topic. Output: produced, the topic, the
+#              last offset per partition, duration_ms and sent_bytes.
+#   wait       kafka.consume, reading the topic from the beginning, so it takes what the step
+#              before it just wrote.
+#   summarise  a jq transform over that batch, so the messages end as ordinary step output.
+#
+# THE CONNECTION IS NAMED, NOT CARRIED. requires.connections says the instance must already
+# hold `orders-topic`, and the document is refused at apply if it does not. Bootstrap servers,
+# a security setting and a sealed password belong to an environment, not to a pipeline.
+#
+# TWO SHAPES OF RECORD. An element of `records` is read as an envelope when it is an object
+# carrying `value` and nothing besides `key`, `value` and `headers`; every other element is
+# itself the value. So a list of documents from an earlier step publishes unchanged, and a
+# record that needs a key or a header says so in an envelope. An object that means to be a
+# value and would read as an envelope is written `{"value": {...}}`.
+#
+# WHERE THE KEY COMES FROM. `key: region` takes each record's key from that field of its
+# value, which is what puts every record of one region on one partition and keeps them in
+# order relative to each other. A record whose envelope names its own `key` keeps that one, and
+# a record with no such field fails the step rather than being published unkeyed.
+#
+# ACKS AND WHAT IDEMPOTENCE ACTUALLY BUYS. `acks: all` is the default and the only setting the
+# idempotent producer runs under: the broker recognises a record the client itself retried and
+# writes it once, so a connection that stumbles inside this step does not double-publish. A
+# RETRY OF THE STEP is a different matter -- it opens a new producer session, and all three
+# records go again. This example does not retry the publish, which is the simplest way to mean
+# it; a pipeline that must not publish twice either does the same or lets a compacted topic or
+# the reader settle the duplicates.
+#
+# WHY start: earliest HERE. The consume sensor's usual setting is `latest`, because a pipeline
+# started by a message wants what arrives next. This one reads a batch its own earlier step
+# wrote, which is already on the topic by the time the sensor pokes, so it starts at the oldest
+# record instead. On a topic something else is also writing to, that would replay the lot.
+#
+# TO MAKE IT YOURS: point the connection at your own cluster, change the topic, and feed
+# `records` from an earlier step instead of writing them inline -- an export in storage reaches
+# the publish through a storage.read, which is the one door a value comes in by.
+
+format: dirigent/v1
+kind: pipeline
+code: kafka-produce-then-consume
+name: Publish to a Kafka topic, then consume the batch
+description: Publish records to a Kafka topic and read the same batch back off it.
+
+tags: [queues, kafka, sensor, transform]
+
+requires:
+  blocks:
+    - kafka.produce
+    - kafka.consume
+    - transform.jq
+  connections:
+    - orders-topic
+
+steps:
+  publish:
+    block: kafka.produce
+    config:
+      connection: orders-topic
+      topic: orders
+      # Each record's key is this field of its value, so one region's records share a
+      # partition and keep their order.
+      key: region
+      records:
+        # A bare element is the value.
+        - {order: 1, region: north, total: 240}
+        - {order: 2, region: south, total: 95}
+        # An envelope, because this record carries headers as well as a value.
+        - value: {order: 3, region: north, total: 610}
+          headers:
+            source: till-4
+      # The default, and the only setting the idempotent producer runs under.
+      acks: all
+      # The whole publish, from the first send to the last acknowledgement.
+      timeout: 30s
+
+  wait:
+    block: kafka.consume
+    depends_on: [publish]
+    poll: 5s
+    deadline: 5m
+    config:
+      connection: orders-topic
+      topic: orders
+      # The batch the step before this one wrote is already on the topic, so this poke reads
+      # the whole topic rather than waiting for what arrives next.
+      start: earliest
+      min_messages: 3
+      max_messages: 50
+      poll_timeout: 5s
+      # The keys were written as text, so they are read back as text; base64 is the default,
+      # because a key is often not text at all.
+      key_format: text
+      value_format: json
+
+  summarise:
+    block: transform.jq
+    depends_on: [wait]
+    config:
+      input: "${steps.wait.output.messages}"
+      program: >-
+        {
+          count: length,
+          keys: (map(.key) | unique),
+          total: (map(.value.total) | add),
+          headers: (map(.headers) | map(select(length > 0)))
+        }

dirigent_examples/shelves/queues/rabbitmq-consume-ack-on-success.yaml

@@ -0,0 +1,102 @@
+# A run that starts from what is on a RabbitMQ queue, and why the default ack mode is the one
+# that hands a short batch back.
+#
+# NEEDS A BROKER. Bring one up and put something on the queue first:
+#   docker compose -f infra/compose.queues.yaml up -d --wait
+#   open http://127.0.0.1:15672   # dirigent / dirigent; declare a queue named `shop-orders`
+#                                 # and publish a JSON body to it from the management UI
+#   dg connection create rabbitmq shop-queue \
+#     --set url=amqp://dirigent@127.0.0.1:5672/ --set password=dirigent
+#   dg run examples/queues/rabbitmq-consume-ack-on-success.yaml
+# On the compose stack the broker is the infra/compose.brokers.yaml overlay and the connection
+# names it as `amqp://dirigent@rabbitmq:5672/`; docs/queues.md is the family's home.
+#
+# Two hops, and what each one hands on:
+#   wait       rabbitmq.consume, parked until at least one message is on the queue. Output:
+#              messages and count.
+#   summarise  a jq transform over that batch, which is the only thing this example does with
+#              the messages.
+#
+# THE CONNECTION IS NAMED, NOT CARRIED. requires.connections says the instance must already
+# hold `shop-queue`. The password is never written into the url: a plain field is neither
+# encrypted at rest nor redacted in an API response, so the secret goes in the connection's
+# sealed password field, which is both, and is merged into the url when a connection is opened
+# and nowhere else. A url written amqp://user:secret@host is refused, the same way the sql
+# kind refuses one.
+#
+# WHY ack: on_success IS THE DEFAULT. RabbitMQ, unlike Kafka, keeps its own place: a message is
+# held unacknowledged until it is acked or nacked. So the question is not where to read but
+# when to acknowledge, and the safe answer is "only in the poke that succeeded".
+#
+# Work through what a poke does with min_messages: 3 and two messages on the queue. It takes
+# both, finds the batch too small, and nacks both back onto the queue with requeue -- so they
+# are still there for the next poke, or for another consumer, and nothing was lost to a batch
+# that was never acted on. Then a third message arrives, the next poke takes all three, hands
+# them to the transform, and acknowledges them.
+#
+# ack: always is the other choice, and it is a real one with a real cost: every message is
+# acknowledged the moment it is taken, park or no park, so a batch that parks is gone. It
+# suits a queue nothing else reads and a step that would rather drop a partial batch than see
+# a message twice.
+#
+# ON_SUCCESS IS STILL AT-LEAST-ONCE. Acknowledging happens inside the poke, and the attempt's
+# outcome commits after it: a worker that dies in between leaves the messages acked and the
+# step unfinished, and a worker that dies before the ack leaves them on the queue to be taken
+# again. Neither ordering gives exactly-once -- that would need the broker and this instance's
+# database in one transaction -- so a step that must not act twice makes itself idempotent.
+#
+# THE QUEUE MUST ALREADY EXIST. This block declares nothing: a queue a pipeline invented is a
+# typo that reads as a working pipeline, so a name nobody created is rejected, with the name in
+# the message, rather than waiting forever on a queue that will never have anything in it.
+#
+# TO MAKE IT YOURS: point the connection at your own broker, change the queue, and replace the
+# summarise step with whatever actually consumes the batch.
+
+format: dirigent/v1
+kind: pipeline
+code: rabbitmq-consume-ack-on-success
+name: Consume a RabbitMQ queue, acknowledging only on success
+description: Wait for messages on a RabbitMQ queue and reshape the batch, handing a short batch back to the queue.
+
+tags: [queues, rabbitmq, sensor, transform, starter]
+
+requires:
+  blocks:
+    - rabbitmq.consume
+    - transform.jq
+  connections:
+    - shop-queue
+
+steps:
+  wait:
+    block: rabbitmq.consume
+    poll: 10s
+    deadline: 1h
+    # A wait that ran out is not a failure: nothing arrived inside the hour, so the run is
+    # skipped and the next trigger tries again.
+    on_timeout: skip
+    config:
+      connection: shop-queue
+      queue: shop-orders
+      min_messages: 1
+      max_messages: 50
+      # How long one poke waits on the broker before answering with what it has.
+      poll_timeout: 5s
+      # Written out although it is the default, because it is the decision this example is
+      # about. Change it to always and a batch below min_messages is acknowledged and dropped.
+      ack: on_success
+      value_format: json
+
+  summarise:
+    block: transform.jq
+    depends_on: [wait]
+    config:
+      # Each message carries the routing key, the exchange, the delivery tag, the decoded
+      # body, the headers, and the publisher's timestamp when it stamped one.
+      input: "${steps.wait.output.messages}"
+      program: >-
+        {
+          count: length,
+          routing_keys: (map(.routing_key) | unique),
+          bodies: map(.body)
+        }

dirigent_examples/shelves/queues/report-to-kafka.yaml

@@ -0,0 +1,105 @@
+# A rendered page published to a Kafka topic, for whatever reads the topic to deliver.
+#
+# NEEDS A BROKER, and a topic to write to:
+#   docker compose -f infra/compose.queues.yaml up -d --wait
+#   docker exec -it dirigent-queues-redpanda-1 rpk topic create orders
+#   dg connection create kafka orders-topic --set bootstrap_servers='["127.0.0.1:9092"]'
+#   dg run examples/queues/report-to-kafka.yaml
+# On the compose stack the broker is the infra/compose.brokers.yaml overlay and the
+# connection names it `redpanda:9092`; docs/queues.md is the family's home.
+#
+# Two hops, and what each one hands on:
+#   page      report.render, the markdown. Output: text, content_type, text_bytes.
+#   publish   kafka.produce, one record carrying the page. Output: produced, the topic, the
+#             last offset per partition, duration_ms and sent_bytes.
+#
+# THE RENDER IS A STEP, THE SENDING IS ANOTHER. report.render has no destination of its own:
+# it renders text and hands it on, and the step below it decides where the text goes. The
+# same template writes a file in examples/recipes/report-to-file.yaml, an object in
+# examples/s3/report-to-s3.yaml, and a queue message in report-to-rabbitmq.yaml.
+#
+# WHY THE RECORD CARRIES A CODE. A consumer reading this topic gets one JSON record whose
+# `text` is the whole page, so `code` is what tells it which report arrived without parsing
+# the markdown. It is the record's key too, so every day's page for one report lands on one
+# partition and the days stay in order relative to each other.
+#
+# A PUBLISH IS NOT IDEMPOTENT ACROSS A RETRY: a retried step opens a new producer session and
+# publishes the record again. This document does not retry it, which is the simplest way to
+# mean that.
+#
+# TO MAKE IT YOURS: change the template, change the topic, and give the record whatever
+# envelope the consumer on the other side expects.
+
+format: dirigent/v1
+kind: pipeline
+code: report-to-kafka
+name: Publish a rendered report to a Kafka topic
+description: Render a markdown page from a Jinja template and publish it as one record on a Kafka topic.
+
+tags: [queues, report, kafka, transform]
+
+requires:
+  blocks:
+    - transform.jq
+    - report.render
+    - kafka.produce
+  connections:
+    - orders-topic
+
+params:
+  type: object
+  properties:
+    day:
+      type: string
+      format: date
+      default: "2026-01-01"
+      description: The day the report is about.
+
+steps:
+  summary:
+    block: transform.jq
+    config:
+      input:
+        day: ${params.day}
+        orders:
+          - {order: 1, region: north, total: 240}
+          - {order: 2, region: south, total: 95}
+          - {order: 3, region: north, total: 610}
+      program: |
+        {day: .day,
+         orders: (.orders | length),
+         total: (.orders | map(.total) | add),
+         by_region: (.orders | group_by(.region)
+                     | map({region: .[0].region, total: map(.total) | add}))}
+
+  page:
+    block: report.render
+    depends_on: [summary]
+    config:
+      values:
+        summary: ${steps.summary.output.value}
+      content_type: text/markdown
+      template: |
+        # Orders for {{ summary.day }}
+
+        {{ summary.orders }} orders, {{ summary.total }} in total.
+
+        {% for row in summary.by_region %}
+        - {{ row.region }}: {{ row.total }}
+        {% endfor %}
+
+  publish:
+    block: kafka.produce
+    depends_on: [page]
+    config:
+      connection: orders-topic
+      topic: orders
+      # The record's key, so one report's days share a partition and keep their order.
+      key: code
+      records:
+        - code: daily-orders
+          day: ${params.day}
+          text: ${steps.page.output.text}
+          content_type: ${steps.page.output.content_type}
+      acks: all
+      timeout: 30s

dirigent_examples/shelves/queues/report-to-rabbitmq.yaml

@@ -0,0 +1,105 @@
+# A rendered page published to a RabbitMQ queue, for whatever consumes the queue to deliver.
+#
+# NEEDS A BROKER, and a queue to publish to:
+#   docker compose -f infra/compose.queues.yaml up -d --wait
+#   open http://127.0.0.1:15672   # dirigent / dirigent; declare a queue named `shop-reports`
+#   dg connection create rabbitmq shop-queue \
+#     --set url=amqp://dirigent@127.0.0.1:5672/ --set password=dirigent
+#   dg run examples/queues/report-to-rabbitmq.yaml
+# On the compose stack the broker is the infra/compose.brokers.yaml overlay and the
+# connection names it `amqp://dirigent@rabbitmq:5672/`; docs/queues.md is the family's home.
+#
+# Two hops, and what each one hands on:
+#   page      report.render, the markdown. Output: text, content_type, text_bytes.
+#   publish   rabbitmq.publish, one message carrying that text. Output: published, which is
+#             one, and message_bytes.
+#
+# THE DEFAULT EXCHANGE IS THE ONE WITH NO NAME, and on it a routing key is a queue name. So
+# `exchange: ""` with `routing_key: shop-reports` puts the message on that queue and nowhere
+# else. The queue has to exist: the publish is mandatory, so a routing key nothing takes is
+# refused by the broker and fails the step, rather than dropped without a word. Naming an
+# exchange instead routes by whatever bindings the broker holds, and an exchange the broker
+# does not have is rejected by name the same way.
+#
+# WHAT THE MESSAGE CARRIES. A string message is sent as UTF-8 text and anything else as
+# canonical JSON, and the content type follows -- `text/plain` or `application/json` unless
+# the step names one. This one sends the page itself and says it is markdown, so a consumer
+# reads the body rather than unwrapping an envelope first.
+#
+# PERSISTENT: TRUE is the default, and it is what makes a durable queue keep the message
+# across a broker restart. A publish is not idempotent either way: a retry of this step puts
+# a second copy on the queue, so a pipeline that must not do that either does not retry it or
+# gives the consumer something to settle duplicates by.
+#
+# TO MAKE IT YOURS: change the template, change the queue, and point the connection at your
+# own broker.
+
+format: dirigent/v1
+kind: pipeline
+code: report-to-rabbitmq
+name: Publish a rendered report to a RabbitMQ queue
+description: Render a markdown page from a Jinja template and publish it as one message on a RabbitMQ queue.
+
+tags: [queues, report, rabbitmq, transform]
+
+requires:
+  blocks:
+    - transform.jq
+    - report.render
+    - rabbitmq.publish
+  connections:
+    - shop-queue
+
+params:
+  type: object
+  properties:
+    day:
+      type: string
+      format: date
+      default: "2026-01-01"
+      description: The day the report is about.
+
+steps:
+  summary:
+    block: transform.jq
+    config:
+      input:
+        day: ${params.day}
+        orders:
+          - {order: 1, status: shipped}
+          - {order: 2, status: held}
+          - {order: 3, status: shipped}
+      program: |
+        {day: .day,
+         orders: (.orders | length),
+         held: [.orders[] | select(.status == "held") | .order]}
+
+  page:
+    block: report.render
+    depends_on: [summary]
+    config:
+      values:
+        summary: ${steps.summary.output.value}
+      content_type: text/markdown
+      template: |
+        # Shop report for {{ summary.day }}
+
+        {{ summary.orders }} orders, {{ summary.held | length }} held.
+
+        {% for order in summary.held %}
+        - order {{ order }} is held
+        {% endfor %}
+
+  publish:
+    block: rabbitmq.publish
+    depends_on: [page]
+    config:
+      connection: shop-queue
+      # The default exchange, where the routing key is the queue's own name.
+      exchange: ""
+      routing_key: shop-reports
+      message: ${steps.page.output.text}
+      content_type: ${steps.page.output.content_type}
+      # The default, written out because it is what a durable queue needs to survive a restart.
+      persistent: true
+      timeout: 30s