dirigent-examples 0.15.0__tar.gz

dirigent_examples-0.15.0/LICENSE

@@ -0,0 +1,18 @@
+Copyright (c) 2026 Morten Olav Hansen <morten@winterop.com>. All rights reserved.
+
+This source code and accompanying documentation are the property of
+Morten Olav Hansen. No license, express or implied, is granted to use, copy,
+modify, merge, publish, distribute, sublicense, or sell copies of this
+software or its derivatives.
+
+The source is published for reference only. Any use beyond reading
+requires written permission from the copyright holder.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES,
+OR OTHER LIABILITY ARISING FROM THE USE OF THE SOFTWARE.
+
+Third-party components redistributed with this software, and the licences they
+carry, are listed in THIRD_PARTY_NOTICES.md.

dirigent_examples-0.15.0/PKG-INFO

@@ -0,0 +1,21 @@
+Metadata-Version: 2.4
+Name: dirigent-examples
+Version: 0.15.0
+Summary: The dirigent example corpus: the shelves of runnable documents, installed.
+License-Expression: LicenseRef-Proprietary
+License-File: LICENSE
+Requires-Dist: dirigent-client
+Requires-Dist: dirigent-plugin
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+
+# dirigent-examples
+
+The dirigent example corpus as an installed distribution: 22 topic shelves of runnable
+`dirigent/v1` documents, contributed to a host through the `examples()` extension point and
+read by `dg examples`, `dg pipeline new`, and the instance's Examples screen.
+
+The shelves are the repository's `examples/` directory; the root path is a symlink to the
+real files here, so every `dg run --local examples/...` line keeps working in a checkout.
+[`shelves/README.md`](src/dirigent_examples/shelves/README.md) is the corpus's own guide: the
+shelves, the tag vocabulary, and what each document needs to run.

dirigent_examples-0.15.0/README.md

@@ -0,0 +1,10 @@
+# dirigent-examples
+
+The dirigent example corpus as an installed distribution: 22 topic shelves of runnable
+`dirigent/v1` documents, contributed to a host through the `examples()` extension point and
+read by `dg examples`, `dg pipeline new`, and the instance's Examples screen.
+
+The shelves are the repository's `examples/` directory; the root path is a symlink to the
+real files here, so every `dg run --local examples/...` line keeps working in a checkout.
+[`shelves/README.md`](src/dirigent_examples/shelves/README.md) is the corpus's own guide: the
+shelves, the tag vocabulary, and what each document needs to run.

dirigent_examples-0.15.0/pyproject.toml

@@ -0,0 +1,25 @@
+[project]
+name = "dirigent-examples"
+version = "0.15.0"
+description = "The dirigent example corpus: the shelves of runnable documents, installed."
+readme = "README.md"
+requires-python = ">=3.13"
+license = "LicenseRef-Proprietary"
+license-files = ["LICENSE"]
+dependencies = [
+    "dirigent-client",
+    "dirigent-plugin",
+]
+
+[project.entry-points."dirigent.plugins.v1"]
+examples = "dirigent_examples:plugin"
+
+[build-system]
+requires = ["uv_build>=0.12.0,<0.13.0"]
+build-backend = "uv_build"
+
+[tool.uv.sources.dirigent-client]
+workspace = true
+
+[tool.uv.sources.dirigent-plugin]
+workspace = true

dirigent_examples-0.15.0/pyproject.toml.orig

@@ -0,0 +1,24 @@
+[project]
+name = "dirigent-examples"
+version = "0.15.0"
+description = "The dirigent example corpus: the shelves of runnable documents, installed."
+readme = "README.md"
+requires-python = ">=3.13"
+license = "LicenseRef-Proprietary"
+license-files = ["LICENSE"]
+dependencies = [
+    # The python/ shelf is SDK scripts a reader runs, so the corpus carries the client.
+    "dirigent-client",
+    "dirigent-plugin",
+]
+
+[project.entry-points."dirigent.plugins.v1"]
+examples = "dirigent_examples:plugin"
+
+[build-system]
+requires = ["uv_build>=0.12.0,<0.13.0"]
+build-backend = "uv_build"
+
+[tool.uv.sources]
+dirigent-client = { workspace = true }
+dirigent-plugin = { workspace = true }

dirigent_examples-0.15.0/src/dirigent_examples/__init__.py

@@ -0,0 +1,22 @@
+"""The dirigent example corpus, installed: shelves of runnable ``dirigent/v1`` documents."""
+
+from collections.abc import Sequence
+from importlib.resources import files
+from importlib.resources.abc import Traversable
+
+from dirigent_plugin import extension
+
+#: The directory the topic shelves live in, inside this distribution.
+SHELVES_DIRECTORY = "shelves"
+
+
+class ExamplesPlugin:
+    """The plugin object the host discovers under the dirigent.plugins.v1 entry-point group."""
+
+    @extension
+    def examples(self) -> Sequence[Traversable]:
+        """Contribute the shelves this distribution carries, and nothing else."""
+        return [files(__package__ or "dirigent_examples") / SHELVES_DIRECTORY]
+
+
+plugin = ExamplesPlugin()

dirigent_examples-0.15.0/src/dirigent_examples/shelves/README.md

@@ -0,0 +1,299 @@
+# Example pipelines
+
+Every file here is a real `dirigent/v1` document, and every one of them runs. A test walks
+this directory on every CI run: the documents in `examples/` are validated against the real
+block catalog, and every document, including the previews, must survive a
+parse-export-reparse round trip unchanged.
+
+[`python/`](python) is the other half of the corpus: the documents here describe pipelines,
+and the scripts there drive an instance that holds them, through the `dirigent-client` SDK.
+
+The documents live on topic shelves, each with its own README:
+
+| Shelf | What lives there |
+| --- | --- |
+| [`graph/`](graph) | The shapes a DAG takes: lines, branches, fans, joins, and skips. |
+| [`patterns/`](patterns) | One engine behaviour per file: rules, retries, the two clocks, item policies, triggers, concurrency, and the reference language. |
+| [`failure/`](failure) | Retries, budgets, timeouts, tolerated failures, and the cleanup edge. |
+| [`transform/`](transform) | The reshaping verbs: jq programs and the `convert.std` codec, no allowlist anywhere. |
+| [`recipes/`](recipes) | One question per file: how to group, join, pivot, clean, convert, check, store, and call. |
+| [`triggers/`](triggers) | What starts a run on its own: the three clocks, and the inbound webhook. |
+| [`sensors/`](sensors) | Steps that wait for the world: a drop landing, a clock window opening. |
+| [`queues/`](queues) | A run started by a message: a Kafka topic and a RabbitMQ queue, waited on as sensors. |
+| [`execute/`](execute) | Code on the worker: a shell step behind the allowlist. |
+| [`docker/`](docker) | The container family: run one, watch it live, build an image and push it, and bring a whole compose stack up and down, on the worker's daemon or one a connection names. |
+| [`git/`](git) | An existing project brought into a run: a repository checked out, then built and brought up from what it ships. |
+| [`sql/`](sql) | A database read and written: bound parameters, one transaction, and a result handed to storage when it belongs in a file. |
+| [`composition/`](composition) | Pipelines made of pipelines, and the handoff to a second instance. |
+| [`s3/`](s3) | Object storage through the `s3://` scheme, with no S3 block anywhere. |
+| [`demo/`](demo) | Surfaces shown off: the run form, rendered markdown, the requires preflight, the weekly-import shared-name pair. |
+| [`open-data/`](open-data) | Real feeds against public, mostly keyless APIs: acme, health indicators, maps, earthquakes, humanitarian data. |
+| [`validate/`](validate) | A gate that checks a value's shape and passes it through: `validate.schema`, with the shape carried and named. |
+
+**The tag rule.** Every document's `tags:` is drawn from one vocabulary of three groups and
+nothing else. The **shelf** is the directory the file sits in, exactly one, and every document
+wears it. The **block families** are what its steps use, read off the block ids: `http`,
+`transform` (the jq verbs and the `convert.std` codec), `storage` (a storage block, or a
+converter's `source` and `target`), `execute` (shell and docker), `sql`, `git`, `webhook`
+(`webhook.post`, or an inbound webhook trigger), `kafka`, `rabbitmq`, `pipeline`, `validate`,
+and `sensor` for any sensor block. The **behaviours** are what the document teaches and what a
+reader filters by: `schedule`, `fan-out`, `graph`, `failure`, `retry`, `timeout`,
+`concurrency`, `params`, `references`, `rules`, `priority`, `window`, `composition`,
+`credential`, `outputs`, `filter`, `map`, `relay`, `csv`, `parquet`, `jq`, `sparql`,
+`geocode`, `briefing`, `observability`, `conventions`, `report` for a document that
+declares a `report:` section, so every run of it writes its own account of itself, or renders
+a page of its own with `report.render`, and `starter` for a document `dg pipeline new` may
+copy into a project: a real multi-step flow on a real source, carrying nothing. A source's
+name is never a tag: the pipeline's `code` and `name` already say it is USGS or WHO GHO, and a
+word one document wears is a filter nobody can use.
+
+**The starter rule.** `starter` is the one tag a document opts into rather than wears by
+description, and it is the narrowest. A document earns it by being a real multi-step flow on
+a real source -- a public endpoint, or a service a connection names. What disqualifies one is
+a top-level `connections:` or `schemas:` section, which an instance refuses at apply; a run
+that fails by design; a single step; and a hello-world. `dg pipeline new <code>` copies a
+starter's text verbatim into `pipelines/`, rewriting only the `code:` line and dropping
+`starter` from the `tags:` line, so the comments come with it and its `requires` is the list
+of what to create first.
+
+**The tiering rule.** `examples/` is what runs: every document there is runnable, though some
+need infrastructure whose setup their headers document -- the [`docker/`](docker) shelf needs a
+Docker socket or a daemon a connection names, [`queues/`](queues) needs the brokers
+`infra/compose.queues.yaml` starts, `sql-postgres-readonly.yaml` needs a PostgreSQL, and
+`s3-round-trip.yaml` needs an S3-compatible endpoint and a connection to it.
+`preview/` is for what cannot run yet because the *code* does not exist: documents whose block
+ids belong to adapter packs nobody has written. A missing credential is not a preview; a
+missing package is.
+
+The HTTP examples call [Postman Echo](https://postman-echo.com), a public request-and-
+response service, so you can run any of them for real without standing anything up:
+
+```bash
+dg run --local examples/hello-world.yaml
+dg run --local examples/graph/linear.yaml -p day=2026-01-01 --enable-unsafe shell.run
+dg run --local examples/failure/retries.yaml                    # watch the retry policy fire
+dg run --local examples/composition/chained-instances.yaml \
+  --connections examples/connections.yaml -p day=2026-01-01
+```
+
+Most of these use `shell.run`, which executes code on the worker and is refused unless the
+instance allowlists it. `--enable-unsafe shell.run` allows it for one command; the instance
+setting allows it for good:
+
+```bash
+export DIRIGENT_ENABLED_UNSAFE_BLOCKS='["shell.run"]'
+```
+
+Not every one of them succeeds, and that is the point of three of them:
+
+| Example | Ends as | Because |
+| --- | --- | --- |
+| `failure/retries.yaml` | `completed_with_errors` | The publish endpoint refuses three times; the failure is tolerated |
+| `failure/error-handler.yaml` | `failed` | The load fails, so the alert branch runs and the success branch is skipped |
+| `sensors/sensor-gate.yaml` | `succeeded` | The drop never lands, so the sensor skips and the branch skips with it |
+| `failure/optional-step.yaml` | `completed_with_errors` | The metrics push exits 3 and is tolerated, so the branch below it still runs |
+| `failure/retry-budget.yaml` | `failed` | A command that always exits 1, given three attempts to prove it |
+| `failure/step-timeout.yaml` | `failed` | A thirty-second sleep against a two-second budget, so it fails in about two |
+| `graph/skip-diamond.yaml` | `succeeded` | Green, with one step skipped: the handler had nothing to handle |
+
+`dg run --local` applies and runs the document in a throwaway SQLite instance that is
+deleted afterwards -- no server, no Docker, no database. When a step fails it prints the
+failing step, its block, the error class, the message, and that attempt's log lines before
+the database goes away. Against a real instance it is the same two commands:
+
+```bash
+dg apply examples/graph/linear.yaml
+dg run linear -p day=2026-01-01 --watch
+```
+
+The CI lane that walks these files never touches the network: it validates the documents
+and round-trips them, and the execution tests elsewhere use a mock transport. Postman Echo
+is for humans running the examples for real.
+
+## Engine patterns
+
+[`patterns/`](patterns) is the reference shelf. Every other shelf is organised by what a
+pipeline is *for*; that one is organised by what the engine *does*, one behaviour per file,
+named for what it shows: the four trigger rules, the retry policy's five fields, `timeout`
+against `deadline` against `on_timeout`, the two item policies, the three clocks a trigger can
+be, the four concurrency policies, the four questions `pipeline.run` answers, and every
+`${...}` form the reference language allows.
+
+Each header says what to expect before you run it, because several of them fail on purpose:
+
+```bash
+dg run --local examples/patterns/rule-one-failed.yaml          # failed, and the alert branch ran
+dg run --local examples/patterns/fan-out-continue.yaml         # completed_with_errors, two items of three
+dg run --local examples/patterns/timeout-skips-the-step.yaml   # succeeded, having skipped the branch
+```
+
+Nothing on that shelf runs code on the worker, so none of it needs `--enable-unsafe`. Four
+files are handed something a bare local run does not have -- a window, a connections file, a
+child document -- and [patterns/README.md](patterns/README.md) indexes all of them with the
+outcome each one settles as.
+
+## Connections
+
+Three of the examples reference a connection by code. Documents are portable precisely because
+they name credentials rather than carrying them, so the credential has to exist wherever the
+document is applied. On a server, create it once:
+
+```bash
+dg connection create http postman-echo --set base_url=https://postman-echo.com
+```
+
+A local run has no server to hold it, so it is handed the connection instead:
+
+```bash
+dg run --local examples/demo/requires.yaml --connections examples/connections.yaml
+```
+
+[connections.yaml](connections.yaml) is that file. The basic-auth credentials in it are
+Postman's own public demo pair. `basic_password` is a `SecretStr`, so it is encrypted at
+rest and redacted in every API response.
+
+## Schemas
+
+[`schemas/`](schemas) holds a few named JSON Schemas: the shape a read is expected to return,
+written down so a pipeline can be refused the moment it sees a payload that moved. A schema is
+locally authored -- it is a picture you hold of the payload, never something fetched from the
+source -- so it lives here as a plain JSON Schema and is applied on its own, not as part of a
+pipeline document:
+
+```bash
+dg schema create examples/schemas/ou-record.json
+```
+
+A schema reads its own identity from its keywords: `$id` becomes the `code` it is addressed
+by, `title` its name, and `description` its body. A server that applies a directory at boot
+stores the schemas it finds there too, before the pipelines that require them, so mounting this
+whole directory lands the schemas as well as the documents. See
+[schemas/README.md](schemas/README.md) and the [JSON Schema guide](../docs/json-schema.md).
+
+## Recipes
+
+[`recipes/`](recipes) is the widest shelf and the one to read while doing the work rather
+than while learning the format. Each file answers one question a person doing data work
+actually asks -- how do I group and sum, how do I join two lists, which types survive a
+parquet round trip, where do credentials live -- in a complete pipeline whose header comment
+is the lesson. Nothing on the shelf needs infrastructure, a credential, or an allowlist
+entry: the data is inline, generated by a jq program, or fetched from Postman Echo.
+
+```bash
+dg run --local examples/recipes/jq-group-by-and-sum.yaml
+dg run --local examples/recipes/etl-csv-clean-validate-parquet.yaml
+dg run --local examples/recipes/schema-refuses-then-rule.yaml   # fails, by design
+```
+
+Two of them end `failed` on purpose and say so in their headers:
+`recipes/schema-refuses-then-rule.yaml`, where a gate refuses and the error branch runs, and
+`recipes/reconcile-two-sources.yaml`, where the two sources disagree past the tolerance the
+run was given. [recipes/README.md](recipes/README.md) indexes all forty-seven by the question
+each one answers.
+
+## The examples
+
+Each shelf's README indexes its own documents; what stays at the root is the front door:
+
+| File | What it demonstrates |
+| --- | --- |
+| [hello-world.yaml](hello-world.yaml) | The smallest runnable document: one step, no parameters, nothing granted |
+| [connections.yaml](connections.yaml) | The credential file a `--local` run is handed |
+
+Two of them are worth running twice, because a parameter flips which branch is taken:
+
+```bash
+dg run --local examples/failure/retries.yaml -p status=200        # publish succeeds; still completed_with_errors as the cache times out
+dg run --local examples/failure/error-handler.yaml -p status=200  # the success branch, not the alert
+```
+
+## Preview documents
+
+[`preview/`](preview) holds documents whose blocks or packages an instance may not have. They
+are valid `dirigent/v1` and they round-trip, but applying one to an instance without what it
+names is refused by the `requires` preflight with a list of what is missing. They are here to
+show that the format never grows per-integration syntax: everything domain-specific lives
+inside `config`, which the format treats as opaque and each block validates against its own
+published schema.
+
+[`preview/s3-parquet-to-ingestion.yaml`](preview/s3-parquet-to-ingestion.yaml) is the one on
+the shelf today: every block it names is installed, and what it needs is the
+`dirigent-storage-s3` package and an endpoint for the `s3://` scheme it addresses.
+
+## Composing pipelines
+
+A dirigent document holds exactly one pipeline, so a parent and its child are two files, and
+the apply order matters: the child has to exist before the parent will apply at all. The
+parent says so in `requires.pipelines`, so a missing child is refused up front, with the
+code to apply first, rather than failing at the step.
+
+```bash
+dg apply examples/composition/composition-child.yaml
+dg apply examples/composition/composition-parent.yaml
+dg run composition-parent -p day=2026-01-01 --watch
+```
+
+A local run starts on an empty throwaway instance, so the child is handed over with
+`--also-apply`, which applies a document without running it:
+
+```bash
+dg run --local examples/composition/composition-parent.yaml \
+  --also-apply examples/composition/composition-child.yaml \
+  -p day=2026-01-01 --enable-unsafe shell.run
+```
+
+## Why every example is self-contained
+
+Almost every document here defines the pipeline it schedules or hooks, because a pipeline
+document carries its own `triggers:` section and one file is then one deployable unit. That
+is what keeps the schedule examples runnable on their own: each teaches one clock, and a
+reader can fire it ad hoc without applying anything else first.
+
+The other form has its own example. A `kind: triggers` document declares clocks and webhooks
+for a pipeline defined somewhere else and names it -- the ops team's clock file over a
+pipeline another team owns. [triggers/document-nightly.yaml](triggers/document-nightly.yaml)
+is that document, and it targets the pipeline in
+[triggers/managed-and-manual.yaml](triggers/managed-and-manual.yaml), so the two teach
+together: apply the pipeline, then apply its clocks.
+
+## Putting the whole corpus into one instance
+
+`make dev-seeded` boots a `dg dev --seed examples` with everything on this page already
+applied, so there is something to look at without applying forty documents by hand. The
+instance walks this directory for `dirigent/v1` documents, creates the connections
+[connections.yaml](connections.yaml) carries, applies each document with its schedules
+paused, and creates what a document carries for its own standalone run before applying it
+without that section. The target adds what only the demo wants: the object store
+`s3-round-trip.yaml` names, a document pointed at a connection nothing holds, six runs, and
+one health check per connection. Then it hands the terminal to `dg dev`; one ctrl-c ends both.
+
+```bash
+make dev-seeded | dg format
+```
+
+**Seeing failures is the point.** An instance where everything is green teaches nothing about
+what a failure looks like, so the seeded one is deliberately mixed:
+
+| What lands | How many | Why |
+| --- | --- | --- |
+| Pipelines stored | 159 | Every document here that an instance will hold, including the ones that carry their own connections or schemas: the seed creates what they carry and applies the rest, which is the only form an instance stores |
+| Documents refused | 18 | Nine in [docker/](docker) and two in [git/](git) name a compose or a build block the seed does not allowlist, some of them also a connection it does not create; four name a connection that does not exist, one of them (`warehouse-nobody-created`) built by the seed on purpose; one names a schema no instance here holds; and two require a pipeline or a target applied after them |
+| Schedules | 17, all paused | `--paused` is what stops seventeen clocks starting to fire at somebody who has not looked at them |
+| Runs | 3 succeeded, 1 with errors, 2 failed | `hello-world`, `transform/jq-reshape.yaml` and `triggers/cron-windowed.yaml` settle green; `optional-step.yaml` settles `completed_with_errors`, which is a third status rather than a shade of failed; `error-handler.yaml` fails by design, and `s3-round-trip.yaml` cannot reach an object store nobody started |
+| Connections | 10 healthy, 4 red | The ten that answer are `postman-echo` and the public endpoints the documents carry for their own standalone runs; `artifacts` points at `127.0.0.1:9000`, `work-db` at a database nobody started, and `orders-topic` and `shop-queue` at the two brokers in `infra/compose.queues.yaml`, where nothing is listening unless you started what their headers document |
+
+The refusals are reported by code with the reason the instance gave, and the seeding carries
+on past each one -- a refusal is a thing to look at, not an error to fix.
+
+State lands in `.dirigent/state` under the working directory: the SQLite database and the
+artifacts. `dg dev` empties that directory on every start, so each seeding builds a fresh
+instance and the key its connections are sealed with is minted per run and thrown away with
+them. `SEED_ROOT=/somewhere/else` puts it elsewhere, and `SEED_PORT` moves the instance off
+3333.
+
+Re-running is safe, and it starts over: the instance is emptied first, so every apply is a
+`created` against a database with nothing in it, nothing accumulates between seedings, and a
+schedule you resumed by hand is gone with the rest. That is the trade the target makes for
+never answering out of an older schema.
+
+The target is for looking at things by hand. Nothing in `make check` or CI runs it.

dirigent_examples-0.15.0/src/dirigent_examples/shelves/composition/README.md

@@ -0,0 +1,18 @@
+# Composition examples
+
+Pipelines made of pipelines: a child that stands on its own, a parent that runs it per
+region, and a handoff to a second instance when the next stage belongs to somebody else.
+`pipeline.run` keeps the child a first-class run -- its own id, its own log, its own row --
+rather than steps inlined into the parent.
+
+```bash
+dg run --local examples/composition/composition-parent.yaml --also-apply examples/composition/composition-child.yaml -p day=2026-01-01 --enable-unsafe shell.run
+```
+
+## Pipelines
+
+| File | What it teaches |
+| --- | --- |
+| [composition-child.yaml](composition-child.yaml) | The reusable unit: callable on its own, or by whatever composes it. |
+| [composition-parent.yaml](composition-parent.yaml) | `pipeline.run` per region: one child waited for, one fired and left to finish alone. |
+| [chained-instances.yaml](chained-instances.yaml) | Across the boundary: the result handed to another dirigent instance as a signed webhook, because the next stage is theirs. |

dirigent_examples-0.15.0/src/dirigent_examples/shelves/composition/chained-instances.yaml

@@ -0,0 +1,97 @@
+# Handing work to another dirigent instance, signed the way that instance verifies it.
+#
+# pipeline.run composes pipelines on one instance. This is the other shape: two instances,
+# each with its own database, schedules, and operators, where one hands the next a payload
+# and stops caring. The receiving side is an ordinary inbound webhook -- `dg webhook create`
+# on that instance -- whose token is the whole credential and whose params_from_payload maps
+# the body onto its own parameter schema.
+#
+# What webhook.post adds over http.request with method: POST is the signature, and that is
+# the only reason to reach for it. It renders the JSON body itself and computes an HMAC over
+# exactly those bytes, sent as X-Dirigent-Signature, in the format dirigent's own
+# /hooks/{token} intake verifies. Posting a body nobody signs is http.request's job.
+#
+# The secret is the connection's hmac_secret, named by sign_with. The receiving webhook is
+# declared with the same one:
+#   dg webhook create downstream-pipeline handoff --hmac-secret "$SHARED_SECRET"
+#
+# The secret lives on the connection, never in this file: a document names a credential, it
+# does not carry one.
+#
+# The receiver here is Postman Echo, which reflects whatever it is posted, so this runs
+# against no instance at all and the reflected body shows what went on the wire, signature
+# header included:
+#   dg run --local examples/composition/chained-instances.yaml \
+#     --connections examples/connections.yaml -p day=2026-01-01
+#
+# Against a real peer the only change is the connection: base_url becomes that instance's
+# URL, path becomes /hooks/<the token it minted>, and sign_with names the same connection.
+
+format: dirigent/v1
+kind: pipeline
+code: chained-instances
+name: One instance triggers another
+description: Produce a result, then hand it to another dirigent instance as a signed webhook.
+
+tags: [composition, http, webhook]
+
+requires:
+  blocks:
+    - http.request
+    - webhook.post
+  connections:
+    - postman-echo
+
+params:
+  type: object
+  required: [day]
+  additionalProperties: false
+  properties:
+    day:
+      type: string
+      format: date
+      description: The day whose result is handed downstream.
+
+steps:
+  produce:
+    block: http.request
+    config:
+      connection: postman-echo
+      path: /post
+      method: POST
+      body:
+        day: "${params.day}"
+        rows: 4200
+
+  hand_off:
+    block: webhook.post
+    depends_on: [produce]
+    retry:
+      max_attempts: 3
+      backoff: 5s
+    config:
+      connection: postman-echo
+      path: /post
+      # The receiving instance verifies the signature against the same secret, so a handoff
+      # cannot be forged by anything that merely knows the URL.
+      sign_with: postman-echo
+      # A receiver that queues the work answers 202, which is a success here even though the
+      # work has not happened yet: the handoff is what this step is responsible for.
+      success_status: [200, 201, 202]
+      headers:
+        X-Dirigent-Source: chained-instances
+      body:
+        day: "${params.day}"
+        upstream_run: "${run.id}"
+        rows: "${steps.produce.output.body.json.rows}"
+
+  confirm:
+    block: http.request
+    depends_on: [hand_off]
+    config:
+      connection: postman-echo
+      path: /post
+      method: POST
+      body:
+        handed_off_with_status: "${steps.hand_off.output.status}"
+        signed: "${steps.hand_off.output.signed}"

dirigent_examples-0.15.0/src/dirigent_examples/shelves/composition/composition-child.yaml

@@ -0,0 +1,64 @@
+# The child half of a composed pair: an ordinary pipeline that knows nothing about parents.
+#
+# Nothing here says it is a child: it has its own name, its own parameter schema, and its
+# own run history, and it can be run on its own, scheduled on its own, or called by three
+# different parents. Being callable is not something a document opts into.
+#
+# The parameter schema is the interface. composition-parent.yaml writes these parameters out
+# explicitly rather than forwarding whatever it was run with.
+#
+# Apply this one first -- composition-parent.yaml requires it by name:
+#   dg apply examples/composition/composition-child.yaml
+#   dg apply examples/composition/composition-parent.yaml
+#
+# Or run the parent locally, which applies this one into the throwaway instance for you:
+#   dg run --local examples/composition/composition-parent.yaml --also-apply examples/composition/composition-child.yaml
+
+format: dirigent/v1
+kind: pipeline
+code: composition-child
+name: The called pipeline
+description: Load one region's data for one day; callable on its own or by a parent pipeline.
+
+tags: [composition, http, storage]
+
+params:
+  type: object
+  required: [region, day]
+  additionalProperties: false
+  properties:
+    region:
+      type: string
+      description: Which region this run loads.
+    day:
+      type: string
+      format: date
+      description: The day being loaded.
+
+steps:
+  fetch:
+    block: http.request
+    retry:
+      max_attempts: 3
+    config:
+      url: https://postman-echo.com/post
+      method: POST
+      body:
+        region: "${params.region}"
+        day: "${params.day}"
+
+  store:
+    block: storage.write
+    depends_on: [fetch]
+    config:
+      # The child writes under its own run's scratch, so two children of the same parent
+      # never write to the same place, whatever region they were given.
+      target: "${run.scratch}/regions/${params.region}.json"
+      value: "${steps.fetch.output.body}"
+
+  archive:
+    block: storage.copy
+    depends_on: [store]
+    config:
+      source: "${steps.store.output.uri}"
+      target: "${run.scratch}/archive/${params.day}/${params.region}.json"