datadog-structurizr 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. datadog_structurizr-0.1.0/LICENSE +21 -0
  2. datadog_structurizr-0.1.0/MANIFEST.in +4 -0
  3. datadog_structurizr-0.1.0/PKG-INFO +521 -0
  4. datadog_structurizr-0.1.0/README.md +496 -0
  5. datadog_structurizr-0.1.0/examples/README.md +24 -0
  6. datadog_structurizr-0.1.0/examples/checkout-web/c4.toml +36 -0
  7. datadog_structurizr-0.1.0/examples/checkout-web/raw/definition.json +13 -0
  8. datadog_structurizr-0.1.0/examples/checkout-web/raw/dependencies-checkout-worker.json +9 -0
  9. datadog_structurizr-0.1.0/examples/checkout-web/raw/dependencies.json +17 -0
  10. datadog_structurizr-0.1.0/examples/checkout-web/raw/resources.json +26 -0
  11. datadog_structurizr-0.1.0/examples/checkout-web/raw/types.json +11 -0
  12. datadog_structurizr-0.1.0/pyproject.toml +43 -0
  13. datadog_structurizr-0.1.0/setup.cfg +4 -0
  14. datadog_structurizr-0.1.0/src/datadog_structurizr/__init__.py +3 -0
  15. datadog_structurizr-0.1.0/src/datadog_structurizr/client.py +263 -0
  16. datadog_structurizr-0.1.0/src/datadog_structurizr/config.py +183 -0
  17. datadog_structurizr-0.1.0/src/datadog_structurizr/emitter.py +212 -0
  18. datadog_structurizr-0.1.0/src/datadog_structurizr/main.py +119 -0
  19. datadog_structurizr-0.1.0/src/datadog_structurizr/mapper.py +260 -0
  20. datadog_structurizr-0.1.0/src/datadog_structurizr/model.py +58 -0
  21. datadog_structurizr-0.1.0/src/datadog_structurizr/render.py +39 -0
  22. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/PKG-INFO +521 -0
  23. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/SOURCES.txt +31 -0
  24. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/dependency_links.txt +1 -0
  25. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/entry_points.txt +2 -0
  26. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/requires.txt +8 -0
  27. datadog_structurizr-0.1.0/src/datadog_structurizr.egg-info/top_level.txt +1 -0
  28. datadog_structurizr-0.1.0/tests/conftest.py +50 -0
  29. datadog_structurizr-0.1.0/tests/test_client.py +234 -0
  30. datadog_structurizr-0.1.0/tests/test_config.py +146 -0
  31. datadog_structurizr-0.1.0/tests/test_emitter.py +100 -0
  32. datadog_structurizr-0.1.0/tests/test_examples.py +92 -0
  33. datadog_structurizr-0.1.0/tests/test_mapper.py +206 -0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 veritas-sovereign
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,4 @@
1
+ include tests/conftest.py
2
+ # The example config and saved responses (made up) are test fixtures.
3
+ include examples/README.md
4
+ recursive-include examples/checkout-web *.toml *.json
@@ -0,0 +1,521 @@
1
+ Metadata-Version: 2.4
2
+ Name: datadog-structurizr
3
+ Version: 0.1.0
4
+ Summary: Generate C4 system context, container and component diagrams of a Datadog APM service as Structurizr DSL and Mermaid.
5
+ Author: Amit Kumar
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/veritas-sovereign/datadog-structurizr
8
+ Project-URL: Changelog, https://github.com/veritas-sovereign/datadog-structurizr/blob/main/CHANGELOG.md
9
+ Keywords: c4,c4-model,datadog,apm,structurizr,mermaid,architecture,diagrams
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Environment :: Console
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Operating System :: OS Independent
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Topic :: Software Development :: Documentation
16
+ Requires-Python: >=3.9
17
+ Description-Content-Type: text/markdown
18
+ License-File: LICENSE
19
+ Requires-Dist: requests>=2.31
20
+ Requires-Dist: python-dotenv>=1.0
21
+ Requires-Dist: tomli>=1.1; python_version < "3.11"
22
+ Provides-Extra: dev
23
+ Requires-Dist: pytest>=8; extra == "dev"
24
+ Dynamic: license-file
25
+
26
+ <div align="center">
27
+
28
+ # datadog-structurizr
29
+
30
+ ### Generate C4 diagrams of a Datadog APM service as Structurizr DSL and Mermaid
31
+
32
+ <br>
33
+
34
+ [![Full documentation](https://img.shields.io/badge/📖_full_documentation-github.com%2Fveritas--sovereign%2Fdatadog--structurizr-2f7ed8?style=for-the-badge&labelColor=555555)](https://github.com/veritas-sovereign/datadog-structurizr#readme)
35
+
36
+ <br>
37
+
38
+ 🚀 [Quick Start](#quick-start) · 📦 [PyPI](https://pypi.org/project/datadog-structurizr/) · 🔑 [Datadog Access](#datadog-access) · 📋 [Mapping Conventions](#mapping-conventions) · ⚠️ [Limitations](#limitations) · 🧪 [Examples](examples/README.md) · 📝 [Changelog](CHANGELOG.md)
39
+
40
+ <br>
41
+
42
+ [![PyPI](https://img.shields.io/pypi/v/datadog-structurizr?logo=pypi&logoColor=white&label=PyPI)](https://pypi.org/project/datadog-structurizr/)
43
+ [![License: MIT](https://img.shields.io/badge/License-MIT-c9a227)](LICENSE)
44
+ [![Python](https://img.shields.io/badge/Python-3.9%2B-3776ab?logo=python&logoColor=white)](pyproject.toml)
45
+ [![Input](https://img.shields.io/badge/Input-Datadog%20APM-632ca6?logo=datadog&logoColor=white)](https://docs.datadoghq.com/tracing/)
46
+ [![Output](https://img.shields.io/badge/Output-Structurizr%20DSL-438dd5)](https://docs.structurizr.com/dsl)
47
+ [![Output](https://img.shields.io/badge/Output-Mermaid%20C4-ff3670?logo=mermaid&logoColor=white)](https://mermaid.js.org/syntax/c4.html)
48
+ [![Tests](https://github.com/veritas-sovereign/datadog-structurizr/actions/workflows/test.yml/badge.svg)](https://github.com/veritas-sovereign/datadog-structurizr/actions/workflows/test.yml)
49
+ [![Last commit](https://img.shields.io/github/last-commit/veritas-sovereign/datadog-structurizr)](https://github.com/veritas-sovereign/datadog-structurizr/commits)
50
+
51
+ </div>
52
+
53
+ ---
54
+
55
+ ## Overview
56
+
57
+ `datadog-structurizr` reads one service's topology from Datadog APM and writes three C4 views of it:
58
+
59
+ | Here | C4 model | C4 view | Shows | Built from |
60
+ | --- | --- | --- | --- | --- |
61
+ | L0 | Level 1 | System Context | the user, the software system containing the service, the services it calls and is called by | service dependencies |
62
+ | L1 | Level 2 | Container | the service, the other services you name as part of the system, and their datastores inside the system boundary, with the neighbouring systems | service dependencies |
63
+ | L2 | Level 3 | Component | the service's entry points: HTTP endpoint groups and handlers | span resources |
64
+
65
+ The [C4 model](https://c4model.com/) numbers these views 1, 2 and 3. This project names them L0, L1 and L2, as do the view names and file names it writes.
66
+
67
+ L2 shows the API surface that APM sees, not the code structure behind it. Treat its components as a first draft of the service's entry points, not as its controllers, services and repositories.
68
+
69
+ Each run writes:
70
+
71
+ - a [Structurizr DSL](https://docs.structurizr.com/dsl) workspace with all three views. The generated model and views go into two files that every run rewrites; `workspace.dsl` includes them, is created once, and is yours to edit
72
+ - one [Mermaid C4](https://mermaid.js.org/syntax/c4.html) diagram per view, which renders on GitHub and in VS Code, and as SVG with `mmdc`
73
+ - the raw Datadog responses, so you can generate the diagrams again with `--offline` and no API calls
74
+
75
+ It is meant as a starting point for architecture documentation. Keep `workspace.dsl` as the source you maintain, and run the tool again whenever the services change: your edits survive.
76
+
77
+ ## Quick Start
78
+
79
+ With Python (3.9 or later):
80
+
81
+ ```bash
82
+ pip install datadog-structurizr
83
+ export DD_API_KEY=... DD_APP_KEY=... DD_SITE=datadoghq.com
84
+ datadog-structurizr --service checkout-web --env prod
85
+ ```
86
+
87
+ For a web component made of several services, name the system and the other services in it:
88
+
89
+ ```bash
90
+ datadog-structurizr --service checkout-web --env prod --system-name Checkout --include checkout-worker
91
+ ```
92
+
93
+ To try it without Datadog keys, download [`examples/checkout-web/`](examples/checkout-web) and run, from inside that folder:
94
+
95
+ ```bash
96
+ datadog-structurizr -c c4.toml --offline -o .
97
+ ```
98
+
99
+ Open the output folder in [Structurizr local](#viewing-locally), paste `workspace-inline.dsl` (one file, nothing to include) into the [Structurizr playground](https://playground.structurizr.com), or open the `.mmd` files in any Mermaid viewer.
100
+
101
+ ## Installation
102
+
103
+ Pick one:
104
+
105
+ | Option | Best for | You need |
106
+ | --- | --- | --- |
107
+ | [PyPI](#from-pypi) | everyday use | Python 3.9+; optionally `mmdc` (Node.js) to render SVGs |
108
+ | [From source](#from-source) | changing the tool or running its tests | Python 3.9+, git |
109
+ | [Docker](#docker) | running without Python | Docker |
110
+
111
+ You also need a Datadog API key and application key. See [Datadog access](#datadog-access).
112
+
113
+ ### From PyPI
114
+
115
+ [`datadog-structurizr`](https://pypi.org/project/datadog-structurizr/) is published to PyPI for every release.
116
+
117
+ ```bash
118
+ python3 -m venv .venv && source .venv/bin/activate # Windows: .venv\Scripts\activate
119
+ pip install datadog-structurizr # latest release
120
+ pip install datadog-structurizr==0.1.0 # or a specific version
121
+ datadog-structurizr --help
122
+ ```
123
+
124
+ To render the diagrams as SVG, also install the Mermaid CLI:
125
+
126
+ ```bash
127
+ npm install -g @mermaid-js/mermaid-cli
128
+ ```
129
+
130
+ ### From source
131
+
132
+ 1. Clone the repository and enter it:
133
+
134
+ ```bash
135
+ git clone https://github.com/veritas-sovereign/datadog-structurizr.git
136
+ cd datadog-structurizr
137
+ ```
138
+
139
+ 2. Create and activate a virtual environment. `.venv/` is already git-ignored.
140
+
141
+ ```bash
142
+ python3 -m venv .venv
143
+ source .venv/bin/activate # Windows: .venv\Scripts\activate
144
+ ```
145
+
146
+ 3. Install the package in editable mode. Add `[dev]` to also install pytest.
147
+
148
+ ```bash
149
+ pip install -e . # or: pip install -e '.[dev]'
150
+ ```
151
+
152
+ 4. Check the install:
153
+
154
+ ```bash
155
+ datadog-structurizr --version
156
+ ```
157
+
158
+ To use the code without installing it, run `pip install -r requirements.txt` and prefix commands with `PYTHONPATH=src`.
159
+
160
+ ### Docker
161
+
162
+ Each release publishes an image with the tool and no renderers to `ghcr.io/veritas-sovereign/datadog-structurizr`, for amd64 and arm64, tagged with the version (`0.1.0`) and the minor version (`0.1`). It is built from the [`Dockerfile`](Dockerfile); `docker build -t datadog-structurizr .` builds the same image from a clone.
163
+
164
+ Run it in the folder that holds `c4.toml`. `--user` makes the output files yours, and the keys come from the environment:
165
+
166
+ ```bash
167
+ docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/work" \
168
+ -e DD_API_KEY -e DD_APP_KEY \
169
+ ghcr.io/veritas-sovereign/datadog-structurizr:0.1.0 -c c4.toml -o . --no-render
170
+ ```
171
+
172
+ The image has no `mmdc` or structurizr-cli, so use `--no-render`, and [view](#viewing-locally) or validate the output with the `structurizr/structurizr` image.
173
+
174
+ ## Usage
175
+
176
+ ### Generate the diagrams
177
+
178
+ ```bash
179
+ datadog-structurizr [-c c4.toml] [--service NAME] [--env ENV] [--hours N] [-o DIR]
180
+ [--system-name NAME] [--include SERVICE[,SERVICE]]... [--no-person]
181
+ [--offline] [--no-render] [--no-input]
182
+ ```
183
+
184
+ | Option | Meaning |
185
+ | --- | --- |
186
+ | `-c`, `--config` | [config file](#config-file) with the service, system boundary, classification and person |
187
+ | `--service` | APM service name |
188
+ | `--env` | APM environment, such as `prod`. The dependencies API requires it |
189
+ | `--hours` | lookback window in hours for dependencies and span resources (default 24) |
190
+ | `-o`, `--output` | output directory (default `output`) |
191
+ | `--system-name` | name of the software system (default: the service name) |
192
+ | `--include` | other APM services that are containers of the same system. Comma-separated, and repeatable. Adds to `include` in the config file |
193
+ | `--no-person` | draw no person; for services only called by other services |
194
+ | `--offline` | rebuild from `<output>/raw/*.json` instead of calling Datadog; no keys needed |
195
+ | `--no-render` | write `.dsl` and `.mmd` sources only; do not render images |
196
+ | `--no-input` | never prompt (see [Prompts](#prompts)) |
197
+ | `--version` | print the version |
198
+
199
+ ### What to tell the tool
200
+
201
+ Only the service and environment are required. The other inputs fix the cases where Datadog data alone gives the wrong picture:
202
+
203
+ | Input | Why you would set it | Flag | Config file |
204
+ | --- | --- | --- | --- |
205
+ | Service and environment | required | `--service`, `--env` | `service`, `env` |
206
+ | Other services in the same system | A web component is often several services (web, BFF, worker). Without this, each one is drawn as a separate system and the container view is almost empty | `--include` | `[system] include` |
207
+ | System name | The system is usually called something other than its main service, for example `Checkout` rather than `checkout-web` | `--system-name` | `[system] name`, `description` |
208
+ | Classification | A dependency whose name does not match the [name hints](#elements) ends up in the wrong group | — | `[classify] datastores`, `external`, `internal` |
209
+ | Services to leave out | Telemetry collectors, agents and sidecars clutter every view | — | `ignore` |
210
+ | The person | The default `User` is generic. Name the real user, or turn the person off for a service that is only called by other services | `--no-person` | `[person] enabled`, `name`, `description` |
211
+
212
+ ### Config file
213
+
214
+ Keep the inputs in a TOML file next to the generated `workspace.dsl`, and commit it, so a later run gives the same diagrams. A complete example is [`examples/checkout-web/c4.toml`](examples/checkout-web/c4.toml):
215
+
216
+ ```toml
217
+ service = "checkout-web"
218
+ env = "prod"
219
+ hours = 24
220
+ ignore = ["otel-collector", "datadog-agent*"]
221
+
222
+ [system]
223
+ name = "Checkout"
224
+ description = "Lets shoppers review their cart and pay."
225
+ include = ["checkout-worker"]
226
+
227
+ [person]
228
+ enabled = true
229
+ name = "Shopper"
230
+ description = "Buys products on the web shop."
231
+
232
+ [classify]
233
+ datastores = ["ledger"]
234
+ external = []
235
+ internal = []
236
+
237
+ [components]
238
+ max = 12
239
+ strip_prefixes = []
240
+ ```
241
+
242
+ | Key | Type | Meaning |
243
+ | --- | --- | --- |
244
+ | `service`, `env` | string | APM service and environment |
245
+ | `hours` | integer | lookback window (default 24) |
246
+ | `site` | string | Datadog site (default `datadoghq.com`) |
247
+ | `output` | string | output directory (default `output`) |
248
+ | `ignore` | list | services left out of every view |
249
+ | `[system] name`, `description` | string | the software system's name and description |
250
+ | `[system] include` | list | other APM services that are containers of this system. Exact names; each one costs one more API call |
251
+ | `[person] enabled` | boolean | draw a person (default `true`) |
252
+ | `[person] name`, `description` | string | the person (default `User`) |
253
+ | `[classify] datastores`, `external`, `internal` | list | force a dependency into a group, overriding the name hints |
254
+ | `[components] max` | integer | most components shown in L2, the last being `Other` (default 12, at least 2) |
255
+ | `[components] strip_prefixes` | list | path prefixes removed before routes are grouped, such as an app's context path `"/shop"` |
256
+
257
+ `ignore` and the `[classify]` lists accept `fnmatch` patterns (`*`, `?`, `[abc]`) and ignore case. Top-level keys such as `ignore` must come before the first `[table]`, as TOML requires. Unknown keys and wrong types stop the run with an error naming the key, so a typo cannot be silently ignored. Datadog keys are refused in the config file.
258
+
259
+ ### Environment
260
+
261
+ Datadog keys only come from the environment. Environment variables can also come from a `.env` file in the current directory. Copy [`.env.example`](.env.example) to start:
262
+
263
+ | Variable | Required | Meaning |
264
+ | --- | --- | --- |
265
+ | `DD_API_KEY` | yes, unless `--offline` | Datadog API key |
266
+ | `DD_APP_KEY` | yes, unless `--offline` | Datadog application key with `apm_read` |
267
+ | `DD_SITE` | no | Datadog site: `datadoghq.com` (default), `datadoghq.eu`, `us3.datadoghq.com`, `us5.datadoghq.com`, `ap1.datadoghq.com`. A copied `api.` or `app.` prefix, `https://` and a trailing `/` are removed |
268
+ | `DD_SERVICE` | no, if given elsewhere | APM service name |
269
+ | `DD_ENV` | no, if given elsewhere | APM environment |
270
+ | `DD_LOOKBACK_HOURS` | no | lookback window in hours (default 24) |
271
+ | `OUTPUT_DIR` | no | output directory (default `output`) |
272
+
273
+ `.env` and `.env.*` are git-ignored, except `.env.example`. Do not commit keys.
274
+
275
+ Where the settings come from, highest priority first:
276
+
277
+ 1. command-line flags
278
+ 2. the config file
279
+ 3. environment variables and `.env`
280
+ 4. defaults
281
+
282
+ ### Prompts
283
+
284
+ If the service or environment is still missing after these, the tool asks for it, but only when it runs in a terminal. In CI, or with `--no-input`, it stops with an error listing what is missing. It never asks for keys.
285
+
286
+ ### Outputs
287
+
288
+ ```
289
+ output/
290
+ ├── workspace.dsl yours: created once, never overwritten; includes the two files below
291
+ ├── datadog-model.dsl generated elements and relationships; rewritten every run
292
+ ├── datadog-views.dsl generated L0-SystemContext, L1-Containers, L2-Components views; rewritten every run
293
+ ├── workspace-inline.dsl workspace.dsl with both files pasted in, for the Structurizr playground
294
+ ├── L0-SystemContext.mmd Mermaid C4Context
295
+ ├── L1-Containers.mmd Mermaid C4Container
296
+ ├── L2-Components.mmd Mermaid C4Component
297
+ ├── *.svg rendered views, if mmdc is installed
298
+ ├── plantuml/ PlantUML export and images, if structurizr-cli and plantuml are installed
299
+ └── raw/
300
+ ├── dependencies.json dependencies of the service
301
+ ├── dependencies-<service>.json dependencies of each included service
302
+ ├── resources.json resource names and span counts
303
+ ├── definition.json service definition, if the service has one
304
+ └── types.json span counts per span type of each neighbouring service
305
+ ```
306
+
307
+ `raw/` is replaced as a whole, and only after every Datadog call of the run has succeeded. It never mixes responses from different runs, and a failed run leaves the previous one in place. Do not keep other files in it.
308
+
309
+ `raw/` holds your internal service and route names. Check it before you commit it anywhere public.
310
+
311
+ ### Typical workflow
312
+
313
+ 1. Copy [`examples/checkout-web/c4.toml`](examples/checkout-web/c4.toml), and set `service`, `env` and the system name.
314
+ 2. Run against Datadog once: `datadog-structurizr -c c4.toml`.
315
+ 3. Look at the SVGs. Add the services that belong to the system to `include`, move wrongly grouped dependencies with `[classify]`, and drop noise with `ignore`. Then run again with `--offline`, which makes no API calls. A service newly added to `include` needs one more online run, to fetch its dependencies.
316
+ 4. Add to `workspace.dsl` the elements and relationships Datadog cannot see, such as which component calls which downstream service. Commit `c4.toml`, `workspace.dsl` and the two `datadog-*.dsl` files to your documentation repository. Later runs rewrite only the `datadog-*.dsl` files.
317
+ 5. Validate it, so a hand-written relationship to a renamed element is caught:
318
+
319
+ ```bash
320
+ docker run --rm -v "$PWD/output:/usr/local/structurizr" structurizr/structurizr validate -workspace /usr/local/structurizr/workspace.dsl
321
+ ```
322
+
323
+ [`examples/github-actions/c4.yml`](examples/github-actions/c4.yml) does this in CI: pull requests rebuild the model from the committed `raw/` responses and validate it, and a manual or weekly run fetches fresh data with the Datadog keys from the repository secrets. Until the package is on PyPI, the example installs it from GitHub; pin a commit there rather than `main`.
324
+
325
+ ### Rendering
326
+
327
+ | Renderer | Used when | Output |
328
+ | --- | --- | --- |
329
+ | [`mmdc`](https://github.com/mermaid-js/mermaid-cli) | on `PATH` | `L0-SystemContext.svg`, `L1-Containers.svg`, `L2-Components.svg` |
330
+ | [structurizr-cli](https://docs.structurizr.com/cli) + [PlantUML](https://plantuml.com/) | both on `PATH` | `plantuml/*.puml` and `plantuml/*.svg` |
331
+
332
+ When neither is installed, only the sources are written. The generated workspace sets no theme, so structurizr-cli needs no network access to read it.
333
+
334
+ structurizr-cli is end of life; it still works when installed. Its replacement is the `export` command of the `structurizr/structurizr` image, which takes the same options. It writes PlantUML or Mermaid sources from `workspace.dsl`, so unlike the tool's own `.mmd` files they include your hand edits:
335
+
336
+ ```bash
337
+ docker run --rm --user "$(id -u):$(id -g)" -v "$PWD/output:/usr/local/structurizr" structurizr/structurizr \
338
+ export -w /usr/local/structurizr/workspace.dsl -f plantuml/c4plantuml -o /usr/local/structurizr/export
339
+ ```
340
+
341
+ Use `-f mermaid` for Mermaid. The image renders PNG or SVG only from a running Structurizr page (`-url`), so render the exported sources with `plantuml` or `mmdc`.
342
+
343
+ ### Viewing locally
344
+
345
+ [Structurizr local](https://docs.structurizr.com/local) serves the workspace in a browser, with a diagram editor for layout. It reads `workspace.dsl` from the mounted folder, so the `!include` files are found next to it:
346
+
347
+ ```bash
348
+ docker run -it --rm -p 8080:8080 -v "$PWD/output:/usr/local/structurizr" structurizr/structurizr local
349
+ ```
350
+
351
+ Then open http://localhost:8080. Structurizr Lite and the separate `structurizr/cli` image are end of life; the `structurizr/structurizr` image replaces both.
352
+
353
+ ## Datadog access
354
+
355
+ ### Endpoints
356
+
357
+ | API | Used for | If it fails |
358
+ | --- | --- | --- |
359
+ | [`GET /api/v1/service_dependencies/{service}`](https://docs.datadoghq.com/api/latest/service-dependencies/) | `calls` and `called_by` of the service and of each included service: L0 and L1 | the run stops |
360
+ | [`POST /api/v2/spans/analytics/aggregate`](https://docs.datadoghq.com/api/latest/spans/) grouped by `resource_name` | components: L2 | the run stops. L2 shows one placeholder component only when the call succeeds and finds no indexed spans |
361
+ | [`POST /api/v2/spans/analytics/aggregate`](https://docs.datadoghq.com/api/latest/spans/) grouped by `service`, then `type`, for the neighbouring services | element types: L0, L1 | the run stops |
362
+ | [`GET /api/v2/services/definitions/{service}`](https://docs.datadoghq.com/api/latest/service-definition/) | description, team and languages | 404 (no definition) is ignored; any other status stops the run |
363
+
364
+ When Datadog answers 429 (rate limited), the call is retried up to 2 times, each after the number of seconds Datadog gives in the `x-ratelimit-reset` header. A 429 without that header, or asking for more than 60 seconds, is not retried.
365
+
366
+ When a call fails, the run prints the HTTP status, the call and Datadog's message to stderr, exits with status 1, and writes no diagrams. A diagram drawn from partial data would look complete but be wrong.
367
+
368
+ ### Keys and permissions
369
+
370
+ - The application key needs the `apm_read` permission. A read-only service account key is enough. The tool makes no write calls.
371
+ - `DD_SITE` must match your Datadog organisation's site, or every call returns 403.
372
+
373
+ ### Limits
374
+
375
+ - The service dependencies endpoint is in public beta.
376
+ - Span aggregates only cover **indexed** spans (those kept by retention filters). The span counts on components show relative traffic, not total requests.
377
+ - Span aggregates were limited to 50 requests per 60 seconds when measured on a live account (the `x-ratelimit-*` response headers), and that allowance is shared with everything else in the organisation that queries spans. A run makes two or three of these requests. When it is used up, Datadog answers 429.
378
+ - Span aggregates return the **100** resource names with the most spans. Resources beyond those are not fetched and do not appear in L2, not even in `Other`. They are the least-used ones. The response has no cursor for the next page, so there is no way to fetch more.
379
+
380
+ ## Mapping conventions
381
+
382
+ ### Elements
383
+
384
+ Each dependency name is checked against these rules in order. The first rule that matches decides:
385
+
386
+ | Rule | C4 element | Where it appears |
387
+ | --- | --- | --- |
388
+ | the service itself, or a name in `include` | container in the target system | L0 (inside the system), L1, L2 (the service, as the boundary) |
389
+ | matches `ignore` | left out | — |
390
+ | matches `[classify] datastores` | database container inside the target system | L1 |
391
+ | matches `[classify] external` | external software system, shown in grey | L0, L1 |
392
+ | matches `[classify] internal` | internal software system | L0, L1 |
393
+ | its own spans' most common type is a datastore type: `sql`, `redis`, `valkey`, `elasticsearch`, `opensearch`, `dynamodb`, `mongodb`, `cosmosdb` | database container inside the target system, with that type as its technology | L1 |
394
+ | its own spans' most common type is a service type: `web`, `http`, `rpc`, `soap`, `serverless` | external system if the name matches `EXTERNAL_HINTS`, otherwise internal system; the datastore name hints are skipped | L0, L1 |
395
+ | name matches `DATASTORE_HINTS` (`postgres`, `redis`, `kafka`, a `db` token, ...) | database container inside the target system | L1 |
396
+ | name matches `EXTERNAL_HINTS` (`stripe`, `aws.`, `s3`, `twilio`, ...) | external software system, shown in grey | L0, L1 |
397
+ | any other service | internal software system | L0, L1 |
398
+
399
+ The span types come from one spans aggregate over the neighbouring services, grouped by service and then by `type`; spans with no type are not counted. The two type lists are the values seen on a live account, so a type outside them (for example `queue` or `custom`) leaves the decision to the name hints. A span type cannot tell an external service from an internal one.
400
+
401
+ The hints are matched case-insensitively, and `db` must be a whole word, so `feedback-service` is not a datastore. The hint lists are in [`mapper.py`](src/datadog_structurizr/mapper.py). Use `[classify]` rather than editing them.
402
+
403
+ The person (`User` by default) uses the service over HTTPS. It appears in L0, L1 and L2 unless it is turned off.
404
+
405
+ A service that calls itself gets no relationship. Relationships between containers of the target system appear in L1 and are hidden in L0, where they are inside the system.
406
+
407
+ ### Components
408
+
409
+ - HTTP resources such as `GET /api/v1/cart/{id}` are grouped by their first path segment after `api`, version segments (`v1`, `v2`), and ids. `GET /api/v1/cart/{id}` and `POST /api/v1/cart/items` both go into `Cart API`, with the technology `HTTP endpoint group`.
410
+ - Other resources, such as queue consumers and jobs, each become one component with the technology `Entry point`.
411
+ - Probe endpoints are dropped: paths whose last segment is `health` or ends in `health` (`app-health`, `service-health`), `healthz`, `healthcheck`, `ready`, `readyz`, `live`, `livez`, `ping` or `metrics`, and any path with an `actuator` segment.
412
+ - Static files (`.js`, `.css`, `.html`, images, fonts, also pre-compressed as `.br` or `.gz`) go into one `Static content` component.
413
+ - A method with no route (`GET`, `POST`), which some tracers record when they cannot name the endpoint, goes into one `Unrouted HTTP` component.
414
+ - When every route starts with the same context path, such as `/shop/...`, they all land in one group. List that prefix in `[components] strip_prefixes` to group by the segment after it.
415
+ - Components are sorted by span count. At most 12 are shown, and the rest are combined into `Other`. Change the number with `[components] max`.
416
+ - The person calls each HTTP endpoint group. Without a person, the services that call the target service do. Entry points get no incoming relationship.
417
+ - Components are only built for the target service, not for included services. Run the tool again with an included service as `service` to get its components.
418
+
419
+ ### Identifiers
420
+
421
+ Element identifiers are the names with every character other than a letter, digit or `_` replaced by `_`. The system's identifier is its name followed by `_system`. Component identifiers are prefixed with their container's identifier and `__`. Output is the same for the same input.
422
+
423
+ ## Limitations
424
+
425
+ - **Span types and names decide the element type** unless you classify them. A dependency with no typed spans in the window, or a type outside the known lists, is classified by name. Check L1, and use `[classify]` for anything in the wrong group.
426
+ - **No component-to-dependency relationships.** Datadog does not say which entry point calls which downstream service, so L2 shows no relationships from components to other services. Add them by hand in `workspace.dsl`.
427
+ - **Hand edits appear only in the Structurizr output.** The Mermaid diagrams are drawn from Datadog data alone.
428
+ - **Generated identifiers can change.** Hand-written relationships refer to generated identifiers such as `checkout_web__Cart_API`. If a service or route group is renamed or disappears, structurizr-cli reports the dangling identifier, and you fix `workspace.dsl` by hand.
429
+ - **One system per run.** Run the command once per system and merge the workspaces by hand if you need a landscape view.
430
+ - **Layout is automatic.** Structurizr views use `autoLayout`. Mermaid's C4 layout is basic; for presentation diagrams, use the Structurizr output.
431
+
432
+ ## Examples
433
+
434
+ | Directory | What it shows |
435
+ | --- | --- |
436
+ | [`checkout-web/`](examples/checkout-web) | A `c4.toml` and saved responses for a web service and its worker: internal, datastore and external dependencies, two callers, route groups, a probe, a queue consumer, an ignored collector and a classified datastore |
437
+
438
+ See [`examples/README.md`](examples/README.md) for how to run it.
439
+
440
+ ## Testing
441
+
442
+ ```bash
443
+ pip install -e '.[dev]'
444
+ pytest
445
+ ```
446
+
447
+ The tests replace the Datadog calls with fakes, so they need no keys or network access. They run `examples/checkout-web/` end to end and cover configuration, the client, the mapper and the emitter.
448
+
449
+ ## Project structure
450
+
451
+ ```
452
+ datadog-structurizr/
453
+ ├── src/
454
+ │ └── datadog_structurizr/ Python package
455
+ │ ├── __init__.py package version
456
+ │ ├── main.py datadog-structurizr command
457
+ │ ├── config.py settings from the command line, config file, environment and .env
458
+ │ ├── client.py Datadog API calls; saves and replays raw responses
459
+ │ ├── model.py C4 model classes
460
+ │ ├── mapper.py maps Datadog data onto the C4 model
461
+ │ ├── emitter.py writes Structurizr DSL and Mermaid C4
462
+ │ └── render.py optional rendering with mmdc, or structurizr-cli and plantuml
463
+ ├── examples/
464
+ │ ├── README.md how to run the sample
465
+ │ ├── checkout-web/ sample c4.toml and saved Datadog responses
466
+ │ └── github-actions/c4.yml example workflow: rebuild, validate, refresh from Datadog
467
+ ├── tests/
468
+ │ ├── conftest.py clean environment and shared settings
469
+ │ ├── test_examples.py end-to-end offline runs
470
+ │ ├── test_config.py config file, precedence, prompts and errors
471
+ │ ├── test_client.py request shapes, fallbacks and errors, with fake responses
472
+ │ ├── test_mapper.py boundary, classification, ignore, person and components
473
+ │ └── test_emitter.py DSL nesting and Mermaid views
474
+ ├── .github/
475
+ │ ├── dependabot.yml weekly updates for GitHub Actions and the Docker base image
476
+ │ └── workflows/
477
+ │ ├── test.yml pytest on Python 3.9 and 3.13, Structurizr validation, Docker image build
478
+ │ ├── publish-pypi.yml builds the package; publishes it to PyPI on v* tags
479
+ │ └── publish-image.yml tests the image; publishes it to GHCR on v* tags
480
+ ├── Dockerfile image with the tool and no renderers; published to GHCR on releases
481
+ ├── .dockerignore
482
+ ├── .env.example settings template
483
+ ├── pyproject.toml package metadata and datadog-structurizr command
484
+ ├── requirements.txt runtime dependencies
485
+ ├── CHANGELOG.md
486
+ ├── LICENSE
487
+ ├── .editorconfig
488
+ ├── .gitattributes
489
+ └── .gitignore
490
+ ```
491
+
492
+ The package lives under `src/` so it can only be imported once installed, and so it is not mistaken for a second copy of the repository folder.
493
+
494
+ ### How a run works
495
+
496
+ ```
497
+ Datadog ──► client.py ──► mapper.py ──► emitter.py ──► datadog-*.dsl, *.mmd ──► render.py (optional)
498
+ fetch, dependencies DSL and mmdc, structurizr-cli
499
+ save raw/ and resources Mermaid text
500
+ to C4 model
501
+ ```
502
+
503
+ ## Naming conventions
504
+
505
+ - **Repository, distribution and command:** `datadog-structurizr` (kebab-case)
506
+ - **Python package:** `datadog_structurizr` (snake_case), in `src/`
507
+ - **Modules:** short snake_case nouns, without a `datadog_` prefix, since the package name already provides it
508
+ - **Views:** `L0-SystemContext`, `L1-Containers`, `L2-Components`, both as Structurizr view keys and as file names
509
+ - **C4 terms:** element, relationship, software system, container, component, person
510
+
511
+ ## Related
512
+
513
+ [drawio-structurizr](https://github.com/veritas-sovereign/drawio-structurizr) converts C4 diagrams drawn in draw.io into Structurizr DSL. Use it for the parts of the architecture that Datadog does not trace.
514
+
515
+ ## Contributing
516
+
517
+ Issues and pull requests are welcome at [veritas-sovereign/datadog-structurizr](https://github.com/veritas-sovereign/datadog-structurizr). Run `pytest` before opening a pull request (GitHub Actions runs it too), and add saved responses to `examples/` when you change how Datadog data is read.
518
+
519
+ ## License
520
+
521
+ Released under the [MIT License](LICENSE).