cloudmap 1.3.2__tar.gz → 1.3.3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (88) hide show
  1. cloudmap-1.3.3/.github/ISSUE_TEMPLATE/bug_report.yml +43 -0
  2. cloudmap-1.3.3/.github/ISSUE_TEMPLATE/config.yml +5 -0
  3. cloudmap-1.3.3/.github/ISSUE_TEMPLATE/feature_request.yml +20 -0
  4. cloudmap-1.3.3/.gitignore +30 -0
  5. cloudmap-1.3.3/ARCHITECTURE.md +168 -0
  6. {cloudmap-1.3.2 → cloudmap-1.3.3}/PKG-INFO +160 -49
  7. {cloudmap-1.3.2 → cloudmap-1.3.3}/README.md +159 -48
  8. cloudmap-1.3.3/SECURITY.md +101 -0
  9. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/__init__.py +1 -1
  10. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/extract/extractors.py +7 -4
  11. cloudmap-1.3.3/tests/test_deterministic_output.py +56 -0
  12. cloudmap-1.3.2/.gitignore +0 -22
  13. cloudmap-1.3.2/ARCHITECTURE.md +0 -158
  14. cloudmap-1.3.2/SECURITY.md +0 -46
  15. {cloudmap-1.3.2 → cloudmap-1.3.3}/.github/workflows/ci.yml +0 -0
  16. {cloudmap-1.3.2 → cloudmap-1.3.3}/.github/workflows/publish.yml +0 -0
  17. {cloudmap-1.3.2 → cloudmap-1.3.3}/CONTRIBUTING.md +0 -0
  18. {cloudmap-1.3.2 → cloudmap-1.3.3}/FORMAT.md +0 -0
  19. {cloudmap-1.3.2 → cloudmap-1.3.3}/LICENSE +0 -0
  20. {cloudmap-1.3.2 → cloudmap-1.3.3}/PLAN.md +0 -0
  21. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/__main__.py +0 -0
  22. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/adapters/__init__.py +0 -0
  23. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ask/__init__.py +0 -0
  24. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ask/intent.py +0 -0
  25. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ask/narration.py +0 -0
  26. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ask/queries.py +0 -0
  27. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/cli.py +0 -0
  28. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/data/contoso.json +0 -0
  29. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/extract/__init__.py +0 -0
  30. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/extract/llm.py +0 -0
  31. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/graph.py +0 -0
  32. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ingest/__init__.py +0 -0
  33. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ingest/azure.py +0 -0
  34. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/ingest/fixture.py +0 -0
  35. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/interactive.py +0 -0
  36. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/local_model.py +0 -0
  37. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/model.py +0 -0
  38. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/__init__.py +0 -0
  39. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/azure_icons.py +0 -0
  40. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/csv_export.py +0 -0
  41. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/drawio.py +0 -0
  42. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/html.py +0 -0
  43. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/json_out.py +0 -0
  44. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/render/mermaid.py +0 -0
  45. {cloudmap-1.3.2 → cloudmap-1.3.3}/cloudmap/scrub.py +0 -0
  46. {cloudmap-1.3.2 → cloudmap-1.3.3}/docs/social-preview.html +0 -0
  47. {cloudmap-1.3.2 → cloudmap-1.3.3}/docs/social-preview.png +0 -0
  48. {cloudmap-1.3.2 → cloudmap-1.3.3}/estate-viewer.png +0 -0
  49. {cloudmap-1.3.2 → cloudmap-1.3.3}/fixtures/acme_orders.json +0 -0
  50. {cloudmap-1.3.2 → cloudmap-1.3.3}/fixtures/contoso.json +0 -0
  51. {cloudmap-1.3.2 → cloudmap-1.3.3}/fixtures/estate.json +0 -0
  52. {cloudmap-1.3.2 → cloudmap-1.3.3}/pyproject.toml +0 -0
  53. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/01_input_complex_random.json +0 -0
  54. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/04_scrubbed_output.json +0 -0
  55. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/06_trace_output.json +0 -0
  56. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/07_trace_output.html +0 -0
  57. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/08_trace_output.csv +0 -0
  58. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/09_trace_output.drawio +0 -0
  59. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/10_input_enterprise_architecture.json +0 -0
  60. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/12_enterprise_trace.html +0 -0
  61. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/13_enterprise_trace.json +0 -0
  62. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/14_enterprise_scrubbed.json +0 -0
  63. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/complex_mock_demo/README.md +0 -0
  64. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/data/azure_resource_types.txt +0 -0
  65. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_adapters.py +0 -0
  66. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_arg_rows.py +0 -0
  67. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_ask.py +0 -0
  68. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_azure.py +0 -0
  69. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_cli_exports.py +0 -0
  70. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_common_types.py +0 -0
  71. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_containerapps.py +0 -0
  72. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_containment_vs_association.py +0 -0
  73. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_drawio_xml.py +0 -0
  74. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_enrich.py +0 -0
  75. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_estate.py +0 -0
  76. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_fixtures_safe.py +0 -0
  77. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_golden_orders.py +0 -0
  78. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_graph.py +0 -0
  79. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_html.py +0 -0
  80. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_ingest_paging.py +0 -0
  81. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_interactive_wizard.py +0 -0
  82. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_llm.py +0 -0
  83. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_local_model.py +0 -0
  84. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_render_text.py +0 -0
  85. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_scrub.py +0 -0
  86. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_trust.py +0 -0
  87. {cloudmap-1.3.2 → cloudmap-1.3.3}/tests/test_type_agnostic.py +0 -0
  88. {cloudmap-1.3.2 → cloudmap-1.3.3}/uv.lock +0 -0
@@ -0,0 +1,43 @@
1
+ name: Bug report
2
+ description: Something broke, or the map is wrong
3
+ labels: [bug]
4
+ body:
5
+ - type: input
6
+ id: version
7
+ attributes:
8
+ label: cloudmap version
9
+ description: "`pip show cloudmap` or `cloudmap --help` header"
10
+ placeholder: "1.3.2"
11
+ validations:
12
+ required: true
13
+ - type: input
14
+ id: command
15
+ attributes:
16
+ label: The exact command you ran
17
+ placeholder: "cloudmap trace my-app --live --allow-live --single-sub"
18
+ validations:
19
+ required: true
20
+ - type: textarea
21
+ id: what-happened
22
+ attributes:
23
+ label: What happened, and what did you expect
24
+ description: Paste the output. If the map itself is wrong (a missing or bogus edge), say which edge.
25
+ validations:
26
+ required: true
27
+ - type: textarea
28
+ id: fixture
29
+ attributes:
30
+ label: Minimal synthetic fixture (for wrong-map bugs)
31
+ description: >
32
+ The perfect report reproduces the wrong edge with a minimal synthetic
33
+ fixture - see fixtures/contoso.json for the shape. NEVER paste a real
34
+ capture; run `cloudmap scrub` on it first and read the file before
35
+ posting.
36
+ render: json
37
+ - type: checkboxes
38
+ id: scrubbed
39
+ attributes:
40
+ label: Safety check
41
+ options:
42
+ - label: Anything I pasted above contains no real resource names, subscription ids or credentials.
43
+ required: true
@@ -0,0 +1,5 @@
1
+ blank_issues_enabled: true
2
+ contact_links:
3
+ - name: Security issue
4
+ url: https://github.com/KatsaounisThanasis/cloudmap/security/advisories/new
5
+ about: Anything that could expose a user's infrastructure data - report it privately, never as a public issue.
@@ -0,0 +1,20 @@
1
+ name: Feature request
2
+ description: A resource type mapped poorly, a missing edge kind, a workflow that needs a flag
3
+ labels: [enhancement]
4
+ body:
5
+ - type: textarea
6
+ id: problem
7
+ attributes:
8
+ label: The situation
9
+ description: What were you trying to learn from the map, and what did it give you instead?
10
+ validations:
11
+ required: true
12
+ - type: textarea
13
+ id: proposal
14
+ attributes:
15
+ label: What you would expect
16
+ description: >
17
+ If it is a missing dependency for a resource type: which property of
18
+ which type expresses it? (That is usually a small extractor rule plus a
19
+ fixture test.) Keep in mind the rules in CONTRIBUTING.md - read-only,
20
+ local-first, every edge carries evidence.
@@ -0,0 +1,30 @@
1
+ __pycache__/
2
+ *.pyc
3
+ .venv/
4
+ venv/
5
+ build/
6
+ dist/
7
+ *.egg-info/
8
+ .pytest_cache/
9
+ .ruff_cache/
10
+
11
+ # generated diagram output
12
+ *.blast.drawio
13
+ *.mmd
14
+ out/
15
+
16
+ # never commit anything pulled from a live cloud
17
+ live/
18
+ *.live.json
19
+ .team/
20
+
21
+ # A live trace writes artifacts named after the resource it traced, so their
22
+ # names cannot be known in advance. Point --out-dir at live/ or out/ above.
23
+ # These root-anchored rules are the belt-and-braces for the ones that land in
24
+ # the repo root anyway - no tracked file at the root has these extensions, and
25
+ # the committed sample artifacts live under tests/ and docs/, which are unaffected.
26
+ /*.drawio
27
+ /*.json
28
+ /*.html
29
+ /*.csv
30
+ /*.mmd
@@ -0,0 +1,168 @@
1
+ # Architecture
2
+
3
+ > Written in English to match the rest of the repo (README, code, comments).
4
+ > `file.py:NN` references are indicative - they drift as the code moves. Trust the
5
+ > function and file names; grep for the symbol rather than jumping to the line.
6
+
7
+ ## Overview
8
+
9
+ cloudmap is a local-first CLI (Python 3.9+; two runtime deps, `rich` and
10
+ `questionary`, both for the terminal UI) that takes the **name of one Azure
11
+ resource** and produces its full **dependency graph (blast radius)** as an
12
+ **editable draw.io diagram**, plus an interactive HTML viewer, Mermaid, JSON and
13
+ CSV. It is a clean four-stage pipeline: **ingest** (fixture or live `az`) →
14
+ **extract** (properties / hostnames / RBAC / secrets → typed edges) → **graph**
15
+ (direction-consistent blast-radius BFS) → **render**. The core value is that
16
+ Azure Resource Graph has no "dependencies" table, so `extract/extractors.py`
17
+ infers dependencies and **verifies** each one - anything referenced but not
18
+ verified becomes an explicit `external` node instead of being silently dropped.
19
+
20
+ ## Architecture
21
+
22
+ ```mermaid
23
+ flowchart TD
24
+ CLI["__main__.py → cli.main()<br/>argparse: trace / capture / scrub / ask"] --> TRACE["_cmd_trace"]
25
+ CLI -->|"no args, tty"| WIZ["interactive.py<br/>wizard: sub → rg → resource"]
26
+ WIZ --> TRACE
27
+
28
+ TRACE -->|"--from (or 'demo')"| FIX["adapters/load_graph()<br/>raw export or neutral map"]
29
+ TRACE -->|"--live"| AZ["ingest/azure.py<br/>query_live()"]
30
+
31
+ AZ --> GUARD["_guard()<br/>optional ALLOW_SUB pin"]
32
+ AZ --> SUBS["_target_subscriptions()<br/>tenant-wide by default"]
33
+ AZ --> PAGE["_graph_paged()<br/>ALL types, skip_token paging"]
34
+
35
+ FIX --> BUILD["graph.build_graph()"]
36
+ AZ --> BUILD
37
+ BUILD --> EXTRACT["extract_edges()<br/>typed rules + 4 generic passes"]
38
+
39
+ TRACE --> ENRICH["_enrich_live() per seed type<br/>enrich_webapps() / enrich_aks_clusters()<br/>appsettings+RBAC+diag, secrets in-memory"]
40
+ ENRICH --> EXTERNAL["seed_external_dependencies()<br/>never-drop → external nodes"]
41
+
42
+ BUILD --> BLAST["blast_radius() graph.py<br/>BFS, direction consistency"]
43
+ EXTERNAL --> BLAST
44
+ BLAST --> COLLAPSE["collapse_high_level()<br/>singletons keep their name"]
45
+ COLLAPSE --> R1["render/drawio.py<br/>native Azure icons"]
46
+ COLLAPSE --> R2["render/mermaid.py"]
47
+ COLLAPSE --> R3["render/json_out.py<br/>meta: truncated/gaps/blind spots"]
48
+ COLLAPSE --> R4["render/html.py<br/>self-contained viewer"]
49
+ COLLAPSE --> R5["render/csv_export.py<br/>edge list + evidence"]
50
+
51
+ CLI --> ASK["_cmd_ask"]
52
+ R3 -.->|"saved map, reloaded"| FIX
53
+ ASK --> INTENT["ask/intent.py<br/>rules first, model only as<br/>validated fallback (--llm)"]
54
+ INTENT --> Q["ask/queries.py<br/>impact / depends / paths /<br/>shared / guesses / summary<br/>COMPUTED from edges"]
55
+ Q --> NARR["ask/narration.py (--explain)<br/>prose from the computed facts"]
56
+ INTENT -.-> LM["local_model.py<br/>the only outbound call:<br/>ollama or OpenAI-compatible, local"]
57
+ NARR -.-> LM
58
+ ```
59
+
60
+ ## File Map
61
+
62
+ | File | Role | Why it's built this way |
63
+ |---|---|---|
64
+ | `cloudmap/model.py` | `Node`/`Edge`/`Graph` dataclasses | Provider-neutral core. The `external`+`note` fields exist for the "never silent drop" principle - a referenced-but-unverified target still needs a place in the model. |
65
+ | `cloudmap/ingest/fixture.py` | Load synthetic/captured JSON | Accepts both a bare list and `{"data":[...]}` so the same file works as a fixture and as captured `az graph` output. |
66
+ | `cloudmap/ingest/azure.py` | Live `az` ingest + enrichment | The most sensitive file, so it concentrates every guard: the single `subprocess.run` boundary with timeouts, the optional `CLOUDMAP_ALLOW_SUBSCRIPTION` pin (never injected into `az account` commands, so a mismatch is reported as a mismatch), `classify_gap()` so artifacts carry categories instead of raw stderr, and the rule that Kubernetes secret text never reaches an export. |
67
+ | `cloudmap/extract/extractors.py` | **The heart**: properties/hosts/RBAC/secrets → edges | Typed rules for ~17 services plus four generic passes that make "any type" true: ARM-id references (with containment property paths excluded, not whole types), host mentions, nested child-of, and generic host indexing (a host counts as identity only when it carries the resource's own name - two apps on one App Service scale unit share `ftpsHostName`). Observers (alert rules, dashboards) emit `observes` instead of vanishing. |
68
+ | `cloudmap/graph.py` | Build graph + blast-radius BFS + the high-level collapse | Direction consistency in `blast_radius` (never reverse once you have stepped, tracked per direction) is why a shared VNet/plan does not pull in unrelated apps. `collapse_high_level` folds instances into one box per type, keeps a singleton's real name, and carries kinds AND evidence across. |
69
+ | `cloudmap/interactive.py` | The zero-argument wizard | Offers every resource in the subscription (no type allowlist - the engine has none), delegates all `az` I/O to `ingest.azure`, and pins the chosen subscription before handing off to `_cmd_trace`. |
70
+ | `cloudmap/scrub.py` | Global consistent pseudonymisation for captures | Identifier positions are renamed consistently (a reference and its target stay correlated); credentials are redacted, and its docstrings carry the postmortems of the two leak classes that shipped and were caught. |
71
+ | `cloudmap/render/drawio.py` | `.drawio` with Azure2 icons | Verified icon paths; falls back to a box, external nodes render dashed, so an icon is never broken. |
72
+ | `cloudmap/render/html.py` | One self-contained interactive viewer | No server, no CDN; the INCOMPLETE badge opens its "why" in the side panel, grouped ×N boxes list their instances on click. |
73
+ | `cloudmap/render/{mermaid,json_out,csv_export}.py` | Secondary outputs | Preview, machine-readable inventory (with the honesty meta), and a flat evidence-bearing edge list for auditors. |
74
+ | `cloudmap/ask/queries.py` | **The Ask layer's heart**: the six queries, each computed by traversal | An answer must be auditable, so it is derived from edges, never generated. `_trust()` grades a whole path by its weakest hop. |
75
+ | `cloudmap/ask/intent.py` | Question → one query | Rules first; the model is a fallback that may only name a query from a fixed list and a resource, both validated against the graph - it routes, it never answers. |
76
+ | `cloudmap/ask/narration.py` | Optional prose (`--explain`) | Handed the computed facts only, printed *below* them. |
77
+ | `cloudmap/local_model.py` | The single outbound model call | One module = one auditable promise: localhost (ollama's API by default, any OpenAI-compatible server via `CLOUDMAP_LLM_URL`), and failure returns an empty value. |
78
+ | `cloudmap/cli.py` | Orchestration + argparse | `_cmd_trace` wires the stages; `_cmd_capture`/`_cmd_scrub` produce committable fixtures; `_cmd_ask` loads a saved map and prints the computed answer, proof lines included. |
79
+ | `cloudmap/data/contoso.json` | The packaged demo estate | `--from demo` must work for a pip-only install - the first command a stranger runs cannot depend on cloning the repo. |
80
+ | `fixtures/` | Synthetic + scrubbed-golden estates | Fixture-first development → zero cloud contact in tests. |
81
+ | `tests/` | 468 tests | Behaviour-named; includes a conformance sweep of all 4,687 published Azure resource types (`tests/data/azure_resource_types.txt`) through every dependency channel, a realistic-estate suite with the property names Azure genuinely returns, and `test_fixtures_safe.py`, which fails CI if a fixture ever carries a credential. |
82
+ | `.gitignore` | Safety | `live/`, `*.live.json`, `*.blast.drawio` and every artifact extension at the repo root are ignored → output from a real tenant cannot be committed by accident. |
83
+
84
+ ## Execution Flow
85
+
86
+ **Fixture path** (`cloudmap trace contoso-web --from demo`):
87
+
88
+ 1. `__main__.py` → `cli.main()` - argparse, or the wizard when run bare on a tty.
89
+ 2. `_cmd_trace` - fixture branch → `adapters.load_graph()` (raw export or neutral
90
+ map, auto-detected; a reloaded capture keeps its truncated/gaps meta).
91
+ 3. `graph.build_graph()` → builds `Node`s and calls `extract_edges()`.
92
+ 4. `extract_edges()` - constructs a `Resolver` and derives edges: typed rules
93
+ (`serverFarmId`→hosted-on, VM `has-nic`/`uses-disk`, NIC `in-subnet`, config
94
+ hostnames/vault refs/IK, RBAC via system AND user-assigned identity
95
+ principals, `managedBy`), then the generic passes for every other shape.
96
+ 5. `find_seeds()` - exact id, exact name, else substring; >1 → each match is
97
+ printed with its resource group and full id.
98
+ 6. `blast_radius()` - BFS from the seed in both directions, never reversing.
99
+ 7. Render: draw.io / HTML / Mermaid / JSON / CSV, `_print_summary` shows
100
+ "Depends on" and "What depends on it" as separate panels.
101
+
102
+ **Live path** (`--live --allow-live [--resolve-secrets]`) - additionally:
103
+
104
+ 8. `query_live()` → `_guard()` (optional pin check) → `_target_subscriptions()`
105
+ → `_graph_paged()` (ALL resource types + tenant-wide role assignments,
106
+ `skip_token` paging, loud warning at the page cap).
107
+ 9. `_enrich_live()` picks targets by seed type: a workload seed enriches itself;
108
+ a config-referenced data service (Key Vault, SQL, storage, Redis, ...)
109
+ enriches every app in scope, because only their config reveals its
110
+ dependents; compute/network seeds stay seed-only - ARM already has their
111
+ relationships. `enrich_webapps()`/`enrich_aks_clusters()` run concurrently
112
+ (`CLOUDMAP_ENRICH_WORKERS`, default 12). Secrets resolve **in-memory** only.
113
+ 10. Rebuild, `seed_external_dependencies()` resurfaces unresolved references as
114
+ dashed external nodes, and whatever was *not* enriched is recorded as a
115
+ blind spot in `meta` - so the artifact, and every `ask` answer drawn from
116
+ it, repeats it.
117
+
118
+ ## Design Decisions
119
+
120
+ - **extract vs ingest separation** (`extractors.py` knows nothing about `az`).
121
+ All dependency logic is pure Python over dicts, so the full suite runs in ~2s
122
+ without Azure.
123
+ - **Direction consistency** (`blast_radius`). Real-world testing showed shared
124
+ plan/VNet connect dozens of unrelated apps. "From the seed go both ways, but
125
+ never reverse afterwards" solves it without a hub-type list; the visited set
126
+ is tracked per direction so a node reachable both ways loses neither cone.
127
+ - **Containment is not dependency** (`_CONTAINMENT_PATHS`). A VNet's
128
+ `subnets[].ipConfigurations` enumerate what is plugged INTO it; following them
129
+ made a shared VNet a hub. But its DDoS plan and peerings are real
130
+ dependencies, so the exclusion is on property paths, not on types - an
131
+ earlier type-wide exclusion silently lost all of those edges.
132
+ - **Generic before hand-written, resolver as the safety net.** A type with no
133
+ rule still maps through ARM ids, advertised hosts, child ids and RBAC; a
134
+ reference that names nothing scanned yields no edge, so the generic passes
135
+ cannot invent a dependency. This is what a 4,687-type conformance sweep pins.
136
+ - **Enrichment is scoped by seed type, and the gap is declared**
137
+ (`_enrichment_targets`). Tracing a VM must not deep-read 289 web apps to
138
+ learn what ARM already says; tracing a vault must, because config is the only
139
+ place its dependents exist. Whatever is skipped becomes a `blind_spot`.
140
+ - **Scrub preserves structure, not identity** (`scrub.py`), and a capture never
141
+ carries Kubernetes secret text at all - the field is stripped before any
142
+ export, under every flag combination.
143
+ - **Never silent drop** (`seed_external_dependencies` + `model.py`).
144
+ - **The model proposes, code verifies** (`extract/llm.py`): only proposals that
145
+ resolve to a scanned resource survive, drawn dashed and marked; `ask` may use
146
+ the model to route a phrasing or narrate a computed answer, never to state a
147
+ fact.
148
+
149
+ ## Open Questions / Risks
150
+
151
+ - **Live validation breadth**: the engine is conformance-tested against every
152
+ published type, but has been exercised against one real tenant. Property
153
+ shapes in other tenants/clouds may surface new false-positive classes like
154
+ the shared `ftpsHostName` one (found live, fixed, regression-pinned).
155
+ - **Deep config readers exist for App Service and AKS only.** Other types'
156
+ runtime config (Logic Apps run history, Data Factory linked-service secrets)
157
+ needs per-type API calls; today those dependencies appear only if expressed
158
+ in ARM properties, and the blind spot says so.
159
+ - **Cost of `--enrich all`** on large tenants: one `az` round-trip per app,
160
+ `CLOUDMAP_ENRICH_WORKERS` (default 12) at a time. No cache between runs yet -
161
+ every trace pays the scan again.
162
+ - **`az graph --skip-token`** depends on the resource-graph extension version.
163
+ - **`az aks command invoke`** starts a short-lived pod in the cluster to run
164
+ its read-only kubectl - documented in the README as the one asterisk on
165
+ "read-only".
166
+ - **Secret resolution reachability**: a vault behind an unreachable private
167
+ endpoint fails the in-memory resolve and the dependency shows as external
168
+ rather than resolved.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: cloudmap
3
- Version: 1.3.2
3
+ Version: 1.3.3
4
4
  Summary: Trace the blast radius of an Azure resource: one name in, a verified dependency graph out.
5
5
  Project-URL: Homepage, https://github.com/KatsaounisThanasis/cloudmap
6
6
  Project-URL: Repository, https://github.com/KatsaounisThanasis/cloudmap
@@ -48,6 +48,7 @@ Description-Content-Type: text/markdown
48
48
  <a href="#60-second-demo"><b>Demo</b></a> ·
49
49
  <a href="#interactive-wizard"><b>Wizard</b></a> ·
50
50
  <a href="#why"><b>Why</b></a> ·
51
+ <a href="#what-you-will-see---and-what-you-wont"><b>What you'll see</b></a> ·
51
52
  <a href="#how-it-works"><b>How it works</b></a> ·
52
53
  <a href="#ask-a-map-questions"><b>Ask</b></a>
53
54
  </p>
@@ -100,43 +101,57 @@ cloudmap trace contoso-web --from demo -o contoso-web.drawio
100
101
 
101
102
  ```text
102
103
  Blast radius: 9 resources (0 external), 9 dependencies
103
- ╭────────────────────────────── Dependency Graph ──────────────────────────────╮
104
+ ╭─────────────────────────────── Depends on (6) ───────────────────────────────╮
104
105
  │ 🌐 contoso-web │
105
- │ ├── --hosted-on--> 📦 App Service Plan
106
- │ ├── --vnet-integration--> 📦 Virtual Network
107
- │ ├── --connects-to--> 🗄️ SQL Server
108
- ├── --sends-telemetry--> 📦 App Insights
109
- │ └── --uses-workspace--> 📦 Log Analytics
110
- │ ├── --reads-secret; role: Key Vault Secrets User--> 🔐 Key Vault
111
- │ └── --connects-to--> 📦 Storage
106
+ │ ├── --hosted-on--> 📦 contoso-plan
107
+ │ ├── --vnet-integration--> 📦 contoso-vnet
108
+ │ ├── --sends-telemetry--> 📦 contoso-appi
109
+ │ └── --uses-workspace--> 📦 contoso-law
110
+ ├── --connects-to--> 🗄️ contoso-sql
111
+ │ ├── --connects-to--> 📦 contosostg
112
+ │ └── --reads-secret; role: Key Vault Secrets User--> 🔐 contoso-kv
113
+ ╰──────────────────────────────────────────────────────────────────────────────╯
114
+ ╭─────────────────────────── What depends on it (1) ───────────────────────────╮
115
+ │ 🌐 contoso-web │
116
+ │ └── <--routes-to-- 📦 contoso-agw │
112
117
  ╰──────────────────────────────────────────────────────────────────────────────╯
113
118
  🔗 draw.io: contoso-web.drawio
114
119
  ```
115
120
 
116
- That is the default **high-level** view - resources grouped by type. Add
117
- `--level detail` to see every instance with its real name, and on a real estate
118
- it goes several layers deep:
121
+ The two panels are the two halves of a blast radius: what the seed needs, and
122
+ what needs the seed. Arrows in the second panel are drawn backwards (`<--`)
123
+ because that is the direction the dependency actually runs.
124
+
125
+ That is the default **high-level** view - one box per resource type. Here every
126
+ type happens to have a single instance, so each box keeps its real name; where a
127
+ type has several, the box becomes `Web App ×4` and lists its members. Add
128
+ `--level detail` to always get every instance by name. On a larger estate the
129
+ map goes several layers deep:
119
130
 
120
131
  <details>
121
132
  <summary>A deeper map</summary>
122
133
 
134
+ Against the enterprise fixture in the repo
135
+ (`tests/complex_mock_demo/10_input_enterprise_architecture.json`):
136
+
123
137
  ```text
124
- Blast radius: 15 resources (0 external), 14 dependencies
125
- ╭────────────────────────────── Dependency Graph ──────────────────────────────╮
138
+ Blast radius: 16 resources (0 external), 15 dependencies
139
+ ╭─────────────────────────────── Depends on (2) ───────────────────────────────╮
126
140
  │ 🌐 app-spa-frontend │
127
141
  │ ├── --calls--> 🌐 app-auth-service │
128
- │ │ ├── --reads-secret--> 🔐 kv-core-prod
129
- │ │ └── --connects-to--> 🗄️ cosmos-auth
142
+ │ │ ├── --connects-to--> 🗄️ cosmos-auth
143
+ │ │ └── --reads-secret--> 🔐 kv-core-prod
130
144
  │ └── --calls--> 🌐 app-api-gateway │
145
+ │ ├── --connects-to--> 🗄️ redis-gateway │
131
146
  │ ├── --calls--> 📦 capp-payment-service │
132
- │ │ └── --reads-secret--> 🔐 kv-payments-prod │
147
+ │ │ ├── --reads-secret--> 🔐 kv-payments-prod │
148
+ │ │ └── --connects-to--> 📦 eh-transactions │
133
149
  │ ├── --calls--> 🌐 app-inventory-api │
134
150
  │ │ ├── --connects-to--> 🗄️ pg-inventory-prod │
135
151
  │ │ └── --connects-to--> 📦 stinventoryprod │
136
- │ ├── --connects-to--> 🗄️ redis-gateway │
137
152
  │ └── --calls--> 🌐 app-orders-api │
138
- │ ├── --connects-to--> 🗄️ redis-orders │
139
153
  │ ├── --connects-to--> 📦 sb-enterprise │
154
+ │ ├── --connects-to--> 🗄️ redis-orders │
140
155
  │ └── --connects-to--> 🗄️ sql-orders-prod │
141
156
  ╰──────────────────────────────────────────────────────────────────────────────╯
142
157
  ```
@@ -175,8 +190,10 @@ the results. It reads live Azure, so `az login` first.
175
190
 
176
191
  ## Why
177
192
 
178
- - **Live-cloud tools upload your data.** cloudmap runs locally and reads only
179
- what you point it at. Nothing leaves your machine.
193
+ - **Live-cloud tools upload your data.** cloudmap runs locally: the only traffic
194
+ it generates is the `az` calls to Azure's own API that fetch your inventory.
195
+ Nothing is uploaded anywhere else - no SaaS backend, no telemetry, no model API.
196
+ The graph, the diagrams and the raw properties stay on disk, on your machine.
180
197
  - **Existing OSS is siloed** - Terraform-only or Kubernetes-only. cloudmap works
181
198
  from Azure's own inventory (Resource Graph) and correlates across services.
182
199
  - **Impact analysis, onboarding, change reviews.** "What breaks if I touch this?"
@@ -186,8 +203,8 @@ the results. It reads live Azure, so `az login` first.
186
203
 
187
204
  **Is this safe to delete?** An Azure SQL database looks orphaned in the portal and
188
205
  someone wants it gone to save the monthly bill. `cloudmap trace sql-orders-dev
189
- --direction up` deep-enriches the connection strings of the web apps in the
190
- subscription and shows what still points at it - including, occasionally, a
206
+ --live --allow-live --direction up` deep-enriches the connection strings of the
207
+ web apps in scope and shows what still points at it - including, occasionally, a
191
208
  production app that was never supposed to.
192
209
 
193
210
  **What is actually broken?** An AKS cluster starts failing at 3am. Tracing it
@@ -196,33 +213,129 @@ evidence for that edge ("found in Kubernetes secret X"), so the next question -
196
213
  did anything change on that vault - has a place to start.
197
214
 
198
215
  **Who has access to this?** An auditor asks which systems can reach the storage
199
- account holding customer data. `cloudmap trace pii-storage --direction up --csv
200
- pii-audit.csv` hands back a spreadsheet of the web apps and clusters with managed
216
+ account holding customer data. `cloudmap trace pii-storage --live --allow-live
217
+ --direction up --csv pii-audit.csv` hands back a spreadsheet of the apps and clusters with managed
201
218
  identity RBAC on it, with the role assignments as proof.
202
219
 
203
220
  ## What it maps
204
221
 
205
- **Any Azure resource type can be a seed.** The scan is not filtered by type, and
206
- resources are mapped at two levels:
207
-
208
- - **Typed rules** for the services where the relationship has a specific meaning:
209
- App Service / Functions, Container Apps (+ environments), AKS, App Gateway, API
210
- Management, Key Vault, Storage, SQL / PostgreSQL / MySQL / Cosmos, Redis, Service
211
- Bus, Event Hub, Cognitive Search, Azure OpenAI, Container Registry, ML workspaces,
212
- Log Analytics, App Insights, VNets, Private Endpoints and managed identities.
213
- These produce edges like `hosted-on`, `reads-secret`, `pulls-image`, `routes-to`.
214
- - **A generic ARM-reference pass** for everything else: any resolvable resource id
215
- found in a resource's properties becomes a `references` edge, with the property
216
- path as proof. So a type cloudmap has never heard of is still mapped, still
217
- deterministically, still with evidence.
218
-
219
- RBAC edges (`role: Key Vault Secrets User`) are extracted tenant-wide, so
220
- "who has access to this" works for any resource that can be a role scope.
222
+ **Any Azure resource type can be a seed.** The scan is not filtered by type - the
223
+ KQL is `resources | project ...` with no type filter, and the same is true of the
224
+ wizard's picker. What differs between types is not *whether* they are mapped but
225
+ *how precisely*, and that comes from four channels every resource goes through:
226
+
227
+ | Channel | What it finds | Evidence it records |
228
+ |---|---|---|
229
+ | **ARM ids in properties** | any resolvable resource id anywhere in the blob, plus `managedBy` and assigned identities | the property path (`properties.serverFarmId`) |
230
+ | **Advertised hostnames** | free text naming a scanned resource's own endpoint (`kv-x.vault.azure.net`, `acr.azurecr.io`) | the host and where it was found |
231
+ | **Nested child ids** | a child that cannot exist without its parent (a SQL database, a Service Bus queue) | `child-of`, derived from the id's shape |
232
+ | **RBAC role assignments** | which identity holds which role on which scope | `role: <name>`, principal scope |
233
+
234
+ On top of those, ~17 services have **typed rules** that give the edge its real
235
+ meaning instead of a generic `references`: App Service / Functions, Container Apps
236
+ (+ managed environments), AKS, VMs, NICs, Key Vault, Storage, SQL, PostgreSQL,
237
+ MySQL, Cosmos, Redis, Service Bus, Cognitive Search, Cognitive Services / Azure
238
+ OpenAI, Container Registry, ML workspaces, Log Analytics, App Insights and public
239
+ IPs - producing `hosted-on`, `reads-secret`, `pulls-image`, `vnet-integration`,
240
+ `uses-workspace`. A few more rules key off a **property shape** rather than a type
241
+ name, so they fire wherever that shape appears: `privateLinkServiceConnections` →
242
+ `private-link-to` (Private Endpoints), `backendAddressPools[].fqdn` → `routes-to`
243
+ and `gatewayIPConfigurations` → `in-subnet` (Application Gateway), `subnet.id` →
244
+ `in-subnet`.
245
+
246
+ Everything else - API Management, Data Factory, Front Door, a type published last
247
+ week - still lands on the map through the four channels above, as a `references`
248
+ edge carrying the property path that produced it. That is the claim a conformance
249
+ sweep over all **4,687 published Azure resource types** pins
250
+ (`tests/test_type_agnostic.py`): every one of them can be a seed and can be found.
251
+ Deliberately excluded as *sources*: alert rules and dashboards, which reference
252
+ half the estate without depending on it - they emit `observes` instead, so they
253
+ are still visible without turning into hubs.
254
+
255
+ RBAC edges are extracted **tenant-wide by default** (`--single-sub` narrows this to
256
+ the active subscription), so "who has access to this" works for any resource that
257
+ can be a role scope.
221
258
 
222
259
  Depth is honest about itself: apps and clusters have rich outbound edges because
223
260
  their config names other resources. Infrastructure resources are usually leaves
224
261
  going outward, and their value is the reverse view (`--direction up`).
225
262
 
263
+ ## What you will see - and what you won't
264
+
265
+ cloudmap has **no credentials of its own**. Every read is a subprocess call to the
266
+ Azure CLI using your `az login` token, so the map is bounded by your own RBAC. If
267
+ you are about to try this on a tenant and want to know what to expect, this
268
+ section is the whole answer.
269
+
270
+ ### The two ways a resource can be missing
271
+
272
+ **Invisible.** Azure Resource Graph filters by RBAC *server-side*. A resource you
273
+ have no role on is not returned and **no error is raised** - it simply is not in
274
+ the data. cloudmap cannot tell the difference between "does not exist" and "you
275
+ cannot see it", and neither can you from the output alone. This is the failure
276
+ mode to keep in mind: a thin map on a narrow account is not evidence of a small
277
+ blast radius.
278
+
279
+ **Denied.** The deep reads (app settings, Key Vault values, AKS manifests) are
280
+ ordinary API calls that fail loudly when your role is short. cloudmap catches
281
+ each one, classifies it, and writes it on the map as a **read gap** rather than
282
+ dropping the edges it would have produced. You get told.
283
+
284
+ ### What each level of access buys you
285
+
286
+ | To get this on the map | cloudmap runs | You need |
287
+ |---|---|---|
288
+ | The resources themselves, and every role assignment | `az graph query` | **Reader** on the subscriptions in scope. Reader is literally `*/read`, and ARG returns only what that covers. |
289
+ | A workload's identity, VNet integration, runtime | `az webapp show` | Reader |
290
+ | Where a resource sends its diagnostics | `az monitor diagnostic-settings list` | Reader |
291
+ | The RBAC an app's managed identity holds elsewhere | `az role assignment list --assignee --all` | Reader at the scopes you want counted |
292
+ | **Key Vault references and connection strings** (`--enrich`) | `az webapp config appsettings list`, `az webapp config connection-string list` | `Microsoft.Web/sites/config/list/Action`. **Reader is not enough** - this is a POST, not a read. *Website Contributor* or *Contributor* on the app have it. |
293
+ | **What a KV-backed connection string actually points at** (`--resolve-secrets`) | `az keyvault secret show` | Key Vault **data-plane** access: *Key Vault Secrets User* (or an equivalent access policy). Values are used in memory and never printed or written. |
294
+ | **AKS workload edges** - images, configmaps, optionally secrets | `az aks command invoke` | `Microsoft.ContainerService/managedClusters/runCommand/action`. *Azure Kubernetes Service Contributor* has it; *Cluster User* does not. |
295
+
296
+ That is the complete list of `az` commands cloudmap can run. All of them are
297
+ `show` / `list` / `query` verbs; `aks command invoke` is the one asterisk, and it
298
+ is explained under [Live Azure](#live-azure).
299
+
300
+ The practical consequence: **on a plain Reader account, tracing a Key Vault will
301
+ usually come back nearly empty**, because the apps that read it declare that only
302
+ in their app settings, and app settings need more than Reader. The map will say so
303
+ as a read gap - it will not quietly show you an unused vault.
304
+
305
+ ### How the map tells you it is incomplete
306
+
307
+ Three separate signals, all written into the artifact (`--json` `meta`, the HTML
308
+ viewer's `INCOMPLETE ⓘ` badge, and repeated by every `cloudmap ask` answer drawn
309
+ from that map):
310
+
311
+ | Signal | Means | Fix |
312
+ |---|---|---|
313
+ | `truncated` | the Resource Graph scan hit the pagination cap, so the inventory itself is partial | narrow the scope: `--single-sub` |
314
+ | `read_gaps` | a specific deep read failed - permission, network, or the resource refusing | grant the action in the table above, or accept the gap knowingly |
315
+ | `blind_spots` | a whole class of edge was never attempted, because those workloads were not deep-enriched | `--enrich all` |
316
+
317
+ `complete: false` in the JSON is set if any of the three is non-empty. An empty
318
+ answer from an incomplete map is never allowed to read as "nothing depends on
319
+ this".
320
+
321
+ ### My map came back empty or thinner than expected
322
+
323
+ In order:
324
+
325
+ 1. **Did it find the right resource?** Matching is case-insensitive substring; on
326
+ more than one match cloudmap prints each candidate with its resource group and
327
+ full id instead of guessing.
328
+ 2. **Is the scope right?** By default cloudmap scans **every enabled subscription
329
+ in the tenant**. If the terminal reports fewer subscriptions than you expect,
330
+ your account has no role on the rest - they are invisible, not excluded.
331
+ 3. **Read the meta.** `truncated` / `read_gaps` / `blind_spots` name what is
332
+ missing, in that order of severity.
333
+ 4. **Was it the right direction?** Infrastructure (a vault, a subnet, a plan) has
334
+ few outbound edges by nature. `--direction up` is the view that matters for it.
335
+ 5. **Was it enriched?** If the seed is a data service and you did not enrich, the
336
+ dependents that live in app config were never looked for. Re-run with
337
+ `--enrich all`.
338
+
226
339
  ## How it works
227
340
 
228
341
  1. **Ingest** - a JSON fixture (default) or live `az graph query` (opt-in, guarded).
@@ -298,7 +411,10 @@ run the (read-only) `kubectl get` commands. Nothing of yours is modified, but an
298
411
  audit of the cluster's control plane will see that ephemeral pod.
299
412
 
300
413
  Tenant-wide enrichment runs its `az` reads concurrently; `CLOUDMAP_ENRICH_WORKERS`
301
- (default 12) tunes how many at once.
414
+ (default 12) tunes how many at once. Measured on a real subscription: tracing a
415
+ Key Vault's dependents with `--enrich` fanning out over 287 web apps and 2 AKS
416
+ clusters took 6m08s wall-clock at the default concurrency, peaking at ~100 MB
417
+ of memory. Seed-only traces (a web app, a VM) come back in seconds.
302
418
 
303
419
  ```
304
420
  cloudmap trace my-app --live --allow-live
@@ -306,13 +422,8 @@ cloudmap trace my-app --live --allow-live
306
422
 
307
423
  **cloudmap runs as you.** It has no credentials of its own - every read goes through
308
424
  the Azure CLI with your `az login` token, so it sees exactly what your account can
309
- see and nothing more. Resource Graph filters by RBAC, so resources you cannot read
310
- simply do not appear (Azure raises no error for them - they are invisible, not
311
- denied). Deep reads your role does not allow (app settings, Key Vault values, AKS
312
- manifests) fail visibly instead: cloudmap **reports the gap** on the map rather than
313
- silently dropping edges, and warns when a scan is truncated - so a small graph on a
314
- low-privilege account reads as "this is what I was allowed to see", not "this is
315
- everything".
425
+ see and nothing more. What that means in practice, per permission level, is in
426
+ [What you will see - and what you won't](#what-you-will-see---and-what-you-wont).
316
427
 
317
428
  <details>
318
429
  <summary><b>Flags, enrichment and the subscription pin</b></summary>