cloudmap 1.1.0__tar.gz → 1.2.0__tar.gz

cloudmap-1.2.0/CONTRIBUTING.md

@@ -0,0 +1,55 @@
+# Contributing
+
+Thanks for looking. The project is small on purpose; contributions that keep it
+small are the easiest to merge.
+
+## Setup
+
+```
+git clone https://github.com/KatsaounisThanasis/cloudmap && cd cloudmap
+pip install -e ".[dev]"
+pytest && ruff check .
+```
+
+Python 3.9+ (CI runs 3.9 and 3.13). No test touches a real cloud: `subprocess`
+is faked everywhere, and several test files have an autouse fixture that turns
+any real subprocess call into a failure - keep it that way.
+
+## The rules that are not up for debate
+
+These are the project's identity; PRs that violate them will be declined even
+if the feature is useful:
+
+1. **Read-only.** No `az` command that mutates anything.
+2. **Local-first.** No telemetry, no upload, no outbound call except the
+   optional user-configured local model endpoint.
+3. **Every edge carries evidence.** An edge without a provable source
+   (property path, config reference, RBAC assignment) does not go on the map.
+4. **A model proposes, code verifies.** LLM output may only suggest candidates
+   that a deterministic check confirms against scanned resources. A guess that
+   cannot be verified is dropped, not shown.
+5. **Incompleteness is declared.** If a scan was truncated or a read failed,
+   the artifact says so. Never let an empty result pass for "nothing depends
+   on this".
+6. **Nothing sensitive in the repo.** Fixtures are synthetic or scrubbed;
+   `tests/test_fixtures_safe.py` enforces it in CI.
+
+## Practical notes
+
+- Match the existing style: docstrings explain *why*, tests are named as
+  behaviour claims (`test_a_truncated_capture_stays_truncated_when_retraced`).
+- A bug fix needs a regression test that fails without the fix.
+- New resource-type support usually means: a Resolver index entry, an
+  extractor rule (or nothing, if the generic ARM-reference pass covers it),
+  a FRIENDLY name, and a fixture-based test.
+- Run `ruff check .` before pushing - CI treats lint as a failure.
+
+## Reporting bugs
+
+Open an issue with the command you ran and the output. If the map itself is
+wrong (missing or bogus edge), the perfect report includes a minimal synthetic
+fixture that reproduces it - see `fixtures/contoso.json` for the shape. Never
+paste a real capture; scrub it first (`cloudmap scrub`) and read it before
+posting.
+
+Security issues: see [SECURITY.md](SECURITY.md) - do not open a public issue.

{cloudmap-1.1.0 → cloudmap-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: cloudmap
-Version: 1.1.0
+Version: 1.2.0
 Summary: Trace the blast radius of an Azure resource: one name in, a verified dependency graph out.
 Project-URL: Homepage, https://github.com/KatsaounisThanasis/cloudmap
 Project-URL: Repository, https://github.com/KatsaounisThanasis/cloudmap

cloudmap-1.2.0/SECURITY.md

@@ -0,0 +1,46 @@
+# Security
+
+cloudmap reads cloud infrastructure and handles the output, so security reports
+get priority over everything else.
+
+## Reporting a vulnerability
+
+Use [GitHub private vulnerability reporting](https://github.com/KatsaounisThanasis/cloudmap/security/advisories/new)
+(Security tab → Report a vulnerability). Please do not open a public issue for
+anything that could expose a user's infrastructure data before a fix exists.
+
+You can expect an acknowledgement within a few days. There is no bounty - this
+is a solo open-source project - but reports are credited in the release notes
+unless you ask otherwise.
+
+## What counts as a vulnerability here
+
+The interesting failure modes for a tool like this:
+
+- **Credential leakage into artifacts**: any way a secret, key, token or
+  password ends up in a `.drawio` / `.json` / `.html` / `.csv` / capture file.
+  The scrubber (`cloudmap/scrub.py`) redacts credentials and pseudonymises
+  identifiers; a value that survives it is a bug of the highest priority -
+  one shipped before (base64 padding, fixed in `2b1e06d`) and the regression
+  test suite grew from it.
+- **Scope escalation**: cloudmap must only ever read what the caller's own
+  `az login` token can read, and must never perform a write operation against
+  the tenant.
+- **Injection through cloud-controlled data**: resource names, tags and
+  properties are attacker-influenceable in shared tenants; anything that lets
+  them break out of an `az` argument list, the HTML viewer, or the draw.io XML
+  is in scope.
+
+## Design promises you can hold us to
+
+- Read-only: no `az` mutation commands, ever.
+- Local-first: the only outbound network call in the codebase is to a
+  user-configured local model endpoint (`cloudmap/local_model.py`), off by
+  default. Resource data never leaves the machine.
+- Secrets stay in memory: `--resolve-secrets` substitutes Key Vault values
+  in-memory for edge extraction and never prints or writes them.
+- `tests/test_fixtures_safe.py` fails CI if a committed fixture ever carries
+  a credential or a real-looking identifier.
+
+If you find code contradicting any of these, that is a valid report even if
+you cannot demonstrate an exploit.

{cloudmap-1.1.0 → cloudmap-1.2.0}/cloudmap/__init__.py

@@ -1,3 +1,3 @@
 """cloudmap - trace an Azure resource's full dependency graph and export it."""
 
-__version__ = "1.1.0"
+__version__ = "1.2.0"

{cloudmap-1.1.0 → cloudmap-1.2.0}/cloudmap/adapters/__init__.py

@@ -88,4 +88,10 @@ def load_graph(path):
         data = json.load(f)
     if _looks_neutral(data):
         return graph_from_neutral(data)
-    return AzureAdapter.to_graph(data)
+    graph = AzureAdapter.to_graph(data)
+    # A capture's own honesty flags (truncated, enriched, scrubbed) ride along:
+    # re-tracing a truncated export must not produce a map that claims to be
+    # complete just because the file was re-read from disk.
+    if isinstance(data, dict) and isinstance(data.get("meta"), dict):
+        graph.meta.update(data["meta"])
+    return graph

{cloudmap-1.1.0 → cloudmap-1.2.0}/cloudmap/cli.py

@@ -15,6 +15,14 @@ from .render.mermaid import to_mermaid
 def main(argv=None):
     # Interactive mode if no arguments are provided
     if (argv is None and len(sys.argv) == 1) or (argv is not None and len(argv) == 0):
+        # The wizard hands the terminal to questionary; in a pipe / cron / CI
+        # there is no terminal to hand over, so fail with a pointer instead of
+        # letting the prompt library misbehave against a non-tty stream.
+        if not (sys.stdin.isatty() and sys.stdout.isatty()):
+            print("cloudmap: running with no arguments starts the interactive wizard, "
+                  "which needs a terminal. Use `cloudmap --help` for the scriptable "
+                  "commands.", file=sys.stderr)
+            return 2
         from .interactive import interactive_main
         return interactive_main()
         
@@ -114,8 +122,14 @@ def _cmd_trace(args):
         graph = build_graph(resources)
     else:
         from .adapters import load_graph
-        resources, truncated = [], False
+        resources = []
         graph = load_graph(args.fixture)   # auto-detects raw export vs neutral cloudmap graph
+        # A re-traced capture keeps its own honesty flags: a scan that was
+        # truncated at capture time must not become a map that claims
+        # completeness just because it was re-read from disk.
+        truncated = bool(graph.meta.get("truncated", False))
+        read_gaps = list(graph.meta.get("read_gaps") or [])
+        blind_spots = list(graph.meta.get("blind_spots") or [])
 
     seeds = find_seeds(graph, args.name)
     if not seeds:
@@ -281,6 +295,13 @@ def _enrich_live(args, graph, seed, resources):
             print("Read gaps while enriching (edges below may be INCOMPLETE):", file=sys.stderr)
             for msg in read_gaps:
                 print(f"  ! could not read {msg}", file=sys.stderr)
+            # The artifact is meant to be shared; az stderr carries correlation
+            # ids and GUIDs. The terminal (above) keeps the full text, the map
+            # records a short classified reason.
+            from .ingest.azure import classify_gap
+            read_gaps = [f"{m.partition(': ')[0]}: {classify_gap(m.partition(': ')[2])}"
+                         if ": " in m else classify_gap(m)
+                         for m in read_gaps]
                 
         graph = build_graph(resources)            # re-extract, now that config is present
         resolver = Resolver(graph.nodes)

{cloudmap-1.1.0 → cloudmap-1.2.0}/cloudmap/extract/extractors.py

@@ -183,6 +183,7 @@ class Resolver:
         self.storage_by_name = {}
         self.law_by_customer_id = {}   # Log Analytics workspace GUID -> node_id
         self.by_name = {}          # resource name -> node_id (for verifying LLM proposals)
+        self.sqldb_by_name = {}    # db name -> [(parent server id lower, db node_id)]
         for n in nodes.values():
             self.by_id[n.id.lower()] = n.id
             if n.name:
@@ -213,6 +214,11 @@ class Resolver:
             self.kv_by_name[name] = n.id
         elif t == "microsoft.sql/servers":
             self._add_host(p.get("fullyQualifiedDomainName") or f"{name}.database.windows.net", n.id)
+        elif t == "microsoft.sql/servers/databases":
+            # a connection string names the SERVER as host and the database by
+            # name; keep the pair so config edges can land on the database itself
+            self.sqldb_by_name.setdefault(name, []).append(
+                (n.id.lower().rsplit("/", 2)[0], n.id))
         elif t.startswith("microsoft.dbforpostgresql/"):
             self._add_host(p.get("fullyQualifiedDomainName") or f"{name}.postgres.database.azure.com", n.id)
         elif t.startswith("microsoft.dbformysql/"):
@@ -410,6 +416,19 @@ def extract_edges(nodes):
         elif t == "microsoft.containerservice/managedclusters":
             _aks_config_edges(n, p, r, add)
 
+    # Nested-child pass. An id like .../servers/sqlsrv/databases/orders is a
+    # child resource: it cannot exist without its parent, so a scanned parent
+    # gets a child-of edge. Without this a nested seed (a SQL database, a
+    # Service Bus queue) is an island - no property on either side names the
+    # other, the relationship exists only in the id's shape.
+    for n in nodes.values():
+        if n.type == "microsoft.authorization/roleassignments":
+            continue
+        if len(n.id.split("/")) > 9:                       # provider path + child segments
+            parent = r.by_resource_id(n.id.rsplit("/", 2)[0])
+            if parent and parent != n.id:
+                add(n.id, parent, "child-of", "nested ARM resource id")
+
     # Generic ARM-reference pass. Any resolvable resource id sitting in a
     # resource's properties is a real dependency, whatever the type - this is
     # what lets cloudmap map resource types it has no hand-written rule for,
@@ -462,13 +481,27 @@ def _config_edges(n, values, r, add, label="app config"):
     # O(blob) extraction instead of O(hosts * blob)
     import re
     words = set(re.findall(r"[a-z0-9.-]+", blob))
-    
+
+    hit_hosts = {}                # resolved node id (lower) -> the host that named it
     for word in words:
         if word in r.by_host:
+            hit_hosts[r.by_host[word].lower()] = word
             add(n.id, r.by_host[word], domain_kind(word)[0], f"{label} references host {word}")
         if word in r.by_ik:
             add(n.id, r.by_ik[word], "sends-telemetry", f"{label} contains instrumentation key")
 
+    # A SQL connection string names the server as host and the database by name.
+    # When that database was scanned, land the edge on the database itself (in
+    # addition to the server) - it is what a database seed's upward view needs.
+    # The parent-server check keeps a same-named database under an unreferenced
+    # server from matching.
+    for m in re.finditer(r"(?:database|initial catalog)\s*=\s*([a-z0-9._-]+)", blob):
+        for srv_id, db_id in r.sqldb_by_name.get(m.group(1), []):
+            if srv_id in hit_hosts:
+                add(n.id, db_id, "connects-to",
+                    f"{label} connection string names database {m.group(1)} "
+                    f"on referenced server host {hit_hosts[srv_id]}")
+
     # Storage accounts are often in connection strings: accountname=xyz
     for m in re.finditer(r"accountname=([a-z0-9-]+)", blob):
         acct = m.group(1)

{cloudmap-1.1.0 → cloudmap-1.2.0}/cloudmap/ingest/azure.py

@@ -44,6 +44,41 @@ def _az(args):
     return out.stdout
 
 
+_GUID_RE = re.compile(
+    r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}")
+
+# Matched against the lowercased failure text, first hit wins.
+_GAP_CATEGORIES = (
+    ("authorizationfailed", "authorization denied (RBAC)"),
+    ("forbidden", "authorization denied (RBAC)"),
+    ("denyassignment", "blocked by a deny assignment"),
+    ("invalidaadclustertoken", "cluster token rejected"),
+    ("aadsts", "Entra token rejected"),
+    ("was not found", "not found"),
+    ("notfound", "not found"),
+    ("timed out", "timeout"),
+    ("failed to resolve", "network (DNS)"),
+    ("connection refused", "network"),
+)
+
+
+def classify_gap(message):
+    """Short, artifact-safe reason for a failed read.
+
+    Full az stderr carries correlation ids, subscription GUIDs and sometimes
+    principal ids - useful on the operator's own terminal, wrong inside a
+    JSON/HTML artifact meant to be shared. The terminal keeps the full text;
+    the artifact records the category this returns. Unknown failures fall back
+    to the first line with every GUID masked, so nothing identifying rides
+    along even when the category list has never seen the error."""
+    low = str(message).lower()
+    for needle, label in _GAP_CATEGORIES:
+        if needle in low:
+            return label
+    first_line = str(message).strip().splitlines()[0] if str(message).strip() else ""
+    return _GUID_RE.sub("<id>", first_line)[:120] or "read failed"
+
+
 def _guard():
     try:
         acct = json.loads(_az(["account", "show", "-o", "json"]))

{cloudmap-1.1.0 → cloudmap-1.2.0}/tests/test_arg_rows.py

@@ -164,12 +164,11 @@ def test_a_neutral_graph_json_is_reloaded_without_re_extracting(tmp_path):
     assert graph.meta["seed"] == "/web"          # a reloaded map keeps its provenance
 
 
-@pytest.mark.xfail(strict=True, reason="BUG: a microsoft.sql/servers/databases seed has an "
-                                       "empty blast radius - no child-to-parent edge links a "
-                                       "database to its server, and consumers' connection "
-                                       "strings resolve to the server, so the wizard offers a "
-                                       "seed type that can only ever produce an empty map")
 def test_a_sql_database_seed_reaches_its_server_and_its_consumer():
+    # Regression for a real bug: a database seed used to produce an empty map -
+    # no child-of edge linked it to its server, and consumers' connection strings
+    # resolved only to the server. Fixed by the nested-child pass + landing
+    # config edges on the scanned database itself.
     server = f"{S}/microsoft.sql/servers/sqlsrv"
     database = f"{server}/databases/orders"
     graph = build_graph([

cloudmap-1.2.0/tests/test_azure.py

@@ -0,0 +1,45 @@
+import pytest
+
+from cloudmap.ingest import azure
+
+
+def test_resolve_secret_handles_failures(monkeypatch):
+    def fake_az(*args, **kwargs):
+        raise RuntimeError("az command failed")
+    monkeypatch.setattr(azure, "_az", fake_az)
+    
+    assert azure._resolve_secret("@Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/)") == ""
+
+def test_guard_raises_system_exit_on_mismatch(monkeypatch):
+    monkeypatch.setenv("CLOUDMAP_ALLOW_SUBSCRIPTION", "sub-2")
+    def fake_az(*args, **kwargs):
+        return '{"id": "sub-1", "tenantId": "t-1"}'
+    monkeypatch.setattr(azure, "_az", fake_az)
+    
+    with pytest.raises(SystemExit, match="Refusing the mismatch"):
+        azure._guard()
+
+
+def test_gap_classification_names_the_category_not_the_incident():
+    cases = {
+        "az webapp show failed: AuthorizationFailed: client 'x' with object id "
+        "'11111111-2222-3333-4444-555555555555' does not have authorization":
+            "authorization denied (RBAC)",
+        "ERROR: (ResourceNotFound) the resource was not found": "not found",
+        "az aks command invoke timed out after 120s": "timeout",
+        "AADSTS50076: MFA required": "Entra token rejected",
+        "Failed to resolve 'management.azure.com'": "network (DNS)",
+    }
+    for raw, expected in cases.items():
+        assert azure.classify_gap(raw) == expected, raw
+
+
+def test_an_unknown_gap_is_kept_but_every_guid_is_masked():
+    short = azure.classify_gap(
+        "some novel failure, correlation id 12345678-abcd-ef01-2345-6789abcdef01\n"
+        "second line with more detail")
+
+    assert "12345678-abcd" not in short          # nothing identifying rides along
+    assert "<id>" in short
+    assert "second line" not in short            # first line only
+    assert len(short) <= 120

{cloudmap-1.1.0 → cloudmap-1.2.0}/tests/test_cli_exports.py

@@ -225,3 +225,32 @@ def test_no_subcommand_is_rejected_rather_than_falling_into_the_wizard():
     # interactive session in a non-interactive context (CI, pipes).
     with pytest.raises(SystemExit):
         cli.main(["--nonsense"])
+
+
+def test_a_truncated_capture_stays_truncated_when_retraced(tmp_path):
+    # The capture said "I did not see everything". Re-tracing that file offline
+    # must not launder the warning into complete:true.
+    capture = tmp_path / "capture.json"
+    capture.write_text(json.dumps({
+        "data": [{"id": "/subscriptions/s/resourceGroups/rg/providers/Microsoft.Web/sites/web",
+                  "name": "web", "type": "microsoft.web/sites", "properties": {}}],
+        "meta": {"truncated": True},
+        "scrubbed": True,
+    }), encoding="utf-8")
+
+    out = tmp_path / "map.json"
+    rc = cli.main(["trace", "web", "--from", str(capture), "--json", str(out)])
+    meta = json.loads(out.read_text())["meta"]
+
+    assert rc == 0
+    assert meta["truncated"] is True
+    assert meta["complete"] is False
+
+
+def test_bare_cloudmap_without_a_terminal_refuses_instead_of_prompting(monkeypatch, capsys):
+    monkeypatch.setattr("sys.stdin", type("S", (), {"isatty": staticmethod(lambda: False)})())
+
+    rc = cli.main([])
+
+    assert rc == 2
+    assert "terminal" in capsys.readouterr().err

cloudmap-1.2.0/tests/test_render_text.py

@@ -0,0 +1,70 @@
+"""The two text renderers (Mermaid, CSV).
+
+Mermaid has exactly two escape rules - double quotes in labels and pipes in edge
+kinds break its syntax - and the CSV is read by spreadsheets and auditors, so it
+must round-trip through a real csv.reader with hostile evidence text intact.
+"""
+
+import csv
+import io
+
+from cloudmap.model import Edge, Graph, Node
+from cloudmap.render.csv_export import to_csv
+from cloudmap.render.mermaid import to_mermaid
+
+
+def _graph():
+    web = Node(id="/web", name='app "prod"', type="microsoft.web/sites")
+    kv = Node(id="/kv", name="kv-1", type="microsoft.keyvault/vaults")
+    ext = Node(id="/ext", name="unknown.example.com", type="external", external=True)
+    return Graph(
+        nodes={n.id: n for n in (web, kv, ext)},
+        edges=[
+            Edge("/web", "/kv", "reads-secret; role: KV | Secrets User",
+                 evidence='setting "CONN", value contains , and\nnewline'),
+            Edge("/web", "/ext", "connects-to", origin="model"),
+        ],
+        distances={"/web": 0, "/kv": 1, "/ext": 1},
+    )
+
+
+# --- mermaid ----------------------------------------------------------------------
+
+def test_mermaid_escapes_the_only_two_things_that_break_it():
+    text = to_mermaid(_graph(), "/web")
+
+    assert '"' not in text.split("\n")[1].split("[")[1].replace('["', "").replace('"]', "") \
+        or "app 'prod'" in text                      # quotes in names become single quotes
+    assert "KV / Secrets User" in text               # pipes in kinds become slashes
+    assert "-->|" in text                            # verified edge, solid
+    assert '-. "connects-to (model)" .->' in text    # model edge, dashed and labelled
+
+
+def test_mermaid_marks_seed_and_external_nodes():
+    text = to_mermaid(_graph(), "/web")
+
+    assert "style N0" in text                        # seed styled
+    assert "stroke-dasharray" in text                # external node dashed
+
+
+# --- csv --------------------------------------------------------------------------
+
+def test_csv_round_trips_through_a_real_reader():
+    rows = list(csv.reader(io.StringIO(to_csv(_graph(), "/web"))))
+    header, body = rows[0], rows[1:]
+
+    assert header[0] == "Source Name" and "Evidence" in header
+    assert len(body) == 2
+    by_target = {r[3]: r for r in body}
+    assert by_target["kv-1"][5] == "Verified"
+    assert by_target["kv-1"][6] == 'setting "CONN", value contains , and\nnewline'
+    assert by_target["unknown.example.com"][5] == "GUESS (LLM) (Unverified Target)"
+
+
+def test_csv_falls_back_to_the_id_tail_for_unscanned_targets():
+    g = _graph()
+    g.edges.append(Edge("/web", "/subscriptions/s/x/gone-resource", "references"))
+
+    rows = list(csv.reader(io.StringIO(to_csv(g, "/web"))))
+
+    assert any(r[3] == "gone-resource" and "Unverified Target" in r[5] for r in rows[1:])

cloudmap-1.1.0/tests/test_azure.py

@@ -1,20 +0,0 @@
-import pytest
-
-from cloudmap.ingest import azure
-
-
-def test_resolve_secret_handles_failures(monkeypatch):
-    def fake_az(*args, **kwargs):
-        raise RuntimeError("az command failed")
-    monkeypatch.setattr(azure, "_az", fake_az)
-    
-    assert azure._resolve_secret("@Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/)") == ""
-
-def test_guard_raises_system_exit_on_mismatch(monkeypatch):
-    monkeypatch.setenv("CLOUDMAP_ALLOW_SUBSCRIPTION", "sub-2")
-    def fake_az(*args, **kwargs):
-        return '{"id": "sub-1", "tenantId": "t-1"}'
-    monkeypatch.setattr(azure, "_az", fake_az)
-    
-    with pytest.raises(SystemExit, match="Refusing the mismatch"):
-        azure._guard()