heimdall-code 0.1.1__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/.claude-plugin/plugin.json +1 -1
  2. heimdall_code-0.3.0/CHANGELOG.md +129 -0
  3. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/CONTRIBUTING.md +21 -3
  4. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/PKG-INFO +89 -19
  5. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/README.md +88 -18
  6. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/QUESTIONS.md +11 -11
  7. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/README.md +98 -2
  8. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/RESULTS.md +20 -16
  9. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/compare.py +61 -7
  10. heimdall_code-0.3.0/bench/kubernetes.py +295 -0
  11. heimdall_code-0.3.0/bench/terraform.py +187 -0
  12. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/__init__.py +1 -1
  13. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/graph.py +1 -1
  14. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/index.py +83 -8
  15. heimdall_code-0.3.0/heimdall/parse.py +871 -0
  16. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/pyproject.toml +1 -1
  17. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/skills/heimdall/SKILL.md +1 -1
  18. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/tests/test_heimdall.py +192 -3
  19. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/uv.lock +1 -1
  20. heimdall_code-0.1.1/CHANGELOG.md +0 -54
  21. heimdall_code-0.1.1/heimdall/parse.py +0 -409
  22. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/.claude-plugin/marketplace.json +0 -0
  23. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/.github/workflows/release.yml +0 -0
  24. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/.gitignore +0 -0
  25. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/.mcp.json +0 -0
  26. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/CLAUDE.md +0 -0
  27. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/LICENSE +0 -0
  28. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/.gitignore +0 -0
  29. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/dump_lbug.mjs +0 -0
  30. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/bench/questions.py +0 -0
  31. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/__main__.py +0 -0
  32. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/cli.py +0 -0
  33. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/install.py +0 -0
  34. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/mcp_server.py +0 -0
  35. {heimdall_code-0.1.1 → heimdall_code-0.3.0}/heimdall/registry.py +0 -0
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "heimdall",
3
- "version": "0.1.1",
3
+ "version": "0.3.0",
4
4
  "description": "A code knowledge graph that sees everything. Indexes any repo into a SQLite property graph and answers what calls this, what breaks if I change it, and how does A reach B.",
5
5
  "author": {
6
6
  "name": "Bogdan Sapovskyi",
@@ -0,0 +1,129 @@
1
+ # Changelog
2
+
3
+ Notable changes, newest first. Versions follow [semver](https://semver.org); until 1.0
4
+ the graph schema and the JSON payloads may still move.
5
+
6
+ ## 0.3.0 — 2026-08-01
7
+
8
+ ### Added
9
+
10
+ - **Kubernetes manifests**, as the 20th language and the second one the generic walker
11
+ could not reach. A document with `apiVersion`, `kind` and `metadata.name` is indexed
12
+ under the address `kubectl` uses for it — `Deployment/api` — and every field holding
13
+ the name of another document becomes a `CALLS` edge. The field decides the kind: a name
14
+ under `configMapRef.name` can only be a ConfigMap, and where the field does not say
15
+ (`roleRef`, `scaleTargetRef`) the sibling `kind` does. So `impact ConfigMap/app-config`
16
+ now answers with the workloads that would restart.
17
+ - **Label selectors are edges too**, which is the one Kubernetes reference not written as
18
+ a name: a Service's selector is matched against the labels a workload puts on its pods,
19
+ within one namespace, so a prod Service cannot end up wired to staging pods. Scope
20
+ settles which copy was meant — a directory of manifests is a scope the way a Terraform
21
+ module is, and `kustomization.yaml`'s `resources` say which other directories join it.
22
+ - `bench/kubernetes.py`, graded by an oracle that re-reads the manifests by indentation
23
+ and shares no code with the tree-sitter reader. Across four public repos — 1,988 files,
24
+ 2,748 documents — every reference by name and by label resolves to the right document
25
+ and none of the edges is wrong. The one reference with no edge is `##CRBNAME##`, a sed
26
+ placeholder in an argo-cd test fixture, which is not a name.
27
+ - Yaml that is not a manifest is a file in the graph and nothing more. An unrendered Helm
28
+ chart is skipped rather than guessed at: `{{ .Release.Name }}-api` is not a name.
29
+
30
+ ### Fixed
31
+
32
+ - A yaml file is now parsed one document at a time, split on YAML's own `---` before the
33
+ grammar sees it. A single document the grammar could not follow used to take every
34
+ document after it down with it — in argo-cd's 1.9 MB rendered `install.yaml` that was
35
+ 82 of 130 objects, left out of the graph with nothing to say they were missing.
36
+
37
+ ## 0.2.0 — 2026-08-01
38
+
39
+ ### Added
40
+
41
+ - **Constants are symbols.** A `SCREAMING_SNAKE` module- or class-level binding is indexed
42
+ as a new node kind, `Constant`, and reading one is an edge. Endpoint tables, thresholds
43
+ and feature flags were previously not in the graph at all, so `impact SOME_THRESHOLD`
44
+ answered nothing and the question had to go back to `rg`.
45
+ - **A function handed over rather than called** is an edge too — `register(on_save)`,
46
+ `key=parse`, `@decorator`, a class named as a receiver. On Django the oracle finds
47
+ 12,000 such sites; recall went from 8.4% / 4.8% (local / imported) to 97.8% / 96.9%,
48
+ where GitNexus scores 1.9% / 0.1% and graphify 13.8% / 5.3%. Constant reads score
49
+ 100.0% / 100.0%; neither other tool models them at all.
50
+ - Both are `CALLS` edges, on the same argument the decorator edge already made: they break
51
+ the same way when the target changes. They differ from a call in the fallback — an
52
+ unresolvable *call* is emitted as up to five `0.3` candidates, an unresolvable
53
+ *reference* is dropped, because a bare name with no evidence behind it is almost always
54
+ a local variable. Django's `0.3` edge count came out *below* what it was before the
55
+ feature, and the referee still refutes 0 of 204,713 checkable edges.
56
+ - The cost: Django indexes in 2.7 s instead of 1.7 s and the graph is 78 MB instead of 48.
57
+ `bench/RESULTS.md`, `bench/QUESTIONS.md` and the README tables are regenerated against
58
+ all three tools rather than edited.
59
+
60
+ - **Terraform**, as the 19th language and the first one the generic walker could not
61
+ reach: `.tf` and `.hcl` are read by `_parse_hcl`, which indexes a block under the
62
+ address the rest of the configuration names it by (`resource "aws_instance" "web"` ->
63
+ `aws_instance.web`, `variable "region"` -> `var.region`) and turns every reference to
64
+ one into a `CALLS` edge. Scoping is Terraform's: a directory is a module, and a `module`
65
+ block's arguments resolve to the `variable` blocks in the directory its `source` names.
66
+ - `bench/terraform.py`, graded by a regex oracle that shares no code with the reader.
67
+ Across three public repos — 432 HCL files, 5,104 blocks, 10,206 edges — one reference
68
+ of 2,828 goes unresolved and none of the edges is wrong.
69
+
70
+ ### Fixed
71
+
72
+ - A name being *introduced* is no longer read as a reference to something else: a
73
+ parameter, a `*args` splat, a loop variable, a destructuring pattern and a keyword
74
+ argument's name all bind rather than read. Found by the referee, which refuted exactly
75
+ one edge — `def handle(self, *labels)` pointing at an unrelated `labels` method.
76
+ - A relative import with more than one `..` (`from '../../store'`) resolved to nothing:
77
+ the dots after the first were read as module separators, which is right for
78
+ `..pkg.mod` and wrong for a path. Every such import in every JS/TS repo was silently
79
+ missing its edge.
80
+
81
+ ## 0.1.1 — 2026-08-01
82
+
83
+ - `heimdall --version`.
84
+ - Works on Python 3.12 again: `os.process_cpu_count()` is 3.13+, so every index crashed
85
+ on the version the package declares as its floor. The release workflow now runs the
86
+ suite on 3.12 rather than on whatever the runner image ships.
87
+ - `install` no longer treats a heimdall in the default uv cache as durable when
88
+ `UV_CACHE_DIR` points elsewhere — that config would have rotted at the next `uv cache
89
+ prune`.
90
+
91
+ ## 0.1.0 — 2026-08-01
92
+
93
+ First release.
94
+
95
+ ### Added
96
+
97
+ - **Index.** `heimdall index` builds `.heimdall/graph.db` in four passes — discover
98
+ (`git ls-files`, submodules included), parse (tree-sitter, one process per core),
99
+ resolve, FTS5. 1.7 s and 48 MB for Django's 2,927 files.
100
+ - **Query.** `search`, `context`, `impact`, `trace`, `stats`, `projects` and a read-only
101
+ `sql` escape hatch over a two-table schema. Every command takes `--json` and
102
+ `-p/--project`.
103
+ - **18 languages**, declared as data in `parse.py` and each covered by a test that fails
104
+ if a grammar renames a node type.
105
+ - **MCP server** (`heimdall mcp`): one process serves every repo in
106
+ `~/.heimdall/registry.json`, eight tools returning assembled answers rather than raw
107
+ edges.
108
+ - **`heimdall install [claude|codex|all]`** — writes `~/.claude.json`, `./.mcp.json`,
109
+ `~/.codex/config.toml` or `./.codex/config.toml`, merging around your existing
110
+ settings; `--cli` installs the skill instead of the server.
111
+ - **Claude Code plugin**: `/plugin marketplace add stPhoenix/heimdall`.
112
+ - **Benchmark** (`bench/`) against GitNexus and graphify, graded by a CPython `ast`
113
+ oracle.
114
+
115
+ ### Resolution fixes found by the benchmark
116
+
117
+ Each of these was a wrong answer before it was a test:
118
+
119
+ - Imports resolve per scope, so a function with its own `from x import y` is no longer
120
+ matched against the whole file.
121
+ - `pkg/main.py` is reachable as `pkg.main`, not only as `pkg`.
122
+ - `as` renames are followed home — but only for in-repo targets, so
123
+ `from asyncio import run as arun` cannot bind to your own `run`.
124
+ - A decorator is a call by the function it wraps, not by the enclosing class. Worth 26%
125
+ of imported-call recall on Django.
126
+ - Recursion produces an edge.
127
+ - A class name used as a receiver resolves inside that class — including when several
128
+ classes share the name, because the one that defines the member is the one that was
129
+ meant. Took *who calls `Model.check`* from 0% to 94% precision.
@@ -18,7 +18,7 @@ There is no CI. The gate is `uv run pytest && uv run ruff check .` before you pu
18
18
 
19
19
  | | |
20
20
  |---|---|
21
- | `heimdall/parse.py` | tree-sitter → symbols, calls, imports, heritage. Language support is **data** (`EXT_LANG`, `DEFS`, `CALL_NODES`, `IMPORT_NODES`, `HERITAGE_*`), walked by one generic function. |
21
+ | `heimdall/parse.py` | tree-sitter → symbols, calls, imports, heritage. Language support is **data** (`EXT_LANG`, `DEFS`, `CALL_NODES`, `IMPORT_NODES`, `CONST_NODES`, `HERITAGE_*`), walked by one generic function — plus `_parse_hcl` and `_parse_yaml`, the two readers that could not be rows. |
22
22
  | `heimdall/index.py` | the four passes: discover → parse (process pool) → resolve → store. Everything a query needs is precomputed here. |
23
23
  | `heimdall/graph.py` | schema + reads. Two tables, recursive CTEs, FTS5. `query()` is the read-only SQL gate. |
24
24
  | `heimdall/cli.py` | argparse; every command takes `--json` and `-p/--project`. |
@@ -29,11 +29,20 @@ There is no CI. The gate is `uv run pytest && uv run ruff check .` before you pu
29
29
 
30
30
  ## Adding a language
31
31
 
32
- Rows in three dicts in `parse.py` — definition node types, call node types, import node
33
- types — plus a snippet in `SNIPPETS` in `tests/test_heimdall.py`. No new module. The test
32
+ Rows in four dicts in `parse.py` — definition node types, call node types, import node
33
+ types, and the node types that bind a name (which is how constants are found) — plus a
34
+ snippet in `SNIPPETS` in `tests/test_heimdall.py`. No new module. The test
34
35
  asserts the snippet still yields symbols *and* call edges, so a grammar renaming a node
35
36
  type fails the build instead of quietly emptying the graph.
36
37
 
38
+ HCL and YAML are the two that did not fit, and they are worth reading before deciding
39
+ yours does not either: neither has definition nodes or call nodes, so no set of rows could
40
+ express them. Each gets a reader beside the walker rather than a branch inside it —
41
+ `_parse_hcl` with its three tables of Terraform addressing rules, `_parse_yaml` with its
42
+ table of Kubernetes field paths — and the walker stays untouched. That is the shape to
43
+ copy if a language genuinely needs it, and the bar is "the grammar has nothing the walker
44
+ is looking for", not "the walker gets a few nodes wrong".
45
+
37
46
  ## Tests
38
47
 
39
48
  One file, `tests/test_heimdall.py`, runnable with or without pytest. Every resolution bug
@@ -48,6 +57,15 @@ resolution bug, add the smallest fixture that would have caught it.
48
57
  (it wants Node, npm and a clone of Django), but it is what a change to resolution should
49
58
  be measured with: `python bench/compare.py /path/to/django` and `python bench/questions.py`.
50
59
 
60
+ `bench/terraform.py` does the same for HCL against a regex oracle, and
61
+ `bench/kubernetes.py` for manifests against an oracle that re-reads them by indentation.
62
+ No other tool to compare against, so both only grade Heimdall's own edges:
63
+
64
+ ```console
65
+ python bench/terraform.py /path/to/some-terraform-repo
66
+ python bench/kubernetes.py /path/to/some-manifest-repo
67
+ ```
68
+
51
69
  ## Release
52
70
 
53
71
  Bump `version` in `pyproject.toml`, `__version__` in `heimdall/__init__.py` and `version`
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: heimdall-code
3
- Version: 0.1.1
3
+ Version: 0.3.0
4
4
  Summary: A code knowledge graph that sees everything -- indexes any repository into a SQLite property graph and serves it to AI agents over MCP.
5
5
  Project-URL: Homepage, https://github.com/stPhoenix/heimdall
6
6
  Project-URL: Source, https://github.com/stPhoenix/heimdall
@@ -32,14 +32,14 @@ Apache-2.0. No server, no cloud, no embedded graph engine. **The graph is a SQLi
32
32
  ```console
33
33
  $ heimdall index ~/code/django
34
34
  pass 1/4 discovering files
35
- 2974 source files
35
+ 2998 source files
36
36
  pass 2/4 parsing (24 workers)
37
37
  pass 3/4 building graph
38
38
  pass 4/4 building search index
39
- done in 1.49s # 43,482 symbols, 353,101 edges
39
+ done in 2.35s # 45,515 symbols, 418,951 edges
40
40
 
41
41
  $ heimdall impact django/shortcuts.py:get_object_or_404 -d 2 -c 0.7
42
- impact of django/shortcuts.py:get_object_or_404: 6 symbols, 4 files
42
+ impact of django/shortcuts.py:get_object_or_404: 10 symbols, 8 files
43
43
 
44
44
  depth 1 (5):
45
45
  Function django/contrib/flatpages/views.py:flatpage (0.80)
@@ -53,7 +53,7 @@ path of length 2:
53
53
  -> django/http/response.py:HttpResponse
54
54
  ```
55
55
 
56
- A second and a half, cold, for the whole of Django on a laptop.
56
+ Two and a half seconds, cold, for the whole of Django on a laptop.
57
57
 
58
58
  ## Why
59
59
 
@@ -89,16 +89,18 @@ against [graphify](https://pypi.org/project/graphifyy/) in its AST-only mode:
89
89
  |---|---|---|---|
90
90
  | `self.m()` bound correctly | 100.0% | 97.1% | 92.6% |
91
91
  | same-file call bound correctly | 100.0% | 93.4% | 89.3% |
92
- | imported call bound correctly | 99.7% | 71.5% | 75.4% |
92
+ | imported call bound correctly | 99.7% | 71.4% | 75.3% |
93
93
  | class instantiation, same file / imported | 99.5% / 98.8% | not modelled | 95.2% / 98.2% |
94
+ | constant read, same file / imported | 100.0% / 100.0% | not modelled | not modelled |
95
+ | function passed as a value, same file / imported | 97.8% / 96.9% | 1.9% / 0.1% | 13.8% / 5.3% |
94
96
  | mean targets emitted per call | 1.00–1.06 | 1.00 | 1.00 |
95
- | edges the referee can refute outright | **0** of 157,036 | 0 of 23,736 | 202 of 30,882 |
96
- | live symbols wrongly shown as uncalled | **1** (0.0%) | 1,373 (29.4%) | 205 (4.4%) |
97
+ | edges the referee can refute outright | **0** of 204,713 | 0 of 23,736 | 202 of 30,882 |
98
+ | live symbols wrongly shown as uncalled | **1** (0.0%) | 3,635 (52.3%) | 2,381 (34.2%) |
97
99
  | hand-written edge cases missed or fabricated | **1** of 13 | 4 of 13 | 3 of 13 |
98
100
  | *who calls `Model.check`*, precision | 94% | 0% | 95% |
99
- | index from scratch | **1.7 s** | 23.0 s | 28.8 s |
100
- | index on disk | **48 MB** | 642 MB | 151 MB |
101
- | `impact X` from cold | **56 ms** | 949 ms | 856 ms |
101
+ | index from scratch | **2.6 s** | 23.1 s | 28.6 s |
102
+ | index on disk | **78 MB** | 730 MB | 151 MB |
103
+ | `impact X` from cold | **59 ms** | 977 ms | 830 ms |
102
104
 
103
105
  Six of those Heimdall numbers only got there because the benchmark existed. It found that
104
106
  imports were resolved file-wide rather than per-scope, that a Python `pkg/main.py`
@@ -216,10 +218,18 @@ Four passes, all precomputed at index time so a query is one SQL statement:
216
218
  Receivers are used wherever they carry information: `self.foo()` resolves inside the
217
219
  enclosing class, `utils.foo()` inside the file imported as `utils`, and `Model.foo()`
218
220
  inside the class `Model` — including when *several* classes are called `Model`, because
219
- the one that defines `foo` is the one that was meant. On Django that leaves 46,260 of
220
- 286,346 call edges resolved by name alone — and every one of them is labelled
221
+ the one that defines `foo` is the one that was meant. On Django that leaves 46,277 of
222
+ 350,104 call edges resolved by name alone — and every one of them is labelled
221
223
  `confidence = 0.3`.
222
224
 
225
+ A `CALLS` edge covers every way one symbol depends on another by name, not only
226
+ invocation: reading a constant, handing a function over as a value (`register(on_save)`,
227
+ `key=parse`), naming a class as a receiver, and a decorator. All of those break the same
228
+ way when the target changes, which is the question `impact` answers. Where they differ is
229
+ the fallback: an unresolvable *call* is emitted as up to five `0.3` candidates, while an
230
+ unresolvable *reference* is dropped entirely — a bare name with nothing behind it is
231
+ almost always a local variable, and guessing five ways at a local variable is noise.
232
+
223
233
  ### Schema
224
234
 
225
235
  ```sql
@@ -227,7 +237,7 @@ node(id, kind, name, qname, path, line, end_line, lang, signature)
227
237
  edge(src, dst, kind, confidence, line)
228
238
  ```
229
239
 
230
- `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`.
240
+ `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`, `Constant`.
231
241
  `kind` on edges: `CONTAINS`, `DEFINES`, `IMPORTS`, `CALLS`, `EXTENDS`.
232
242
 
233
243
  That is the whole data model — `heimdall sql` gives you direct access to it.
@@ -235,17 +245,77 @@ That is the whole data model — `heimdall sql` gives you direct access to it.
235
245
  ## Languages
236
246
 
237
247
  Python, JavaScript, TypeScript, TSX, Go, Rust, Java, Kotlin, C#, Ruby, PHP, C, C++,
238
- Swift, Scala, Lua, Dart, Bash. Every one of them is covered by a test that asserts it
239
- still yields symbols and call edges, so a grammar rename fails the build rather than
240
- silently degrading the graph.
248
+ Swift, Scala, Lua, Dart, Bash, Terraform, Kubernetes. Every one of them is covered by a
249
+ test that asserts it still yields symbols and call edges, so a grammar rename fails the
250
+ build rather than silently degrading the graph.
241
251
 
242
- Adding a language means adding rows to three dicts in `heimdall/parse.py` — node types
243
- for definitions, for calls, and for imports — plus a snippet in the test.
252
+ Adding a language means adding rows to four dicts in `heimdall/parse.py` — node types
253
+ for definitions, for calls, for imports and for name bindings — plus a snippet in the test.
244
254
 
245
255
  One known gap: in Ruby, a receiver-less call without parentheses (`foo`) is
246
256
  indistinguishable from a variable reference in the grammar, so only `foo()` and
247
257
  `x.foo` produce call edges.
248
258
 
259
+ ### Terraform
260
+
261
+ `.tf` and `.hcl` are read differently, because HCL has no definitions and no calls. A
262
+ block is indexed under the address the rest of the configuration refers to it by —
263
+ `resource "aws_instance" "web"` becomes `aws_instance.web`, `variable "region"` becomes
264
+ `var.region`, and each `locals` attribute its own `local.x` — and every reference to one
265
+ of those addresses becomes a `CALLS` edge. The scope is Terraform's own: a directory of
266
+ `.tf` files is one module, and a `module` block's arguments resolve to the `variable`
267
+ blocks in the directory its `source` points at, which is what makes *what breaks if I
268
+ rename this module input* answerable.
269
+
270
+ ```console
271
+ heimdall context aws_instance.web # what it reads, and who reads it
272
+ heimdall impact var.region # everything downstream of one variable
273
+ ```
274
+
275
+ Blocks are stored under the existing `Class` kind rather than a new one — the schema's
276
+ six kinds are the same six for every language, and `Class` is already where Go's
277
+ `type_spec` and C's `struct` live.
278
+
279
+ Scored against a regex oracle on three public repos — 432 HCL files, 5,104 blocks, 10,206
280
+ edges — one reference of 2,828 goes unresolved and none of the edges is wrong; see
281
+ [bench/README.md](bench/README.md).
282
+ Packer and Nomad `.hcl` parse with the same grammar and get symbols, but only Terraform's
283
+ addressing rules are modelled.
284
+
285
+ ### Kubernetes
286
+
287
+ `.yaml` and `.yml` are read the same way and for the same reason. A document that carries
288
+ `apiVersion`, `kind` and `metadata.name` is indexed under the address `kubectl` uses for
289
+ it — `Deployment/api`, `ConfigMap/app-config` — and every field that holds the name of
290
+ another document becomes a `CALLS` edge. Which kind a name refers to is decided by the
291
+ field it sits in: the string under `configMapRef.name` can only be a ConfigMap, and where
292
+ the field does not say (`roleRef`, `scaleTargetRef`) the sibling `kind` does.
293
+
294
+ ```console
295
+ heimdall impact ConfigMap/app-config # every workload that would restart
296
+ heimdall context Deployment/api # what it mounts, and who routes to it
297
+ ```
298
+
299
+ A Service names no Deployment, so the one reference that is not written as a name is
300
+ matched by shape instead: a selector's labels against the labels a workload puts on its
301
+ pods, within one namespace. Scope settles the rest — a directory of manifests is a scope
302
+ like a Terraform module, and `kustomization.yaml`'s `resources` say which other
303
+ directories to add, so a `base` and its overlays resolve to the copy that was meant.
304
+
305
+ Scored against an oracle that re-reads the manifests by indentation, on four public repos
306
+ — 1,988 files, 2,748 documents — every reference by name and every reference by label
307
+ resolves to the right document, and none of the edges is wrong.
308
+
309
+ Yaml that is not a manifest is a file in the graph and nothing more; a Helm chart is not
310
+ yaml until it is rendered, and `{{ .Release.Name }}-api` is not a name, so unrendered
311
+ templates are skipped rather than guessed at.
312
+
313
+ One approximation worth knowing: a selector is matched against pod labels whatever kind
314
+ wrote it. A Prometheus `ServiceMonitor` selects *Services*, not pods, so Heimdall links it
315
+ to the workload whose pods carry those labels — the neighbouring object rather than the
316
+ exact one. Matching by kind would need a table of what every CRD selects; matching by
317
+ labels gets a CRD that selects pods right without knowing it exists.
318
+
249
319
  ## Limitations
250
320
 
251
321
  Call resolution is name-based, not type-based. `self.x()` and `module.x()` resolve
@@ -12,14 +12,14 @@ Apache-2.0. No server, no cloud, no embedded graph engine. **The graph is a SQLi
12
12
  ```console
13
13
  $ heimdall index ~/code/django
14
14
  pass 1/4 discovering files
15
- 2974 source files
15
+ 2998 source files
16
16
  pass 2/4 parsing (24 workers)
17
17
  pass 3/4 building graph
18
18
  pass 4/4 building search index
19
- done in 1.49s # 43,482 symbols, 353,101 edges
19
+ done in 2.35s # 45,515 symbols, 418,951 edges
20
20
 
21
21
  $ heimdall impact django/shortcuts.py:get_object_or_404 -d 2 -c 0.7
22
- impact of django/shortcuts.py:get_object_or_404: 6 symbols, 4 files
22
+ impact of django/shortcuts.py:get_object_or_404: 10 symbols, 8 files
23
23
 
24
24
  depth 1 (5):
25
25
  Function django/contrib/flatpages/views.py:flatpage (0.80)
@@ -33,7 +33,7 @@ path of length 2:
33
33
  -> django/http/response.py:HttpResponse
34
34
  ```
35
35
 
36
- A second and a half, cold, for the whole of Django on a laptop.
36
+ Two and a half seconds, cold, for the whole of Django on a laptop.
37
37
 
38
38
  ## Why
39
39
 
@@ -69,16 +69,18 @@ against [graphify](https://pypi.org/project/graphifyy/) in its AST-only mode:
69
69
  |---|---|---|---|
70
70
  | `self.m()` bound correctly | 100.0% | 97.1% | 92.6% |
71
71
  | same-file call bound correctly | 100.0% | 93.4% | 89.3% |
72
- | imported call bound correctly | 99.7% | 71.5% | 75.4% |
72
+ | imported call bound correctly | 99.7% | 71.4% | 75.3% |
73
73
  | class instantiation, same file / imported | 99.5% / 98.8% | not modelled | 95.2% / 98.2% |
74
+ | constant read, same file / imported | 100.0% / 100.0% | not modelled | not modelled |
75
+ | function passed as a value, same file / imported | 97.8% / 96.9% | 1.9% / 0.1% | 13.8% / 5.3% |
74
76
  | mean targets emitted per call | 1.00–1.06 | 1.00 | 1.00 |
75
- | edges the referee can refute outright | **0** of 157,036 | 0 of 23,736 | 202 of 30,882 |
76
- | live symbols wrongly shown as uncalled | **1** (0.0%) | 1,373 (29.4%) | 205 (4.4%) |
77
+ | edges the referee can refute outright | **0** of 204,713 | 0 of 23,736 | 202 of 30,882 |
78
+ | live symbols wrongly shown as uncalled | **1** (0.0%) | 3,635 (52.3%) | 2,381 (34.2%) |
77
79
  | hand-written edge cases missed or fabricated | **1** of 13 | 4 of 13 | 3 of 13 |
78
80
  | *who calls `Model.check`*, precision | 94% | 0% | 95% |
79
- | index from scratch | **1.7 s** | 23.0 s | 28.8 s |
80
- | index on disk | **48 MB** | 642 MB | 151 MB |
81
- | `impact X` from cold | **56 ms** | 949 ms | 856 ms |
81
+ | index from scratch | **2.6 s** | 23.1 s | 28.6 s |
82
+ | index on disk | **78 MB** | 730 MB | 151 MB |
83
+ | `impact X` from cold | **59 ms** | 977 ms | 830 ms |
82
84
 
83
85
  Six of those Heimdall numbers only got there because the benchmark existed. It found that
84
86
  imports were resolved file-wide rather than per-scope, that a Python `pkg/main.py`
@@ -196,10 +198,18 @@ Four passes, all precomputed at index time so a query is one SQL statement:
196
198
  Receivers are used wherever they carry information: `self.foo()` resolves inside the
197
199
  enclosing class, `utils.foo()` inside the file imported as `utils`, and `Model.foo()`
198
200
  inside the class `Model` — including when *several* classes are called `Model`, because
199
- the one that defines `foo` is the one that was meant. On Django that leaves 46,260 of
200
- 286,346 call edges resolved by name alone — and every one of them is labelled
201
+ the one that defines `foo` is the one that was meant. On Django that leaves 46,277 of
202
+ 350,104 call edges resolved by name alone — and every one of them is labelled
201
203
  `confidence = 0.3`.
202
204
 
205
+ A `CALLS` edge covers every way one symbol depends on another by name, not only
206
+ invocation: reading a constant, handing a function over as a value (`register(on_save)`,
207
+ `key=parse`), naming a class as a receiver, and a decorator. All of those break the same
208
+ way when the target changes, which is the question `impact` answers. Where they differ is
209
+ the fallback: an unresolvable *call* is emitted as up to five `0.3` candidates, while an
210
+ unresolvable *reference* is dropped entirely — a bare name with nothing behind it is
211
+ almost always a local variable, and guessing five ways at a local variable is noise.
212
+
203
213
  ### Schema
204
214
 
205
215
  ```sql
@@ -207,7 +217,7 @@ node(id, kind, name, qname, path, line, end_line, lang, signature)
207
217
  edge(src, dst, kind, confidence, line)
208
218
  ```
209
219
 
210
- `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`.
220
+ `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`, `Constant`.
211
221
  `kind` on edges: `CONTAINS`, `DEFINES`, `IMPORTS`, `CALLS`, `EXTENDS`.
212
222
 
213
223
  That is the whole data model — `heimdall sql` gives you direct access to it.
@@ -215,17 +225,77 @@ That is the whole data model — `heimdall sql` gives you direct access to it.
215
225
  ## Languages
216
226
 
217
227
  Python, JavaScript, TypeScript, TSX, Go, Rust, Java, Kotlin, C#, Ruby, PHP, C, C++,
218
- Swift, Scala, Lua, Dart, Bash. Every one of them is covered by a test that asserts it
219
- still yields symbols and call edges, so a grammar rename fails the build rather than
220
- silently degrading the graph.
228
+ Swift, Scala, Lua, Dart, Bash, Terraform, Kubernetes. Every one of them is covered by a
229
+ test that asserts it still yields symbols and call edges, so a grammar rename fails the
230
+ build rather than silently degrading the graph.
221
231
 
222
- Adding a language means adding rows to three dicts in `heimdall/parse.py` — node types
223
- for definitions, for calls, and for imports — plus a snippet in the test.
232
+ Adding a language means adding rows to four dicts in `heimdall/parse.py` — node types
233
+ for definitions, for calls, for imports and for name bindings — plus a snippet in the test.
224
234
 
225
235
  One known gap: in Ruby, a receiver-less call without parentheses (`foo`) is
226
236
  indistinguishable from a variable reference in the grammar, so only `foo()` and
227
237
  `x.foo` produce call edges.
228
238
 
239
+ ### Terraform
240
+
241
+ `.tf` and `.hcl` are read differently, because HCL has no definitions and no calls. A
242
+ block is indexed under the address the rest of the configuration refers to it by —
243
+ `resource "aws_instance" "web"` becomes `aws_instance.web`, `variable "region"` becomes
244
+ `var.region`, and each `locals` attribute its own `local.x` — and every reference to one
245
+ of those addresses becomes a `CALLS` edge. The scope is Terraform's own: a directory of
246
+ `.tf` files is one module, and a `module` block's arguments resolve to the `variable`
247
+ blocks in the directory its `source` points at, which is what makes *what breaks if I
248
+ rename this module input* answerable.
249
+
250
+ ```console
251
+ heimdall context aws_instance.web # what it reads, and who reads it
252
+ heimdall impact var.region # everything downstream of one variable
253
+ ```
254
+
255
+ Blocks are stored under the existing `Class` kind rather than a new one — the schema's
256
+ six kinds are the same six for every language, and `Class` is already where Go's
257
+ `type_spec` and C's `struct` live.
258
+
259
+ Scored against a regex oracle on three public repos — 432 HCL files, 5,104 blocks, 10,206
260
+ edges — one reference of 2,828 goes unresolved and none of the edges is wrong; see
261
+ [bench/README.md](bench/README.md).
262
+ Packer and Nomad `.hcl` parse with the same grammar and get symbols, but only Terraform's
263
+ addressing rules are modelled.
264
+
265
+ ### Kubernetes
266
+
267
+ `.yaml` and `.yml` are read the same way and for the same reason. A document that carries
268
+ `apiVersion`, `kind` and `metadata.name` is indexed under the address `kubectl` uses for
269
+ it — `Deployment/api`, `ConfigMap/app-config` — and every field that holds the name of
270
+ another document becomes a `CALLS` edge. Which kind a name refers to is decided by the
271
+ field it sits in: the string under `configMapRef.name` can only be a ConfigMap, and where
272
+ the field does not say (`roleRef`, `scaleTargetRef`) the sibling `kind` does.
273
+
274
+ ```console
275
+ heimdall impact ConfigMap/app-config # every workload that would restart
276
+ heimdall context Deployment/api # what it mounts, and who routes to it
277
+ ```
278
+
279
+ A Service names no Deployment, so the one reference that is not written as a name is
280
+ matched by shape instead: a selector's labels against the labels a workload puts on its
281
+ pods, within one namespace. Scope settles the rest — a directory of manifests is a scope
282
+ like a Terraform module, and `kustomization.yaml`'s `resources` say which other
283
+ directories to add, so a `base` and its overlays resolve to the copy that was meant.
284
+
285
+ Scored against an oracle that re-reads the manifests by indentation, on four public repos
286
+ — 1,988 files, 2,748 documents — every reference by name and every reference by label
287
+ resolves to the right document, and none of the edges is wrong.
288
+
289
+ Yaml that is not a manifest is a file in the graph and nothing more; a Helm chart is not
290
+ yaml until it is rendered, and `{{ .Release.Name }}-api` is not a name, so unrendered
291
+ templates are skipped rather than guessed at.
292
+
293
+ One approximation worth knowing: a selector is matched against pod labels whatever kind
294
+ wrote it. A Prometheus `ServiceMonitor` selects *Services*, not pods, so Heimdall links it
295
+ to the workload whose pods carry those labels — the neighbouring object rather than the
296
+ exact one. Matching by kind would need a table of what every CRD selects; matching by
297
+ labels gets a CRD that selects pods right without knowing it exists.
298
+
229
299
  ## Limitations
230
300
 
231
301
  Call resolution is name-based, not type-based. `self.x()` and `module.x()` resolve
@@ -45,7 +45,7 @@ The *who writes into this table* question: a method name that several classes sh
45
45
 
46
46
  | | callers of the right one | callers of anything named `check` |
47
47
  |---|---|---|
48
- | Heimdall | 100% recall, 94% precision (109 returned) | 100% recall, 23% precision (442 returned) |
48
+ | Heimdall | 100% recall, 94% precision (110 returned) | 100% recall, 23% precision (453 returned) |
49
49
  | GitNexus | 0% recall, 0% precision (7 returned) | 0% recall, 0% precision (42 returned) |
50
50
  | graphify | 99% recall, 95% precision (107 returned) | 99% recall, 61% precision (168 returned) |
51
51
  | graphify `+uses` | 99% recall, 95% precision (107 returned) | 99% recall, 61% precision (168 returned) |
@@ -58,11 +58,11 @@ Blast radius at depth 3 for the five most-called symbols the oracle is sure abou
58
58
 
59
59
  | target | truth | Heimdall | GitNexus | graphify | graphify `+uses` |
60
60
  |---|---|---|---|---|---|
61
- | `utils.py:setup` | 912 | 100% of truth, 971 total | 0% of truth, 103 total | 0% of truth, 59 total | 0% of truth, 59 total |
62
- | `utils.py:override_settings` | 200 | 100% of truth, 1,792 total | 0% of truth, 0 total | 100% of truth, 1,549 total | 100% of truth, 1,623 total |
63
- | `safestring.py:mark_safe` | 338 | 100% of truth, 11,830 total | 100% of truth, 430 total | 97% of truth, 482 total | 97% of truth, 482 total |
64
- | `utils.py:isolate_apps` | 132 | 100% of truth, 201 total | 0% of truth, 0 total | 100% of truth, 201 total | 100% of truth, 201 total |
65
- | `utils.py:CaptureQueriesContext` | 91 | 100% of truth, 91 total | 0% of truth, 0 total | 99% of truth, 90 total | 99% of truth, 90 total |
61
+ | `utils.py:setup` | 912 | 100% of truth, 982 total | 0% of truth, 103 total | 0% of truth, 59 total | 0% of truth, 59 total |
62
+ | `exceptions.py:ValidationError` | 393 | 100% of truth, 1,503 total | 0% of truth, 0 total | 29% of truth, 122 total | 29% of truth, 122 total |
63
+ | `models.py:User` | 322 | 99% of truth, 2,527 total | 0% of truth, 0 total | 6% of truth, 20 total | 6% of truth, 20 total |
64
+ | `functional.py:cached_property` | 298 | 100% of truth, 14,504 total | 0% of truth, 0 total | 1% of truth, 3 total | 9% of truth, 7,545 total |
65
+ | `exceptions.py:ImproperlyConfigured` | 473 | 100% of truth, 9,413 total | 0% of truth, 0 total | 75% of truth, 444 total | 75% of truth, 444 total |
66
66
 
67
67
  ### Is any of this made up?
68
68
 
@@ -70,19 +70,19 @@ An edge `f -> g` is refuted when neither `g` nor anything imported *as* `g` occu
70
70
 
71
71
  | | refuted edges |
72
72
  |---|---|
73
- | Heimdall | 0 of 157,036 checkable (0.00%) |
73
+ | Heimdall | 0 of 204,713 checkable (0.00%) |
74
74
  | GitNexus | 0 of 23,736 checkable (0.00%) |
75
75
  | graphify | 202 of 30,882 checkable (0.65%) |
76
76
  | graphify `+uses` | 48,856 of 84,172 checkable (58.04%) |
77
77
 
78
78
  ### Would it tell me live code is dead?
79
79
 
80
- 4,668 symbols the oracle proves are called. A tool that reports no callers for one of them would send an agent to delete live code.
80
+ 6,952 symbols the oracle proves are called. A tool that reports no callers for one of them would send an agent to delete live code.
81
81
 
82
82
  | | symbols wrongly shown as uncalled |
83
83
  |---|---|
84
84
  | Heimdall | 1 (0.0%) |
85
- | GitNexus | 1,373 (29.4%) |
86
- | graphify | 205 (4.4%) |
87
- | graphify `+uses` | 199 (4.3%) |
85
+ | GitNexus | 3,635 (52.3%) |
86
+ | graphify | 2,381 (34.2%) |
87
+ | graphify `+uses` | 1,134 (16.3%) |
88
88