heimdall-code 0.1.0__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (33) hide show
  1. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/.claude-plugin/plugin.json +1 -1
  2. heimdall_code-0.2.0/CHANGELOG.md +98 -0
  3. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/CONTRIBUTING.md +15 -3
  4. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/PKG-INFO +54 -18
  5. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/README.md +53 -17
  6. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/QUESTIONS.md +11 -11
  7. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/README.md +49 -2
  8. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/RESULTS.md +20 -16
  9. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/compare.py +61 -7
  10. heimdall_code-0.2.0/bench/terraform.py +187 -0
  11. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/__init__.py +1 -1
  12. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/cli.py +2 -1
  13. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/graph.py +1 -1
  14. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/index.py +34 -5
  15. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/parse.py +284 -6
  16. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/pyproject.toml +1 -1
  17. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/skills/heimdall/SKILL.md +1 -1
  18. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/tests/test_heimdall.py +95 -3
  19. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/uv.lock +1 -1
  20. heimdall_code-0.1.0/CHANGELOG.md +0 -44
  21. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/.claude-plugin/marketplace.json +0 -0
  22. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/.github/workflows/release.yml +0 -0
  23. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/.gitignore +0 -0
  24. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/.mcp.json +0 -0
  25. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/CLAUDE.md +0 -0
  26. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/LICENSE +0 -0
  27. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/.gitignore +0 -0
  28. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/dump_lbug.mjs +0 -0
  29. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/bench/questions.py +0 -0
  30. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/__main__.py +0 -0
  31. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/install.py +0 -0
  32. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/mcp_server.py +0 -0
  33. {heimdall_code-0.1.0 → heimdall_code-0.2.0}/heimdall/registry.py +0 -0
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "heimdall",
3
- "version": "0.1.0",
3
+ "version": "0.2.0",
4
4
  "description": "A code knowledge graph that sees everything. Indexes any repo into a SQLite property graph and answers what calls this, what breaks if I change it, and how does A reach B.",
5
5
  "author": {
6
6
  "name": "Bogdan Sapovskyi",
@@ -0,0 +1,98 @@
1
+ # Changelog
2
+
3
+ Notable changes, newest first. Versions follow [semver](https://semver.org); until 1.0
4
+ the graph schema and the JSON payloads may still move.
5
+
6
+ ## 0.2.0 — 2026-08-01
7
+
8
+ ### Added
9
+
10
+ - **Constants are symbols.** A `SCREAMING_SNAKE` module- or class-level binding is indexed
11
+ as a new node kind, `Constant`, and reading one is an edge. Endpoint tables, thresholds
12
+ and feature flags were previously not in the graph at all, so `impact SOME_THRESHOLD`
13
+ answered nothing and the question had to go back to `rg`.
14
+ - **A function handed over rather than called** is an edge too — `register(on_save)`,
15
+ `key=parse`, `@decorator`, a class named as a receiver. On Django the oracle finds
16
+ 12,000 such sites; recall went from 8.4% / 4.8% (local / imported) to 97.8% / 96.9%,
17
+ where GitNexus scores 1.9% / 0.1% and graphify 13.8% / 5.3%. Constant reads score
18
+ 100.0% / 100.0%; neither other tool models them at all.
19
+ - Both are `CALLS` edges, on the same argument the decorator edge already made: they break
20
+ the same way when the target changes. They differ from a call in the fallback — an
21
+ unresolvable *call* is emitted as up to five `0.3` candidates, an unresolvable
22
+ *reference* is dropped, because a bare name with no evidence behind it is almost always
23
+ a local variable. Django's `0.3` edge count came out *below* what it was before the
24
+ feature, and the referee still refutes 0 of 204,713 checkable edges.
25
+ - The cost: Django indexes in 2.7 s instead of 1.7 s and the graph is 78 MB instead of 48.
26
+ `bench/RESULTS.md`, `bench/QUESTIONS.md` and the README tables are regenerated against
27
+ all three tools rather than edited.
28
+
29
+ - **Terraform**, as the 19th language and the first one the generic walker could not
30
+ reach: `.tf` and `.hcl` are read by `_parse_hcl`, which indexes a block under the
31
+ address the rest of the configuration names it by (`resource "aws_instance" "web"` ->
32
+ `aws_instance.web`, `variable "region"` -> `var.region`) and turns every reference to
33
+ one into a `CALLS` edge. Scoping is Terraform's: a directory is a module, and a `module`
34
+ block's arguments resolve to the `variable` blocks in the directory its `source` names.
35
+ - `bench/terraform.py`, graded by a regex oracle that shares no code with the reader.
36
+ Across three public repos — 432 HCL files, 5,104 blocks, 10,206 edges — one reference
37
+ of 2,828 goes unresolved and none of the edges is wrong.
38
+
39
+ ### Fixed
40
+
41
+ - A name being *introduced* is no longer read as a reference to something else: a
42
+ parameter, a `*args` splat, a loop variable, a destructuring pattern and a keyword
43
+ argument's name all bind rather than read. Found by the referee, which refuted exactly
44
+ one edge — `def handle(self, *labels)` pointing at an unrelated `labels` method.
45
+ - A relative import with more than one `..` (`from '../../store'`) resolved to nothing:
46
+ the dots after the first were read as module separators, which is right for
47
+ `..pkg.mod` and wrong for a path. Every such import in every JS/TS repo was silently
48
+ missing its edge.
49
+
50
+ ## 0.1.1 — 2026-08-01
51
+
52
+ - `heimdall --version`.
53
+ - Works on Python 3.12 again: `os.process_cpu_count()` is 3.13+, so every index crashed
54
+ on the version the package declares as its floor. The release workflow now runs the
55
+ suite on 3.12 rather than on whatever the runner image ships.
56
+ - `install` no longer treats a heimdall in the default uv cache as durable when
57
+ `UV_CACHE_DIR` points elsewhere — that config would have rotted at the next `uv cache
58
+ prune`.
59
+
60
+ ## 0.1.0 — 2026-08-01
61
+
62
+ First release.
63
+
64
+ ### Added
65
+
66
+ - **Index.** `heimdall index` builds `.heimdall/graph.db` in four passes — discover
67
+ (`git ls-files`, submodules included), parse (tree-sitter, one process per core),
68
+ resolve, FTS5. 1.7 s and 48 MB for Django's 2,927 files.
69
+ - **Query.** `search`, `context`, `impact`, `trace`, `stats`, `projects` and a read-only
70
+ `sql` escape hatch over a two-table schema. Every command takes `--json` and
71
+ `-p/--project`.
72
+ - **18 languages**, declared as data in `parse.py` and each covered by a test that fails
73
+ if a grammar renames a node type.
74
+ - **MCP server** (`heimdall mcp`): one process serves every repo in
75
+ `~/.heimdall/registry.json`, eight tools returning assembled answers rather than raw
76
+ edges.
77
+ - **`heimdall install [claude|codex|all]`** — writes `~/.claude.json`, `./.mcp.json`,
78
+ `~/.codex/config.toml` or `./.codex/config.toml`, merging around your existing
79
+ settings; `--cli` installs the skill instead of the server.
80
+ - **Claude Code plugin**: `/plugin marketplace add stPhoenix/heimdall`.
81
+ - **Benchmark** (`bench/`) against GitNexus and graphify, graded by a CPython `ast`
82
+ oracle.
83
+
84
+ ### Resolution fixes found by the benchmark
85
+
86
+ Each of these was a wrong answer before it was a test:
87
+
88
+ - Imports resolve per scope, so a function with its own `from x import y` is no longer
89
+ matched against the whole file.
90
+ - `pkg/main.py` is reachable as `pkg.main`, not only as `pkg`.
91
+ - `as` renames are followed home — but only for in-repo targets, so
92
+ `from asyncio import run as arun` cannot bind to your own `run`.
93
+ - A decorator is a call by the function it wraps, not by the enclosing class. Worth 26%
94
+ of imported-call recall on Django.
95
+ - Recursion produces an edge.
96
+ - A class name used as a receiver resolves inside that class — including when several
97
+ classes share the name, because the one that defines the member is the one that was
98
+ meant. Took *who calls `Model.check`* from 0% to 94% precision.
@@ -18,7 +18,7 @@ There is no CI. The gate is `uv run pytest && uv run ruff check .` before you pu
18
18
 
19
19
  | | |
20
20
  |---|---|
21
- | `heimdall/parse.py` | tree-sitter → symbols, calls, imports, heritage. Language support is **data** (`EXT_LANG`, `DEFS`, `CALL_NODES`, `IMPORT_NODES`, `HERITAGE_*`), walked by one generic function. |
21
+ | `heimdall/parse.py` | tree-sitter → symbols, calls, imports, heritage. Language support is **data** (`EXT_LANG`, `DEFS`, `CALL_NODES`, `IMPORT_NODES`, `CONST_NODES`, `HERITAGE_*`), walked by one generic function. |
22
22
  | `heimdall/index.py` | the four passes: discover → parse (process pool) → resolve → store. Everything a query needs is precomputed here. |
23
23
  | `heimdall/graph.py` | schema + reads. Two tables, recursive CTEs, FTS5. `query()` is the read-only SQL gate. |
24
24
  | `heimdall/cli.py` | argparse; every command takes `--json` and `-p/--project`. |
@@ -29,11 +29,19 @@ There is no CI. The gate is `uv run pytest && uv run ruff check .` before you pu
29
29
 
30
30
  ## Adding a language
31
31
 
32
- Rows in three dicts in `parse.py` — definition node types, call node types, import node
33
- types — plus a snippet in `SNIPPETS` in `tests/test_heimdall.py`. No new module. The test
32
+ Rows in four dicts in `parse.py` — definition node types, call node types, import node
33
+ types, and the node types that bind a name (which is how constants are found) — plus a
34
+ snippet in `SNIPPETS` in `tests/test_heimdall.py`. No new module. The test
34
35
  asserts the snippet still yields symbols *and* call edges, so a grammar renaming a node
35
36
  type fails the build instead of quietly emptying the graph.
36
37
 
38
+ HCL is the one language that did not fit, and it is worth reading before deciding yours
39
+ does not either: it has no definition nodes and no call nodes, so no set of rows could
40
+ express it. It gets `_parse_hcl`, a reader beside the walker rather than a branch inside
41
+ it, and its own three tables of Terraform addressing rules. That is the shape to copy if
42
+ a language genuinely needs it — and the bar is "the grammar has nothing the walker is
43
+ looking for", not "the walker gets a few nodes wrong".
44
+
37
45
  ## Tests
38
46
 
39
47
  One file, `tests/test_heimdall.py`, runnable with or without pytest. Every resolution bug
@@ -48,6 +56,10 @@ resolution bug, add the smallest fixture that would have caught it.
48
56
  (it wants Node, npm and a clone of Django), but it is what a change to resolution should
49
57
  be measured with: `python bench/compare.py /path/to/django` and `python bench/questions.py`.
50
58
 
59
+ `bench/terraform.py` does the same for HCL against a regex oracle — no other tool to
60
+ compare against, so it only grades Heimdall's own edges: `python bench/terraform.py
61
+ /path/to/some-terraform-repo`.
62
+
51
63
  ## Release
52
64
 
53
65
  Bump `version` in `pyproject.toml`, `__version__` in `heimdall/__init__.py` and `version`
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: heimdall-code
3
- Version: 0.1.0
3
+ Version: 0.2.0
4
4
  Summary: A code knowledge graph that sees everything -- indexes any repository into a SQLite property graph and serves it to AI agents over MCP.
5
5
  Project-URL: Homepage, https://github.com/stPhoenix/heimdall
6
6
  Project-URL: Source, https://github.com/stPhoenix/heimdall
@@ -36,10 +36,10 @@ pass 1/4 discovering files
36
36
  pass 2/4 parsing (24 workers)
37
37
  pass 3/4 building graph
38
38
  pass 4/4 building search index
39
- done in 1.49s # 43,482 symbols, 353,101 edges
39
+ done in 2.42s # 45,515 symbols, 418,925 edges
40
40
 
41
41
  $ heimdall impact django/shortcuts.py:get_object_or_404 -d 2 -c 0.7
42
- impact of django/shortcuts.py:get_object_or_404: 6 symbols, 4 files
42
+ impact of django/shortcuts.py:get_object_or_404: 10 symbols, 8 files
43
43
 
44
44
  depth 1 (5):
45
45
  Function django/contrib/flatpages/views.py:flatpage (0.80)
@@ -53,7 +53,7 @@ path of length 2:
53
53
  -> django/http/response.py:HttpResponse
54
54
  ```
55
55
 
56
- A second and a half, cold, for the whole of Django on a laptop.
56
+ Two and a half seconds, cold, for the whole of Django on a laptop.
57
57
 
58
58
  ## Why
59
59
 
@@ -89,16 +89,18 @@ against [graphify](https://pypi.org/project/graphifyy/) in its AST-only mode:
89
89
  |---|---|---|---|
90
90
  | `self.m()` bound correctly | 100.0% | 97.1% | 92.6% |
91
91
  | same-file call bound correctly | 100.0% | 93.4% | 89.3% |
92
- | imported call bound correctly | 99.7% | 71.5% | 75.4% |
92
+ | imported call bound correctly | 99.7% | 71.4% | 75.3% |
93
93
  | class instantiation, same file / imported | 99.5% / 98.8% | not modelled | 95.2% / 98.2% |
94
+ | constant read, same file / imported | 100.0% / 100.0% | not modelled | not modelled |
95
+ | function passed as a value, same file / imported | 97.8% / 96.9% | 1.9% / 0.1% | 13.8% / 5.3% |
94
96
  | mean targets emitted per call | 1.00–1.06 | 1.00 | 1.00 |
95
- | edges the referee can refute outright | **0** of 157,036 | 0 of 23,736 | 202 of 30,882 |
96
- | live symbols wrongly shown as uncalled | **1** (0.0%) | 1,373 (29.4%) | 205 (4.4%) |
97
+ | edges the referee can refute outright | **0** of 204,713 | 0 of 23,736 | 202 of 30,882 |
98
+ | live symbols wrongly shown as uncalled | **1** (0.0%) | 3,635 (52.3%) | 2,381 (34.2%) |
97
99
  | hand-written edge cases missed or fabricated | **1** of 13 | 4 of 13 | 3 of 13 |
98
100
  | *who calls `Model.check`*, precision | 94% | 0% | 95% |
99
- | index from scratch | **1.7 s** | 23.0 s | 28.8 s |
100
- | index on disk | **48 MB** | 642 MB | 151 MB |
101
- | `impact X` from cold | **56 ms** | 949 ms | 856 ms |
101
+ | index from scratch | **2.7 s** | 23.6 s | 29.8 s |
102
+ | index on disk | **78 MB** | 730 MB | 151 MB |
103
+ | `impact X` from cold | **58 ms** | 969 ms | 837 ms |
102
104
 
103
105
  Six of those Heimdall numbers only got there because the benchmark existed. It found that
104
106
  imports were resolved file-wide rather than per-scope, that a Python `pkg/main.py`
@@ -216,10 +218,18 @@ Four passes, all precomputed at index time so a query is one SQL statement:
216
218
  Receivers are used wherever they carry information: `self.foo()` resolves inside the
217
219
  enclosing class, `utils.foo()` inside the file imported as `utils`, and `Model.foo()`
218
220
  inside the class `Model` — including when *several* classes are called `Model`, because
219
- the one that defines `foo` is the one that was meant. On Django that leaves 46,260 of
220
- 286,346 call edges resolved by name alone — and every one of them is labelled
221
+ the one that defines `foo` is the one that was meant. On Django that leaves 46,277 of
222
+ 350,104 call edges resolved by name alone — and every one of them is labelled
221
223
  `confidence = 0.3`.
222
224
 
225
+ A `CALLS` edge covers every way one symbol depends on another by name, not only
226
+ invocation: reading a constant, handing a function over as a value (`register(on_save)`,
227
+ `key=parse`), naming a class as a receiver, and a decorator. All of those break the same
228
+ way when the target changes, which is the question `impact` answers. Where they differ is
229
+ the fallback: an unresolvable *call* is emitted as up to five `0.3` candidates, while an
230
+ unresolvable *reference* is dropped entirely — a bare name with nothing behind it is
231
+ almost always a local variable, and guessing five ways at a local variable is noise.
232
+
223
233
  ### Schema
224
234
 
225
235
  ```sql
@@ -227,7 +237,7 @@ node(id, kind, name, qname, path, line, end_line, lang, signature)
227
237
  edge(src, dst, kind, confidence, line)
228
238
  ```
229
239
 
230
- `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`.
240
+ `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`, `Constant`.
231
241
  `kind` on edges: `CONTAINS`, `DEFINES`, `IMPORTS`, `CALLS`, `EXTENDS`.
232
242
 
233
243
  That is the whole data model — `heimdall sql` gives you direct access to it.
@@ -235,17 +245,43 @@ That is the whole data model — `heimdall sql` gives you direct access to it.
235
245
  ## Languages
236
246
 
237
247
  Python, JavaScript, TypeScript, TSX, Go, Rust, Java, Kotlin, C#, Ruby, PHP, C, C++,
238
- Swift, Scala, Lua, Dart, Bash. Every one of them is covered by a test that asserts it
239
- still yields symbols and call edges, so a grammar rename fails the build rather than
240
- silently degrading the graph.
248
+ Swift, Scala, Lua, Dart, Bash, Terraform. Every one of them is covered by a test that
249
+ asserts it still yields symbols and call edges, so a grammar rename fails the build
250
+ rather than silently degrading the graph.
241
251
 
242
- Adding a language means adding rows to three dicts in `heimdall/parse.py` — node types
243
- for definitions, for calls, and for imports — plus a snippet in the test.
252
+ Adding a language means adding rows to four dicts in `heimdall/parse.py` — node types
253
+ for definitions, for calls, for imports and for name bindings — plus a snippet in the test.
244
254
 
245
255
  One known gap: in Ruby, a receiver-less call without parentheses (`foo`) is
246
256
  indistinguishable from a variable reference in the grammar, so only `foo()` and
247
257
  `x.foo` produce call edges.
248
258
 
259
+ ### Terraform
260
+
261
+ `.tf` and `.hcl` are read differently, because HCL has no definitions and no calls. A
262
+ block is indexed under the address the rest of the configuration refers to it by —
263
+ `resource "aws_instance" "web"` becomes `aws_instance.web`, `variable "region"` becomes
264
+ `var.region`, and each `locals` attribute its own `local.x` — and every reference to one
265
+ of those addresses becomes a `CALLS` edge. The scope is Terraform's own: a directory of
266
+ `.tf` files is one module, and a `module` block's arguments resolve to the `variable`
267
+ blocks in the directory its `source` points at, which is what makes *what breaks if I
268
+ rename this module input* answerable.
269
+
270
+ ```console
271
+ heimdall context aws_instance.web # what it reads, and who reads it
272
+ heimdall impact var.region # everything downstream of one variable
273
+ ```
274
+
275
+ Blocks are stored under the existing `Class` kind rather than a new one — the schema's
276
+ six kinds are the same six for every language, and `Class` is already where Go's
277
+ `type_spec` and C's `struct` live.
278
+
279
+ Scored against a regex oracle on three public repos — 432 HCL files, 5,104 blocks, 10,206
280
+ edges — one reference of 2,828 goes unresolved and none of the edges is wrong; see
281
+ [bench/README.md](bench/README.md).
282
+ Packer and Nomad `.hcl` parse with the same grammar and get symbols, but only Terraform's
283
+ addressing rules are modelled.
284
+
249
285
  ## Limitations
250
286
 
251
287
  Call resolution is name-based, not type-based. `self.x()` and `module.x()` resolve
@@ -16,10 +16,10 @@ pass 1/4 discovering files
16
16
  pass 2/4 parsing (24 workers)
17
17
  pass 3/4 building graph
18
18
  pass 4/4 building search index
19
- done in 1.49s # 43,482 symbols, 353,101 edges
19
+ done in 2.42s # 45,515 symbols, 418,925 edges
20
20
 
21
21
  $ heimdall impact django/shortcuts.py:get_object_or_404 -d 2 -c 0.7
22
- impact of django/shortcuts.py:get_object_or_404: 6 symbols, 4 files
22
+ impact of django/shortcuts.py:get_object_or_404: 10 symbols, 8 files
23
23
 
24
24
  depth 1 (5):
25
25
  Function django/contrib/flatpages/views.py:flatpage (0.80)
@@ -33,7 +33,7 @@ path of length 2:
33
33
  -> django/http/response.py:HttpResponse
34
34
  ```
35
35
 
36
- A second and a half, cold, for the whole of Django on a laptop.
36
+ Two and a half seconds, cold, for the whole of Django on a laptop.
37
37
 
38
38
  ## Why
39
39
 
@@ -69,16 +69,18 @@ against [graphify](https://pypi.org/project/graphifyy/) in its AST-only mode:
69
69
  |---|---|---|---|
70
70
  | `self.m()` bound correctly | 100.0% | 97.1% | 92.6% |
71
71
  | same-file call bound correctly | 100.0% | 93.4% | 89.3% |
72
- | imported call bound correctly | 99.7% | 71.5% | 75.4% |
72
+ | imported call bound correctly | 99.7% | 71.4% | 75.3% |
73
73
  | class instantiation, same file / imported | 99.5% / 98.8% | not modelled | 95.2% / 98.2% |
74
+ | constant read, same file / imported | 100.0% / 100.0% | not modelled | not modelled |
75
+ | function passed as a value, same file / imported | 97.8% / 96.9% | 1.9% / 0.1% | 13.8% / 5.3% |
74
76
  | mean targets emitted per call | 1.00–1.06 | 1.00 | 1.00 |
75
- | edges the referee can refute outright | **0** of 157,036 | 0 of 23,736 | 202 of 30,882 |
76
- | live symbols wrongly shown as uncalled | **1** (0.0%) | 1,373 (29.4%) | 205 (4.4%) |
77
+ | edges the referee can refute outright | **0** of 204,713 | 0 of 23,736 | 202 of 30,882 |
78
+ | live symbols wrongly shown as uncalled | **1** (0.0%) | 3,635 (52.3%) | 2,381 (34.2%) |
77
79
  | hand-written edge cases missed or fabricated | **1** of 13 | 4 of 13 | 3 of 13 |
78
80
  | *who calls `Model.check`*, precision | 94% | 0% | 95% |
79
- | index from scratch | **1.7 s** | 23.0 s | 28.8 s |
80
- | index on disk | **48 MB** | 642 MB | 151 MB |
81
- | `impact X` from cold | **56 ms** | 949 ms | 856 ms |
81
+ | index from scratch | **2.7 s** | 23.6 s | 29.8 s |
82
+ | index on disk | **78 MB** | 730 MB | 151 MB |
83
+ | `impact X` from cold | **58 ms** | 969 ms | 837 ms |
82
84
 
83
85
  Six of those Heimdall numbers only got there because the benchmark existed. It found that
84
86
  imports were resolved file-wide rather than per-scope, that a Python `pkg/main.py`
@@ -196,10 +198,18 @@ Four passes, all precomputed at index time so a query is one SQL statement:
196
198
  Receivers are used wherever they carry information: `self.foo()` resolves inside the
197
199
  enclosing class, `utils.foo()` inside the file imported as `utils`, and `Model.foo()`
198
200
  inside the class `Model` — including when *several* classes are called `Model`, because
199
- the one that defines `foo` is the one that was meant. On Django that leaves 46,260 of
200
- 286,346 call edges resolved by name alone — and every one of them is labelled
201
+ the one that defines `foo` is the one that was meant. On Django that leaves 46,277 of
202
+ 350,104 call edges resolved by name alone — and every one of them is labelled
201
203
  `confidence = 0.3`.
202
204
 
205
+ A `CALLS` edge covers every way one symbol depends on another by name, not only
206
+ invocation: reading a constant, handing a function over as a value (`register(on_save)`,
207
+ `key=parse`), naming a class as a receiver, and a decorator. All of those break the same
208
+ way when the target changes, which is the question `impact` answers. Where they differ is
209
+ the fallback: an unresolvable *call* is emitted as up to five `0.3` candidates, while an
210
+ unresolvable *reference* is dropped entirely — a bare name with nothing behind it is
211
+ almost always a local variable, and guessing five ways at a local variable is noise.
212
+
203
213
  ### Schema
204
214
 
205
215
  ```sql
@@ -207,7 +217,7 @@ node(id, kind, name, qname, path, line, end_line, lang, signature)
207
217
  edge(src, dst, kind, confidence, line)
208
218
  ```
209
219
 
210
- `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`.
220
+ `kind` on nodes: `Folder`, `File`, `Class`, `Interface`, `Function`, `Method`, `Constant`.
211
221
  `kind` on edges: `CONTAINS`, `DEFINES`, `IMPORTS`, `CALLS`, `EXTENDS`.
212
222
 
213
223
  That is the whole data model — `heimdall sql` gives you direct access to it.
@@ -215,17 +225,43 @@ That is the whole data model — `heimdall sql` gives you direct access to it.
215
225
  ## Languages
216
226
 
217
227
  Python, JavaScript, TypeScript, TSX, Go, Rust, Java, Kotlin, C#, Ruby, PHP, C, C++,
218
- Swift, Scala, Lua, Dart, Bash. Every one of them is covered by a test that asserts it
219
- still yields symbols and call edges, so a grammar rename fails the build rather than
220
- silently degrading the graph.
228
+ Swift, Scala, Lua, Dart, Bash, Terraform. Every one of them is covered by a test that
229
+ asserts it still yields symbols and call edges, so a grammar rename fails the build
230
+ rather than silently degrading the graph.
221
231
 
222
- Adding a language means adding rows to three dicts in `heimdall/parse.py` — node types
223
- for definitions, for calls, and for imports — plus a snippet in the test.
232
+ Adding a language means adding rows to four dicts in `heimdall/parse.py` — node types
233
+ for definitions, for calls, for imports and for name bindings — plus a snippet in the test.
224
234
 
225
235
  One known gap: in Ruby, a receiver-less call without parentheses (`foo`) is
226
236
  indistinguishable from a variable reference in the grammar, so only `foo()` and
227
237
  `x.foo` produce call edges.
228
238
 
239
+ ### Terraform
240
+
241
+ `.tf` and `.hcl` are read differently, because HCL has no definitions and no calls. A
242
+ block is indexed under the address the rest of the configuration refers to it by —
243
+ `resource "aws_instance" "web"` becomes `aws_instance.web`, `variable "region"` becomes
244
+ `var.region`, and each `locals` attribute its own `local.x` — and every reference to one
245
+ of those addresses becomes a `CALLS` edge. The scope is Terraform's own: a directory of
246
+ `.tf` files is one module, and a `module` block's arguments resolve to the `variable`
247
+ blocks in the directory its `source` points at, which is what makes *what breaks if I
248
+ rename this module input* answerable.
249
+
250
+ ```console
251
+ heimdall context aws_instance.web # what it reads, and who reads it
252
+ heimdall impact var.region # everything downstream of one variable
253
+ ```
254
+
255
+ Blocks are stored under the existing `Class` kind rather than a new one — the schema's
256
+ six kinds are the same six for every language, and `Class` is already where Go's
257
+ `type_spec` and C's `struct` live.
258
+
259
+ Scored against a regex oracle on three public repos — 432 HCL files, 5,104 blocks, 10,206
260
+ edges — one reference of 2,828 goes unresolved and none of the edges is wrong; see
261
+ [bench/README.md](bench/README.md).
262
+ Packer and Nomad `.hcl` parse with the same grammar and get symbols, but only Terraform's
263
+ addressing rules are modelled.
264
+
229
265
  ## Limitations
230
266
 
231
267
  Call resolution is name-based, not type-based. `self.x()` and `module.x()` resolve
@@ -45,7 +45,7 @@ The *who writes into this table* question: a method name that several classes sh
45
45
 
46
46
  | | callers of the right one | callers of anything named `check` |
47
47
  |---|---|---|
48
- | Heimdall | 100% recall, 94% precision (109 returned) | 100% recall, 23% precision (442 returned) |
48
+ | Heimdall | 100% recall, 94% precision (110 returned) | 100% recall, 23% precision (453 returned) |
49
49
  | GitNexus | 0% recall, 0% precision (7 returned) | 0% recall, 0% precision (42 returned) |
50
50
  | graphify | 99% recall, 95% precision (107 returned) | 99% recall, 61% precision (168 returned) |
51
51
  | graphify `+uses` | 99% recall, 95% precision (107 returned) | 99% recall, 61% precision (168 returned) |
@@ -58,11 +58,11 @@ Blast radius at depth 3 for the five most-called symbols the oracle is sure abou
58
58
 
59
59
  | target | truth | Heimdall | GitNexus | graphify | graphify `+uses` |
60
60
  |---|---|---|---|---|---|
61
- | `utils.py:setup` | 912 | 100% of truth, 971 total | 0% of truth, 103 total | 0% of truth, 59 total | 0% of truth, 59 total |
62
- | `utils.py:override_settings` | 200 | 100% of truth, 1,792 total | 0% of truth, 0 total | 100% of truth, 1,549 total | 100% of truth, 1,623 total |
63
- | `safestring.py:mark_safe` | 338 | 100% of truth, 11,830 total | 100% of truth, 430 total | 97% of truth, 482 total | 97% of truth, 482 total |
64
- | `utils.py:isolate_apps` | 132 | 100% of truth, 201 total | 0% of truth, 0 total | 100% of truth, 201 total | 100% of truth, 201 total |
65
- | `utils.py:CaptureQueriesContext` | 91 | 100% of truth, 91 total | 0% of truth, 0 total | 99% of truth, 90 total | 99% of truth, 90 total |
61
+ | `utils.py:setup` | 912 | 100% of truth, 982 total | 0% of truth, 103 total | 0% of truth, 59 total | 0% of truth, 59 total |
62
+ | `exceptions.py:ValidationError` | 393 | 100% of truth, 1,503 total | 0% of truth, 0 total | 29% of truth, 122 total | 29% of truth, 122 total |
63
+ | `models.py:User` | 322 | 99% of truth, 2,527 total | 0% of truth, 0 total | 6% of truth, 20 total | 6% of truth, 20 total |
64
+ | `functional.py:cached_property` | 298 | 100% of truth, 14,504 total | 0% of truth, 0 total | 1% of truth, 3 total | 9% of truth, 7,545 total |
65
+ | `exceptions.py:ImproperlyConfigured` | 473 | 100% of truth, 9,413 total | 0% of truth, 0 total | 75% of truth, 444 total | 75% of truth, 444 total |
66
66
 
67
67
  ### Is any of this made up?
68
68
 
@@ -70,19 +70,19 @@ An edge `f -> g` is refuted when neither `g` nor anything imported *as* `g` occu
70
70
 
71
71
  | | refuted edges |
72
72
  |---|---|
73
- | Heimdall | 0 of 157,036 checkable (0.00%) |
73
+ | Heimdall | 0 of 204,713 checkable (0.00%) |
74
74
  | GitNexus | 0 of 23,736 checkable (0.00%) |
75
75
  | graphify | 202 of 30,882 checkable (0.65%) |
76
76
  | graphify `+uses` | 48,856 of 84,172 checkable (58.04%) |
77
77
 
78
78
  ### Would it tell me live code is dead?
79
79
 
80
- 4,668 symbols the oracle proves are called. A tool that reports no callers for one of them would send an agent to delete live code.
80
+ 6,952 symbols the oracle proves are called. A tool that reports no callers for one of them would send an agent to delete live code.
81
81
 
82
82
  | | symbols wrongly shown as uncalled |
83
83
  |---|---|
84
84
  | Heimdall | 1 (0.0%) |
85
- | GitNexus | 1,373 (29.4%) |
86
- | graphify | 205 (4.4%) |
87
- | graphify `+uses` | 199 (4.3%) |
85
+ | GitNexus | 3,635 (52.3%) |
86
+ | graphify | 2,381 (34.2%) |
87
+ | graphify `+uses` | 1,134 (16.3%) |
88
88
 
@@ -8,6 +8,7 @@ Two runners, because they measure different things:
8
8
  |---|---|---|
9
9
  | `compare.py` | are the *edges* right, and what do they cost? | [RESULTS.md](RESULTS.md) |
10
10
  | `questions.py` | is the *answer* right? | [QUESTIONS.md](QUESTIONS.md) |
11
+ | `terraform.py` | the same question for HCL, where the oracle is a regex | stdout |
11
12
 
12
13
  The repository is [Django](https://github.com/django/django) — 2,927 Python files, deep
13
14
  inheritance, decorators everywhere, and `tests/` full of same-named methods, which is the
@@ -67,6 +68,13 @@ sites where the language leaves no room for interpretation:
67
68
  | `self.m()` inside class `C` | `C` defines `m` — a receiver of `self` cannot mean another class |
68
69
  | `m()` where `m` is defined once at module level | one definition in scope, nothing shadowing it |
69
70
  | `m()` where `m` came from `from <mod> import m` | the binding names the module, and the module is a file in this repo |
71
+ | `m` *read* rather than called, same two rules | a constant or a function passed as a value is bound exactly as a call is |
72
+
73
+ A name read rather than called is scored separately from a call, and split again by what
74
+ it turned out to name: a `SCREAMING_SNAKE` module-level binding is a constant, anything
75
+ else that resolves to a definition is a function or class handed over as a value. Base
76
+ classes are excluded — `class A(B)` is an `EXTENDS` edge, and this runner only loads
77
+ `CALLS`, so counting them would score a question it never asked.
70
78
 
71
79
  Class instantiation (`Thing()`) is scored separately from function calls, because it is a
72
80
  call in Python but not every tool models it as a `CALLS` edge — folding the two together
@@ -88,11 +96,13 @@ qualified-name convention, so neither tool's naming scheme wins by default.
88
96
 
89
97
  | # | Question | Metric |
90
98
  |---|---|---|
91
- | 1 | Does `self.m()` bind to the enclosing class? | recall over 5,389 sites |
99
+ | 1 | Does `self.m()` bind to the enclosing class? | recall over 5,391 sites |
92
100
  | 2 | Does a same-file call bind to the right function? | recall over 1,255 sites |
93
101
  | 3 | Does a same-file class instantiation produce an edge? | recall over 943 sites |
94
- | 4 | Does an imported call bind to the right file? | recall over 4,238 sites |
102
+ | 4 | Does an imported call bind to the right file? | recall over 4,242 sites |
95
103
  | 5 | Does an imported class instantiation produce an edge? | recall over 4,299 sites |
104
+ | 5a | Is a constant read an edge, same file / imported? | recall over 765 / 659 sites |
105
+ | 5b | Is a function *passed* rather than called an edge? | recall over 1,196 / 10,779 sites |
96
106
  | 6 | How many candidates does the answer arrive buried in? | mean fan-out per resolved site |
97
107
  | 7 | Who reaches these 5 hot symbols within 3 call hops? | set size, overlap, wall time |
98
108
  | 8 | Does clustering beat the directory tree? | Newman modularity, shared subgraph |
@@ -107,6 +117,43 @@ Case 9 is run twice — once on Heimdall's call graph and once on GitNexus's —
107
117
  label propagation is fitted on the graph it is then scored on, and Leiden is not. The
108
118
  second table is the one where Leiden has home advantage.
109
119
 
120
+ ## Terraform (`terraform.py`)
121
+
122
+ No competitor here — neither of the other two reads HCL — so this one only asks whether
123
+ Heimdall's own edges are right. The oracle is a regular expression, chosen precisely
124
+ because it shares no code with the tree-sitter reader it grades.
125
+
126
+ ```console
127
+ git clone --depth 1 https://github.com/terraform-aws-modules/terraform-aws-eks
128
+ git clone --depth 1 https://github.com/terraform-aws-modules/terraform-aws-vpc
129
+ git clone --depth 1 https://github.com/terraform-google-modules/terraform-example-foundation
130
+ python bench/terraform.py terraform-aws-eks terraform-aws-vpc terraform-example-foundation
131
+ ```
132
+
133
+ | repo | files | blocks | refs | edges | recall | precision | mean confidence |
134
+ |---|---|---|---|---|---|---|---|
135
+ | terraform-aws-eks | 90 | 1,363 | 950 | 2,989 | 100.0% | 100.0% | 0.87 |
136
+ | terraform-aws-vpc | 77 | 1,872 | 472 | 3,461 | 99.8% | 100.0% | 0.84 |
137
+ | terraform-example-foundation | 265 | 1,869 | 1,406 | 3,756 | 100.0% | 100.0% | 0.85 |
138
+ | **total** | **432** | **5,104** | **2,828** | **10,206** | **100.0%** | **100.0%** | |
139
+
140
+ *recall* — a `var.X` / `local.X` / `module.X` / `data.T.N` written in a directory where
141
+ exactly one file declares that block must produce an edge to that file. *precision* — an
142
+ edge is endorsed only if its target really declares that address and the target is either
143
+ a sibling of the source or the directory a `module` block points `source` at.
144
+
145
+ The one row that falls short is worth keeping: the missing reference is `var.azs` quoted
146
+ inside a `description` string in terraform-aws-vpc, where the oracle cannot tell prose
147
+ from code and Heimdall is right not to link it. Precision was 99.8% until references
148
+ stopped being guessed at — the 17 wrong edges were all in terraform-aws-eks's
149
+ `modules/karpenter/migrations.tf`, which names resources declared in no directory the
150
+ oracle can reach, and all 17 were name-only matches that are now dropped rather than
151
+ emitted at `confidence = 0.3`.
152
+
153
+ Bare resource references (`aws_s3_bucket.logs.arn`) are excluded from the recall claim —
154
+ no regex can tell one from an attribute lookup on something else — but the edges Heimdall
155
+ emits for them are still graded for precision.
156
+
110
157
  ## Edge cases (`questions.py`, part 1)
111
158
 
112
159
  Cases 1–11 are aggregates over a real repository, which is the right way to measure the
@@ -8,25 +8,29 @@ Repo: `django` — 2927 Python files. Oracle: CPython `ast`, restricted to call
8
8
 
9
9
  | Call kind | sites | Heimdall recall | fan-out | GitNexus recall | fan-out | graphify recall | fan-out | graphify `+uses` recall | fan-out |
10
10
  |---|---|---|---|---|---|---|---|---|---|
11
- | `self` | 5389 | 100.0% | 1.00 | 97.1% | 1.00 | 92.6% | 1.00 | 92.6% | 1.00 |
11
+ | `self` | 5391 | 100.0% | 1.01 | 97.1% | 1.00 | 92.6% | 1.00 | 92.6% | 1.00 |
12
12
  | `local call` | 1255 | 100.0% | 1.06 | 93.4% | 1.00 | 89.3% | 1.00 | 89.3% | 1.00 |
13
13
  | `local new` | 943 | 99.5% | 1.00 | 0.0% | nan | 95.2% | 1.00 | 95.2% | 1.00 |
14
- | `import call` | 4238 | 99.7% | 1.02 | 71.5% | 1.00 | 75.4% | 1.00 | 75.4% | 1.00 |
14
+ | `local const` | 765 | 100.0% | 1.00 | 0.0% | nan | 0.0% | nan | 0.0% | nan |
15
+ | `local ref` | 1196 | 97.8% | 1.02 | 1.9% | 1.00 | 13.8% | 1.00 | 13.8% | 1.00 |
16
+ | `import call` | 4242 | 99.7% | 1.02 | 71.4% | 1.00 | 75.3% | 1.00 | 75.3% | 1.00 |
15
17
  | `import new` | 4299 | 98.8% | 1.03 | 0.0% | 1.00 | 98.2% | 1.00 | 98.2% | 1.00 |
18
+ | `import const` | 659 | 100.0% | 1.02 | 0.0% | nan | 0.0% | nan | 0.0% | nan |
19
+ | `import ref` | 10779 | 96.9% | 1.00 | 0.1% | 1.00 | 5.3% | 1.00 | 5.3% | 1.00 |
16
20
 
17
- Total CALLS edges: Heimdall 159,132, GitNexus 24,190, graphify 31,289, graphify `+uses` 84,579
21
+ Total CALLS edges: Heimdall 208,517, GitNexus 24,190, graphify 31,289, graphify `+uses` 84,579
18
22
 
19
23
  ## Is Leiden clustering worth it?
20
24
 
21
- Newman modularity of each grouping on one shared subgraph (8,936 symbols, 36,310 call edges) — the symbols Leiden placed, so no partition is scored on edges another never saw. Leiden was fitted on GitNexus's own graph and is being replayed on Heimdall's.
25
+ Newman modularity of each grouping on one shared subgraph (9,850 symbols, 41,274 call edges) — the symbols Leiden placed, so no partition is scored on edges another never saw. Leiden was fitted on GitNexus's own graph and is being replayed on Heimdall's.
22
26
 
23
27
  | Grouping | groups | modularity |
24
28
  |---|---|---|
25
- | Leiden (GitNexus) | 1,541 | 0.257 |
26
- | directory | 298 | 0.224 |
27
- | file | 1,006 | 0.194 |
28
- | label propagation (stdlib, ~15 lines) | 74 | 0.188 |
29
- | shuffled control | 1,541 | 0.016 |
29
+ | Leiden (GitNexus) | 1,642 | 0.257 |
30
+ | directory | 308 | 0.242 |
31
+ | file | 1,154 | 0.203 |
32
+ | label propagation (stdlib, ~15 lines) | 87 | 0.187 |
33
+ | shuffled control | 1,642 | 0.016 |
30
34
 
31
35
  Same question on GitNexus's *own* graph (15,573 edges), where Leiden was actually fitted:
32
36
 
@@ -40,10 +44,10 @@ Same question on GitNexus's *own* graph (15,573 edges), where Leiden was actuall
40
44
 
41
45
  | | callers found | shared with the other | wall time |
42
46
  |---|---|---|---|
43
- | Heimdall (SQLite recursive CTE, in-process) | 2,047 | 1,904 | 13 ms |
44
- | GitNexus (LadybugDB Cypher, node subprocess) | 1,904 | 1,904 | 395 ms |
47
+ | Heimdall (SQLite recursive CTE, in-process) | 2,063 | 1,904 | 14 ms |
48
+ | GitNexus (LadybugDB Cypher, node subprocess) | 1,904 | 1,904 | 419 ms |
45
49
 
46
- Targets: `django/contrib/contenttypes/fields.py:create`, `django/contrib/sessions/backends/cache.py:create`, `django/contrib/sessions/backends/base.py:create`, `django/apps/config.py:create`, `django/test/testcases.py:assertRaisesMessage`
50
+ Targets: `django/contrib/contenttypes/fields.py:create`, `django/contrib/sessions/backends/base.py:create`, `django/contrib/sessions/backends/cache.py:create`, `django/apps/config.py:create`, `django/test/testcases.py:assertRaisesMessage`
47
51
 
48
52
  The two answers differ mostly because the graphs differ, not because the traversals do — see the call-resolution table above. graphify is absent from this table on purpose: its traversal is in-process Python over the JSON graph, so the row would mostly measure how long that file takes to load, which the Cost table already reports.
49
53
 
@@ -51,8 +55,8 @@ The two answers differ mostly because the graphs differ, not because the travers
51
55
 
52
56
  | | Heimdall | GitNexus | graphify |
53
57
  |---|---|---|---|
54
- | index from scratch | 1.7 s | 23.0 s | 28.8 s |
55
- | re-index, nothing changed | 1.7 s | 17.9 s | 28.6 s |
56
- | index on disk | 48 MB | 642 MB | 151 MB |
57
- | `impact create` from cold, median of 5 | 56 ms | 949 ms | 856 ms |
58
+ | index from scratch | 2.7 s | 23.6 s | 29.8 s |
59
+ | re-index, nothing changed | 2.7 s | 18.6 s | 29.8 s |
60
+ | index on disk | 78 MB | 730 MB | 151 MB |
61
+ | `impact create` from cold, median of 5 | 58 ms | 969 ms | 837 ms |
58
62