codeflowmap 0.1.0 β†’ 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,101 +1,128 @@
1
- # code-mapper
1
+ <div align="center">
2
2
 
3
- Map your codebase's **dependencies** and **data flows** (writes / reads / auth) into an
4
- Obsidian-linkable Markdown vault, with a web UI to navigate and highlight paths.
3
+ # πŸ—ΊοΈ codeflowmap
5
4
 
6
- > Open source (MIT). Hybrid architecture: deterministic static analysis builds the graph,
7
- > an open-weight LLM adds the semantic layer.
8
- >
9
- > ⚠️ **Early / experimental** β€” interfaces and output format may change.
5
+ ### Map your codebase's **dependencies** and **data flows** β€” what writes, what reads, what's behind auth.
6
+
7
+ <p align="center">
8
+ <a href="https://www.npmjs.com/package/codeflowmap"><img alt="npm" src="https://img.shields.io/npm/v/codeflowmap?logo=npm&color=cb3837"></a>
9
+ <a href="LICENSE"><img alt="license: MIT" src="https://img.shields.io/badge/license-MIT-blue"></a>
10
+ <img alt="runtime: Bun" src="https://img.shields.io/badge/runtime-Bun-000?logo=bun&logoColor=white">
11
+ <img alt="status: experimental" src="https://img.shields.io/badge/status-early%20%2F%20experimental-f59e0b">
12
+ </p>
13
+
14
+ <p align="center">
15
+ <em>Deterministic static analysis builds the graph; an open-weight LLM adds the semantic layer.</em>
16
+ </p>
17
+
18
+ <!-- Swap this for a real demo gif/png whenever you grab one (scan β†’ graph β†’ focus β†’ annotate). -->
19
+ <p align="center">
20
+ <img src="https://raw.githubusercontent.com/man-consult/code-mapper/main/assets/hero.svg" alt="codeflowmap: focus a file to see its callers (left), dependencies (right), and its writes / reads / auth" width="860">
21
+ </p>
22
+
23
+ ```bash
24
+ bunx codeflowmap serve # β†’ http://localhost:4321 Β· no install, no clone
25
+ ```
26
+
27
+ </div>
28
+
29
+ ---
10
30
 
11
31
  ## Why this exists
12
32
 
13
- I wanted to get better at reading code I didn't write. Two kinds, really. The first: understanding the code an LLM wrote for me. Generating it is the fast part. Getting across what it actually did (what it touched, what it reads and writes, whether the auth path is where I think it is) was the gap I needed to fix. This tool is my way of improving the verification step. Generate, then graph it and see what's really there.
33
+ I wanted to get better at reading code I didn't write.
34
+
35
+ Firstly: understanding the code an LLM wrote for me. I pretty soon discovered that bashing out a few prompts and getting an LLM to compile some code isn't enough. Getting across what it actually did (what it touched, what it reads and writes, whether the auth path is where I think it is) was the gap I needed to fix. This tool is my way of improving the verification step. Generate, then graph it and see what's really there.
14
36
 
15
37
  The second is dropping into an unfamiliar repo and tracing how it stitches together, what calls what, where data gets written, which paths run through auth. Slow work, and easy to get wrong when it's not your local playground. This part is all about learning good patterns from people who know what they're doing.
16
38
 
39
+ The next challenge was knowing where to start, which I'd been answering with a second tool that worked in conjunction with the graph. So I folded it in, and now it'll generate a guided tour of the codebase to go with the graph.
40
+
17
41
  It's experimental because it was built to be useful to me first, not to be a finished product. If it helps you better understand a codebase the same way, good.
18
42
 
43
+ ---
19
44
 
20
- ## Requirements
45
+ ## What you get
21
46
 
22
- - [Bun](https://bun.sh) β‰₯ 1.0 β€” the runtime (the tool uses Bun's built-in server).
23
- - *(optional, for the annotation layer)* a local [Ollama](https://ollama.com) install, or a
24
- Mistral / OpenAI API key. The graph itself needs no model.
47
+ | | |
48
+ |---|---|
49
+ | πŸ”— **Exact dependency graph** | File-level imports resolved by the TypeScript compiler (aliases, barrels, index files) β€” not guessed. |
50
+ | πŸ” **Function-level call graph** | Who calls what, symbol to symbol, for TS/JS and Python. Toggle between the **Files** and **Functions** views. |
51
+ | 🧭 **Focus any node** | Click (or search) a file/function β†’ a clean, directional view: callers on the left, dependencies on the right. Big neighbourhoods collapse by directory. |
52
+ | 🧠 **Semantic layer (optional)** | An LLM annotates each file: a summary plus **writes / reads / config / auth / flows** β€” the meaning the graph can't infer. |
53
+ | 🧾 **Flow-trace** | Aggregates the data footprint across a whole capability: *"this route writes these tables, reads these, is gated by this auth."* |
54
+ | 🧭 **Guided tour** *(new in 0.1.2)* | A suggested reading order for an unfamiliar repo, built from the import graph: foundations first, or entry points first. Step through it in the UI or read `TOUR.md`. |
55
+ | πŸ““ **Obsidian-linkable vault** | One `.md` per file with `[[wikilinks]]` that are real edges β€” open the output folder as an Obsidian vault for free. |
25
56
 
26
- ## Install
57
+ ---
27
58
 
28
- No clone needed:
59
+ ## Quick start
29
60
 
30
61
  ```bash
31
62
  bunx codeflowmap serve # opens http://localhost:4321
32
63
  ```
33
64
 
34
- Then, in the browser: **Browse** to a repo β†’ **Scan** β†’ explore. Optionally paste a Mistral API
35
- key and click **Annotate** for the semantic layer. (See [Usage](#usage) for the full walkthrough.)
65
+ Then, in the browser:
66
+
67
+ 1. **Scan** β€” click **Browse…** to pick a folder (or type/paste a path; `~` works), then **Scan**. The import graph, call graph, and Obsidian vault are written to `<repo>/.codemap`.
68
+ 2. **Annotate** *(optional)* β€” add the semantic layer. Leave the key blank to run locally via **Ollama**, or paste an **API key** + base URL for any OpenAI-compatible provider (Mistral, OpenAI, …).
69
+ 3. **Explore** β€” toggle **Files / Functions**, click a node to focus its neighbourhood and highlight up/downstream paths, search to jump anywhere, and filter by data-flow type.
70
+ 4. **Take the tour** β€” open the **guided tour** panel to step through the repo in reading order, and flip between **foundations-first** and **entry-points-first**.
71
+
72
+ Open the same `.codemap/` folder as an Obsidian vault for the graph view + backlinks.
73
+ Add `.codemap/` to your `.gitignore` if you don't want it committed.
74
+
75
+ ---
36
76
 
37
77
  ## How it works
38
78
 
39
79
  ```
40
- AST / static analysis β†’ import graph + function-level call graph (deterministic, no tokens)
80
+ AST / static analysis β†’ import graph + function-level call graph (deterministic, no tokens)
41
81
  ↓
42
- LLM annotation β†’ writes / reads / auth / flows (provider-agnostic; default: local Ollama)
82
+ LLM annotation β†’ writes / reads / config / auth / flows (provider-agnostic; default: local Ollama)
43
83
  ↓
44
- Markdown vault β†’ one .md per file, [[wikilinks]] = real edges (link into Obsidian)
84
+ Markdown vault β†’ one .md per file, [[wikilinks]] = real edges (link into Obsidian)
45
85
  ↓
46
- Web UI β†’ Files / Functions views, highlight up/downstream paths, filter by flow
86
+ Web UI β†’ Files / Functions views, focus + flow-trace, filter by flow
47
87
  ```
48
88
 
49
- ## Status
50
-
51
- | Phase | What | State |
52
- |-------|------|-------|
53
- | 1 | Core graph (TS/JS via ts-morph) β†’ `graph.json` + Obsidian vault | βœ… |
54
- | 2 | Tree-sitter Python extractor (imports + symbols, AST-accurate) | βœ… |
55
- | 3 | LLM annotation pass (provider-agnostic, local-first via Ollama) | βœ… |
56
- | 4 | Vite + React web UI over `graph.json` | βœ… |
57
- | 5 | Function-level call graph for TS/JS (`Functions` view) | βœ… |
89
+ > **Graph edges are never inferred by the LLM.** Imports come from real module resolution and call edges from symbol resolution (TypeScript Compiler API), so they're exact, free, and CI-reproducible. The model only adds the semantic layer.
58
90
 
59
- Python currently contributes import edges + symbols; function-level call edges are TS/JS only.
91
+ ---
60
92
 
61
- ## Usage
93
+ ## Requirements
62
94
 
63
- Everything runs from the browser. The only command is launching the UI (`bunx codeflowmap serve`,
64
- or `bun run codemap serve` from a source checkout). Then, in the browser:
95
+ - [**Bun**](https://bun.sh) β‰₯ 1.0 β€” the runtime (the tool uses Bun's built-in server).
96
+ - *(optional, for annotation)* a local [**Ollama**](https://ollama.com) install **or** an API key for any OpenAI-compatible provider. The graph itself needs no model.
65
97
 
66
- 1. **Scan** β€” click **Browse…** to pick a folder (or type/paste a path; `~` works), then **Scan**.
67
- The import graph, call graph, and Obsidian vault are written to `<repo>/.codemap`.
68
- 2. **Explore** β€” toggle **Files / Functions**, click a node to highlight its up/downstream (or
69
- caller/callee) paths, search, and filter by data-flow type.
70
- 3. **Annotate** *(optional)* β€” paste a **Mistral API key**, click **Annotate** to add the semantic
71
- layer (summary + writes/reads/auth/flows). The key is held in memory and sent only to your local
72
- server, which calls Mistral and writes the result back to the vault β€” never stored in the browser
73
- or on disk.
98
+ ---
74
99
 
75
- Open the same `.codemap/` folder as an Obsidian vault for the graph view + backlinks.
76
- Add `.codemap/` to your `.gitignore` if you don't want it sent to the cloud...
100
+ <details>
101
+ <summary><b>Scriptable / CI usage</b> β€” the same operations as plain CLI commands</summary>
77
102
 
78
- ### Scriptable / CI (optional)
79
-
80
- The same operations are plain CLI commands, for automation or a local-model workflow:
103
+ <br>
81
104
 
82
105
  ```bash
83
106
  bun run codemap scan ./repo -o .codemap # graph + vault, deterministic, no tokens
84
107
  ollama pull qwen2.5-coder:7b # one-time, ~4.7 GB
85
108
  bun run codemap annotate -o .codemap # local Ollama (default)
86
109
  bun run codemap annotate -o .codemap -p mistral # Mistral API (needs CODEMAP_API_KEY)
110
+ bun run codemap onboard ./repo -o .codemap # guided reading order β†’ .codemap/TOUR.md
111
+ bun run codemap onboard ./repo --order entrypoints # …or start from the entry points
87
112
  ```
88
113
 
89
- ### Annotation providers
114
+ From npm, swap `bun run codemap` for `bunx codeflowmap` (e.g. `bunx codeflowmap onboard ./repo`).
115
+
116
+ Annotations are cached by file-content hash under `.codemap/.cache`, so re-runs only re-send changed files.
117
+
118
+ </details>
90
119
 
91
- Annotation hits a real model. Configured by flags or environment; secrets are **read only** from
92
- the environment and are never written to the vault, `graph.json`, the cache, or logs.
120
+ <details>
121
+ <summary><b>Annotation providers & data egress</b> β€” read before annotating real code</summary>
93
122
 
94
- > ⚠️ **Data egress:** annotating sends the **full contents of every scanned file** to the
95
- > configured provider. With `mistral` / `openai` (or any remote `--base-url`) that means your
96
- > source code leaves the machine and is processed by a third party β€” do not annotate
97
- > proprietary or sensitive code against a remote provider without authorization. For a fully
98
- > local workflow that sends nothing off-box, use the default `ollama` provider.
123
+ <br>
124
+
125
+ > ⚠️ **Data egress:** annotating sends the **full contents of every scanned file** to the configured provider. With `mistral` / `openai` (or any remote `--base-url`) your source code leaves the machine and is processed by a third party β€” don't annotate proprietary or sensitive code against a remote provider without authorization. For a fully local workflow that sends nothing off-box, use the default **`ollama`** provider.
99
126
 
100
127
  | Provider | Endpoint | Default model | Key |
101
128
  |----------|----------|---------------|-----|
@@ -103,35 +130,63 @@ the environment and are never written to the vault, `graph.json`, the cache, or
103
130
  | `mistral` | `https://api.mistral.ai/v1` | `devstral-small-2512` | `CODEMAP_API_KEY` / `MISTRAL_API_KEY` |
104
131
  | `openai` | `https://api.openai.com/v1` | `gpt-4o-mini` | `CODEMAP_API_KEY` / `OPENAI_API_KEY` |
105
132
 
106
- Env overrides: `CODEMAP_PROVIDER`, `CODEMAP_MODEL`, `CODEMAP_BASE_URL`, `CODEMAP_API_KEY`.
107
- Any OpenAI-compatible endpoint works via `--base-url`. Annotations are cached by file-content
108
- hash under `.codemap/.cache`, so re-runs only re-send changed files.
133
+ Env overrides: `CODEMAP_PROVIDER`, `CODEMAP_MODEL`, `CODEMAP_BASE_URL`, `CODEMAP_API_KEY`. Any OpenAI-compatible endpoint works via `--base-url`. Secrets are read **only** from the environment β€” never written to the vault, `graph.json`, the cache, or logs.
134
+
135
+ **Model sizing:** `qwen2.5-coder:7b` (~4.7 GB, Apache-2.0) is the default β€” it runs on a 16 GB laptop and the per-file task doesn't need more. For higher-quality summaries on a 32 GB+ machine, use `-m devstral` (24B). For 8 GB, `-m qwen2.5-coder:3b`.
136
+
137
+ </details>
138
+
139
+ ---
140
+
141
+ ## Status
109
142
 
110
- **Model sizing:** `qwen2.5-coder:7b` (~4.7 GB, Apache-2.0) is the default because it runs on a
111
- 16 GB laptop and the per-file task doesn't need more. For higher-quality summaries on a 32 GB+
112
- machine, use `-m devstral` (24B). For 8 GB, `-m qwen2.5-coder:3b`.
143
+ | | Capability | State |
144
+ |---|---|---|
145
+ | βœ… | Import graph (TS/JS via ts-morph) β†’ `graph.json` + Obsidian vault | done |
146
+ | βœ… | Tree-sitter Python extractor (imports + symbols) | done |
147
+ | βœ… | Function-level call graph for TS/JS (**Functions** view) | done |
148
+ | βœ… | LLM annotation β€” provider-agnostic, local-first | done |
149
+ | βœ… | Web UI (Cytoscape): focus, directory grouping, flow-trace | done |
150
+ | βœ… | Function-level call edges for Python (resolved through imports) | done |
151
+ | βœ… | Guided tour / repo onboarding (`onboard` + web tour) | done |
113
152
 
114
- ## Design notes
153
+ > **Early / experimental** β€” interfaces and output format may change. Function-level call edges now cover both TS/JS and Python; Python calls are resolved through imports (including packages under a source root such as `src/`) and verified against the target file's symbols.
115
154
 
116
- - **Graph edges are never inferred by the LLM** β€” imports come from real module resolution and
117
- call edges from symbol resolution (TypeScript Compiler API), so they're exact, free, and
118
- CI-reproducible. The LLM only adds the semantic layer.
119
- - **The LLM is provider-agnostic** behind one `Annotator` interface. Default is local, open-weight
120
- **Qwen2.5-Coder 7B** (Apache-2.0) via Ollama; Mistral / OpenAI / any OpenAI-compatible endpoint
121
- are drop-in alternatives via `--provider` / `--base-url`.
155
+ ---
122
156
 
123
157
  ## Development
124
158
 
125
159
  ```bash
126
160
  git clone https://github.com/man-consult/code-mapper.git && cd code-mapper
127
161
  bun install
128
- bun run codemap serve # runs from source; rebuilds the UI on first run
162
+ bun run codemap serve # runs from source; builds the UI on first run
129
163
  ```
130
164
 
131
- Monorepo layout (Bun workspaces): `packages/core` (graph extraction), `packages/annotate`
132
- (LLM layer), `packages/cli` (the `serve` command), `packages/web` (Cytoscape UI).
133
- `bun run build:pkg` assembles the publishable `code-mapper` package into `pkg/`.
165
+ Monorepo (Bun workspaces): `packages/core` (graph extraction) Β· `packages/annotate` (LLM layer) Β· `packages/cli` (`serve`, `scan`, `annotate`, `onboard`) Β· `packages/web` (Cytoscape UI). `bun run build:pkg` assembles the publishable npm package into `pkg/`.
166
+
167
+ ---
134
168
 
135
169
  ## License
136
170
 
137
171
  [MIT](LICENSE) Β© 2026 Brian Mangano
172
+
173
+ ---
174
+
175
+ ## Changelog
176
+
177
+ ### 0.1.2
178
+
179
+ Folds in a guided reading order for dropping into an unfamiliar repo, built on the same dependency graph asked a different question. Python also gains function-level call edges.
180
+
181
+ - **Guided tour / repo onboarding.** `codemap onboard <dir>` writes a `TOUR.md` reading order, and the web UI gains a guided-tour panel that steps through the repo stop by stop. The order is derived from the import graph β€” cycles folded, dependencies introduced before the files that use them β€” with a **foundations-first ↔ entry-points-first** toggle for two ways to tell the story.
182
+ - **Python function-level call edges.** The **Functions** view now works for Python. Calls are resolved through imports and verified against the target file's declared symbols, so edges stay exact rather than guessed.
183
+ - **Python source-root import resolution.** Absolute imports for packages living under a source root (e.g. `backend/src/myapp/…`) now resolve by suffix-matching the scanned file set β€” fixing both import and call edges for "src layout" projects.
184
+ - **Housekeeping.** Fixed a few bugs, removed a couple of dependencies (`zod` is replaced by a small built-in check on model output, and `@types/cytoscape` is no longer needed), and bumped a few others to clear some security advisories. `bun audit` is now clean.
185
+
186
+ **NOTE** Repos analysed before 0.1.2 need a fresh scan to surface the Guided Tour and Python call edges as both come from static analysis. A re-scan rewrites graph.json and clears existing summaries, so re-run annotate afterwards to restore them (cached by file content, so it's cheap).
187
+
188
+ ### 0.1.1
189
+ - Packaging and security hardening: the local server is loopback-only with a Host/Origin guard against DNS-rebinding and cross-origin requests.
190
+
191
+ ### 0.1.0
192
+ - Initial release: deterministic import graph + function-level call graph (TS/JS) via the TypeScript compiler, tree-sitter Python extractor (imports + symbols), provider-agnostic LLM annotation (local-first), Cytoscape web UI, and Obsidian-linkable vault.