repo-blueprint 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -2
- package/README.md +15 -8
- package/bin/repo-blueprint.mjs +1 -1
- package/package.json +7 -2
- package/skills/repo-blueprint/SKILL.md +9 -9
- package/src/blueprint.config.json +26 -8
- package/src/build.mjs +5 -2
- package/src/extract-py.py +34 -15
- package/src/extract-ts.mjs +154 -15
- package/src/model.mjs +29 -9
- package/src/template.html +18 -7
- package/src/views.mjs +4 -3
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "repo-blueprint",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "A clearer way to see how a repository is built. Maps TypeScript, JavaScript and Python codebases into dependency layers you can zoom through.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "Mo Ganji"
|
package/README.md
CHANGED
|
@@ -2,11 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
A clearer way to see how a repository is built. repo-blueprint maps a codebase into layers, with each part placed above its dependencies. Detail is handled by zoom: the top level shows packages, and zooming in reveals folders, then files.
|
|
4
4
|
|
|
5
|
-

|
|
6
6
|
|
|
7
7
|
- Works with TypeScript, JavaScript and Python, monorepos included
|
|
8
|
-
- Draws imports, RPC and HTTP calls, events, and
|
|
8
|
+
- Draws imports, RPC and HTTP calls, events, and every third-party package the code imports, named by system (PostgreSQL, Stripe…) when it is a known driver or SDK
|
|
9
9
|
- Flags import cycles, unused packages and orphan files
|
|
10
|
+
- Reports every file it could not map, and every import it could not follow, so nothing drops off the map silently
|
|
10
11
|
- Produces one self-contained HTML file
|
|
11
12
|
- Uses static analysis only, so the output is deterministic
|
|
12
13
|
|
|
@@ -36,8 +37,8 @@ Serves the map on localhost and rebuilds it when files change.
|
|
|
36
37
|
|
|
37
38
|
## Supported projects
|
|
38
39
|
|
|
39
|
-
- **TypeScript and JavaScript:** any folder with a `package.json
|
|
40
|
-
- **Python:** any folder with a `pyproject.toml
|
|
40
|
+
- **TypeScript and JavaScript:** any folder with a `package.json`, at any depth. ES modules and CommonJS (`require`, `module.exports`) are both read. Imports are resolved through the package's `tsconfig.json` or `jsconfig.json` when it has one. Vue, Svelte and Astro files are not parsed yet and are reported as not mapped.
|
|
41
|
+
- **Python:** any folder with a `pyproject.toml`, flat or `src/` layout.
|
|
41
42
|
|
|
42
43
|
Each package is shown as an app, a service or a library. A declared type is used when there is one (Backstage `catalog-info.yaml`, Nx `project.json`). Otherwise the role is inferred: an entry file that mounts a UI makes an app, one that starts a server or worker makes a service, and anything else is a library.
|
|
43
44
|
|
|
@@ -53,21 +54,25 @@ You can also copy [`SKILL.md`](https://github.com/MohGanji/repo-blueprint/blob/m
|
|
|
53
54
|
|
|
54
55
|
## Configuration
|
|
55
56
|
|
|
56
|
-
Put a `.blueprint.json` in the root of the repo being mapped to extend the [defaults](https://github.com/MohGanji/repo-blueprint/blob/main/src/blueprint.config.json). For example, to
|
|
57
|
+
None is needed: every imported package is on the map without it. Put a `.blueprint.json` in the root of the repo being mapped to extend the [defaults](https://github.com/MohGanji/repo-blueprint/blob/main/src/blueprint.config.json). For example, to group your own SDKs under one system name:
|
|
57
58
|
|
|
58
59
|
```json
|
|
59
60
|
{
|
|
60
|
-
"externalSystems": { "
|
|
61
|
+
"externalSystems": { "Billing API": ["@acme/billing-client"] }
|
|
61
62
|
}
|
|
62
63
|
```
|
|
63
64
|
|
|
64
65
|
| Key | Description |
|
|
65
66
|
| --- | --- |
|
|
66
|
-
| `externalSystems` | System names and the packages that indicate them |
|
|
67
|
+
| `externalSystems` | System names and the packages that indicate them. Labels only: a package not listed still appears under its own name |
|
|
67
68
|
| `messaging` | Method and property names for RPC calls, publishes and subscriptions |
|
|
68
69
|
| `folderRoles` | Folder names that imply a role, used by the role-mismatch finding |
|
|
69
70
|
| `sdkDirs` | Folders whose subfolders are generated API clients, named after the services they call |
|
|
70
71
|
|
|
72
|
+
## Examples
|
|
73
|
+
|
|
74
|
+
[`examples/`](https://github.com/MohGanji/repo-blueprint/tree/main/examples) holds blueprints of open-source projects, each pinned to a commit in [`corpus.json`](https://github.com/MohGanji/repo-blueprint/blob/main/examples/corpus.json): Express, Koa, debug, Chalk, nanoid, Ky, Zustand and Zod from npm, and Requests, Click, HTTPX, Flask, Rich, Pydantic, FastAPI, Typer, SQLModel, Instructor, LiteLLM and Karpathy's nanochat from PyPI and GitHub. Download one and open it in a browser. `node examples/build-examples.mjs` rebuilds them.
|
|
75
|
+
|
|
71
76
|
## How it works
|
|
72
77
|
|
|
73
78
|
The TypeScript compiler API and Python's `ast` module extract facts from the source. Those become a model of packages, folders, edges and findings, which is rendered as one diagram per folder.
|
|
@@ -75,13 +80,15 @@ The TypeScript compiler API and Python's `ast` module extract facts from the sou
|
|
|
75
80
|
```
|
|
76
81
|
bin/repo-blueprint.mjs CLI
|
|
77
82
|
src/
|
|
78
|
-
├── extract-ts.mjs TypeScript extractor
|
|
83
|
+
├── extract-ts.mjs TypeScript and JavaScript extractor
|
|
79
84
|
├── extract-py.py Python extractor
|
|
80
85
|
├── model.mjs packages, edges, roles, findings
|
|
81
86
|
├── views.mjs one diagram per folder
|
|
82
87
|
├── build.mjs writes the HTML
|
|
83
88
|
└── template.html the page
|
|
84
89
|
skills/repo-blueprint/ agent skill
|
|
90
|
+
test/ fixture repos and tests (npm test)
|
|
91
|
+
examples/ blueprints of open-source repos
|
|
85
92
|
```
|
|
86
93
|
|
|
87
94
|
## Contributing
|
package/bin/repo-blueprint.mjs
CHANGED
|
@@ -12,7 +12,7 @@ const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
|
|
|
12
12
|
const SRC = path.join(ROOT, 'src');
|
|
13
13
|
const HOME = path.join(os.homedir(), '.repo-blueprint');
|
|
14
14
|
const SKIP_DIRS = new Set(['node_modules', '.git', 'dist', 'build', 'coverage', '.venv', 'venv', '.nx', 'tmp', '.next', 'out', '.turbo']);
|
|
15
|
-
const WATCHED = /\.(tsx?|py)$|(^|\/)(package\.json|
|
|
15
|
+
const WATCHED = /\.(tsx?|jsx?|mts|cts|mjs|cjs|py)$|(^|\/)(package\.json|[tj]sconfig[\w.-]*\.json|pyproject\.toml|project\.json|catalog-info\.ya?ml|\.blueprint\.json|Dockerfile)$/;
|
|
16
16
|
const IGNORED = /(^|\/)(node_modules|\.git|dist|build|coverage|\.venv|venv|__pycache__|\.nx|tmp|\.next|out|\.turbo|playwright-report|test-results)(\/|$)/;
|
|
17
17
|
|
|
18
18
|
const HELP = `repo-blueprint: map a code repository by static analysis.
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "repo-blueprint",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "A clearer way to see how a repository is built. Maps TypeScript, JavaScript and Python codebases into dependency layers you can zoom through.",
|
|
5
5
|
"author": "Mo Ganji <mo@ganji.me>",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"repository": {
|
|
@@ -16,6 +16,9 @@
|
|
|
16
16
|
"bin": {
|
|
17
17
|
"repo-blueprint": "bin/repo-blueprint.mjs"
|
|
18
18
|
},
|
|
19
|
+
"scripts": {
|
|
20
|
+
"test": "node --test test/"
|
|
21
|
+
},
|
|
19
22
|
"files": [
|
|
20
23
|
"bin",
|
|
21
24
|
"src",
|
|
@@ -37,6 +40,8 @@
|
|
|
37
40
|
"static-analysis",
|
|
38
41
|
"diagram",
|
|
39
42
|
"typescript",
|
|
43
|
+
"javascript",
|
|
44
|
+
"python",
|
|
40
45
|
"claude-code",
|
|
41
46
|
"skill"
|
|
42
47
|
]
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: repo-blueprint
|
|
3
|
-
description: Generate a blueprint of a code repository, a zoomable, dependency-layered architecture map built by static analysis (TypeScript and Python). Use when the user asks for a blueprint, architecture map, dependency map, codebase diagram or overview of how a repo fits together, or wants to watch that map update live while they code.
|
|
3
|
+
description: Generate a blueprint of a code repository, a zoomable, dependency-layered architecture map built by static analysis (TypeScript, JavaScript and Python). Use when the user asks for a blueprint, architecture map, dependency map, codebase diagram or overview of how a repo fits together, or wants to watch that map update live while they code.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# repo-blueprint
|
|
7
7
|
|
|
8
|
-
Builds a single standalone HTML page that maps a repository: packages, folders and files stacked in dependency tiers, with imports, RPC calls, HTTP calls, events and
|
|
8
|
+
Builds a single standalone HTML page that maps a repository: packages, folders and files stacked in dependency tiers, with imports, RPC calls, HTTP calls, events and every third-party dependency drawn between them, plus deterministic findings (import cycles, misplaced packages, unused files, and any file or import the analysis could not map). Everything comes from static analysis, so the same commit always gives the same page.
|
|
9
9
|
|
|
10
10
|
## Commands
|
|
11
11
|
|
|
@@ -14,13 +14,13 @@ The skill runs the `repo-blueprint` npm package, pinned to the version it was wr
|
|
|
14
14
|
One-time build, writes the HTML and prints its path plus a summary:
|
|
15
15
|
|
|
16
16
|
```bash
|
|
17
|
-
npx -y repo-blueprint@0.
|
|
17
|
+
npx -y repo-blueprint@0.2.0 build <repo> --open
|
|
18
18
|
```
|
|
19
19
|
|
|
20
20
|
Live mode, serves the page and rebuilds it when source files change (run it in the background; it keeps running until stopped):
|
|
21
21
|
|
|
22
22
|
```bash
|
|
23
|
-
npx -y repo-blueprint@0.
|
|
23
|
+
npx -y repo-blueprint@0.2.0 watch <repo> --open
|
|
24
24
|
```
|
|
25
25
|
|
|
26
26
|
Keep the pinned version as written, even if a newer one exists: the same commit and the same version always produce the same page, so users can compare blueprints over time.
|
|
@@ -34,22 +34,22 @@ Keep the pinned version as written, even if a newer one exists: the same commit
|
|
|
34
34
|
|
|
35
35
|
1. Use the repo the user names, otherwise the current working directory.
|
|
36
36
|
2. If they want to keep it open while they work, use `watch` in the background and give them the URL it prints. Otherwise use `build --open`.
|
|
37
|
-
3. Relay the
|
|
37
|
+
3. Relay the summary lines the build prints (packages by role, coverage, findings by category) and the output path or URL. If the coverage line says files were skipped, say how many; the page lists them with the reason. Don't describe the page at length; the user can press **?** inside it for help.
|
|
38
38
|
4. If `node` is missing, tell the user the skill needs Node.js 18.17 or newer. If the build fails, show the error. Python packages are skipped with a notice when neither Python 3.10+ nor `uv` is available; say so if it happens.
|
|
39
39
|
|
|
40
40
|
## Supported repos
|
|
41
41
|
|
|
42
|
-
- **TypeScript and JavaScript:** any folder with a `package.json
|
|
43
|
-
- **Python:** any folder with a `pyproject.toml
|
|
42
|
+
- **TypeScript and JavaScript:** any folder with a `package.json`, at any depth, ES modules or CommonJS. Imports are resolved with the package's `tsconfig.json` or `jsconfig.json` when it has one. Vue, Svelte and Astro files are reported as not mapped.
|
|
43
|
+
- **Python:** any folder with a `pyproject.toml`, flat or `src/` layout.
|
|
44
44
|
- **Roles:** a declared type wins (Backstage `catalog-info.yaml` `spec.type`, Nx `project.json` `projectType`). Otherwise roles come from the code: an entry file that mounts a UI makes an app, one that starts a server or worker (or a Dockerfile or start script) makes a service, and anything else is a library.
|
|
45
45
|
|
|
46
46
|
## Per-repo settings
|
|
47
47
|
|
|
48
48
|
The package ships defaults. A repo can extend or override them with a `.blueprint.json` at its root, using the same keys:
|
|
49
49
|
|
|
50
|
-
- `externalSystems`: maps a system name to the npm or Python packages that indicate it, for example `{ "
|
|
50
|
+
- `externalSystems`: maps a system name to the npm or Python packages that indicate it, for example `{ "Billing API": ["@acme/billing-client"] }`. It only renames: every imported package is on the map without it.
|
|
51
51
|
- `messaging`: method and property names for RPC calls, publishes and subscriptions, for repos whose message-bus wrapper uses different names.
|
|
52
52
|
- `folderRoles`: folder names that imply a role, used only for the role-mismatch finding.
|
|
53
53
|
- `sdkDirs`: folder names whose subfolders are generated API clients named after the services they call.
|
|
54
54
|
|
|
55
|
-
Only suggest a `.blueprint.json` when
|
|
55
|
+
Only suggest a `.blueprint.json` when the user wants packages grouped under a system name, or message links are missing from the map.
|
|
@@ -25,10 +25,20 @@
|
|
|
25
25
|
"server.ts",
|
|
26
26
|
"worker.ts",
|
|
27
27
|
"app.module.ts",
|
|
28
|
+
"main.js",
|
|
29
|
+
"index.js",
|
|
30
|
+
"server.js",
|
|
31
|
+
"worker.js",
|
|
28
32
|
"main.py",
|
|
29
33
|
"__main__.py",
|
|
30
34
|
"__init__.py",
|
|
31
|
-
"*.config.ts"
|
|
35
|
+
"*.config.ts",
|
|
36
|
+
"*.config.js",
|
|
37
|
+
"*.config.mjs",
|
|
38
|
+
"*.config.cjs",
|
|
39
|
+
".*rc.js",
|
|
40
|
+
".*rc.cjs",
|
|
41
|
+
".*rc.mjs"
|
|
32
42
|
],
|
|
33
43
|
"frameworkLoaded": [
|
|
34
44
|
"*.entity.ts",
|
|
@@ -70,15 +80,23 @@
|
|
|
70
80
|
},
|
|
71
81
|
"externalSystems": {
|
|
72
82
|
"PostgreSQL": [
|
|
73
|
-
"typeorm",
|
|
74
|
-
"@nestjs/typeorm",
|
|
75
83
|
"pg",
|
|
76
|
-
"sqlalchemy",
|
|
77
|
-
"knex",
|
|
78
84
|
"postgres",
|
|
79
|
-
"
|
|
80
|
-
"
|
|
81
|
-
"
|
|
85
|
+
"pg-boss",
|
|
86
|
+
"psycopg",
|
|
87
|
+
"psycopg2",
|
|
88
|
+
"asyncpg"
|
|
89
|
+
],
|
|
90
|
+
"Stripe": [
|
|
91
|
+
"stripe",
|
|
92
|
+
"@stripe/stripe-js",
|
|
93
|
+
"@stripe/react-stripe-js"
|
|
94
|
+
],
|
|
95
|
+
"SQLite": [
|
|
96
|
+
"better-sqlite3",
|
|
97
|
+
"sqlite3",
|
|
98
|
+
"sqlite",
|
|
99
|
+
"aiosqlite"
|
|
82
100
|
],
|
|
83
101
|
"MySQL": [
|
|
84
102
|
"mysql",
|
package/src/build.mjs
CHANGED
|
@@ -13,8 +13,10 @@ const OUT = path.resolve(process.argv[3]);
|
|
|
13
13
|
const FACTS = path.resolve(process.argv[4]);
|
|
14
14
|
const name = path.basename(REPO);
|
|
15
15
|
|
|
16
|
-
const read = (f) => (fs.existsSync(f) ? JSON.parse(fs.readFileSync(f, 'utf8'))
|
|
17
|
-
const
|
|
16
|
+
const read = (f) => (fs.existsSync(f) ? JSON.parse(fs.readFileSync(f, 'utf8')) : { workspaces: [] });
|
|
17
|
+
const tsFacts = read(path.join(FACTS, `${name}-ts.json`));
|
|
18
|
+
const pyFacts = read(path.join(FACTS, `${name}-py.json`));
|
|
19
|
+
const model = buildModel(name, [...tsFacts.workspaces, ...pyFacts.workspaces], loadConfig(REPO), [tsFacts.coverage, pyFacts.coverage]);
|
|
18
20
|
const t0 = Date.now();
|
|
19
21
|
const views = buildViews(model);
|
|
20
22
|
|
|
@@ -69,4 +71,5 @@ const byType = {};
|
|
|
69
71
|
for (const f of model.findings) byType[f.type] = (byType[f.type] || 0) + 1;
|
|
70
72
|
const nFiles = Object.keys(model.files).length;
|
|
71
73
|
console.log(`${name}${commit ? ` (${branch}@${commit})` : ''}: ${Object.keys(model.packages).length} packages (${Object.entries(roles).map(([r, n]) => `${n} ${r}`).join(', ')}), ${nFiles} files, ${Object.keys(views).length} views`);
|
|
74
|
+
if (model.coverage) console.log(`coverage: ${model.coverage.analyzed} of ${model.coverage.total} source files mapped${model.coverage.skipped ? `, ${model.coverage.skipped} skipped (see findings)` : ''}`);
|
|
72
75
|
console.log(`findings: ${model.findings.length ? Object.entries(byType).map(([t, n]) => `${t} ${n}`).join(', ') : 'none'}`);
|
package/src/extract-py.py
CHANGED
|
@@ -7,7 +7,7 @@ import sys
|
|
|
7
7
|
REPO = os.path.abspath(sys.argv[1])
|
|
8
8
|
OUT = sys.argv[2]
|
|
9
9
|
SUBJECT_RE = re.compile(r"^[A-Za-z][\w-]*(\.[\w#*-]+)+$")
|
|
10
|
-
TEST_RE = re.compile(r"(^|/)
|
|
10
|
+
TEST_RE = re.compile(r"(^|/)tests?/|(^|/)(test_|conftest)|_test\.py$")
|
|
11
11
|
|
|
12
12
|
|
|
13
13
|
def ext_name(mod):
|
|
@@ -62,19 +62,35 @@ def enums_of(tree):
|
|
|
62
62
|
return out
|
|
63
63
|
|
|
64
64
|
|
|
65
|
+
def py_files(d, nested=()):
|
|
66
|
+
out = []
|
|
67
|
+
for root, dirs, fs in os.walk(d):
|
|
68
|
+
dirs[:] = sorted(x for x in dirs if not x.startswith(".") and x not in SKIP and os.path.join(root, x) not in nested)
|
|
69
|
+
out += [os.path.join(root, f) for f in sorted(fs) if f.endswith(".py")]
|
|
70
|
+
return out
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
def parse(p, skipped, wid):
|
|
74
|
+
try:
|
|
75
|
+
return ast.parse(open(p, encoding="utf-8").read(), filename=p)
|
|
76
|
+
except (SyntaxError, UnicodeDecodeError, ValueError) as e:
|
|
77
|
+
why = f"invalid syntax on line {e.lineno}" if isinstance(e, SyntaxError) else "not valid UTF-8 text" if isinstance(e, UnicodeDecodeError) else str(e)
|
|
78
|
+
skipped.append({"pkg": wid, "file": rel(p), "reason": f"could not be parsed: {why}"})
|
|
79
|
+
return None
|
|
80
|
+
|
|
81
|
+
|
|
65
82
|
def main():
|
|
66
83
|
result = []
|
|
84
|
+
skipped = []
|
|
67
85
|
packages = discover()
|
|
86
|
+
# Every .py file (tests excluded) is either analysed or listed in skipped with the reason.
|
|
87
|
+
for p in py_files(REPO):
|
|
88
|
+
if not TEST_RE.search(rel(p)) and not any(p.startswith(d + os.sep) for _, d in packages):
|
|
89
|
+
skipped.append({"pkg": None, "file": rel(p), "reason": "outside any package: no pyproject.toml above it"})
|
|
68
90
|
for wid, wdir in packages:
|
|
69
91
|
nested = [d for _, d in packages if d != wdir and d.startswith(wdir + os.sep)]
|
|
70
|
-
paths = []
|
|
71
|
-
for
|
|
72
|
-
dirs[:] = sorted(d for d in dirs if not d.startswith(".") and d not in SKIP and os.path.join(root, d) not in nested)
|
|
73
|
-
for f in sorted(fs):
|
|
74
|
-
p = os.path.join(root, f)
|
|
75
|
-
if f.endswith(".py") and not TEST_RE.search(rel(p)[len(rel(wdir)) + 1 :]):
|
|
76
|
-
paths.append(p)
|
|
77
|
-
trees = {p: ast.parse(open(p, encoding="utf-8").read(), filename=p) for p in paths}
|
|
92
|
+
paths = [p for p in py_files(wdir, nested) if not TEST_RE.search(os.path.relpath(p, wdir).replace(os.sep, "/"))]
|
|
93
|
+
trees = {p: t for p in paths if (t := parse(p, skipped, wid)) is not None}
|
|
78
94
|
enums = {}
|
|
79
95
|
for t in trees.values():
|
|
80
96
|
enums.update(enums_of(t))
|
|
@@ -92,12 +108,14 @@ def main():
|
|
|
92
108
|
for _ in range(level - 1):
|
|
93
109
|
base = os.path.dirname(base)
|
|
94
110
|
parts = mod.split(".") if mod else []
|
|
95
|
-
|
|
111
|
+
cands = [os.path.join(base, *parts)]
|
|
96
112
|
else:
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
113
|
+
# Absolute imports resolve from the package root, or from src/ in a src layout.
|
|
114
|
+
cands = [os.path.join(wdir, *mod.split(".")), os.path.join(wdir, "src", *mod.split("."))]
|
|
115
|
+
for cand in cands:
|
|
116
|
+
for c in (cand + ".py", os.path.join(cand, "__init__.py")):
|
|
117
|
+
if c in trees:
|
|
118
|
+
return rel(c)
|
|
101
119
|
return None
|
|
102
120
|
|
|
103
121
|
files = {}
|
|
@@ -159,7 +177,8 @@ def main():
|
|
|
159
177
|
continue
|
|
160
178
|
result.append({"id": wid, "dir": rel(wdir) if wdir != REPO else ".", "parent": os.path.basename(os.path.dirname(wdir)) if wdir != REPO else "", "aliases": [], "files": files, "mq": mq, "lang": "python", "meta": meta})
|
|
161
179
|
sys.stdout.write(f"{wid}: {len(files)} files, {len(mq)} mq facts\n")
|
|
162
|
-
|
|
180
|
+
analyzed = sum(len(w["files"]) for w in result)
|
|
181
|
+
json.dump({"workspaces": result, "coverage": {"total": analyzed + len(skipped), "analyzed": analyzed, "skipped": skipped}}, open(OUT, "w"))
|
|
163
182
|
|
|
164
183
|
|
|
165
184
|
main()
|
package/src/extract-ts.mjs
CHANGED
|
@@ -10,7 +10,7 @@ const OUT = path.resolve(process.argv[3] || 'facts-ts.json');
|
|
|
10
10
|
|
|
11
11
|
import { builtinModules } from 'node:module';
|
|
12
12
|
const BUILTINS = new Set([...builtinModules, 'bun', 'bun:test']);
|
|
13
|
-
const TEST_RE = /(\.spec\.|\.test\.|\.d\.ts
|
|
13
|
+
const TEST_RE = /(\.spec\.|\.test\.|\.d\.ts$|(^|\/)(__tests__|__fixtures__|__test-fixtures__|__mocks__|test|tests|test-d|simulations)\/|vitest\.|playwright\.)/;
|
|
14
14
|
const ROUTE_DECORATORS = { Get: 'GET', Post: 'POST', Put: 'PUT', Patch: 'PATCH', Delete: 'DELETE', All: 'ALL' };
|
|
15
15
|
const BOOT_RE = /^(NestFactory\.(create|createMicroservice|createApplicationContext)|Bun\.serve|serve|http\.createServer|https\.createServer|createServer|Worker\.create|[\w.]*\.listen)$/;
|
|
16
16
|
const MOUNT_RE = /^(createRoot|hydrateRoot|ReactDOM\.render|ReactDOM\.createRoot|ReactDOM\.hydrate|createApp|bootstrapApplication|platformBrowserDynamic)$/;
|
|
@@ -23,11 +23,55 @@ const clip = (s, n = 70) => {
|
|
|
23
23
|
};
|
|
24
24
|
|
|
25
25
|
const SKIP_DIRS = new Set(['node_modules', '.git', 'dist', 'build', 'coverage', '.venv', 'venv', '.nx', 'tmp', '.next', 'out', '.turbo', 'playwright-report', 'test-results']);
|
|
26
|
+
const SOURCE_RE = /\.(tsx?|jsx?|mts|cts|mjs|cjs)$/;
|
|
27
|
+
const isSource = (f) => SOURCE_RE.test(f) && !/\.d\.[mc]?ts$/.test(f);
|
|
28
|
+
// Single-file components hold imports too, but are not parsed yet: they are counted and reported as skipped.
|
|
29
|
+
const COMPONENT_RE = /\.(vue|svelte|astro)$/;
|
|
30
|
+
const inSkippedDir = (relPath) => relPath.split('/').slice(0, -1).some((seg) => SKIP_DIRS.has(seg) || seg.startsWith('.'));
|
|
31
|
+
|
|
32
|
+
// Used when a package has neither a tsconfig.json nor a jsconfig.json.
|
|
33
|
+
const DEFAULT_OPTIONS = {
|
|
34
|
+
allowJs: true,
|
|
35
|
+
jsx: ts.JsxEmit.Preserve,
|
|
36
|
+
module: ts.ModuleKind.ESNext,
|
|
37
|
+
moduleResolution: ts.ModuleResolutionKind.Bundler,
|
|
38
|
+
target: ts.ScriptTarget.ES2022,
|
|
39
|
+
resolveJsonModule: true,
|
|
40
|
+
};
|
|
41
|
+
|
|
42
|
+
// Source files under dir. With nested: true, it descends into nested packages and also lists component files.
|
|
43
|
+
function walkSources(dir, nested = false) {
|
|
44
|
+
const out = [];
|
|
45
|
+
const walk = (d) => {
|
|
46
|
+
for (const e of fs.readdirSync(d, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) {
|
|
47
|
+
const abs = path.join(d, e.name);
|
|
48
|
+
if (e.isDirectory()) {
|
|
49
|
+
if (!SKIP_DIRS.has(e.name) && !e.name.startsWith('.') && (nested || !fs.existsSync(path.join(abs, 'package.json')))) walk(abs);
|
|
50
|
+
} else if (isSource(e.name) || (nested && COMPONENT_RE.test(e.name))) out.push(abs);
|
|
51
|
+
}
|
|
52
|
+
};
|
|
53
|
+
walk(dir);
|
|
54
|
+
return out;
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// Compiler options and file list for a package: its tsconfig.json, else its jsconfig.json, else defaults over every source file.
|
|
58
|
+
// JavaScript is always analysed, whatever the config says about allowJs.
|
|
59
|
+
function compilerSetup(dir) {
|
|
60
|
+
for (const name of ['tsconfig.json', 'jsconfig.json']) {
|
|
61
|
+
const cfgPath = path.join(dir, name);
|
|
62
|
+
if (!fs.existsSync(cfgPath)) continue;
|
|
63
|
+
const raw = ts.readConfigFile(cfgPath, ts.sys.readFile).config || {};
|
|
64
|
+
raw.compilerOptions = { ...raw.compilerOptions, allowJs: true };
|
|
65
|
+
const parsed = ts.parseJsonConfigFileContent(raw, ts.sys, dir, { noEmit: true }, cfgPath);
|
|
66
|
+
return { options: parsed.options, fileNames: parsed.fileNames, config: name };
|
|
67
|
+
}
|
|
68
|
+
return { options: DEFAULT_OPTIONS, fileNames: walkSources(dir), config: null };
|
|
69
|
+
}
|
|
26
70
|
|
|
27
71
|
function discoverWorkspaces() {
|
|
28
72
|
const out = [];
|
|
29
73
|
const walk = (dir, depth) => {
|
|
30
|
-
if (fs.existsSync(path.join(dir, '
|
|
74
|
+
if (fs.existsSync(path.join(dir, 'package.json'))) out.push(dir);
|
|
31
75
|
if (depth >= 4) return;
|
|
32
76
|
for (const e of fs.readdirSync(dir, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) {
|
|
33
77
|
if (e.isDirectory() && !SKIP_DIRS.has(e.name) && !e.name.startsWith('.')) walk(path.join(dir, e.name), depth + 1);
|
|
@@ -75,6 +119,16 @@ function declFileOf(checker, expr) {
|
|
|
75
119
|
return d ? rel(d.getSourceFile().fileName) : null;
|
|
76
120
|
}
|
|
77
121
|
|
|
122
|
+
const isStringish = (n) => ts.isStringLiteral(n) || ts.isNoSubstitutionTemplateLiteral(n);
|
|
123
|
+
|
|
124
|
+
// Names a require() call takes: `const { a } = require(x)` → a, `require(x).a` → a, anything else → the whole module.
|
|
125
|
+
function requiredNames(call) {
|
|
126
|
+
const p = call.parent;
|
|
127
|
+
if (p && ts.isVariableDeclaration(p) && ts.isObjectBindingPattern(p.name)) return p.name.elements.map((e) => (e.propertyName || e.name).getText());
|
|
128
|
+
if (p && ts.isPropertyAccessExpression(p) && p.expression === call) return [p.name.text];
|
|
129
|
+
return ['*'];
|
|
130
|
+
}
|
|
131
|
+
|
|
78
132
|
function isParamOf(id, checker) {
|
|
79
133
|
const sym = checker.getSymbolAtLocation(id);
|
|
80
134
|
return !!(sym && sym.declarations && sym.declarations.some((d) => ts.isParameter(d)));
|
|
@@ -113,12 +167,51 @@ function paramsSig(fn) {
|
|
|
113
167
|
|
|
114
168
|
function classifyFn(name, file) {
|
|
115
169
|
if (/^use[A-Z]/.test(name)) return 'hook';
|
|
116
|
-
if (/^[A-Z]/.test(name) && file
|
|
170
|
+
if (/^[A-Z]/.test(name) && /\.[jt]sx$/.test(file)) return 'component';
|
|
117
171
|
return 'function';
|
|
118
172
|
}
|
|
119
173
|
|
|
120
|
-
|
|
174
|
+
// Target name of a CommonJS export assignment: `module.exports` → '', `exports.a` / `module.exports.a` → 'a', anything else → null.
|
|
175
|
+
function cjsTarget(left) {
|
|
176
|
+
const t = left.getText();
|
|
177
|
+
if (t === 'module.exports') return '';
|
|
178
|
+
const m = /^(?:module\.)?exports\.([A-Za-z_$][\w$]*)$/.exec(t);
|
|
179
|
+
return m ? m[1] : null;
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
// CommonJS exports: `module.exports = { a, b: fn }`, `module.exports = fn`, `exports.a = …`.
|
|
183
|
+
function cjsDecls(sf, file) {
|
|
184
|
+
const local = new Map();
|
|
185
|
+
for (const st of sf.statements) {
|
|
186
|
+
if ((ts.isFunctionDeclaration(st) || ts.isClassDeclaration(st)) && st.name) local.set(st.name.text, st);
|
|
187
|
+
if (ts.isVariableStatement(st)) for (const d of st.declarationList.declarations) if (ts.isIdentifier(d.name) && d.initializer) local.set(d.name.text, d.initializer);
|
|
188
|
+
}
|
|
189
|
+
const declOf = (name, value) => {
|
|
190
|
+
const v = value && ts.isIdentifier(value) && local.has(value.text) ? local.get(value.text) : value;
|
|
191
|
+
if (v && (ts.isClassDeclaration(v) || ts.isClassExpression(v))) return { name, kind: 'class', decorators: [], extends: [], members: [], routes: [] };
|
|
192
|
+
if (v && (ts.isFunctionDeclaration(v) || ts.isFunctionExpression(v) || ts.isArrowFunction(v) || ts.isMethodDeclaration(v))) return { name, kind: classifyFn(name, file), sig: paramsSig(v) };
|
|
193
|
+
return { name, kind: 'const' };
|
|
194
|
+
};
|
|
121
195
|
const decls = [];
|
|
196
|
+
for (const st of sf.statements) {
|
|
197
|
+
if (!ts.isExpressionStatement(st) || !ts.isBinaryExpression(st.expression) || st.expression.operatorToken.kind !== ts.SyntaxKind.EqualsToken) continue;
|
|
198
|
+
const { left, right } = st.expression;
|
|
199
|
+
const target = cjsTarget(left);
|
|
200
|
+
if (target === null) continue;
|
|
201
|
+
if (target) decls.push(declOf(target, right));
|
|
202
|
+
else if (ts.isObjectLiteralExpression(right)) {
|
|
203
|
+
for (const p of right.properties) {
|
|
204
|
+
if (ts.isShorthandPropertyAssignment(p)) decls.push(declOf(p.name.text, p.name));
|
|
205
|
+
else if (ts.isPropertyAssignment(p) && p.name) decls.push(declOf(p.name.getText(sf), p.initializer));
|
|
206
|
+
else if (ts.isMethodDeclaration(p) && p.name) decls.push(declOf(p.name.getText(sf), p));
|
|
207
|
+
}
|
|
208
|
+
} else decls.push(declOf('default', right));
|
|
209
|
+
}
|
|
210
|
+
return decls;
|
|
211
|
+
}
|
|
212
|
+
|
|
213
|
+
function extractDecls(sf, file) {
|
|
214
|
+
const decls = cjsDecls(sf, file);
|
|
122
215
|
for (const st of sf.statements) {
|
|
123
216
|
if (ts.isClassDeclaration(st) && st.name && isExported(st)) {
|
|
124
217
|
const decos = decoratorsOf(st);
|
|
@@ -159,6 +252,22 @@ function extractDecls(sf, file) {
|
|
|
159
252
|
return decls;
|
|
160
253
|
}
|
|
161
254
|
|
|
255
|
+
// Files a package.json publishes through main, module, browser, bin and exports, as repo-relative paths.
|
|
256
|
+
function publishedFiles(dir, pj) {
|
|
257
|
+
const out = new Set();
|
|
258
|
+
const add = (v) => {
|
|
259
|
+
if (typeof v === 'string') {
|
|
260
|
+
if (!v.startsWith('.') && !/^[\w@-]/.test(v)) return;
|
|
261
|
+
const abs = path.resolve(dir, v);
|
|
262
|
+
for (const c of [abs, ...['.js', '.ts', '.mjs', '.cjs', '/index.js', '/index.ts'].map((x) => abs + x)]) {
|
|
263
|
+
if (fs.existsSync(c) && fs.statSync(c).isFile()) return out.add(rel(c));
|
|
264
|
+
}
|
|
265
|
+
} else if (v && typeof v === 'object') Object.values(v).forEach(add);
|
|
266
|
+
};
|
|
267
|
+
for (const k of ['main', 'module', 'browser', 'bin', 'exports', 'types']) add(pj[k]);
|
|
268
|
+
return [...out].sort();
|
|
269
|
+
}
|
|
270
|
+
|
|
162
271
|
function declaredRole(dir) {
|
|
163
272
|
const out = {};
|
|
164
273
|
const pj = path.join(dir, 'project.json');
|
|
@@ -184,14 +293,13 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
184
293
|
bin: !!pj.bin,
|
|
185
294
|
exportsField: !!(pj.exports || pj.main || pj.module || pj.types),
|
|
186
295
|
dockerfile: fs.existsSync(path.join(ws.dir, 'Dockerfile')),
|
|
296
|
+
published: publishedFiles(ws.dir, pj),
|
|
187
297
|
declared: declaredRole(ws.dir),
|
|
188
298
|
};
|
|
189
|
-
const
|
|
190
|
-
const
|
|
191
|
-
const parsed = ts.parseJsonConfigFileContent(cfg.config, ts.sys, ws.dir, { noEmit: true }, cfgPath);
|
|
192
|
-
const options = { ...parsed.options, noEmit: true, skipLibCheck: true, allowJs: false };
|
|
299
|
+
const setup = compilerSetup(ws.dir);
|
|
300
|
+
const options = { ...setup.options, noEmit: true, skipLibCheck: true, allowJs: true, checkJs: false, maxNodeModuleJsDepth: 0 };
|
|
193
301
|
const owner = (f) => allWs.filter((w) => f.startsWith(w.dir + path.sep)).sort((a, b) => b.dir.length - a.dir.length)[0];
|
|
194
|
-
let roots =
|
|
302
|
+
let roots = setup.fileNames.filter((f) => isSource(f) && !TEST_RE.test(rel(f)) && !inSkippedDir(rel(f)) && owner(f) === ws);
|
|
195
303
|
const rootIndex = path.join(ws.dir, 'index.ts');
|
|
196
304
|
if (fs.existsSync(rootIndex) && !roots.includes(rootIndex)) roots.push(rootIndex);
|
|
197
305
|
roots = roots.sort();
|
|
@@ -202,17 +310,30 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
202
310
|
|
|
203
311
|
const files = {};
|
|
204
312
|
const mq = [];
|
|
313
|
+
const skipped = [];
|
|
314
|
+
const inRoots = new Set(roots);
|
|
315
|
+
const configName = setup.config || 'package settings';
|
|
316
|
+
for (const abs of walkSources(ws.dir, true)) {
|
|
317
|
+
if (owner(abs) !== ws || TEST_RE.test(rel(abs)) || inRoots.has(abs)) continue;
|
|
318
|
+
skipped.push({ file: rel(abs), reason: COMPONENT_RE.test(abs) ? 'not parsed: Vue, Svelte and Astro files are not supported yet' : `left out by its ${configName}` });
|
|
319
|
+
}
|
|
205
320
|
for (const abs of roots) {
|
|
206
321
|
const sf = program.getSourceFile(abs);
|
|
207
|
-
if (!sf)
|
|
322
|
+
if (!sf) {
|
|
323
|
+
skipped.push({ file: rel(abs), reason: 'could not be read' });
|
|
324
|
+
continue;
|
|
325
|
+
}
|
|
208
326
|
const file = rel(abs);
|
|
209
327
|
const imports = [];
|
|
328
|
+
const unresolved = new Set();
|
|
329
|
+
const dynamic = new Set();
|
|
210
330
|
const externals = new Set();
|
|
211
331
|
const boot = new Set();
|
|
212
332
|
const mount = new Set();
|
|
213
333
|
const lines = sf.getLineAndCharacterOfPosition(sf.end).line + 1;
|
|
214
334
|
|
|
215
335
|
const addImport = (spec, names, node) => {
|
|
336
|
+
if (spec.startsWith('node:') || BUILTINS.has(spec)) return;
|
|
216
337
|
const pkg = Object.keys(PKG_NAMES).find((n) => spec === n || spec.startsWith(n + '/'));
|
|
217
338
|
if (pkg && PKG_NAMES[pkg] !== ws.id) {
|
|
218
339
|
imports.push({ spec, target: `pkg:${PKG_NAMES[pkg]}`, names });
|
|
@@ -224,7 +345,12 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
224
345
|
if (!ts.isBuiltinModule?.(pkgName) && !BUILTINS.has(pkgName)) externals.add(pkgName);
|
|
225
346
|
return;
|
|
226
347
|
}
|
|
227
|
-
if (!r
|
|
348
|
+
if (!r) {
|
|
349
|
+
const bare = spec.replace(/[?#].*$/, '');
|
|
350
|
+
if (!/\.[A-Za-z0-9]+$/.test(bare) || SOURCE_RE.test(bare)) unresolved.add(spec);
|
|
351
|
+
return;
|
|
352
|
+
}
|
|
353
|
+
if (r.isExternalLibraryImport || /node_modules/.test(r.resolvedFileName)) return;
|
|
228
354
|
const target = rel(r.resolvedFileName);
|
|
229
355
|
if (target.startsWith('..')) return;
|
|
230
356
|
imports.push({ spec, target, names });
|
|
@@ -246,7 +372,12 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
246
372
|
addImport(node.moduleSpecifier.text, names, node);
|
|
247
373
|
} else if (ts.isCallExpression(node)) {
|
|
248
374
|
const callee = node.expression;
|
|
249
|
-
|
|
375
|
+
const isImport = callee.kind === ts.SyntaxKind.ImportKeyword;
|
|
376
|
+
const isRequire = ts.isIdentifier(callee) && callee.text === 'require' && node.arguments.length === 1;
|
|
377
|
+
if ((isImport || isRequire) && node.arguments[0]) {
|
|
378
|
+
if (isStringish(node.arguments[0])) addImport(node.arguments[0].text, isImport ? ['*'] : requiredNames(node), node);
|
|
379
|
+
else dynamic.add(clip(node.arguments[0].getText(sf), 50));
|
|
380
|
+
}
|
|
250
381
|
const text = callee.getText(sf);
|
|
251
382
|
if (BOOT_RE.test(text)) boot.add(text.replace(/^.*\.(?=listen$)/, '.'));
|
|
252
383
|
if (MOUNT_RE.test(text)) mount.add(text);
|
|
@@ -263,7 +394,7 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
263
394
|
const subj = prop(node.arguments[1], MSG.rpcSubjectProperty);
|
|
264
395
|
if (subj) {
|
|
265
396
|
const { values, inferred } = subjectsOf(checker, subj);
|
|
266
|
-
for (const s of values) mq.push({ kind: 'rpc-call', subject: s, file, inferred });
|
|
397
|
+
for (const s of values.filter((v) => SUBJECT_RE.test(v))) mq.push({ kind: 'rpc-call', subject: s, file, inferred });
|
|
267
398
|
if (!values.length) mq.push({ kind: 'rpc-call-dynamic', file, expr: clip(subj.getText(sf), 50) });
|
|
268
399
|
}
|
|
269
400
|
}
|
|
@@ -280,9 +411,9 @@ function analyzeWorkspace(ws, allWs) {
|
|
|
280
411
|
ts.forEachChild(node, visit);
|
|
281
412
|
};
|
|
282
413
|
visit(sf);
|
|
283
|
-
files[file] = { lines, imports, externals: [...externals].sort(), boot: [...boot].sort(), mount: [...mount].sort(), decls: extractDecls(sf, file) };
|
|
414
|
+
files[file] = { lines, imports, unresolved: [...unresolved].sort(), dynamic: [...dynamic].sort(), externals: [...externals].sort(), boot: [...boot].sort(), mount: [...mount].sort(), decls: extractDecls(sf, file) };
|
|
284
415
|
}
|
|
285
|
-
return { id: ws.id, dir: rel(ws.dir) || '.', parent: ws.parent, aliases, files, mq, meta, lang: 'typescript' };
|
|
416
|
+
return { id: ws.id, dir: rel(ws.dir) || '.', parent: ws.parent, aliases, files, mq, meta, skipped, lang: Object.keys(files).some((f) => /\.[mc]?tsx?$/.test(f)) ? 'typescript' : 'javascript' };
|
|
286
417
|
}
|
|
287
418
|
|
|
288
419
|
function readMqModule(checker, obj, file) {
|
|
@@ -323,11 +454,19 @@ for (const ws of wss) {
|
|
|
323
454
|
if (name && localDeps.has(name)) PKG_NAMES[name] = ws.id;
|
|
324
455
|
}
|
|
325
456
|
const result = { repo: REPO, workspaces: [] };
|
|
457
|
+
// Every JS/TS source file in the repo (tests excluded) is either analysed or listed here with the reason.
|
|
458
|
+
const skipped = [];
|
|
459
|
+
const owned = (f) => wss.some((w) => f.startsWith(w.dir + path.sep));
|
|
460
|
+
for (const abs of walkSources(REPO, true)) if (!TEST_RE.test(rel(abs)) && !owned(abs)) skipped.push({ pkg: null, file: rel(abs), reason: 'outside any package: no package.json above it' });
|
|
326
461
|
for (const ws of wss) {
|
|
327
462
|
const t0 = Date.now();
|
|
328
463
|
const r = analyzeWorkspace(ws, wss);
|
|
464
|
+
for (const x of r.skipped) skipped.push({ pkg: ws.id, ...x });
|
|
465
|
+
delete r.skipped;
|
|
329
466
|
if (!Object.keys(r.files).length) continue;
|
|
330
467
|
result.workspaces.push(r);
|
|
331
468
|
process.stdout.write(`${ws.id}: ${Object.keys(r.files).length} files, ${r.mq.length} mq facts, ${Date.now() - t0}ms\n`);
|
|
332
469
|
}
|
|
470
|
+
const analyzed = result.workspaces.reduce((n, w) => n + Object.keys(w.files).length, 0);
|
|
471
|
+
result.coverage = { total: analyzed + skipped.length, analyzed, skipped };
|
|
333
472
|
fs.writeFileSync(OUT, JSON.stringify(result));
|
package/src/model.mjs
CHANGED
|
@@ -1,16 +1,19 @@
|
|
|
1
1
|
const glob = (pats) => new RegExp('(^|/)(' + pats.map((p) => p.replace(/[.+^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '[^/]*').replace(/\/$/, '/.*')).join('|') + ')$');
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
// coverage: one { total, analyzed, skipped: [{ pkg, file, reason }] } per extractor that ran.
|
|
4
|
+
export function buildModel(repoName, facts, config, coverages = []) {
|
|
5
|
+
const parts = coverages.filter(Boolean);
|
|
6
|
+
const coverage = parts.length ? { total: parts.reduce((n, c) => n + c.total, 0), analyzed: parts.reduce((n, c) => n + c.analyzed, 0), skipped: parts.flatMap((c) => c.skipped) } : null;
|
|
4
7
|
const EXTERNAL_SYSTEMS = config.externalSystems;
|
|
5
8
|
const SYSTEM_OF = Object.fromEntries(Object.entries(EXTERNAL_SYSTEMS).flatMap(([sys, pkgs]) => pkgs.map((p) => [p, sys])));
|
|
6
9
|
const FOLDER_ROLE = config.folderRoles;
|
|
7
10
|
const CONVENTIONAL_ENTRY_RE = glob(config.conventionalEntries);
|
|
8
11
|
const FRAMEWORK_LOADED_RE = glob(config.frameworkLoaded);
|
|
9
12
|
const SDK_RE = new RegExp(`/(${config.sdkDirs.join('|')})/([^/]+)/`);
|
|
10
|
-
return buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_ROLE, CONVENTIONAL_ENTRY_RE, FRAMEWORK_LOADED_RE, SDK_RE });
|
|
13
|
+
return buildModelWith({ repoName, facts, coverage, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_ROLE, CONVENTIONAL_ENTRY_RE, FRAMEWORK_LOADED_RE, SDK_RE });
|
|
11
14
|
}
|
|
12
15
|
|
|
13
|
-
function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_ROLE, CONVENTIONAL_ENTRY_RE, FRAMEWORK_LOADED_RE, SDK_RE }) {
|
|
16
|
+
function buildModelWith({ repoName, facts, coverage, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_ROLE, CONVENTIONAL_ENTRY_RE, FRAMEWORK_LOADED_RE, SDK_RE }) {
|
|
14
17
|
const wsList = [...facts].sort((a, b) => a.id.localeCompare(b.id));
|
|
15
18
|
const packages = {};
|
|
16
19
|
const nodes = {};
|
|
@@ -38,7 +41,7 @@ function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_R
|
|
|
38
41
|
for (const [file, f] of Object.entries(files)) {
|
|
39
42
|
for (const d of f.decls) {
|
|
40
43
|
const k = `${f.pkg}::${d.name}`;
|
|
41
|
-
if (!exportIndex[k] || /index\.
|
|
44
|
+
if (!exportIndex[k] || /index\.[mc]?[jt]sx?$/.test(exportIndex[k])) exportIndex[k] = file;
|
|
42
45
|
}
|
|
43
46
|
}
|
|
44
47
|
|
|
@@ -139,8 +142,7 @@ function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_R
|
|
|
139
142
|
const thirdParty = {};
|
|
140
143
|
for (const [file, f] of Object.entries(files)) {
|
|
141
144
|
for (const ext of f.externals) {
|
|
142
|
-
|
|
143
|
-
if (sys) addEdge(file, `ext:${sys}`, 'ext', ext);
|
|
145
|
+
addEdge(file, `ext:${SYSTEM_OF[ext] || ext}`, 'ext', ext);
|
|
144
146
|
for (const k of chainOf(file)) {
|
|
145
147
|
const t = (thirdParty[k] ||= {});
|
|
146
148
|
t[ext] = (t[ext] || 0) + 1;
|
|
@@ -193,7 +195,7 @@ function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_R
|
|
|
193
195
|
if (p.role !== 'library') {
|
|
194
196
|
const importers = [...new Set(PE.filter((e) => e.to === p.id && e.kind === 'import').map((e) => e.from))].sort();
|
|
195
197
|
if (importers.length) findings.push({ type: 'runnable-imported', severity: 'design', pkg: p.id, title: `${p.id} is a ${p.role} but ${importers.join(', ')} import${importers.length > 1 ? '' : 's'} its code`, detail: 'Runnable packages imported as libraries couple deploys together.' });
|
|
196
|
-
} else if (!PE.some((e) => e.to === p.id)) {
|
|
198
|
+
} else if (Object.keys(packages).length > 1 && !PE.some((e) => e.to === p.id)) {
|
|
197
199
|
findings.push({ type: 'unused-library', severity: 'check', pkg: p.id, title: `${p.id} is a library nothing else uses`, detail: 'No entry point and no other package imports it.' });
|
|
198
200
|
}
|
|
199
201
|
}
|
|
@@ -209,9 +211,26 @@ function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_R
|
|
|
209
211
|
for (const c of unmatched.calls) findings.push({ type: 'rpc-no-server', severity: 'check', pkg: c.pkg, title: `RPC "${c.subject}" has no server in the repo`, detail: `Called from ${c.file}.` });
|
|
210
212
|
for (const p of unmatched.pubs) findings.push({ type: 'event-no-subscriber', severity: 'check', pkg: p.pkg, title: `Event "${p.subject}" has no subscriber in the repo`, detail: `Published from ${p.file}.` });
|
|
211
213
|
for (const s of unmatched.subs) findings.push({ type: 'subscription-no-publisher', severity: 'check', pkg: s.pkg, title: `"${s.subject}" is subscribed but never published in the repo`, detail: `Handled in ${s.file}.${dyn}` });
|
|
214
|
+
for (const w of wsList) {
|
|
215
|
+
for (const [file, info] of Object.entries(w.files).sort()) {
|
|
216
|
+
for (const spec of info.unresolved || []) findings.push({ type: 'unresolved-import', severity: 'check', pkg: w.id, title: `${file} imports ${spec}, which does not resolve to a file`, detail: 'The import is missing from the map. The file may be missing, generated at build time, or reached through an alias the analysis does not know.', files: [file] });
|
|
217
|
+
for (const expr of info.dynamic || []) findings.push({ type: 'dynamic-import', severity: 'check', pkg: w.id, title: `${file} imports a module named at runtime: ${expr}`, detail: 'Static analysis cannot tell which module this loads, so the import is missing from the map.', files: [file] });
|
|
218
|
+
}
|
|
219
|
+
}
|
|
220
|
+
const skippedBy = new Map();
|
|
221
|
+
for (const x of coverage ? coverage.skipped : []) {
|
|
222
|
+
const k = `${x.pkg || ''}|${x.reason}`;
|
|
223
|
+
if (!skippedBy.has(k)) skippedBy.set(k, { pkg: x.pkg, reason: x.reason, files: [] });
|
|
224
|
+
skippedBy.get(k).files.push(x.file);
|
|
225
|
+
}
|
|
226
|
+
for (const g of skippedBy.values()) {
|
|
227
|
+
const n = `${g.files.length} file${g.files.length > 1 ? 's' : ''}`;
|
|
228
|
+
findings.push({ type: 'skipped-files', severity: 'check', pkg: g.pkg || undefined, title: g.pkg ? `${n} in ${g.pkg} ${g.reason}` : `${n} ${g.reason}`, detail: 'These files are not on the map.', files: g.files.sort() });
|
|
229
|
+
}
|
|
230
|
+
const published = new Set(Object.values(packages).flatMap((p) => p.meta.published || []));
|
|
212
231
|
const orphans = {};
|
|
213
232
|
for (const [file, f] of Object.entries(files)) {
|
|
214
|
-
if (importedFiles.has(file) || CONVENTIONAL_ENTRY_RE.test(file) || FRAMEWORK_LOADED_RE.test(file) || f.boot.length || f.mount.length) continue;
|
|
233
|
+
if (importedFiles.has(file) || published.has(file) || CONVENTIONAL_ENTRY_RE.test(file) || FRAMEWORK_LOADED_RE.test(file) || f.boot.length || f.mount.length) continue;
|
|
215
234
|
(orphans[f.pkg] ||= []).push(file);
|
|
216
235
|
}
|
|
217
236
|
for (const [pkg, list] of Object.entries(orphans).sort()) findings.push({ type: 'orphan-files', severity: 'check', pkg, title: `${list.length} file${list.length > 1 ? 's' : ''} in ${pkg} never imported`, detail: 'Not imported by any other file and not an entry point.', files: list.sort() });
|
|
@@ -232,6 +251,7 @@ function buildModelWith({ repoName, facts, EXTERNAL_SYSTEMS, SYSTEM_OF, FOLDER_R
|
|
|
232
251
|
thirdParty: thirdPartyOut,
|
|
233
252
|
fineEdges: FE,
|
|
234
253
|
findings,
|
|
254
|
+
coverage: coverage ? { total: coverage.total, analyzed: coverage.analyzed, skipped: coverage.skipped.length } : null,
|
|
235
255
|
mq: { calls: mq.call.length, serves: mq.serve.length, pubs: mq.pub.length, subs: mq.sub.length, dynamic: mq.dynamic },
|
|
236
256
|
};
|
|
237
257
|
}
|
|
@@ -244,7 +264,7 @@ function runtimeOf(p, used) {
|
|
|
244
264
|
const ui = [['react', 'React'], ['react-dom', 'React'], ['vue', 'Vue'], ['@angular/core', 'Angular'], ['svelte', 'Svelte'], ['solid-js', 'Solid']];
|
|
245
265
|
const server = [['@nestjs/core', 'NestJS'], ['@nestjs/common', 'NestJS'], ['express', 'Express'], ['fastify', 'Fastify'], ['hono', 'Hono'], ['@temporalio/worker', 'Temporal'], ['koa', 'Koa']];
|
|
246
266
|
for (const [mod, name] of p.role === 'app' ? [...ui, ...server] : [...server, ...ui]) if (used[mod]) return name;
|
|
247
|
-
return 'TypeScript';
|
|
267
|
+
return p.lang === 'javascript' ? 'JavaScript' : 'TypeScript';
|
|
248
268
|
}
|
|
249
269
|
|
|
250
270
|
function sdkTarget(name, packages, self) {
|
package/src/template.html
CHANGED
|
@@ -271,7 +271,7 @@ details summary { cursor: pointer; color: var(--muted); font-size: 13px; }
|
|
|
271
271
|
<dt>Dependents · outside</dt><dd>What uses this folder from elsewhere.</dd>
|
|
272
272
|
<dt>Frame</dt><dd>The folder itself. Its parts sit in tiers from <b>Ln · Entry</b> (nothing in here uses it) down to <b>L0 · Base</b> (uses nothing else in here). A part sits one tier above the highest thing it imports or calls.</dd>
|
|
273
273
|
<dt>Dependencies · outside</dt><dd>Other code in the repo this folder uses.</dd>
|
|
274
|
-
<dt>External
|
|
274
|
+
<dt>External dependencies</dt><dd>Every third-party package the code imports. Known drivers and SDKs are named by the system they reach, such as PostgreSQL or Stripe.</dd>
|
|
275
275
|
</dl>
|
|
276
276
|
<p>A dashed red box is an import cycle: its parts import each other, so they share a tier.</p>
|
|
277
277
|
</section>
|
|
@@ -280,7 +280,7 @@ details summary { cursor: pointer; color: var(--muted); font-size: 13px; }
|
|
|
280
280
|
<dl>
|
|
281
281
|
<dt>Border colour</dt><dd>Green library, blue service, pink app. Dashed boxes are outside the current folder.</dd>
|
|
282
282
|
<dt><span class="plus">+</span></dt><dd>The box has inner parts you can zoom into.</dd>
|
|
283
|
-
<dt>Lines</dt><dd>Import (solid), RPC call (thick), HTTP call (blue dashed), event subscription (amber, dot end), external
|
|
283
|
+
<dt>Lines</dt><dd>Import (solid), RPC call (thick), HTTP call (blue dashed), event subscription (amber, dot end), external dependency (grey dotted). Each points from the part that depends to the part it depends on.</dd>
|
|
284
284
|
</dl>
|
|
285
285
|
</section>
|
|
286
286
|
<section>
|
|
@@ -309,7 +309,7 @@ details summary { cursor: pointer; color: var(--muted); font-size: 13px; }
|
|
|
309
309
|
</section>
|
|
310
310
|
<section>
|
|
311
311
|
<h3>Running it</h3>
|
|
312
|
-
<p><code>npx repo-blueprint build <repo></code> writes a standalone HTML file. <code>npx repo-blueprint watch <repo></code> serves it and rebuilds on code changes. A repo can
|
|
312
|
+
<p><code>npx repo-blueprint build <repo></code> writes a standalone HTML file. <code>npx repo-blueprint watch <repo></code> serves it and rebuilds on code changes. A repo can name more external systems, add message-bus method names and folder-role hints in a <code>.blueprint.json</code> at its root.</p>
|
|
313
313
|
</section>
|
|
314
314
|
</div>
|
|
315
315
|
</div>
|
|
@@ -472,6 +472,13 @@ function frameBox(layer) {
|
|
|
472
472
|
const c = layer.querySelector('.pele-cluster[data-id="FRAME"] rect');
|
|
473
473
|
return c ? svgBox(layer, c) : null;
|
|
474
474
|
}
|
|
475
|
+
// The dependency tiers of a view, without the external band: where the opening camera aims when there is no frame.
|
|
476
|
+
function levelsBox(layer) {
|
|
477
|
+
const boxes = [...layer.querySelectorAll('.pele-cluster')].filter((c) => /^L\d+$/.test(c.getAttribute('data-id'))).map((c) => svgBox(layer, c.querySelector('rect') || c));
|
|
478
|
+
if (!boxes.length) return null;
|
|
479
|
+
const x = Math.min(...boxes.map((b) => b.x));
|
|
480
|
+
return { x, w: Math.max(...boxes.map((b) => b.x + b.w)) - x };
|
|
481
|
+
}
|
|
475
482
|
function fitCam(box, pad = 0.92, maxS = 1.5) {
|
|
476
483
|
const W = stage.clientWidth, H = stage.clientHeight;
|
|
477
484
|
const s = Math.min(maxS, Math.min(W / box.w, H / box.h) * pad);
|
|
@@ -488,7 +495,7 @@ function homeCam(layer) {
|
|
|
488
495
|
const readable = W < 600 ? 0.45 : 0.7;
|
|
489
496
|
if (fit.s >= readable) return fit;
|
|
490
497
|
const s = Math.min(1, Math.max(readable, (W / v.width) * 0.96));
|
|
491
|
-
const fb = frameBox(layer) || { x: 0, w: v.width };
|
|
498
|
+
const fb = frameBox(layer) || levelsBox(layer) || { x: 0, w: v.width };
|
|
492
499
|
const x = fb.w * s > W - 24 ? 12 - fb.x * s : W / 2 - (fb.x + fb.w / 2) * s;
|
|
493
500
|
return { s, x, y: 12 };
|
|
494
501
|
}
|
|
@@ -812,6 +819,9 @@ const FINDING_TYPES = {
|
|
|
812
819
|
'rpc-no-server': { label: 'RPC without server', desc: 'An RPC subject is called, but no handler for it exists in the repo.' },
|
|
813
820
|
'event-no-subscriber': { label: 'Unheard event', desc: 'An event is published, but nothing in the repo subscribes to it.' },
|
|
814
821
|
'subscription-no-publisher': { label: 'Orphan subscription', desc: 'Something subscribes to an event that nothing in the repo publishes.' },
|
|
822
|
+
'skipped-files': { label: 'Not mapped', desc: 'Source files the analysis did not read, with the reason. Nothing they import or export is on the map.' },
|
|
823
|
+
'unresolved-import': { label: 'Unresolved import', desc: 'An import that does not resolve to a file, so it is missing from the map.' },
|
|
824
|
+
'dynamic-import': { label: 'Runtime import', desc: 'An import whose module is only known at runtime, so it is missing from the map.' },
|
|
815
825
|
'orphan-files': { label: 'Unimported files', desc: 'Files no other file imports that are not entry points. Often dead code.' },
|
|
816
826
|
};
|
|
817
827
|
const SEVERITY = {
|
|
@@ -903,9 +913,10 @@ function edgeHtml(v, e) {
|
|
|
903
913
|
|
|
904
914
|
function extHtml(key, v) {
|
|
905
915
|
const name = key.slice(4);
|
|
906
|
-
const pkgs = M.externalSystems[name]
|
|
916
|
+
const pkgs = M.externalSystems[name];
|
|
907
917
|
const users = v.all.filter((e) => e.to === key);
|
|
908
|
-
|
|
918
|
+
const what = pkgs ? `External system, outside the repo. Detected from imports of: <span class="mono">${esc(pkgs.join(', '))}</span>` : 'Third-party package, imported from outside the repo.';
|
|
919
|
+
return `<h2>${esc(name)}</h2><p class="det">${what}</p>
|
|
909
920
|
<div><h3>Used by, in this view</h3><ul>${users.map((e) => `<li><button class="link" data-k="${esc(e.from)}">${esc(labelOf(state.view, e.from))}</button><div class="det mono">${esc(e.details.join(', '))} · ${e.count} file${e.count > 1 ? 's' : ''}</div></li>`).join('')}</ul></div>`;
|
|
910
921
|
}
|
|
911
922
|
|
|
@@ -1063,7 +1074,7 @@ function renderLegend() {
|
|
|
1063
1074
|
const on = loadEdgeKinds();
|
|
1064
1075
|
$('legend').innerHTML = `<span><span class="sw" style="border-color:var(--green)"></span>library</span><span><span class="sw" style="border-color:var(--in)"></span>service</span><span><span class="sw" style="border-color:var(--out)"></span>app</span>
|
|
1065
1076
|
<span><span class="sw" style="border-color:var(--red);border-style:dashed"></span>import cycle</span>
|
|
1066
|
-
<span><span class="sw" style="border-color:var(--muted);border-style:dotted"></span>external
|
|
1077
|
+
<span><span class="sw" style="border-color:var(--muted);border-style:dotted"></span>external</span>
|
|
1067
1078
|
${EDGE_KINDS.map((e) => `<button class="tog" data-kind="${e.k}" aria-pressed="${on.has(e.k)}" title="${e.label} lines: on shows them always, off shows them only for the hovered box">${line(e.dash, e.color, e.end)}${e.label}</button>`).join('')}
|
|
1068
1079
|
`;
|
|
1069
1080
|
for (const b of $('legend').querySelectorAll('.tog')) b.addEventListener('click', () => {
|
package/src/views.mjs
CHANGED
|
@@ -206,8 +206,9 @@ function makeView(v) {
|
|
|
206
206
|
const outerNodes = (side) =>
|
|
207
207
|
[...outer.entries()]
|
|
208
208
|
.filter(([, o]) => o.side === side)
|
|
209
|
-
.map(([k]) => (k.startsWith('more:') ? { key: k, label: `+${more[side].length} more`, cls: ['outer', 'more'], outerSide: side } : { ...outerOf(k), outerSide: side }))
|
|
210
|
-
|
|
209
|
+
.map(([k, o]) => (k.startsWith('more:') ? { key: k, label: `+${more[side].length} more`, cls: ['outer', 'more'], outerSide: side, weight: 0 } : { ...outerOf(k), outerSide: side, weight: o.weight }))
|
|
210
|
+
// External dependencies read most-used first; neighbouring packages stay alphabetical.
|
|
211
|
+
.sort((a, b) => (a.key.startsWith('more:') ? 1 : 0) - (b.key.startsWith('more:') ? 1 : 0) || (side === 'ext' ? b.weight - a.weight : 0) || a.label.localeCompare(b.label));
|
|
211
212
|
const ups = v.frame ? outerNodes('up') : [];
|
|
212
213
|
const downs = v.frame ? outerNodes('down') : [];
|
|
213
214
|
const exts = outerNodes('ext');
|
|
@@ -246,7 +247,7 @@ function makeView(v) {
|
|
|
246
247
|
lines.push(' end');
|
|
247
248
|
}
|
|
248
249
|
if (exts.length) {
|
|
249
|
-
lines.push(' subgraph EXT["External
|
|
250
|
+
lines.push(' subgraph EXT["External dependencies"]');
|
|
250
251
|
chunk(exts, ' ');
|
|
251
252
|
lines.push(' end');
|
|
252
253
|
}
|