@vibgrate/cli 1.0.79 → 1.0.81
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DOCS.md +14 -3
- package/HCS-EXTRACT.md +442 -0
- package/HCS-RUNTIME-SETUP.md +73 -0
- package/LICENSE.md +1 -1
- package/README.md +1 -1
- package/dist/{baseline-NRKROYVK.js → baseline-5AZAIOQ6.js} +2 -2
- package/dist/{chunk-DKSPLRJV.js → chunk-DEPG5EIH.js} +1 -1
- package/dist/{chunk-TKNDQ337.js → chunk-W6TXQI4D.js} +27 -22
- package/dist/cli.js +170 -16
- package/dist/hcs-worker.js +562 -25
- package/dist/index.d.ts +3 -3
- package/dist/index.js +1 -1
- package/package.json +4 -2
package/DOCS.md
CHANGED
|
@@ -59,12 +59,13 @@ For a quick overview, see the [README](./README.md). This document covers everyt
|
|
|
59
59
|
- [Privacy & Security](#privacy--security)
|
|
60
60
|
- [Exit Codes](#exit-codes)
|
|
61
61
|
- [Programmatic API](#programmatic-api)
|
|
62
|
+
- [HCS Extract Runtime Prerequisites](#hcs-extract-runtime-prerequisites)
|
|
62
63
|
|
|
63
64
|
---
|
|
64
65
|
|
|
65
66
|
## How It Works
|
|
66
67
|
|
|
67
|
-
Vibgrate recursively scans your repository for `package.json` (Node/TypeScript), `.sln`/`.csproj` (.NET), Python manifests, and Java build manifests. For each project it discovers, it:
|
|
68
|
+
Vibgrate recursively scans your repository for `package.json` (Node/TypeScript), `.sln`/`.csproj`/`.vbproj` (.NET), Python manifests, and Java build manifests. For each project it discovers, it:
|
|
68
69
|
|
|
69
70
|
1. **Detects** the runtime version, target framework, and all dependencies
|
|
70
71
|
2. **Queries** the npm/NuGet registry for latest stable versions (with built-in caching and concurrency control)
|
|
@@ -101,7 +102,7 @@ This section summarizes what the CLI supports today and how to use each capabili
|
|
|
101
102
|
Vibgrate currently discovers and evaluates projects in:
|
|
102
103
|
|
|
103
104
|
- **Node.js / TypeScript** (`package.json`, lockfiles)
|
|
104
|
-
- **.NET** (`.sln`, `.csproj`)
|
|
105
|
+
- **.NET** (`.sln`, `.csproj`, `.vbproj`)
|
|
105
106
|
- **Python** (`requirements.txt`, `pyproject.toml`-style manifests)
|
|
106
107
|
- **Java** (`pom.xml`, Gradle-style manifests)
|
|
107
108
|
|
|
@@ -704,7 +705,7 @@ Vibgrate is built with a privacy-first architecture. Here's what it **never** do
|
|
|
704
705
|
|
|
705
706
|
What it **does** collect:
|
|
706
707
|
|
|
707
|
-
- Package names and version numbers (from `package.json`, `.csproj`, lockfiles)
|
|
708
|
+
- Package names and version numbers (from `package.json`, `.csproj`/`.vbproj`, lockfiles)
|
|
708
709
|
- Config structure flags (e.g. `strict: true` from `tsconfig.json`)
|
|
709
710
|
- File names and sizes (paths and metadata, never contents)
|
|
710
711
|
- Public npm/NuGet registry metadata (latest versions, deprecation flags)
|
|
@@ -752,3 +753,13 @@ import type {
|
|
|
752
753
|
---
|
|
753
754
|
|
|
754
755
|
Copyright © 2026 Vibgrate. All rights reserved. See [LICENSE](https://vibgrate.com/license) for terms.
|
|
756
|
+
|
|
757
|
+
## HCS Extract Runtime Prerequisites
|
|
758
|
+
|
|
759
|
+
When using `vibgrate extract`, language workers run as subprocesses.
|
|
760
|
+
|
|
761
|
+
- Swift currently uses the Node worker (text extraction), so no Swift/Xcode install is required for HCS extract at this time.
|
|
762
|
+
- Go/Python/Java/.NET may require local runtimes (`go`, `python3`, `java`, `dotnet`) when platform-native workers are not bundled.
|
|
763
|
+
- The CLI now emits install hints when these commands are missing on PATH.
|
|
764
|
+
|
|
765
|
+
For OS-specific setup and packaging strategy, see [`HCS-RUNTIME-SETUP.md`](./HCS-RUNTIME-SETUP.md).
|
package/HCS-EXTRACT.md
ADDED
|
@@ -0,0 +1,442 @@
|
|
|
1
|
+
# HCS Extract — Architecture & Interface Guide
|
|
2
|
+
|
|
3
|
+
How `vibgrate extract` discovers source code, spawns HCS workers, validates
|
|
4
|
+
facts, and renders progress.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Command synopsis
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
vibgrate extract [path] [options]
|
|
12
|
+
|
|
13
|
+
Options:
|
|
14
|
+
-o, --out <file> Write NDJSON to file (default: stdout)
|
|
15
|
+
--language <langs> Comma-separated languages (default: auto-detect)
|
|
16
|
+
--include-tests Include test files in analysis
|
|
17
|
+
--push Stream validated facts to dashboard API
|
|
18
|
+
--dsn <dsn> DSN token for push (or VIBGRATE_DSN env)
|
|
19
|
+
--concurrency <n> Parallel file workers (default: CPU count)
|
|
20
|
+
--timeout-mins <mins> Total timeout in minutes (default: 60)
|
|
21
|
+
--feedback <file> Load NDJSON diff artifact for refinement
|
|
22
|
+
--verbose Print worker stderr and summary statistics
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## High-level architecture
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
31
|
+
│ vibgrate CLI (ESM) │
|
|
32
|
+
│ packages/vibgrate-cli │
|
|
33
|
+
├─────────────────────────────────────────────────────────────┤
|
|
34
|
+
│ cli.ts │
|
|
35
|
+
│ └── extractCommand (src/commands/extract.ts) │
|
|
36
|
+
│ └── diffCommand (src/commands/diff.ts) │
|
|
37
|
+
│ │
|
|
38
|
+
│ Extract orchestration: │
|
|
39
|
+
│ 1. Validate args & options │
|
|
40
|
+
│ 2. Detect languages (walk filesystem for extensions) │
|
|
41
|
+
│ 3. Spawn HCS worker process per language │
|
|
42
|
+
│ 4. Parse stdout (NDJSON facts) + stderr (progress/logs) │
|
|
43
|
+
│ 5. Validate FactEnvelope schema for every line │
|
|
44
|
+
│ 6. Write to stdout or --out file │
|
|
45
|
+
│ 7. Optionally --push to dashboard API │
|
|
46
|
+
└───────────────┬─────────────────────────────────────────────┘
|
|
47
|
+
│ spawns: node --enable-source-maps <worker> --project <dir>
|
|
48
|
+
▼
|
|
49
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
50
|
+
│ HCS Node Worker (ESM, tsup bundle) │
|
|
51
|
+
│ packages/vibgrate-hcs/node │
|
|
52
|
+
├─────────────────────────────────────────────────────────────┤
|
|
53
|
+
│ dist/main.js (~4.8 MB, bundles antlr4ts + vb6-antlr4) │
|
|
54
|
+
│ │
|
|
55
|
+
│ AST extractors (ts-morph): │
|
|
56
|
+
│ typescript, javascript │
|
|
57
|
+
│ │
|
|
58
|
+
│ Text/regex extractors: │
|
|
59
|
+
│ swift, rust, ruby, php, dart, scala, vb6, cobol │
|
|
60
|
+
│ │
|
|
61
|
+
│ STDOUT ─▶ NDJSON FactEnvelope lines │
|
|
62
|
+
│ STDERR ─▶ [info], [warn], [error], [progress] lines │
|
|
63
|
+
└─────────────────────────────────────────────────────────────┘
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Package dependency chain
|
|
69
|
+
|
|
70
|
+
```
|
|
71
|
+
@vibgrate/cli
|
|
72
|
+
└── @vibgrate/hcs-node-worker (workspace:*)
|
|
73
|
+
├── ts-morph (external — resolved from node_modules at runtime)
|
|
74
|
+
├── commander (external)
|
|
75
|
+
├── antlr4ts (bundled into dist/main.js by tsup)
|
|
76
|
+
└── vb6-antlr4 (bundled into dist/main.js by tsup)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### Build order
|
|
80
|
+
|
|
81
|
+
The CLI's `build` and `build:only` scripts automatically build the HCS worker
|
|
82
|
+
first via `pnpm run build:hcs`:
|
|
83
|
+
|
|
84
|
+
```json
|
|
85
|
+
{
|
|
86
|
+
"build": "pnpm run build:hcs && tsup ... && pnpm test:solutions",
|
|
87
|
+
"build:only": "pnpm run build:hcs && tsup ...",
|
|
88
|
+
"build:hcs": "pnpm --filter @vibgrate/hcs-node-worker build"
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
The root-level `pnpm -r build` also resolves the correct order because pnpm
|
|
93
|
+
sees the `workspace:*` dependency and topologically sorts accordingly.
|
|
94
|
+
|
|
95
|
+
### Publishing
|
|
96
|
+
|
|
97
|
+
| Package | `files` field | Notes |
|
|
98
|
+
|---------|---------------|-------|
|
|
99
|
+
| `@vibgrate/cli` | `["dist", "DOCS.md"]` | Built CLI + library |
|
|
100
|
+
| `@vibgrate/hcs-node-worker` | `["dist"]` | Single-file worker bundle |
|
|
101
|
+
|
|
102
|
+
When `@vibgrate/cli` is installed via npm, `@vibgrate/hcs-node-worker` is
|
|
103
|
+
pulled in as a normal dependency. Its `dist/main.js` self-contains the ANTLR
|
|
104
|
+
parsers; only `ts-morph` and `commander` are resolved transitively from
|
|
105
|
+
node_modules.
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## Worker resolution
|
|
110
|
+
|
|
111
|
+
`extract.ts` → `resolveHcsWorkerBin()` locates the worker binary at runtime:
|
|
112
|
+
|
|
113
|
+
| Priority | Method | When |
|
|
114
|
+
|----------|--------|------|
|
|
115
|
+
| 1 | `import.meta.resolve('@vibgrate/hcs-node-worker')` | Installed package or pnpm workspace link |
|
|
116
|
+
| 2 | Monorepo sibling path: `../../vibgrate-hcs/node/dist/main.js` | Dev fallback |
|
|
117
|
+
| 3 | Source path: `../../vibgrate-hcs/node/src/main.ts` | `tsx` dev mode |
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## Language detection
|
|
122
|
+
|
|
123
|
+
When `--language` is omitted, `extract.ts` walks the target directory and
|
|
124
|
+
counts files by extension:
|
|
125
|
+
|
|
126
|
+
| Language | Extensions |
|
|
127
|
+
|----------|-----------|
|
|
128
|
+
| typescript | `.ts`, `.tsx`, `.mts`, `.cts` |
|
|
129
|
+
| javascript | `.js`, `.jsx`, `.mjs`, `.cjs` |
|
|
130
|
+
| swift | `.swift` |
|
|
131
|
+
| rust | `.rs` |
|
|
132
|
+
| ruby | `.rb`, `.rake`, `.gemspec` |
|
|
133
|
+
| php | `.php` |
|
|
134
|
+
| dart | `.dart` |
|
|
135
|
+
| scala | `.scala`, `.sc` |
|
|
136
|
+
| cobol | `.cbl`, `.cob`, `.cpy`, `.cob85`, `.cobol` |
|
|
137
|
+
| vb6 | `.vbp`, `.bas`, `.cls`, `.frm`, `.ctl`, `.dsr` |
|
|
138
|
+
| go | `.go` |
|
|
139
|
+
| python | `.py` |
|
|
140
|
+
| java | `.java` |
|
|
141
|
+
| csharp | `.cs` |
|
|
142
|
+
| cplusplus | `.vcxproj`, `.cpp`, `.cc`, `.cxx`, `.hpp`, `.hh`, `.hxx`, `.h`, `.ixx` |
|
|
143
|
+
| vbnet | `.vb` |
|
|
144
|
+
|
|
145
|
+
Directories in `SKIP_DIRS` (`node_modules`, `.git`, `dist`, `build`, `target`,
|
|
146
|
+
`vendor`, etc.) are skipped. Test files are excluded unless `--include-tests`
|
|
147
|
+
is set.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
151
|
+
## Worker dispatch
|
|
152
|
+
|
|
153
|
+
Each detected language spawns a separate worker process:
|
|
154
|
+
|
|
155
|
+
```
|
|
156
|
+
node --enable-source-maps <worker> --project <dir> # TS/JS
|
|
157
|
+
node --enable-source-maps <worker> --project <dir> --swift-project <dir> # text langs
|
|
158
|
+
node --enable-source-maps <worker> --project <dir> --cobol-project <dir> \
|
|
159
|
+
--cobol-copybook-paths <paths> # COBOL
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
The `--concurrency` flag controls how many workers run in parallel (default:
|
|
163
|
+
CPU count), managed by a `Semaphore` utility.
|
|
164
|
+
|
|
165
|
+
For native workers (Go/Python/Java/.NET), if no bundled worker is available, the CLI falls back to local toolchains and emits installation hints when commands are missing. See [HCS-RUNTIME-SETUP.md](./HCS-RUNTIME-SETUP.md).
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Fact Envelope schema (ABI v0.2)
|
|
170
|
+
|
|
171
|
+
Every NDJSON line emitted on stdout must conform to:
|
|
172
|
+
|
|
173
|
+
```typescript
|
|
174
|
+
interface FactEnvelope {
|
|
175
|
+
factId: string; // "hcs:<FactType>:<sha256-16>"
|
|
176
|
+
factType: string; // e.g. "SymbolDeclared", "CallObserved"
|
|
177
|
+
language: string; // e.g. "typescript", "cobol"
|
|
178
|
+
scanner: string; // "ts-morph"
|
|
179
|
+
scannerVersion: string; // "0.2.0"
|
|
180
|
+
emittedAt: string; // ISO 8601 timestamp
|
|
181
|
+
payload: unknown; // Fact-type-specific payload object
|
|
182
|
+
}
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
The CLI validates every line against this schema. Invalid lines cause exit
|
|
186
|
+
code `1` (schema failure).
|
|
187
|
+
|
|
188
|
+
### Fact types emitted per language
|
|
189
|
+
|
|
190
|
+
| Language | Fact types |
|
|
191
|
+
|----------|-----------|
|
|
192
|
+
| typescript | CallObserved, DataAccessOperation, EndpointMetadata, FileIndexed, ImportObserved, RouteDeclared, SqlStatementObserved, SqlUsage, SymbolDeclared |
|
|
193
|
+
| javascript | ApiClientCall, CallObserved, DataAccessOperation, EndpointMetadata, ExternalServiceCall, FileIndexed, ImportObserved, RouteDeclared, SqlStatementObserved, SqlUsage, SymbolDeclared |
|
|
194
|
+
| swift | ApiClientCall, ConfigAccess, PageRouteDeclared |
|
|
195
|
+
| cplusplus | FileIndexed, ImportObserved, RouteDeclared, SqlUsage, SymbolDeclared, TestCaseDeclared |
|
|
196
|
+
| rust | AnnotationFound, ExternalServiceCall, FileIndexed, ImportObserved, RouteDeclared, SymbolDeclared, TestCaseDeclared |
|
|
197
|
+
| ruby | ConfigAccess, ExternalServiceCall, FileIndexed, ImportObserved, RouteDeclared, SqlUsage, SymbolDeclared |
|
|
198
|
+
| php | ConfigAccess, ExternalServiceCall, FileIndexed, ImportObserved, SqlUsage, SymbolDeclared |
|
|
199
|
+
| dart | ExternalServiceCall, FileIndexed, ImportObserved, RouteDeclared, SymbolDeclared |
|
|
200
|
+
| scala | ConfigAccess, FileIndexed, ImportObserved, SymbolDeclared |
|
|
201
|
+
| vb6 | AnnotationFound, CallObserved, ConfigAccess, FieldDeclared, SqlUsage, SymbolDeclared |
|
|
202
|
+
| cobol | CallObserved, CobolConfidenceReport, CobolProgramDeclared, ExecBlockObserved, FileResourceDeclared, RecordLayoutDeclared, SqlStatementObserved, SqlUsage, SymbolDeclared |
|
|
203
|
+
|
|
204
|
+
---
|
|
205
|
+
|
|
206
|
+
## AST extractors (TypeScript / JavaScript)
|
|
207
|
+
|
|
208
|
+
These use [ts-morph](https://ts-morph.com) to build a full TypeScript AST and
|
|
209
|
+
run extractors in dependency order:
|
|
210
|
+
|
|
211
|
+
| # | Extractor | Emits |
|
|
212
|
+
|---|-----------|-------|
|
|
213
|
+
| 1 | `FileIndexedExtractor` | `FileIndexed` |
|
|
214
|
+
| 2 | `SymbolAndCallGraphExtractor` | `SymbolDeclared`, `CallObserved` |
|
|
215
|
+
| 3 | `ImportObservedExtractor` | `ImportObserved` |
|
|
216
|
+
| 4 | `RouteExtractor` | `RouteDeclared`, `EndpointMetadata` |
|
|
217
|
+
| 5 | `EntityExtractor` | `EntityDeclared`, `FieldDeclared` |
|
|
218
|
+
| 6 | `AuthExtractor` | `AuthBoundaryDeclared`, `RoleRequirement` |
|
|
219
|
+
| 7 | `SqlUsageExtractor` | `SqlUsage` |
|
|
220
|
+
| 8 | `DataAccessOperationExtractor` | `DataAccessOperation` |
|
|
221
|
+
| 9 | `ConfigExtractor` | `ConfigAccess` |
|
|
222
|
+
| 10 | `TestCaseDeclaredExtractor` | `TestCaseDeclared` |
|
|
223
|
+
| 11 | `MigrationDeclaredExtractor` | `MigrationDeclared` |
|
|
224
|
+
| 12 | `EventEmittedExtractor` | `EventEmitted` |
|
|
225
|
+
| 13 | `EventConsumedExtractor` | `EventConsumed` |
|
|
226
|
+
| 14 | `ExternalServiceCallExtractor` | `ExternalServiceCall`, `ApiClientCall` |
|
|
227
|
+
| 15 | `SqlObjectDeclaredExtractor` | `SqlObjectDeclared` |
|
|
228
|
+
| 16 | `ComponentExtractor` | `ComponentDeclared` |
|
|
229
|
+
| 17 | `ClientRouteExtractor` | `PageRouteDeclared` |
|
|
230
|
+
| 18 | `StateExtractor` | `StateStoreDeclared` |
|
|
231
|
+
|
|
232
|
+
### Text extractors (regex-based, no AST)
|
|
233
|
+
|
|
234
|
+
| Language | Extractor classes |
|
|
235
|
+
|----------|------------------|
|
|
236
|
+
| Swift | `SwiftApiCallExtractor`, `SwiftConfigExtractor`, `SwiftNavigationExtractor` |
|
|
237
|
+
| VB6 | `VB6ProjectExtractor`, `VB6SymbolExtractor`, `VB6SqlExtractor`, `VB6ConfigExtractor`, `VB6FormsExtractor` |
|
|
238
|
+
| Rust | `RustExtractor` |
|
|
239
|
+
| Ruby | `RubyExtractor` |
|
|
240
|
+
| PHP | `PhpExtractor` |
|
|
241
|
+
| Dart | `DartExtractor` |
|
|
242
|
+
| Scala | `ScalaExtractor` |
|
|
243
|
+
| C++ | `CppExtractor` |
|
|
244
|
+
| COBOL | `CobolAdapter` (wraps ANTLR parser + preprocessor + SQL/data/procedure extractors) |
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## Progress reporting protocol
|
|
249
|
+
|
|
250
|
+
The HCS worker emits structured progress events on **stderr** using the
|
|
251
|
+
`[progress]` prefix. The CLI parses these for live terminal display.
|
|
252
|
+
|
|
253
|
+
### Worker → CLI protocol
|
|
254
|
+
|
|
255
|
+
```
|
|
256
|
+
[progress] {"phase":"discovering","language":"typescript"}
|
|
257
|
+
[progress] {"phase":"scanning","language":"rust","fileCount":142,"fileIndex":0}
|
|
258
|
+
[progress] {"phase":"scanning","language":"rust","file":"src/main.rs","fileIndex":1,"fileCount":142}
|
|
259
|
+
[progress] {"phase":"extracting","extractor":"SymbolAndCallGraphExtractor","fileIndex":2,"fileCount":18}
|
|
260
|
+
[progress] {"phase":"done","language":"rust","fileCount":142}
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
### Progress event schema
|
|
264
|
+
|
|
265
|
+
```typescript
|
|
266
|
+
interface ProgressEvent {
|
|
267
|
+
phase: "discovering" | "scanning" | "extracting" | "done";
|
|
268
|
+
language?: string;
|
|
269
|
+
file?: string; // Absolute path of file being processed
|
|
270
|
+
fileIndex?: number; // 1-based current file/extractor index
|
|
271
|
+
fileCount?: number; // Total files or extractors
|
|
272
|
+
extractor?: string; // Extractor class name (AST extractors)
|
|
273
|
+
}
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
### CLI progress rendering
|
|
277
|
+
|
|
278
|
+
The `ProgressTracker` class in `extract.ts` renders a single-line progress
|
|
279
|
+
bar on stderr:
|
|
280
|
+
|
|
281
|
+
```
|
|
282
|
+
typescript 12/42 29% rust ✓ ruby extracting 23 facts
|
|
283
|
+
```
|
|
284
|
+
|
|
285
|
+
| Behaviour | TTY (interactive) | Pipe / CI |
|
|
286
|
+
|-----------|-------------------|-----------|
|
|
287
|
+
| Display | `\r`-overwriting single line, 12 fps throttle | No progress line (clean logs) |
|
|
288
|
+
| Facts | Live counter | `--verbose` summary only |
|
|
289
|
+
| Completion | Line cleared, summary printed | Summary only |
|
|
290
|
+
|
|
291
|
+
stderr is used exclusively so that stdout remains pure NDJSON.
|
|
292
|
+
|
|
293
|
+
---
|
|
294
|
+
|
|
295
|
+
## I/O channel separation
|
|
296
|
+
|
|
297
|
+
```
|
|
298
|
+
┌──────────────┐ stdout (NDJSON) ┌──────────────┐
|
|
299
|
+
│ HCS Worker │ ─────────────────────▶ │ CLI │ ──▶ stdout / --out file
|
|
300
|
+
│ │ stderr │ │
|
|
301
|
+
│ │ ─────────────────────▶ │ (progress │ ──▶ stderr (progress/logs)
|
|
302
|
+
│ │ [info] [progress] etc │ tracker) │
|
|
303
|
+
└──────────────┘ └──────────────┘
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
- **stdout**: Only NDJSON `FactEnvelope` lines. One JSON object per line.
|
|
307
|
+
The CLI validates each line, collects valid facts, sorts them, and writes
|
|
308
|
+
to stdout (or `--out` file).
|
|
309
|
+
- **stderr**: Human-readable logs (`[info]`, `[warn]`, `[error]`) and
|
|
310
|
+
machine-readable `[progress]` JSON lines. Shown in `--verbose` mode
|
|
311
|
+
(except progress, which is always parsed).
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
## Exit codes
|
|
316
|
+
|
|
317
|
+
| Code | Meaning |
|
|
318
|
+
|------|---------|
|
|
319
|
+
| 0 | Success |
|
|
320
|
+
| 1 | Schema validation failure (invalid fact envelope) |
|
|
321
|
+
| 2 | Worker parse failure |
|
|
322
|
+
| 3 | Timeout exceeded |
|
|
323
|
+
| 4 | Push to dashboard API failed |
|
|
324
|
+
| 5 | Usage error (bad args) |
|
|
325
|
+
|
|
326
|
+
---
|
|
327
|
+
|
|
328
|
+
## ESM bundling details
|
|
329
|
+
|
|
330
|
+
The HCS worker is built with [tsup](https://tsup.egoist.dev) using
|
|
331
|
+
`packages/vibgrate-hcs/node/tsup.config.ts`:
|
|
332
|
+
|
|
333
|
+
```typescript
|
|
334
|
+
export default defineConfig({
|
|
335
|
+
entry: ["src/main.ts"],
|
|
336
|
+
format: ["esm"],
|
|
337
|
+
target: "es2022",
|
|
338
|
+
platform: "node",
|
|
339
|
+
noExternal: ["antlr4ts", "vb6-antlr4"], // Bundle ANTLR parsers
|
|
340
|
+
esbuildOptions(options) {
|
|
341
|
+
options.banner = {
|
|
342
|
+
js: [
|
|
343
|
+
'#!/usr/bin/env node',
|
|
344
|
+
'import { createRequire as __cjsRequire } from "node:module";',
|
|
345
|
+
'const require = __cjsRequire(import.meta.url);',
|
|
346
|
+
].join('\n'),
|
|
347
|
+
};
|
|
348
|
+
},
|
|
349
|
+
});
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
Key decisions:
|
|
353
|
+
- `antlr4ts` and `vb6-antlr4` are **bundled** because their generated ANTLR
|
|
354
|
+
files use extensionless imports that fail under Node ESM strict resolution.
|
|
355
|
+
- A `createRequire` banner shim is injected **before** esbuild's `__require`
|
|
356
|
+
IIFE so that bundled CJS code (antlr4ts internals) can call `require()`
|
|
357
|
+
without throwing `ERR_REQUIRE_ESM`.
|
|
358
|
+
- `ts-morph` and `commander` remain external — resolved normally from
|
|
359
|
+
node_modules at runtime.
|
|
360
|
+
|
|
361
|
+
---
|
|
362
|
+
|
|
363
|
+
## Integration tests
|
|
364
|
+
|
|
365
|
+
```
|
|
366
|
+
pnpm test:hcs # from packages/vibgrate-cli
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
Runs `test-hcs/test-runner.ts` which exercises all 10 languages against the
|
|
370
|
+
**built** CLI binary. Each language has fixture files under
|
|
371
|
+
`test-hcs/fixtures/<language>/`.
|
|
372
|
+
|
|
373
|
+
| Check | Description |
|
|
374
|
+
|-------|-------------|
|
|
375
|
+
| Exit code | Must be 0 |
|
|
376
|
+
| NDJSON validity | Every stdout line must parse as valid JSON |
|
|
377
|
+
| FactEnvelope schema | All required fields present and typed |
|
|
378
|
+
| Min fact count | Language-specific minimum (3–5) |
|
|
379
|
+
| Required fact types | Language-specific fact types must appear |
|
|
380
|
+
| Language field | Must match the expected language |
|
|
381
|
+
| factId uniqueness | All factIds should be unique (warning if not) |
|
|
382
|
+
|
|
383
|
+
### Running tests
|
|
384
|
+
|
|
385
|
+
```bash
|
|
386
|
+
# Full build + test
|
|
387
|
+
pnpm build:only && pnpm test:hcs
|
|
388
|
+
|
|
389
|
+
# Just tests (assumes built)
|
|
390
|
+
pnpm test:hcs
|
|
391
|
+
|
|
392
|
+
# Verbose output
|
|
393
|
+
pnpm test:hcs -- --verbose
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
---
|
|
397
|
+
|
|
398
|
+
## Directories
|
|
399
|
+
|
|
400
|
+
```
|
|
401
|
+
packages/vibgrate-cli/
|
|
402
|
+
src/commands/extract.ts # CLI extract orchestration + progress tracker
|
|
403
|
+
src/commands/diff.ts # CLI diff command (NDJSON delta comparison)
|
|
404
|
+
test-hcs/
|
|
405
|
+
test-runner.ts # Integration test runner (10 languages)
|
|
406
|
+
fixtures/ # Per-language fixture projects
|
|
407
|
+
typescript/ # Express + Prisma app
|
|
408
|
+
javascript/ # Express + SQL + axios app
|
|
409
|
+
swift/ # SwiftUI + URLSession app
|
|
410
|
+
rust/ # actix-web + sqlx + reqwest app
|
|
411
|
+
ruby/ # Sinatra + ActiveRecord app
|
|
412
|
+
php/ # Laravel controller
|
|
413
|
+
dart/ # Flutter + http app
|
|
414
|
+
scala/ # Akka HTTP + Slick app
|
|
415
|
+
vb6/ # VB6 module (Type, Sub, Function, SQL)
|
|
416
|
+
cobol/ # COBOL with EXEC SQL + file I/O
|
|
417
|
+
|
|
418
|
+
packages/vibgrate-hcs/node/
|
|
419
|
+
src/main.ts # Worker entry — commander, dispatch, progress
|
|
420
|
+
src/facts/
|
|
421
|
+
fact-emitter.ts # NdjsonFactEmitter (FactEnvelope, emit helpers)
|
|
422
|
+
fact-types.ts # TypeScript interfaces for all payload types
|
|
423
|
+
src/extractors/
|
|
424
|
+
extractor.ts # Extractor interface (AST-based)
|
|
425
|
+
text-extractor.ts # TextExtractor interface (regex-based)
|
|
426
|
+
file-indexed.ts # FileIndexedExtractor
|
|
427
|
+
symbol-call-graph.ts # SymbolAndCallGraphExtractor
|
|
428
|
+
import-observed.ts # ImportObservedExtractor
|
|
429
|
+
routes.ts # RouteExtractor
|
|
430
|
+
... # 18 AST extractors total
|
|
431
|
+
rust-extractor.ts # RustExtractor
|
|
432
|
+
ruby-extractor.ts # RubyExtractor
|
|
433
|
+
php-extractor.ts # PhpExtractor
|
|
434
|
+
dart-extractor.ts # DartExtractor
|
|
435
|
+
scala-extractor.ts # ScalaExtractor
|
|
436
|
+
swift-*.ts # 3 Swift extractors
|
|
437
|
+
vb6-*.ts # 5 VB6 extractors
|
|
438
|
+
cobol-adapter.ts # CobolAdapter (wraps ANTLR pipeline)
|
|
439
|
+
cobol-*.ts # COBOL parser + 5 sub-extractors
|
|
440
|
+
tsup.config.ts # Build config (bundle ANTLR, createRequire shim)
|
|
441
|
+
dist/main.js # Built worker (~4.8 MB)
|
|
442
|
+
```
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# HCS Runtime Setup (Windows, macOS, Linux)
|
|
2
|
+
|
|
3
|
+
This guide explains what must be installed to run `vibgrate extract` across languages.
|
|
4
|
+
|
|
5
|
+
## Important behavior
|
|
6
|
+
|
|
7
|
+
- **Swift scanning currently uses the Node worker (text/regex extraction)**, so no Swift/Xcode toolchain is required today.
|
|
8
|
+
- **Native AST scanners** currently exist for: Go, Python, Java, and .NET (C#).
|
|
9
|
+
- The CLI tries to run **bundled native workers** first when present in `dist/workers/`.
|
|
10
|
+
- If no bundled worker exists, the CLI falls back to local toolchains (`go`, `python3`, `java`, `dotnet`).
|
|
11
|
+
|
|
12
|
+
## What the CLI can and cannot auto-install
|
|
13
|
+
|
|
14
|
+
- The CLI **does not auto-install system runtimes/toolchains**.
|
|
15
|
+
- On first run, if a required runtime is missing, CLI errors include install hints.
|
|
16
|
+
- Recommended enterprise approach: pre-provision toolchains in base images or publish bundled workers for each target OS.
|
|
17
|
+
|
|
18
|
+
## Prerequisites by language
|
|
19
|
+
|
|
20
|
+
### Go (`--language go`)
|
|
21
|
+
|
|
22
|
+
- Runtime command needed when no bundled worker exists: `go`
|
|
23
|
+
- Install docs: https://go.dev/doc/install
|
|
24
|
+
|
|
25
|
+
Quick install examples:
|
|
26
|
+
- macOS: `brew install go`
|
|
27
|
+
- Windows: `winget install GoLang.Go`
|
|
28
|
+
- Linux: install from `https://go.dev/dl/` or distro package manager
|
|
29
|
+
|
|
30
|
+
### Python (`--language python`)
|
|
31
|
+
|
|
32
|
+
- Runtime command needed when no bundled worker exists: `python3`
|
|
33
|
+
- Install docs: https://www.python.org/downloads/
|
|
34
|
+
|
|
35
|
+
Quick install examples:
|
|
36
|
+
- macOS: `brew install python`
|
|
37
|
+
- Windows: `winget install Python.Python.3.12`
|
|
38
|
+
- Linux: install Python 3.10+ from distro package manager
|
|
39
|
+
|
|
40
|
+
### Java (`--language java`)
|
|
41
|
+
|
|
42
|
+
- Runtime command needed when no bundled worker exists: `java` (JRE/JDK 17+)
|
|
43
|
+
- Install docs: https://adoptium.net/
|
|
44
|
+
|
|
45
|
+
Quick install examples:
|
|
46
|
+
- macOS: `brew install --cask temurin`
|
|
47
|
+
- Windows: `winget install EclipseAdoptium.Temurin.17.JRE`
|
|
48
|
+
- Linux: install OpenJDK 17+
|
|
49
|
+
|
|
50
|
+
### C# / .NET (`--language csharp`) and VB.NET (`--language vbnet` or `--language vb.net`)
|
|
51
|
+
|
|
52
|
+
Resolution order in CLI:
|
|
53
|
+
1. Platform-native bundled worker executable (if present)
|
|
54
|
+
2. `dotnet VibgrateHcsWorker.dll`
|
|
55
|
+
3. `dotnet run --project VibgrateHcsWorker.csproj` (source mode)
|
|
56
|
+
|
|
57
|
+
- Runtime command needed for fallback modes: `dotnet`
|
|
58
|
+
- Install docs: https://dotnet.microsoft.com/download
|
|
59
|
+
|
|
60
|
+
Quick install examples:
|
|
61
|
+
- macOS: `brew install --cask dotnet-sdk`
|
|
62
|
+
- Windows: `winget install Microsoft.DotNet.SDK.8`
|
|
63
|
+
- Linux: install .NET 8 SDK/runtime from Microsoft packages
|
|
64
|
+
|
|
65
|
+
## Packaging guidance for full coverage
|
|
66
|
+
|
|
67
|
+
For easiest user experience across Windows/macOS/Linux:
|
|
68
|
+
|
|
69
|
+
1. Build and ship worker artifacts per OS/arch in `dist/workers/`.
|
|
70
|
+
2. Keep runtime fallback for dev workflows.
|
|
71
|
+
3. Document minimum versions in release notes and this setup guide.
|
|
72
|
+
4. In CI images, pre-install toolchains if not shipping bundled workers.
|
|
73
|
+
|
package/LICENSE.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Vibgrate Proprietary License
|
|
2
2
|
|
|
3
|
-
Copyright
|
|
3
|
+
Copyright © 2026 Vibgrate. All rights reserved.
|
|
4
4
|
|
|
5
5
|
Permission is hereby granted, free of charge, to any person or organisation
|
|
6
6
|
obtaining a copy of this software in compiled/distributed form (the "Software"),
|
package/README.md
CHANGED
|
@@ -116,7 +116,7 @@ We keep output plain and operational: easy to convert into backlog items and CI
|
|
|
116
116
|
Vibgrate recursively scans mixed repositories and supports:
|
|
117
117
|
|
|
118
118
|
- Node.js / TypeScript (`package.json`)
|
|
119
|
-
- .NET (`.sln`, `.csproj`)
|
|
119
|
+
- .NET (`.sln`, `.csproj`, `.vbproj`)
|
|
120
120
|
- Python (`requirements.txt`, `pyproject.toml` style ecosystems)
|
|
121
121
|
- Java (`pom.xml`, Gradle-style manifest ecosystems)
|
|
122
122
|
|