@dailephd/my-dev-kit-lab 0.3.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -8
- package/dist/scripts/audits/runAudit.js +1 -0
- package/dist/src/audits/codeRot/detectors/deadCodeCandidateDetector.js +43 -10
- package/dist/src/audits/codeRot/detectors/duplicateImplementationDetector.js +78 -1
- package/dist/src/audits/codeRot/detectors/testRotDetector.js +62 -0
- package/dist/src/audits/codeRot/utils/filePatternUtils.js +32 -0
- package/dist/src/audits/codeRot/utils/sourceFactsLookup.js +53 -0
- package/dist/src/audits/core/auditRunner.js +4 -1
- package/dist/src/audits/core/collectSourceFacts.js +142 -0
- package/dist/src/audits/core/languageAnalyzerRegistry.js +33 -0
- package/dist/src/audits/core/projectInventory.js +150 -0
- package/dist/src/audits/core/sourceFacts.js +17 -0
- package/dist/src/audits/core/typescriptJavaScriptAnalyzer.js +299 -0
- package/dist/src/audits/report/auditReportModel.js +12 -0
- package/dist/src/audits/report/renderAuditJsonReport.js +1 -0
- package/dist/src/audits/report/renderAuditTextReport.js +19 -3
- package/package.json +8 -5
package/README.md
CHANGED
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
# my-dev-kit-lab
|
|
2
2
|
|
|
3
|
-
my-dev-kit-lab is the experiment, evidence, reporting, security-validation, and
|
|
4
|
-
|
|
5
|
-
The current
|
|
3
|
+
my-dev-kit-lab is the experiment, evidence, reporting, security-validation, and audit companion for my-dev-kit. It runs reproducible experiments that test whether my-dev-kit's graph-guided retrieval helps coding-agent workflows, collects metrics, renders reports, generates plots, captures screenshots, builds gallery outputs, performs automated CLI/package security validation, and runs the generic audit framework.
|
|
4
|
+
|
|
5
|
+
The current published baseline is `v0.3.0`. `v0.3.0` introduced the generic audit framework and the first implemented code-rot detector family. The generic experiment-plugin framework was introduced in `v0.2.0`. The first plugin is `context-strategy-comparison`, which preserves the existing raw-full-file vs my-dev-kit-guided workflow through the plugin runner.
|
|
6
|
+
|
|
7
|
+
The checked-out package is release-prepared for `v0.3.1` on top of the published baseline: normalized language/file-role inventory, source-facts collection, a TypeScript/JavaScript analyzer, source-facts-aware code-rot signals, and source-facts summaries in text/JSON audit reports. `v0.3.1` is not published yet; Python, Java, Kotlin, Android validation, framework-aware profiles, and broader audit types remain future roadmap work.
|
|
6
8
|
|
|
7
9
|
my-dev-kit is most useful when the repository is larger than the task. It helps coding agents work with large codebases through reusable structural indexing, graph-guided retrieval, targeted source slices, and auditable context selection. Results are scoped evidence; the project does not claim that my-dev-kit always saves tokens.
|
|
8
10
|
|
|
@@ -32,7 +34,7 @@ my-dev-kit is most useful when the repository is larger than the task. It helps
|
|
|
32
34
|
- Target-aware experiment execution for local projects via `experiment:run -- --target <path>`
|
|
33
35
|
- Plugin-aware JSON and HTML reports with plugin, target, variant, metric, artifact, warning, skip, and failure metadata
|
|
34
36
|
- Security validation framework: dependency audit, package tarball inspection, CLI adversarial tests, static scans (CodeQL/Semgrep), bounded fuzz smoke, attack-scenario checks, and structured verdict/report output — runnable against any local project via `security:validate --target <path>`
|
|
35
|
-
- Generic audit framework
|
|
37
|
+
- Generic audit framework in the current published `v0.3.0` baseline: `npm run audit` runs heuristic, conservative code-rot detectors and writes text/JSON reports under `reports/audits/code-rot/` by default. Only the `code-rot` audit type is currently implemented; `quality`, `security`, `project`, and `all` audit types are planned and fail cleanly instead of running. `npm run audit` supports `--target`, `--types code-rot`, and other flags — see [docs/COMMANDS.md](docs/COMMANDS.md). Audit does not modify target files, does not auto-fix issues, does not run `security:validate`, and does not perform release-readiness determination.
|
|
36
38
|
|
|
37
39
|
---
|
|
38
40
|
|
|
@@ -201,9 +203,16 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
|
|
|
201
203
|
|
|
202
204
|
---
|
|
203
205
|
|
|
204
|
-
## Current baseline release positioning
|
|
205
|
-
|
|
206
|
-
my-dev-kit-lab is at a working baseline. The raw-vs-indexed experiment pipeline is fully implemented and produces reproducible artifacts. Real-agent campaign support exists for Codex and Claude. The
|
|
206
|
+
## Current baseline release positioning
|
|
207
|
+
|
|
208
|
+
my-dev-kit-lab is at a working baseline. The raw-vs-indexed experiment pipeline is fully implemented and produces reproducible artifacts. Real-agent campaign support exists for Codex and Claude. The current published npm baseline is `v0.3.0`, which includes the generic audit framework and the first implemented code-rot detector family on top of the earlier experiment and automated security-validation work.
|
|
209
|
+
|
|
210
|
+
Planned roadmap direction after the published baseline:
|
|
211
|
+
|
|
212
|
+
- `v0.3.1`: language-aware code-rot substrate plus TypeScript/JavaScript support, release-prepared and not yet published
|
|
213
|
+
- `v0.3.2` through `v0.3.4`: Python, Java/Kotlin, and cross-language stability for the language-aware code-rot track
|
|
214
|
+
- `v0.4.0` through `v0.4.2`: Android automated security validation
|
|
215
|
+
- manual pentest: post-v1 / version TBD
|
|
207
216
|
|
|
208
217
|
---
|
|
209
218
|
|
|
@@ -241,7 +250,7 @@ Each security command can validate my-dev-kit-lab itself or another local projec
|
|
|
241
250
|
|
|
242
251
|
`security:validate` supports `--checks`, `--profile`, `--format`, `--fail-on`, `--out`, and `--report-prefix`. The no-flag path remains backward compatible and still runs the classic implemented check groups: `deps`, `package`, `static`, `cli-adversarial`, and `fuzz`. Supplying `--profile` without `--checks` swaps in that profile's default checks; supplying explicit `--checks` always wins over profile defaults.
|
|
243
252
|
|
|
244
|
-
The current attack-scenario checks cover boundary, subprocess, secrets, and network assumptions. They are automated adversarial checks, not a manual pentest. `verdictImpact` metadata from each registered scenario drives blocker categorization in report reasoning, and narrowed `--checks` runs are labeled as scoped rather than described as a full release gate.
|
|
253
|
+
The current attack-scenario checks cover boundary, subprocess, secrets, and network assumptions. They are automated adversarial checks, not a manual pentest. Manual pentest is deferred until after `v1.0.0`. `verdictImpact` metadata from each registered scenario drives blocker categorization in report reasoning, and narrowed `--checks` runs are labeled as scoped rather than described as a full release gate.
|
|
245
254
|
|
|
246
255
|
JSON report poisoning/config-injection checks use a baseline-diff schema guard so legitimate additive JSON fields do not fail the guard while payload-created trusted top-level fields still do. Text reports are sanitized to strip ANSI/control-byte payloads before rendering.
|
|
247
256
|
|
|
@@ -86,6 +86,7 @@ if (model.summary.noDetectorsRegistered) {
|
|
|
86
86
|
console.log(`\nInventory: ${model.inventory.totalScannedFileCount} file(s) scanned, ${model.inventory.skippedFileCount} skipped`);
|
|
87
87
|
console.log(` source=${model.inventory.filesByCategory.source} tests=${model.inventory.filesByCategory.tests} docs=${model.inventory.filesByCategory.docs} package=${model.inventory.filesByCategory.package} config=${model.inventory.filesByCategory.config} scripts=${model.inventory.filesByCategory.scripts} ci=${model.inventory.filesByCategory.ci}`);
|
|
88
88
|
console.log(`Source of truth: package=${model.sourceOfTruth.packageName ?? "(none)"}@${model.sourceOfTruth.packageVersion ?? "?"} readme=${model.sourceOfTruth.hasReadme} changelog=${model.sourceOfTruth.hasChangelog} ciWorkflows=${model.sourceOfTruth.ciWorkflowCount}`);
|
|
89
|
+
console.log(`Source facts: ${model.sourceFacts.totalFilesAnalyzed} file(s) analyzed (parsed=${model.sourceFacts.filesByParseStatus.parsed} file-level-only=${model.sourceFacts.filesByParseStatus["file-level-only"]} unsupported=${model.sourceFacts.filesByParseStatus.unsupported})`);
|
|
89
90
|
if (writtenPaths.length > 0) {
|
|
90
91
|
console.log(`\nReports written:`);
|
|
91
92
|
for (const p of writtenPaths) {
|
|
@@ -2,6 +2,7 @@ import path from "node:path";
|
|
|
2
2
|
import { readBoundedFileText } from "../utils/boundedRead.js";
|
|
3
3
|
import { baseNameNoExt } from "../utils/filePatternUtils.js";
|
|
4
4
|
import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
|
|
5
|
+
import { collectRelativeImportBasenames, getParsedSourceFacts, indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
|
|
5
6
|
// ---------------------------------------------------------------------------
|
|
6
7
|
// v0.3.0 Batch 4 — dead-code candidate detector.
|
|
7
8
|
//
|
|
@@ -140,7 +141,13 @@ function findUnreferencedScriptFiles(ctx, pkg) {
|
|
|
140
141
|
return issues;
|
|
141
142
|
}
|
|
142
143
|
function buildReverseReferenceIndex(ctx) {
|
|
143
|
-
|
|
144
|
+
// v0.3.1 Batch 4 -- seeded with basenames the TypeScript/JavaScript
|
|
145
|
+
// analyzer already resolved from real import declarations (parsed files
|
|
146
|
+
// only). This only ever adds references, never removes them, so it can
|
|
147
|
+
// only reduce false positives relative to the regex-only scan below --
|
|
148
|
+
// safe even though ctx.sourceFacts is optional (returns an empty set when
|
|
149
|
+
// absent, e.g. in existing detector-only unit tests).
|
|
150
|
+
const referencedBasenames = collectRelativeImportBasenames(ctx.sourceFacts);
|
|
144
151
|
const allCandidateFiles = [
|
|
145
152
|
...ctx.inventory.sourceFiles,
|
|
146
153
|
...ctx.inventory.testFiles,
|
|
@@ -164,6 +171,7 @@ function buildReverseReferenceIndex(ctx) {
|
|
|
164
171
|
function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
|
|
165
172
|
const referencedBasenames = buildReverseReferenceIndex(ctx);
|
|
166
173
|
const binCandidates = pkg ? binTargetsAsSourceCandidates(pkg) : new Set();
|
|
174
|
+
const sourceFactsIndex = indexSourceFactsByPath(ctx.sourceFacts);
|
|
167
175
|
const issues = [];
|
|
168
176
|
for (const file of ctx.inventory.sourceFiles) {
|
|
169
177
|
if (file.sizeBytes > MAX_READ_BYTES)
|
|
@@ -187,6 +195,26 @@ function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
|
|
|
187
195
|
continue;
|
|
188
196
|
if (referencedBasenames.has(base.toLowerCase()))
|
|
189
197
|
continue;
|
|
198
|
+
const evidence = [
|
|
199
|
+
{
|
|
200
|
+
kind: "observation",
|
|
201
|
+
message: "No relative import/require specifier in the scanned source/test/script set resolves to this file's basename.",
|
|
202
|
+
filePath: file.relativePath,
|
|
203
|
+
source: DETECTOR_ID,
|
|
204
|
+
confidence: "low",
|
|
205
|
+
},
|
|
206
|
+
];
|
|
207
|
+
// v0.3.1 Batch 4 -- when the TypeScript/JavaScript analyzer actually
|
|
208
|
+
// parsed this file, surface its export count as extra candidate
|
|
209
|
+
// evidence (an exported symbol with zero detected external references
|
|
210
|
+
// is more suspicious than an unexported one). This never changes
|
|
211
|
+
// whether the issue is emitted, only its evidence -- the flagging
|
|
212
|
+
// decision above is unchanged from the existing basename-reference
|
|
213
|
+
// heuristic.
|
|
214
|
+
const facts = getParsedSourceFacts(sourceFactsIndex, file.relativePath);
|
|
215
|
+
if (facts) {
|
|
216
|
+
evidence.push(buildSourceFactsEvidence(facts));
|
|
217
|
+
}
|
|
190
218
|
issues.push(makeCodeRotIssue({
|
|
191
219
|
auditType: "code-rot",
|
|
192
220
|
detectorId: DETECTOR_ID,
|
|
@@ -203,20 +231,25 @@ function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
|
|
|
203
231
|
releaseBlocking: false,
|
|
204
232
|
implementationBlocking: false,
|
|
205
233
|
autoFixEligible: false,
|
|
206
|
-
evidence
|
|
207
|
-
{
|
|
208
|
-
kind: "observation",
|
|
209
|
-
message: "No relative import/require specifier in the scanned source/test/script set resolves to this file's basename.",
|
|
210
|
-
filePath: file.relativePath,
|
|
211
|
-
source: DETECTOR_ID,
|
|
212
|
-
confidence: "low",
|
|
213
|
-
},
|
|
214
|
-
],
|
|
234
|
+
evidence,
|
|
215
235
|
affectedFiles: [file.relativePath],
|
|
216
236
|
}));
|
|
217
237
|
}
|
|
218
238
|
return issues;
|
|
219
239
|
}
|
|
240
|
+
// v0.3.1 Batch 4 -- describes a parsed file's source-facts shape as an
|
|
241
|
+
// evidence entry. Only called for parseStatus "parsed" facts (see callers),
|
|
242
|
+
// since file-level-only/parse-error facts carry empty fact arrays that
|
|
243
|
+
// would misleadingly read as "zero exports" rather than "unknown".
|
|
244
|
+
function buildSourceFactsEvidence(facts) {
|
|
245
|
+
return {
|
|
246
|
+
kind: "reference",
|
|
247
|
+
message: `Source facts: the TypeScript/JavaScript analyzer parsed this file and recorded ${facts.exports.length} export(s), ${facts.declarations.length} declaration(s), and ${facts.imports.length} import(s).`,
|
|
248
|
+
filePath: facts.relativePath,
|
|
249
|
+
source: DETECTOR_ID,
|
|
250
|
+
confidence: "medium",
|
|
251
|
+
};
|
|
252
|
+
}
|
|
220
253
|
function findOldOrDeprecatedFilesNotInDocs(ctx, docsConcat) {
|
|
221
254
|
const issues = [];
|
|
222
255
|
for (const file of ctx.inventory.files) {
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
import { readBoundedFileText } from "../utils/boundedRead.js";
|
|
2
|
-
import { firstPathSegment, GENERIC_INFRA_BASENAMES, baseNameNoExt } from "../utils/filePatternUtils.js";
|
|
2
|
+
import { firstPathSegment, GENERIC_DECLARATION_NAMES, GENERIC_INFRA_BASENAMES, baseNameNoExt } from "../utils/filePatternUtils.js";
|
|
3
3
|
import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
|
|
4
|
+
import { indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
|
|
4
5
|
// ---------------------------------------------------------------------------
|
|
5
6
|
// v0.3.0 Batch 4 — duplicate/parallel implementation candidate detector.
|
|
6
7
|
//
|
|
@@ -60,6 +61,7 @@ export const DUPLICATE_IMPLEMENTATION_DETECTOR = {
|
|
|
60
61
|
if (ctx.config.include.includes("architecture")) {
|
|
61
62
|
issues.push(...findParallelDirectories(ctx.inventory.files.map((f) => f.relativePath)));
|
|
62
63
|
issues.push(...findDuplicateBasenamesAcrossRoots(ctx.inventory.sourceFiles.map((f) => f.relativePath)));
|
|
64
|
+
issues.push(...findDuplicateDeclarationCandidates(ctx));
|
|
63
65
|
}
|
|
64
66
|
issues.push(...findDuplicateDetectorRegistryIds(ctx));
|
|
65
67
|
return deduplicateIssuesById(issues);
|
|
@@ -214,6 +216,81 @@ function firstTopLevelUnderSrc(relativePath) {
|
|
|
214
216
|
return null;
|
|
215
217
|
return `${parts[0]}/${parts[1]}`;
|
|
216
218
|
}
|
|
219
|
+
// v0.3.1 Batch 4 -- source-facts-aware duplicate-declaration candidate
|
|
220
|
+
// check. Weak/low-confidence signal, same spirit as
|
|
221
|
+
// findDuplicateBasenamesAcrossRoots(): the same exported declaration name
|
|
222
|
+
// and kind (e.g. two exported `class Logger`) appears in 2-4 distinct
|
|
223
|
+
// "source"-role files that a registered analyzer actually parsed. Bounded
|
|
224
|
+
// to declaration kinds that carry real implementation weight
|
|
225
|
+
// (function/class/interface/type/enum) -- "variable"/"constant"/"method"/
|
|
226
|
+
// "unknown" are excluded as too noisy (methods repeat across unrelated
|
|
227
|
+
// classes by convention; module-scope variables/constants repeat even more
|
|
228
|
+
// often). Generic declaration names (run, init, handler, ...) are excluded
|
|
229
|
+
// via GENERIC_DECLARATION_NAMES for the same reason
|
|
230
|
+
// GENERIC_INFRA_BASENAMES excludes generic file basenames above. Requires
|
|
231
|
+
// ctx.sourceFacts -- returns no issues when it is absent (e.g. existing
|
|
232
|
+
// detector unit tests that build a literal AuditDetectorContext without
|
|
233
|
+
// it), so this check is purely additive.
|
|
234
|
+
const NOISY_DECLARATION_KINDS = new Set(["variable", "constant", "method", "unknown"]);
|
|
235
|
+
function findDuplicateDeclarationCandidates(ctx) {
|
|
236
|
+
if (!ctx.sourceFacts)
|
|
237
|
+
return [];
|
|
238
|
+
const index = indexSourceFactsByPath(ctx.sourceFacts);
|
|
239
|
+
const byKey = new Map();
|
|
240
|
+
for (const facts of index.values()) {
|
|
241
|
+
if (facts.role !== "source")
|
|
242
|
+
continue;
|
|
243
|
+
if (facts.parseStatus !== "parsed")
|
|
244
|
+
continue;
|
|
245
|
+
for (const decl of facts.declarations) {
|
|
246
|
+
if (!decl.exported)
|
|
247
|
+
continue;
|
|
248
|
+
if (NOISY_DECLARATION_KINDS.has(decl.kind))
|
|
249
|
+
continue;
|
|
250
|
+
const nameLower = decl.name.toLowerCase();
|
|
251
|
+
if (GENERIC_DECLARATION_NAMES.has(nameLower))
|
|
252
|
+
continue;
|
|
253
|
+
const key = `${decl.kind}:${nameLower}`;
|
|
254
|
+
const entry = byKey.get(key) ?? { name: decl.name, kind: decl.kind, paths: new Set() };
|
|
255
|
+
entry.paths.add(facts.relativePath);
|
|
256
|
+
byKey.set(key, entry);
|
|
257
|
+
}
|
|
258
|
+
}
|
|
259
|
+
const issues = [];
|
|
260
|
+
for (const entry of byKey.values()) {
|
|
261
|
+
if (entry.paths.size < 2 || entry.paths.size > 4)
|
|
262
|
+
continue;
|
|
263
|
+
const sortedPaths = [...entry.paths].sort((a, b) => a.localeCompare(b));
|
|
264
|
+
issues.push(makeCodeRotIssue({
|
|
265
|
+
auditType: "code-rot",
|
|
266
|
+
detectorId: DETECTOR_ID,
|
|
267
|
+
idCues: ["duplicate-declaration-candidate", entry.kind, entry.name.toLowerCase(), ...sortedPaths],
|
|
268
|
+
title: `Exported ${entry.kind} "${entry.name}" is declared in ${sortedPaths.length} unrelated files (weak signal)`,
|
|
269
|
+
description: `An exported ${entry.kind} named "${entry.name}" is declared in: ${sortedPaths.join(", ")}. This is a source-facts-derived candidate signal (same declaration name and kind, not a semantic-equivalence check) for parallel implementations -- inspect manually before consolidating.`,
|
|
270
|
+
severity: "info",
|
|
271
|
+
confidence: "low",
|
|
272
|
+
falsePositiveRisk: "high",
|
|
273
|
+
category: "duplicate-implementation-candidate",
|
|
274
|
+
recommendedAction: "Manually confirm whether these declarations serve genuinely different purposes before taking any action.",
|
|
275
|
+
suggestedFixStrategy: "No automatic action -- rename or consolidate only after manual review.",
|
|
276
|
+
validationCommands: ["npm run audit -- --types code-rot --include architecture"],
|
|
277
|
+
releaseBlocking: false,
|
|
278
|
+
implementationBlocking: false,
|
|
279
|
+
autoFixEligible: false,
|
|
280
|
+
evidence: [
|
|
281
|
+
{
|
|
282
|
+
kind: "reference",
|
|
283
|
+
message: `Source facts: an exported ${entry.kind} named "${entry.name}" was parsed in ${sortedPaths.length} distinct files.`,
|
|
284
|
+
excerpt: sortedPaths.join(", "),
|
|
285
|
+
source: DETECTOR_ID,
|
|
286
|
+
confidence: "low",
|
|
287
|
+
},
|
|
288
|
+
],
|
|
289
|
+
affectedFiles: sortedPaths,
|
|
290
|
+
}));
|
|
291
|
+
}
|
|
292
|
+
return issues;
|
|
293
|
+
}
|
|
217
294
|
// Cheap regex scan (not a semantic parse) of already-small detector source
|
|
218
295
|
// files for exported `id: "..."` string literals. Flags duplicates only if
|
|
219
296
|
// genuinely found via this file-based check -- per the spec's own allowed
|
|
@@ -2,6 +2,7 @@ import fs from "node:fs";
|
|
|
2
2
|
import path from "node:path";
|
|
3
3
|
import { readBoundedFileText } from "../utils/boundedRead.js";
|
|
4
4
|
import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
|
|
5
|
+
import { getParsedSourceFacts, indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
|
|
5
6
|
import { splitLines } from "../utils/textLines.js";
|
|
6
7
|
// ---------------------------------------------------------------------------
|
|
7
8
|
// v0.3.0 Batch 4 — test-rot detector.
|
|
@@ -42,6 +43,7 @@ export const TEST_ROT_DETECTOR = {
|
|
|
42
43
|
const issues = [];
|
|
43
44
|
const scriptNames = new Set(ctx.sourceOfTruth.commands.allScriptNames);
|
|
44
45
|
const currentVersion = ctx.sourceOfTruth.package?.version ?? null;
|
|
46
|
+
const sourceFactsIndex = indexSourceFactsByPath(ctx.sourceFacts);
|
|
45
47
|
for (const testFile of ctx.inventory.testFiles) {
|
|
46
48
|
const content = readBoundedFileText(ctx.target.rootPath, testFile.relativePath, testFile.sizeBytes, MAX_READ_BYTES);
|
|
47
49
|
if (content === null)
|
|
@@ -61,6 +63,17 @@ export const TEST_ROT_DETECTOR = {
|
|
|
61
63
|
if (testFile.relativePath.startsWith("tests/audits/codeRot/"))
|
|
62
64
|
continue;
|
|
63
65
|
issues.push(...findMissingSourceImports(ctx, testFile.relativePath, content));
|
|
66
|
+
// v0.3.1 Batch 4 -- when the TypeScript/JavaScript analyzer actually
|
|
67
|
+
// parsed this test file, also check its structured import facts.
|
|
68
|
+
// This catches import forms the regex above misses (e.g. multi-line
|
|
69
|
+
// import statements) without duplicate-reporting a specifier both
|
|
70
|
+
// checks already found: idCues use the same shape
|
|
71
|
+
// (testRelativePath, "missing-source-import", specifier), so an
|
|
72
|
+
// identical id is deduplicated by deduplicateIssuesById() below.
|
|
73
|
+
const parsedFacts = getParsedSourceFacts(sourceFactsIndex, testFile.relativePath);
|
|
74
|
+
if (parsedFacts) {
|
|
75
|
+
issues.push(...findMissingSourceImportsFromSourceFacts(ctx, testFile.relativePath, parsedFacts));
|
|
76
|
+
}
|
|
64
77
|
issues.push(...findStaleNpmRunReferences(testFile.relativePath, content, scriptNames));
|
|
65
78
|
issues.push(...findSkipUsage(testFile.relativePath, content));
|
|
66
79
|
issues.push(...findOnlyUsage(testFile.relativePath, content));
|
|
@@ -115,6 +128,55 @@ function findMissingSourceImports(ctx, testRelativePath, content) {
|
|
|
115
128
|
}
|
|
116
129
|
return issues;
|
|
117
130
|
}
|
|
131
|
+
// v0.3.1 Batch 4 -- source-facts-derived counterpart to
|
|
132
|
+
// findMissingSourceImports() above. Uses the TypeScript/JavaScript
|
|
133
|
+
// analyzer's already-parsed ImportFact list instead of a regex scan of raw
|
|
134
|
+
// content, so it also catches relative import specifiers the regex above
|
|
135
|
+
// misses. Only called for facts with parseStatus "parsed" (see caller) --
|
|
136
|
+
// file-level-only/parse-error facts carry an empty imports array that
|
|
137
|
+
// would otherwise silently look like "no imports" rather than "unknown".
|
|
138
|
+
function findMissingSourceImportsFromSourceFacts(ctx, testRelativePath, facts) {
|
|
139
|
+
const issues = [];
|
|
140
|
+
const testDir = path.dirname(testRelativePath);
|
|
141
|
+
for (const imp of facts.imports) {
|
|
142
|
+
if (!imp.source.startsWith("."))
|
|
143
|
+
continue;
|
|
144
|
+
const resolvedBase = path.join(testDir, imp.source).replace(/\\/g, "/");
|
|
145
|
+
const strippedBase = resolvedBase.replace(/\.[cm]?[jt]sx?$/, "");
|
|
146
|
+
const resolved = RESOLVE_EXTENSIONS.some((ext) => fs.existsSync(path.join(ctx.target.rootPath, `${strippedBase}${ext}`)));
|
|
147
|
+
if (resolved)
|
|
148
|
+
continue;
|
|
149
|
+
issues.push(makeCodeRotIssue({
|
|
150
|
+
auditType: "code-rot",
|
|
151
|
+
detectorId: DETECTOR_ID,
|
|
152
|
+
idCues: [testRelativePath, "missing-source-import", imp.source],
|
|
153
|
+
title: `Test "${testRelativePath}" imports a missing source file: "${imp.source}"`,
|
|
154
|
+
description: `${testRelativePath} imports "${imp.source}" (source-facts-derived via the TypeScript/JavaScript analyzer), which does not resolve to an existing file on disk (checked common TS/JS extensions relative to the test file).`,
|
|
155
|
+
severity: "medium",
|
|
156
|
+
confidence: "high",
|
|
157
|
+
falsePositiveRisk: "low",
|
|
158
|
+
category: "test-rot",
|
|
159
|
+
recommendedAction: "Fix or remove the stale import.",
|
|
160
|
+
suggestedFixStrategy: `Update the import specifier "${imp.source}" in ${testRelativePath} to point to an existing file.`,
|
|
161
|
+
validationCommands: [`npx vitest run ${testRelativePath}`],
|
|
162
|
+
releaseBlocking: false,
|
|
163
|
+
implementationBlocking: false,
|
|
164
|
+
autoFixEligible: false,
|
|
165
|
+
evidence: [
|
|
166
|
+
{
|
|
167
|
+
kind: "file",
|
|
168
|
+
message: "Analyzer-recorded relative import specifier does not resolve to an existing file.",
|
|
169
|
+
filePath: testRelativePath,
|
|
170
|
+
excerpt: imp.source,
|
|
171
|
+
source: DETECTOR_ID,
|
|
172
|
+
confidence: "high",
|
|
173
|
+
},
|
|
174
|
+
],
|
|
175
|
+
affectedFiles: [testRelativePath],
|
|
176
|
+
}));
|
|
177
|
+
}
|
|
178
|
+
return issues;
|
|
179
|
+
}
|
|
118
180
|
function findStaleNpmRunReferences(testRelativePath, content, scriptNames) {
|
|
119
181
|
const issues = [];
|
|
120
182
|
const seen = new Set();
|
|
@@ -45,3 +45,35 @@ export const GENERIC_INFRA_BASENAMES = new Set([
|
|
|
45
45
|
"errors",
|
|
46
46
|
"error",
|
|
47
47
|
]);
|
|
48
|
+
// v0.3.1 Batch 4 -- common, generic exported-declaration names that are
|
|
49
|
+
// expected to repeat across unrelated files legitimately (CLI entrypoints,
|
|
50
|
+
// handler/lifecycle conventions, etc.) -- excluded from the source-facts
|
|
51
|
+
// duplicate-declaration-candidate heuristic in duplicateImplementationDetector.ts
|
|
52
|
+
// to keep false positives near zero.
|
|
53
|
+
export const GENERIC_DECLARATION_NAMES = new Set([
|
|
54
|
+
"run",
|
|
55
|
+
"main",
|
|
56
|
+
"init",
|
|
57
|
+
"initialize",
|
|
58
|
+
"setup",
|
|
59
|
+
"handler",
|
|
60
|
+
"handle",
|
|
61
|
+
"execute",
|
|
62
|
+
"process",
|
|
63
|
+
"validate",
|
|
64
|
+
"parse",
|
|
65
|
+
"format",
|
|
66
|
+
"render",
|
|
67
|
+
"build",
|
|
68
|
+
"create",
|
|
69
|
+
"start",
|
|
70
|
+
"stop",
|
|
71
|
+
"load",
|
|
72
|
+
"save",
|
|
73
|
+
"update",
|
|
74
|
+
"remove",
|
|
75
|
+
"apply",
|
|
76
|
+
"get",
|
|
77
|
+
"set",
|
|
78
|
+
"default",
|
|
79
|
+
]);
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
import { baseNameNoExt } from "./filePatternUtils.js";
|
|
2
|
+
// ---------------------------------------------------------------------------
|
|
3
|
+
// v0.3.1 Batch 4 -- shared source-facts lookup helpers for code-rot
|
|
4
|
+
// detectors.
|
|
5
|
+
//
|
|
6
|
+
// ctx.sourceFacts is optional (see auditRegistry.ts) precisely so existing
|
|
7
|
+
// detector tests that build a literal AuditDetectorContext without it keep
|
|
8
|
+
// working unchanged -- every helper here accepts `undefined` and degrades to
|
|
9
|
+
// an empty result rather than throwing, so a caller never needs its own
|
|
10
|
+
// null-check before using them.
|
|
11
|
+
// ---------------------------------------------------------------------------
|
|
12
|
+
// Indexes a SourceFactsSnapshot's files by relativePath for O(1) per-file
|
|
13
|
+
// lookups. Returns an empty map when sourceFacts is undefined.
|
|
14
|
+
export function indexSourceFactsByPath(sourceFacts) {
|
|
15
|
+
const index = new Map();
|
|
16
|
+
if (!sourceFacts)
|
|
17
|
+
return index;
|
|
18
|
+
for (const file of sourceFacts.files)
|
|
19
|
+
index.set(file.relativePath, file);
|
|
20
|
+
return index;
|
|
21
|
+
}
|
|
22
|
+
// Returns the facts for relativePath only when parseStatus is "parsed" --
|
|
23
|
+
// callers that want analyzer-derived evidence (imports/exports/declarations)
|
|
24
|
+
// should use this rather than reading the index directly, since
|
|
25
|
+
// file-level-only/parse-error/skipped/unsupported entries carry empty fact
|
|
26
|
+
// arrays that would silently look like "no imports" rather than "unknown".
|
|
27
|
+
export function getParsedSourceFacts(index, relativePath) {
|
|
28
|
+
const facts = index.get(relativePath);
|
|
29
|
+
return facts && facts.parseStatus === "parsed" ? facts : null;
|
|
30
|
+
}
|
|
31
|
+
// Basenames referenced by any parsed file's relative import/re-export
|
|
32
|
+
// specifier, lowercased. Mirrors deadCodeCandidateDetector.ts's regex-based
|
|
33
|
+
// buildReverseReferenceIndex() basename matching, but sourced from
|
|
34
|
+
// structured analyzer facts -- so it also catches import forms the regex
|
|
35
|
+
// misses (e.g. multi-line import statements) for files a registered
|
|
36
|
+
// analyzer actually parsed. Returns an empty set when sourceFacts is
|
|
37
|
+
// undefined, so merging it into an existing Set is always safe.
|
|
38
|
+
export function collectRelativeImportBasenames(sourceFacts) {
|
|
39
|
+
const basenames = new Set();
|
|
40
|
+
if (!sourceFacts)
|
|
41
|
+
return basenames;
|
|
42
|
+
for (const file of sourceFacts.files) {
|
|
43
|
+
if (file.parseStatus !== "parsed")
|
|
44
|
+
continue;
|
|
45
|
+
for (const imp of file.imports) {
|
|
46
|
+
if (!imp.source.startsWith("."))
|
|
47
|
+
continue;
|
|
48
|
+
const base = baseNameNoExt(imp.source.split("?")[0]);
|
|
49
|
+
basenames.add(base.toLowerCase());
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
return basenames;
|
|
53
|
+
}
|
|
@@ -3,14 +3,16 @@ import { DEFAULT_AUDIT_REGISTRY, selectDetectors } from "./auditRegistry.js";
|
|
|
3
3
|
import { resolveAuditTarget } from "./auditTarget.js";
|
|
4
4
|
import { scanProjectInventory } from "./projectInventory.js";
|
|
5
5
|
import { collectSourceOfTruth } from "./sourceOfTruth.js";
|
|
6
|
+
import { collectSourceFacts } from "./collectSourceFacts.js";
|
|
6
7
|
export async function runAudit(options) {
|
|
7
8
|
const { config, toolRoot } = options;
|
|
8
9
|
const registry = options.registry ?? DEFAULT_AUDIT_REGISTRY;
|
|
9
10
|
const target = options.target ?? resolveAuditTarget(config.targetPathArg, toolRoot);
|
|
10
11
|
const inventory = scanProjectInventory(target.rootPath);
|
|
11
12
|
const sourceOfTruth = collectSourceOfTruth(target.rootPath, inventory);
|
|
13
|
+
const sourceFacts = await collectSourceFacts(target.rootPath, inventory);
|
|
12
14
|
const selected = selectDetectors(registry, config.types, config.include);
|
|
13
|
-
const ctx = { target, config, inventory, sourceOfTruth };
|
|
15
|
+
const ctx = { target, config, inventory, sourceOfTruth, sourceFacts };
|
|
14
16
|
const issues = [];
|
|
15
17
|
const skippedDetectors = [];
|
|
16
18
|
const detectorErrors = [];
|
|
@@ -59,6 +61,7 @@ export async function runAudit(options) {
|
|
|
59
61
|
exitReason: reason,
|
|
60
62
|
inventory,
|
|
61
63
|
sourceOfTruth,
|
|
64
|
+
sourceFacts,
|
|
62
65
|
noDetectorsRegistered: selected.length === 0,
|
|
63
66
|
};
|
|
64
67
|
}
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
import fs from "node:fs";
|
|
2
|
+
import path from "node:path";
|
|
3
|
+
import { NORMALIZED_LANGUAGES } from "./projectInventory.js";
|
|
4
|
+
import { DEFAULT_LANGUAGE_ANALYZER_REGISTRY, selectLanguageAnalyzer, } from "./languageAnalyzerRegistry.js";
|
|
5
|
+
import { SOURCE_FACT_PARSE_STATUSES, } from "./sourceFacts.js";
|
|
6
|
+
// ---------------------------------------------------------------------------
|
|
7
|
+
// v0.3.1 Batch 2 -- source facts collector.
|
|
8
|
+
//
|
|
9
|
+
// Consumes the Batch 1 ProjectInventorySnapshot (never re-walks the
|
|
10
|
+
// filesystem) and produces a SourceFactsSnapshot: analyzer facts where a
|
|
11
|
+
// registered analyzer exists, deterministic fallback facts otherwise. Same
|
|
12
|
+
// spirit as sourceOfTruth.ts -- pure data collection, never an AuditIssue,
|
|
13
|
+
// never a write to a target file.
|
|
14
|
+
//
|
|
15
|
+
// Eligibility / omission policy (documented here since it's the single
|
|
16
|
+
// source of truth for the decision):
|
|
17
|
+
// - Only inventory entries with role "source" or "test" are considered.
|
|
18
|
+
// docs/config/package/generated/vendor/build-output/report-output/
|
|
19
|
+
// unknown-role files are omitted entirely from `files` -- source facts
|
|
20
|
+
// are meaningless for them, and omitting (rather than emitting a
|
|
21
|
+
// "skipped" placeholder for every doc/config file in the repo) keeps
|
|
22
|
+
// the snapshot proportional to actual candidate source files.
|
|
23
|
+
// - For an eligible file in a *known* language (typescript, javascript,
|
|
24
|
+
// python, java, kotlin) with no registered analyzer: parseStatus
|
|
25
|
+
// "file-level-only" -- we know the language and already have
|
|
26
|
+
// inventory-level evidence (lineCount), just no analyzer parsed it yet.
|
|
27
|
+
// - For an eligible file in an *unknown* language: parseStatus
|
|
28
|
+
// "unsupported" -- no file-level evidence attempted either.
|
|
29
|
+
// - File content is only read when a registered analyzer actually claims
|
|
30
|
+
// the file -- with an empty default registry (Batch 2), no file is ever
|
|
31
|
+
// read by this collector.
|
|
32
|
+
// ---------------------------------------------------------------------------
|
|
33
|
+
const ANALYZABLE_ROLES = new Set(["source", "test"]);
|
|
34
|
+
const KNOWN_LANGUAGES = new Set(["typescript", "javascript", "python", "java", "kotlin"]);
|
|
35
|
+
export async function collectSourceFacts(targetRoot, inventory, registry = DEFAULT_LANGUAGE_ANALYZER_REGISTRY) {
|
|
36
|
+
const resolvedRoot = path.resolve(targetRoot);
|
|
37
|
+
const files = [];
|
|
38
|
+
const analyzerDiagnostics = [];
|
|
39
|
+
const warnings = [];
|
|
40
|
+
// inventory.files is already sorted by relativePath (Batch 1 guarantee) --
|
|
41
|
+
// filtering preserves that order, so no re-sort is needed here.
|
|
42
|
+
const eligible = inventory.files.filter((entry) => ANALYZABLE_ROLES.has(entry.role));
|
|
43
|
+
for (const entry of eligible) {
|
|
44
|
+
const analyzer = selectLanguageAnalyzer(registry, entry.language, entry.extension);
|
|
45
|
+
if (!analyzer) {
|
|
46
|
+
files.push(buildFallbackFacts(entry));
|
|
47
|
+
continue;
|
|
48
|
+
}
|
|
49
|
+
const absolutePath = path.join(resolvedRoot, entry.relativePath);
|
|
50
|
+
try {
|
|
51
|
+
const content = fs.readFileSync(absolutePath, "utf8");
|
|
52
|
+
const facts = await analyzer.analyzeFile({
|
|
53
|
+
targetRoot: resolvedRoot,
|
|
54
|
+
relativePath: entry.relativePath,
|
|
55
|
+
absolutePath,
|
|
56
|
+
content,
|
|
57
|
+
inventoryEntry: entry,
|
|
58
|
+
language: entry.language,
|
|
59
|
+
role: entry.role,
|
|
60
|
+
});
|
|
61
|
+
files.push(facts);
|
|
62
|
+
}
|
|
63
|
+
catch (err) {
|
|
64
|
+
const message = err instanceof Error ? err.message : String(err);
|
|
65
|
+
const diagnostic = {
|
|
66
|
+
severity: "error",
|
|
67
|
+
code: "analyzer-error",
|
|
68
|
+
message: `Analyzer "${analyzer.id}" failed on "${entry.relativePath}": ${message}`,
|
|
69
|
+
path: entry.relativePath,
|
|
70
|
+
analyzerId: analyzer.id,
|
|
71
|
+
};
|
|
72
|
+
analyzerDiagnostics.push(diagnostic);
|
|
73
|
+
files.push({
|
|
74
|
+
relativePath: entry.relativePath,
|
|
75
|
+
language: entry.language,
|
|
76
|
+
role: entry.role,
|
|
77
|
+
parseStatus: "parse-error",
|
|
78
|
+
analyzerId: analyzer.id,
|
|
79
|
+
lineCount: entry.lineCount,
|
|
80
|
+
imports: [],
|
|
81
|
+
exports: [],
|
|
82
|
+
declarations: [],
|
|
83
|
+
references: [],
|
|
84
|
+
diagnostics: [diagnostic],
|
|
85
|
+
});
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
return {
|
|
89
|
+
targetRoot: resolvedRoot,
|
|
90
|
+
files,
|
|
91
|
+
filesByLanguage: countByLanguage(files),
|
|
92
|
+
filesByParseStatus: countByParseStatus(files),
|
|
93
|
+
analyzerDiagnostics,
|
|
94
|
+
warnings,
|
|
95
|
+
};
|
|
96
|
+
}
|
|
97
|
+
function buildFallbackFacts(entry) {
|
|
98
|
+
const known = KNOWN_LANGUAGES.has(entry.language);
|
|
99
|
+
const parseStatus = known ? "file-level-only" : "unsupported";
|
|
100
|
+
const diagnostic = known
|
|
101
|
+
? {
|
|
102
|
+
severity: "info",
|
|
103
|
+
code: "no-analyzer-registered",
|
|
104
|
+
message: `No language analyzer is registered for "${entry.language}" yet; only inventory-level facts are available.`,
|
|
105
|
+
path: entry.relativePath,
|
|
106
|
+
analyzerId: null,
|
|
107
|
+
}
|
|
108
|
+
: {
|
|
109
|
+
severity: "info",
|
|
110
|
+
code: "unsupported-language",
|
|
111
|
+
message: `Language "${entry.language}" is not supported for source-fact extraction.`,
|
|
112
|
+
path: entry.relativePath,
|
|
113
|
+
analyzerId: null,
|
|
114
|
+
};
|
|
115
|
+
return {
|
|
116
|
+
relativePath: entry.relativePath,
|
|
117
|
+
language: entry.language,
|
|
118
|
+
role: entry.role,
|
|
119
|
+
parseStatus,
|
|
120
|
+
analyzerId: null,
|
|
121
|
+
lineCount: entry.lineCount,
|
|
122
|
+
imports: [],
|
|
123
|
+
exports: [],
|
|
124
|
+
declarations: [],
|
|
125
|
+
references: [],
|
|
126
|
+
diagnostics: [diagnostic],
|
|
127
|
+
};
|
|
128
|
+
}
|
|
129
|
+
function countByLanguage(files) {
|
|
130
|
+
const counts = Object.fromEntries(NORMALIZED_LANGUAGES.map((l) => [l, 0]));
|
|
131
|
+
for (const file of files) {
|
|
132
|
+
counts[file.language] += 1;
|
|
133
|
+
}
|
|
134
|
+
return counts;
|
|
135
|
+
}
|
|
136
|
+
function countByParseStatus(files) {
|
|
137
|
+
const counts = Object.fromEntries(SOURCE_FACT_PARSE_STATUSES.map((s) => [s, 0]));
|
|
138
|
+
for (const file of files) {
|
|
139
|
+
counts[file.parseStatus] += 1;
|
|
140
|
+
}
|
|
141
|
+
return counts;
|
|
142
|
+
}
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import { TYPESCRIPT_JAVASCRIPT_ANALYZER } from "./typescriptJavaScriptAnalyzer.js";
|
|
2
|
+
// Validates a list has no duplicate analyzer ids. Later batches call this
|
|
3
|
+
// when assembling their own registries; Batch 2's empty default trivially
|
|
4
|
+
// passes.
|
|
5
|
+
export function createLanguageAnalyzerRegistry(analyzers) {
|
|
6
|
+
const seen = new Set();
|
|
7
|
+
for (const analyzer of analyzers) {
|
|
8
|
+
if (seen.has(analyzer.id)) {
|
|
9
|
+
throw new Error(`Duplicate language analyzer id: "${analyzer.id}".`);
|
|
10
|
+
}
|
|
11
|
+
seen.add(analyzer.id);
|
|
12
|
+
}
|
|
13
|
+
return analyzers;
|
|
14
|
+
}
|
|
15
|
+
// First matching analyzer in registry order (deterministic), or null when
|
|
16
|
+
// none supports the given language/extension. Order is preserved from the
|
|
17
|
+
// input registry -- not re-sorted.
|
|
18
|
+
export function selectLanguageAnalyzer(registry, language, extension) {
|
|
19
|
+
for (const analyzer of registry) {
|
|
20
|
+
if (!analyzer.supportedLanguages.includes(language))
|
|
21
|
+
continue;
|
|
22
|
+
if (analyzer.supportedExtensions && !analyzer.supportedExtensions.includes(extension))
|
|
23
|
+
continue;
|
|
24
|
+
return analyzer;
|
|
25
|
+
}
|
|
26
|
+
return null;
|
|
27
|
+
}
|
|
28
|
+
// v0.3.1 Batch 3 -- registers the TypeScript/JavaScript analyzer. Python,
|
|
29
|
+
// Java, and Kotlin remain fallback-only (no analyzer registered for them) --
|
|
30
|
+
// see collectSourceFacts.ts's fallback policy.
|
|
31
|
+
export const DEFAULT_LANGUAGE_ANALYZER_REGISTRY = createLanguageAnalyzerRegistry([
|
|
32
|
+
TYPESCRIPT_JAVASCRIPT_ANALYZER,
|
|
33
|
+
]);
|
|
@@ -27,6 +27,112 @@ export const INVENTORY_FILE_CATEGORIES = [
|
|
|
27
27
|
"report",
|
|
28
28
|
"unknown",
|
|
29
29
|
];
|
|
30
|
+
// ---------------------------------------------------------------------------
|
|
31
|
+
// v0.3.1 Batch 1 — normalized language and file-role metadata.
|
|
32
|
+
//
|
|
33
|
+
// Purely additive to the v0.3.0 category model above: `category` keeps its
|
|
34
|
+
// existing meaning and existing consumers (detectors, report model) are
|
|
35
|
+
// untouched. `language` and `role` are deterministic, extension/path-based
|
|
36
|
+
// classifications layered on top so later v0.3.1 batches (language analyzer
|
|
37
|
+
// registry, per-language code-rot detectors) can select files by language
|
|
38
|
+
// without re-deriving it from extensions themselves. Language detection is
|
|
39
|
+
// classification-only here -- it does not imply parsing/analysis support for
|
|
40
|
+
// python/java/kotlin exists yet.
|
|
41
|
+
// ---------------------------------------------------------------------------
|
|
42
|
+
export const NORMALIZED_LANGUAGES = [
|
|
43
|
+
"typescript",
|
|
44
|
+
"javascript",
|
|
45
|
+
"python",
|
|
46
|
+
"java",
|
|
47
|
+
"kotlin",
|
|
48
|
+
"json",
|
|
49
|
+
"markdown",
|
|
50
|
+
"yaml",
|
|
51
|
+
"xml",
|
|
52
|
+
"toml",
|
|
53
|
+
"unknown",
|
|
54
|
+
];
|
|
55
|
+
const EXTENSION_TO_LANGUAGE = {
|
|
56
|
+
".ts": "typescript",
|
|
57
|
+
".tsx": "typescript",
|
|
58
|
+
".mts": "typescript",
|
|
59
|
+
".cts": "typescript",
|
|
60
|
+
".js": "javascript",
|
|
61
|
+
".jsx": "javascript",
|
|
62
|
+
".mjs": "javascript",
|
|
63
|
+
".cjs": "javascript",
|
|
64
|
+
".py": "python",
|
|
65
|
+
".java": "java",
|
|
66
|
+
".kt": "kotlin",
|
|
67
|
+
".kts": "kotlin",
|
|
68
|
+
".json": "json",
|
|
69
|
+
".md": "markdown",
|
|
70
|
+
".markdown": "markdown",
|
|
71
|
+
".yml": "yaml",
|
|
72
|
+
".yaml": "yaml",
|
|
73
|
+
".xml": "xml",
|
|
74
|
+
".toml": "toml",
|
|
75
|
+
};
|
|
76
|
+
function classifyLanguage(extension) {
|
|
77
|
+
return EXTENSION_TO_LANGUAGE[extension] ?? "unknown";
|
|
78
|
+
}
|
|
79
|
+
export const FILE_ROLES = [
|
|
80
|
+
"source",
|
|
81
|
+
"test",
|
|
82
|
+
"docs",
|
|
83
|
+
"config",
|
|
84
|
+
"package",
|
|
85
|
+
"generated",
|
|
86
|
+
"build-output",
|
|
87
|
+
"vendor",
|
|
88
|
+
"report-output",
|
|
89
|
+
"unknown",
|
|
90
|
+
];
|
|
91
|
+
// Path-segment markers used only when a file wasn't already excluded from
|
|
92
|
+
// traversal entirely (EXCLUDED_DIR_NAMES below) -- e.g. paths reachable from
|
|
93
|
+
// nested benchmark/fixture projects, or directories not yet in the hard
|
|
94
|
+
// exclusion set. Deliberately conservative: matched as whole path segments,
|
|
95
|
+
// not substrings.
|
|
96
|
+
const VENDOR_PATH_SEGMENTS = new Set(["node_modules", "vendor"]);
|
|
97
|
+
const BUILD_OUTPUT_PATH_SEGMENTS = new Set(["dist", "build", "out", ".next", ".turbo", "target", ".gradle"]);
|
|
98
|
+
const GENERATED_PATH_SEGMENTS = new Set([
|
|
99
|
+
"generated",
|
|
100
|
+
".cache",
|
|
101
|
+
".my-dev-kit",
|
|
102
|
+
".my-dev-kit-v1",
|
|
103
|
+
".my-dev-kit-lab",
|
|
104
|
+
]);
|
|
105
|
+
const REPORT_OUTPUT_PATH_SEGMENTS = new Set(["reports", "lab-output"]);
|
|
106
|
+
function classifyFileRole(relativePath, basename, extension, category) {
|
|
107
|
+
const segments = relativePath.split("/");
|
|
108
|
+
if (category === "report" ||
|
|
109
|
+
isStrayReportFile(basename) ||
|
|
110
|
+
segments.some((s) => REPORT_OUTPUT_PATH_SEGMENTS.has(s))) {
|
|
111
|
+
return "report-output";
|
|
112
|
+
}
|
|
113
|
+
if (segments.some((s) => VENDOR_PATH_SEGMENTS.has(s)))
|
|
114
|
+
return "vendor";
|
|
115
|
+
if (extension === ".tsbuildinfo" || segments.some((s) => BUILD_OUTPUT_PATH_SEGMENTS.has(s)))
|
|
116
|
+
return "build-output";
|
|
117
|
+
if (extension === ".map" || segments.some((s) => GENERATED_PATH_SEGMENTS.has(s)))
|
|
118
|
+
return "generated";
|
|
119
|
+
switch (category) {
|
|
120
|
+
case "tests":
|
|
121
|
+
return "test";
|
|
122
|
+
case "docs":
|
|
123
|
+
return "docs";
|
|
124
|
+
case "package":
|
|
125
|
+
return "package";
|
|
126
|
+
case "config":
|
|
127
|
+
case "ci":
|
|
128
|
+
return "config";
|
|
129
|
+
case "source":
|
|
130
|
+
case "scripts":
|
|
131
|
+
return "source";
|
|
132
|
+
default:
|
|
133
|
+
return "unknown";
|
|
134
|
+
}
|
|
135
|
+
}
|
|
30
136
|
// Directory names excluded from traversal entirely -- never descended into.
|
|
31
137
|
const EXCLUDED_DIR_NAMES = new Set([
|
|
32
138
|
"node_modules",
|
|
@@ -96,6 +202,24 @@ const KNOWN_SOURCE_EXTENSIONS = new Set([
|
|
|
96
202
|
".go",
|
|
97
203
|
".rs",
|
|
98
204
|
".java",
|
|
205
|
+
// v0.3.1 Batch 2 fix: .kt/.kts were classified as a NormalizedLanguage
|
|
206
|
+
// ("kotlin") since Batch 1 but were missing here, so a Kotlin file under
|
|
207
|
+
// src/ got category "unknown" (role "unknown") instead of "source" --
|
|
208
|
+
// silently excluding it from source-facts collection. .go/.rs above are
|
|
209
|
+
// intentionally left as "source" despite having no NormalizedLanguage
|
|
210
|
+
// mapping yet (they normalize to language "unknown"), which is the
|
|
211
|
+
// legitimate "known role, unknown language" case collectSourceFacts.ts's
|
|
212
|
+
// fallback policy handles.
|
|
213
|
+
".kt",
|
|
214
|
+
".kts",
|
|
215
|
+
// v0.3.1 Batch 3 fix: same class of gap as the .kt/.kts fix above -- .mts
|
|
216
|
+
// and .cts were mapped to NormalizedLanguage "typescript" since Batch 1
|
|
217
|
+
// but were missing here, so a .mts/.cts file under src/ got category
|
|
218
|
+
// "unknown" (role "unknown") and would never reach the Batch 3
|
|
219
|
+
// TypeScript/JavaScript analyzer's eligibility filter (role source/test
|
|
220
|
+
// only, see collectSourceFacts.ts).
|
|
221
|
+
".mts",
|
|
222
|
+
".cts",
|
|
99
223
|
]);
|
|
100
224
|
// A file this large or larger is skipped entirely (not even stat'd into the
|
|
101
225
|
// inventory) -- protects against pathological huge-file scan cost.
|
|
@@ -123,6 +247,8 @@ export function scanProjectInventory(targetRoot, options = {}) {
|
|
|
123
247
|
}
|
|
124
248
|
const filesByCategory = countByCategory(files);
|
|
125
249
|
const filesByExtension = countByExtension(files);
|
|
250
|
+
const filesByLanguage = countByLanguage(files);
|
|
251
|
+
const filesByRole = countByRole(files);
|
|
126
252
|
const excludedDirectorySummary = [...excludedDirCounts.entries()]
|
|
127
253
|
.map(([name, occurrences]) => ({ name, occurrences }))
|
|
128
254
|
.sort((a, b) => a.name.localeCompare(b.name));
|
|
@@ -134,6 +260,8 @@ export function scanProjectInventory(targetRoot, options = {}) {
|
|
|
134
260
|
skippedFileCount,
|
|
135
261
|
filesByCategory,
|
|
136
262
|
filesByExtension,
|
|
263
|
+
filesByLanguage,
|
|
264
|
+
filesByRole,
|
|
137
265
|
packageFiles: files.filter((f) => f.category === "package"),
|
|
138
266
|
docsFiles: files.filter((f) => f.category === "docs"),
|
|
139
267
|
sourceFiles: files.filter((f) => f.category === "source"),
|
|
@@ -192,11 +320,19 @@ export function scanProjectInventory(targetRoot, options = {}) {
|
|
|
192
320
|
const relativePath = relativeWithinRoot(resolvedRoot, fullPath);
|
|
193
321
|
const extension = path.extname(entry.name).toLowerCase();
|
|
194
322
|
const category = classifyFile(relativePath, entry.name, extension);
|
|
323
|
+
const language = classifyLanguage(extension);
|
|
324
|
+
const role = classifyFileRole(relativePath, entry.name, extension, category);
|
|
195
325
|
const entryResult = {
|
|
196
326
|
relativePath,
|
|
197
327
|
normalizedPath: relativePath.toLowerCase(),
|
|
198
328
|
extension,
|
|
199
329
|
category,
|
|
330
|
+
language,
|
|
331
|
+
role,
|
|
332
|
+
isGenerated: role === "generated",
|
|
333
|
+
isBuildOutput: role === "build-output",
|
|
334
|
+
isVendor: role === "vendor",
|
|
335
|
+
isReportOutput: role === "report-output",
|
|
200
336
|
sizeBytes: stat.size,
|
|
201
337
|
likelyGenerated: category === "generated" || category === "report",
|
|
202
338
|
likelyConfig: category === "config" || category === "package",
|
|
@@ -297,6 +433,20 @@ function countByCategory(files) {
|
|
|
297
433
|
}
|
|
298
434
|
return counts;
|
|
299
435
|
}
|
|
436
|
+
function countByLanguage(files) {
|
|
437
|
+
const counts = Object.fromEntries(NORMALIZED_LANGUAGES.map((l) => [l, 0]));
|
|
438
|
+
for (const file of files) {
|
|
439
|
+
counts[file.language] += 1;
|
|
440
|
+
}
|
|
441
|
+
return counts;
|
|
442
|
+
}
|
|
443
|
+
function countByRole(files) {
|
|
444
|
+
const counts = Object.fromEntries(FILE_ROLES.map((r) => [r, 0]));
|
|
445
|
+
for (const file of files) {
|
|
446
|
+
counts[file.role] += 1;
|
|
447
|
+
}
|
|
448
|
+
return counts;
|
|
449
|
+
}
|
|
300
450
|
function countByExtension(files) {
|
|
301
451
|
const counts = new Map();
|
|
302
452
|
for (const file of files) {
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
// ---------------------------------------------------------------------------
|
|
2
|
+
// v0.3.1 Batch 2 -- normalized source facts model.
|
|
3
|
+
//
|
|
4
|
+
// Purely additive substrate: no concrete language analyzer exists yet (that
|
|
5
|
+
// is Batch 3's TypeScript/JavaScript analyzer), so every SourceFileFacts
|
|
6
|
+
// produced in this batch is either a deterministic fallback (see
|
|
7
|
+
// collectSourceFacts.ts) or a parse-error record. The fact arrays
|
|
8
|
+
// (imports/exports/declarations/references) are always empty in this batch
|
|
9
|
+
// -- the shape exists so Batch 3 can populate them without a type change.
|
|
10
|
+
// ---------------------------------------------------------------------------
|
|
11
|
+
export const SOURCE_FACT_PARSE_STATUSES = [
|
|
12
|
+
"parsed",
|
|
13
|
+
"file-level-only",
|
|
14
|
+
"unsupported",
|
|
15
|
+
"parse-error",
|
|
16
|
+
"skipped",
|
|
17
|
+
];
|
|
@@ -0,0 +1,299 @@
|
|
|
1
|
+
import ts from "typescript";
|
|
2
|
+
// ---------------------------------------------------------------------------
|
|
3
|
+
// v0.3.1 Batch 3 -- TypeScript/JavaScript source analyzer.
|
|
4
|
+
//
|
|
5
|
+
// Single-file, syntax-only analysis via the TypeScript compiler API:
|
|
6
|
+
// - ts.createSourceFile() builds a per-file AST (no Program, no project
|
|
7
|
+
// resolution, no type checking) that is walked once to extract facts.
|
|
8
|
+
// - ts.transpileModule({ reportDiagnostics: true }) is used purely to
|
|
9
|
+
// obtain syntactic diagnostics -- it is itself a single-file operation
|
|
10
|
+
// (no cross-file resolution), matching the "single-file parsing only"
|
|
11
|
+
// requirement for this batch.
|
|
12
|
+
//
|
|
13
|
+
// Parse-status policy (deliberately conservative, see docs on
|
|
14
|
+
// SourceFactParseStatus): zero syntax diagnostics -> "parsed". Any syntax
|
|
15
|
+
// diagnostic -> "parse-error", even though the AST walk below still runs
|
|
16
|
+
// against TypeScript's best-effort (error-tolerant) parse and whatever facts
|
|
17
|
+
// it manages to extract are still attached -- the parseStatus signals
|
|
18
|
+
// "don't treat these facts as fully reliable", not "no facts exist". This
|
|
19
|
+
// mirrors how collectSourceFacts.ts already reports a caught analyzer
|
|
20
|
+
// exception as parse-error with an empty-fact fallback -- this analyzer just
|
|
21
|
+
// reaches parse-error through diagnostics instead of a thrown exception, and
|
|
22
|
+
// still gets to keep best-effort facts (which a thrown-exception parse-error
|
|
23
|
+
// never has, since collectSourceFacts.ts only has the exception itself).
|
|
24
|
+
// ---------------------------------------------------------------------------
|
|
25
|
+
export const TYPESCRIPT_JAVASCRIPT_ANALYZER_ID = "typescript-javascript-analyzer";
|
|
26
|
+
const SUPPORTED_EXTENSIONS = [".ts", ".tsx", ".mts", ".cts", ".js", ".jsx", ".mjs", ".cjs"];
|
|
27
|
+
// AST construction is meaningfully more expensive than a line count -- large
|
|
28
|
+
// files (generated bundles, vendored single-file libraries that slipped past
|
|
29
|
+
// role/path exclusions, etc.) are not parsed. Mirrors the size-bounding
|
|
30
|
+
// philosophy of projectInventory.ts's own read limits, just at a tighter
|
|
31
|
+
// threshold appropriate for actually building an AST.
|
|
32
|
+
const MAX_ANALYZABLE_BYTES = 1_000_000;
|
|
33
|
+
export const TYPESCRIPT_JAVASCRIPT_ANALYZER = {
|
|
34
|
+
id: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
|
|
35
|
+
supportedLanguages: ["typescript", "javascript"],
|
|
36
|
+
supportedExtensions: SUPPORTED_EXTENSIONS,
|
|
37
|
+
analyzeFile,
|
|
38
|
+
};
|
|
39
|
+
function analyzeFile(input) {
|
|
40
|
+
const { relativePath, language, role, content, inventoryEntry } = input;
|
|
41
|
+
if (Buffer.byteLength(content, "utf8") > MAX_ANALYZABLE_BYTES) {
|
|
42
|
+
return {
|
|
43
|
+
relativePath,
|
|
44
|
+
language,
|
|
45
|
+
role,
|
|
46
|
+
parseStatus: "file-level-only",
|
|
47
|
+
analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
|
|
48
|
+
lineCount: inventoryEntry.lineCount,
|
|
49
|
+
imports: [],
|
|
50
|
+
exports: [],
|
|
51
|
+
declarations: [],
|
|
52
|
+
references: [],
|
|
53
|
+
diagnostics: [
|
|
54
|
+
{
|
|
55
|
+
severity: "info",
|
|
56
|
+
code: "file-too-large-for-analysis",
|
|
57
|
+
message: `File exceeds the ${MAX_ANALYZABLE_BYTES}-byte analyzer bound; only inventory-level facts are available.`,
|
|
58
|
+
path: relativePath,
|
|
59
|
+
analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
|
|
60
|
+
},
|
|
61
|
+
],
|
|
62
|
+
};
|
|
63
|
+
}
|
|
64
|
+
const scriptKind = resolveScriptKind(inventoryEntry.extension);
|
|
65
|
+
const sourceFile = ts.createSourceFile(relativePath, content, ts.ScriptTarget.Latest, true, scriptKind);
|
|
66
|
+
const diagnostics = getSyntaxDiagnostics(relativePath, sourceFile);
|
|
67
|
+
const imports = [];
|
|
68
|
+
const exports = [];
|
|
69
|
+
const declarations = [];
|
|
70
|
+
const references = [];
|
|
71
|
+
const lineOf = (node) => sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile)).line + 1;
|
|
72
|
+
function pushDeclaration(node, name, kind) {
|
|
73
|
+
const isExported = hasModifier(node, ts.SyntaxKind.ExportKeyword);
|
|
74
|
+
const isDefault = hasModifier(node, ts.SyntaxKind.DefaultKeyword);
|
|
75
|
+
if (name) {
|
|
76
|
+
declarations.push({ name, kind, exported: isExported, line: lineOf(node) });
|
|
77
|
+
}
|
|
78
|
+
if (isDefault) {
|
|
79
|
+
exports.push({ name: name ?? "default", kind: "default", line: lineOf(node) });
|
|
80
|
+
}
|
|
81
|
+
else if (isExported && name) {
|
|
82
|
+
exports.push({ name, kind: "named", line: lineOf(node) });
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
function handleImportDeclaration(node) {
|
|
86
|
+
if (!ts.isStringLiteralLike(node.moduleSpecifier))
|
|
87
|
+
return;
|
|
88
|
+
const source = node.moduleSpecifier.text;
|
|
89
|
+
const importedNames = [];
|
|
90
|
+
const clause = node.importClause;
|
|
91
|
+
if (clause) {
|
|
92
|
+
if (clause.name)
|
|
93
|
+
importedNames.push(clause.name.text);
|
|
94
|
+
const bindings = clause.namedBindings;
|
|
95
|
+
if (bindings && ts.isNamespaceImport(bindings)) {
|
|
96
|
+
importedNames.push(bindings.name.text);
|
|
97
|
+
}
|
|
98
|
+
else if (bindings && ts.isNamedImports(bindings)) {
|
|
99
|
+
for (const element of bindings.elements)
|
|
100
|
+
importedNames.push(element.name.text);
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
imports.push({
|
|
104
|
+
source,
|
|
105
|
+
kind: "static",
|
|
106
|
+
line: lineOf(node),
|
|
107
|
+
...(importedNames.length > 0 ? { importedNames } : {}),
|
|
108
|
+
});
|
|
109
|
+
}
|
|
110
|
+
function handleExportDeclaration(node) {
|
|
111
|
+
const source = node.moduleSpecifier && ts.isStringLiteralLike(node.moduleSpecifier) ? node.moduleSpecifier.text : undefined;
|
|
112
|
+
if (source) {
|
|
113
|
+
imports.push({ source, kind: "re-export", line: lineOf(node) });
|
|
114
|
+
}
|
|
115
|
+
if (!node.exportClause) {
|
|
116
|
+
if (source)
|
|
117
|
+
exports.push({ name: "*", kind: "namespace", line: lineOf(node) });
|
|
118
|
+
return;
|
|
119
|
+
}
|
|
120
|
+
if (ts.isNamespaceExport(node.exportClause)) {
|
|
121
|
+
exports.push({ name: node.exportClause.name.text, kind: "namespace", line: lineOf(node) });
|
|
122
|
+
return;
|
|
123
|
+
}
|
|
124
|
+
for (const element of node.exportClause.elements) {
|
|
125
|
+
exports.push({ name: element.name.text, kind: "named", line: lineOf(node) });
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
function handleExportAssignment(node) {
|
|
129
|
+
if (node.isExportEquals) {
|
|
130
|
+
exports.push({ name: "module.exports", kind: "commonjs", line: lineOf(node) });
|
|
131
|
+
return;
|
|
132
|
+
}
|
|
133
|
+
const expr = node.expression;
|
|
134
|
+
let name = "default";
|
|
135
|
+
if (ts.isIdentifier(expr)) {
|
|
136
|
+
name = expr.text;
|
|
137
|
+
}
|
|
138
|
+
else if ((ts.isFunctionExpression(expr) || ts.isClassExpression(expr)) && expr.name) {
|
|
139
|
+
name = expr.name.text;
|
|
140
|
+
}
|
|
141
|
+
exports.push({ name, kind: "default", line: lineOf(node) });
|
|
142
|
+
}
|
|
143
|
+
function handleVariableStatement(node) {
|
|
144
|
+
const isExported = hasModifier(node, ts.SyntaxKind.ExportKeyword);
|
|
145
|
+
const isConst = (node.declarationList.flags & ts.NodeFlags.Const) !== 0;
|
|
146
|
+
for (const decl of node.declarationList.declarations) {
|
|
147
|
+
// Destructuring patterns (`const { a, b } = x`) are conservatively
|
|
148
|
+
// skipped -- only simple identifier bindings are recorded.
|
|
149
|
+
if (!ts.isIdentifier(decl.name))
|
|
150
|
+
continue;
|
|
151
|
+
declarations.push({
|
|
152
|
+
name: decl.name.text,
|
|
153
|
+
kind: isConst ? "constant" : "variable",
|
|
154
|
+
exported: isExported,
|
|
155
|
+
line: lineOf(decl),
|
|
156
|
+
});
|
|
157
|
+
if (isExported) {
|
|
158
|
+
exports.push({ name: decl.name.text, kind: "named", line: lineOf(decl) });
|
|
159
|
+
}
|
|
160
|
+
}
|
|
161
|
+
}
|
|
162
|
+
function handleCommonJsAssignment(node) {
|
|
163
|
+
const left = node.left;
|
|
164
|
+
if (!ts.isPropertyAccessExpression(left) || !ts.isIdentifier(left.expression))
|
|
165
|
+
return;
|
|
166
|
+
if (left.expression.text === "module" && left.name.text === "exports") {
|
|
167
|
+
exports.push({ name: "module.exports", kind: "commonjs", line: lineOf(node) });
|
|
168
|
+
}
|
|
169
|
+
else if (left.expression.text === "exports") {
|
|
170
|
+
exports.push({ name: left.name.text, kind: "commonjs", line: lineOf(node) });
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
function handleCallExpression(node) {
|
|
174
|
+
if (ts.isIdentifier(node.expression) && node.expression.text === "require") {
|
|
175
|
+
const arg = node.arguments[0];
|
|
176
|
+
if (arg && ts.isStringLiteralLike(arg)) {
|
|
177
|
+
imports.push({ source: arg.text, kind: "commonjs", line: lineOf(node) });
|
|
178
|
+
}
|
|
179
|
+
return; // already represented as an import fact -- do not also reference "require".
|
|
180
|
+
}
|
|
181
|
+
if (node.expression.kind === ts.SyntaxKind.ImportKeyword) {
|
|
182
|
+
const arg = node.arguments[0];
|
|
183
|
+
if (arg && ts.isStringLiteralLike(arg)) {
|
|
184
|
+
imports.push({ source: arg.text, kind: "dynamic", line: lineOf(node) });
|
|
185
|
+
}
|
|
186
|
+
return;
|
|
187
|
+
}
|
|
188
|
+
if (ts.isIdentifier(node.expression)) {
|
|
189
|
+
// Conservative reference extraction: only bare-identifier call
|
|
190
|
+
// expressions (`foo()`), not property-access calls (`a.b()`) or any
|
|
191
|
+
// other identifier usage -- no scope analysis, no unused-detection.
|
|
192
|
+
references.push({ name: node.expression.text, kind: "call", line: lineOf(node) });
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
function visit(node) {
|
|
196
|
+
if (ts.isImportDeclaration(node)) {
|
|
197
|
+
handleImportDeclaration(node);
|
|
198
|
+
}
|
|
199
|
+
else if (ts.isExportDeclaration(node)) {
|
|
200
|
+
handleExportDeclaration(node);
|
|
201
|
+
}
|
|
202
|
+
else if (ts.isExportAssignment(node)) {
|
|
203
|
+
handleExportAssignment(node);
|
|
204
|
+
}
|
|
205
|
+
else if (ts.isFunctionDeclaration(node)) {
|
|
206
|
+
pushDeclaration(node, node.name?.text, "function");
|
|
207
|
+
}
|
|
208
|
+
else if (ts.isClassDeclaration(node)) {
|
|
209
|
+
pushDeclaration(node, node.name?.text, "class");
|
|
210
|
+
}
|
|
211
|
+
else if (ts.isInterfaceDeclaration(node)) {
|
|
212
|
+
pushDeclaration(node, node.name.text, "interface");
|
|
213
|
+
}
|
|
214
|
+
else if (ts.isTypeAliasDeclaration(node)) {
|
|
215
|
+
pushDeclaration(node, node.name.text, "type");
|
|
216
|
+
}
|
|
217
|
+
else if (ts.isEnumDeclaration(node)) {
|
|
218
|
+
pushDeclaration(node, node.name.text, "enum");
|
|
219
|
+
}
|
|
220
|
+
else if (ts.isVariableStatement(node)) {
|
|
221
|
+
handleVariableStatement(node);
|
|
222
|
+
}
|
|
223
|
+
else if (ts.isMethodDeclaration(node) && ts.isIdentifier(node.name)) {
|
|
224
|
+
declarations.push({ name: node.name.text, kind: "method", exported: false, line: lineOf(node) });
|
|
225
|
+
}
|
|
226
|
+
else if (ts.isCallExpression(node)) {
|
|
227
|
+
handleCallExpression(node);
|
|
228
|
+
}
|
|
229
|
+
else if (ts.isBinaryExpression(node) && node.operatorToken.kind === ts.SyntaxKind.EqualsToken) {
|
|
230
|
+
handleCommonJsAssignment(node);
|
|
231
|
+
}
|
|
232
|
+
ts.forEachChild(node, visit);
|
|
233
|
+
}
|
|
234
|
+
visit(sourceFile);
|
|
235
|
+
const parseStatus = diagnostics.length > 0 ? "parse-error" : "parsed";
|
|
236
|
+
return {
|
|
237
|
+
relativePath,
|
|
238
|
+
language,
|
|
239
|
+
role,
|
|
240
|
+
parseStatus,
|
|
241
|
+
analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
|
|
242
|
+
lineCount: inventoryEntry.lineCount,
|
|
243
|
+
imports,
|
|
244
|
+
exports,
|
|
245
|
+
declarations,
|
|
246
|
+
references,
|
|
247
|
+
diagnostics,
|
|
248
|
+
};
|
|
249
|
+
}
|
|
250
|
+
function resolveScriptKind(extension) {
|
|
251
|
+
switch (extension) {
|
|
252
|
+
case ".tsx":
|
|
253
|
+
return ts.ScriptKind.TSX;
|
|
254
|
+
case ".jsx":
|
|
255
|
+
return ts.ScriptKind.JSX;
|
|
256
|
+
case ".js":
|
|
257
|
+
case ".mjs":
|
|
258
|
+
case ".cjs":
|
|
259
|
+
return ts.ScriptKind.JS;
|
|
260
|
+
case ".ts":
|
|
261
|
+
case ".mts":
|
|
262
|
+
case ".cts":
|
|
263
|
+
default:
|
|
264
|
+
return ts.ScriptKind.TS;
|
|
265
|
+
}
|
|
266
|
+
}
|
|
267
|
+
function hasModifier(node, kind) {
|
|
268
|
+
if (!ts.canHaveModifiers(node))
|
|
269
|
+
return false;
|
|
270
|
+
const modifiers = ts.getModifiers(node);
|
|
271
|
+
return modifiers?.some((m) => m.kind === kind) ?? false;
|
|
272
|
+
}
|
|
273
|
+
// Reads syntax diagnostics directly off the already-built SourceFile rather
|
|
274
|
+
// than running a second full parse via ts.transpileModule(). `parseDiagnostics`
|
|
275
|
+
// is not part of the public TypeScript.d.ts surface, but it is populated by
|
|
276
|
+
// createSourceFile() itself (every syntax error the parser recovers from is
|
|
277
|
+
// recorded there) and has been stable across TypeScript versions for years --
|
|
278
|
+
// this is a widely-used pattern for exactly this "syntax diagnostics without
|
|
279
|
+
// a second parse/transpile pass" use case. Benchmarked at ~5x faster than
|
|
280
|
+
// transpileModule() over this repo's own ~375 TS/JS files, which matters
|
|
281
|
+
// because collectSourceFacts.ts calls this analyzer once per eligible file
|
|
282
|
+
// on every audit run (including this repo's own self-audit).
|
|
283
|
+
function getSyntaxDiagnostics(relativePath, sourceFile) {
|
|
284
|
+
const parseDiagnostics = sourceFile.parseDiagnostics ?? [];
|
|
285
|
+
return parseDiagnostics.map((diagnostic) => toSourceFactDiagnostic(relativePath, diagnostic));
|
|
286
|
+
}
|
|
287
|
+
function toSourceFactDiagnostic(relativePath, diagnostic) {
|
|
288
|
+
const message = ts.flattenDiagnosticMessageText(diagnostic.messageText, "\n");
|
|
289
|
+
const line = diagnostic.file && diagnostic.start !== undefined
|
|
290
|
+
? diagnostic.file.getLineAndCharacterOfPosition(diagnostic.start).line + 1
|
|
291
|
+
: undefined;
|
|
292
|
+
return {
|
|
293
|
+
severity: diagnostic.category === ts.DiagnosticCategory.Warning ? "warning" : "error",
|
|
294
|
+
code: `TS${diagnostic.code}`,
|
|
295
|
+
message: line !== undefined ? `Line ${line}: ${message}` : message,
|
|
296
|
+
path: relativePath,
|
|
297
|
+
analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
|
|
298
|
+
};
|
|
299
|
+
}
|
|
@@ -22,6 +22,8 @@ export function summarizeInventory(inventory) {
|
|
|
22
22
|
skippedFileCount: inventory.skippedFileCount,
|
|
23
23
|
filesByCategory: inventory.filesByCategory,
|
|
24
24
|
filesByExtension: inventory.filesByExtension,
|
|
25
|
+
filesByLanguage: inventory.filesByLanguage,
|
|
26
|
+
filesByRole: inventory.filesByRole,
|
|
25
27
|
excludedDirectorySummary: inventory.excludedDirectorySummary,
|
|
26
28
|
warnings: inventory.warnings,
|
|
27
29
|
};
|
|
@@ -47,6 +49,15 @@ export function summarizeSourceOfTruth(sourceOfTruth) {
|
|
|
47
49
|
warnings: sourceOfTruth.warnings,
|
|
48
50
|
};
|
|
49
51
|
}
|
|
52
|
+
export function summarizeSourceFacts(sourceFacts) {
|
|
53
|
+
return {
|
|
54
|
+
totalFilesAnalyzed: sourceFacts.files.length,
|
|
55
|
+
filesByLanguage: sourceFacts.filesByLanguage,
|
|
56
|
+
filesByParseStatus: sourceFacts.filesByParseStatus,
|
|
57
|
+
analyzerDiagnosticCount: sourceFacts.analyzerDiagnostics.length,
|
|
58
|
+
warnings: sourceFacts.warnings,
|
|
59
|
+
};
|
|
60
|
+
}
|
|
50
61
|
const MAX_JSON_RECOMMENDATIONS = 200;
|
|
51
62
|
const MAX_TEXT_RECOMMENDATIONS = 10;
|
|
52
63
|
const SEVERITY_ORDER = ["blocker", "high", "medium", "low", "info"];
|
|
@@ -176,6 +187,7 @@ export function buildAuditReportModel(result, opts) {
|
|
|
176
187
|
},
|
|
177
188
|
inventory: summarizeInventory(result.inventory),
|
|
178
189
|
sourceOfTruth: summarizeSourceOfTruth(result.sourceOfTruth),
|
|
190
|
+
sourceFacts: summarizeSourceFacts(result.sourceFacts),
|
|
179
191
|
detectors,
|
|
180
192
|
issues,
|
|
181
193
|
skippedDetectors: result.skippedDetectors,
|
|
@@ -18,6 +18,7 @@ export function renderAuditJsonReport(model) {
|
|
|
18
18
|
summary: model.summary,
|
|
19
19
|
inventory: model.inventory,
|
|
20
20
|
sourceOfTruth: model.sourceOfTruth,
|
|
21
|
+
sourceFacts: model.sourceFacts,
|
|
21
22
|
detectors: model.detectors,
|
|
22
23
|
issues: model.issues,
|
|
23
24
|
skippedDetectors: model.skippedDetectors,
|
|
@@ -70,10 +70,20 @@ export function renderAuditTextReport(model) {
|
|
|
70
70
|
lines.push(divider("-"));
|
|
71
71
|
lines.push(` total=${model.inventory.totalFileCount} scanned=${model.inventory.totalScannedFileCount} skipped=${model.inventory.skippedFileCount}`);
|
|
72
72
|
lines.push(` source=${model.inventory.filesByCategory.source} tests=${model.inventory.filesByCategory.tests} docs=${model.inventory.filesByCategory.docs} package=${model.inventory.filesByCategory.package} config=${model.inventory.filesByCategory.config} scripts=${model.inventory.filesByCategory.scripts} ci=${model.inventory.filesByCategory.ci} generated=${model.inventory.filesByCategory.generated} report=${model.inventory.filesByCategory.report} unknown=${model.inventory.filesByCategory.unknown}`);
|
|
73
|
+
lines.push(` languages: typescript=${model.inventory.filesByLanguage.typescript} javascript=${model.inventory.filesByLanguage.javascript} python=${model.inventory.filesByLanguage.python} java=${model.inventory.filesByLanguage.java} kotlin=${model.inventory.filesByLanguage.kotlin} json=${model.inventory.filesByLanguage.json} markdown=${model.inventory.filesByLanguage.markdown} yaml=${model.inventory.filesByLanguage.yaml} xml=${model.inventory.filesByLanguage.xml} toml=${model.inventory.filesByLanguage.toml} unknown=${model.inventory.filesByLanguage.unknown}`);
|
|
74
|
+
lines.push(` roles: source=${model.inventory.filesByRole.source} test=${model.inventory.filesByRole.test} docs=${model.inventory.filesByRole.docs} config=${model.inventory.filesByRole.config} package=${model.inventory.filesByRole.package} generated=${model.inventory.filesByRole.generated} build-output=${model.inventory.filesByRole["build-output"]} vendor=${model.inventory.filesByRole.vendor} report-output=${model.inventory.filesByRole["report-output"]} unknown=${model.inventory.filesByRole.unknown}`);
|
|
73
75
|
if (model.inventory.warnings.length > 0) {
|
|
74
76
|
lines.push(` warnings: ${model.inventory.warnings.length}`);
|
|
75
77
|
}
|
|
76
78
|
lines.push("");
|
|
79
|
+
lines.push("Source facts summary");
|
|
80
|
+
lines.push(divider("-"));
|
|
81
|
+
lines.push(` analyzed=${model.sourceFacts.totalFilesAnalyzed}`);
|
|
82
|
+
lines.push(` parsed=${model.sourceFacts.filesByParseStatus.parsed} file-level-only=${model.sourceFacts.filesByParseStatus["file-level-only"]} unsupported=${model.sourceFacts.filesByParseStatus.unsupported} parse-error=${model.sourceFacts.filesByParseStatus["parse-error"]} skipped=${model.sourceFacts.filesByParseStatus.skipped}`);
|
|
83
|
+
if (model.sourceFacts.analyzerDiagnosticCount > 0) {
|
|
84
|
+
lines.push(` analyzer diagnostics: ${model.sourceFacts.analyzerDiagnosticCount}`);
|
|
85
|
+
}
|
|
86
|
+
lines.push("");
|
|
77
87
|
lines.push("Source-of-truth summary");
|
|
78
88
|
lines.push(divider("-"));
|
|
79
89
|
lines.push(` package: ${model.sourceOfTruth.packageName ?? "(none)"}@${model.sourceOfTruth.packageVersion ?? "?"}`);
|
|
@@ -153,12 +163,18 @@ function renderIssueBlock(issue) {
|
|
|
153
163
|
lines.push(` severity=${issue.severity} confidence=${issue.confidence} category=${sanitizeLine(issue.category)}`);
|
|
154
164
|
lines.push(` detector=${issue.detectorId} id=${issue.id}`);
|
|
155
165
|
for (const evidence of issue.evidence.slice(0, MAX_EVIDENCE_PER_ISSUE)) {
|
|
166
|
+
// v0.3.1 Batch 5 -- an evidence entry's `message` (the human-readable
|
|
167
|
+
// "why") and `excerpt` (a quoted snippet/reference list) are
|
|
168
|
+
// independent fields on the same entry -- render both when both are
|
|
169
|
+
// present. Previously an `excerpt` silently suppressed `message`
|
|
170
|
+
// entirely, which dropped the only readable explanation for
|
|
171
|
+
// source-facts-derived findings (e.g. duplicateImplementationDetector's
|
|
172
|
+
// "Source facts: an exported ... was parsed in N distinct files.") and
|
|
173
|
+
// left only a bare file-path excerpt in text output.
|
|
174
|
+
lines.push(` > ${sanitizeLine(evidence.message)}`);
|
|
156
175
|
if (evidence.excerpt) {
|
|
157
176
|
lines.push(...renderQuotedExcerpt(evidence.excerpt, " "));
|
|
158
177
|
}
|
|
159
|
-
else {
|
|
160
|
-
lines.push(` > ${sanitizeLine(evidence.message)}`);
|
|
161
|
-
}
|
|
162
178
|
}
|
|
163
179
|
lines.push(` recommendedAction: ${sanitizeLine(issue.recommendedAction)}`);
|
|
164
180
|
return lines;
|
package/package.json
CHANGED
|
@@ -1,15 +1,16 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@dailephd/my-dev-kit-lab",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.1",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Evidence, benchmark, and evaluation companion for my-dev-kit.",
|
|
6
6
|
"bin": {
|
|
7
7
|
"my-dev-kit-lab": "dist/scripts/run-final-demo.js"
|
|
8
8
|
},
|
|
9
|
-
"files": [
|
|
10
|
-
"dist/scripts/",
|
|
11
|
-
"dist/
|
|
12
|
-
"
|
|
9
|
+
"files": [
|
|
10
|
+
"dist/scripts/",
|
|
11
|
+
"!dist/scripts/docs/",
|
|
12
|
+
"dist/src/",
|
|
13
|
+
"benchmarks/",
|
|
13
14
|
"docs/METRICS.md",
|
|
14
15
|
"examples/",
|
|
15
16
|
"tests/fixtures/"
|
|
@@ -63,6 +64,8 @@
|
|
|
63
64
|
"experiment:list": "tsx scripts/experiments/listExperiments.ts",
|
|
64
65
|
"experiment:describe": "tsx scripts/experiments/describeExperiment.ts",
|
|
65
66
|
"experiment:run": "tsx scripts/experiments/runExperiment.ts",
|
|
67
|
+
"ecosystem:sync": "tsx scripts/docs/syncEcosystemDocs.ts",
|
|
68
|
+
"ecosystem:check": "tsx scripts/docs/checkEcosystemDocs.ts",
|
|
66
69
|
"run-final-demo": "tsx scripts/run-final-demo.ts",
|
|
67
70
|
"run-agent-prompt": "tsx scripts/run-agent-prompt.ts",
|
|
68
71
|
"run-controlled-experiment": "tsx scripts/run-controlled-experiment.ts",
|