@dailephd/my-dev-kit-lab 0.3.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,8 +1,10 @@
1
1
  # my-dev-kit-lab
2
2
 
3
- my-dev-kit-lab is the experiment, evidence, reporting, security-validation, and future audit companion for my-dev-kit. It runs reproducible experiments that test whether my-dev-kit's graph-guided retrieval helps coding-agent workflows, collects metrics, renders reports, generates plots, captures screenshots, builds gallery outputs, and performs automated CLI/package security validation.
4
-
5
- The current package version is `v0.3.0` (release-prepared; not yet published to npm, tagged, or released on GitHub). The latest published npm baseline is `v0.2.2`. Its generic experiment-plugin framework was introduced in v0.2.0. The first plugin is `context-strategy-comparison`, which preserves the existing raw-full-file vs my-dev-kit-guided workflow through the plugin runner.
3
+ my-dev-kit-lab is the experiment, evidence, reporting, security-validation, and audit companion for my-dev-kit. It runs reproducible experiments that test whether my-dev-kit's graph-guided retrieval helps coding-agent workflows, collects metrics, renders reports, generates plots, captures screenshots, builds gallery outputs, performs automated CLI/package security validation, and runs the generic audit framework.
4
+
5
+ The current published baseline is `v0.3.0`. `v0.3.0` introduced the generic audit framework and the first implemented code-rot detector family. The generic experiment-plugin framework was introduced in `v0.2.0`. The first plugin is `context-strategy-comparison`, which preserves the existing raw-full-file vs my-dev-kit-guided workflow through the plugin runner.
6
+
7
+ The checked-out package is release-prepared for `v0.3.1` on top of the published baseline: normalized language/file-role inventory, source-facts collection, a TypeScript/JavaScript analyzer, source-facts-aware code-rot signals, and source-facts summaries in text/JSON audit reports. `v0.3.1` is not published yet; Python, Java, Kotlin, Android validation, framework-aware profiles, and broader audit types remain future roadmap work.
6
8
 
7
9
  my-dev-kit is most useful when the repository is larger than the task. It helps coding agents work with large codebases through reusable structural indexing, graph-guided retrieval, targeted source slices, and auditable context selection. Results are scoped evidence; the project does not claim that my-dev-kit always saves tokens.
8
10
 
@@ -32,7 +34,7 @@ my-dev-kit is most useful when the repository is larger than the task. It helps
32
34
  - Target-aware experiment execution for local projects via `experiment:run -- --target <path>`
33
35
  - Plugin-aware JSON and HTML reports with plugin, target, variant, metric, artifact, warning, skip, and failure metadata
34
36
  - Security validation framework: dependency audit, package tarball inspection, CLI adversarial tests, static scans (CodeQL/Semgrep), bounded fuzz smoke, attack-scenario checks, and structured verdict/report output — runnable against any local project via `security:validate --target <path>`
35
- - Generic audit framework (package.json now specifies version `v0.3.0`; release-prepared but not yet published to npm, tagged, or released on GitHub): `npm run audit` runs heuristic, conservative code-rot detectors and writes text/JSON reports under `reports/audits/code-rot/` by default. Only the `code-rot` audit type is currently implemented; `quality`, `security`, `project`, and `all` audit types are planned and fail cleanly instead of running. `npm run audit` supports `--target`, `--types code-rot`, and other flags — see [docs/COMMANDS.md](docs/COMMANDS.md). Audit does not modify target files, does not auto-fix issues, does not run `security:validate`, and does not perform release-readiness determination.
37
+ - Generic audit framework in the current published `v0.3.0` baseline: `npm run audit` runs heuristic, conservative code-rot detectors and writes text/JSON reports under `reports/audits/code-rot/` by default. Only the `code-rot` audit type is currently implemented; `quality`, `security`, `project`, and `all` audit types are planned and fail cleanly instead of running. `npm run audit` supports `--target`, `--types code-rot`, and other flags — see [docs/COMMANDS.md](docs/COMMANDS.md). Audit does not modify target files, does not auto-fix issues, does not run `security:validate`, and does not perform release-readiness determination.
36
38
 
37
39
  ---
38
40
 
@@ -201,9 +203,16 @@ See [docs/METRICS.md](docs/METRICS.md) for full metric definitions.
201
203
 
202
204
  ---
203
205
 
204
- ## Current baseline release positioning
205
-
206
- my-dev-kit-lab is at a working baseline. The raw-vs-indexed experiment pipeline is fully implemented and produces reproducible artifacts. Real-agent campaign support exists for Codex and Claude. The latest published npm baseline is `v0.2.2`, which implements the automated security-validation fortification work described below and has passed pre-release readiness and cross-platform validation. package.json now specifies version `v0.3.0`, which adds the generic audit framework described below; `v0.3.0` is release-prepared but has not yet been published to npm, tagged, or released on GitHub.
206
+ ## Current baseline release positioning
207
+
208
+ my-dev-kit-lab is at a working baseline. The raw-vs-indexed experiment pipeline is fully implemented and produces reproducible artifacts. Real-agent campaign support exists for Codex and Claude. The current published npm baseline is `v0.3.0`, which includes the generic audit framework and the first implemented code-rot detector family on top of the earlier experiment and automated security-validation work.
209
+
210
+ Planned roadmap direction after the published baseline:
211
+
212
+ - `v0.3.1`: language-aware code-rot substrate plus TypeScript/JavaScript support, release-prepared and not yet published
213
+ - `v0.3.2` through `v0.3.4`: Python, Java/Kotlin, and cross-language stability for the language-aware code-rot track
214
+ - `v0.4.0` through `v0.4.2`: Android automated security validation
215
+ - manual pentest: post-v1 / version TBD
207
216
 
208
217
  ---
209
218
 
@@ -241,7 +250,7 @@ Each security command can validate my-dev-kit-lab itself or another local projec
241
250
 
242
251
  `security:validate` supports `--checks`, `--profile`, `--format`, `--fail-on`, `--out`, and `--report-prefix`. The no-flag path remains backward compatible and still runs the classic implemented check groups: `deps`, `package`, `static`, `cli-adversarial`, and `fuzz`. Supplying `--profile` without `--checks` swaps in that profile's default checks; supplying explicit `--checks` always wins over profile defaults.
243
252
 
244
- The current attack-scenario checks cover boundary, subprocess, secrets, and network assumptions. They are automated adversarial checks, not a manual pentest. `verdictImpact` metadata from each registered scenario drives blocker categorization in report reasoning, and narrowed `--checks` runs are labeled as scoped rather than described as a full release gate.
253
+ The current attack-scenario checks cover boundary, subprocess, secrets, and network assumptions. They are automated adversarial checks, not a manual pentest. Manual pentest is deferred until after `v1.0.0`. `verdictImpact` metadata from each registered scenario drives blocker categorization in report reasoning, and narrowed `--checks` runs are labeled as scoped rather than described as a full release gate.
245
254
 
246
255
  JSON report poisoning/config-injection checks use a baseline-diff schema guard so legitimate additive JSON fields do not fail the guard while payload-created trusted top-level fields still do. Text reports are sanitized to strip ANSI/control-byte payloads before rendering.
247
256
 
@@ -86,6 +86,7 @@ if (model.summary.noDetectorsRegistered) {
86
86
  console.log(`\nInventory: ${model.inventory.totalScannedFileCount} file(s) scanned, ${model.inventory.skippedFileCount} skipped`);
87
87
  console.log(` source=${model.inventory.filesByCategory.source} tests=${model.inventory.filesByCategory.tests} docs=${model.inventory.filesByCategory.docs} package=${model.inventory.filesByCategory.package} config=${model.inventory.filesByCategory.config} scripts=${model.inventory.filesByCategory.scripts} ci=${model.inventory.filesByCategory.ci}`);
88
88
  console.log(`Source of truth: package=${model.sourceOfTruth.packageName ?? "(none)"}@${model.sourceOfTruth.packageVersion ?? "?"} readme=${model.sourceOfTruth.hasReadme} changelog=${model.sourceOfTruth.hasChangelog} ciWorkflows=${model.sourceOfTruth.ciWorkflowCount}`);
89
+ console.log(`Source facts: ${model.sourceFacts.totalFilesAnalyzed} file(s) analyzed (parsed=${model.sourceFacts.filesByParseStatus.parsed} file-level-only=${model.sourceFacts.filesByParseStatus["file-level-only"]} unsupported=${model.sourceFacts.filesByParseStatus.unsupported})`);
89
90
  if (writtenPaths.length > 0) {
90
91
  console.log(`\nReports written:`);
91
92
  for (const p of writtenPaths) {
@@ -2,6 +2,7 @@ import path from "node:path";
2
2
  import { readBoundedFileText } from "../utils/boundedRead.js";
3
3
  import { baseNameNoExt } from "../utils/filePatternUtils.js";
4
4
  import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
5
+ import { collectRelativeImportBasenames, getParsedSourceFacts, indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
5
6
  // ---------------------------------------------------------------------------
6
7
  // v0.3.0 Batch 4 — dead-code candidate detector.
7
8
  //
@@ -140,7 +141,13 @@ function findUnreferencedScriptFiles(ctx, pkg) {
140
141
  return issues;
141
142
  }
142
143
  function buildReverseReferenceIndex(ctx) {
143
- const referencedBasenames = new Set();
144
+ // v0.3.1 Batch 4 -- seeded with basenames the TypeScript/JavaScript
145
+ // analyzer already resolved from real import declarations (parsed files
146
+ // only). This only ever adds references, never removes them, so it can
147
+ // only reduce false positives relative to the regex-only scan below --
148
+ // safe even though ctx.sourceFacts is optional (returns an empty set when
149
+ // absent, e.g. in existing detector-only unit tests).
150
+ const referencedBasenames = collectRelativeImportBasenames(ctx.sourceFacts);
144
151
  const allCandidateFiles = [
145
152
  ...ctx.inventory.sourceFiles,
146
153
  ...ctx.inventory.testFiles,
@@ -164,6 +171,7 @@ function buildReverseReferenceIndex(ctx) {
164
171
  function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
165
172
  const referencedBasenames = buildReverseReferenceIndex(ctx);
166
173
  const binCandidates = pkg ? binTargetsAsSourceCandidates(pkg) : new Set();
174
+ const sourceFactsIndex = indexSourceFactsByPath(ctx.sourceFacts);
167
175
  const issues = [];
168
176
  for (const file of ctx.inventory.sourceFiles) {
169
177
  if (file.sizeBytes > MAX_READ_BYTES)
@@ -187,6 +195,26 @@ function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
187
195
  continue;
188
196
  if (referencedBasenames.has(base.toLowerCase()))
189
197
  continue;
198
+ const evidence = [
199
+ {
200
+ kind: "observation",
201
+ message: "No relative import/require specifier in the scanned source/test/script set resolves to this file's basename.",
202
+ filePath: file.relativePath,
203
+ source: DETECTOR_ID,
204
+ confidence: "low",
205
+ },
206
+ ];
207
+ // v0.3.1 Batch 4 -- when the TypeScript/JavaScript analyzer actually
208
+ // parsed this file, surface its export count as extra candidate
209
+ // evidence (an exported symbol with zero detected external references
210
+ // is more suspicious than an unexported one). This never changes
211
+ // whether the issue is emitted, only its evidence -- the flagging
212
+ // decision above is unchanged from the existing basename-reference
213
+ // heuristic.
214
+ const facts = getParsedSourceFacts(sourceFactsIndex, file.relativePath);
215
+ if (facts) {
216
+ evidence.push(buildSourceFactsEvidence(facts));
217
+ }
190
218
  issues.push(makeCodeRotIssue({
191
219
  auditType: "code-rot",
192
220
  detectorId: DETECTOR_ID,
@@ -203,20 +231,25 @@ function findUnreferencedSourceFiles(ctx, pkg, docsConcat) {
203
231
  releaseBlocking: false,
204
232
  implementationBlocking: false,
205
233
  autoFixEligible: false,
206
- evidence: [
207
- {
208
- kind: "observation",
209
- message: "No relative import/require specifier in the scanned source/test/script set resolves to this file's basename.",
210
- filePath: file.relativePath,
211
- source: DETECTOR_ID,
212
- confidence: "low",
213
- },
214
- ],
234
+ evidence,
215
235
  affectedFiles: [file.relativePath],
216
236
  }));
217
237
  }
218
238
  return issues;
219
239
  }
240
+ // v0.3.1 Batch 4 -- describes a parsed file's source-facts shape as an
241
+ // evidence entry. Only called for parseStatus "parsed" facts (see callers),
242
+ // since file-level-only/parse-error facts carry empty fact arrays that
243
+ // would misleadingly read as "zero exports" rather than "unknown".
244
+ function buildSourceFactsEvidence(facts) {
245
+ return {
246
+ kind: "reference",
247
+ message: `Source facts: the TypeScript/JavaScript analyzer parsed this file and recorded ${facts.exports.length} export(s), ${facts.declarations.length} declaration(s), and ${facts.imports.length} import(s).`,
248
+ filePath: facts.relativePath,
249
+ source: DETECTOR_ID,
250
+ confidence: "medium",
251
+ };
252
+ }
220
253
  function findOldOrDeprecatedFilesNotInDocs(ctx, docsConcat) {
221
254
  const issues = [];
222
255
  for (const file of ctx.inventory.files) {
@@ -1,6 +1,7 @@
1
1
  import { readBoundedFileText } from "../utils/boundedRead.js";
2
- import { firstPathSegment, GENERIC_INFRA_BASENAMES, baseNameNoExt } from "../utils/filePatternUtils.js";
2
+ import { firstPathSegment, GENERIC_DECLARATION_NAMES, GENERIC_INFRA_BASENAMES, baseNameNoExt } from "../utils/filePatternUtils.js";
3
3
  import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
4
+ import { indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
4
5
  // ---------------------------------------------------------------------------
5
6
  // v0.3.0 Batch 4 — duplicate/parallel implementation candidate detector.
6
7
  //
@@ -60,6 +61,7 @@ export const DUPLICATE_IMPLEMENTATION_DETECTOR = {
60
61
  if (ctx.config.include.includes("architecture")) {
61
62
  issues.push(...findParallelDirectories(ctx.inventory.files.map((f) => f.relativePath)));
62
63
  issues.push(...findDuplicateBasenamesAcrossRoots(ctx.inventory.sourceFiles.map((f) => f.relativePath)));
64
+ issues.push(...findDuplicateDeclarationCandidates(ctx));
63
65
  }
64
66
  issues.push(...findDuplicateDetectorRegistryIds(ctx));
65
67
  return deduplicateIssuesById(issues);
@@ -214,6 +216,81 @@ function firstTopLevelUnderSrc(relativePath) {
214
216
  return null;
215
217
  return `${parts[0]}/${parts[1]}`;
216
218
  }
219
+ // v0.3.1 Batch 4 -- source-facts-aware duplicate-declaration candidate
220
+ // check. Weak/low-confidence signal, same spirit as
221
+ // findDuplicateBasenamesAcrossRoots(): the same exported declaration name
222
+ // and kind (e.g. two exported `class Logger`) appears in 2-4 distinct
223
+ // "source"-role files that a registered analyzer actually parsed. Bounded
224
+ // to declaration kinds that carry real implementation weight
225
+ // (function/class/interface/type/enum) -- "variable"/"constant"/"method"/
226
+ // "unknown" are excluded as too noisy (methods repeat across unrelated
227
+ // classes by convention; module-scope variables/constants repeat even more
228
+ // often). Generic declaration names (run, init, handler, ...) are excluded
229
+ // via GENERIC_DECLARATION_NAMES for the same reason
230
+ // GENERIC_INFRA_BASENAMES excludes generic file basenames above. Requires
231
+ // ctx.sourceFacts -- returns no issues when it is absent (e.g. existing
232
+ // detector unit tests that build a literal AuditDetectorContext without
233
+ // it), so this check is purely additive.
234
+ const NOISY_DECLARATION_KINDS = new Set(["variable", "constant", "method", "unknown"]);
235
+ function findDuplicateDeclarationCandidates(ctx) {
236
+ if (!ctx.sourceFacts)
237
+ return [];
238
+ const index = indexSourceFactsByPath(ctx.sourceFacts);
239
+ const byKey = new Map();
240
+ for (const facts of index.values()) {
241
+ if (facts.role !== "source")
242
+ continue;
243
+ if (facts.parseStatus !== "parsed")
244
+ continue;
245
+ for (const decl of facts.declarations) {
246
+ if (!decl.exported)
247
+ continue;
248
+ if (NOISY_DECLARATION_KINDS.has(decl.kind))
249
+ continue;
250
+ const nameLower = decl.name.toLowerCase();
251
+ if (GENERIC_DECLARATION_NAMES.has(nameLower))
252
+ continue;
253
+ const key = `${decl.kind}:${nameLower}`;
254
+ const entry = byKey.get(key) ?? { name: decl.name, kind: decl.kind, paths: new Set() };
255
+ entry.paths.add(facts.relativePath);
256
+ byKey.set(key, entry);
257
+ }
258
+ }
259
+ const issues = [];
260
+ for (const entry of byKey.values()) {
261
+ if (entry.paths.size < 2 || entry.paths.size > 4)
262
+ continue;
263
+ const sortedPaths = [...entry.paths].sort((a, b) => a.localeCompare(b));
264
+ issues.push(makeCodeRotIssue({
265
+ auditType: "code-rot",
266
+ detectorId: DETECTOR_ID,
267
+ idCues: ["duplicate-declaration-candidate", entry.kind, entry.name.toLowerCase(), ...sortedPaths],
268
+ title: `Exported ${entry.kind} "${entry.name}" is declared in ${sortedPaths.length} unrelated files (weak signal)`,
269
+ description: `An exported ${entry.kind} named "${entry.name}" is declared in: ${sortedPaths.join(", ")}. This is a source-facts-derived candidate signal (same declaration name and kind, not a semantic-equivalence check) for parallel implementations -- inspect manually before consolidating.`,
270
+ severity: "info",
271
+ confidence: "low",
272
+ falsePositiveRisk: "high",
273
+ category: "duplicate-implementation-candidate",
274
+ recommendedAction: "Manually confirm whether these declarations serve genuinely different purposes before taking any action.",
275
+ suggestedFixStrategy: "No automatic action -- rename or consolidate only after manual review.",
276
+ validationCommands: ["npm run audit -- --types code-rot --include architecture"],
277
+ releaseBlocking: false,
278
+ implementationBlocking: false,
279
+ autoFixEligible: false,
280
+ evidence: [
281
+ {
282
+ kind: "reference",
283
+ message: `Source facts: an exported ${entry.kind} named "${entry.name}" was parsed in ${sortedPaths.length} distinct files.`,
284
+ excerpt: sortedPaths.join(", "),
285
+ source: DETECTOR_ID,
286
+ confidence: "low",
287
+ },
288
+ ],
289
+ affectedFiles: sortedPaths,
290
+ }));
291
+ }
292
+ return issues;
293
+ }
217
294
  // Cheap regex scan (not a semantic parse) of already-small detector source
218
295
  // files for exported `id: "..."` string literals. Flags duplicates only if
219
296
  // genuinely found via this file-based check -- per the spec's own allowed
@@ -2,6 +2,7 @@ import fs from "node:fs";
2
2
  import path from "node:path";
3
3
  import { readBoundedFileText } from "../utils/boundedRead.js";
4
4
  import { deduplicateIssuesById, makeCodeRotIssue } from "../utils/issueFactories.js";
5
+ import { getParsedSourceFacts, indexSourceFactsByPath } from "../utils/sourceFactsLookup.js";
5
6
  import { splitLines } from "../utils/textLines.js";
6
7
  // ---------------------------------------------------------------------------
7
8
  // v0.3.0 Batch 4 — test-rot detector.
@@ -42,6 +43,7 @@ export const TEST_ROT_DETECTOR = {
42
43
  const issues = [];
43
44
  const scriptNames = new Set(ctx.sourceOfTruth.commands.allScriptNames);
44
45
  const currentVersion = ctx.sourceOfTruth.package?.version ?? null;
46
+ const sourceFactsIndex = indexSourceFactsByPath(ctx.sourceFacts);
45
47
  for (const testFile of ctx.inventory.testFiles) {
46
48
  const content = readBoundedFileText(ctx.target.rootPath, testFile.relativePath, testFile.sizeBytes, MAX_READ_BYTES);
47
49
  if (content === null)
@@ -61,6 +63,17 @@ export const TEST_ROT_DETECTOR = {
61
63
  if (testFile.relativePath.startsWith("tests/audits/codeRot/"))
62
64
  continue;
63
65
  issues.push(...findMissingSourceImports(ctx, testFile.relativePath, content));
66
+ // v0.3.1 Batch 4 -- when the TypeScript/JavaScript analyzer actually
67
+ // parsed this test file, also check its structured import facts.
68
+ // This catches import forms the regex above misses (e.g. multi-line
69
+ // import statements) without duplicate-reporting a specifier both
70
+ // checks already found: idCues use the same shape
71
+ // (testRelativePath, "missing-source-import", specifier), so an
72
+ // identical id is deduplicated by deduplicateIssuesById() below.
73
+ const parsedFacts = getParsedSourceFacts(sourceFactsIndex, testFile.relativePath);
74
+ if (parsedFacts) {
75
+ issues.push(...findMissingSourceImportsFromSourceFacts(ctx, testFile.relativePath, parsedFacts));
76
+ }
64
77
  issues.push(...findStaleNpmRunReferences(testFile.relativePath, content, scriptNames));
65
78
  issues.push(...findSkipUsage(testFile.relativePath, content));
66
79
  issues.push(...findOnlyUsage(testFile.relativePath, content));
@@ -115,6 +128,55 @@ function findMissingSourceImports(ctx, testRelativePath, content) {
115
128
  }
116
129
  return issues;
117
130
  }
131
+ // v0.3.1 Batch 4 -- source-facts-derived counterpart to
132
+ // findMissingSourceImports() above. Uses the TypeScript/JavaScript
133
+ // analyzer's already-parsed ImportFact list instead of a regex scan of raw
134
+ // content, so it also catches relative import specifiers the regex above
135
+ // misses. Only called for facts with parseStatus "parsed" (see caller) --
136
+ // file-level-only/parse-error facts carry an empty imports array that
137
+ // would otherwise silently look like "no imports" rather than "unknown".
138
+ function findMissingSourceImportsFromSourceFacts(ctx, testRelativePath, facts) {
139
+ const issues = [];
140
+ const testDir = path.dirname(testRelativePath);
141
+ for (const imp of facts.imports) {
142
+ if (!imp.source.startsWith("."))
143
+ continue;
144
+ const resolvedBase = path.join(testDir, imp.source).replace(/\\/g, "/");
145
+ const strippedBase = resolvedBase.replace(/\.[cm]?[jt]sx?$/, "");
146
+ const resolved = RESOLVE_EXTENSIONS.some((ext) => fs.existsSync(path.join(ctx.target.rootPath, `${strippedBase}${ext}`)));
147
+ if (resolved)
148
+ continue;
149
+ issues.push(makeCodeRotIssue({
150
+ auditType: "code-rot",
151
+ detectorId: DETECTOR_ID,
152
+ idCues: [testRelativePath, "missing-source-import", imp.source],
153
+ title: `Test "${testRelativePath}" imports a missing source file: "${imp.source}"`,
154
+ description: `${testRelativePath} imports "${imp.source}" (source-facts-derived via the TypeScript/JavaScript analyzer), which does not resolve to an existing file on disk (checked common TS/JS extensions relative to the test file).`,
155
+ severity: "medium",
156
+ confidence: "high",
157
+ falsePositiveRisk: "low",
158
+ category: "test-rot",
159
+ recommendedAction: "Fix or remove the stale import.",
160
+ suggestedFixStrategy: `Update the import specifier "${imp.source}" in ${testRelativePath} to point to an existing file.`,
161
+ validationCommands: [`npx vitest run ${testRelativePath}`],
162
+ releaseBlocking: false,
163
+ implementationBlocking: false,
164
+ autoFixEligible: false,
165
+ evidence: [
166
+ {
167
+ kind: "file",
168
+ message: "Analyzer-recorded relative import specifier does not resolve to an existing file.",
169
+ filePath: testRelativePath,
170
+ excerpt: imp.source,
171
+ source: DETECTOR_ID,
172
+ confidence: "high",
173
+ },
174
+ ],
175
+ affectedFiles: [testRelativePath],
176
+ }));
177
+ }
178
+ return issues;
179
+ }
118
180
  function findStaleNpmRunReferences(testRelativePath, content, scriptNames) {
119
181
  const issues = [];
120
182
  const seen = new Set();
@@ -45,3 +45,35 @@ export const GENERIC_INFRA_BASENAMES = new Set([
45
45
  "errors",
46
46
  "error",
47
47
  ]);
48
+ // v0.3.1 Batch 4 -- common, generic exported-declaration names that are
49
+ // expected to repeat across unrelated files legitimately (CLI entrypoints,
50
+ // handler/lifecycle conventions, etc.) -- excluded from the source-facts
51
+ // duplicate-declaration-candidate heuristic in duplicateImplementationDetector.ts
52
+ // to keep false positives near zero.
53
+ export const GENERIC_DECLARATION_NAMES = new Set([
54
+ "run",
55
+ "main",
56
+ "init",
57
+ "initialize",
58
+ "setup",
59
+ "handler",
60
+ "handle",
61
+ "execute",
62
+ "process",
63
+ "validate",
64
+ "parse",
65
+ "format",
66
+ "render",
67
+ "build",
68
+ "create",
69
+ "start",
70
+ "stop",
71
+ "load",
72
+ "save",
73
+ "update",
74
+ "remove",
75
+ "apply",
76
+ "get",
77
+ "set",
78
+ "default",
79
+ ]);
@@ -0,0 +1,53 @@
1
+ import { baseNameNoExt } from "./filePatternUtils.js";
2
+ // ---------------------------------------------------------------------------
3
+ // v0.3.1 Batch 4 -- shared source-facts lookup helpers for code-rot
4
+ // detectors.
5
+ //
6
+ // ctx.sourceFacts is optional (see auditRegistry.ts) precisely so existing
7
+ // detector tests that build a literal AuditDetectorContext without it keep
8
+ // working unchanged -- every helper here accepts `undefined` and degrades to
9
+ // an empty result rather than throwing, so a caller never needs its own
10
+ // null-check before using them.
11
+ // ---------------------------------------------------------------------------
12
+ // Indexes a SourceFactsSnapshot's files by relativePath for O(1) per-file
13
+ // lookups. Returns an empty map when sourceFacts is undefined.
14
+ export function indexSourceFactsByPath(sourceFacts) {
15
+ const index = new Map();
16
+ if (!sourceFacts)
17
+ return index;
18
+ for (const file of sourceFacts.files)
19
+ index.set(file.relativePath, file);
20
+ return index;
21
+ }
22
+ // Returns the facts for relativePath only when parseStatus is "parsed" --
23
+ // callers that want analyzer-derived evidence (imports/exports/declarations)
24
+ // should use this rather than reading the index directly, since
25
+ // file-level-only/parse-error/skipped/unsupported entries carry empty fact
26
+ // arrays that would silently look like "no imports" rather than "unknown".
27
+ export function getParsedSourceFacts(index, relativePath) {
28
+ const facts = index.get(relativePath);
29
+ return facts && facts.parseStatus === "parsed" ? facts : null;
30
+ }
31
+ // Basenames referenced by any parsed file's relative import/re-export
32
+ // specifier, lowercased. Mirrors deadCodeCandidateDetector.ts's regex-based
33
+ // buildReverseReferenceIndex() basename matching, but sourced from
34
+ // structured analyzer facts -- so it also catches import forms the regex
35
+ // misses (e.g. multi-line import statements) for files a registered
36
+ // analyzer actually parsed. Returns an empty set when sourceFacts is
37
+ // undefined, so merging it into an existing Set is always safe.
38
+ export function collectRelativeImportBasenames(sourceFacts) {
39
+ const basenames = new Set();
40
+ if (!sourceFacts)
41
+ return basenames;
42
+ for (const file of sourceFacts.files) {
43
+ if (file.parseStatus !== "parsed")
44
+ continue;
45
+ for (const imp of file.imports) {
46
+ if (!imp.source.startsWith("."))
47
+ continue;
48
+ const base = baseNameNoExt(imp.source.split("?")[0]);
49
+ basenames.add(base.toLowerCase());
50
+ }
51
+ }
52
+ return basenames;
53
+ }
@@ -3,14 +3,16 @@ import { DEFAULT_AUDIT_REGISTRY, selectDetectors } from "./auditRegistry.js";
3
3
  import { resolveAuditTarget } from "./auditTarget.js";
4
4
  import { scanProjectInventory } from "./projectInventory.js";
5
5
  import { collectSourceOfTruth } from "./sourceOfTruth.js";
6
+ import { collectSourceFacts } from "./collectSourceFacts.js";
6
7
  export async function runAudit(options) {
7
8
  const { config, toolRoot } = options;
8
9
  const registry = options.registry ?? DEFAULT_AUDIT_REGISTRY;
9
10
  const target = options.target ?? resolveAuditTarget(config.targetPathArg, toolRoot);
10
11
  const inventory = scanProjectInventory(target.rootPath);
11
12
  const sourceOfTruth = collectSourceOfTruth(target.rootPath, inventory);
13
+ const sourceFacts = await collectSourceFacts(target.rootPath, inventory);
12
14
  const selected = selectDetectors(registry, config.types, config.include);
13
- const ctx = { target, config, inventory, sourceOfTruth };
15
+ const ctx = { target, config, inventory, sourceOfTruth, sourceFacts };
14
16
  const issues = [];
15
17
  const skippedDetectors = [];
16
18
  const detectorErrors = [];
@@ -59,6 +61,7 @@ export async function runAudit(options) {
59
61
  exitReason: reason,
60
62
  inventory,
61
63
  sourceOfTruth,
64
+ sourceFacts,
62
65
  noDetectorsRegistered: selected.length === 0,
63
66
  };
64
67
  }
@@ -0,0 +1,142 @@
1
+ import fs from "node:fs";
2
+ import path from "node:path";
3
+ import { NORMALIZED_LANGUAGES } from "./projectInventory.js";
4
+ import { DEFAULT_LANGUAGE_ANALYZER_REGISTRY, selectLanguageAnalyzer, } from "./languageAnalyzerRegistry.js";
5
+ import { SOURCE_FACT_PARSE_STATUSES, } from "./sourceFacts.js";
6
+ // ---------------------------------------------------------------------------
7
+ // v0.3.1 Batch 2 -- source facts collector.
8
+ //
9
+ // Consumes the Batch 1 ProjectInventorySnapshot (never re-walks the
10
+ // filesystem) and produces a SourceFactsSnapshot: analyzer facts where a
11
+ // registered analyzer exists, deterministic fallback facts otherwise. Same
12
+ // spirit as sourceOfTruth.ts -- pure data collection, never an AuditIssue,
13
+ // never a write to a target file.
14
+ //
15
+ // Eligibility / omission policy (documented here since it's the single
16
+ // source of truth for the decision):
17
+ // - Only inventory entries with role "source" or "test" are considered.
18
+ // docs/config/package/generated/vendor/build-output/report-output/
19
+ // unknown-role files are omitted entirely from `files` -- source facts
20
+ // are meaningless for them, and omitting (rather than emitting a
21
+ // "skipped" placeholder for every doc/config file in the repo) keeps
22
+ // the snapshot proportional to actual candidate source files.
23
+ // - For an eligible file in a *known* language (typescript, javascript,
24
+ // python, java, kotlin) with no registered analyzer: parseStatus
25
+ // "file-level-only" -- we know the language and already have
26
+ // inventory-level evidence (lineCount), just no analyzer parsed it yet.
27
+ // - For an eligible file in an *unknown* language: parseStatus
28
+ // "unsupported" -- no file-level evidence attempted either.
29
+ // - File content is only read when a registered analyzer actually claims
30
+ // the file -- with an empty default registry (Batch 2), no file is ever
31
+ // read by this collector.
32
+ // ---------------------------------------------------------------------------
33
+ const ANALYZABLE_ROLES = new Set(["source", "test"]);
34
+ const KNOWN_LANGUAGES = new Set(["typescript", "javascript", "python", "java", "kotlin"]);
35
+ export async function collectSourceFacts(targetRoot, inventory, registry = DEFAULT_LANGUAGE_ANALYZER_REGISTRY) {
36
+ const resolvedRoot = path.resolve(targetRoot);
37
+ const files = [];
38
+ const analyzerDiagnostics = [];
39
+ const warnings = [];
40
+ // inventory.files is already sorted by relativePath (Batch 1 guarantee) --
41
+ // filtering preserves that order, so no re-sort is needed here.
42
+ const eligible = inventory.files.filter((entry) => ANALYZABLE_ROLES.has(entry.role));
43
+ for (const entry of eligible) {
44
+ const analyzer = selectLanguageAnalyzer(registry, entry.language, entry.extension);
45
+ if (!analyzer) {
46
+ files.push(buildFallbackFacts(entry));
47
+ continue;
48
+ }
49
+ const absolutePath = path.join(resolvedRoot, entry.relativePath);
50
+ try {
51
+ const content = fs.readFileSync(absolutePath, "utf8");
52
+ const facts = await analyzer.analyzeFile({
53
+ targetRoot: resolvedRoot,
54
+ relativePath: entry.relativePath,
55
+ absolutePath,
56
+ content,
57
+ inventoryEntry: entry,
58
+ language: entry.language,
59
+ role: entry.role,
60
+ });
61
+ files.push(facts);
62
+ }
63
+ catch (err) {
64
+ const message = err instanceof Error ? err.message : String(err);
65
+ const diagnostic = {
66
+ severity: "error",
67
+ code: "analyzer-error",
68
+ message: `Analyzer "${analyzer.id}" failed on "${entry.relativePath}": ${message}`,
69
+ path: entry.relativePath,
70
+ analyzerId: analyzer.id,
71
+ };
72
+ analyzerDiagnostics.push(diagnostic);
73
+ files.push({
74
+ relativePath: entry.relativePath,
75
+ language: entry.language,
76
+ role: entry.role,
77
+ parseStatus: "parse-error",
78
+ analyzerId: analyzer.id,
79
+ lineCount: entry.lineCount,
80
+ imports: [],
81
+ exports: [],
82
+ declarations: [],
83
+ references: [],
84
+ diagnostics: [diagnostic],
85
+ });
86
+ }
87
+ }
88
+ return {
89
+ targetRoot: resolvedRoot,
90
+ files,
91
+ filesByLanguage: countByLanguage(files),
92
+ filesByParseStatus: countByParseStatus(files),
93
+ analyzerDiagnostics,
94
+ warnings,
95
+ };
96
+ }
97
+ function buildFallbackFacts(entry) {
98
+ const known = KNOWN_LANGUAGES.has(entry.language);
99
+ const parseStatus = known ? "file-level-only" : "unsupported";
100
+ const diagnostic = known
101
+ ? {
102
+ severity: "info",
103
+ code: "no-analyzer-registered",
104
+ message: `No language analyzer is registered for "${entry.language}" yet; only inventory-level facts are available.`,
105
+ path: entry.relativePath,
106
+ analyzerId: null,
107
+ }
108
+ : {
109
+ severity: "info",
110
+ code: "unsupported-language",
111
+ message: `Language "${entry.language}" is not supported for source-fact extraction.`,
112
+ path: entry.relativePath,
113
+ analyzerId: null,
114
+ };
115
+ return {
116
+ relativePath: entry.relativePath,
117
+ language: entry.language,
118
+ role: entry.role,
119
+ parseStatus,
120
+ analyzerId: null,
121
+ lineCount: entry.lineCount,
122
+ imports: [],
123
+ exports: [],
124
+ declarations: [],
125
+ references: [],
126
+ diagnostics: [diagnostic],
127
+ };
128
+ }
129
+ function countByLanguage(files) {
130
+ const counts = Object.fromEntries(NORMALIZED_LANGUAGES.map((l) => [l, 0]));
131
+ for (const file of files) {
132
+ counts[file.language] += 1;
133
+ }
134
+ return counts;
135
+ }
136
+ function countByParseStatus(files) {
137
+ const counts = Object.fromEntries(SOURCE_FACT_PARSE_STATUSES.map((s) => [s, 0]));
138
+ for (const file of files) {
139
+ counts[file.parseStatus] += 1;
140
+ }
141
+ return counts;
142
+ }
@@ -0,0 +1,33 @@
1
+ import { TYPESCRIPT_JAVASCRIPT_ANALYZER } from "./typescriptJavaScriptAnalyzer.js";
2
+ // Validates a list has no duplicate analyzer ids. Later batches call this
3
+ // when assembling their own registries; Batch 2's empty default trivially
4
+ // passes.
5
+ export function createLanguageAnalyzerRegistry(analyzers) {
6
+ const seen = new Set();
7
+ for (const analyzer of analyzers) {
8
+ if (seen.has(analyzer.id)) {
9
+ throw new Error(`Duplicate language analyzer id: "${analyzer.id}".`);
10
+ }
11
+ seen.add(analyzer.id);
12
+ }
13
+ return analyzers;
14
+ }
15
+ // First matching analyzer in registry order (deterministic), or null when
16
+ // none supports the given language/extension. Order is preserved from the
17
+ // input registry -- not re-sorted.
18
+ export function selectLanguageAnalyzer(registry, language, extension) {
19
+ for (const analyzer of registry) {
20
+ if (!analyzer.supportedLanguages.includes(language))
21
+ continue;
22
+ if (analyzer.supportedExtensions && !analyzer.supportedExtensions.includes(extension))
23
+ continue;
24
+ return analyzer;
25
+ }
26
+ return null;
27
+ }
28
+ // v0.3.1 Batch 3 -- registers the TypeScript/JavaScript analyzer. Python,
29
+ // Java, and Kotlin remain fallback-only (no analyzer registered for them) --
30
+ // see collectSourceFacts.ts's fallback policy.
31
+ export const DEFAULT_LANGUAGE_ANALYZER_REGISTRY = createLanguageAnalyzerRegistry([
32
+ TYPESCRIPT_JAVASCRIPT_ANALYZER,
33
+ ]);
@@ -27,6 +27,112 @@ export const INVENTORY_FILE_CATEGORIES = [
27
27
  "report",
28
28
  "unknown",
29
29
  ];
30
+ // ---------------------------------------------------------------------------
31
+ // v0.3.1 Batch 1 — normalized language and file-role metadata.
32
+ //
33
+ // Purely additive to the v0.3.0 category model above: `category` keeps its
34
+ // existing meaning and existing consumers (detectors, report model) are
35
+ // untouched. `language` and `role` are deterministic, extension/path-based
36
+ // classifications layered on top so later v0.3.1 batches (language analyzer
37
+ // registry, per-language code-rot detectors) can select files by language
38
+ // without re-deriving it from extensions themselves. Language detection is
39
+ // classification-only here -- it does not imply parsing/analysis support for
40
+ // python/java/kotlin exists yet.
41
+ // ---------------------------------------------------------------------------
42
+ export const NORMALIZED_LANGUAGES = [
43
+ "typescript",
44
+ "javascript",
45
+ "python",
46
+ "java",
47
+ "kotlin",
48
+ "json",
49
+ "markdown",
50
+ "yaml",
51
+ "xml",
52
+ "toml",
53
+ "unknown",
54
+ ];
55
+ const EXTENSION_TO_LANGUAGE = {
56
+ ".ts": "typescript",
57
+ ".tsx": "typescript",
58
+ ".mts": "typescript",
59
+ ".cts": "typescript",
60
+ ".js": "javascript",
61
+ ".jsx": "javascript",
62
+ ".mjs": "javascript",
63
+ ".cjs": "javascript",
64
+ ".py": "python",
65
+ ".java": "java",
66
+ ".kt": "kotlin",
67
+ ".kts": "kotlin",
68
+ ".json": "json",
69
+ ".md": "markdown",
70
+ ".markdown": "markdown",
71
+ ".yml": "yaml",
72
+ ".yaml": "yaml",
73
+ ".xml": "xml",
74
+ ".toml": "toml",
75
+ };
76
+ function classifyLanguage(extension) {
77
+ return EXTENSION_TO_LANGUAGE[extension] ?? "unknown";
78
+ }
79
+ export const FILE_ROLES = [
80
+ "source",
81
+ "test",
82
+ "docs",
83
+ "config",
84
+ "package",
85
+ "generated",
86
+ "build-output",
87
+ "vendor",
88
+ "report-output",
89
+ "unknown",
90
+ ];
91
+ // Path-segment markers used only when a file wasn't already excluded from
92
+ // traversal entirely (EXCLUDED_DIR_NAMES below) -- e.g. paths reachable from
93
+ // nested benchmark/fixture projects, or directories not yet in the hard
94
+ // exclusion set. Deliberately conservative: matched as whole path segments,
95
+ // not substrings.
96
+ const VENDOR_PATH_SEGMENTS = new Set(["node_modules", "vendor"]);
97
+ const BUILD_OUTPUT_PATH_SEGMENTS = new Set(["dist", "build", "out", ".next", ".turbo", "target", ".gradle"]);
98
+ const GENERATED_PATH_SEGMENTS = new Set([
99
+ "generated",
100
+ ".cache",
101
+ ".my-dev-kit",
102
+ ".my-dev-kit-v1",
103
+ ".my-dev-kit-lab",
104
+ ]);
105
+ const REPORT_OUTPUT_PATH_SEGMENTS = new Set(["reports", "lab-output"]);
106
+ function classifyFileRole(relativePath, basename, extension, category) {
107
+ const segments = relativePath.split("/");
108
+ if (category === "report" ||
109
+ isStrayReportFile(basename) ||
110
+ segments.some((s) => REPORT_OUTPUT_PATH_SEGMENTS.has(s))) {
111
+ return "report-output";
112
+ }
113
+ if (segments.some((s) => VENDOR_PATH_SEGMENTS.has(s)))
114
+ return "vendor";
115
+ if (extension === ".tsbuildinfo" || segments.some((s) => BUILD_OUTPUT_PATH_SEGMENTS.has(s)))
116
+ return "build-output";
117
+ if (extension === ".map" || segments.some((s) => GENERATED_PATH_SEGMENTS.has(s)))
118
+ return "generated";
119
+ switch (category) {
120
+ case "tests":
121
+ return "test";
122
+ case "docs":
123
+ return "docs";
124
+ case "package":
125
+ return "package";
126
+ case "config":
127
+ case "ci":
128
+ return "config";
129
+ case "source":
130
+ case "scripts":
131
+ return "source";
132
+ default:
133
+ return "unknown";
134
+ }
135
+ }
30
136
  // Directory names excluded from traversal entirely -- never descended into.
31
137
  const EXCLUDED_DIR_NAMES = new Set([
32
138
  "node_modules",
@@ -96,6 +202,24 @@ const KNOWN_SOURCE_EXTENSIONS = new Set([
96
202
  ".go",
97
203
  ".rs",
98
204
  ".java",
205
+ // v0.3.1 Batch 2 fix: .kt/.kts were classified as a NormalizedLanguage
206
+ // ("kotlin") since Batch 1 but were missing here, so a Kotlin file under
207
+ // src/ got category "unknown" (role "unknown") instead of "source" --
208
+ // silently excluding it from source-facts collection. .go/.rs above are
209
+ // intentionally left as "source" despite having no NormalizedLanguage
210
+ // mapping yet (they normalize to language "unknown"), which is the
211
+ // legitimate "known role, unknown language" case collectSourceFacts.ts's
212
+ // fallback policy handles.
213
+ ".kt",
214
+ ".kts",
215
+ // v0.3.1 Batch 3 fix: same class of gap as the .kt/.kts fix above -- .mts
216
+ // and .cts were mapped to NormalizedLanguage "typescript" since Batch 1
217
+ // but were missing here, so a .mts/.cts file under src/ got category
218
+ // "unknown" (role "unknown") and would never reach the Batch 3
219
+ // TypeScript/JavaScript analyzer's eligibility filter (role source/test
220
+ // only, see collectSourceFacts.ts).
221
+ ".mts",
222
+ ".cts",
99
223
  ]);
100
224
  // A file this large or larger is skipped entirely (not even stat'd into the
101
225
  // inventory) -- protects against pathological huge-file scan cost.
@@ -123,6 +247,8 @@ export function scanProjectInventory(targetRoot, options = {}) {
123
247
  }
124
248
  const filesByCategory = countByCategory(files);
125
249
  const filesByExtension = countByExtension(files);
250
+ const filesByLanguage = countByLanguage(files);
251
+ const filesByRole = countByRole(files);
126
252
  const excludedDirectorySummary = [...excludedDirCounts.entries()]
127
253
  .map(([name, occurrences]) => ({ name, occurrences }))
128
254
  .sort((a, b) => a.name.localeCompare(b.name));
@@ -134,6 +260,8 @@ export function scanProjectInventory(targetRoot, options = {}) {
134
260
  skippedFileCount,
135
261
  filesByCategory,
136
262
  filesByExtension,
263
+ filesByLanguage,
264
+ filesByRole,
137
265
  packageFiles: files.filter((f) => f.category === "package"),
138
266
  docsFiles: files.filter((f) => f.category === "docs"),
139
267
  sourceFiles: files.filter((f) => f.category === "source"),
@@ -192,11 +320,19 @@ export function scanProjectInventory(targetRoot, options = {}) {
192
320
  const relativePath = relativeWithinRoot(resolvedRoot, fullPath);
193
321
  const extension = path.extname(entry.name).toLowerCase();
194
322
  const category = classifyFile(relativePath, entry.name, extension);
323
+ const language = classifyLanguage(extension);
324
+ const role = classifyFileRole(relativePath, entry.name, extension, category);
195
325
  const entryResult = {
196
326
  relativePath,
197
327
  normalizedPath: relativePath.toLowerCase(),
198
328
  extension,
199
329
  category,
330
+ language,
331
+ role,
332
+ isGenerated: role === "generated",
333
+ isBuildOutput: role === "build-output",
334
+ isVendor: role === "vendor",
335
+ isReportOutput: role === "report-output",
200
336
  sizeBytes: stat.size,
201
337
  likelyGenerated: category === "generated" || category === "report",
202
338
  likelyConfig: category === "config" || category === "package",
@@ -297,6 +433,20 @@ function countByCategory(files) {
297
433
  }
298
434
  return counts;
299
435
  }
436
+ function countByLanguage(files) {
437
+ const counts = Object.fromEntries(NORMALIZED_LANGUAGES.map((l) => [l, 0]));
438
+ for (const file of files) {
439
+ counts[file.language] += 1;
440
+ }
441
+ return counts;
442
+ }
443
+ function countByRole(files) {
444
+ const counts = Object.fromEntries(FILE_ROLES.map((r) => [r, 0]));
445
+ for (const file of files) {
446
+ counts[file.role] += 1;
447
+ }
448
+ return counts;
449
+ }
300
450
  function countByExtension(files) {
301
451
  const counts = new Map();
302
452
  for (const file of files) {
@@ -0,0 +1,17 @@
1
+ // ---------------------------------------------------------------------------
2
+ // v0.3.1 Batch 2 -- normalized source facts model.
3
+ //
4
+ // Purely additive substrate: no concrete language analyzer exists yet (that
5
+ // is Batch 3's TypeScript/JavaScript analyzer), so every SourceFileFacts
6
+ // produced in this batch is either a deterministic fallback (see
7
+ // collectSourceFacts.ts) or a parse-error record. The fact arrays
8
+ // (imports/exports/declarations/references) are always empty in this batch
9
+ // -- the shape exists so Batch 3 can populate them without a type change.
10
+ // ---------------------------------------------------------------------------
11
+ export const SOURCE_FACT_PARSE_STATUSES = [
12
+ "parsed",
13
+ "file-level-only",
14
+ "unsupported",
15
+ "parse-error",
16
+ "skipped",
17
+ ];
@@ -0,0 +1,299 @@
1
+ import ts from "typescript";
2
+ // ---------------------------------------------------------------------------
3
+ // v0.3.1 Batch 3 -- TypeScript/JavaScript source analyzer.
4
+ //
5
+ // Single-file, syntax-only analysis via the TypeScript compiler API:
6
+ // - ts.createSourceFile() builds a per-file AST (no Program, no project
7
+ // resolution, no type checking) that is walked once to extract facts.
8
+ // - ts.transpileModule({ reportDiagnostics: true }) is used purely to
9
+ // obtain syntactic diagnostics -- it is itself a single-file operation
10
+ // (no cross-file resolution), matching the "single-file parsing only"
11
+ // requirement for this batch.
12
+ //
13
+ // Parse-status policy (deliberately conservative, see docs on
14
+ // SourceFactParseStatus): zero syntax diagnostics -> "parsed". Any syntax
15
+ // diagnostic -> "parse-error", even though the AST walk below still runs
16
+ // against TypeScript's best-effort (error-tolerant) parse and whatever facts
17
+ // it manages to extract are still attached -- the parseStatus signals
18
+ // "don't treat these facts as fully reliable", not "no facts exist". This
19
+ // mirrors how collectSourceFacts.ts already reports a caught analyzer
20
+ // exception as parse-error with an empty-fact fallback -- this analyzer just
21
+ // reaches parse-error through diagnostics instead of a thrown exception, and
22
+ // still gets to keep best-effort facts (which a thrown-exception parse-error
23
+ // never has, since collectSourceFacts.ts only has the exception itself).
24
+ // ---------------------------------------------------------------------------
25
+ export const TYPESCRIPT_JAVASCRIPT_ANALYZER_ID = "typescript-javascript-analyzer";
26
+ const SUPPORTED_EXTENSIONS = [".ts", ".tsx", ".mts", ".cts", ".js", ".jsx", ".mjs", ".cjs"];
27
+ // AST construction is meaningfully more expensive than a line count -- large
28
+ // files (generated bundles, vendored single-file libraries that slipped past
29
+ // role/path exclusions, etc.) are not parsed. Mirrors the size-bounding
30
+ // philosophy of projectInventory.ts's own read limits, just at a tighter
31
+ // threshold appropriate for actually building an AST.
32
+ const MAX_ANALYZABLE_BYTES = 1_000_000;
33
+ export const TYPESCRIPT_JAVASCRIPT_ANALYZER = {
34
+ id: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
35
+ supportedLanguages: ["typescript", "javascript"],
36
+ supportedExtensions: SUPPORTED_EXTENSIONS,
37
+ analyzeFile,
38
+ };
39
+ function analyzeFile(input) {
40
+ const { relativePath, language, role, content, inventoryEntry } = input;
41
+ if (Buffer.byteLength(content, "utf8") > MAX_ANALYZABLE_BYTES) {
42
+ return {
43
+ relativePath,
44
+ language,
45
+ role,
46
+ parseStatus: "file-level-only",
47
+ analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
48
+ lineCount: inventoryEntry.lineCount,
49
+ imports: [],
50
+ exports: [],
51
+ declarations: [],
52
+ references: [],
53
+ diagnostics: [
54
+ {
55
+ severity: "info",
56
+ code: "file-too-large-for-analysis",
57
+ message: `File exceeds the ${MAX_ANALYZABLE_BYTES}-byte analyzer bound; only inventory-level facts are available.`,
58
+ path: relativePath,
59
+ analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
60
+ },
61
+ ],
62
+ };
63
+ }
64
+ const scriptKind = resolveScriptKind(inventoryEntry.extension);
65
+ const sourceFile = ts.createSourceFile(relativePath, content, ts.ScriptTarget.Latest, true, scriptKind);
66
+ const diagnostics = getSyntaxDiagnostics(relativePath, sourceFile);
67
+ const imports = [];
68
+ const exports = [];
69
+ const declarations = [];
70
+ const references = [];
71
+ const lineOf = (node) => sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile)).line + 1;
72
+ function pushDeclaration(node, name, kind) {
73
+ const isExported = hasModifier(node, ts.SyntaxKind.ExportKeyword);
74
+ const isDefault = hasModifier(node, ts.SyntaxKind.DefaultKeyword);
75
+ if (name) {
76
+ declarations.push({ name, kind, exported: isExported, line: lineOf(node) });
77
+ }
78
+ if (isDefault) {
79
+ exports.push({ name: name ?? "default", kind: "default", line: lineOf(node) });
80
+ }
81
+ else if (isExported && name) {
82
+ exports.push({ name, kind: "named", line: lineOf(node) });
83
+ }
84
+ }
85
+ function handleImportDeclaration(node) {
86
+ if (!ts.isStringLiteralLike(node.moduleSpecifier))
87
+ return;
88
+ const source = node.moduleSpecifier.text;
89
+ const importedNames = [];
90
+ const clause = node.importClause;
91
+ if (clause) {
92
+ if (clause.name)
93
+ importedNames.push(clause.name.text);
94
+ const bindings = clause.namedBindings;
95
+ if (bindings && ts.isNamespaceImport(bindings)) {
96
+ importedNames.push(bindings.name.text);
97
+ }
98
+ else if (bindings && ts.isNamedImports(bindings)) {
99
+ for (const element of bindings.elements)
100
+ importedNames.push(element.name.text);
101
+ }
102
+ }
103
+ imports.push({
104
+ source,
105
+ kind: "static",
106
+ line: lineOf(node),
107
+ ...(importedNames.length > 0 ? { importedNames } : {}),
108
+ });
109
+ }
110
+ function handleExportDeclaration(node) {
111
+ const source = node.moduleSpecifier && ts.isStringLiteralLike(node.moduleSpecifier) ? node.moduleSpecifier.text : undefined;
112
+ if (source) {
113
+ imports.push({ source, kind: "re-export", line: lineOf(node) });
114
+ }
115
+ if (!node.exportClause) {
116
+ if (source)
117
+ exports.push({ name: "*", kind: "namespace", line: lineOf(node) });
118
+ return;
119
+ }
120
+ if (ts.isNamespaceExport(node.exportClause)) {
121
+ exports.push({ name: node.exportClause.name.text, kind: "namespace", line: lineOf(node) });
122
+ return;
123
+ }
124
+ for (const element of node.exportClause.elements) {
125
+ exports.push({ name: element.name.text, kind: "named", line: lineOf(node) });
126
+ }
127
+ }
128
+ function handleExportAssignment(node) {
129
+ if (node.isExportEquals) {
130
+ exports.push({ name: "module.exports", kind: "commonjs", line: lineOf(node) });
131
+ return;
132
+ }
133
+ const expr = node.expression;
134
+ let name = "default";
135
+ if (ts.isIdentifier(expr)) {
136
+ name = expr.text;
137
+ }
138
+ else if ((ts.isFunctionExpression(expr) || ts.isClassExpression(expr)) && expr.name) {
139
+ name = expr.name.text;
140
+ }
141
+ exports.push({ name, kind: "default", line: lineOf(node) });
142
+ }
143
+ function handleVariableStatement(node) {
144
+ const isExported = hasModifier(node, ts.SyntaxKind.ExportKeyword);
145
+ const isConst = (node.declarationList.flags & ts.NodeFlags.Const) !== 0;
146
+ for (const decl of node.declarationList.declarations) {
147
+ // Destructuring patterns (`const { a, b } = x`) are conservatively
148
+ // skipped -- only simple identifier bindings are recorded.
149
+ if (!ts.isIdentifier(decl.name))
150
+ continue;
151
+ declarations.push({
152
+ name: decl.name.text,
153
+ kind: isConst ? "constant" : "variable",
154
+ exported: isExported,
155
+ line: lineOf(decl),
156
+ });
157
+ if (isExported) {
158
+ exports.push({ name: decl.name.text, kind: "named", line: lineOf(decl) });
159
+ }
160
+ }
161
+ }
162
+ function handleCommonJsAssignment(node) {
163
+ const left = node.left;
164
+ if (!ts.isPropertyAccessExpression(left) || !ts.isIdentifier(left.expression))
165
+ return;
166
+ if (left.expression.text === "module" && left.name.text === "exports") {
167
+ exports.push({ name: "module.exports", kind: "commonjs", line: lineOf(node) });
168
+ }
169
+ else if (left.expression.text === "exports") {
170
+ exports.push({ name: left.name.text, kind: "commonjs", line: lineOf(node) });
171
+ }
172
+ }
173
+ function handleCallExpression(node) {
174
+ if (ts.isIdentifier(node.expression) && node.expression.text === "require") {
175
+ const arg = node.arguments[0];
176
+ if (arg && ts.isStringLiteralLike(arg)) {
177
+ imports.push({ source: arg.text, kind: "commonjs", line: lineOf(node) });
178
+ }
179
+ return; // already represented as an import fact -- do not also reference "require".
180
+ }
181
+ if (node.expression.kind === ts.SyntaxKind.ImportKeyword) {
182
+ const arg = node.arguments[0];
183
+ if (arg && ts.isStringLiteralLike(arg)) {
184
+ imports.push({ source: arg.text, kind: "dynamic", line: lineOf(node) });
185
+ }
186
+ return;
187
+ }
188
+ if (ts.isIdentifier(node.expression)) {
189
+ // Conservative reference extraction: only bare-identifier call
190
+ // expressions (`foo()`), not property-access calls (`a.b()`) or any
191
+ // other identifier usage -- no scope analysis, no unused-detection.
192
+ references.push({ name: node.expression.text, kind: "call", line: lineOf(node) });
193
+ }
194
+ }
195
+ function visit(node) {
196
+ if (ts.isImportDeclaration(node)) {
197
+ handleImportDeclaration(node);
198
+ }
199
+ else if (ts.isExportDeclaration(node)) {
200
+ handleExportDeclaration(node);
201
+ }
202
+ else if (ts.isExportAssignment(node)) {
203
+ handleExportAssignment(node);
204
+ }
205
+ else if (ts.isFunctionDeclaration(node)) {
206
+ pushDeclaration(node, node.name?.text, "function");
207
+ }
208
+ else if (ts.isClassDeclaration(node)) {
209
+ pushDeclaration(node, node.name?.text, "class");
210
+ }
211
+ else if (ts.isInterfaceDeclaration(node)) {
212
+ pushDeclaration(node, node.name.text, "interface");
213
+ }
214
+ else if (ts.isTypeAliasDeclaration(node)) {
215
+ pushDeclaration(node, node.name.text, "type");
216
+ }
217
+ else if (ts.isEnumDeclaration(node)) {
218
+ pushDeclaration(node, node.name.text, "enum");
219
+ }
220
+ else if (ts.isVariableStatement(node)) {
221
+ handleVariableStatement(node);
222
+ }
223
+ else if (ts.isMethodDeclaration(node) && ts.isIdentifier(node.name)) {
224
+ declarations.push({ name: node.name.text, kind: "method", exported: false, line: lineOf(node) });
225
+ }
226
+ else if (ts.isCallExpression(node)) {
227
+ handleCallExpression(node);
228
+ }
229
+ else if (ts.isBinaryExpression(node) && node.operatorToken.kind === ts.SyntaxKind.EqualsToken) {
230
+ handleCommonJsAssignment(node);
231
+ }
232
+ ts.forEachChild(node, visit);
233
+ }
234
+ visit(sourceFile);
235
+ const parseStatus = diagnostics.length > 0 ? "parse-error" : "parsed";
236
+ return {
237
+ relativePath,
238
+ language,
239
+ role,
240
+ parseStatus,
241
+ analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
242
+ lineCount: inventoryEntry.lineCount,
243
+ imports,
244
+ exports,
245
+ declarations,
246
+ references,
247
+ diagnostics,
248
+ };
249
+ }
250
+ function resolveScriptKind(extension) {
251
+ switch (extension) {
252
+ case ".tsx":
253
+ return ts.ScriptKind.TSX;
254
+ case ".jsx":
255
+ return ts.ScriptKind.JSX;
256
+ case ".js":
257
+ case ".mjs":
258
+ case ".cjs":
259
+ return ts.ScriptKind.JS;
260
+ case ".ts":
261
+ case ".mts":
262
+ case ".cts":
263
+ default:
264
+ return ts.ScriptKind.TS;
265
+ }
266
+ }
267
+ function hasModifier(node, kind) {
268
+ if (!ts.canHaveModifiers(node))
269
+ return false;
270
+ const modifiers = ts.getModifiers(node);
271
+ return modifiers?.some((m) => m.kind === kind) ?? false;
272
+ }
273
+ // Reads syntax diagnostics directly off the already-built SourceFile rather
274
+ // than running a second full parse via ts.transpileModule(). `parseDiagnostics`
275
+ // is not part of the public TypeScript.d.ts surface, but it is populated by
276
+ // createSourceFile() itself (every syntax error the parser recovers from is
277
+ // recorded there) and has been stable across TypeScript versions for years --
278
+ // this is a widely-used pattern for exactly this "syntax diagnostics without
279
+ // a second parse/transpile pass" use case. Benchmarked at ~5x faster than
280
+ // transpileModule() over this repo's own ~375 TS/JS files, which matters
281
+ // because collectSourceFacts.ts calls this analyzer once per eligible file
282
+ // on every audit run (including this repo's own self-audit).
283
+ function getSyntaxDiagnostics(relativePath, sourceFile) {
284
+ const parseDiagnostics = sourceFile.parseDiagnostics ?? [];
285
+ return parseDiagnostics.map((diagnostic) => toSourceFactDiagnostic(relativePath, diagnostic));
286
+ }
287
+ function toSourceFactDiagnostic(relativePath, diagnostic) {
288
+ const message = ts.flattenDiagnosticMessageText(diagnostic.messageText, "\n");
289
+ const line = diagnostic.file && diagnostic.start !== undefined
290
+ ? diagnostic.file.getLineAndCharacterOfPosition(diagnostic.start).line + 1
291
+ : undefined;
292
+ return {
293
+ severity: diagnostic.category === ts.DiagnosticCategory.Warning ? "warning" : "error",
294
+ code: `TS${diagnostic.code}`,
295
+ message: line !== undefined ? `Line ${line}: ${message}` : message,
296
+ path: relativePath,
297
+ analyzerId: TYPESCRIPT_JAVASCRIPT_ANALYZER_ID,
298
+ };
299
+ }
@@ -22,6 +22,8 @@ export function summarizeInventory(inventory) {
22
22
  skippedFileCount: inventory.skippedFileCount,
23
23
  filesByCategory: inventory.filesByCategory,
24
24
  filesByExtension: inventory.filesByExtension,
25
+ filesByLanguage: inventory.filesByLanguage,
26
+ filesByRole: inventory.filesByRole,
25
27
  excludedDirectorySummary: inventory.excludedDirectorySummary,
26
28
  warnings: inventory.warnings,
27
29
  };
@@ -47,6 +49,15 @@ export function summarizeSourceOfTruth(sourceOfTruth) {
47
49
  warnings: sourceOfTruth.warnings,
48
50
  };
49
51
  }
52
+ export function summarizeSourceFacts(sourceFacts) {
53
+ return {
54
+ totalFilesAnalyzed: sourceFacts.files.length,
55
+ filesByLanguage: sourceFacts.filesByLanguage,
56
+ filesByParseStatus: sourceFacts.filesByParseStatus,
57
+ analyzerDiagnosticCount: sourceFacts.analyzerDiagnostics.length,
58
+ warnings: sourceFacts.warnings,
59
+ };
60
+ }
50
61
  const MAX_JSON_RECOMMENDATIONS = 200;
51
62
  const MAX_TEXT_RECOMMENDATIONS = 10;
52
63
  const SEVERITY_ORDER = ["blocker", "high", "medium", "low", "info"];
@@ -176,6 +187,7 @@ export function buildAuditReportModel(result, opts) {
176
187
  },
177
188
  inventory: summarizeInventory(result.inventory),
178
189
  sourceOfTruth: summarizeSourceOfTruth(result.sourceOfTruth),
190
+ sourceFacts: summarizeSourceFacts(result.sourceFacts),
179
191
  detectors,
180
192
  issues,
181
193
  skippedDetectors: result.skippedDetectors,
@@ -18,6 +18,7 @@ export function renderAuditJsonReport(model) {
18
18
  summary: model.summary,
19
19
  inventory: model.inventory,
20
20
  sourceOfTruth: model.sourceOfTruth,
21
+ sourceFacts: model.sourceFacts,
21
22
  detectors: model.detectors,
22
23
  issues: model.issues,
23
24
  skippedDetectors: model.skippedDetectors,
@@ -70,10 +70,20 @@ export function renderAuditTextReport(model) {
70
70
  lines.push(divider("-"));
71
71
  lines.push(` total=${model.inventory.totalFileCount} scanned=${model.inventory.totalScannedFileCount} skipped=${model.inventory.skippedFileCount}`);
72
72
  lines.push(` source=${model.inventory.filesByCategory.source} tests=${model.inventory.filesByCategory.tests} docs=${model.inventory.filesByCategory.docs} package=${model.inventory.filesByCategory.package} config=${model.inventory.filesByCategory.config} scripts=${model.inventory.filesByCategory.scripts} ci=${model.inventory.filesByCategory.ci} generated=${model.inventory.filesByCategory.generated} report=${model.inventory.filesByCategory.report} unknown=${model.inventory.filesByCategory.unknown}`);
73
+ lines.push(` languages: typescript=${model.inventory.filesByLanguage.typescript} javascript=${model.inventory.filesByLanguage.javascript} python=${model.inventory.filesByLanguage.python} java=${model.inventory.filesByLanguage.java} kotlin=${model.inventory.filesByLanguage.kotlin} json=${model.inventory.filesByLanguage.json} markdown=${model.inventory.filesByLanguage.markdown} yaml=${model.inventory.filesByLanguage.yaml} xml=${model.inventory.filesByLanguage.xml} toml=${model.inventory.filesByLanguage.toml} unknown=${model.inventory.filesByLanguage.unknown}`);
74
+ lines.push(` roles: source=${model.inventory.filesByRole.source} test=${model.inventory.filesByRole.test} docs=${model.inventory.filesByRole.docs} config=${model.inventory.filesByRole.config} package=${model.inventory.filesByRole.package} generated=${model.inventory.filesByRole.generated} build-output=${model.inventory.filesByRole["build-output"]} vendor=${model.inventory.filesByRole.vendor} report-output=${model.inventory.filesByRole["report-output"]} unknown=${model.inventory.filesByRole.unknown}`);
73
75
  if (model.inventory.warnings.length > 0) {
74
76
  lines.push(` warnings: ${model.inventory.warnings.length}`);
75
77
  }
76
78
  lines.push("");
79
+ lines.push("Source facts summary");
80
+ lines.push(divider("-"));
81
+ lines.push(` analyzed=${model.sourceFacts.totalFilesAnalyzed}`);
82
+ lines.push(` parsed=${model.sourceFacts.filesByParseStatus.parsed} file-level-only=${model.sourceFacts.filesByParseStatus["file-level-only"]} unsupported=${model.sourceFacts.filesByParseStatus.unsupported} parse-error=${model.sourceFacts.filesByParseStatus["parse-error"]} skipped=${model.sourceFacts.filesByParseStatus.skipped}`);
83
+ if (model.sourceFacts.analyzerDiagnosticCount > 0) {
84
+ lines.push(` analyzer diagnostics: ${model.sourceFacts.analyzerDiagnosticCount}`);
85
+ }
86
+ lines.push("");
77
87
  lines.push("Source-of-truth summary");
78
88
  lines.push(divider("-"));
79
89
  lines.push(` package: ${model.sourceOfTruth.packageName ?? "(none)"}@${model.sourceOfTruth.packageVersion ?? "?"}`);
@@ -153,12 +163,18 @@ function renderIssueBlock(issue) {
153
163
  lines.push(` severity=${issue.severity} confidence=${issue.confidence} category=${sanitizeLine(issue.category)}`);
154
164
  lines.push(` detector=${issue.detectorId} id=${issue.id}`);
155
165
  for (const evidence of issue.evidence.slice(0, MAX_EVIDENCE_PER_ISSUE)) {
166
+ // v0.3.1 Batch 5 -- an evidence entry's `message` (the human-readable
167
+ // "why") and `excerpt` (a quoted snippet/reference list) are
168
+ // independent fields on the same entry -- render both when both are
169
+ // present. Previously an `excerpt` silently suppressed `message`
170
+ // entirely, which dropped the only readable explanation for
171
+ // source-facts-derived findings (e.g. duplicateImplementationDetector's
172
+ // "Source facts: an exported ... was parsed in N distinct files.") and
173
+ // left only a bare file-path excerpt in text output.
174
+ lines.push(` > ${sanitizeLine(evidence.message)}`);
156
175
  if (evidence.excerpt) {
157
176
  lines.push(...renderQuotedExcerpt(evidence.excerpt, " "));
158
177
  }
159
- else {
160
- lines.push(` > ${sanitizeLine(evidence.message)}`);
161
- }
162
178
  }
163
179
  lines.push(` recommendedAction: ${sanitizeLine(issue.recommendedAction)}`);
164
180
  return lines;
package/package.json CHANGED
@@ -1,15 +1,16 @@
1
1
  {
2
2
  "name": "@dailephd/my-dev-kit-lab",
3
- "version": "0.3.0",
3
+ "version": "0.3.1",
4
4
  "type": "module",
5
5
  "description": "Evidence, benchmark, and evaluation companion for my-dev-kit.",
6
6
  "bin": {
7
7
  "my-dev-kit-lab": "dist/scripts/run-final-demo.js"
8
8
  },
9
- "files": [
10
- "dist/scripts/",
11
- "dist/src/",
12
- "benchmarks/",
9
+ "files": [
10
+ "dist/scripts/",
11
+ "!dist/scripts/docs/",
12
+ "dist/src/",
13
+ "benchmarks/",
13
14
  "docs/METRICS.md",
14
15
  "examples/",
15
16
  "tests/fixtures/"
@@ -63,6 +64,8 @@
63
64
  "experiment:list": "tsx scripts/experiments/listExperiments.ts",
64
65
  "experiment:describe": "tsx scripts/experiments/describeExperiment.ts",
65
66
  "experiment:run": "tsx scripts/experiments/runExperiment.ts",
67
+ "ecosystem:sync": "tsx scripts/docs/syncEcosystemDocs.ts",
68
+ "ecosystem:check": "tsx scripts/docs/checkEcosystemDocs.ts",
66
69
  "run-final-demo": "tsx scripts/run-final-demo.ts",
67
70
  "run-agent-prompt": "tsx scripts/run-agent-prompt.ts",
68
71
  "run-controlled-experiment": "tsx scripts/run-controlled-experiment.ts",