@shubh000/macrc 0.1.0 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -143,6 +143,25 @@ Compares two audit JSON files and reports regressions and improvements.
143
143
 
144
144
  ---
145
145
 
146
+ ## GitHub Code Scanning
147
+
148
+ MACRC outputs SARIF, which GitHub Code Scanning ingests natively:
149
+
150
+ ```yaml
151
+ - name: MACRC audit
152
+ run: npx @shubh000/macrc gate . --format sarif --out macrc.sarif || true
153
+
154
+ - name: Upload MACRC results
155
+ uses: github/codeql-action/upload-sarif@v3
156
+ with:
157
+ sarif_file: macrc.sarif
158
+ category: macrc
159
+ ```
160
+
161
+ Results appear in the **Security → Code scanning** tab. Use `|| true` if you want findings without blocking the workflow, or drop it to block on absent Tier-1 dims.
162
+
163
+ ---
164
+
146
165
  ## GitHub Action
147
166
 
148
167
  ```yaml
@@ -162,7 +181,7 @@ Full inputs: `directory`, `format`, `policy-dir`, `stack-category`, `llm`, `sari
162
181
 
163
182
  **Step 1: Heuristic pass (all 20 dims, deterministic)**
164
183
 
165
- Three signal extractors run per repo. For the 10 patterns with high false-positive risk (e.g. `zod_usage`, `bearer_auth`, `health_route`), hits are verified with a context-window analysis: ±10 surrounding lines must confirm the signal is at an actual call site, not a schema definition, test fixture, or comment.
184
+ Three signal extractors run per repo. For 10 patterns with high false-positive risk (e.g. `zod_usage`, `bearer_auth`, `health_route`), hits go through a context-window pass: ±10 surrounding lines are checked with rule-based string analysis to confirm the signal is at a real call site, not a schema definition, test fixture, or comment. This is not an AST parser — no tree-sitter or Babel bindings. It is a fast, dependency-free signal verifier (`context-verifier.mjs`).
166
185
 
167
186
  | Extractor | What it checks |
168
187
  |---|---|
@@ -172,11 +191,17 @@ Three signal extractors run per repo. For the 10 patterns with high false-positi
172
191
 
173
192
  **Step 2: Targeted LLM pass (Tier-1 dims only, `--llm` flag)**
174
193
 
175
- 8 dimensions have ≥25pp divergence between heuristic and LLM scores on the 231-repo calibration corpus. For these, a structured prompt is sent to `claude-haiku-4-5` with the actual route/controller code as context, only when the heuristic says `partial` or `present`.
194
+ 5 dimensions have ≥20pp divergence between heuristic and LLM scores on the 232-repo corpus (231 used in LLM calibration — 1 quarantined during ingest). For these, a structured prompt is sent to `claude-haiku-4-5` with the actual route/controller code as context, only when the heuristic says `partial` or `present`.
195
+
196
+ **Why verify `partial/present`, not `absent`?** The Tier-1 gap is a *false-negative* gap: the heuristic says a repo has `auth` or `input_validation` (partial or present) when the LLM finds it actually doesn't. The heuristic almost never calls a truly present dim absent — it errs on the side of optimism. So the false negatives sit entirely in the `partial/present` bucket. Sending the LLM only those repos cuts the verification cost by ~70% (roughly 30% of repos per Tier-1 dim have partial/present) while catching the cases that matter. Verifying `absent` repos would add cost with zero TPR gain.
176
197
 
177
198
  **Step 3: Patch generation (`fix` subcommand)**
178
199
 
179
- For each absent/partial Tier-1 dim, a language-aware patch builder finds the right insertion anchor and generates a minimal, syntactically safe patch: injecting middleware factories after the last import block, adding timeout config to HTTP client instances, replacing hardcoded `localhost` URLs with env var expressions, wiring a Redis session store.
200
+ For each absent/partial Tier-1 dim, a language-aware patch builder generates a minimal scaffolding patch: injecting middleware stubs after the last import block, adding timeout config to HTTP client calls, replacing hardcoded `localhost` URLs with env var expressions, wiring a Redis session store.
201
+
202
+ **Scope note:** patches target straightforward Express/FastAPI apps. They are starter scaffolding, not universal codemods — complex TypeScript generics, custom axios wrappers, or monorepo structures will need manual adjustment after apply.
203
+
204
+ **AST query roadmap:** Current acorn queries cover JS/TS only. Python equivalent via `python3 -m ast` subprocess (stdlib, no install) is the planned next addition — relevant for the AI/ML backend ecosystem where most AI-native services are FastAPI or Flask.
180
205
 
181
206
  ---
182
207
 
@@ -200,7 +225,7 @@ Gap = LLM absent% minus verified heuristic absent%. Tier-1 (≥20pp gap) means t
200
225
 
201
226
  Contextual verification closes large parts of the gap for health_checks (+56pp raw vs +3pp verified) and load_balancing_awareness (+55pp raw vs +2pp verified), confirming the verifier is doing real work. The remaining Tier-1 gaps reflect structural limits of static analysis.
202
227
 
203
- The full calibration corpus (231 public AI-native repos across 8 stack categories) is documented at the [MACRC Corpus dashboard](https://trinetra.life).
228
+ The full calibration corpus (232 public AI-native repos across 8 stack categories, 231 in the LLM calibration subset) is documented at the [MACRC Corpus dashboard](https://trinetra.life).
204
229
 
205
230
  ---
206
231
 
@@ -236,8 +261,15 @@ Default: all dims warn on absent, pass on partial or present.
236
261
  ## Tests
237
262
 
238
263
  ```bash
239
- node --test tests/ast-verifier.test.mjs tests/remediator.test.mjs
240
- # 24 tests, 0 failures
264
+ # Unit tests (fast, no network)
265
+ node --test tests/context-verifier.test.mjs tests/remediator.test.mjs tests/syntax-queries.test.mjs
266
+
267
+ # Integration tests (runs full pipeline on demo/bad-express-app)
268
+ node --test tests/integration.test.mjs
269
+
270
+ # All at once
271
+ node --test tests/*.test.mjs
272
+ # 43 tests, 0 failures
241
273
  ```
242
274
 
243
275
  ---
@@ -1,11 +1,11 @@
1
1
  /**
2
2
  * Contextual verification for HIGH-FP grep patterns.
3
3
  *
4
- * "AST" here means context-window analysis: for each regex match we look at
5
- * ±N surrounding lines to determine whether the hit is at a real call site
6
- * rather than in a schema definition, test fixture, comment, or README.
4
+ * For each regex match, reads ±N surrounding lines to determine whether the
5
+ * hit is at a real call site rather than a schema definition, test fixture,
6
+ * comment, or README. Rule-based string checks on the context window.
7
7
  *
8
- * This closes the wiring gap without requiring native Tree-sitter bindings.
8
+ * Not an AST parser: no tree-sitter or Babel. Fast and dependency-free.
9
9
  *
10
10
  * Exports:
11
11
  * HIGH_FP_KEYS set of pattern keys that need verification
@@ -10,7 +10,8 @@
10
10
  import { readFileSync, existsSync } from 'fs';
11
11
  import { resolve } from 'path';
12
12
  import { execSync } from 'child_process';
13
- import { grepSourcesVerified } from './ast-verifier.mjs';
13
+ import { grepSourcesVerified } from './context-verifier.mjs';
14
+ import { queryTimeoutHandling, queryDistributedAssumptions } from './syntax-queries.mjs';
14
15
 
15
16
  export const PENDING_DIM = { state: 'pending', severity: null, notes: '', snippets: [] };
16
17
  export const ABSENT_DIM = { state: 'absent', severity: null, notes: '', snippets: [] };
@@ -383,6 +384,8 @@ export function computeHeuristicSummary(layers) {
383
384
  * stackCategory: one of ai-web-app, api-proxy, mcp-server, agent-framework,
384
385
  * cli-tool, desktop-native, plugin-extension, skills-config, or ''
385
386
  */
387
+ const JS_TS_RE = /\.(js|ts|mjs|cjs|jsx|tsx)$/;
388
+
386
389
  export function runHeuristic(cloneDir, stackCategory = '', { verified = true } = {}) {
387
390
  const deps = parseDeps(cloneDir);
388
391
  // verified=true uses context-window analysis for HIGH-FP patterns,
@@ -391,5 +394,47 @@ export function runHeuristic(cloneDir, stackCategory = '', { verified = true } =
391
394
  ? grepSourcesVerified(cloneDir, GREP_PATTERNS)
392
395
  : grepSources(cloneDir);
393
396
  const configs = scanConfigs(cloneDir);
394
- return heuristicScore(deps, hits, configs, stackCategory);
397
+ const layers = heuristicScore(deps, hits, configs, stackCategory);
398
+
399
+ // AST refinement: run acorn syntax queries on JS/TS files for the two
400
+ // dimensions with the largest verified-heuristic gap (+52pp / +20pp).
401
+ // Only runs when verified=true; replaces the grep result when non-null.
402
+ if (verified) {
403
+ let sourceFiles = [];
404
+ try {
405
+ sourceFiles = execSync('git ls-files', { cwd: cloneDir, timeout: 10000 })
406
+ .toString().split('\n')
407
+ .filter(f => f && JS_TS_RE.test(f))
408
+ .map(f => resolve(cloneDir, f));
409
+ } catch {}
410
+
411
+ // timeout_handling — merge per-file AST results
412
+ const timeoutResults = sourceFiles.map(f => queryTimeoutHandling(f)).filter(Boolean);
413
+ if (timeoutResults.length > 0) {
414
+ const totalCalls = timeoutResults.reduce((s, r) => s + r.callsFound, 0);
415
+ const timedOutCalls = timeoutResults.reduce((s, r) => s + r.callsWithTimeout, 0);
416
+ if (timedOutCalls === 0) {
417
+ layers.reliability.timeout_handling = { state: 'absent', severity: 'high', notes: `AST: ${totalCalls} HTTP call(s) with no timeout config`, snippets: [] };
418
+ } else if (timedOutCalls < totalCalls) {
419
+ layers.reliability.timeout_handling = { state: 'partial', severity: 'high', notes: `AST: ${timedOutCalls}/${totalCalls} HTTP calls have timeout config`, snippets: [] };
420
+ } else {
421
+ layers.reliability.timeout_handling = { state: 'present', severity: null, notes: `AST: all ${totalCalls} HTTP call(s) have timeout config`, snippets: [] };
422
+ }
423
+ }
424
+
425
+ // distributed_assumptions — absent if any file has hardcoded localhost in a call arg
426
+ if (!isNA(stackCategory, 'distributed_assumptions')) {
427
+ const distResults = sourceFiles.map(f => queryDistributedAssumptions(f)).filter(Boolean);
428
+ if (distResults.length > 0) {
429
+ const urls = distResults.flatMap(r => r.hardcodedUrls);
430
+ layers.scale_contract.distributed_assumptions = {
431
+ state: 'absent', severity: 'high',
432
+ notes: `AST: hardcoded localhost URL(s) in call arguments: ${[...new Set(urls)].slice(0, 3).join(', ')}`,
433
+ snippets: [],
434
+ };
435
+ }
436
+ }
437
+ }
438
+
439
+ return layers;
395
440
  }
@@ -0,0 +1,169 @@
1
+ /**
2
+ * Syntax-tree queries for Tier-1 dimensions with high grep false-positive rates.
3
+ *
4
+ * Uses acorn (pure-JS ES2022 parser, no native deps) for JS/TS files.
5
+ * Returns null when the file is not parseable or has no relevant calls —
6
+ * callers should fall back to the grep result in that case.
7
+ *
8
+ * Exported functions:
9
+ * queryTimeoutHandling(filePath) → { state, callsFound, callsWithTimeout } | null
10
+ * queryDistributedAssumptions(filePath) → { state, hardcodedUrls } | null
11
+ */
12
+
13
+ import { readFileSync } from 'fs';
14
+ import { createRequire } from 'module';
15
+
16
+ // acorn is a CJS package — load via createRequire
17
+ const require = createRequire(import.meta.url);
18
+ const { parse } = require('acorn');
19
+
20
+ const PARSE_OPTS = { ecmaVersion: 2022, sourceType: 'module' };
21
+
22
+ // ── AST walker — depth-first ───────────────────────────────────────────────────
23
+
24
+ function walk(node, visit, parent = null) {
25
+ if (!node || typeof node !== 'object' || !node.type) return;
26
+ visit(node, parent);
27
+ for (const key of Object.keys(node)) {
28
+ const child = node[key];
29
+ if (Array.isArray(child)) {
30
+ child.forEach(c => walk(c, visit, node));
31
+ } else if (child && typeof child === 'object' && child.type) {
32
+ walk(child, visit, node);
33
+ }
34
+ }
35
+ }
36
+
37
+ // Return callee name as a string, e.g. "axios.get" or "fetch"
38
+ function getCalleeName(callee) {
39
+ if (!callee) return '';
40
+ if (callee.type === 'Identifier') return callee.name;
41
+ if (callee.type === 'MemberExpression') {
42
+ const obj = getCalleeName(callee.object);
43
+ const prop = callee.computed ? '' : (callee.property?.name ?? '');
44
+ return prop ? `${obj}.${prop}` : obj;
45
+ }
46
+ return '';
47
+ }
48
+
49
+ // Known HTTP client call patterns that accept a config object with timeout/signal
50
+ const HTTP_CALLEE_RE = /^(axios\.(get|post|put|delete|patch|request)|fetch)$/;
51
+
52
+ // Keys that signal timeout configuration in an options object
53
+ const TIMEOUT_KEYS = new Set(['timeout', 'signal']);
54
+
55
+ function hasTimeoutOption(args) {
56
+ return args.some(arg =>
57
+ arg.type === 'ObjectExpression' &&
58
+ arg.properties.some(p => {
59
+ const key = p.key?.name ?? p.key?.value ?? '';
60
+ return TIMEOUT_KEYS.has(key);
61
+ })
62
+ );
63
+ }
64
+
65
+ // ── queryTimeoutHandling ───────────────────────────────────────────────────────
66
+
67
+ /**
68
+ * Parse a JS/TS file and check whether axios/fetch calls have timeout config.
69
+ *
70
+ * Returns:
71
+ * null – file not parseable or no HTTP calls found
72
+ * { state: 'absent', callsFound, callsWithTimeout }
73
+ * { state: 'partial', callsFound, callsWithTimeout }
74
+ * { state: 'present', callsFound, callsWithTimeout }
75
+ */
76
+ export function queryTimeoutHandling(filePath) {
77
+ let src;
78
+ try { src = readFileSync(filePath, 'utf8'); } catch { return null; }
79
+
80
+ let ast;
81
+ try { ast = parse(src, PARSE_OPTS); } catch {
82
+ // Try script mode as fallback (CJS files)
83
+ try { ast = parse(src, { ...PARSE_OPTS, sourceType: 'script' }); } catch { return null; }
84
+ }
85
+
86
+ let callsFound = 0;
87
+ let callsWithTimeout = 0;
88
+
89
+ walk(ast, node => {
90
+ if (node.type !== 'CallExpression') return;
91
+ const name = getCalleeName(node.callee);
92
+ if (!HTTP_CALLEE_RE.test(name)) return;
93
+ callsFound++;
94
+ if (hasTimeoutOption(node.arguments)) callsWithTimeout++;
95
+ });
96
+
97
+ if (callsFound === 0) return null;
98
+ if (callsWithTimeout === 0) return { state: 'absent', callsFound, callsWithTimeout };
99
+ if (callsWithTimeout < callsFound) return { state: 'partial', callsFound, callsWithTimeout };
100
+ return { state: 'present', callsFound, callsWithTimeout };
101
+ }
102
+
103
+ // ── queryDistributedAssumptions ────────────────────────────────────────────────
104
+
105
+ const LOCALHOST_RE = /^https?:\/\/(localhost|127\.0\.0\.1)(:\d+)?/;
106
+
107
+ // Test-block callee names that make a localhost string safe to ignore
108
+ const TEST_CALLEE_NAMES = new Set(['describe', 'it', 'test', 'beforeEach', 'afterEach', 'before', 'after', 'beforeAll', 'afterAll']);
109
+
110
+ /**
111
+ * Parse a JS/TS file and check whether localhost string literals appear
112
+ * as call arguments outside of test blocks.
113
+ *
114
+ * Returns:
115
+ * null – not parseable or no localhost in call args
116
+ * { state: 'absent', hardcodedUrls }
117
+ */
118
+ export function queryDistributedAssumptions(filePath) {
119
+ let src;
120
+ try { src = readFileSync(filePath, 'utf8'); } catch { return null; }
121
+
122
+ let ast;
123
+ try { ast = parse(src, PARSE_OPTS); } catch {
124
+ try { ast = parse(src, { ...PARSE_OPTS, sourceType: 'script' }); } catch { return null; }
125
+ }
126
+
127
+ // Build parent map so we can walk ancestors
128
+ const parentMap = new Map();
129
+ walk(ast, (node, parent) => { if (parent) parentMap.set(node, parent); });
130
+
131
+ function isInsideTestBlock(node) {
132
+ let cur = parentMap.get(node);
133
+ while (cur) {
134
+ if (cur.type === 'CallExpression') {
135
+ const name = getCalleeName(cur.callee);
136
+ if (TEST_CALLEE_NAMES.has(name)) return true;
137
+ }
138
+ cur = parentMap.get(cur);
139
+ }
140
+ return false;
141
+ }
142
+
143
+ function isCallArgument(node) {
144
+ const parent = parentMap.get(node);
145
+ if (!parent) return false;
146
+ if (parent.type === 'CallExpression') {
147
+ return parent.arguments.includes(node);
148
+ }
149
+ // Template literal expression inside a call
150
+ if (parent.type === 'TemplateLiteral') {
151
+ const grandparent = parentMap.get(parent);
152
+ if (grandparent?.type === 'CallExpression' && grandparent.arguments.includes(parent)) return true;
153
+ }
154
+ return false;
155
+ }
156
+
157
+ const hardcodedUrls = [];
158
+
159
+ walk(ast, node => {
160
+ if (node.type !== 'Literal' || typeof node.value !== 'string') return;
161
+ if (!LOCALHOST_RE.test(node.value)) return;
162
+ if (isInsideTestBlock(node)) return;
163
+ if (!isCallArgument(node)) return;
164
+ hardcodedUrls.push(node.value);
165
+ });
166
+
167
+ if (hardcodedUrls.length === 0) return null;
168
+ return { state: 'absent', hardcodedUrls };
169
+ }
package/package.json CHANGED
@@ -1,9 +1,11 @@
1
1
  {
2
2
  "name": "@shubh000/macrc",
3
- "version": "0.1.0",
3
+ "version": "1.0.0",
4
4
  "description": "Structural audit CLI for AI-generated code: 20 dimensions across Security, Reliability, Observability, and Scale Contract",
5
5
  "type": "module",
6
- "bin": "cli/macrc.js",
6
+ "bin": {
7
+ "macrc": "cli/macrc.mjs"
8
+ },
7
9
  "files": [
8
10
  "cli/",
9
11
  "demo/",
@@ -16,6 +18,9 @@
16
18
  "scripts": {
17
19
  "test": "node --test tests/*.test.mjs"
18
20
  },
21
+ "dependencies": {
22
+ "acorn": "^8.11.0"
23
+ },
19
24
  "keywords": [
20
25
  "audit",
21
26
  "ai",
@@ -1,10 +1,10 @@
1
1
  /**
2
- * Unit tests for cli/lib/ast-verifier.mjs
3
- * Run: node --test tests/ast-verifier.test.mjs
2
+ * Unit tests for cli/lib/context-verifier.mjs
3
+ * Run: node --test tests/context-verifier.test.mjs
4
4
  */
5
5
  import { test } from 'node:test';
6
6
  import assert from 'node:assert/strict';
7
- import { verifyMatch } from '../cli/lib/ast-verifier.mjs';
7
+ import { verifyMatch } from '../cli/lib/context-verifier.mjs';
8
8
 
9
9
  // ── zod_usage ─────────────────────────────────────────────────────────────────
10
10
 
@@ -0,0 +1,36 @@
1
+ {
2
+ "schema_version": "1.0",
3
+ "repo_id": "fixture",
4
+ "audited_at": "2026-01-02",
5
+ "auditor": "macrc-heuristic",
6
+ "layers": {
7
+ "security": {
8
+ "auth": { "state": "present", "severity": null, "notes": "fixture: auth wired", "snippets": [] },
9
+ "input_validation": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
10
+ "transport_security": { "state": "absent", "severity": null, "notes": "fixture", "snippets": [] },
11
+ "dependency_hygiene": { "state": "pending", "severity": null, "notes": "fixture", "snippets": [] },
12
+ "secrets_management": { "state": "pending", "severity": null, "notes": "fixture", "snippets": [] }
13
+ },
14
+ "reliability": {
15
+ "error_handling": { "state": "absent", "severity": "critical", "notes": "fixture", "snippets": [] },
16
+ "retry_logic": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
17
+ "circuit_breaking": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
18
+ "graceful_degradation": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
19
+ "timeout_handling": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] }
20
+ },
21
+ "observability": {
22
+ "logging": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
23
+ "metrics_emission": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
24
+ "tracing": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
25
+ "alerting": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
26
+ "health_checks": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] }
27
+ },
28
+ "scale_contract": {
29
+ "statelessness": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
30
+ "load_balancing_awareness": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
31
+ "distributed_assumptions": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
32
+ "rate_limiting": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
33
+ "caching_strategy": { "state": "absent", "severity": "low", "notes": "fixture", "snippets": [] }
34
+ }
35
+ }
36
+ }
@@ -0,0 +1,36 @@
1
+ {
2
+ "schema_version": "1.0",
3
+ "repo_id": "fixture",
4
+ "audited_at": "2026-01-01",
5
+ "auditor": "macrc-heuristic",
6
+ "layers": {
7
+ "security": {
8
+ "auth": { "state": "absent", "severity": "high", "notes": "fixture: no auth", "snippets": [] },
9
+ "input_validation": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
10
+ "transport_security": { "state": "absent", "severity": null, "notes": "fixture", "snippets": [] },
11
+ "dependency_hygiene": { "state": "pending", "severity": null, "notes": "fixture", "snippets": [] },
12
+ "secrets_management": { "state": "pending", "severity": null, "notes": "fixture", "snippets": [] }
13
+ },
14
+ "reliability": {
15
+ "error_handling": { "state": "absent", "severity": "critical", "notes": "fixture", "snippets": [] },
16
+ "retry_logic": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
17
+ "circuit_breaking": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
18
+ "graceful_degradation": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
19
+ "timeout_handling": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] }
20
+ },
21
+ "observability": {
22
+ "logging": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
23
+ "metrics_emission": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
24
+ "tracing": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
25
+ "alerting": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
26
+ "health_checks": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] }
27
+ },
28
+ "scale_contract": {
29
+ "statelessness": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
30
+ "load_balancing_awareness": { "state": "absent", "severity": "medium", "notes": "fixture", "snippets": [] },
31
+ "distributed_assumptions": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
32
+ "rate_limiting": { "state": "absent", "severity": "high", "notes": "fixture", "snippets": [] },
33
+ "caching_strategy": { "state": "absent", "severity": "low", "notes": "fixture", "snippets": [] }
34
+ }
35
+ }
36
+ }
@@ -0,0 +1,82 @@
1
+ import { test } from 'node:test';
2
+ import assert from 'node:assert/strict';
3
+ import { resolve } from 'path';
4
+ import { spawnSync } from 'child_process';
5
+ import { fileURLToPath } from 'url';
6
+ import { runHeuristic } from '../cli/lib/heuristic.mjs';
7
+ import { buildPatches } from '../cli/lib/remediator.mjs';
8
+
9
+ const __dir = fileURLToPath(new URL('.', import.meta.url));
10
+ const DEMO = resolve(__dir, '../demo/bad-express-app');
11
+
12
+ // ── Absent-dim assertions ──────────────────────────────────────────────────────
13
+ // bad-express-app:
14
+ // no auth packages/patterns → secEnhancement.auth absent
15
+ // no validation packages/patterns → secEnhancement.input_validation absent
16
+ // no retry packages/patterns → reliability.retry_logic absent
17
+ // no timeout patterns → reliability.timeout_handling absent
18
+ // no health route → observability.health_checks absent
19
+ // express-session, no redis → scale_contract.statelessness absent
20
+ // localhost URLs in api.js → scale_contract.distributed_assumptions absent
21
+ // no rate-limit packages → scale_contract.rate_limiting absent
22
+
23
+ const ABSENT = [
24
+ ['secEnhancement', 'auth'],
25
+ ['secEnhancement', 'input_validation'],
26
+ ['reliability', 'retry_logic'],
27
+ ['reliability', 'timeout_handling'],
28
+ ['observability', 'health_checks'],
29
+ ['scale_contract', 'statelessness'],
30
+ ['scale_contract', 'distributed_assumptions'],
31
+ ['scale_contract', 'rate_limiting'],
32
+ ];
33
+
34
+ // Run once; reuse across all tests
35
+ const heuristicResult = runHeuristic(DEMO, 'ai-web-app', { verified: true });
36
+
37
+ for (const [layer, dim] of ABSENT) {
38
+ test(`${layer}.${dim} is absent in bad-express-app`, () => {
39
+ const dimResult = heuristicResult[layer]?.[dim];
40
+ // secEnhancement only records present/partial — undefined means absent
41
+ const state = dimResult?.state ?? 'absent';
42
+ assert.equal(state, 'absent', `expected absent, got "${state}"`);
43
+ });
44
+ }
45
+
46
+ // ── Patch count test ───────────────────────────────────────────────────────────
47
+ // buildPatches expects auditResult.layers with 'security' (not 'secEnhancement')
48
+
49
+ test('buildPatches produces ≥7 patches on bad-express-app audit', () => {
50
+ const security = {
51
+ auth: heuristicResult.secEnhancement.auth || { state: 'absent', severity: null, notes: '', snippets: [] },
52
+ input_validation: heuristicResult.secEnhancement.input_validation || { state: 'absent', severity: null, notes: '', snippets: [] },
53
+ transport_security: heuristicResult.secEnhancement.transport_security || { state: 'absent', severity: null, notes: '', snippets: [] },
54
+ dependency_hygiene: { state: 'pending', severity: null, notes: '', snippets: [] },
55
+ secrets_management: { state: 'pending', severity: null, notes: '', snippets: [] },
56
+ };
57
+ const auditResult = {
58
+ layers: {
59
+ security,
60
+ reliability: heuristicResult.reliability,
61
+ observability: heuristicResult.observability,
62
+ scale_contract: heuristicResult.scale_contract,
63
+ },
64
+ };
65
+ const patches = buildPatches(auditResult, DEMO);
66
+ assert.ok(patches.length >= 7, `expected ≥7 patches, got ${patches.length}: ${patches.map(p => p._dim).join(', ')}`);
67
+ });
68
+
69
+ // ── Diff regression test ───────────────────────────────────────────────────────
70
+
71
+ test('diff detects auth absent→present as IMPROVED', () => {
72
+ const FIX_BEFORE = resolve(__dir, 'fixtures/diff-before.json');
73
+ const FIX_AFTER = resolve(__dir, 'fixtures/diff-after.json');
74
+ const MACRC = resolve(__dir, '../cli/macrc.mjs');
75
+
76
+ const r = spawnSync(process.execPath, [MACRC, 'diff', FIX_BEFORE, FIX_AFTER], { encoding: 'utf8' });
77
+ assert.equal(r.status, 0, `CLI exited ${r.status}: ${r.stderr}`);
78
+ assert.ok(
79
+ r.stdout.includes('IMPROVED') && r.stdout.includes('auth'),
80
+ `expected "IMPROVED" and "auth" in diff output, got:\n${r.stdout}`
81
+ );
82
+ });
@@ -0,0 +1,98 @@
1
+ import { test } from 'node:test';
2
+ import assert from 'node:assert/strict';
3
+ import { writeFileSync, unlinkSync } from 'fs';
4
+ import { resolve } from 'path';
5
+ import { fileURLToPath } from 'url';
6
+ import { queryTimeoutHandling, queryDistributedAssumptions } from '../cli/lib/syntax-queries.mjs';
7
+
8
+ const TMP_DIR = fileURLToPath(new URL('.', import.meta.url));
9
+
10
+ function tmp(name, src) {
11
+ const p = resolve(TMP_DIR, `_tmp_sq_${name}.js`);
12
+ writeFileSync(p, src);
13
+ return p;
14
+ }
15
+ function rm(p) { try { unlinkSync(p); } catch {} }
16
+
17
+ // ── queryTimeoutHandling ───────────────────────────────────────────────────────
18
+
19
+ test('timeout_handling: absent when axios.get has no timeout', () => {
20
+ const p = tmp('t1', `import axios from 'axios';\naxios.get('http://api.example.com/data');`);
21
+ const r = queryTimeoutHandling(p);
22
+ rm(p);
23
+ assert.equal(r?.state, 'absent');
24
+ assert.equal(r?.callsFound, 1);
25
+ assert.equal(r?.callsWithTimeout, 0);
26
+ });
27
+
28
+ test('timeout_handling: present when axios.get has { timeout }', () => {
29
+ const p = tmp('t2', `import axios from 'axios';\naxios.get('http://api.example.com/data', { timeout: 5000 });`);
30
+ const r = queryTimeoutHandling(p);
31
+ rm(p);
32
+ assert.equal(r?.state, 'present');
33
+ assert.equal(r?.callsWithTimeout, 1);
34
+ });
35
+
36
+ test('timeout_handling: present when fetch has AbortSignal in { signal }', () => {
37
+ const p = tmp('t3', `fetch('http://api.example.com', { signal: AbortSignal.timeout(5000) });`);
38
+ const r = queryTimeoutHandling(p);
39
+ rm(p);
40
+ assert.equal(r?.state, 'present');
41
+ });
42
+
43
+ test('timeout_handling: null for file with no HTTP calls', () => {
44
+ const p = tmp('t4', `const x = 1 + 2;\nconsole.log(x);`);
45
+ const r = queryTimeoutHandling(p);
46
+ rm(p);
47
+ assert.equal(r, null);
48
+ });
49
+
50
+ test('timeout_handling: partial when mixed (some with, some without)', () => {
51
+ const p = tmp('t5', [
52
+ `import axios from 'axios';`,
53
+ `axios.get('/a');`,
54
+ `axios.get('/b', { timeout: 3000 });`,
55
+ ].join('\n'));
56
+ const r = queryTimeoutHandling(p);
57
+ rm(p);
58
+ assert.equal(r?.state, 'partial');
59
+ assert.equal(r?.callsFound, 2);
60
+ assert.equal(r?.callsWithTimeout, 1);
61
+ });
62
+
63
+ // ── queryDistributedAssumptions ────────────────────────────────────────────────
64
+
65
+ test('distributed_assumptions: absent when localhost in call argument', () => {
66
+ const p = tmp('d1', `fetch('http://localhost:3000/api');`);
67
+ const r = queryDistributedAssumptions(p);
68
+ rm(p);
69
+ assert.equal(r?.state, 'absent');
70
+ assert.ok(r?.hardcodedUrls.length >= 1);
71
+ });
72
+
73
+ test('distributed_assumptions: null when localhost not in call argument', () => {
74
+ const p = tmp('d2', `const URL = 'http://localhost:3000';`);
75
+ const r = queryDistributedAssumptions(p);
76
+ rm(p);
77
+ assert.equal(r, null);
78
+ });
79
+
80
+ test('distributed_assumptions: null when localhost in test block', () => {
81
+ const p = tmp('d3', [
82
+ `describe('test', () => {`,
83
+ ` it('calls local', () => {`,
84
+ ` fetch('http://localhost:3000/api');`,
85
+ ` });`,
86
+ `});`,
87
+ ].join('\n'));
88
+ const r = queryDistributedAssumptions(p);
89
+ rm(p);
90
+ assert.equal(r, null);
91
+ });
92
+
93
+ test('distributed_assumptions: absent for 127.0.0.1 in call argument', () => {
94
+ const p = tmp('d4', `axios.get('http://127.0.0.1:8080/health');`);
95
+ const r = queryDistributedAssumptions(p);
96
+ rm(p);
97
+ assert.equal(r?.state, 'absent');
98
+ });