@shubh000/macrc 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,247 @@
1
- # Temporary Holding Version
1
+ # MACRC: Multi-layer AI Component Readiness Checklist
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ **AI coding agents ship PRs without timeouts, circuit breakers, or input validation. By the time a human reviews it, the code is merged. MACRC catches it at the diff.**
4
+
5
+ MACRC audits any codebase across 20 structural dimensions in four layers (Security, Reliability, Observability, Scale Contract) and optionally generates patches to fix what's absent.
6
+
7
+ ```
8
+ $ macrc audit ./my-service
9
+
10
+ MARC Audit: ./my-service [macrc-heuristic] 2026-11-16
11
+ ──────────────────────────────────────────────────────────────────────
12
+
13
+ Security
14
+ ✗ auth no auth signals found
15
+ ✗ input_validation no validation signals found
16
+ ✗ transport_security no TLS config found
17
+
18
+ Reliability
19
+ ✗ retry_logic [high] No retry logic found
20
+ ✗ timeout_handling [high] No timeout handling found
21
+
22
+ Observability
23
+ ✗ health_checks [medium] No health check endpoint found
24
+
25
+ Scale Contract
26
+ ✗ statelessness [high] In-process session store, not horizontally scalable
27
+
28
+ ──────────────────────────────────────────────────────────────────────
29
+ 0 present 1 partial 17 absent 0 n/a (18 applicable)
30
+ ⚠ Critical severity dimension(s) present
31
+ ```
32
+
33
+ ---
34
+
35
+ ## Install
36
+
37
+ ```bash
38
+ npm install -g macrc
39
+ # or run without installing:
40
+ npx macrc audit .
41
+ ```
42
+
43
+ **Requirements:** Node.js ≥ 18
44
+
45
+ ---
46
+
47
+ ## Quick start
48
+
49
+ ```bash
50
+ # 1. Audit: see what's missing
51
+ macrc audit ./my-api
52
+
53
+ # 2. Gate: block CI on policy violations (exits non-zero)
54
+ macrc gate ./my-api
55
+
56
+ # 3. Fix: preview patches
57
+ macrc fix ./my-api --dry-run
58
+
59
+ # 4. Fix: apply patches
60
+ macrc fix ./my-api
61
+
62
+ # 5. Diff: compare two audit snapshots
63
+ macrc diff before.json after.json
64
+ ```
65
+
66
+ ---
67
+
68
+ ## Demo
69
+
70
+ The `demo/bad-express-app/` directory is an intentionally under-hardened Express app. Run it to see the full pipeline:
71
+
72
+ ```bash
73
+ # Audit: 17 of 18 dims absent
74
+ macrc audit demo/bad-express-app --no-color
75
+
76
+ # Fix dry-run: 9 patches across 7 Tier-1 dims
77
+ macrc fix demo/bad-express-app --dry-run --no-color
78
+ ```
79
+
80
+ **Fix output:**
81
+ ```
82
+ MARC Remediator patches
83
+ ──────────────────────────────────────────────────
84
+ ✓ auth → src/routes/users.js (would-apply)
85
+ note: Wire it: router.use(requireAuth) on protected route groups
86
+ ✓ input_validation → src/routes/users.js (would-apply)
87
+ note: Add validateBody(YourSchema) as middleware before each POST/PUT/PATCH handler
88
+ ✓ retry_logic → package.json (would-apply)
89
+ ✓ retry_logic → src/services/api.js (would-apply)
90
+ ✓ timeout_handling → src/services/api.js (would-apply)
91
+ note: Replace axios.get/post/... with http.get/post/... in api.js
92
+ ✓ health_checks → src/index.js (would-apply)
93
+ ✓ statelessness → src/index.js (would-apply)
94
+ note: npm install connect-redis ioredis, then apply this patch
95
+ ✓ statelessness → src/index.js (would-apply)
96
+ ✓ load_balancing → src/index.js (would-apply)
97
+ ──────────────────────────────────────────────────
98
+ 9 applied 0 manual 0 skipped 0 errors
99
+ ```
100
+
101
+ ---
102
+
103
+ ## Subcommands
104
+
105
+ ### `macrc audit <dir>`
106
+
107
+ Runs the full heuristic scan plus optional targeted LLM verification on Tier-1 dimensions.
108
+
109
+ ```bash
110
+ macrc audit ./my-api
111
+ macrc audit ./my-api --format json > audit.json
112
+ macrc audit ./my-api --llm # targeted LLM on Tier-1 dims (needs ANTHROPIC_API_KEY)
113
+ macrc audit ./my-api --stack-category mcp-server # explicit stack override
114
+ macrc audit ./my-api --no-verify # skip contextual grep (use raw regex counts)
115
+ ```
116
+
117
+ ### `macrc gate <dir>`
118
+
119
+ Like `audit` but exits non-zero if policy blocks. Use in CI to prevent merging absent Tier-1 dims.
120
+
121
+ ```bash
122
+ macrc gate ./my-api --format sarif --out results.sarif
123
+ ```
124
+
125
+ ### `macrc fix <dir>`
126
+
127
+ Generates and applies deterministic patches for absent/partial Tier-1 dimensions. Language-aware (TypeScript, JavaScript, Python).
128
+
129
+ ```bash
130
+ macrc fix ./my-api --dry-run # preview only
131
+ macrc fix ./my-api --dims timeout_handling,auth # patch specific dims
132
+ macrc fix ./my-api --from audit.json # skip re-scan, use saved audit
133
+ ```
134
+
135
+ ### `macrc diff <before.json> <after.json>`
136
+
137
+ Compares two audit JSON files and reports regressions and improvements.
138
+
139
+ ```
140
+ ↑ IMPROVED reliability.retry_logic absent → present
141
+ ↓ REGRESSED security.auth present → absent
142
+ ```
143
+
144
+ ---
145
+
146
+ ## GitHub Action
147
+
148
+ ```yaml
149
+ - name: MACRC gate
150
+ uses: trinetra/macrc@v1
151
+ with:
152
+ directory: .
153
+ fail-on-block: true
154
+ llm: false # set true + ANTHROPIC_API_KEY for Tier-1 LLM verification
155
+ ```
156
+
157
+ Full inputs: `directory`, `format`, `policy-dir`, `stack-category`, `llm`, `sarif-upload`, `fail-on-block`.
158
+
159
+ ---
160
+
161
+ ## How it works
162
+
163
+ **Step 1: Heuristic pass (all 20 dims, deterministic)**
164
+
165
+ Three signal extractors run per repo. For the 10 patterns with high false-positive risk (e.g. `zod_usage`, `bearer_auth`, `health_route`), hits are verified with a context-window analysis: ±10 surrounding lines must confirm the signal is at an actual call site, not a schema definition, test fixture, or comment.
166
+
167
+ | Extractor | What it checks |
168
+ |---|---|
169
+ | `parseDeps` | 5 package-manager manifest formats |
170
+ | `grepSources` | 32 regex patterns across ≤400 source files |
171
+ | `scanConfigs` | 12 infra config file types (k8s, docker-compose, Dockerfile, nginx…) |
172
+
173
+ **Step 2: Targeted LLM pass (Tier-1 dims only, `--llm` flag)**
174
+
175
+ 8 dimensions have ≥25pp divergence between heuristic and LLM scores on the 231-repo calibration corpus. For these, a structured prompt is sent to `claude-haiku-4-5` with the actual route/controller code as context, only when the heuristic says `partial` or `present`.
176
+
177
+ **Step 3: Patch generation (`fix` subcommand)**
178
+
179
+ For each absent/partial Tier-1 dim, a language-aware patch builder finds the right insertion anchor and generates a minimal, syntactically safe patch: injecting middleware factories after the last import block, adding timeout config to HTTP client instances, replacing hardcoded `localhost` URLs with env var expressions, wiring a Redis session store.
180
+
181
+ ---
182
+
183
+ ## Divergence map: why Tier-1 matters
184
+
185
+ Running both the heuristic and LLM auditor on the same 231 repos reveals which dimensions the heuristic systematically underestimates:
186
+
187
+ | Dimension | LLM absent% | Heuristic (raw)% | Heuristic (verified)% | Gap (verified) | Tier |
188
+ |---|---|---|---|---|---|
189
+ | input_validation | 100% | 29% | 45% | +55pp | **1** |
190
+ | distributed_assumptions | 85% | 13% | 33% | +52pp | **1** |
191
+ | auth | 100% | 51% | 54% | +46pp | **1** |
192
+ | statelessness | 36% | 1% | 1% | +35pp | **1** |
193
+ | timeout_handling | 70% | 35% | 50% | +20pp | **1** |
194
+ | retry_logic | 92% | 66% | 80% | +12pp | 2 |
195
+ | logging | 53% | 35% | 35% | +18pp | 2 |
196
+ | health_checks | 89% | 33% | 86% | +3pp | 2 |
197
+ | load_balancing_awareness | 90% | 35% | 88% | +2pp | 2 |
198
+
199
+ Gap = LLM absent% minus verified heuristic absent%. Tier-1 (≥20pp gap) means the heuristic alone misses too many real absences to be reliable. Columns use verified mode (default). Raw mode is shown for reference.
200
+
201
+ Contextual verification closes large parts of the gap for health_checks (+56pp raw vs +3pp verified) and load_balancing_awareness (+55pp raw vs +2pp verified), confirming the verifier is doing real work. The remaining Tier-1 gaps reflect structural limits of static analysis.
202
+
203
+ The full calibration corpus (231 public AI-native repos across 8 stack categories) is documented at the [MACRC Corpus dashboard](https://trinetra.life).
204
+
205
+ ---
206
+
207
+ ## Stack categories and N/A classification
208
+
209
+ | Category | N/A dims | Count |
210
+ |---|---|---|
211
+ | `plugin-extension` | 8 (all network dims) | 25 repos |
212
+ | `skills-config` | 8 (all network dims) | 25 repos |
213
+ | `desktop-native` | 8 (all network dims) | 24 repos |
214
+ | `cli-tool` | 7 (network dims except distributed_assumptions) | 24 repos |
215
+
216
+ Dims classified N/A are excluded from the heuristic denominator. LLM mode scores all dims without N/A.
217
+
218
+ ---
219
+
220
+ ## Policy
221
+
222
+ Place a `.macrc.yml` in your repo root to customise gate behaviour:
223
+
224
+ ```yaml
225
+ policy:
226
+ security.auth: block # fail gate if absent
227
+ security.input_validation: block
228
+ reliability.timeout_handling: warn
229
+ observability.health_checks: warn
230
+ ```
231
+
232
+ Default: all dims warn on absent, pass on partial or present.
233
+
234
+ ---
235
+
236
+ ## Tests
237
+
238
+ ```bash
239
+ node --test tests/ast-verifier.test.mjs tests/remediator.test.mjs
240
+ # 24 tests, 0 failures
241
+ ```
242
+
243
+ ---
244
+
245
+ ## License
246
+
247
+ MIT
@@ -0,0 +1,247 @@
1
+ /**
2
+ * Contextual verification for HIGH-FP grep patterns.
3
+ *
4
+ * "AST" here means context-window analysis: for each regex match we look at
5
+ * ±N surrounding lines to determine whether the hit is at a real call site
6
+ * rather than in a schema definition, test fixture, comment, or README.
7
+ *
8
+ * This closes the wiring gap without requiring native Tree-sitter bindings.
9
+ *
10
+ * Exports:
11
+ * HIGH_FP_KEYS set of pattern keys that need verification
12
+ * verifyMatch (key, line, contextLines[]) returns 'confirmed'|'probable'|'unlikely'
13
+ * grepSourcesVerified verified drop-in for grepSources
14
+ */
15
+
16
+ import { readFileSync, readdirSync } from 'fs';
17
+ import { resolve } from 'path';
18
+ import { execSync } from 'child_process';
19
+
20
+ export const HIGH_FP_KEYS = new Set([
21
+ 'retry_pattern',
22
+ 'fallback_pattern',
23
+ 'feature_flag',
24
+ 'timeout_config',
25
+ 'py_timeout',
26
+ 'health_route',
27
+ 'bearer_auth',
28
+ 'env_service_url',
29
+ 'zod_usage',
30
+ 'pydantic_usage',
31
+ ]);
32
+
33
+ // ── Per-pattern context verifiers ─────────────────────────────────────────────
34
+ // Each receives:
35
+ // line: the matched line text
36
+ // context: surrounding ±10 lines joined as one string
37
+ // Returns: 'confirmed' | 'probable' | 'unlikely'
38
+
39
+ const VERIFIERS = {
40
+
41
+ // zod_usage: z.string() schema definition ≠ .parse(req.body) at route
42
+ zod_usage(line, context) {
43
+ if (/\.(safe)?[Pp]arse\s*\(\s*(req|request|ctx\.request)\.(body|params|query)/.test(context))
44
+ return 'confirmed';
45
+ // Inside a router handler scope
46
+ if (/router\.|app\.(get|post|put|patch|delete)\s*\(/.test(context))
47
+ return 'probable';
48
+ // Schema-only file (no req reference at all)
49
+ if (!/\breq\b|\brequest\b|\bctx\b/.test(context))
50
+ return 'unlikely';
51
+ return 'probable';
52
+ },
53
+
54
+ // pydantic_usage: BaseModel class ≠ used as FastAPI request body param
55
+ pydantic_usage(line, context) {
56
+ // FastAPI route decorator near a BaseModel subclass usage
57
+ if (/@app\.(post|put|patch|delete)\s*\(/.test(context))
58
+ return 'confirmed';
59
+ if (/async def \w+\s*\([^)]*:\s*\w+/.test(context))
60
+ return 'probable';
61
+ // Just a plain data class definition with no route nearby
62
+ if (/^class \w+\(BaseModel\)/.test(line) && !/@app\./.test(context))
63
+ return 'unlikely';
64
+ return 'probable';
65
+ },
66
+
67
+ // bearer_auth: "Bearer" string in comments/tests ≠ middleware wired to routes
68
+ bearer_auth(line, context) {
69
+ // Looks like a comment or docstring
70
+ if (/^\s*(\/\/|\/\*|\*|#)/.test(line))
71
+ return 'unlikely';
72
+ // Actual middleware or auth function application
73
+ if (/app\.use\s*\(|router\.use\s*\(|authenticate|verifyToken|passport\.authenticate/.test(context))
74
+ return 'confirmed';
75
+ // Authorization header being SET (client code), not verified (server code)
76
+ if (/headers\s*:\s*\{|fetch\s*\(|axios\./.test(context))
77
+ return 'unlikely';
78
+ return 'probable';
79
+ },
80
+
81
+ // timeout_config: `timeout: 30` in any object literal
82
+ timeout_config(line, context) {
83
+ // Inside an HTTP client options object
84
+ if (/fetch\s*\(|axios\.(create|get|post|put|patch|delete)\s*\(|httpClient|createClient|\.request\s*\(/.test(context))
85
+ return 'confirmed';
86
+ // Inside a test/jest/describe block
87
+ if (/\b(jest|describe|it|test|beforeEach|afterEach)\s*\(/.test(context))
88
+ return 'unlikely';
89
+ // Inside a known non-HTTP config object (lint, build tools)
90
+ if (/eslint|webpack|vite|rollup|tsconfig/.test(context))
91
+ return 'unlikely';
92
+ return 'unlikely'; // too many non-HTTP objects have a timeout key
93
+ },
94
+
95
+ // py_timeout: `timeout=N` as keyword argument
96
+ py_timeout(line, context) {
97
+ if (/requests\.(get|post|put|patch|delete|head|options)\s*\(|httpx\.|aiohttp\.ClientSession/.test(context))
98
+ return 'confirmed';
99
+ if (/subprocess\.|os\.system|sleep\s*\(/.test(context))
100
+ return 'unlikely';
101
+ return 'unlikely'; // conservative
102
+ },
103
+
104
+ // health_route: "/health" string anywhere
105
+ health_route(line, context) {
106
+ // Server-side route registration
107
+ if (/app\.(get|route)\s*\(\s*['"`]\/(health|healthz|ping|status)|router\.(get|route)\s*\(\s*['"`]\/(health|healthz|ping|status)|@app\.(get|route)\s*\(\s*['"`]\/(health|healthz|ping|status)/.test(context))
108
+ return 'confirmed';
109
+ // Client calling the health endpoint (fetch/axios)
110
+ if (/fetch\s*\(|axios\.(get|create)|requests\.(get|head)/.test(context))
111
+ return 'unlikely';
112
+ // Test asserting health endpoint exists
113
+ if (/expect\s*\(|assert|supertest|chai/.test(context))
114
+ return 'unlikely';
115
+ return 'unlikely'; // /health in README or openapi spec is common
116
+ },
117
+
118
+ // env_service_url: `process.env.X_URL` read
119
+ env_service_url(line, context) {
120
+ // Used as fetch/axios URL argument
121
+ if (/fetch\s*\(\s*process\.env|baseURL\s*:\s*process\.env|baseUrl\s*:\s*process\.env|url\s*:\s*process\.env/.test(context))
122
+ return 'confirmed';
123
+ if (/fetch\(|axios\.(create|get|post)|new URL\s*\(/.test(context))
124
+ return 'probable';
125
+ // Just reading PORT or DATABASE_URL for non-HTTP purposes
126
+ if (/PORT|DATABASE_URL|REDIS_URL|MONGO_URL/.test(line))
127
+ return 'unlikely';
128
+ return 'unlikely'; // env reads for NODE_ENV, LOG_LEVEL, etc. don't count
129
+ },
130
+
131
+ // retry_pattern: `maxRetries` / `retryCount` config key
132
+ retry_pattern(line, context) {
133
+ // Inside a retry library configuration
134
+ if (/axiosRetry\s*\(|p-retry|retrying\(|@retry|tenacity|backoff\.on_exception/.test(context))
135
+ return 'confirmed';
136
+ // Near an actual HTTP call site
137
+ if (/fetch\s*\(|axios\.|requests\.(get|post)|await/.test(context))
138
+ return 'probable';
139
+ // Config object in a test file or non-call-site
140
+ return 'unlikely';
141
+ },
142
+
143
+ // fallback_pattern: variable/function named "fallback"
144
+ fallback_pattern(line, context) {
145
+ if (/catch\s*\(|\.catch\s*\(|except\s/.test(context))
146
+ return 'probable';
147
+ if (/return\s+fallback|fallbackData|fallbackResponse/.test(line))
148
+ return 'probable';
149
+ return 'unlikely';
150
+ },
151
+
152
+ // feature_flag: isEnabled() / featureFlag
153
+ feature_flag(line, context) {
154
+ if (/if\s*\(\s*(isEnabled|featureFlag|feature_flag)/.test(context))
155
+ return 'probable';
156
+ return 'unlikely';
157
+ },
158
+ };
159
+
160
+ // ── Context-window grep ───────────────────────────────────────────────────────
161
+
162
+ const SOURCE_EXTS_RE = /\.(js|ts|jsx|tsx|mjs|cjs|py|go|rs|rb|java|cs|php)$/;
163
+ const TEST_PATH_RE = /\.(test|spec)\.|__tests__|\/tests?\/|test_[\w]+\.py$|_test\.go$/;
164
+ const CONTEXT_WINDOW = 10; // lines before + after match
165
+
166
+ /**
167
+ * Drop-in replacement for grepSources.
168
+ * Returns the same shape {key: count} but for HIGH_FP_KEYS the count is the
169
+ * number of VERIFIED (confirmed + probable) matches, not raw matches.
170
+ */
171
+ function listSourceFiles(cloneDir) {
172
+ try {
173
+ const tracked = execSync('git ls-files', { cwd: cloneDir, timeout: 10000 })
174
+ .toString().split('\n').filter(f => f && SOURCE_EXTS_RE.test(f));
175
+ // If git returned files, use them; empty means untracked/non-git, fall through
176
+ if (tracked.length > 0) return tracked;
177
+ } catch {}
178
+ // Fallback for non-git dirs or untracked working trees: shallow walk up to 2 levels
179
+ const results = [];
180
+ function walk(dir, depth) {
181
+ if (depth > 2) return;
182
+ try {
183
+ for (const entry of readdirSync(dir, { withFileTypes: true })) {
184
+ if (entry.name.startsWith('.') || entry.name === 'node_modules') continue;
185
+ const full = resolve(dir, entry.name);
186
+ if (entry.isDirectory()) walk(full, depth + 1);
187
+ else if (SOURCE_EXTS_RE.test(entry.name)) results.push(full);
188
+ }
189
+ } catch {}
190
+ }
191
+ walk(cloneDir, 0);
192
+ return results;
193
+ }
194
+
195
+ export function grepSourcesVerified(cloneDir, GREP_PATTERNS) {
196
+ const H = {};
197
+ const files = listSourceFiles(cloneDir);
198
+
199
+ for (const relPath of files.slice(0, 400)) {
200
+ const isTest = TEST_PATH_RE.test(relPath);
201
+ let content = '';
202
+ try { content = readFileSync(resolve(cloneDir, relPath), 'utf8').replace(/\0/g, ''); } catch { continue; }
203
+
204
+ const lines = content.split('\n');
205
+
206
+ for (const [key, pattern] of Object.entries(GREP_PATTERNS)) {
207
+ if (key === 'localhost_hardcoded' && isTest) continue;
208
+
209
+ if (!HIGH_FP_KEYS.has(key)) {
210
+ // low/medium risk: use raw count
211
+ const matches = content.match(pattern);
212
+ if (matches) H[key] = (H[key] || 0) + matches.length;
213
+ continue;
214
+ }
215
+
216
+ // HIGH-FP: scan line by line and apply context verification
217
+ const verifier = VERIFIERS[key];
218
+ let verifiedCount = 0;
219
+
220
+ lines.forEach((line, idx) => {
221
+ const p = new RegExp(pattern.source, pattern.flags);
222
+ if (!p.test(line)) return;
223
+
224
+ const start = Math.max(0, idx - CONTEXT_WINDOW);
225
+ const end = Math.min(lines.length, idx + CONTEXT_WINDOW + 1);
226
+ const ctx = lines.slice(start, end).join('\n');
227
+ const result = verifier ? verifier(line, ctx) : 'probable';
228
+
229
+ if (result === 'confirmed' || result === 'probable') verifiedCount++;
230
+ });
231
+
232
+ if (verifiedCount > 0) H[key] = (H[key] || 0) + verifiedCount;
233
+ }
234
+ }
235
+
236
+ return H;
237
+ }
238
+
239
+ /**
240
+ * Convenience: verify a single match externally.
241
+ */
242
+ export function verifyMatch(key, line, contextLines) {
243
+ if (!HIGH_FP_KEYS.has(key)) return 'confirmed';
244
+ const verifier = VERIFIERS[key];
245
+ if (!verifier) return 'probable';
246
+ return verifier(line, contextLines.join('\n'));
247
+ }