@quantracode/vibecheck 0.3.2 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,17 +1,34 @@
1
1
  # @quantracode/vibecheck
2
2
 
3
- A deterministic, local-only security scanner for modern web applications. Designed to catch common security issues in Next.js, Express, and other Node.js projects with high precision and low false positives.
3
+ The first AI Enforcement Security tool. Proves whether AI-written code actually enforces the security it claims not just implied, commented, or assumed.
4
4
 
5
- ## Quickstart (No Install)
5
+ ## What is AI Enforcement Security?
6
6
 
7
- Run VibeCheck instantly without installation:
7
+ Traditional security tools scan for vulnerabilities. VibeCheck verifies enforcement reality. That's a fundamentally different job.
8
+
9
+ AI-generated code often hallucinates security guarantees. It writes comments claiming protection exists, imports security libraries but doesn't use them, or creates middleware that never gets wired up. VibeCheck detects these patterns and proves what's actually enforced.
10
+
11
+ **Key Principles:**
12
+ - **Deterministic** — No LLM calls, results are reproducible
13
+ - **Local-only** — All analysis runs on your machine, code never uploaded
14
+ - **Low false positives** — Precision over recall
15
+ - **Framework-aware** — Built for Next.js, Express patterns
16
+ - **Enforcement-focused** — Proves what's enforced, not just scanned
17
+
18
+ ## Quick Start
8
19
 
9
20
  ```bash
10
- # Using npx
21
+ # Install globally
22
+ npm install -g @quantracode/vibecheck
23
+
24
+ # Or use without installing
11
25
  npx @quantracode/vibecheck scan --fail-on off --out vibecheck-scan.json
12
26
 
13
- # Using pnpm dlx
14
- pnpm dlx @quantracode/vibecheck scan --fail-on off --out vibecheck-scan.json
27
+ # With auto-fix enabled
28
+ npx @quantracode/vibecheck scan --apply-fixes
29
+
30
+ # With custom security rules
31
+ npx @quantracode/vibecheck scan --rules ./my-custom-rules
15
32
  ```
16
33
 
17
34
  ### Scan Another Folder
@@ -20,14 +37,6 @@ pnpm dlx @quantracode/vibecheck scan --fail-on off --out vibecheck-scan.json
20
37
  npx @quantracode/vibecheck scan --target ../my-other-app --out scan.json
21
38
  ```
22
39
 
23
- ## Installation
24
-
25
- ```bash
26
- npm install -g @quantracode/vibecheck
27
- # or
28
- pnpm add -g @quantracode/vibecheck
29
- ```
30
-
31
40
  ## Usage
32
41
 
33
42
  ```bash
@@ -61,6 +70,18 @@ vibecheck scan --include-tests
61
70
  # Generate intent map with coverage metrics
62
71
  vibecheck scan --emit-intent-map
63
72
 
73
+ # Auto-fix security issues (with confirmation)
74
+ vibecheck scan --apply-fixes
75
+
76
+ # Auto-fix without confirmation prompts
77
+ vibecheck scan --apply-fixes --force
78
+
79
+ # Load custom YAML security rules
80
+ vibecheck scan --rules ./my-custom-rules
81
+
82
+ # Load a single custom rule file
83
+ vibecheck scan -r my-rule.yaml
84
+
64
85
  # Explain a scan report
65
86
  vibecheck explain ./scan.json
66
87
 
@@ -129,7 +150,9 @@ vibecheck view
129
150
  # Browser opens to http://localhost:3000 with results loaded
130
151
  ```
131
152
 
132
- ### Command Line Options
153
+ ## Command Line Options
154
+
155
+ ### Scan Options
133
156
 
134
157
  | Option | Description | Default |
135
158
  |--------|-------------|---------|
@@ -141,7 +164,9 @@ vibecheck view
141
164
  | `-e, --exclude <glob>` | Glob pattern to exclude (repeatable) | See below |
142
165
  | `--include-tests` | Include test files in scan | `false` |
143
166
  | `--emit-intent-map` | Include route map and coverage metrics | `false` |
144
- | `--changed` | Only scan changed files (not implemented) | `false` |
167
+ | `--apply-fixes` | Apply patches from findings after scan | `false` |
168
+ | `--force` | Skip confirmation when applying patches | `false` |
169
+ | `-r, --rules <path>` | Load custom YAML rules from directory or file | - |
145
170
 
146
171
  ### Default Excludes
147
172
 
@@ -156,618 +181,325 @@ The following patterns are excluded by default:
156
181
  - `test/`, `tests/`, `fixtures/`, `__mocks__/`, `__fixtures__/`
157
182
  - `cypress/`, `e2e/`, `*.stories.*`
158
183
 
159
- ## Scanner Packs
160
-
161
- VibeCheck organizes security rules into modular scanner packs. Each pack focuses on a specific security domain.
162
-
163
- ### Auth Pack
164
-
165
- Rules for authentication and authorization issues.
166
-
167
- #### VC-AUTH-001: Unprotected State-Changing API Route
184
+ ## Auto-Fix Security Issues
168
185
 
169
- **Severity:** High / Critical
170
- **Category:** auth
186
+ VibeCheck can automatically apply security patches from findings:
171
187
 
172
- Detects Next.js App Router API route handlers (POST, PUT, PATCH, DELETE) that perform database operations without authentication checks.
173
-
174
- **What it looks for:**
175
- - Route handlers in `app/**/route.ts` files
176
- - Handlers that use Prisma, Drizzle, or other database operations
177
- - Missing calls to `getServerSession`, `auth()`, or similar auth checks
178
-
179
- **Example (vulnerable):**
180
- ```typescript
181
- // app/api/users/route.ts
182
- export async function POST(request: Request) {
183
- const body = await request.json();
184
- await prisma.user.create({ data: body }); // No auth check!
185
- return Response.json({ success: true });
186
- }
187
- ```
188
-
189
- **Example (safe):**
190
- ```typescript
191
- // app/api/users/route.ts
192
- export async function POST(request: Request) {
193
- const session = await getServerSession();
194
- if (!session) {
195
- return Response.json({ error: "Unauthorized" }, { status: 401 });
196
- }
197
- const body = await request.json();
198
- await prisma.user.create({ data: body });
199
- return Response.json({ success: true });
200
- }
201
- ```
202
-
203
- ---
204
-
205
- #### VC-MW-001: Middleware Matcher Gap
206
-
207
- **Severity:** High
208
- **Category:** middleware
209
-
210
- Detects Next.js middleware that doesn't cover API routes, potentially leaving them unprotected.
211
-
212
- **What it looks for:**
213
- - `middleware.ts` files with `config.matcher` exports
214
- - Matchers that exclude `/api` routes
215
- - Projects using next-auth without middleware protection
216
-
217
- **Example (vulnerable):**
218
- ```typescript
219
- // middleware.ts
220
- export const config = {
221
- matcher: ['/dashboard/:path*'] // Missing /api routes!
222
- };
223
- ```
188
+ ```bash
189
+ # Apply fixes with confirmation prompts for each patch
190
+ vibecheck scan --apply-fixes
224
191
 
225
- **Example (safe):**
226
- ```typescript
227
- // middleware.ts
228
- export const config = {
229
- matcher: ['/api/:path*', '/dashboard/:path*']
230
- };
192
+ # Apply all fixes automatically without prompts (use with caution!)
193
+ vibecheck scan --apply-fixes --force
231
194
  ```
232
195
 
233
- ---
234
-
235
- ### Validation Pack
236
-
237
- Rules for input validation issues.
238
-
239
- #### VC-VAL-001: Validation Defined But Output Ignored
240
-
241
- **Severity:** Medium
242
- **Category:** validation
243
-
244
- Detects cases where validation libraries (Zod, Yup, Joi) are called but the validated result is not used.
245
-
246
- **What it looks for:**
247
- - Calls to `.parse()`, `.validate()`, `.parseAsync()`, etc.
248
- - Result not assigned to a variable
249
- - Raw `request.body` or `req.body` used after validation
196
+ ### How It Works
250
197
 
251
- **Example (vulnerable):**
252
- ```typescript
253
- const schema = z.object({ name: z.string() });
254
- schema.parse(body); // Result ignored!
255
- await prisma.user.create({ data: body }); // Uses unvalidated body
256
- ```
198
+ 1. **Scan completes** - VibeCheck identifies security issues
199
+ 2. **Patches available** - Findings with `remediation.patch` field are candidates for auto-fix
200
+ 3. **Preview shown** - Each patch is displayed with before/after comparison
201
+ 4. **User confirmation** - You approve or reject each patch (unless `--force` is used)
202
+ 5. **Applied to files** - Approved patches are written to your source files
257
203
 
258
- **Example (safe):**
259
- ```typescript
260
- const schema = z.object({ name: z.string() });
261
- const validated = schema.parse(body);
262
- await prisma.user.create({ data: validated });
263
- ```
204
+ ### Safety Features
264
205
 
265
- ---
206
+ - **Unified diff format**: Only standard git-style diffs are supported for safety
207
+ - **Context validation**: Patches verify that file content matches before applying
208
+ - **Confirmation required**: Interactive approval by default
209
+ - **Detailed errors**: Clear messages if a patch fails to apply
266
210
 
267
- ### Privacy Pack
211
+ ### Best Practices
268
212
 
269
- Rules for data privacy and logging issues.
213
+ 1. **Commit first**: Always have a clean git status before applying fixes
214
+ 2. **Review changes**: Run `git diff` after applying patches
215
+ 3. **Test thoroughly**: Run your test suite to ensure nothing broke
216
+ 4. **Use force carefully**: Only use `--force` when you fully trust the patches
270
217
 
271
- #### VC-PRIV-001: Sensitive Data Logged
218
+ **Example:**
272
219
 
273
- **Severity:** High
274
- **Category:** privacy
220
+ ```bash
221
+ # 1. Ensure clean working directory
222
+ git status
275
223
 
276
- Detects logging statements that include sensitive variable names.
224
+ # 2. Run scan with auto-fix
225
+ vibecheck scan --apply-fixes
277
226
 
278
- **What it looks for:**
279
- - `console.log`, `console.info`, `console.debug`, `logger.info`, etc.
280
- - Variables containing: password, secret, token, apiKey, creditCard, ssn, etc.
227
+ # 3. Review what changed
228
+ git diff
281
229
 
282
- **Example (vulnerable):**
283
- ```typescript
284
- console.log("User login:", { email, password }); // Logs password!
285
- ```
230
+ # 4. Test the changes
231
+ npm test
286
232
 
287
- **Example (safe):**
288
- ```typescript
289
- console.log("User login:", { email, timestamp: Date.now() });
233
+ # 5. Commit if everything looks good
234
+ git add .
235
+ git commit -m "fix: apply VibeCheck security patches"
290
236
  ```
291
237
 
292
- ---
293
-
294
- ### Config Pack
295
-
296
- Rules for configuration and secrets management issues.
297
-
298
- #### VC-CONFIG-001: Undocumented Environment Variable
299
-
300
- **Severity:** Low
301
- **Category:** config
238
+ ## Custom Security Rules
302
239
 
303
- Detects `process.env.VAR` references that aren't documented in `.env.example`.
240
+ Extend VibeCheck with your own YAML-based security rules - no TypeScript required!
304
241
 
305
- ---
306
-
307
- #### VC-CONFIG-002: Insecure Default Secret Fallback
308
-
309
- **Severity:** Critical
310
- **Category:** secrets
311
-
312
- Detects hardcoded fallback values for security-critical environment variables.
313
-
314
- **What it looks for:**
315
- - `process.env.VAR || "fallback"` patterns
316
- - Variables named: SECRET, KEY, TOKEN, PASSWORD, etc.
317
- - Hardcoded string fallbacks
318
-
319
- **Example (vulnerable):**
320
- ```typescript
321
- const jwtSecret = process.env.JWT_SECRET || "development-secret";
322
- ```
242
+ ```bash
243
+ # Load custom rules from a directory
244
+ vibecheck scan --rules ./my-custom-rules
323
245
 
324
- **Example (safe):**
325
- ```typescript
326
- const jwtSecret = process.env.JWT_SECRET;
327
- if (!jwtSecret) throw new Error("JWT_SECRET is required");
246
+ # Load a single rule file
247
+ vibecheck scan -r my-security-rule.yaml
328
248
  ```
329
249
 
330
- ---
250
+ ### Example Custom Rule
331
251
 
332
- ### Network Pack
252
+ ```yaml
253
+ id: CUSTOM-AUTH-001
254
+ severity: high
255
+ category: auth
256
+ title: "Missing authentication on POST endpoint"
257
+ description: |
258
+ POST endpoints should have authentication checks to prevent
259
+ unauthorized access to state-changing operations.
333
260
 
334
- Rules for network security issues.
261
+ files:
262
+ file_type: [ts, tsx]
263
+ include: ["**/api/**/*.ts"]
264
+ exclude: ["**/*.test.*"]
335
265
 
336
- #### VC-NET-001: SSRF-Prone Fetch
266
+ match:
267
+ contains: "export async function POST"
268
+ not_contains: "getServerSession"
269
+ case_sensitive: false
337
270
 
338
- **Severity:** High
339
- **Category:** network
271
+ context:
272
+ in_function: [POST]
273
+ file_not_contains: ["test", "mock"]
340
274
 
341
- Detects fetch/axios calls where the URL is derived from user input without validation.
275
+ recommended_fix: |
276
+ Add authentication to your POST handler:
342
277
 
343
- **What it looks for:**
344
- - `fetch()` or `axios.get()` calls
345
- - URL constructed from request parameters, query strings, or body
346
- - No URL validation or allowlist checks
278
+ ```typescript
279
+ import { getServerSession } from "next-auth";
347
280
 
348
- **Example (vulnerable):**
349
- ```typescript
350
- export async function GET(request: Request) {
351
- const { searchParams } = new URL(request.url);
352
- const url = searchParams.get("url");
353
- const response = await fetch(url); // SSRF risk!
354
- return Response.json(await response.json());
355
- }
356
- ```
357
-
358
- **Example (safe):**
359
- ```typescript
360
- const ALLOWED_HOSTS = ["api.example.com", "cdn.example.com"];
361
-
362
- export async function GET(request: Request) {
363
- const { searchParams } = new URL(request.url);
364
- const url = searchParams.get("url");
365
- const parsed = new URL(url);
366
- if (!ALLOWED_HOSTS.includes(parsed.hostname)) {
367
- return Response.json({ error: "Invalid host" }, { status: 400 });
281
+ export async function POST(request: Request) {
282
+ const session = await getServerSession(authOptions);
283
+ if (!session) {
284
+ return new Response("Unauthorized", { status: 401 });
285
+ }
286
+ // ... rest of handler
368
287
  }
369
- const response = await fetch(url);
370
- return Response.json(await response.json());
371
- }
372
- ```
373
-
374
- ---
375
-
376
- ### Hallucinations Pack
377
-
378
- Rules for detecting security libraries that are imported but not properly used.
379
-
380
- #### VC-HALL-001: Security Library Imported But Not Used
381
-
382
- **Severity:** Medium
383
- **Category:** middleware
384
-
385
- Detects security libraries (helmet, cors, csurf, etc.) that are imported but the import is never used.
288
+ ```
386
289
 
387
- **What it looks for:**
388
- - Imports from security packages: helmet, cors, csurf, express-rate-limit, hpp, etc.
389
- - Import identifier not referenced after the import statement
290
+ links:
291
+ owasp: https://owasp.org/API-Security/editions/2023/en/0xa2-broken-authentication/
292
+ cwe: https://cwe.mitre.org/data/definitions/306.html
390
293
 
391
- **Example (vulnerable):**
392
- ```typescript
393
- import helmet from "helmet"; // Imported but never used!
394
- import cors from "cors";
395
-
396
- const app = express();
397
- app.use(cors());
398
- // Missing: app.use(helmet());
294
+ metadata:
295
+ author: "Your Name"
296
+ tags: ["authentication", "api-security"]
297
+ created: "2025-01-07"
399
298
  ```
400
299
 
401
- ---
402
-
403
- #### VC-HALL-002: NextAuth Imported But Not Enforced
404
-
405
- **Severity:** High
406
- **Category:** auth
407
-
408
- Detects next-auth imported but `getServerSession` never called, suggesting auth is configured but not enforced.
409
-
410
- **What it looks for:**
411
- - Imports from `next-auth` or `next-auth/next`
412
- - No calls to `getServerSession` anywhere in the file
413
-
414
- ---
300
+ ### Rule Structure
415
301
 
416
- ## Phase 2 Rules
302
+ **Required Fields:**
303
+ - `id`: Unique identifier (format: `XXX-XXX-000`)
304
+ - `severity`: `critical`, `high`, `medium`, `low`, or `info`
305
+ - `category`: Security category (auth, validation, secrets, etc.)
306
+ - `title`: Short description
307
+ - `description`: Detailed explanation
308
+ - `match`: What to look for (see Match Conditions below)
309
+ - `recommended_fix`: How to fix the issue
417
310
 
418
- ### Network Pack (Extended)
311
+ **Optional Fields:**
312
+ - `files`: File filters (type, include/exclude patterns, directories)
313
+ - `context`: Advanced conditions (imports, function types, file content)
314
+ - `patch`: Unified diff for auto-fixing
315
+ - `links`: Reference URLs (OWASP, CWE, docs)
316
+ - `metadata`: Author, tags, version, dates
317
+ - `enabled`: Whether the rule is active (default: true)
419
318
 
420
- #### VC-NET-002: Open Redirect
319
+ ### Match Conditions
421
320
 
422
- **Severity:** High
423
- **Category:** network
424
-
425
- Detects server-side redirects where user-controlled input determines the destination.
426
-
427
- **Two-signal requirement:** Must identify user-controlled source AND redirect call uses that value.
428
-
429
- **Example (vulnerable):**
430
- ```typescript
431
- export async function GET(request: Request) {
432
- const { searchParams } = new URL(request.url);
433
- const next = searchParams.get("next");
434
- return NextResponse.redirect(next!); // Open redirect!
435
- }
321
+ **String Match:**
322
+ ```yaml
323
+ match:
324
+ contains: "eval("
325
+ case_sensitive: true
436
326
  ```
437
327
 
438
- ---
439
-
440
- #### VC-NET-003: Over-permissive CORS with Credentials
441
-
442
- **Severity:** High
443
- **Category:** network
444
-
445
- Detects CORS configurations that combine `origin: "*"` with `credentials: true`.
446
-
447
- **Example (vulnerable):**
448
- ```typescript
449
- cors({ origin: "*", credentials: true }) // Dangerous combination!
328
+ **Negative Match (should NOT contain):**
329
+ ```yaml
330
+ match:
331
+ not_contains: "getServerSession" # Flag files without auth
450
332
  ```
451
333
 
452
- ---
453
-
454
- #### VC-NET-004: Missing Request Timeout
455
-
456
- **Severity:** Low
457
- **Category:** network
458
-
459
- Detects fetch/axios calls without timeout in API route handlers.
460
-
461
- **Example (vulnerable):**
462
- ```typescript
463
- const response = await fetch("https://external-api.com/data"); // No timeout!
334
+ **Regular Expression:**
335
+ ```yaml
336
+ match:
337
+ regex: "password\\s*=\\s*['\"][^'\"]+['\"]"
338
+ case_sensitive: false
464
339
  ```
465
340
 
466
- ---
467
-
468
- ### Middleware Pack
469
-
470
- #### VC-RATE-001: Missing Rate Limiting
471
-
472
- **Severity:** Medium
473
- **Category:** middleware
474
- **Confidence:** 0.65
475
-
476
- Detects unauthenticated state-changing endpoints without rate limiting.
477
-
478
- **What it looks for:**
479
- - POST/PUT/PATCH/DELETE handlers without auth checks
480
- - Handlers with database writes or sensitive operations
481
- - No rate limiting signals in handler or middleware
482
-
483
- ---
484
-
485
- ### Validation Pack (Extended)
486
-
487
- #### VC-VAL-002: Client-Side Only Validation
488
-
489
- **Severity:** Medium
490
- **Category:** validation
491
-
492
- Detects validation in frontend components but missing in API routes.
493
-
494
- ---
495
-
496
- ### Privacy Pack (Extended)
497
-
498
- #### VC-PRIV-002: Over-broad API Response
499
-
500
- **Severity:** Medium/High
501
- **Category:** privacy
502
-
503
- Detects Prisma queries returning full models without `select` restrictions.
504
-
505
- **Example (vulnerable):**
506
- ```typescript
507
- const users = await prisma.user.findMany(); // Returns password hash!
508
- return Response.json(users);
341
+ **Combined Conditions:**
342
+ ```yaml
343
+ match:
344
+ contains: "export async function POST"
345
+ not_contains: "await auth()"
346
+ regex: "prisma\\.(create|update|delete)"
509
347
  ```
510
348
 
511
- **Example (safe):**
512
- ```typescript
513
- const users = await prisma.user.findMany({
514
- select: { id: true, name: true, email: true }
515
- });
516
- return Response.json(users);
517
- ```
518
-
519
- ---
520
-
521
- #### VC-PRIV-003: Debug Flags in Production
522
-
523
- **Severity:** Medium
524
- **Category:** config
525
-
526
- Detects `debug: true` or `dev: true` in config files without NODE_ENV guards.
527
-
528
- ---
529
-
530
- ### Crypto Pack
531
-
532
- #### VC-CRYPTO-001: Math.random for Tokens
533
-
534
- **Severity:** High
535
- **Category:** crypto
536
-
537
- Detects Math.random used to generate tokens, keys, or session IDs.
349
+ ### File Filters
538
350
 
539
- **Example (vulnerable):**
540
- ```typescript
541
- const token = Math.random().toString(36).substring(2); // Predictable!
351
+ ```yaml
352
+ files:
353
+ file_type: [ts, tsx, js, jsx]
354
+ include: ["**/api/**/*.ts", "**/routes/**/*.ts"]
355
+ exclude: ["**/*.test.*", "**/*.spec.*"]
356
+ directories: ["/api/", "/routes/"]
542
357
  ```
543
358
 
544
- **Example (safe):**
545
- ```typescript
546
- const token = crypto.randomBytes(32).toString('hex');
547
- ```
359
+ Available file types: `ts`, `tsx`, `js`, `jsx`, `json`, `env`, `yaml`, `yml`, `md`, `config`, `any`
548
360
 
549
- ---
361
+ ### Context Conditions
550
362
 
551
- #### VC-CRYPTO-002: JWT Decode Without Verify
363
+ ```yaml
364
+ context:
365
+ # Only flag if file imports these packages
366
+ requires_import: ["next-auth", "@prisma/client"]
552
367
 
553
- **Severity:** Critical
554
- **Category:** crypto
368
+ # Don't flag if file imports these
369
+ excludes_import: ["vitest", "@testing-library"]
555
370
 
556
- Detects jwt.decode() used without jwt.verify() in the same file.
371
+ # File must contain all of these
372
+ file_contains: ["database", "user"]
557
373
 
558
- **Example (vulnerable):**
559
- ```typescript
560
- const payload = jwt.decode(token); // Signature not verified!
561
- ```
374
+ # File must NOT contain any of these
375
+ file_not_contains: ["test", "mock"]
562
376
 
563
- **Example (safe):**
564
- ```typescript
565
- const payload = jwt.verify(token, secret); // Signature verified
377
+ # Only match in specific handler types
378
+ in_function: [POST, PUT, DELETE, route_handler]
566
379
  ```
567
380
 
568
- ---
381
+ ### Multiple Rules in One File
569
382
 
570
- #### VC-CRYPTO-003: Weak Password Hashing
383
+ ```yaml
384
+ schema_version: "1.0"
571
385
 
572
- **Severity:** High
573
- **Category:** crypto
386
+ rules:
387
+ - id: CUSTOM-001
388
+ severity: high
389
+ category: auth
390
+ title: "First rule"
391
+ # ... rule config
574
392
 
575
- Detects MD5/SHA1 for passwords or bcrypt with saltRounds < 10.
576
-
577
- **Example (vulnerable):**
578
- ```typescript
579
- crypto.createHash('md5').update(password).digest('hex'); // Weak!
580
- bcrypt.hash(password, 5); // Too few rounds!
393
+ - id: CUSTOM-002
394
+ severity: medium
395
+ category: validation
396
+ title: "Second rule"
397
+ # ... rule config
581
398
  ```
582
399
 
583
- ---
400
+ ### Complete Guide
584
401
 
585
- ### Uploads Pack
402
+ See [examples/CUSTOM_RULES_GUIDE.md](./examples/CUSTOM_RULES_GUIDE.md) for:
403
+ - Full rule specification
404
+ - Advanced examples (SQL injection, hardcoded secrets, etc.)
405
+ - Best practices and troubleshooting
406
+ - Community contribution guidelines
586
407
 
587
- #### VC-UP-001: File Upload Without Constraints
408
+ ### Example Rules
588
409
 
589
- **Severity:** High
590
- **Category:** uploads
410
+ Check out [examples/custom-rules/](./examples/custom-rules/) for production-ready examples:
411
+ - `hardcoded-secret.yaml` - Detect secrets in .env files
412
+ - `missing-rate-limit.yaml` - Find API routes without rate limiting
413
+ - `console-log-production.yaml` - Flag console.log in production code
414
+ - `collection-example.yaml` - Multiple rules in one file
591
415
 
592
- Detects file uploads without size or type validation.
416
+ ## Scanner Categories
593
417
 
594
- **Example (vulnerable):**
595
- ```typescript
596
- const file = formData.get('file') as File;
597
- // No size or type check before processing
598
- ```
418
+ VibeCheck includes 30+ enforcement verification scanners across these categories:
599
419
 
600
- ---
420
+ ### Auth & Authorization
601
421
 
602
- #### VC-UP-002: Upload to Public Path
422
+ | Rule ID | Title | Severity |
423
+ |---------|-------|----------|
424
+ | VC-AUTH-001 | Unprotected State-Changing API Route | High/Critical |
425
+ | VC-MW-001 | Middleware Matcher Gap | High |
426
+ | VC-AUTH-010 | Auth-by-UI with Server Gap | Critical |
603
427
 
604
- **Severity:** High
605
- **Category:** uploads
428
+ ### Input Validation
606
429
 
607
- Detects uploaded files written to public directories.
430
+ | Rule ID | Title | Severity |
431
+ |---------|-------|----------|
432
+ | VC-VAL-001 | Validation Defined But Output Ignored | Medium |
433
+ | VC-VAL-002 | Client-Side Only Validation | Medium |
608
434
 
609
- **Example (vulnerable):**
610
- ```typescript
611
- fs.writeFileSync(`public/uploads/${filename}`, buffer); // Publicly accessible!
612
- ```
435
+ ### Network Security
613
436
 
614
- ---
437
+ | Rule ID | Title | Severity |
438
+ |---------|-------|----------|
439
+ | VC-NET-001 | SSRF-Prone Fetch | High |
440
+ | VC-NET-002 | Open Redirect | High |
441
+ | VC-NET-003 | Over-permissive CORS with Credentials | High |
442
+ | VC-NET-004 | Missing Request Timeout | Low |
615
443
 
616
- ## Phase 3: Hallucination Detection Engine
444
+ ### Secrets & Config
617
445
 
618
- Advanced cross-file analysis for detecting security intent vs implementation gaps.
446
+ | Rule ID | Title | Severity |
447
+ |---------|-------|----------|
448
+ | VC-CONFIG-001 | Undocumented Environment Variable | Low |
449
+ | VC-CONFIG-002 | Insecure Default Secret Fallback | Critical |
450
+ | VC-PRIV-003 | Debug Flags in Production | Medium |
619
451
 
620
- ### Intent Command
452
+ ### Privacy & Data
621
453
 
622
- Generate a security intent map baseline for your codebase:
454
+ | Rule ID | Title | Severity |
455
+ |---------|-------|----------|
456
+ | VC-PRIV-001 | Sensitive Data Logged | High |
457
+ | VC-PRIV-002 | Over-broad API Response | Medium/High |
623
458
 
624
- ```bash
625
- # Generate intent map
626
- vibecheck intent ./my-project --out intent-map.json
459
+ ### Cryptography
627
460
 
628
- # Include intent map in scan output
629
- vibecheck scan ./my-project --emit-intent-map
630
- ```
631
-
632
- ### Hallucinations Pack (Phase 3)
461
+ | Rule ID | Title | Severity |
462
+ |---------|-------|----------|
463
+ | VC-CRYPTO-001 | Math.random for Tokens | High |
464
+ | VC-CRYPTO-002 | JWT Decode Without Verify | Critical |
465
+ | VC-CRYPTO-003 | Weak Password Hashing | High |
633
466
 
634
- #### VC-HALL-010: Comment Claims Protection But Unproven
467
+ ### File Uploads
635
468
 
636
- **Severity:** Medium
637
- **Category:** hallucinations
638
- **Confidence:** 0.75
469
+ | Rule ID | Title | Severity |
470
+ |---------|-------|----------|
471
+ | VC-UP-001 | File Upload Without Constraints | High |
472
+ | VC-UP-002 | Upload to Public Path | High |
639
473
 
640
- Detects comments that claim security protection but the implementation doesn't prove it.
474
+ ### Middleware
641
475
 
642
- **Example (vulnerable):**
643
- ```typescript
644
- // This route is protected by authentication middleware
645
- export async function POST(request: Request) {
646
- // No auth check here, and middleware doesn't cover /api routes
647
- await prisma.user.create({ data: body });
648
- }
649
- ```
476
+ | Rule ID | Title | Severity |
477
+ |---------|-------|----------|
478
+ | VC-RATE-001 | Missing Rate Limiting | Medium |
650
479
 
651
- ---
480
+ ### AI Hallucinations
652
481
 
653
- #### VC-HALL-011: Middleware Assumed But Not Matching
482
+ | Rule ID | Title | Severity |
483
+ |---------|-------|----------|
484
+ | VC-HALL-001 | Security Library Imported But Not Used | Medium |
485
+ | VC-HALL-010 | Comment Claims Protection But Unproven | Medium |
486
+ | VC-HALL-011 | Middleware Assumed But Not Matching | High |
487
+ | VC-HALL-012 | Validation Claimed But Missing/Ignored | Medium |
654
488
 
655
- **Severity:** High
656
- **Category:** hallucinations
657
- **Confidence:** 0.70
489
+ ### Supply Chain
658
490
 
659
- Detects routes that expect middleware protection but are not covered by matcher patterns.
491
+ | Rule ID | Title | Severity |
492
+ |---------|-------|----------|
493
+ | VC-SC-001 | Unpinned Dependencies | Medium |
494
+ | VC-SC-002 | Suspicious Postinstall Scripts | High |
495
+ | VC-SC-003 | Deprecated Packages | Low |
660
496
 
661
- **Example (vulnerable):**
662
- ```typescript
663
- // middleware.ts
664
- export const config = {
665
- matcher: ['/dashboard/:path*'], // Missing /api routes!
666
- };
497
+ ### Abuse & Compute
667
498
 
668
- // app/api/users/route.ts
669
- // Auth handled by middleware (but middleware doesn't cover this!)
670
- export async function DELETE(request: Request) {
671
- await prisma.user.delete({ where: { id } });
672
- }
673
- ```
674
-
675
- ---
676
-
677
- #### VC-HALL-012: Validation Claimed But Missing/Ignored
678
-
679
- **Severity:** Medium
680
- **Category:** hallucinations
681
- **Confidence:** 0.80
682
-
683
- Detects validation that is claimed but not properly implemented or used.
684
-
685
- **Example (vulnerable):**
686
- ```typescript
687
- const schema = z.object({ name: z.string() });
688
- schema.parse(body); // Result ignored!
689
- await prisma.user.create({ data: body }); // Uses raw body
690
- ```
691
-
692
- ---
693
-
694
- #### VC-AUTH-010: Auth-by-UI with Server Gap
695
-
696
- **Severity:** Critical
697
- **Category:** auth
698
- **Confidence:** 0.85
699
-
700
- Detects client-side auth checks without corresponding server-side protection.
701
-
702
- **Example (vulnerable):**
703
- ```tsx
704
- // Client component
705
- const { session } = useSession();
706
- if (session) {
707
- // Only render if logged in
708
- await fetch('/api/users', { method: 'DELETE' }); // Server has no auth!
709
- }
710
- ```
711
-
712
- ---
713
-
714
- ### Coverage Metrics
715
-
716
- With `--emit-intent-map`, the scan artifact includes coverage metrics:
717
-
718
- - **authCoverage**: Percentage of state-changing routes with proven auth
719
- - **validationCoverage**: Percentage of routes with request bodies that have validation
720
- - **middlewareCoverage**: Percentage of routes covered by middleware matchers
721
-
722
- ### Intent Map Structure
723
-
724
- ```json
725
- {
726
- "routeMap": [
727
- {
728
- "routeId": "abc123",
729
- "method": "POST",
730
- "path": "/api/users",
731
- "file": "app/api/users/route.ts",
732
- "startLine": 10,
733
- "endLine": 25
734
- }
735
- ],
736
- "middlewareMap": [
737
- {
738
- "file": "middleware.ts",
739
- "matchers": ["/api/:path*"],
740
- "protectsApi": true,
741
- "startLine": 15
742
- }
743
- ],
744
- "intentMap": [
745
- {
746
- "intentId": "def456",
747
- "type": "AUTH_ENFORCED",
748
- "scope": "route",
749
- "source": "comment",
750
- "textEvidence": "// Protected by auth"
751
- }
752
- ],
753
- "proofTraces": {
754
- "abc123": {
755
- "routeId": "abc123",
756
- "authProven": true,
757
- "validationProven": false,
758
- "middlewareCovered": true,
759
- "steps": [...]
760
- }
761
- },
762
- "coverage": {
763
- "authCoverage": 0.85,
764
- "validationCoverage": 0.60,
765
- "middlewareCoverage": 1.0
766
- }
767
- }
768
- ```
769
-
770
- ---
499
+ | Rule ID | Title | Severity |
500
+ |---------|-------|----------|
501
+ | VC-ABUSE-001 | Unbounded AI API Calls | High |
502
+ | VC-ABUSE-002 | Missing Cost Controls | Medium |
771
503
 
772
504
  ## Output Formats
773
505
 
@@ -779,11 +511,11 @@ The default format, defined by `@vibecheck/schema`:
779
511
 
780
512
  ```json
781
513
  {
782
- "artifactVersion": "0.1",
514
+ "artifactVersion": "0.3",
783
515
  "generatedAt": "2024-01-15T10:30:00.000Z",
784
516
  "tool": {
785
517
  "name": "vibecheck",
786
- "version": "1.0.0"
518
+ "version": "0.3.2"
787
519
  },
788
520
  "summary": {
789
521
  "totalFindings": 2,
@@ -824,21 +556,79 @@ The default format, defined by `@vibecheck/schema`:
824
556
 
825
557
  ### SARIF Format
826
558
 
827
- [SARIF (Static Analysis Results Interchange Format)](https://sarifweb.azurewebsites.net/) is an OASIS standard for static analysis tools. Use `--format sarif` to generate SARIF 2.1.0 output, compatible with:
559
+ SARIF (Static Analysis Results Interchange Format) is an OASIS standard for static analysis tools. Use `--format sarif` to generate SARIF 2.1.0 output, compatible with:
828
560
 
829
- - **GitHub Code Scanning** - Upload via `github/codeql-action/upload-sarif`
830
- - **Azure DevOps** - Native SARIF support in security reports
831
- - **VS Code** - SARIF Viewer extension
832
- - **Other tools** - Any SARIF 2.1.0 compatible viewer
561
+ - **GitHub Code Scanning** Upload via `github/codeql-action/upload-sarif`
562
+ - **Azure DevOps** Native SARIF support in security reports
563
+ - **VS Code** SARIF Viewer extension
564
+ - **Other tools** Any SARIF 2.1.0 compatible viewer
833
565
 
834
566
  ```bash
835
567
  # Generate SARIF for GitHub Code Scanning
836
568
  vibecheck scan --format sarif --out results.sarif
569
+ ```
570
+
571
+ ## CI/CD Integration
572
+
573
+ ### GitHub Actions Example
574
+
575
+ ```yaml
576
+ name: Security Scan
577
+
578
+ on: [push, pull_request]
579
+
580
+ jobs:
581
+ vibecheck:
582
+ runs-on: ubuntu-latest
583
+ steps:
584
+ - uses: actions/checkout@v4
585
+
586
+ - name: Setup Node.js
587
+ uses: actions/setup-node@v4
588
+ with:
589
+ node-version: '20'
590
+
591
+ - name: Run VibeCheck
592
+ run: npx @quantracode/vibecheck scan --format sarif --out results.sarif --fail-on high
593
+
594
+ - name: Upload SARIF
595
+ uses: github/codeql-action/upload-sarif@v3
596
+ if: always()
597
+ with:
598
+ sarif_file: results.sarif
599
+ ```
837
600
 
838
- # Upload to GitHub (in CI workflow)
839
- - uses: github/codeql-action/upload-sarif@v2
840
- with:
841
- sarif_file: results.sarif
601
+ ### Policy Evaluation
602
+
603
+ Compare scans against baselines for regression detection:
604
+
605
+ ```bash
606
+ # Evaluate against startup profile
607
+ vibecheck evaluate \
608
+ --artifact vibecheck-scan.json \
609
+ --profile startup \
610
+ --out policy-report.json
611
+
612
+ # Compare against baseline (regression detection)
613
+ vibecheck evaluate \
614
+ --artifact vibecheck-scan.json \
615
+ --baseline main-branch-scan.json \
616
+ --profile enterprise
617
+ ```
618
+
619
+ Available profiles: `startup`, `growth`, `enterprise`
620
+
621
+ ## Intent Map & Coverage Metrics
622
+
623
+ With `--emit-intent-map`, the scan artifact includes coverage metrics:
624
+
625
+ - **authCoverage**: Percentage of state-changing routes with proven auth
626
+ - **validationCoverage**: Percentage of routes with request bodies that have validation
627
+ - **middlewareCoverage**: Percentage of routes covered by middleware matchers
628
+
629
+ ```bash
630
+ # Generate intent map
631
+ vibecheck scan ./my-project --emit-intent-map --out scan.json
842
632
  ```
843
633
 
844
634
  ## Architecture
@@ -847,57 +637,27 @@ vibecheck scan --format sarif --out results.sarif
847
637
  packages/cli/src/
848
638
  ├── commands/
849
639
  │ ├── scan.ts # Main scan orchestrator
640
+ │ ├── evaluate.ts # Policy evaluation
850
641
  │ └── explain.ts # Report viewer
851
642
  ├── scanners/
852
643
  │ ├── types.ts # ScanContext, types
853
644
  │ ├── helpers/
854
645
  │ │ ├── ast-helpers.ts # ts-morph utilities
855
646
  │ │ └── context-builder.ts # ScanContext factory
856
- │ ├── auth/
857
- ├── unprotected-api-route.ts # VC-AUTH-001
858
- │ └── middleware-gap.ts # VC-MW-001
859
- │ ├── validation/
860
- │ └── ignored-validation.ts # VC-VAL-001
861
- │ ├── privacy/
862
- │ └── sensitive-logging.ts # VC-PRIV-001
863
- │ ├── config/
864
- ├── undocumented-env.ts # VC-CONFIG-001
865
- │ └── insecure-defaults.ts # VC-CONFIG-002
866
- ├── network/
867
- │ │ └── ssrf-prone-fetch.ts # VC-NET-001
868
- │ └── hallucinations/
869
- │ └── unused-security-imports.ts # VC-HALL-001, VC-HALL-002
647
+ │ ├── auth/ # Auth & authorization scanners
648
+ │ ├── validation/ # Input validation scanners
649
+ ├── privacy/ # Privacy & data scanners
650
+ │ ├── config/ # Config & secrets scanners
651
+ ├── network/ # Network security scanners
652
+ │ ├── crypto/ # Cryptography scanners
653
+ ├── uploads/ # File upload scanners
654
+ │ ├── middleware/ # Middleware scanners
655
+ │ ├── hallucinations/ # AI hallucination scanners
656
+ ├── supply-chain/ # Supply chain scanners
657
+ └── abuse/ # Abuse & compute scanners
870
658
  └── index.ts
871
659
  ```
872
660
 
873
- ## Design Principles
874
-
875
- 1. **Deterministic** - No LLM calls, results are reproducible
876
- 2. **Local-only** - All analysis runs on your machine
877
- 3. **Low false positives** - Precision over recall
878
- 4. **Framework-aware** - Built for Next.js, Express patterns
879
- 5. **Schema-compliant** - Output conforms to `@vibecheck/schema`
880
-
881
- ## Adding New Scanners
882
-
883
- Each scanner must:
884
-
885
- 1. Accept a `ScanContext` with repo info and AST helpers
886
- 2. Return `Finding[]` conforming to the schema
887
- 3. Generate deterministic fingerprints for deduplication
888
- 4. Include clear evidence with file locations
889
-
890
- ```typescript
891
- import { type ScanContext } from "../types.js";
892
- import { type Finding } from "@vibecheck/schema";
893
-
894
- export async function scanMyRule(ctx: ScanContext): Promise<Finding[]> {
895
- const findings: Finding[] = [];
896
- // ... detection logic
897
- return findings;
898
- }
899
- ```
900
-
901
661
  ## License
902
662
 
903
663
  MIT