@nolans01/agent-validator 0.1.0

package/README.md

@@ -0,0 +1,299 @@
+# agent-validator
+
+Deterministic code quality validation orchestrator for TypeScript, Python, and Ruby projects. Designed for AI coding agents to validate their generated code against measurable metrics.
+
+## Features
+
+- **5 Quality Gates**: type safety, complexity, security, architecture, test quality (mutation testing)
+- **Multi-language**: TypeScript/JavaScript, Python, Ruby
+- **Deterministic**: No LLM-based analysis — only measurable, reproducible metrics
+- **Composable**: Use all gates or cherry-pick specific checks
+- **Profiles**: Pre-configured thresholds (default, critical, prototype)
+- **Fail-fast**: Gates ordered by speed; expensive checks run last
+
+## Installation
+
+```bash
+npm install -g @nolans01/agent-validator
+```
+
+Or as a dev dependency in your project:
+
+```bash
+npm install --save-dev @nolans01/agent-validator
+```
+
+## External Tools
+
+agent-validator orchestrates external tools. Install the ones you need:
+
+```bash
+# Type Safety
+npm install -g typescript    # for TypeScript projects
+pip install mypy             # for Python projects
+
+# Complexity
+pip install lizard
+
+# Security
+pip install semgrep
+brew install gitleaks        # or: https://github.com/gitleaks/gitleaks
+
+# Architecture
+npm install -g madge jscpd knip
+
+# Test Quality (Mutation Testing)
+npm install -D @stryker-mutator/core  # for TypeScript/JavaScript
+pip install mutmut                     # for Python
+gem install mutant mutant-rspec        # for Ruby
+```
+
+Check which tools are available:
+
+```bash
+validator doctor
+```
+
+## Quick Start
+
+### CLI Usage
+
+Scan entire project:
+
+```bash
+validator scan
+```
+
+Analyze specific files:
+
+```bash
+validator files src/index.ts src/utils.ts
+```
+
+Analyze git diff:
+
+```bash
+validator diff --base main --head HEAD
+```
+
+Run specific gates only:
+
+```bash
+validator scan --gates complexity,security
+```
+
+Use a stricter profile:
+
+```bash
+validator scan --profile critical
+```
+
+JSON output for CI:
+
+```bash
+validator scan --format json
+```
+
+Fail on warnings (default: fail on blockers only):
+
+```bash
+validator scan --fail-on warning
+```
+
+### Programmatic API (TypeScript/JavaScript)
+
+```typescript
+import { validate } from '@nolans01/agent-validator';
+
+const report = await validate({
+  mode: 'scan',
+  profile: 'default',
+  gates: ['complexity', 'security'],
+  workdir: process.cwd(),
+});
+
+console.log(`Status: ${report.status}`);
+console.log(`Blockers: ${report.blockers}`);
+console.log(`Warnings: ${report.warnings}`);
+
+// Access detailed results
+for (const gate of report.gates) {
+  console.log(`${gate.gate}: ${gate.status} (${gate.score}%)`);
+  for (const finding of gate.findings) {
+    console.log(`  ${finding.file}:${finding.line} [${finding.severity}] ${finding.message}`);
+  }
+}
+```
+
+### Python Integration
+
+Create a shell wrapper or use subprocess:
+
+```python
+import subprocess
+import json
+
+result = subprocess.run(
+    ['validator', 'scan', '--format', 'json'],
+    capture_output=True,
+    text=True
+)
+
+report = json.loads(result.stdout)
+if report['status'] == 'fail':
+    print(f"Validation failed: {report['blockers']} blockers")
+    exit(1)
+```
+
+### Ruby Integration
+
+```ruby
+require 'json'
+
+output = `validator scan --format json`
+report = JSON.parse(output)
+
+if report['status'] == 'fail'
+  puts "Validation failed: #{report['blockers']} blockers"
+  exit 1
+end
+```
+
+## Quality Gates
+
+### 1. Type Safety Gate
+- **TypeScript**: `tsc --noEmit`
+- **Python**: `mypy`
+- **Scoring**: -10 per blocker, -5 per warning, -2 per info
+- **Fail threshold**: score < 70
+
+### 2. Complexity Gate
+- **Tool**: `lizard`
+- **Metrics**: cyclomatic complexity, function length, parameter count, nesting depth
+- **Scoring**: ratio of clean functions to total functions
+- **Fail threshold**: score < 70 OR any blocker (CCN > 1.5× threshold)
+
+### 3. Security Gate
+- **Tools**: `semgrep` (SAST), `gitleaks` (secret detection)
+- **Scoring**: -25 per blocker, -10 per warning, -3 per info
+- **Fail threshold**: score < 70 OR any blocker
+
+### 4. Architecture Gate
+- **Tools**: `madge` (circular deps), `jscpd` (duplication), `knip` (dead code)
+- **Scoring**: -15 per blocker, -7 per warning, -3 per info
+- **Fail threshold**: score < 70 OR any blocker
+
+### 5. Test Quality Gate (Mutation Testing)
+- **TypeScript/JavaScript**: `stryker`
+- **Python**: `mutmut`
+- **Ruby**: `mutant`
+- **Scoring**: mutation score (% of killed mutants)
+- **Fail threshold**: score < 70
+
+## Profiles
+
+### default
+Balanced thresholds for production code:
+- Cyclomatic: 10, Length: 40, Arguments: 4
+- Mutation score: 80%
+
+### critical
+Strict thresholds for critical systems:
+- Cyclomatic: 6, Length: 30, Arguments: 3
+- Mutation score: 90%
+- All gates enabled with stricter limits
+
+### prototype
+Relaxed thresholds for prototyping:
+- Cyclomatic: 15, Length: 60, Arguments: 6
+- Mutation score: 60%
+- Mutation testing disabled (expensive)
+
+## Configuration
+
+Create a `validator.config.json` in your project root (future support planned):
+
+```json
+{
+  "profile": "default",
+  "gates": ["type_safety", "complexity", "security"],
+  "exclude": ["**/test/**", "**/fixtures/**"]
+}
+```
+
+Currently, use CLI flags or programmatic API to configure.
+
+## CI Integration
+
+### GitHub Actions
+
+```yaml
+- name: Validate Code Quality
+  run: |
+    npm install -g @nolans01/agent-validator
+    pip install lizard semgrep mypy
+    validator scan --format json --fail-on blocker
+```
+
+### GitLab CI
+
+```yaml
+validate:
+  script:
+    - npm install -g @nolans01/agent-validator
+    - pip install lizard semgrep mypy
+    - validator scan --fail-on blocker
+```
+
+## Exit Codes
+
+- `0`: Validation passed
+- `1`: Validation failed (based on `--fail-on` level)
+
+## Development
+
+```bash
+# Clone the repo
+git clone https://github.com/NolanS-OMG/agent-validator
+cd agent-validator
+
+# Install dependencies
+npm install
+
+# Run tests
+npm test
+
+# Build
+npm run build
+
+# Test locally
+npm link
+validator doctor
+```
+
+## Why agent-validator?
+
+AI coding agents generate code fast, but quality validation is still manual. This tool:
+
+1. **Automates validation**: Run all quality checks in one command
+2. **Deterministic**: Same code = same results (no LLM variance)
+3. **Multi-language**: Works across TypeScript, Python, Ruby
+4. **Fast feedback**: Fail-fast ordering saves time in CI
+5. **Agent-friendly**: JSON output for programmatic parsing
+
+## Roadmap
+
+- [ ] Config file support (`validator.config.json`)
+- [ ] Custom gate plugins
+- [ ] More language support (Go, Rust, Java)
+- [ ] Incremental mode (only changed functions)
+- [ ] HTML report generation
+- [ ] VSCode extension
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions welcome! Please open an issue or PR.