codesavant 3.4.0 → 4.0.0

package/README.md

@@ -0,0 +1,826 @@
+# CodeSavant
+
+> AI-powered autonomous coding agent CLI that brings Claude, GPT-4, and other leading AI models directly to your terminal. Features autonomous workflows, a skill marketplace, self-improving brain, and design intelligence.
+
+[![NPM Version](https://img.shields.io/npm/v/codesavant.svg)](https://www.npmjs.com/package/codesavant)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Powered by Bun](https://img.shields.io/badge/Powered%20by-Bun-black)](https://bun.sh)
+
+---
+
+## What's New in v4.0
+
+- **Autonomous /craft Cycle** - End-to-end feature building: research, question, plan, build, verify -- all orchestrated automatically
+- **Skill Marketplace** - Browse, install, and publish reusable AI skills with version management and hash verification
+- **Self-Improving Brain** - Cross-session learning that detects patterns, captures failures, and optimizes over time
+- **Design Intelligence Foundation** - Frontend detection, design token vocabulary, and DesignBrief schemas for style-aware code generation
+- **360+ Test Files** - Comprehensive test coverage across unit, integration, e2e, performance, and security suites
+
+---
+
+## Features
+
+- **Multi-Provider Support** - Works with Anthropic Claude, OpenAI GPT-4, Groq, DeepSeek, Google Gemini, Ollama, and AWS Bedrock
+- **Interactive REPL** - Rich terminal interface with streaming responses and real-time tool execution
+- **20+ Built-in Tools** - File operations, web search, code analysis, task management
+- **Autonomous Agent** - Self-directed research, questioning, planning, and building via /craft
+- **Skill Marketplace** - Browse, install, and publish community skills
+- **Self-Improving Brain** - Pattern detection, failure capture, and cross-session learning
+- **Design Intelligence** - Frontend detection, style vocabulary, and design token resolution
+- **Craft Skills** - AI-powered workflows for TDD, debugging, planning, and code review
+- **Session Management** - Save, resume, and checkpoint your conversations
+- **MCP Integration** - Model Context Protocol support for extensibility
+- **Security First** - Permission system with SSRF protection and command validation
+- **Context Management** - Automatic token tracking and conversation compression
+
+---
+
+## Installation
+
+CodeSavant requires [Bun](https://bun.sh) runtime (v1.0.0+).
+
+### Install Bun
+
+```bash
+# macOS, Linux, WSL
+curl -fsSL https://bun.sh/install | bash
+
+# Or with npm
+npm install -g bun
+```
+
+### Install CodeSavant
+
+```bash
+# Global installation
+bun install -g codesavant
+
+# Or with npm
+npm install -g codesavant
+```
+
+---
+
+## Quick Start
+
+### 1. Launch Interactive REPL
+
+```bash
+codesavant
+```
+
+On first run, you'll be prompted to configure your preferred AI provider:
+
+```
+Welcome to CodeSavant!
+No providers configured. Let's set one up.
+
+Which provider would you like to configure?
+1. Anthropic (Claude)
+2. OpenAI (GPT-4)
+3. Groq (Fast inference)
+4. Ollama (Local models)
+> 1
+
+Enter your Anthropic API key: sk-ant-...
+✓ API key saved!
+```
+
+### 2. Start Chatting
+
+```
+> Hello! I need help refactoring my TypeScript code.
+
+│ I'd be happy to help you refactor your TypeScript code! Could you share
+│ the code you'd like to refactor, along with any specific goals you have?
+
+> Read src/utils/parser.ts and suggest improvements
+
+[Tool: Read] ✓ Complete
+file: src/utils/parser.ts
+
+│ Based on the code, here are my suggestions for refactoring:
+│ 1. Extract magic numbers into constants
+│ 2. Add explicit return types
+│ 3. Use more descriptive variable names...
+```
+
+---
+
+## Usage
+
+### Command-Line Interface
+
+```bash
+# Interactive REPL (default)
+codesavant
+
+# Single command execution
+codesavant -p "explain this error"
+
+# Pipe content as context
+cat error.log | codesavant --pipe
+
+# Resume a previous session
+codesavant -r abc123
+
+# Specify provider and model
+codesavant --provider openai --model gpt-4o
+
+# Plan mode (read-only tools)
+codesavant --plan
+
+# YOLO mode (auto-approve all)
+codesavant --yolo
+
+# Strict mode (approve everything)
+codesavant --strict
+```
+
+### CLI Options
+
+| Option | Description | Example |
+|--------|-------------|---------|
+| `-p, --print <prompt>` | Execute prompt and exit | `codesavant -p "fix bug"` |
+| `--pipe` | Read prompt from stdin | `cat file \| codesavant --pipe` |
+| `-r, --resume <id>` | Resume previous session | `codesavant -r abc123` |
+| `--provider <name>` | Set AI provider | `codesavant --provider anthropic` |
+| `--model <name>` | Set model | `codesavant --model gpt-4o` |
+| `--plan` | Read-only mode | `codesavant --plan` |
+| `--yolo` | Auto-approve all tools | `codesavant --yolo` |
+| `--strict` | Approve all tools manually | `codesavant --strict` |
+| `--cwd <dir>` | Set working directory | `codesavant --cwd /project` |
+| `-o, --output-format` | Output format (text/json/markdown) | `codesavant -o json` |
+| `--version` | Show version | `codesavant --version` |
+| `--help` | Show help | `codesavant --help` |
+
+---
+
+## REPL Commands
+
+Once in the interactive REPL, use these slash commands:
+
+### Session Management
+
+```bash
+/session              # Show current session info
+/session list         # List recent sessions
+/resume [#|id]        # Resume a session
+/checkpoint [name]    # Create checkpoint
+/clear                # Clear conversation
+/exit                 # Exit REPL
+```
+
+### Configuration & Status
+
+```bash
+/config               # Show configuration
+/status               # Show session status
+/provider             # Display current provider
+/model                # Display current model
+/permissions [mode]   # Show/set permissions
+/login                # Show auth status
+/doctor               # Run health checks
+```
+
+### Context & Memory
+
+```bash
+/memory               # Show project memory
+/context              # Show context usage
+/history [count]      # Show recent messages
+/cost                 # Show token usage
+/compact              # Compress conversation
+```
+
+### Features & Tools
+
+```bash
+/help                 # Show all commands
+/skills               # List available skills
+/tasks                # Display task list
+/mcp                  # Show MCP server status
+/plan                 # Toggle plan mode
+/rewind [args]        # Revert file changes
+```
+
+---
+
+## Supported AI Providers
+
+### Anthropic Claude
+
+```bash
+# Set API key
+export ANTHROPIC_API_KEY="sk-ant-..."
+
+# Use Claude
+codesavant --provider anthropic --model claude-sonnet-4-20250514
+```
+
+**Models:**
+- `claude-sonnet-4-20250514` (recommended)
+- `claude-3-opus-20240229`
+- `claude-3-sonnet-20240229`
+- `claude-3-haiku-20240307`
+
+### OpenAI GPT-4
+
+```bash
+# Set API key
+export OPENAI_API_KEY="sk-..."
+
+# Use GPT-4
+codesavant --provider openai --model gpt-4o
+```
+
+**Models:**
+- `gpt-4o` (recommended)
+- `gpt-4o-mini`
+- `gpt-4-turbo`
+- `gpt-4`
+- `gpt-3.5-turbo`
+
+### Groq (Fast Inference)
+
+```bash
+# Set API key
+export GROQ_API_KEY="gsk_..."
+
+# Use Groq
+codesavant --provider groq --model llama-3.1-70b-versatile
+```
+
+**Models:**
+- `llama-3.1-70b-versatile`
+- `llama-3.2-90b-vision-preview`
+- `mixtral-8x7b-32768`
+
+### Google Gemini
+
+```bash
+# Set API key
+export GOOGLE_API_KEY="AIza..."
+
+# Use Gemini
+codesavant --provider gemini --model gemini-2.0-flash-exp
+```
+
+**Models:**
+- `gemini-2.0-flash-exp`
+- `gemini-1.5-pro`
+- `gemini-1.5-flash`
+
+### DeepSeek
+
+```bash
+# Set API key
+export DEEPSEEK_API_KEY="sk-..."
+
+# Use DeepSeek
+codesavant --provider deepseek --model deepseek-chat
+```
+
+**Models:**
+- `deepseek-chat`
+- `deepseek-coder`
+
+### Ollama (Local Models)
+
+```bash
+# Start Ollama server
+ollama serve
+
+# Pull a model
+ollama pull llama3.2
+
+# Use Ollama
+codesavant --provider ollama --model llama3.2
+```
+
+**Models:** Any Ollama model (llama3.2, codellama, mistral, etc.)
+
+### AWS Bedrock
+
+```bash
+# Configure AWS credentials
+aws configure
+
+# Use Bedrock
+codesavant --provider bedrock --model anthropic.claude-3-sonnet-20240229-v1:0
+```
+
+---
+
+## Autonomous /craft Workflows
+
+The `/craft` command provides fully autonomous AI workflows that research, plan, build, and verify features end-to-end.
+
+### Process Skills (Use First)
+
+```bash
+# Brainstorm and explore requirements
+/craft ideate <feature description>
+
+# Debug systematically
+/craft diagnose <bug description>
+```
+
+### Implementation Skills (During Work)
+
+```bash
+# Test-driven development
+/craft proof <feature>
+
+# Create implementation plan
+/craft blueprint <feature>
+
+# Execute plan step-by-step
+/craft build <plan>
+
+# Parallel subagent execution
+/craft assembly-line <plan>
+
+# Create isolated git worktree
+/craft workspace <feature-name>
+```
+
+### Quality Skills (After Work)
+
+```bash
+# Verify work with evidence
+/craft inspect <claim>
+
+# Code review checklist
+/craft review
+
+# Final deployment checks
+/craft ship
+```
+
+### Full Autonomous Cycle
+
+```bash
+# Fully autonomous: research → question → plan → build → verify
+/craft <feature description>
+```
+
+The craft orchestrator spawns research agents in parallel, asks targeted questions, builds a development roadmap, executes plans with atomic commits, and verifies output quality -- all autonomously.
+
+---
+
+## Skill Marketplace
+
+Browse and install community-created skills, or publish your own.
+
+```bash
+# Browse available skills
+/skills marketplace
+
+# Install a skill
+/skills install <skill-name>
+
+# Publish your skill
+/skills publish <skill-path>
+```
+
+Skills are verified with SHA-256 hash integrity checks, support semantic versioning, and handle slug collisions gracefully.
+
+---
+
+## Self-Improving Brain
+
+CodeSavant learns across sessions:
+
+- **Pattern Detection** - Identifies recurring coding patterns and preferences
+- **Failure Capture** - Records what went wrong and how it was fixed
+- **Optimization** - Detects opportunities to improve workflows
+- **Memory Resolution** - Three-tier priority system (project > user > global)
+
+The brain operates transparently -- it writes insights after sessions and reads them before the next one, getting better over time without manual configuration.
+
+---
+
+## Design Intelligence
+
+CodeSavant automatically detects frontend work and applies design-aware context:
+
+- **Frontend Detection** - Identifies frontend tasks from file extensions (.tsx, .jsx, .css, .vue, .svelte), import patterns, and intent keywords
+- **Style Vocabulary** - 50+ design buzzwords (minimal, bold, cinematic, etc.) mapped to concrete design tokens
+- **Design Brief Schema** - Structured, validated design preferences with opinionated defaults
+- **Token Resolution** - Translates style buzzwords into actionable token hints (border-radius, spacing, contrast, etc.)
+
+---
+
+## Built-in Tools
+
+CodeSavant provides 20+ tools that AI can use to help you:
+
+### File Operations
+
+| Tool | Description | Example Use |
+|------|-------------|-------------|
+| **Read** | Read file contents | "Read src/main.ts" |
+| **Write** | Create/overwrite files | "Create a new config.json" |
+| **Edit** | Modify existing files | "Update the port in server.ts to 3000" |
+| **Glob** | Find files by pattern | "Find all .tsx files" |
+| **Grep** | Search file contents | "Search for TODO comments" |
+
+### Code & Execution
+
+| Tool | Description | Example Use |
+|------|-------------|-------------|
+| **Bash** | Execute shell commands | "Run the tests" |
+| **LSP** | Code analysis | "Get diagnostics for this file" |
+| **Notebook** | Jupyter notebook ops | "Read notebook cells" |
+
+### Web & Search
+
+| Tool | Description | Example Use |
+|------|-------------|-------------|
+| **WebSearch** | Search the internet | "Search for React 18 docs" |
+| **WebFetch** | Fetch & analyze pages | "Fetch and summarize this article" |
+
+### Task Management
+
+| Tool | Description | Example Use |
+|------|-------------|-------------|
+| **TaskCreate** | Create structured tasks | Used by workflows |
+| **TaskList** | List all tasks | "Show my tasks" |
+| **TaskUpdate** | Update task status | Automatic during work |
+
+### AI Features
+
+| Tool | Description | Example Use |
+|------|-------------|-------------|
+| **Task (Subagent)** | Spawn parallel agents | For complex multi-step work |
+| **Skill** | Execute custom skills | Via /craft commands |
+| **AskUser** | Interactive questions | AI asks for clarification |
+
+---
+
+## Configuration
+
+### Settings File
+
+CodeSavant uses `.codesavant/settings.json`:
+
+```json
+{
+  "providers": {
+    "anthropic": {
+      "apiKey": "sk-ant-..."
+    },
+    "openai": {
+      "apiKey": "sk-..."
+    }
+  },
+  "defaultProvider": "anthropic",
+  "defaultModel": "claude-sonnet-4-20250514",
+  "hooks": {
+    "session_start": "echo 'Session started!'",
+    "user_prompt_submit": "git status"
+  },
+  "permissions": {
+    "mode": "default"
+  }
+}
+```
+
+### Project Memory (SAVANT.md)
+
+Create `.codesavant/SAVANT.md` for project context:
+
+```markdown
+# Project: MyApp
+
+## Architecture
+- Next.js 14 app router
+- PostgreSQL database
+- tRPC API layer
+
+## Coding Standards
+- Use TypeScript strict mode
+- Prefer functional components
+- Write tests for all features
+```
+
+AI will read this automatically for project context.
+
+### Custom Skills
+
+Create custom skills in `.codesavant/skills/`:
+
+```markdown
+---
+name: deploy
+category: implementation
+description: Deploy to production
+---
+
+# Deploy Skill
+
+Follow these steps to deploy:
+
+1. Run tests: `bun test`
+2. Build: `bun run build`
+3. Deploy: `vercel deploy --prod`
+```
+
+---
+
+## Security
+
+CodeSavant includes comprehensive security features:
+
+### Permission Modes
+
+- **Default** - Approve destructive operations interactively
+- **Plan** (`--plan`) - Read-only, no file modifications
+- **YOLO** (`--yolo`) - Auto-approve (use with caution!)
+- **Strict** (`--strict`) - Approve all tools manually
+
+### Built-in Protections
+
+- **SSRF Protection** - Blocks private IPs, localhost in WebFetch
+- **Path Validation** - Prevents traversal attacks in file operations
+- **Command Safety** - Array-based command spawning (no shell injection)
+- **Secrets Handling** - API keys from environment/config only
+- **Tool Risk Assessment** - High-risk tools require approval
+
+---
+
+## Advanced Features
+
+### Hooks System
+
+Execute commands at key lifecycle events:
+
+```json
+{
+  "hooks": {
+    "session_start": "git pull origin main",
+    "user_prompt_submit": "git status",
+    "session_end": "git stash"
+  }
+}
+```
+
+### MCP Server Integration
+
+Connect external tools via Model Context Protocol:
+
+```json
+{
+  "mcpServers": {
+    "github": {
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-github"],
+      "env": {
+        "GITHUB_TOKEN": "ghp_..."
+      }
+    }
+  }
+}
+```
+
+### Background Tasks
+
+Run long operations in the background:
+
+```bash
+> /bg npm run build
+
+│ Task started in background (ID: bg_abc123)
+│ You can continue working...
+
+> /tasks
+
+│ Active Tasks:
+│ ● bg_abc123: npm run build (Running, 45s)
+```
+
+### File Change Tracking
+
+Revert changes from previous operations:
+
+```bash
+> /rewind
+
+│ Recent file changes:
+│ 1. src/config.ts (2 min ago)
+│ 2. src/auth.ts (5 min ago)
+│
+│ Revert which file? (1-2, or 'all')
+
+> 1
+
+[Tool: Bash] ✓ Complete
+command: git restore src/config.ts
+
+│ ✓ Reverted src/config.ts
+```
+
+---
+
+## Examples
+
+### Example 1: Autonomous Feature Building
+
+```bash
+codesavant
+
+> /craft user authentication with JWT
+
+│ Starting autonomous craft cycle...
+│
+│ [Research] Spawning 4 parallel research agents...
+│ ✓ Stack research complete
+│ ✓ Architecture research complete
+│ ✓ Security research complete
+│ ✓ Patterns research complete
+│
+│ [Questioning] I have 2 questions before building:
+│ 1. Do you want refresh token rotation?
+│ 2. Session storage: Redis or in-memory?
+│
+> Yes to refresh tokens, use Redis
+
+│ [Planning] Creating development roadmap...
+│ [Building] Executing 3 plans with atomic commits...
+│ [Verifying] Running quality checks...
+│
+│ ✓ Feature complete: 12 files, 8 commits, all tests passing
+```
+
+### Example 2: TDD Workflow
+
+```bash
+codesavant
+
+> /craft proof login endpoint
+
+│ Starting TDD workflow. Writing failing tests first...
+
+[Tool: Write] ✓ Complete
+file: tests/api/login.test.ts
+
+│ ✗ 4 tests failing (expected). Now implementing minimal code to pass...
+
+[Tool: Write] ✓ Complete
+file: src/api/login.ts
+
+│ ✓ All 4 tests passing!
+```
+
+### Example 3: Pipe Mode
+
+```bash
+# Analyze an error log
+cat error.log | codesavant --pipe -p "analyze this error and suggest a fix"
+
+# Review a diff
+git diff | codesavant --pipe -p "review these changes"
+
+# Explain code
+cat complex-algorithm.ts | codesavant --pipe -p "explain this algorithm step by step"
+```
+
+### Example 4: Session Management
+
+```bash
+# Start working on a feature
+codesavant
+
+> I need to add OAuth authentication
+
+│ Let's plan the OAuth implementation...
+│ [Long conversation with code changes]
+
+> /checkpoint oauth-implementation
+
+│ ✓ Checkpoint created: oauth-implementation
+
+> /exit
+
+# Later, resume the session
+codesavant -r
+```
+
+---
+
+## Troubleshooting
+
+### Provider Not Working
+
+```bash
+# Check authentication
+codesavant
+> /login
+
+# Run diagnostics
+> /doctor
+
+│ Running health checks...
+│ ✓ Provider: anthropic (authenticated)
+│ ✓ Storage: ~/.codesavant/ (writable)
+│ ✓ Working directory: /project (writable)
+│ ✓ All systems operational
+```
+
+### API Rate Limits
+
+```bash
+> /cost
+
+│ Token Usage (This Session):
+│ - Input:  12,450 tokens
+│ - Output: 8,920 tokens
+│ - Total:  21,370 tokens
+│
+│ Estimated Cost: $0.32
+```
+
+---
+
+## Contributing
+
+We welcome contributions! CodeSavant is open source under the MIT license.
+
+### Development Setup
+
+```bash
+# Clone repository
+git clone https://github.com/vaggarwal039/codesavant.git
+cd codesavant
+
+# Install dependencies
+bun install
+
+# Run in development
+bun run src/cli.ts
+
+# Run tests
+bun test
+
+# Build for production
+bun run build
+```
+
+### Project Structure
+
+```
+codesavant/
+├── src/
+│   ├── cli.ts              # CLI entry point
+│   ├── repl-ink.tsx         # Interactive REPL
+│   ├── agent.ts             # AI agent orchestration
+│   ├── providers/           # AI provider implementations
+│   ├── tools/               # Built-in tools
+│   ├── skills/              # Unified skill system
+│   ├── craft/               # Craft skill engine
+│   ├── craft-orchestrator/  # Autonomous build cycle
+│   ├── marketplace/         # Skill marketplace
+│   ├── brain/               # Self-improving brain
+│   ├── design/              # Design intelligence
+│   ├── components/          # UI components
+│   └── ...
+├── tests/                   # Test suite (360+ test files)
+├── docs/                    # Documentation
+└── package.json
+```
+
+---
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+---
+
+## Acknowledgments
+
+Built with:
+- [Bun](https://bun.sh) - Fast JavaScript runtime
+- [Ink](https://github.com/vadimdemedes/ink) - React for CLIs
+- [Commander.js](https://github.com/tj/commander.js) - CLI framework
+- [Chalk](https://github.com/chalk/chalk) - Terminal styling
+- [Zod](https://zod.dev) - TypeScript schema validation
+
+Powered by:
+- [Anthropic Claude](https://anthropic.com)
+- [OpenAI GPT-4](https://openai.com)
+- [Google Gemini](https://deepmind.google/technologies/gemini/)
+- [Groq](https://groq.com)
+- [DeepSeek](https://deepseek.com)
+
+---
+
+## Support
+
+- **Issues**: [GitHub Issues](https://github.com/vaggarwal039/codesavant/issues)
+- **Discussions**: [GitHub Discussions](https://github.com/vaggarwal039/codesavant/discussions)
+
+---
+
+**Made with care by [Vishal Aggarwal](https://github.com/vaggarwal039)**