tng 0.4.7 → 0.4.9

data/README.md

@@ -1,12 +1,19 @@
-# TNG - AI-Powered Test Generation for Rails
+# TNG - AI-Powered Test Generation & Code Analysis for Rails
 
-TNG is an AI-powered test generation tool that automatically creates comprehensive tests for your Rails applications. It analyzes your code structure, authentication patterns, and configuration to generate high-quality, contextual tests that follow your project's conventions.
+TNG is a comprehensive AI-powered tool for Rails applications that combines intelligent test generation with advanced static analysis. Analyze your code structure, detect issues, find duplicates, and generate high-quality tests that follow your project's conventions.
 
 **Owner:** Binary Dreams LLC
 
-## Installation
+## Features
+
+- **🧪 AI Test Generation** - Generate comprehensive tests for controllers, models, services, and 20+ file types
+- **🔍 Code Audit** - Deep security, performance, and best-practice analysis
+- **🔬 Symbolic Trace** - Visualize execution paths and logic flow with Mermaid diagrams
+- **📊 Clone Detection** - Find duplicate code with 3 detection levels (Token, Structural, Fuzzy)
+- **💀 Dead Code Detection** - Identify unreachable code, unused variables, and unused private methods
+- **⚡ Blazing Fast** - Powered by Rust for near-instant analysis
 
-### Standard Installation
+## Installation
 
 Add this line to your application's Gemfile:
 
@@ -20,415 +27,378 @@ And then execute:
 bundle install
 ```
 
-## Configuration
+## Quick Start
+
+### 1. Initialize Configuration
 
-TNG requires proper configuration to generate high-quality tests. Generate the configuration file by running:
+Generate the configuration file:
 
 ```bash
 rails generate tng:install
 ```
 
-This creates `config/initializers/tng.rb` with a configuration template that you'll need to customize for your application. The generator will attempt to detect your existing setup and pre-configure common settings.
-
-### Configuration Options Explained
-
-#### API Configuration
-- **`api_key`**: Your TNG API key from https://app.tng.sh/
-- **`base_url`**: TNG service endpoint (automatically configured, do not modify)
-
-#### Code Reading
-- **`read_source_code`**: Analyzes the specific code file for which tests are being generated
-- **`read_test_code`**: Reads existing tests for the specific file to match your testing patterns
-
-#### Testing Framework Settings
-- **`testing_framework`**: Automatically detected. Possible values: `minitest` or `rspec`
-
-*Note: The following settings depend on your testing framework.*
-
-**For Minitest:**
-- **`test_style`**: Test organization style. Possible options:
-    - `test_block` - `test "does something" do ... end`
-    - `spec` - `it "does something" do ... end`
-    - `unit` - `def test_does_something ... end`
-- **`assertion_style`**: Assertion syntax preference. Possible values:
-    - `assert/refute` - Uses syntax like `assert`, `assert_equal`, `assert_nil`, `refute`, `refute_nil`
-    - `assert/assert_not` - Uses syntax like `assert`, `assert_equal`, `assert_nil`, `assert_not`, `assert_not_equl`, `assert_not_nil`
-    - `must/wont` - `value.must_equal`, `value.must_be_instance_of`, `value.must_raise`, `value.wont_equal`, `value.wont_be_instance_of`, `value.wont_raise`
-
-**For RSpec:**
-- **`describe_style`**: Include describe blocks. Possible values: `true`, `false`
-- **`context_style`**: Context block style. Possible values: `context`, `describe`
-- **`it_style`**: Example block style. Possible values: `it`, `specify`
-- **`before_style`**: Setup block style. Possible values: `before`, `setup`
-- **`after_style`**: Teardown block style. Possible values: `after`, `teardown`
-- **`let_style`**: Use let helpers. Possible values: `true`, `false`
-- **`subject_style`**: Use subject helpers. Possible values: `true`, `false`
-
-#### Authentication Methods
-Supported authentication types:
-- **`session`**: Session-based authentication
-- **`devise`**: Devise gem integration
-- **`jwt`**: JSON Web Token authentication
-- **`token_auth`**: Token-based authentication
-- **`basic_auth`**: HTTP Basic Authentication
-- **`oauth`**: OAuth authentication
-- **`headers`**: Custom header authentication
-- **`custom`**: Custom authentication logic
-
-#### Mock and Factory Libraries
-- **`mock_library`**: Mock library preference
-    - **For Minitest**: `minitest/mock`, `mocha`, `nil`
-    - **For RSpec**: `rspec-mocks`, `mocha`, `flexmock`, `nil`
-- **`http_mock_library`**: HTTP mocking library. Possible values: `webmock`, `vcr`, `httparty`, `nil`
-- **`factory_library`**: Factory library preference. Possible values: `factory_bot`, `factory_girl`, `fabrication`, `fabricator`, `fixtures`, `active_record`
-
-## Configuration Examples
-
-### Minitest with FactoryBot Setup
+This creates `config/initializers/tng.rb`:
 
 ```ruby
-# config/initializers/tng.rb
 Tng.configure do |config|
+  # Get your API key from https://app.tng.sh/
   config.api_key = ENV["TNG_API_KEY"]
-  config.base_url = "https://app.tng.sh/"
-  
-  config.read_source_code = true
-  config.read_test_code = true
-  
-  config.testing_framework = "minitest"
-  config.test_style = "test_block"
-  config.assertion_style = "assert/refute"
-  config.setup_style = true
-  config.teardown_style = false
-  
-  config.mock_library = "minitest/mock"
-  config.http_mock_library = "webmock"
-  config.factory_library = "factory_bot"
-  
-  config.authentication_enabled = true
-  config.authentication_methods = [
-    {
-      method: "authenticate_user!",
-      file_location: "app/controllers/application_controller.rb",
-      auth_type: "session"
-    }
-  ]
+
+  # API endpoint (do not modify unless instructed)
+  config.base_url = ENV["API_BASE_URI"] || "https://app.tng.sh/"
 end
 ```
 
-### RSpec with FactoryBot Setup
+### 2. Set Your API Key
 
-```ruby
-# config/initializers/tng.rb
-Tng.configure do |config|
-  config.api_key = ENV["TNG_API_KEY"]
-  config.base_url = "https://app.tng.sh/"
-  
-  config.testing_framework = "rspec"
-  config.assertion_style = "expect"
-  config.describe_style = true
-  config.context_style = "context"
-  config.it_style = "it"
-  config.before_style = "before"
-  config.after_style = "after"
-  config.let_style = true
-  config.subject_style = true
-  
-  config.mock_library = "rspec-mocks"
-  config.http_mock_library = "webmock"
-  config.factory_library = "factory_bot"
-  
-  config.authentication_enabled = true
-  config.authentication_methods = [
-    {
-      method: "authenticate_user!",
-      file_location: "app/controllers/application_controller.rb",
-      auth_type: "devise"
-    }
-  ]
-end
+Add to your `.env` file or export:
+
+```bash
+export TNG_API_KEY=your_api_key_here
 ```
 
-### API-Only Application with Token Authentication
+Get your API key from [https://app.tng.sh/](https://app.tng.sh/)
 
-```ruby
-# config/initializers/tng.rb
-Tng.configure do |config|
-  config.api_key = ENV["TNG_API_KEY"]
-  config.base_url = "https://app.tng.sh/"
-  
-  config.read_source_code = true
-  config.read_test_code = true
-  
-  config.testing_framework = "minitest"
-  config.test_style = "test_block"
-  config.assertion_style = "assert/refute"
-  
-  config.mock_library = "minitest/mock"
-  config.http_mock_library = "webmock"
-  config.factory_library = "fixtures"
-  
-  config.authentication_enabled = true
-  config.authentication_methods = [
-    {
-      method: "authenticate_via_token!",
-      file_location: "app/controllers/application_controller.rb",
-      auth_type: "headers"
-    }
-  ]
-end
-```
+### 3. Start Using TNG
 
-### Multiple Authentication Methods
+```bash
+# Interactive mode (recommended)
+bundle exec tng
 
-```ruby
-# config/initializers/tng.rb
-Tng.configure do |config|
-  config.api_key = ENV["TNG_API_KEY"]
-  config.base_url = "https://app.tng.sh/"
-  config.testing_framework = "rspec"
-  config.assertion_style = "expect"
-  config.let_style = true
-  
-  config.mock_library = "rspec-mocks"
-  config.http_mock_library = "webmock"
-  config.factory_library = "factory_bot"
-  
-  config.authentication_enabled = true
-  config.authentication_methods = [
-    {
-      method: "authenticate_user!",
-      file_location: "app/controllers/application_controller.rb",
-      auth_type: "session"
-    },
-    {
-      method: "authenticate_api_user!",
-      file_location: "app/controllers/api/base_controller.rb",
-      auth_type: "token_auth"
-    }
-  ]
-  
-  config.authorization_library = "pundit"
-end
+# Or use direct commands
+bundle exec tng --file app/models/user.rb --clones
 ```
 
-## Advanced Configuration
+## Usage
 
-### Custom Authentication Setup
+### Interactive Mode (TUI)
 
-For applications with custom authentication logic that doesn't fit standard patterns:
+The most powerful way to use TNG:
 
-```ruby
-config.authentication_methods = [
-  {
-    method: "verify_custom_auth!",
-    file_location: "app/controllers/concerns/custom_auth.rb",
-    auth_type: "custom"
-  }
-]
+```bash
+bundle exec tng
 ```
 
-#### Rails API Application
+**Interactive Features:**
+- 📁 **Browse** your application structure
+- 🧪 **Generate Tests** for specific methods
+- 🔍 **Audit** methods for issues and improvements
+- 🔬 **Trace** symbolic execution paths
+- 📊 **Check Duplicates** across your codebase
+- 💀 **Find Dead Code** in files
+- 📈 **View Stats** and usage metrics
 
-```ruby
-config.authentication_enabled = true
-config.authentication_methods = [
-  {
-    method: "authenticate_with_token",
-    file_location: "app/controllers/application_controller.rb",
-    auth_type: "jwt"
-  }
-]
-config.factory_library = "fixtures"  # Often simpler for API apps
-```
+### Command Line Interface
 
-#### Multi-Tenant Application
+#### Test Generation
 
-```ruby
-config.authentication_methods = [
-  {
-    method: "authenticate_tenant!",
-    file_location: "app/controllers/application_controller.rb",
-    auth_type: "custom"
-  },
-  {
-    method: "authenticate_user!",
-    file_location: "app/controllers/application_controller.rb",
-    auth_type: "devise"
-  }
-]
+```bash
+# Generate test for a specific method
+bundle exec tng --file app/controllers/users_controller.rb --method index
+
+# Short form
+bundle exec tng -f users_controller.rb -m show
+
+# Works from subdirectories (auto-detection)
+bundle exec tng users_controller.rb index
 ```
 
-#### Legacy Application with Mixed Authentication
+**Supported File Types:**
+Controllers, Models, Services, Jobs, Helpers, Lib, Policies, Presenters, Mailers, Serializers, Decorators, Forms, Queries, Channels, GraphQL (Resolvers, Types, Mutations, Loaders, Schemas), and more.
 
-```ruby
-config.authentication_methods = [
-  {
-    method: "authenticate_admin!",
-    file_location: "app/controllers/admin/base_controller.rb",
-    auth_type: "basic_auth"
-  },
-  {
-    method: "authenticate_user!",
-    file_location: "app/controllers/application_controller.rb",
-    auth_type: "session"
-  },
-  {
-    method: "authenticate_api_key!",
-    file_location: "app/controllers/api/v1/base_controller.rb",
-    auth_type: "headers"
-  }
-]
+#### Code Audit
+
+Analyze code for security, performance, and best practices:
+
+```bash
+# Audit a specific method
+bundle exec tng --file app/services/payment_service.rb --method process_payment --audit
+
+# Interactive audit mode
+bundle exec tng -f payment_service.rb -m process_payment -a
 ```
 
-## Usage Patterns
+**Audit Checks:**
+- Security vulnerabilities
+- Performance bottlenecks
+- Code smells
+- Best practice violations
+- Rails-specific patterns
 
-### Interactive Mode
+#### Symbolic Trace
 
-The recommended way to use TNG for beginners and exploratory testing:
+Visualize execution paths and generate Mermaid diagrams:
 
 ```bash
-bundle exec tng
+# Trace a method's execution flow
+bundle exec tng --file app/models/order.rb --method calculate_total --trace
+
+# Short form
+bundle exec tng -f order.rb -m calculate_total -t
 ```
 
-This launches an interactive session where you can:
-- Browse your application structure
-- Select files to generate tests for
-- Preview generated tests before saving
-- Adjust configuration on the fly
+**Trace Features:**
+- Control flow visualization
+- Conditional path analysis
+- Loop detection
+- Method call chains
+- Mermaid diagram generation
+
+#### Clone Detection
 
-### Method-Specific Testing
+Find duplicate code with 3 levels of detection:
 
-TNG focuses on precise, method-level test generation:
+```bash
+# Check for all types of clones
+bundle exec tng --file app/services/pricing_service.rb --clones
 
-- **Select specific methods** from 20+ file types including controllers, models, services, jobs, helpers, lib, policies, presenters, mailers, GraphQL components, and more
-- **Interactive browsing** with search and filter capabilities
-- **Focused test generation** for individual methods only
-- **No bulk generation** - intentionally designed for precision
+# Specific detection level
+bundle exec tng --file app/services/pricing_service.rb --clones --level=1
+bundle exec tng --file app/services/pricing_service.rb --clones --level=2
+bundle exec tng --file app/services/pricing_service.rb --clones --level=3
 
-Use `bundle exec tng --help` for more options.
+# Short form
+bundle exec tng -f pricing_service.rb -c
+```
 
-### Direct Mode (for Developers)
+**Detection Levels:**
+- **Level 1 (Token)** - Exact or near-exact duplicates (copy-paste detection)
+- **Level 2 (Structural)** - Same AST structure with different names
+- **Level 3 (Fuzzy)** - Similar logic with variations (up to 20% difference)
 
-For experienced developers who want quick, command-line test generation:
+#### Dead Code Detection
+
+Identify unreachable code and unused elements:
 
 ```bash
-# Generate test for specific method using file path
-bundle exec tng app/controllers/users_controller.rb index
+# Detect dead code in a file
+bundle exec tng --file app/models/user.rb --deadcode
 
-# Short form using shortcuts
-bundle exec tng f=users_controller.rb m=show
+# Short form
+bundle exec tng -f user.rb -d
 
-# Full parameter names
-bundle exec tng --file=app/models/user.rb --method=validate
+# JSON output for CI/CD
+bundle exec tng -f user.rb --deadcode --json
+```
 
-# Works from subdirectories (auto-detection)
-bundle exec tng users_controller.rb index
+**Dead Code Types:**
+- **Unreachable Code** - Code after `return`, `raise`, `exit`, etc.
+- **Unused Variables** - Local variables assigned but never used
+- **Unused Private Methods** - Private methods never called
+- **Unused Parameters** - Method parameters never referenced
+
+### JSON Output Mode
+
+Get machine-readable output for CI/CD integration:
 
-# Mixed parameter order
-bundle exec tng m=index f=app/controllers/users_controller.rb
+```bash
+# Any command with --json flag
+bundle exec tng --file app/models/user.rb --clones --json
+bundle exec tng --file app/services/payment.rb --deadcode --json
+bundle exec tng --file app/controllers/api_controller.rb --trace --json
 ```
 
-**Direct Mode Features:**
-- **Automatic file type detection** - No need to specify `-t controller/model/service`
-- **Flexible file path resolution** - Works with full paths or just filenames
-- **Multiple syntax formats** - Use `f=` shortcuts or `--file=` full parameters
-- **Smart path finding** - Automatically searches Rails directories
+## Advanced Features
+
+### Method-Specific Analysis
+
+TNG focuses on precise, method-level analysis:
+
+- **Select specific methods** from any file type
+- **Interactive browsing** with search and filter
+- **Focused generation** for individual methods
+- **No bulk generation** - designed for precision
+
+### File Type Auto-Detection
+
+No need to specify file types - TNG automatically detects:
+
+```bash
+# All of these work automatically
+bundle exec tng users_controller.rb index
+bundle exec tng user.rb validate_email
+bundle exec tng payment_service.rb process
+bundle exec tng user_mailer.rb welcome_email
+bundle exec tng user_resolver.rb find_user
+```
 
 ### Debug Mode
 
-For troubleshooting configuration or generation issues:
+Enable detailed logging for troubleshooting:
 
 ```bash
-# Interactive mode with debug on
 DEBUG=1 bundle exec tng
+DEBUG=1 bundle exec tng -f users_controller.rb -m index
+```
+
+## Examples
+
+### Example 1: Find Duplicate Code
+
+```bash
+$ bundle exec tng --file app/services/pricing_service.rb --clones
+
+🔍 Clone Detection Results
 
-# Direct mode with debug
-DEBUG=1 bundle exec tng app/controllers/users_controller.rb index
+Level 1 (Token): 2 matches found
+  ├─ Lines 45-52 ↔ Lines 78-85 (8 lines, exact match)
+  └─ Lines 120-127 ↔ Lines 156-163 (8 lines, exact match)
+
+Level 2 (Structural): 1 match found
+  └─ Lines 200-215 ↔ Lines 240-255 (16 nodes, complexity: 85)
+
+Level 3 (Fuzzy): 1 match found
+  └─ Lines 300-320 ↔ Lines 350-370 (similar logic, 15% variation)
 ```
 
-### Help and Options
+### Example 2: Detect Dead Code
 
 ```bash
-bundle exec tng --help              # Show help message
-bundle exec tng -h                  # Short help
+$ bundle exec tng --file app/models/user.rb --deadcode
+
+💀 Dead Code Detection Results
+
+[Unreachable Code] Line 42: Code follows terminal statement
+   42:   puts "This will never execute"
+
+[Unused Variable] Line 67: Variable 'temp' is assigned but never used
+   67:   temp = calculate_something
+
+[Unused Method] Line 120: Private method 'unused_helper' is never used
+   120:   def unused_helper
+
+[Unused Parameter] Line 85: Parameter 'options' is never used
+   85:   def process(data, options)
 ```
 
-## Troubleshooting
+### Example 3: Trace Execution Flow
 
-### Configuration Issues
+```bash
+$ bundle exec tng --file app/models/order.rb --method calculate_total --trace
+
+🔬 Symbolic Trace Results
+
+Execution Paths: 3
+Complexity Score: 12
+
+Path 1: items.empty? → return 0
+Path 2: discount_code.present? → apply_discount → calculate_tax → total
+Path 3: !discount_code.present? → calculate_tax → total
 
-#### API Key Problems
+[Mermaid diagram saved to: trace_order_calculate_total.md]
+```
+
+## CLI Reference
+
+### Flags
+
+| Flag | Short | Description |
+|------|-------|-------------|
+| `--file` | `-f` | Target file path |
+| `--method` | `-m` | Method name to analyze |
+| `--audit` | `-a` | Run audit mode |
+| `--trace` | `-t` | Run symbolic trace mode |
+| `--clones` | `-c` | Run clone detection |
+| `--deadcode` | `-d` | Run dead code detection |
+| `--level` | `-l` | Clone detection level (1, 2, 3, or all) |
+| `--json` | | Output as JSON |
+| `--help` | `-h` | Show help message |
+
+### Examples
+
+```bash
+# Test generation
+bundle exec tng -f users_controller.rb -m index
+
+# Code audit
+bundle exec tng -f payment_service.rb -m process -a
+
+# Symbolic trace
+bundle exec tng -f order.rb -m calculate_total -t
+
+# Clone detection (all levels)
+bundle exec tng -f pricing_service.rb -c
+
+# Clone detection (specific level)
+bundle exec tng -f pricing_service.rb -c -l 2
+
+# Dead code detection
+bundle exec tng -f user.rb -d
+
+# JSON output
+bundle exec tng -f user.rb -c --json
+```
+
+## Troubleshooting
+
+### API Key Issues
 
 **Error: "Invalid API key"**
 - Verify your API key from https://app.tng.sh/
-- Check environment variable: `echo $TNG_API_KEY`
-- Ensure no extra spaces in `.env` file: `TNG_API_KEY=your_key_here`
+- Check: `echo $TNG_API_KEY`
+- Ensure no extra spaces in `.env`: `TNG_API_KEY=your_key_here`
 
 **Error: "API key not found"**
 - Set environment variable: `export TNG_API_KEY=your_key`
 - Or add to `.env` file: `TNG_API_KEY=your_key`
-- Restart Rails server after changing environment variables
+- Restart Rails server after changes
 
-#### Configuration Validation
+### Configuration Issues
 
 **Error: "Invalid configuration"**
-- Run configuration check: `rails console` then `Tng.configuration`
-- Verify all required settings are present
+- Run: `rails console` then `Tng.configuration`
+- Verify `config/initializers/tng.rb` exists
 - Check for typos in configuration file
 
-**Poor Test Quality**
-- Verify authentication methods are correctly configured
-- Check factory library setting matches your setup
-- Ensure `read_source_code` and `read_test_code` are enabled
-- Review testing framework settings match your project
-
-#### Authentication Configuration Issues
-
-**Tests don't include authentication**
-- Verify `authentication_enabled = true`
-- Check authentication methods are correctly specified
-- Ensure file paths exist: `ls app/controllers/application_controller.rb`
-- Verify method names exist in specified files
-
-**Authentication tests fail**
-- Check auth method names match your application
-- Verify file locations are correct
-- Ensure auth_type matches your authentication pattern
-
 ### Generation Issues
 
-#### No Tests Generated
-
-**Check these common issues:**
-1. File path is correct: `ls app/controllers/users_controller.rb`
-2. File contains valid Ruby code
-3. Configuration is valid
-4. API key is working
-5. Internet connection is available
+**No tests generated:**
+1. Verify file path: `ls app/controllers/users_controller.rb`
+2. Check file contains valid Ruby code
+3. Verify API key is working
+4. Check internet connection
 
-#### Generated Tests Are Generic
-
-**Common causes:**
-- Authentication not properly configured
-- Factory library setting doesn't match your setup
-- `read_source_code` or `read_test_code` disabled
-- File analysis failed (check DEBUG mode)
+**Poor test quality:**
+- Ensure you're using the latest version
+- Try audit mode for better insights
+- Check file is properly formatted
 
 ### Debug Mode
 
-Enable debug mode to get detailed information:
+Enable debug output for detailed information:
 
 ```bash
-DEBUG=1 bundle exec tng app/controllers/your_controller.rb method_name
+DEBUG=1 bundle exec tng -f users_controller.rb -m index
 ```
 
 Debug output includes:
-- API request/response information
-
-### Getting Help
-
-If you're still experiencing issues: Contact the gem maintainers or open a GitHub issue for support
+- API request/response details
+- File analysis results
+- Configuration validation
+- Error stack traces
+
+## What's New in v0.4.8
+
+### New Features
+- ✨ **Level 3 Fuzzy Clone Detection** - Find similar code with variations
+- ✨ **Dead Code Detection** - Identify unreachable code and unused elements
+- ✨ **Simplified Configuration** - Only API key required to get started
+- ✨ **Enhanced Clone Detection** - 3 levels of duplicate detection
+- ✨ **Improved UI** - Better visualization of results
+
+### Improvements
+- ⚡ Faster file analysis with optimized Rust parsers
+- 🎨 Better error messages and user feedback
+- 📊 Enhanced reporting for all analysis modes
+- 🔧 Auto-detection of file types and test frameworks
+
+## Getting Help
+
+If you're experiencing issues:
+- Check the troubleshooting section above
+- Enable debug mode: `DEBUG=1 bundle exec tng`
+- Contact support at https://app.tng.sh/
+- Open a GitHub issue for bug reports
 
 ## License