synthra 0.1.0

data/docs/MOCK_SERVER.md

@@ -0,0 +1,280 @@
+# Mock Server
+
+A **production-ready mock server** with API recording and replay capabilities.
+
+## Quick Start
+
+```ruby
+# Start mock server
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  port: 3000
+)
+```
+
+## Features
+
+- 🎲 **Schema-based generation** - Generate data from your DSL schemas
+- 📼 **Recording mode** - Record real API responses for replay
+- ⏱️ **Behavior simulation** - Simulate latency and failures
+- 🔒 **Rate limiting** - Protect against overuse
+- 🌐 **CORS support** - Ready for frontend development
+
+## Configuration
+
+```ruby
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  port: 3000,
+  recording: true,                # Enable recording
+  recordings_dir: "recordings/",  # Where to save recordings
+  cors: true,                     # Enable CORS
+  rate_limit: 100,                # Requests per minute (0 = unlimited)
+  default_mode: :random           # Default generation mode
+)
+```
+
+## API Endpoints
+
+### Schema Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/schemas` | List all schemas |
+| GET | `/api/:schema` | Generate single record |
+| GET | `/api/:schema/batch` | Generate batch |
+| GET | `/api/:schema/stream` | Stream NDJSON |
+| POST | `/api/:schema` | Generate with overrides |
+| GET | `/api/:schema/schema` | Get schema info |
+
+### Recording Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/api/recordings/:name` | Save a recording |
+| GET | `/api/recordings/:name` | Replay a recording |
+
+## Generation Parameters
+
+All generation endpoints accept:
+
+- `seed` - Random seed for deterministic output
+- `mode` - Generation mode: random, edge, invalid, hostile, mixed
+- `count` - Number of records (batch/stream only)
+
+### Examples
+
+```bash
+# Generate with seed
+curl "http://localhost:3000/api/user?seed=42"
+
+# Generate edge cases
+curl "http://localhost:3000/api/user?mode=edge"
+
+# Generate batch
+curl "http://localhost:3000/api/user/batch?count=100&seed=42"
+```
+
+## Recording & Replay
+
+### Save Recording
+
+```bash
+# Record a response manually
+curl -X POST http://localhost:3000/api/recordings/users_list \
+  -H "Content-Type: application/json" \
+  -d '{
+    "data": [
+      {"id": 1, "name": "Real User 1"},
+      {"id": 2, "name": "Real User 2"}
+    ],
+    "content_type": "application/json"
+  }'
+```
+
+### Replay Recording
+
+```bash
+# Get the recorded response
+curl http://localhost:3000/api/recordings/users_list
+```
+
+### Automatic Recording
+
+With `recording: true`, all generation responses are automatically saved:
+
+```ruby
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  recording: true,
+  recordings_dir: "recordings/"
+)
+```
+
+Recordings are saved as `{schema_name}_{timestamp}.json`.
+
+## Behavior Simulation
+
+Schemas with behaviors are honored by the mock server:
+
+```
+User:
+  @latency 50..200ms
+  @failure 5%
+  
+  id: uuid
+  name: name
+```
+
+When this schema is requested:
+- Responses will be delayed 50-200ms
+- 5% of requests will return 500 error
+
+## Rate Limiting
+
+```ruby
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  rate_limit: 100  # 100 requests per minute per IP
+)
+```
+
+When rate limited:
+```json
+{
+  "error": "Rate limit exceeded"
+}
+```
+HTTP Status: 429
+
+## CORS Support
+
+CORS is enabled by default:
+
+```ruby
+# Default headers
+Access-Control-Allow-Origin: *
+Access-Control-Allow-Methods: GET, POST, OPTIONS
+Access-Control-Allow-Headers: Content-Type, Authorization
+Access-Control-Max-Age: 86400
+```
+
+Disable CORS:
+```ruby
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  cors: false
+)
+```
+
+## Async Mode
+
+Start server in background:
+
+```ruby
+server = FakeDataDSL::MockServer.new(schema_dir: "schemas/", port: 3000)
+
+# Start in background thread
+server.start_async
+
+# Do other work...
+
+# Stop when done
+server.stop
+```
+
+## Use Cases
+
+### Frontend Development
+
+```ruby
+# Start mock server for frontend dev
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  port: 3000,
+  cors: true,
+  default_mode: :random
+)
+```
+
+### Integration Testing
+
+```ruby
+# In test setup
+before(:all) do
+  @server = FakeDataDSL::MockServer.new(
+    schema_dir: "spec/fixtures/schemas/",
+    port: 3001
+  )
+  @thread = @server.start_async
+end
+
+after(:all) do
+  @server.stop
+end
+
+it "fetches users from API" do
+  response = HTTParty.get("http://localhost:3001/api/user?seed=42")
+  expect(response["id"]).to be_present
+end
+```
+
+### Contract Testing
+
+```ruby
+# Record real API responses
+FakeDataDSL::MockServer.start(
+  schema_dir: "schemas/",
+  recording: true,
+  recordings_dir: "contracts/"
+)
+
+# Later, replay for testing
+curl http://localhost:3000/api/recordings/user_response
+```
+
+## File Structure
+
+When recording is enabled:
+
+```
+recordings/
+├── User_1706123456.json
+├── User_1706123457.json
+├── Order_1706123458.json
+└── custom_response.json
+```
+
+Recording format:
+```json
+{
+  "data": { "id": "...", "name": "..." },
+  "recorded_at": "2026-01-24T10:30:00Z",
+  "schema": "User",
+  "content_type": "application/json"
+}
+```
+
+## Comparison: MockServer vs APIServer
+
+| Feature | MockServer | APIServer |
+|---------|------------|-----------|
+| Purpose | Development/Testing | Production |
+| Recording | ✅ Yes | ❌ No |
+| Authentication | ❌ No | ✅ Yes |
+| Metrics | ❌ No | ✅ Yes |
+| Rate Limiting | ✅ Basic | ✅ Advanced |
+| Caching | ❌ No | ✅ Yes |
+| Rack Support | ❌ No | ✅ Yes |
+
+Use **MockServer** for:
+- Frontend development
+- Integration testing
+- Contract testing
+- API prototyping
+
+Use **APIServer** for:
+- Production deployment
+- Public APIs
+- Services requiring auth
+- High-traffic scenarios

data/docs/NATIVE_ENGINE.md

@@ -0,0 +1,298 @@
+# Native Rust Engine
+
+The FakeDataDSL Native Engine provides **ultra-high-performance** data generation using Rust and the `fake-rs` library. It achieves **3-4 million records per second**, which is **100-250x faster** than the pure Ruby implementation.
+
+## Features
+
+- ⚡ **3-4 million records/sec** generation speed
+- 🌍 **Multi-locale support** (English, French, German, Japanese, Chinese, etc.)
+- 🔧 **Configurable thread count** (default: 2 threads)
+- 📁 **Direct file output** (JSONL, JSON, CSV)
+- 🎲 **Deterministic seeding** for reproducible results
+- 💾 **Memory efficient** for large datasets
+
+## Installation
+
+### Prerequisites
+
+1. **Rust** (1.70+): Install via [rustup](https://rustup.rs/)
+   ```bash
+   curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+   ```
+
+2. **rb_sys gem**: For Ruby-Rust integration
+   ```bash
+   gem install rb_sys
+   ```
+
+### Compile the Extension
+
+```bash
+cd fake_data_dsl
+bundle exec rake compile
+```
+
+### Verify Installation
+
+```ruby
+require 'fake_data_dsl'
+
+if FakeDataDSL::NativeEngine.available?
+  puts "Native engine available!"
+  puts FakeDataDSL::NativeEngine.performance_info
+else
+  puts "Native engine not available - using Ruby engine"
+end
+```
+
+## Usage
+
+### Basic Generation
+
+```ruby
+require 'fake_data_dsl'
+
+schema = FakeDataDSL.parse(<<~DSL)
+  User:
+    id: uuid
+    name: name
+    email: email
+    age: number(18..80)
+    active: boolean
+DSL
+
+# Generate using native engine
+users = schema.generate_many(100_000, engine: :native, seed: 42)
+```
+
+### Generate to File (Fastest)
+
+For maximum performance with large datasets, generate directly to file:
+
+```ruby
+# Generate 10 million records to JSONL (one JSON per line)
+schema.generate_to_file(10_000_000, "users.jsonl", format: "jsonl")
+
+# Generate CSV
+schema.generate_to_file(1_000_000, "users.csv", format: "csv")
+
+# Generate JSON array
+schema.generate_to_file(100_000, "users.json", format: "json")
+```
+
+### Thread Control
+
+By default, the native engine uses **2 threads** to leave resources for other work. You can customize this:
+
+```ruby
+# Use 4 threads
+schema.generate_to_file(10_000_000, "users.jsonl", threads: 4)
+
+# Use all available cores (fastest for single job)
+schema.generate_to_file(10_000_000, "users.jsonl", threads: 0)
+
+# Use 1 thread (minimal resource usage)
+schema.generate_to_file(10_000_000, "users.jsonl", threads: 1)
+```
+
+| Threads | Use Case |
+|---------|----------|
+| `nil` (default) | 2 threads - balanced for general use |
+| `0` | All cores - maximum speed for single job |
+| `1` | Minimal - when running many parallel jobs |
+| `4-8` | Good balance for multi-process environments |
+
+### Locale Support
+
+The native engine supports multiple locales for authentic names:
+
+```ruby
+# English (uses fast pre-generated pools)
+schema.generate_to_file(1_000_000, "users_en.jsonl", locale: "en")
+
+# French
+schema.generate_to_file(1_000_000, "users_fr.jsonl", locale: "fr_fr")
+
+# German
+schema.generate_to_file(1_000_000, "users_de.jsonl", locale: "de_de")
+
+# Japanese
+schema.generate_to_file(1_000_000, "users_ja.jsonl", locale: "ja_jp")
+
+# Chinese (Simplified)
+schema.generate_to_file(1_000_000, "users_zh.jsonl", locale: "zh_cn")
+```
+
+**Supported Locales:**
+- `en` - English (fast mode)
+- `fr_fr` - French
+- `de_de` - German
+- `ja_jp` - Japanese
+- `zh_cn` - Chinese (Simplified)
+- `zh_tw` - Chinese (Traditional)
+- `pt_br` - Portuguese (Brazil)
+- `it_it` - Italian
+- `ar_sa` - Arabic
+
+### Using NativeEngine Class Directly
+
+For more control, use the `NativeEngine` class directly:
+
+```ruby
+# Create engine with options
+engine = FakeDataDSL::NativeEngine.new(
+  schema,
+  locale: "en",
+  threads: 4
+)
+
+# Generate records
+records = engine.generate_many(100_000, seed: 42)
+
+# Generate to file
+engine.generate_to_file(10_000_000, "users.jsonl", format: "jsonl")
+
+# Stream records
+engine.stream(1_000_000) do |record|
+  process(record)
+end
+```
+
+## Performance Benchmarks
+
+| Records | Time | Rate |
+|---------|------|------|
+| 1M | 0.3s | ~3.5M/sec |
+| 5M | 1.5s | ~3.3M/sec |
+| 10M | 2.7s | **~3.7M/sec** |
+| 20M | 5.3s | ~3.8M/sec |
+
+### Comparison: Ruby vs Native
+
+| Engine | 10M Records | Rate |
+|--------|-------------|------|
+| Pure Ruby | ~10 minutes | ~17K/sec |
+| Native (2 threads) | ~5 seconds | ~2M/sec |
+| Native (all cores) | ~2.7 seconds | **~3.7M/sec** |
+
+## Supported Types
+
+The native engine supports these types:
+
+### Personal
+- `name`, `full_name`, `first_name`, `last_name`, `title`
+
+### Internet
+- `email`, `username`, `password`, `url`
+- `ip`, `ipv4`, `ipv6`, `mac_address`, `user_agent`, `domain`
+
+### Address
+- `city`, `country`, `country_code`, `state`
+- `street`, `street_address`, `postal_code`, `zip_code`
+- `latitude`, `longitude`
+
+### Company
+- `company`, `company_name`, `catch_phrase`, `buzzword`, `industry`
+
+### Text
+- `word`, `words`, `sentence`, `sentences`, `paragraph`, `paragraphs`, `text`
+
+### Phone
+- `phone`, `phone_number`, `cell_phone`
+
+### Date/Time
+- `date`, `time`, `datetime`, `timestamp`, `past_date`, `future_date`
+
+### Identifiers
+- `uuid`, `uuid_v4`, `uuid_v1`, `id_sequence`, `ulid`
+
+### Primitives
+- `number`, `integer`, `float`, `boolean`, `bool`
+
+### Color
+- `hex_color`, `rgb_color`
+
+### Special
+- `enum`, `const`, `array`
+
+## Environment Variables
+
+You can also configure threads via environment variable:
+
+```bash
+# Set before running Ruby
+RAYON_NUM_THREADS=4 ruby my_script.rb
+```
+
+## Troubleshooting
+
+### Native engine not available
+
+```
+Native extension not compiled. To enable native performance:
+1. Install Rust: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
+2. Install rb_sys: gem install rb_sys
+3. Compile extension: bundle exec rake compile
+```
+
+### Compilation errors
+
+Make sure you have:
+- Rust 1.70+ installed
+- Clang/LLVM for linking
+- rb_sys gem installed
+
+```bash
+# Check Rust version
+rustc --version
+
+# Reinstall rb_sys
+gem install rb_sys
+
+# Clean and recompile
+cd fake_data_dsl
+rm -rf ext/fake_data_dsl_native/target
+bundle exec rake compile
+```
+
+### Performance not as expected
+
+1. **Check thread count**: Default is 2 threads. Use `threads: 0` for max speed.
+2. **Use file output**: `generate_to_file` is faster than `generate_many` for large datasets.
+3. **English locale**: English uses fast pre-generated pools; other locales use full fake-rs.
+
+## API Reference
+
+### FakeDataDSL::NativeEngine
+
+#### Class Methods
+
+```ruby
+NativeEngine.available?        # Check if native extension is loaded
+NativeEngine.version           # Get extension version
+NativeEngine.performance_info  # Get performance information
+NativeEngine.thread_count      # Get current thread count
+NativeEngine.set_threads(n)    # Set thread count (call early)
+NativeEngine.available_types   # List supported types
+NativeEngine.available_locales # List supported locales
+```
+
+#### Instance Methods
+
+```ruby
+engine = NativeEngine.new(schema, locale: "en", threads: 2)
+
+engine.generate(seed: nil)                    # Single record
+engine.generate_many(count, seed: nil, threads: nil)  # Multiple records
+engine.generate_to_file(count, path, seed: nil, format: "jsonl", threads: nil)
+engine.stream(count, seed: nil) { |record| }  # Stream records
+```
+
+### Schema Methods
+
+```ruby
+schema.generate_many(count, engine: :native, locale: "en", threads: 2)
+schema.generate_many_native(count, seed: nil, locale: "en", threads: nil)
+schema.generate_to_file(count, path, locale: "en", format: "jsonl", threads: nil)
+```
+