synthra 0.1.0

data/docs/OPENAPI_EXPORT.md

@@ -0,0 +1,252 @@
+# OpenAPI Export
+
+Export your FakeDataDSL schemas to **OpenAPI 3.0 specifications** for API documentation and client generation.
+
+## Quick Start
+
+```bash
+# Export as YAML
+fake_data_dsl export --all -d schemas/ -f openapi -o api.yaml
+
+# Export as JSON
+fake_data_dsl export --all -d schemas/ -f openapi -o api.json
+```
+
+## Generated Specification
+
+The exporter generates a complete OpenAPI 3.0 spec with:
+
+- **Info section** - API title, version, description
+- **Server definitions** - Base URLs
+- **Tags** - One per schema for organization
+- **Paths** - CRUD + generation endpoints
+- **Components/Schemas** - Full schema definitions
+- **Parameters** - Reusable query parameters
+
+## API Endpoints Generated
+
+For each schema (e.g., `User`):
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/user` | Generate single record |
+| GET | `/api/user/batch` | Generate multiple records |
+| GET | `/api/user/stream` | Stream records (NDJSON) |
+| POST | `/api/user` | Generate with overrides |
+| GET | `/api/user/schema` | Get schema definition |
+
+Plus:
+- `GET /api/schemas` - List all schemas
+
+## Ruby API
+
+### Basic Export
+
+```ruby
+exporter = FakeDataDSL::Export::OpenAPI.new(registry)
+yaml_spec = exporter.export
+```
+
+### Custom Options
+
+```ruby
+exporter = FakeDataDSL::Export::OpenAPI.new(
+  registry,
+  title: "My Data Generation API",
+  version: "2.0.0",
+  description: "API for generating test data",
+  server_url: "https://api.example.com",
+  format: :yaml
+)
+
+spec = exporter.export
+```
+
+### Export Formats
+
+```ruby
+# YAML (default)
+yaml = exporter.export_as(:yaml)
+
+# JSON
+json = exporter.export_as(:json)
+```
+
+## Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `title` | String | "FakeDataDSL Data Generation API" | API title |
+| `version` | String | "1.0.0" | API version |
+| `description` | String | "API for generating fake data" | API description |
+| `server_url` | String | "http://localhost:3000" | Base server URL |
+| `format` | Symbol | :yaml | Output format (:yaml or :json) |
+
+## Type Mapping
+
+| FakeDataDSL | OpenAPI |
+|-------------|---------|
+| `text`, `name`, etc. | `string` |
+| `email` | `string`, format: "email" |
+| `url` | `string`, format: "uri" |
+| `uuid` | `string`, format: "uuid" |
+| `date` | `string`, format: "date" |
+| `timestamp`, `datetime` | `string`, format: "date-time" |
+| `password` | `string`, format: "password" |
+| `ip` | `string`, format: "ipv4" |
+| `ipv6` | `string`, format: "ipv6" |
+| `number`, `integer` | `integer`, format: "int64" |
+| `float` | `number`, format: "double" |
+| `boolean` | `boolean` |
+| `binary` | `string`, format: "binary" |
+| `array(T)` | `array`, items: T |
+| `enum(a, b)` | `string`, enum: ["a", "b"] |
+
+## Example Output
+
+```yaml
+openapi: "3.0.3"
+info:
+  title: FakeDataDSL Data Generation API
+  version: "1.0.0"
+  description: API for generating fake data from schemas
+  contact:
+    name: FakeDataDSL
+    url: https://github.com/talaatmagdyx/fake_data_dsl
+
+servers:
+  - url: http://localhost:3000
+    description: Data Generation Server
+
+tags:
+  - name: User
+    description: Operations for User schema
+
+paths:
+  /api/schemas:
+    get:
+      tags: [Schemas]
+      summary: List all available schemas
+      responses:
+        "200":
+          description: List of schema names
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  schemas:
+                    type: array
+                    items:
+                      type: string
+
+  /api/user:
+    get:
+      tags: [User]
+      summary: Generate a single User
+      parameters:
+        - name: seed
+          in: query
+          description: Random seed for deterministic generation
+          schema:
+            type: integer
+        - name: mode
+          in: query
+          description: Generation mode
+          schema:
+            type: string
+            enum: [random, edge, invalid, hostile, mixed]
+      responses:
+        "200":
+          description: Generated User
+          content:
+            application/json:
+              schema:
+                $ref: "#/components/schemas/User"
+
+components:
+  schemas:
+    User:
+      type: object
+      required: [id, name, email]
+      properties:
+        id:
+          type: string
+          format: uuid
+        name:
+          type: string
+        email:
+          type: string
+          format: email
+        age:
+          type: integer
+          format: int64
+        active:
+          type: boolean
+        role:
+          type: string
+          enum: [admin, user, moderator]
+
+    GenerationRequest:
+      type: object
+      properties:
+        count:
+          type: integer
+          default: 10
+        seed:
+          type: integer
+        mode:
+          type: string
+          enum: [random, edge, invalid, hostile, mixed]
+        overrides:
+          type: object
+          additionalProperties: true
+```
+
+## Using Generated Spec
+
+### Swagger UI
+
+```bash
+# Serve with Swagger UI
+docker run -p 8080:8080 \
+  -e SWAGGER_JSON=/api.yaml \
+  -v ./api.yaml:/api.yaml \
+  swaggerapi/swagger-ui
+```
+
+### Client Generation
+
+```bash
+# Generate TypeScript client
+openapi-generator generate -i api.yaml -g typescript-axios -o ./client
+
+# Generate Python client
+openapi-generator generate -i api.yaml -g python -o ./client
+
+# Generate Go client
+openapi-generator generate -i api.yaml -g go -o ./client
+```
+
+### API Testing
+
+```bash
+# Import into Postman
+# 1. Open Postman
+# 2. Import > File > api.yaml
+# 3. All endpoints are ready to test
+```
+
+## Integration with API Server
+
+The OpenAPI export matches the API Server exactly:
+
+```bash
+# Start server
+fake_data_dsl server schemas/ --port 3000
+
+# Export matching spec
+fake_data_dsl export --all -d schemas/ -f openapi -o api.yaml
+```
+
+The generated spec documents exactly what the server provides.

data/docs/PERFORMANCE_MODE.md

@@ -0,0 +1,228 @@
+# Performance Mode
+
+Generate **millions of records** with parallel processing, streaming, and optimized memory usage.
+
+## Quick Start
+
+```bash
+# Generate 1 million records to file
+fake_data_dsl perf User -d schemas/ -c 1000000 -o users.ndjson
+
+# CSV format
+fake_data_dsl perf User -d schemas/ -c 1000000 -f csv -o users.csv
+
+# Benchmark mode
+fake_data_dsl perf User -d schemas/ --benchmark
+```
+
+## CLI Options
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `-d, --dir DIR` | Schema directory | "." |
+| `-c, --count N` | Number of records | 100,000 |
+| `-o, --output FILE` | Output file | stdout |
+| `-f, --format FMT` | Format: ndjson, json, csv | ndjson |
+| `-s, --seed N` | Random seed | random |
+| `-m, --mode MODE` | Generation mode | random |
+| `-b, --benchmark` | Run benchmark | false |
+
+## Ruby API
+
+### Basic Generation
+
+```ruby
+# Generate records in parallel
+records = FakeDataDSL::PerformanceMode.generate(schema, count: 1_000_000)
+```
+
+### With Progress Callback
+
+```ruby
+FakeDataDSL::PerformanceMode.generate(schema, count: 1_000_000) do |progress|
+  puts "#{progress[:current]} / #{progress[:total]}"
+  puts "Rate: #{progress[:rate]} rec/s"
+  puts "ETA: #{progress[:eta]}s"
+end
+```
+
+Progress hash contains:
+- `current` - Records generated so far
+- `total` - Total records to generate
+- `percent` - Completion percentage
+- `rate` - Records per second
+- `elapsed` - Elapsed time (seconds)
+- `eta` - Estimated time remaining (seconds)
+
+### Streaming
+
+Stream records without loading all into memory:
+
+```ruby
+stream = FakeDataDSL::PerformanceMode.stream(schema, count: 10_000_000)
+
+stream.each do |record|
+  # Process one record at a time
+  process(record)
+end
+```
+
+### File Output
+
+Write directly to file with streaming:
+
+```ruby
+result = FakeDataDSL::PerformanceMode.to_file(
+  schema,
+  count: 10_000_000,
+  output: "users.ndjson",
+  format: :ndjson,
+  seed: 42,
+  mode: :random
+) do |progress|
+  puts "#{progress[:percent]}% - #{progress[:rate]} rec/s - ETA: #{progress[:eta]}s"
+end
+
+puts "Generated #{result[:records]} in #{result[:elapsed]}s"
+puts "Rate: #{result[:rate]} records/second"
+```
+
+Supported formats:
+- `:ndjson` - Newline-delimited JSON (default, best for streaming)
+- `:json` - JSON array
+- `:csv` - CSV with headers
+
+### Benchmarking
+
+```ruby
+results = FakeDataDSL::PerformanceMode.benchmark(
+  schema,
+  counts: [1_000, 10_000, 100_000, 1_000_000]
+)
+
+results.each do |count, metrics|
+  puts "#{count} records:"
+  puts "  Time: #{metrics[:elapsed]}s"
+  puts "  Rate: #{metrics[:rate]} rec/s"
+  puts "  Memory: #{metrics[:memory_mb]} MB"
+end
+```
+
+### System Info
+
+Check system capabilities:
+
+```ruby
+info = FakeDataDSL::PerformanceMode.system_info
+
+puts "CPU cores: #{info[:cpu_cores]}"
+puts "Optimal threads: #{info[:optimal_threads]}"
+puts "Native engine: #{info[:native_available]}"
+puts "Recommended batch: #{info[:recommended_batch_size]}"
+```
+
+## Configuration
+
+Customize generation behavior:
+
+```ruby
+records = FakeDataDSL::PerformanceMode.generate(
+  schema,
+  count: 1_000_000,
+  batch_size: 10_000,           # Records per batch
+  threads: 4,                    # Parallel threads (nil = auto)
+  memory_limit_mb: 512,          # Max memory per batch
+  use_native: true,              # Use Rust engine if available
+  progress_interval: 100_000,    # Report every N records
+  gc_interval: 500_000,          # Force GC every N records
+  seed: 42,                      # Random seed
+  mode: :random                  # Generation mode
+)
+```
+
+## Performance Tips
+
+### 1. Use NDJSON for Large Datasets
+
+NDJSON streams records one at a time, minimizing memory:
+
+```ruby
+FakeDataDSL::PerformanceMode.to_file(schema, count: 10_000_000, format: :ndjson, ...)
+```
+
+### 2. Tune Batch Size
+
+Larger batches = faster, but more memory:
+
+```ruby
+# For machines with lots of RAM
+FakeDataDSL::PerformanceMode.generate(schema, count: 1_000_000, batch_size: 50_000)
+
+# For limited memory
+FakeDataDSL::PerformanceMode.generate(schema, count: 1_000_000, batch_size: 1_000)
+```
+
+### 3. Enable Native Engine
+
+The Rust native engine provides 3-4x performance:
+
+```ruby
+# Check if available
+FakeDataDSL::PerformanceMode.system_info[:native_available]
+
+# Force native engine
+FakeDataDSL::PerformanceMode.generate(schema, count: 1_000_000, use_native: true)
+```
+
+### 4. Adjust Thread Count
+
+```ruby
+# Use all CPU cores
+FakeDataDSL::PerformanceMode.generate(
+  schema,
+  count: 1_000_000,
+  threads: Etc.nprocessors
+)
+
+# Leave one core for other work
+FakeDataDSL::PerformanceMode.generate(
+  schema,
+  count: 1_000_000,
+  threads: Etc.nprocessors - 1
+)
+```
+
+### 5. Disable Observability for Speed
+
+```ruby
+FakeDataDSL.configure do |config|
+  config.metrics_collector = nil
+  config.logger = nil
+  config.on_field_generated = nil
+end
+```
+
+## Benchmarks
+
+Typical performance on modern hardware:
+
+| Records | NDJSON | JSON | CSV |
+|---------|--------|------|-----|
+| 10K | 0.2s | 0.3s | 0.2s |
+| 100K | 1.5s | 2.0s | 1.8s |
+| 1M | 12s | 18s | 15s |
+| 10M | 120s | 180s | 150s |
+
+*Results vary based on schema complexity and hardware.*
+
+## Memory Usage
+
+Approximate memory requirements:
+
+| Records | Stream Mode | Batch Mode |
+|---------|-------------|------------|
+| 100K | ~50 MB | ~100 MB |
+| 1M | ~50 MB | ~1 GB |
+| 10M | ~50 MB | ~10 GB |
+
+Stream mode maintains constant memory regardless of total count.