synthra 0.1.0

data/tech_docs/dsl/complex_types.md

@@ -0,0 +1,790 @@
+# Complex Types Reference
+
+Arrays, objects, references, cross-schema linking, and advanced data generation.
+
+## Table of Contents
+
+- [Array Type](#array-type)
+- [Object Type](#object-type)
+- [Reference Type](#reference-type)
+- [Copy Type](#copy-type)
+- [Shared Type](#shared-type)
+- [Map By Field Type](#map-by-field-type)
+- [One Of Type](#one-of-type)
+- [Const Type](#const-type)
+- [Custom Type](#custom-type)
+- [Sequence Types](#sequence-types)
+
+---
+
+## Array Type
+
+### Basic Arrays
+
+```dsl
+Order:
+  # Array of strings
+  tags: array(text, 1..5)
+  
+  # Array of numbers
+  scores: array(number(1..100), 5)
+  
+  # Array of schema references
+  items: array(OrderItem, 1..10)
+```
+
+### Syntax Options
+
+```dsl
+# Positional arguments: element type, size
+tags: array(text, 3)           # Exactly 3 elements
+tags: array(text, 1..5)        # 1 to 5 elements
+
+# Named arguments
+tags: array(element: text, size: 3)
+tags: array(element: text, size: 1..5)
+
+# Complex element types
+addresses: array(Address, 1..3)
+emails: array(email, 0..5)
+prices: array(number(0.99..99.99), 1..10)
+```
+
+### Array Arguments
+
+| Argument | Type | Description |
+|----------|------|-------------|
+| `element` | Type | Element type or schema reference |
+| `size` | Integer or Range | Number of elements |
+| `unique` | Boolean | Ensure unique elements |
+| `min` | Integer | Minimum size (alternative to range) |
+| `max` | Integer | Maximum size (alternative to range) |
+
+### Array Examples
+
+```dsl
+Product:
+  # 1-5 tags, each 3-20 characters
+  tags: array(text(3..20), 1..5)
+  
+  # Exactly 3 images
+  images: array(url, 3)
+  
+  # 0 or more reviews (optional array)
+  reviews: array(Review, 0..10)
+  
+  # Unique categories
+  categories: array(enum(tech, food, fashion), size: 1..3, unique: true)
+
+Review:
+  rating: number(1..5)
+  comment: text(10..500)
+  author: name
+```
+
+### Empty Arrays
+
+```dsl
+# May be empty (0 elements)
+optional_items: array(Item, 0..5)
+
+# Always has at least one
+required_items: array(Item, 1..5)
+```
+
+---
+
+## Object Type
+
+### Inline Objects
+
+For simple, one-off nested structures:
+
+```dsl
+User:
+  metadata: object
+```
+
+Generates a generic object with random keys/values.
+
+### Schema References
+
+For structured nested objects, reference another schema:
+
+```dsl
+User:
+  id: uuid
+  name: name
+  address: Address   # Embeds full Address schema
+
+Address:
+  street: text(10..100)
+  city: name
+  country: country_code
+  postal_code: postal_code
+```
+
+### Nested Schema Example
+
+```ruby
+schema = FakeDataDSL.load('user.dsl')
+user = schema.generate
+
+# Result:
+# {
+#   "id" => "550e8400-...",
+#   "name" => "John Smith",
+#   "address" => {
+#     "street" => "123 Main Street",
+#     "city" => "Springfield",
+#     "country" => "US",
+#     "postal_code" => "12345"
+#   }
+# }
+```
+
+---
+
+## Reference Type
+
+### What is a Reference?
+
+References let you link schemas together without embedding the full object.
+
+### Ref() Syntax
+
+```dsl
+# Reference a specific field from another schema
+Order:
+  id: uuid
+  user_id: Ref(User.id)           # Gets a User id
+  shipping_address_id: Ref(Address.id)  # Gets an Address id
+
+User:
+  id: uuid
+  name: name
+
+Address:
+  id: uuid
+  street: text
+```
+
+### How References Work
+
+1. **Same Batch**: References resolve to values generated in the same batch
+2. **Registry Required**: Cross-schema references need a Registry
+
+```ruby
+registry = FakeDataDSL::Registry.new
+registry.load_file('schemas/user.dsl')
+registry.load_file('schemas/order.dsl')
+
+# Generate with registry for reference resolution
+order = registry.schema('Order').generate(registry: registry)
+# order["user_id"] will be a valid UUID
+```
+
+### Reference Examples
+
+```dsl
+# E-commerce system
+Customer:
+  id: uuid
+  email: email
+
+Order:
+  id: uuid
+  customer_id: Ref(Customer.id)
+  items: array(OrderItem, 1..10)
+
+OrderItem:
+  id: uuid
+  order_id: Ref(Order.id)
+  product_id: Ref(Product.id)
+  quantity: number(1..5)
+
+Product:
+  id: uuid
+  name: text(10..50)
+  price: money
+```
+
+### Reference vs Embed
+
+| Approach | Use Case | Example |
+|----------|----------|---------|
+| **Embed** | Full object needed | `address: Address` |
+| **Ref** | Just ID/foreign key | `user_id: Ref(User.id)` |
+
+```dsl
+# Embed - includes full address
+User:
+  address: Address
+
+# Generates:
+# { "address": { "street": "...", "city": "...", ... } }
+
+# Reference - just the ID
+Order:
+  shipping_address_id: Ref(Address.id)
+
+# Generates:
+# { "shipping_address_id": "550e8400-..." }
+```
+
+### ⚠️ Important: Ref() vs copy() - Know the Difference!
+
+`Ref()` and `copy()` look similar but behave **very differently**:
+
+| Feature | `Ref(Schema.field)` | `copy("path.to.field")` |
+|---------|---------------------|-------------------------|
+| **What it does** | Generates a NEW value | Copies an EXISTING value |
+| **Source** | Generates from another schema's field type | References already-generated field in current context |
+| **Use case** | Foreign key that doesn't need to match existing data | Link IDs across nested objects in same record |
+| **Requires** | Registry for cross-schema resolution | Field must be defined BEFORE the copy |
+
+#### Example: The Wrong Way vs The Right Way
+
+```dsl
+# ❌ WRONG - Using Ref() when you need copy()
+TwitterDM:
+  sender: TwitterUser
+  sender_id: Ref(TwitterUser.id)  # Generates a NEW ID, different from sender.id!
+
+# ✅ CORRECT - Using copy() to link IDs
+TwitterDM:
+  sender: TwitterUser
+  sender_id: copy("sender.id")    # Uses the SAME ID as sender.id
+```
+
+#### When to Use Each
+
+| Scenario | Use | Example |
+|----------|-----|---------|
+| Foreign key to another table | `Ref()` | `customer_id: Ref(Customer.id)` |
+| Copying ID from nested object | `copy()` | `sender_id: copy("sender.id")` |
+| Dynamic key linking with map_by_field | `copy()` | `for_user_id: copy("users.recipient.id")` |
+| Independent ID generation | `Ref()` | `correlation_id: Ref(Request.id)` |
+
+---
+
+## Copy Type
+
+### What is Copy?
+
+`copy()` references a value from another field in the same generation context. This enables linking related IDs across nested schemas.
+
+### Basic Usage
+
+```dsl
+Order:
+  user: User
+  order_user_id: copy("user.id")  # Copies the generated user's ID
+
+User:
+  id: uuid
+  name: full_name
+```
+
+### Path Syntax
+
+Paths use dot notation to traverse nested structures:
+
+```dsl
+TwitterDM:
+  users: map_by_field("id", sender: TwitterUser, recipient: TwitterUser)
+  sender_id: copy("users.sender.id")      # Gets sender's ID
+  recipient_id: copy("users.recipient.id") # Gets recipient's ID
+```
+
+### Important: Field Order
+
+`copy()` can only reference fields that are **defined before** the copy field:
+
+```dsl
+# ✅ CORRECT - user defined before copy
+Order:
+  user: User
+  user_id: copy("user.id")
+
+# ❌ WRONG - user_id defined before user
+Order:
+  user_id: copy("user.id")  # Will return nil!
+  user: User
+```
+
+### Path Validation
+
+FakeDataDSL validates `copy()` paths at load time by default:
+
+```ruby
+registry = FakeDataDSL::Registry.new
+registry.load_string(<<~DSL)
+  Order:
+    user: User
+    user_id: copy("users.id")  # Typo: "users" instead of "user"
+DSL
+# Raises: FakeDataDSL::Errors::PathError
+#   Path 'users.id' not found in schema
+#   Did you mean: user.id?
+```
+
+To disable validation (not recommended):
+
+```ruby
+registry.load_string(dsl, validate: false)
+# OR globally:
+FakeDataDSL.configure { |c| c.validate_paths_on_load = false }
+```
+
+---
+
+## Shared Type
+
+### What is Shared?
+
+`shared()` generates a value once and reuses it across all records in a batch when using `generate_many()`. This is essential for simulating realistic data where certain values should be consistent.
+
+### ⚠️ Important: Only Works with `generate_many()`
+
+**`shared()` requires `generate_many()` to function correctly.**
+
+```ruby
+# ✅ CORRECT - shared values work with generate_many()
+webhooks = schema.generate_many(10, registry: registry)
+# All 10 webhooks have the SAME recipient
+
+# ⚠️ WARNING - shared values don't work with generate()
+webhook1 = schema.generate(registry: registry)
+webhook2 = schema.generate(registry: registry)
+# webhook1 and webhook2 have DIFFERENT recipients!
+```
+
+When `shared()` is used with `generate()`, FakeDataDSL emits a warning:
+
+```
+WARNING: shared() type used outside generate_many() - value will not be shared across calls.
+Use generate_many() for batch generation with shared values.
+```
+
+### Basic Usage
+
+```dsl
+# All webhooks in a batch share the same recipient
+TwitterDMWebhook:
+  users: map_by_field("id", sender: TwitterUser, recipient: shared(TwitterUser))
+  for_user_id: copy("users.recipient.id")  # Same for all records
+```
+
+### How Shared Works
+
+1. First generation: Creates the value and caches it in `shared_context`
+2. Subsequent generations: Returns the cached value
+
+```ruby
+schema = registry.schema("TwitterDMWebhook")
+webhooks = schema.generate_many(10, registry: registry)
+
+# Verify sharing
+recipient_ids = webhooks.map { |w| w.dig("users", "recipient", "id") }.uniq
+# => 1 (all have same recipient)
+
+sender_ids = webhooks.map { |w| w.dig("users", "sender", "id") }.uniq
+# => 10 (all have different senders)
+```
+
+### Shared with Custom Keys
+
+Use the `key:` argument for multiple shared values of the same type:
+
+```dsl
+Conversation:
+  user1: shared(User, key: "participant_1")
+  user2: shared(User, key: "participant_2")
+  messages: array(Message, 5..10)
+
+Message:
+  sender: one_of(shared(User, key: "participant_1"), shared(User, key: "participant_2"))
+  text: message_text
+```
+
+### When to Use Shared
+
+| Scenario | Use Shared? |
+|----------|-------------|
+| Inbox webhooks (same recipient) | ✅ Yes |
+| Random users per record | ❌ No |
+| Consistent config across batch | ✅ Yes |
+| Unique IDs per record | ❌ No |
+
+---
+
+## Map By Field Type
+
+### What is Map By Field?
+
+`map_by_field()` creates dynamic object keys from generated field values. This is essential for APIs that return objects keyed by ID.
+
+### Basic Usage
+
+```dsl
+EventData:
+  users: map_by_field("id", sender: TwitterUser, recipient: TwitterUser)
+
+TwitterUser:
+  id: snowflake_id
+  name: full_name
+  screen_name: social_handle
+```
+
+Generates:
+
+```json
+{
+  "users": {
+    "1234567890123456789": {
+      "id": "1234567890123456789",
+      "name": "John Smith",
+      "screen_name": "johnsmith"
+    },
+    "9876543210987654321": {
+      "id": "9876543210987654321",
+      "name": "Jane Doe",
+      "screen_name": "janedoe"
+    }
+  }
+}
+```
+
+### Syntax
+
+```dsl
+field_name: map_by_field("key_field", alias1: Schema1, alias2: Schema2, ...)
+```
+
+- `"key_field"` - The field to use as object keys (must exist in schemas)
+- `alias: Schema` - Named entries in the map
+
+### Accessing Map Values with Copy
+
+Use the alias names to reference values:
+
+```dsl
+EventData:
+  users: map_by_field("id", sender: TwitterUser, recipient: shared(TwitterUser))
+  sender_id: copy("users.sender.id")       # Access sender's ID
+  recipient_id: copy("users.recipient.id") # Access recipient's ID
+```
+
+### Combining with Shared
+
+```dsl
+# Recipient is shared across all records in generate_many()
+EventData:
+  users: map_by_field("id", sender: TwitterUser, recipient: shared(TwitterUser))
+```
+
+---
+
+## One Of Type
+
+### What is One Of?
+
+`one_of()` randomly selects and generates one of multiple schema options. This enables polymorphic data structures.
+
+### Basic Usage
+
+```dsl
+WebhookEvent:
+  event_id: uuid
+  payload: one_of(MessagePayload, TypingPayload, ReadReceiptPayload)
+
+MessagePayload:
+  type: const("message")
+  text: message_text
+
+TypingPayload:
+  type: const("typing")
+  is_typing: boolean
+
+ReadReceiptPayload:
+  type: const("read_receipt")
+  message_id: snowflake_id
+```
+
+### Weighted Selection
+
+Use weights to control selection probability:
+
+```dsl
+# Messages 80%, typing 15%, read receipts 5%
+payload: one_of(MessagePayload:80%, TypingPayload:15%, ReadReceiptPayload:5%)
+```
+
+### Use Cases
+
+- Polymorphic API responses
+- Mixed event types in streams
+- Testing multiple payload formats
+
+---
+
+## Const Type
+
+### What is Const?
+
+`const()` generates a fixed, literal value. Use this for constant fields like type discriminators.
+
+### Why Not Enum?
+
+`enum()` is for selecting from multiple values. For single literal values, `const()` is more semantically correct:
+
+```dsl
+# ❌ Enum smell - single value enum
+type: enum(message_create)
+
+# ✅ Semantic - constant value
+type: const("message_create")
+```
+
+### Basic Usage
+
+```dsl
+DirectMessageEvent:
+  type: const("message_create")  # String literal
+  version: const(1)              # Integer literal
+  enabled: const(true)           # Boolean literal
+```
+
+### String vs Unquoted Values
+
+```dsl
+# Quoted strings (recommended)
+type: const("message_create")
+
+# Integer
+version: const(42)
+
+# Boolean
+active: const(true)
+```
+
+---
+
+## Custom Type
+
+### What is Custom?
+
+`custom` lets you compute field values dynamically using Ruby functions.
+
+### Basic Usage
+
+```dsl
+Invoice:
+  items: array(LineItem, 1..5)
+  subtotal: custom(:calculate_subtotal)
+  tax: custom(:calculate_tax)
+  total: custom(:calculate_total)
+
+LineItem:
+  quantity: number(1..5)
+  unit_price: number(9.99..99.99)
+```
+
+### Registering Functions
+
+```ruby
+FakeDataDSL.register_function(:calculate_subtotal) do |context|
+  items = context[:items] || []
+  items.sum { |item| item["quantity"] * item["unit_price"] }
+end
+
+FakeDataDSL.register_function(:calculate_tax) do |context|
+  subtotal = context[:subtotal] || 0
+  (subtotal * 0.08).round(2)
+end
+
+FakeDataDSL.register_function(:calculate_total) do |context|
+  subtotal = context[:subtotal] || 0
+  tax = context[:tax] || 0
+  (subtotal + tax).round(2)
+end
+```
+
+### Context Object
+
+The context contains:
+- All previously generated fields
+- Schema metadata
+- Registry (if available)
+
+```ruby
+FakeDataDSL.register_function(:full_name) do |context|
+  "#{context['first_name']} #{context['last_name']}"
+end
+
+FakeDataDSL.register_function(:email_from_name) do |context|
+  name = context['name'].downcase.gsub(' ', '.')
+  "#{name}@company.com"
+end
+```
+
+### Custom Function Examples
+
+```ruby
+# Conditional logic
+FakeDataDSL.register_function(:discount_amount) do |context|
+  total = context[:total] || 0
+  tier = context[:customer_tier] || 'standard'
+  
+  case tier
+  when 'gold' then total * 0.15
+  when 'silver' then total * 0.10
+  else total * 0.05
+  end
+end
+
+# External data
+FakeDataDSL.register_function(:sku) do |context|
+  category = context[:category]
+  id = context[:id][0..7]
+  "#{category.upcase}-#{id}"
+end
+
+# Date calculations
+FakeDataDSL.register_function(:expiration_date) do |context|
+  created = Date.parse(context[:created_at])
+  (created + 365).to_s
+end
+```
+
+---
+
+## Sequence Types
+
+### id_sequence
+
+Auto-incrementing integers:
+
+```dsl
+User:
+  id: id_sequence
+  order_number: id_sequence(start: 1000)
+```
+
+### character_sequence
+
+Sequential letters:
+
+```dsl
+Item:
+  code: character_sequence
+  # Generates: A, B, C, ... Z, AA, AB, ...
+```
+
+### digit_sequence
+
+Sequential numbers as strings:
+
+```dsl
+Invoice:
+  number: digit_sequence(start: "001", pad: 5)
+  # Generates: "00001", "00002", "00003", ...
+```
+
+---
+
+## Formula Type
+
+### Mathematical Expressions
+
+```dsl
+Product:
+  base_price: number(10..100)
+  tax_rate: number(0.05..0.15)
+  final_price: formula("base_price * (1 + tax_rate)")
+```
+
+### Supported Operations
+
+- Arithmetic: `+`, `-`, `*`, `/`, `%`
+- References to other fields
+- Parentheses for grouping
+
+```dsl
+Order:
+  quantity: number(1..10)
+  unit_price: number(9.99..99.99)
+  discount: number(0..20)
+  
+  subtotal: formula("quantity * unit_price")
+  discount_amount: formula("subtotal * discount / 100")
+  total: formula("subtotal - discount_amount")
+```
+
+---
+
+## Template Type
+
+### String Templates
+
+```dsl
+User:
+  first_name: first_name
+  last_name: last_name
+  display_name: template("${first_name} ${last_name}")
+  email: template("${first_name}.${last_name}@company.com")
+```
+
+### Template Syntax
+
+- `${field_name}` - Insert field value
+- Literal text around placeholders
+
+```dsl
+Product:
+  brand: text(5..20)
+  model: text(3..10)
+  sku: template("${brand}-${model}-001")
+  url: template("https://shop.com/products/${sku}")
+```
+
+---
+
+## Type Composition
+
+### Combining Types
+
+```dsl
+Order:
+  id: uuid(unique: true)
+  
+  # Array of referenced items
+  item_ids: array(Ref(Product.id), 1..5)
+  
+  # Nested objects
+  shipping: ShippingInfo
+  billing: BillingInfo
+  
+  # Computed from nested data
+  total: custom(:sum_items)
+
+ShippingInfo:
+  address: Address
+  method: enum(standard, express, overnight)
+  cost: money
+
+BillingInfo:
+  address: Address
+  card_type: enum(visa, mastercard, amex)
+  last_four: text(4)
+```
+
+---
+
+## Next Steps
+
+- [Field Modifiers](field_modifiers.md) - Optional, nullable, conditional
+- [Custom Types Guide](../api/custom_types.md) - Creating your own types
+- [Custom Functions](../api/custom_functions.md) - Computed field functions
+