@zio.dev/zio-blocks 0.0.1

package/reference/reflect-transform.md

@@ -0,0 +1,387 @@
+---
+id: reflect-transform
+title: "Reflect Transform"
+---
+
+# The Transform Method: Architectural Pattern and Design Guide
+
+The `transform` method is a fundamental operation on `Reflect` that enables systematic transformation of schema trees. This document explains the architectural patterns, design decisions, and provides a comprehensive guide for understanding and using `transform`.
+
+## Overview
+
+The `transform` method converts a `Reflect[F, A]` into a `Reflect[G, A]` by recursively traversing the schema tree and applying a `ReflectTransformer[F, G]` at each node:
+
+```scala
+def transform[G[_, _]](path: DynamicOptic, f: ReflectTransformer[F, G]): Lazy[Reflect[G, A]]
+```
+
+**Key Components:**
+- **`path: DynamicOptic`**: Tracks the current location in the schema tree
+- **`f: ReflectTransformer[F, G]`**: The transformation strategy to apply
+- **`Lazy[...]`**: Enables handling of recursive schemas without stack overflow
+
+## Architectural Pattern: Visitor with Lazy Evaluation
+
+The `transform` method implements a variant of the **Visitor Pattern** combined with **Lazy Evaluation** to handle the unique challenges of schema transformation:
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Transform Architecture                    │
+├─────────────────────────────────────────────────────────────┤
+│                                                             │
+│  Reflect[F, A]                                              │
+│       │                                                     │
+│       ▼                                                     │
+│  ┌─────────────┐    ┌──────────────────────┐               │
+│  │  transform  │───▶│ ReflectTransformer   │               │
+│  │   method    │    │     [F, G]           │               │
+│  └─────────────┘    └──────────────────────┘               │
+│       │                      │                              │
+│       │                      ▼                              │
+│       │             ┌──────────────────┐                   │
+│       │             │ transformRecord  │                   │
+│       │             │ transformVariant │                   │
+│       │             │ transformSeq     │                   │
+│       │             │ ...              │                   │
+│       │             └──────────────────┘                   │
+│       │                      │                              │
+│       ▼                      ▼                              │
+│  Lazy[Reflect[G, A]]                                        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Why This Pattern?
+
+1. **Separation of Concerns**: The traversal logic stays in `Reflect`, while transformation logic lives in `ReflectTransformer`
+2. **Extensibility**: New transformations can be added without modifying `Reflect`
+3. **Type Safety**: The type system ensures consistent transformation of the binding type parameter
+4. **Composability**: Transformers can be composed and reused
+
+## Design Decisions
+
+### Decision 1: Two-Phase Transformation (Children First)
+
+Each `transform` implementation follows a **children-first** pattern:
+
+```scala
+// From Reflect.Record.transform
+def transform[G[_, _]](path: DynamicOptic, f: ReflectTransformer[F, G]): Lazy[Record[G, A]] =
+  for {
+    // Phase 1: Transform all children first
+    fields <- Lazy.foreach(fields)(_.transform(path, Term.Type.Record, f))
+    // Phase 2: Then transform the current node with already-transformed children
+    record <- f.transformRecord(path, fields, typeId, recordBinding, doc, modifiers, ...)
+  } yield record
+```
+
+**Rationale:**
+- Ensures children are transformed before parents need them
+- Enables bottom-up construction of the transformed tree
+- Allows transformers to inspect already-transformed children
+
+### Decision 2: Path Tracking with DynamicOptic
+
+The `path` parameter tracks the current location in the schema tree:
+
+```scala
+// Sequence appends 'elements' to the path before transforming its element
+def transform[G[_, _]](path: DynamicOptic, f: ReflectTransformer[F, G]): Lazy[Sequence[G, A, C]] =
+  for {
+    element  <- element.transform(path(DynamicOptic.elements), f)  // path + /elements
+    sequence <- f.transformSequence(path, element, ...)
+  } yield sequence
+
+// Map tracks both keys and values separately
+def transform[G[_, _]](path: DynamicOptic, f: ReflectTransformer[F, G]): Lazy[Map[G, K, V, M]] =
+  for {
+    key   <- key.transform(path(DynamicOptic.mapKeys), f)     // path + /mapKeys
+    value <- value.transform(path(DynamicOptic.mapValues), f) // path + /mapValues
+    map   <- f.transformMap(path, key, value, ...)
+  } yield map
+```
+
+**Use Cases for Path Tracking:**
+- Applying different transformations based on location
+- Error reporting with precise location information
+- Conditional transformation rules
+- Auditing and logging transformations
+
+### Decision 3: Lazy Evaluation for Recursion
+
+The return type `Lazy[Reflect[G, A]]` is critical for handling recursive types:
+
+```scala
+// Deferred uses caching to handle recursive schemas
+case class Deferred[F[_, _], A](...) extends Reflect[F, A] {
+  def transform[G[_, _]](path: DynamicOptic, f: ReflectTransformer[F, G]): Lazy[Reflect[G, A]] =
+    Lazy {
+      val c      = cache.get
+      val key    = new IdentityTuple(this, f)
+      val cached = c.get(key)
+      if (cached ne null) cached.asInstanceOf[Reflect[G, A]]
+      else {
+        // Create deferred wrapper and cache BEFORE recursing
+        val result = Deferred(() => value.transform(path, f).force, ...)
+        c.put(key, result)
+        result
+      }
+    }
+}
+```
+
+**How Recursion is Handled:**
+1. `Deferred` nodes check a thread-local cache before transforming
+2. If found, return the cached (transformed) `Deferred` immediately
+3. If not found, create a new `Deferred` and cache it BEFORE recursing
+4. The lazy thunk in the new `Deferred` will eventually call `transform` on the wrapped value
+
+This pattern breaks infinite recursion by returning a cached reference.
+
+### Decision 4: Node-Specific Transform Methods
+
+Each `Reflect` node type has its own implementation:
+
+| Node Type    | Transforms Children          | Path Updates          |
+|--------------|-----------------------------|-----------------------|
+| `Record`     | All fields                  | Field names           |
+| `Variant`    | All cases                   | Case names            |
+| `Sequence`   | Element type                | `/elements`           |
+| `Map`        | Key and value types         | `/mapKeys`, `/mapValues` |
+| `Wrapper`    | Wrapped type                | (none)                |
+| `Primitive`  | (none - leaf node)          | (none)                |
+| `Dynamic`    | (none - leaf node)          | (none)                |
+| `Deferred`   | Wrapped value (lazy)        | (preserved)           |
+
+## The ReflectTransformer Trait
+
+The `ReflectTransformer[F, G]` trait defines the transformation strategy:
+
+```scala
+trait ReflectTransformer[-F[_, _], G[_, _]] {
+  def transformRecord[A](
+    path: DynamicOptic,
+    fields: IndexedSeq[Term[G, A, ?]],      // Already transformed
+    typeId: TypeId[A],
+    metadata: F[BindingType.Record, A],      // Original binding
+    doc: Doc,
+    modifiers: Seq[Modifier.Reflect],
+    storedDefaultValue: Option[DynamicValue],
+    storedExamples: collection.immutable.Seq[DynamicValue]
+  ): Lazy[Reflect.Record[G, A]]
+
+  def transformVariant[A](...): Lazy[Reflect.Variant[G, A]]
+  def transformSequence[A, C[_]](...): Lazy[Reflect.Sequence[G, A, C]]
+  def transformMap[K, V, M[_, _]](...): Lazy[Reflect.Map[G, K, V, M]]
+  def transformDynamic(...): Lazy[Reflect.Dynamic[G]]
+  def transformPrimitive[A](...): Lazy[Reflect.Primitive[G, A]]
+  def transformWrapper[A, B](...): Lazy[Reflect.Wrapper[G, A, B]]
+}
+```
+
+**Key Observation:** Each method receives already-transformed children plus the original metadata. This enables:
+- Inspecting transformed children before deciding how to transform the current node
+- Combining information from multiple children
+- Conditional transformation based on child structure
+
+### The OnlyMetadata Helper
+
+For transformations that only need to change the binding type:
+
+```scala
+abstract class OnlyMetadata[F[_, _], G[_, _]] extends ReflectTransformer[F, G] {
+  // Only this method needs implementation
+  def transformMetadata[K, A](f: F[K, A]): Lazy[G[K, A]]
+
+  // All node-specific methods are provided automatically
+  def transformRecord[A](
+    path: DynamicOptic,
+    fields: IndexedSeq[Term[G, A, ?]],
+    typeId: TypeId[A],
+    metadata: F[BindingType.Record, A],
+    ...
+  ): Lazy[Reflect.Record[G, A]] =
+    for {
+      binding <- transformMetadata(metadata)
+    } yield new Reflect.Record(fields, typeId, binding, ...)
+
+  // ... similar for other node types
+}
+```
+
+## Built-in Transformers
+
+### noBinding(): Remove Runtime Bindings
+
+The most common built-in transformer strips all runtime bindings:
+
+```scala
+// Definition
+def noBinding[F[_, _]](): ReflectTransformer[F, NoBinding]
+
+// Usage
+val boundSchema: Reflect[Binding, Person] = Schema[Person].reflect
+val unboundSchema: Reflect[NoBinding, Person] =
+  boundSchema.transform(DynamicOptic.root, ReflectTransformer.noBinding()).force
+
+// Or use the convenience property
+val unboundSchema: Reflect[NoBinding, Person] = boundSchema.noBinding
+```
+
+**Use Cases:**
+- Schema serialization (bindings contain functions that can't be serialized)
+- Schema comparison (compare structure without runtime details)
+- Schema transmission over the network
+
+## Practical Example: Custom Transformer
+
+Here's a complete example of a custom transformer that adds documentation to all fields:
+
+```scala
+import zio.blocks.schema._
+import zio.blocks.schema.binding._
+
+object DocumentingTransformer extends ReflectTransformer.OnlyMetadata[Binding, Binding] {
+  // For this transformer, we keep bindings unchanged
+  def transformMetadata[K, A](f: Binding[K, A]): Lazy[Binding[K, A]] = Lazy(f)
+
+  // Override record transformation to add docs
+  override def transformRecord[A](
+    path: DynamicOptic,
+    fields: IndexedSeq[Term[Binding, A, ?]],
+    typeId: TypeId[A],
+    metadata: Binding[BindingType.Record, A],
+    doc: Doc,
+    modifiers: Seq[Modifier.Reflect],
+    storedDefaultValue: Option[DynamicValue],
+    storedExamples: collection.immutable.Seq[DynamicValue]
+  ): Lazy[Reflect.Record[Binding, A]] = {
+    // Add path-based documentation
+    val enhancedDoc = doc match {
+      case Doc.Empty => Doc.Text(s"Record at path: ${path}")
+      case existing  => existing
+    }
+    Lazy(new Reflect.Record(
+      fields, typeId, metadata, enhancedDoc, modifiers,
+      storedDefaultValue, storedExamples
+    ))
+  }
+}
+
+// Usage
+val documented = schema.reflect.transform(DynamicOptic.root, DocumentingTransformer).force
+```
+
+## Transform Flow Visualization
+
+Here's how transform flows through a nested schema:
+
+```
+Input: Reflect[Binding, Person]
+       where Person(name: String, address: Address)
+             Address(street: String, city: String)
+
+Transform Flow:
+═══════════════
+
+1. Person.transform(root, f)
+   │
+   ├─▶ 2. name.transform(root/name, f)
+   │       └─▶ f.transformPrimitive(root/name, String, ...)
+   │           └─▶ Lazy[Reflect.Primitive[G, String]]
+   │
+   ├─▶ 3. address.transform(root/address, f)
+   │       │
+   │       ├─▶ 4. street.transform(root/address/street, f)
+   │       │       └─▶ f.transformPrimitive(root/address/street, String, ...)
+   │       │           └─▶ Lazy[Reflect.Primitive[G, String]]
+   │       │
+   │       ├─▶ 5. city.transform(root/address/city, f)
+   │       │       └─▶ f.transformPrimitive(root/address/city, String, ...)
+   │       │           └─▶ Lazy[Reflect.Primitive[G, String]]
+   │       │
+   │       └─▶ 6. f.transformRecord(root/address, [street', city'], Address, ...)
+   │               └─▶ Lazy[Reflect.Record[G, Address]]
+   │
+   └─▶ 7. f.transformRecord(root, [name', address'], Person, ...)
+           └─▶ Lazy[Reflect.Record[G, Person]]
+
+Output: Lazy[Reflect[G, Person]]
+```
+
+## Best Practices
+
+### 1. Always Start from Root
+
+```scala
+// Good: Start with root path
+schema.reflect.transform(DynamicOptic.root, transformer)
+
+// Also good: Use the noBinding convenience
+schema.reflect.noBinding
+```
+
+### 2. Handle Lazy Properly
+
+```scala
+// Be careful with when you force
+val lazy = schema.reflect.transform(path, f)
+
+// Good: Force when you need the result
+val result = lazy.force
+
+// Good: Chain transformations before forcing
+val chained = for {
+  step1 <- schema.reflect.transform(path, f1)
+  step2 <- step1.transform(path, f2)
+} yield step2
+val result = chained.force
+```
+
+### 3. Use OnlyMetadata When Possible
+
+```scala
+// If you only need to transform bindings, use OnlyMetadata
+object MyTransformer extends ReflectTransformer.OnlyMetadata[Binding, MyBinding] {
+  def transformMetadata[K, A](f: Binding[K, A]): Lazy[MyBinding[K, A]] =
+    Lazy(convertBinding(f))
+}
+```
+
+### 4. Consider Path for Context-Aware Transformations
+
+```scala
+override def transformPrimitive[A](
+  path: DynamicOptic,
+  primitiveType: PrimitiveType[A],
+  ...
+): Lazy[Reflect.Primitive[G, A]] = {
+  // Different transformation based on where we are in the schema
+  val pathStr = path.toString
+  if (pathStr.contains("password") || pathStr.contains("secret")) {
+    // Add security modifier for sensitive fields
+    Lazy(new Reflect.Primitive(..., modifiers = modifiers :+ Modifier.sensitive))
+  } else {
+    Lazy(new Reflect.Primitive(...))
+  }
+}
+```
+
+## Summary
+
+The `transform` method is a powerful mechanism for systematic schema transformation that:
+
+| Aspect | Description |
+|--------|-------------|
+| **Pattern** | Visitor pattern with lazy evaluation |
+| **Traversal** | Bottom-up (children first, then parent) |
+| **Recursion** | Handled via `Lazy` and `Deferred` caching |
+| **Path Tracking** | Via `DynamicOptic` for context-aware transforms |
+| **Extensibility** | Custom `ReflectTransformer` implementations |
+| **Type Safety** | Preserves schema type `A` while changing binding `F` to `G` |
+
+Understanding `transform` is essential for:
+- Implementing custom schema codecs
+- Building schema analysis tools
+- Creating schema-to-schema transformations
+- Understanding how ZIO Blocks handles schema serialization