@structuralists/scaffolding 0.11.0 → 0.13.0

package/src/forms/plan.md

@@ -28,7 +28,8 @@ TS wall can't strand finished work behind it.
    (state/useFormState/errorAt.ts) — the smallest surface that keeps stories
    readable; item 6 folds the same lookup into `getFormFieldPropsAt`'s
    `errorMessage`. The Debugger needed no change (`toInspectable`
-   index-keys arrays).
+   index-keyed arrays at the time; it now passes them through, since
+   `JsonTable` renders arrays natively).
 4. **Item 5 — elements/state split.** Pure churn, no type risk; do it before
    item 6 so the wrappers land in their final home once. ✅ *done* — one
    `src/forms/` umbrella: elements in `src/forms/elements/<Component>/`,
@@ -45,6 +46,18 @@ TS wall can't strand finished work behind it.
    Throwaway `.test-d.ts` of the full grammar at `InsuranceQuoteForm` scale *before*
    any runtime work; three outcomes (works / slow / intractable) each with a
    known response. Worst case, everything above still shipped.
+   - **Spike: ✅ done — verdict WORKS, decisively.** The full recursive
+     grammar (nested specs, `each`, arrays at every level, recursive
+     `Refine`) plus a stress tier (~110-leaf form, 12-level object ladder,
+     4-deep `each` ladder) cost **1.31× instantiations, check time flat** —
+     nowhere near the ~10×/multi-second failure thresholds. No depth caps,
+     lazy-recursion or boxing tricks needed. Two binding grammar
+     adjustments came out of it, both folded into the sketches below:
+     value-model-first branch order in `RefineField`, and the `Refine`
+     identity gate for the default-`V` case.
+   - **Phase 2 (nested object specs + recursive `Refine`): ✅ done** — see
+     "Phases" below.
+   - **Phase 3 (list `each` runtime): ✅ done** — see "Phases" below.
 
 ## Goal
 
@@ -126,30 +139,55 @@ useFormState({
 
 ## Refinement through the grammar
 
-`Refine` becomes recursive, mirroring the grammar. Sketch:
+`Refine` is recursive, mirroring the grammar. As shipped (phase 2, with both
+type-spike adjustments):
 
 ```ts
-type ExcludedOf<V> = V extends Refinement<infer X> ? X : never;
 // MemberExcludes<C>: the per-member-sound union of a validator tuple's
 // excludes — it and SoundExcludedOf landed with phase 1 in state/validations/types.ts.
 
-type RefineField<F, C> =
-  C extends Refinement<infer X>        ? Exclude<F, X>                 // single marked validator
-  : C extends readonly unknown[]       ? Exclude<F, MemberExcludes<C>> // validator array: union of per-member-sound excludes
-  : C extends { readonly each: infer E } ? F extends FormValueList
-      ? Array<RefineObject<F[number], E>> : F                          // per-element
-  : C extends object                   ? F extends FormValuesObject
-      ? RefineObject<F, C> : F                                         // nested spec
-  : F;                                                                 // bare fn / no marker
+type RefineField<F, C> = C extends Refinement<infer Excluded>
+  ? Exclude<F, Excluded>                    // single marked validator
+  : C extends readonly unknown[]
+    ? Exclude<F, MemberExcludes<C>>         // validator array: union of per-member-sound excludes
+    : C extends object                      // structural: interrogate F (the value model) FIRST
+      ? F extends FormValueList
+        ? C extends { readonly each: infer E }
+          ? Array<RefineObject<F[number], E>>  // per-element
+          : F
+        : F extends FormValuesObject
+          ? RefineObject<F, C>              // nested spec
+          : F
+      : F;                                  // bare fn / no marker
 
 type RefineObject<T, V> = { [K in keyof T]: K extends keyof V ? RefineField<T[K], V[K]> : T[K] };
+
+// The identity gate — NOT optional (spike adjustment 2): with constraints
+// omitted, the default V = Validations<T> would distribute through the walk
+// and hand back unions of structurally-identical mapped copies of every
+// section — assignable to T but not identity-equal (mangled hover types,
+// fails toEqualTypeOf). Validations<T> extends V exactly when V is the
+// default (or empty/fully-widened — no markers survive anyway).
+type Refine<T extends FormValuesObject, V extends Validations<T>> =
+  Validations<T> extends V ? T : RefineObject<T, V>;
 ```
 
-Branch-order constraints discovered in the baseline, which this sketch bakes
-in:
+Branch-order constraints baked in (baseline + spike):
 
 - **Check `Refinement` before `object`** — validator functions are objects.
-- **A bare function** (no marker) falls to the final `F` arm — narrows nothing.
+- **The structural arms branch on the value model (`F`) before the
+  constraint's shape** (spike adjustment 1): asking `C extends { each }`
+  first would strand an object field that legitimately owns a key named
+  `each` — its nested spec would match the `each` arm, fail the list test,
+  and fall back unrefined. Pinned by the `audit` disambiguation probe in
+  `useFormState.test-d.ts`.
+- **A bare function** (no marker) falls through: on a scalar field to the
+  final `F` arm; on a structural field into `RefineObject<F, C>` with
+  `keyof C` empty — an identity map, structurally unchanged, as a
+  whole-value validator should be.
+- Nested occurrences need no identity gate: a concrete literal `V` has only
+  the keys actually written, so uncovered fields take the `T[K]` arm
+  verbatim at every level.
 - The runtime cast in `submit()` stays the same single honest cast: valid ⇒
   every constrained node passed, at every depth, which is what the recursive
   `Refine` now encodes.
@@ -200,11 +238,44 @@ multi-second check time:
   0.82 s, 125,022 instantiations, 54,959 types (+10.7% instantiations over
   post-item-5 — the `ValueAt` instantiations at each binding call site are
   real but cheap; check time flat)
+- type spike (full recursive grammar, chunky + stress probes, throwaway
+  worktree — not merged): 163,262 instantiations / 65,036 types / 0.86s
+  check — 1.31× post-item-6; verdict works, no mitigation tricks required.
+  The real phases should land well under that: the spike carried a stress
+  tier (~110-leaf mega form with a large spec inlined twice, 12-level
+  object ladder, 4-deep `each` ladder) the phases don't.
+- post-phase-2 (recursive grammar + recursive `Refine` with identity gate,
+  ported chunky/boundary/negative probe suite, recursive walk): check
+  0.87 s, 133,956 instantiations, 57,762 types (+7.1% instantiations over
+  post-item-6; matches the spike's equivalent non-stress milestone at
+  134,441 almost exactly)
+- post-phase-3 (`each` runtime + the `ListValidation` story with its
+  numeric-path bindings; no new type machinery): check 0.84 s, 136,741
+  instantiations, 58,813 types (+1.8% over the pre-phase-3 HEAD at
+  134,330/58,034 — mostly the story's new hook call site and deep
+  `Path`/`ValueAt` bindings)
 
 ## Runtime consequences (can't be dodged)
 
-- **The validator walk becomes a tree walk.** Recursion mirrors `read()` in
-  `state/path/path.ts`; share the traversal or keep them deliberately parallel.
+- **The validator walk becomes a tree walk.** ✅ *Landed with phase 2*
+  (`validateEntry` in `state/validations/walk.ts` recurses, accumulating the
+  `PathStep[]` address; deliberately parallel to `read()`, not shared —
+  the walk descends a *constraints* tree, `read()` a *path*). Interpretation
+  of a structural spec is directed by the VALUE at the path, mirroring the
+  type level (see "Disambiguation").
+- **Absent sections: a nested spec on an absent/null section is skipped.**
+  *Decided with the type spike.* The type level commits to this implicitly —
+  `F` distributes naked through the structural grammar arms, so a spec on
+  `UsAddress | undefined` refines only the present branch and the
+  nullability survives around the refined interior — and the runtime walk
+  mirrors it: nothing to walk when the section is absent (same for `each`
+  over a null list). A **required section** is expressed as a *leaf*
+  validator on the section field (`mailingAddress: notEmpty(…)`), which
+  stays legal on structural fields — with the recorded caveat that leaf and
+  structural forms are mutually exclusive per key (the "whole-value +
+  structural constraints on the same key" open decision below). Never
+  "help" the grammar arms with `NonNullable<F>` wrappers — that breaks the
+  distribution and blows the recursion stack (TS2589).
 - **`FormErrors` can no longer be `Partial<Record<keyof T, string>>`.**
   ✅ *Landed with working-order step 3* (on the flat baseline, single-key
   paths). Decided: errors become a plain list of structured entries,
@@ -244,13 +315,34 @@ tests, story updates where visible, probe ratchet.
    distributes over a naked constraint parameter, which made union-typed
    *single* constraints (`cond ? v1 : v2`) sound too (union of per-branch
    refinements, pinned in `state/validations/types.test-d.ts`).
-2. **Nested object specs + recursive `Refine`.** The risk phase — the probe
-   ratchet matters most here. (The `{path, error}[]` error model already
-   landed in working-order step 3 — nested errors just carry longer paths;
-   no error plumbing changes in this phase.)
-3. **List `each` specs.** Runtime walks every element; error paths carry the
-   numeric step (`['drivers', 3, 'name']`). Refined element type flows
-   through `Array<...>`.
+2. **Nested object specs + recursive `Refine`.** The risk phase — de-risked
+   by the type spike. ✅ *done* — full recursive grammar
+   (`FieldConstraint`/`ListConstraint`/`Validations` in
+   state/validations/types.ts) and recursive `Refine` with both spike
+   adjustments (value-model-first `RefineField` branch order; the identity
+   gate); recursive `validateEntry` walk with absent-section skip; the
+   spike's chunky/boundary/soundness/negative probe suite ported into
+   `useFormState.test-d.ts` (the error model needed no change, as
+   predicted — nested errors just carry longer paths). **The `each` TYPE
+   arm landed here too** (the spike proved the grammar whole, and carving
+   it out of `RefineField` would have been artificial), but the runtime
+   walk for it was phase 3 — in the interim an `each` constraint on a
+   present list THREW from the walk (pinned in walk.test.ts) rather than
+   silently not validating; on a null list it skipped, which already
+   matched phase-3 semantics.
+3. **List `each` specs — runtime.** ✅ *done* — the walk visits every
+   element; error paths carry the numeric step (`['drivers', 3, 'name']`),
+   replacing the phase-2 throw. (The type level — refined element flowing
+   through `Array<...>` — already landed with phase 2.) Absent/null list ⇒
+   skip, same decided semantics as absent sections; elements fail
+   independently; walk-level semantics pinned React-free in walk.test.ts,
+   the typed boundary in deriveErrors.test.ts, element errors through
+   `errorMessage` at numeric paths in useFieldBinding.test.tsx, and the
+   visible flow in the `ListValidation` story. Two probes the spike said
+   to keep pinned were already in the phase-2 suite: the `audit`-style
+   "object field literally named `each`" disambiguation probe, and the
+   `each: <bare validator>` negative (rejected by TypeScript's weak-type
+   check — an obscure checker rule worth a canary).
 4. **Composition hardening.** Deep mixed fixtures (list-in-object-in-list),
    `perField` still the entry point for pre-built specs, docs
    (forms/CLAUDE.md) updated to the new grammar.

package/src/forms/state/useFormState/FormDebugger.test.tsx

@@ -44,7 +44,8 @@ describe('FormDebugger', () => {
 
     expect(screen.getByText('isValid')).toBeTruthy();
     expect(screen.getByText('submitAttempted')).toBeTruthy();
-    expect(screen.getByText("'nickname' cannot be empty")).toBeTruthy();
+    // String leaves render with surrounding quotes (Chrome-inspector style).
+    expect(screen.getByText('"\'nickname\' cannot be empty"')).toBeTruthy();
   });
 
   test('the open window live-updates as the form changes', () => {
@@ -57,8 +58,8 @@ describe('FormDebugger', () => {
 
     // The window survived the form re-render (stable component identity —
     // a remount would have reset it to closed) and shows the new state.
-    expect(screen.getByText('will')).toBeTruthy();
-    expect(screen.queryByText("'nickname' cannot be empty")).toBeNull();
+    expect(screen.getByText('"will"')).toBeTruthy();
+    expect(screen.queryByText('"\'nickname\' cannot be empty"')).toBeNull();
   });
 
   test('clicking the trigger again closes the window', () => {

package/src/forms/state/useFormState/deriveErrors.test.ts

@@ -74,3 +74,132 @@ describe('deriveFormErrors', () => {
     expect(deriveFormErrors(emptyForm, constraints)).toEqual([]);
   });
 });
+
+// The recursive grammar: nested object specs and list `each` specs walk the
+// value tree and address failures with real multi-step paths — numeric steps
+// for list elements. Walk-level semantics (element iteration, sibling
+// independence, absent-value skips) are pinned in walk.test.ts; these tests
+// pin the typed boundary: a constraints literal checked against the form
+// type produces `Path<T>`-addressed entries.
+
+type ProfileForm = {
+  email: string | undefined;
+  homeAddress: {
+    city: string | undefined;
+    postalCode: string | undefined;
+  };
+  mailingAddress: { city: string | undefined } | undefined;
+  pets: Array<{ name: string | undefined }>;
+  pastPolicies: Array<{ insurer: string | undefined }> | null;
+};
+
+const emptyProfile: ProfileForm = {
+  email: undefined,
+  homeAddress: { city: undefined, postalCode: undefined },
+  mailingAddress: undefined,
+  pets: [],
+  pastPolicies: null,
+};
+
+describe('deriveFormErrors — nested object specs', () => {
+  test('a failing nested leaf contributes an entry addressed by its full path', () => {
+    const errors = deriveFormErrors(emptyProfile, {
+      homeAddress: { city: notEmpty('city') },
+    });
+    expect(errors).toEqual([
+      { path: ['homeAddress', 'city'], error: "'city' cannot be empty" },
+    ]);
+  });
+
+  test('sibling fields fail independently — one entry per failing node', () => {
+    const errors = deriveFormErrors(emptyProfile, {
+      email: notEmpty('email'),
+      homeAddress: {
+        city: notEmpty('city'),
+        postalCode: notEmpty('postalCode'),
+      },
+    });
+    expect(errors).toEqual([
+      { path: ['email'], error: "'email' cannot be empty" },
+      { path: ['homeAddress', 'city'], error: "'city' cannot be empty" },
+      {
+        path: ['homeAddress', 'postalCode'],
+        error: "'postalCode' cannot be empty",
+      },
+    ]);
+  });
+
+  test('passing nested leaves contribute nothing', () => {
+    const errors = deriveFormErrors(
+      {
+        ...emptyProfile,
+        homeAddress: { city: 'SF', postalCode: '94110' },
+      },
+      {
+        homeAddress: {
+          city: notEmpty('city'),
+          postalCode: [notEmpty('postalCode'), matches('postalCode', /^\d{5}$/, '5 digits')],
+        },
+      },
+    );
+    expect(errors).toEqual([]);
+  });
+
+  test('a nested spec on an absent section is skipped', () => {
+    // Decided with the phase-2 type spike: the honest runtime mirror of the
+    // type level (which refines only the present branch of a nullable
+    // section) is to validate nothing when the section is absent.
+    const errors = deriveFormErrors(emptyProfile, {
+      mailingAddress: { city: notEmpty('city') },
+    });
+    expect(errors).toEqual([]);
+  });
+
+  test('the same nested spec fires once the section is present', () => {
+    const errors = deriveFormErrors(
+      { ...emptyProfile, mailingAddress: { city: undefined } },
+      { mailingAddress: { city: notEmpty('city') } },
+    );
+    expect(errors).toEqual([
+      { path: ['mailingAddress', 'city'], error: "'city' cannot be empty" },
+    ]);
+  });
+
+  test('a required section is a leaf validator on the section field', () => {
+    const errors = deriveFormErrors(emptyProfile, {
+      mailingAddress: notEmpty('mailingAddress'),
+    });
+    expect(errors).toEqual([
+      { path: ['mailingAddress'], error: "'mailingAddress' cannot be empty" },
+    ]);
+  });
+});
+
+describe('deriveFormErrors — list `each` specs', () => {
+  test('each element failure is addressed with the numeric index step', () => {
+    const profile: ProfileForm = {
+      ...emptyProfile,
+      pets: [{ name: 'Rex' }, { name: undefined }],
+    };
+    const errors = deriveFormErrors(profile, {
+      pets: { each: { name: notEmpty('name') } },
+    });
+    expect(errors).toEqual([
+      { path: ['pets', 1, 'name'], error: "'name' cannot be empty" },
+    ]);
+  });
+
+  test('an `each` spec on a null list is skipped; a leaf validator is how a list is required', () => {
+    const errors = deriveFormErrors(emptyProfile, {
+      pastPolicies: { each: { insurer: notEmpty('insurer') } },
+    });
+    expect(errors).toEqual([]);
+
+    const required = deriveFormErrors(emptyProfile, {
+      pastPolicies: notEmpty('pastPolicies'),
+    });
+    expect(required).toEqual([
+      { path: ['pastPolicies'], error: "'pastPolicies' cannot be empty" },
+    ]);
+  });
+});

package/src/forms/state/useFormState/deriveErrors.ts

@@ -1,7 +1,7 @@
 import type { Path } from '../path/types';
 import type { Validations } from '../validations/types';
 import { validateEntry } from '../validations/walk';
-import type { FlatConstraintEntry } from '../validations/walk';
+import type { ConstraintEntry } from '../validations/walk';
 import type { FormError, FormValuesObject } from './types';
 
 // The pure half of the hook's error model: current values + constraints in,
@@ -19,16 +19,16 @@ export const deriveFormErrors = <T extends FormValuesObject>(
   for (const key of Object.keys(constraints) as (keyof T & string)[]) {
     // No cast: if the grammar grows a constraint form the walk doesn't
     // understand, this assignment is the compile error that says so.
-    const entry: FlatConstraintEntry | undefined = constraints[key];
+    const entry: ConstraintEntry | undefined = constraints[key];
     if (entry == null) continue;
-    const failure = validateEntry(entry, values[key], [key]);
-    if (failure == null) continue;
-    // The walk returns the address it was handed, and `[key]` — a key of
-    // a constraints object type-checked against T — is a valid single-key
-    // Path<T>. TS can't compute Path<T> for an unresolved generic T, so
-    // the correlation needs the same honest widening as the keys cast
-    // above.
-    errors.push({ path: failure.path as Path<T>, error: failure.error });
+    for (const failure of validateEntry(entry, values[key], [key])) {
+      // The walk extends the address it was handed only along keys of specs
+      // type-checked against T's subtree, so every returned path is a valid
+      // Path<T>. TS can't compute Path<T> for an unresolved generic T to
+      // see the correlation, so it needs the same honest widening as the
+      // keys cast above.
+      errors.push({ path: failure.path as Path<T>, error: failure.error });
+    }
   }
 
   return errors;

package/src/forms/state/useFormState/errorAt.test.ts

@@ -2,9 +2,9 @@ import { describe, test, expect } from 'bun:test';
 import { errorAt } from './errorAt';
 import type { FormErrors } from './types';
 
-// The flat grammar only produces single-key paths today, but errorAt's
-// equality must already be exact over multi-step and numeric-step paths —
-// the recursive grammar (plan phases 2–3) reuses it unchanged.
+// errorAt's equality must be exact over multi-step and numeric-step paths —
+// the recursive grammar produces real multi-step addresses (nested specs,
+// and numeric index steps from runtime `each` over list elements).
 type Form = {
   email: string | undefined;
   address: { city: string | undefined };

package/src/forms/state/useFormState/inspectable.test.ts

@@ -11,15 +11,15 @@ describe('toInspectable', () => {
     expect(toInspectable(true)).toBe(true);
   });
 
-  test('converts arrays to index-keyed objects', () => {
-    expect(toInspectable(['a', 'b'])).toEqual({ 0: 'a', 1: 'b' });
+  test('keeps arrays as arrays (JsonTable renders them natively)', () => {
+    expect(toInspectable(['a', 'b'])).toEqual(['a', 'b']);
   });
 
-  test('converts arrays of objects recursively', () => {
-    expect(toInspectable([{ name: 'ada' }, { name: 'bo' }])).toEqual({
-      0: { name: 'ada' },
-      1: { name: 'bo' },
-    });
+  test('walks array elements recursively (Sets may hide inside)', () => {
+    expect(toInspectable([{ roles: new Set(['admin']) }, { name: 'bo' }])).toEqual([
+      { roles: 'Set(1) { "admin" }' },
+      { name: 'bo' },
+    ]);
   });
 
   test('renders Sets as a descriptive string leaf', () => {
@@ -35,8 +35,8 @@ describe('toInspectable', () => {
     };
     expect(toInspectable(form)).toEqual({
       email: 'a@b.co',
-      address: { city: undefined, tags: { 0: 'home' } },
-      drivers: { 0: { name: 'ada', incidents: {} } },
+      address: { city: undefined, tags: ['home'] },
+      drivers: [{ name: 'ada', incidents: [] }],
     });
   });
 });

package/src/forms/state/useFormState/inspectable.ts

@@ -1,10 +1,10 @@
 // Converts a form-state snapshot into a shape JsonTable renders without
-// throwing. JsonTable dispatches on "plain object vs leaf": arrays and Sets
-// fall to the leaf renderer, and an array of objects (FormValueList) would
-// throw as a React child. A debugger must render *any* legal form state, so:
+// throwing. JsonTable recurses into plain objects and arrays; everything
+// else falls to the leaf renderer, and a Set there would throw as a React
+// child. A debugger must render *any* legal form state, so:
 //
-//   - arrays  → index-keyed plain objects ({ 0: ..., 1: ... }), recursively
 //   - Sets    → a descriptive string leaf: `Set(2) { "a", "b" }`
+//   - arrays  → walked recursively (a Set may hide inside), kept as arrays
 //   - objects → walked recursively
 //   - leaves  → passed through untouched
 const isPlainObject = (value: unknown): value is Record<string, unknown> =>
@@ -20,9 +20,7 @@ export const toInspectable = (value: unknown): unknown => {
   }
 
   if (Array.isArray(value)) {
-    return Object.fromEntries(
-      value.map((element, index) => [index, toInspectable(element)]),
-    );
+    return value.map(toInspectable);
   }
 
   if (isPlainObject(value)) {

package/src/forms/state/useFormState/types.ts

@@ -78,8 +78,8 @@ export type UnionPolicyCheck<T> = HasDisallowedUnion<T> extends true
   : unknown;
 
 // Structured error model: one entry per failing constrained node, addressed
-// by a typed path (single-key paths on the flat grammar; deeper addresses
-// arrive with the recursive grammar). Deliberately a plain list — at form
+// by a typed path as deep as the node (root leaves get single-key paths,
+// nested-spec leaves get the full address). Deliberately a plain list — at form
 // scale a linear scan is fine, and serialized string keys ('drivers.0.name')
 // are never exposed. Read a field's error with `errorAt` (./errorAt.ts),
 // never by hand-assembled keys.

package/src/forms/state/useFormState/useFieldBinding.test.tsx

@@ -138,6 +138,40 @@ describe('getFormFieldPropsAt — error-display policy', () => {
     ).toBeUndefined();
   });
 
+  test('element errors from an `each` spec surface at numeric paths, per element', () => {
+    const { result } = renderHook(() =>
+      useFormState({
+        initialValues,
+        constraints: { pets: { each: { name: notEmpty('name') } } },
+      }),
+    );
+
+    act(() => {
+      result.current.getFormFieldPropsAt(['pets', 1, 'name']).onChange(undefined);
+    });
+
+    // The raw list addresses the failing element by its index step …
+    expect(result.current.errors).toEqual([
+      { path: ['pets', 1, 'name'], error: "'name' cannot be empty" },
+    ]);
+    // … and errorMessage applies the same display policy at that path.
+    expect(
+      result.current.getFormFieldPropsAt(['pets', 1, 'name']).errorMessage,
+    ).toBeUndefined();
+
+    act(() => {
+      result.current.getFormFieldPropsAt(['pets', 1, 'name']).onBlur();
+    });
+
+    expect(
+      result.current.getFormFieldPropsAt(['pets', 1, 'name']).errorMessage,
+    ).toBe("'name' cannot be empty");
+    // The sibling element passed — no error at its own numeric path.
+    expect(
+      result.current.getFormFieldPropsAt(['pets', 0, 'name']).errorMessage,
+    ).toBeUndefined();
+  });
+
   test('repeat blurs on the same path keep the touched list stable', () => {
     // At the useFieldBinding boundary, where the touched list is returned.
     const { result } = renderHook(() =>