graph_weaver 0.0.1 → 0.2.0

data/PLAN.md

@@ -1,7 +1,7 @@
-# Project Plan — typed GraphQL codegen for Ruby/Sorbet
+# Project Plan — GraphWeaver, typed GraphQL client for Ruby/Sorbet
 
-_Resume-from-here notes. The README is the lab notebook (findings); this is
-the plan. Update both when state changes._
+_Resume-from-here notes. README documents the product, NOTES.md is the
+research notebook this grew out of; this is the plan. Update on change._
 
 ## Vision
 
@@ -12,21 +12,40 @@ shape of every query result. Dynamic (eval) mode for development, build
 step for CI/static checking. Runtime deps: graphql + sorbet-runtime only
 (graphql-client is NOT a dependency; the exploration outgrew it).
 
-## State: working prototype, all green
+## State: v0.0.1 on rubygems; v0.0.2 accumulating on main
 
-`bundle exec rspec` (35 examples) + `bundle exec srb tc` + `bin/generate`
-(regenerates lib/generated/ from queries/; parity specs enforce freshness).
+`make check` = bin/generate (spec fixture regeneration; parity specs
+enforce freshness) + rspec + srb tc.
 
-Language coverage: queries, mutations, typed variables (kwargs on execute,
-optional-when-defaulted, enum/scalar serialization), fragments (inline,
-named, interface conditions), union- AND interface-typed fields
-(__typename dispatch, required at generation time), enums (T::Enum),
-custom scalars (name-keyed SCALAR_CASTS / SCALAR_SERIALIZERS registries).
+Language coverage: queries, mutations, typed variables (kwargs on
+execute, optional-when-defaulted), fragments (inline, named, interface
+conditions), union- AND interface-typed fields (__typename dispatch,
+required at generation time), enums (T::Enum), custom scalars via the
+ScalarType registry (register_scalar: codec inference off .parse/.load,
+requires:, opt-in input coercion incl. built-ins).
 Sources: live schema / introspection JSON / SDL — byte-identical output
 (enum values + abstract-type members sorted for determinism).
-Transport: executor: kwarg — in-process schema or HttpExecutor (e2e spec
-against WEBrick). Federation supergraph SDL loads transparently (needs
+Transport: executor precedence per call → per module → baked const →
+GraphWeaver.executor; Transport::HTTP (zero-dep) + opt-in Transport::Faraday
+(url / block middleware / ready connection), e2e specs against WEBrick.
+Errors: typed Response envelope (partial data + extensions survive) and
+a GraphWeaver::Error hierarchy (Transport/Server/Query/Validation/Type)
+with extensible transport-error classification, schema_stale? staleness
+detection, errors_at/each_error/report field-level surfacing (entity
+ids resolved from partial data), and #to_h machine output throughout.
+Dynamic mode: GraphWeaver.parse (paths or raw strings, derived names,
+container-scoped constants) and GraphWeaver.execute one-shots.
+Schema fetching: SchemaLoader.introspect(executor, cache:, ttl:) off
+live endpoints; load takes paths, content, or Hashes.
+Testing (graph_weaver/testing + graph_weaver/rspec): FakeExecutor
+(schema-correct castable fakes, faker semantics, GraphQL-name
+overrides, corrupt:, fail_at: with null propagation), Failure canned
+executors + SequenceExecutor, cassettes with shape-preserving
+anonymization, rspec seed + auto_fake integration. Selection module is
+the single shared query-walk (codegen/fake/anonymizer).
+Federation supergraph SDL loads transparently (needs
 directive_defaults_patch until upstream fix ships).
+Live validation: make integration (GitHub + Countries APIs).
 
 ## Next steps (in rough order)
 
@@ -38,21 +57,32 @@ NOTES.md. Prior-art check partially answered: graphql-client PR #7
 Jan 2024 with users asking; schema-wide typing can't catch
 unfetched-field bugs or type unions/interfaces — the niche looks open.
 
+~~Input objects~~ DONE 2026-07-11: module-level T::Structs, serialize/
+to_h, hash coercion at the execute boundary.
+~~Release~~ 0.1.0 cut 2026-07-11 (breaking: execute returns the
+Response envelope; execute! for raise-or-result).
+
 1. Stable class naming design — names come from GraphQL type names per
    selection site; must not shift when unrelated selections are added
    (generated code is app-code API). Current: one-level field-name
-   disambiguation, then raise.
-2. Input objects as variables (raise NotImplementedError today) — likely
-   generated T::Structs with serialize.
-3. CLI entrypoint (graph_weaver generate --schema X --queries dir) —
+   disambiguation, then raise. Shipped in 0.1.0 as-is — a naming change
+   is fair game pre-1.0 but should land early.
+2. CLI entrypoint (graph_weaver generate --schema X --queries dir) —
    bin/generate is spec-fixture tooling, not shipped.
-4. Subscriptions (unsupported; raise).
-5. First release: 0.1.0 to rubygems once naming design settles.
-6. Nice-to-haves: __typename auto-injection (currently required manually
+3. Subscriptions (unsupported; raise). Recursive input types (raise).
+6. Parse/execute memoization: repeated GraphWeaver.parse/execute of the
+   same [schema, query] re-generates and re-evals every call (~3x the
+   cost of a cached module; benchmarked 2026-07-09) — memo keyed on
+   schema/query/name/executor, minding shared executor= mutation.
+   (Schema-side caching landed: SchemaLoader.introspect cache:/ttl: +
+   introspection_result primitive for Rails.cache et al. Possible
+   follow-up: re-introspect + retry once on validation-shaped
+   QueryErrors, since GraphQL has no standard schema-version signal.)
+7. Nice-to-haves: __typename auto-injection (currently required manually
    on abstract selections), fragment reuse across queries, directives on
    selections (@skip/@include make non-null fields nullable).
-7. Tapioca DSL compiler over dynamic mode (idea from graphql-client
-   PR #7): RBI the GraphWeaver::Codegen.load-eval'd modules so development mode
+8. Tapioca DSL compiler over dynamic mode (idea from graphql-client
+   PR #7): RBI the GraphWeaver.parse-eval'd modules so development mode
    gets static types without the bin/generate build step — tapioca is
    already in every Sorbet shop's workflow. Upstream's
    Tapioca::Dsl::Helpers::GraphqlTypeHelper is prior art for type mapping.
@@ -72,3 +102,36 @@ unfetched-field bugs or type unions/interfaces — the niche looks open.
   resolvers — codegen must stay name-keyed, never call schema runtime hooks
 - graphql-client (the gem) casts scalars via coerce_isolated_input and
   only with a live schema class — documented in the early specs
+
+## From the field-test experiments (2026-07-12)
+
+A junior + senior agent pair exercised the repo cold (clone, examples,
+extensions, a Pokedex app against Hasura's 4,441-type PokeAPI schema).
+Fixed same-day: snake_case type names generated invalid constants
+(camelize), GraphQL::ParseError/NotImplementedError escaping the Error
+umbrella, HttpExecutor timeouts, GitHub's top-level "type" error codes,
+typo'd client registrations silently no-oping. Still open:
+
+~~Recursive input types~~ DONE 2026-07-12: register-before-walk breaks
+the recursion; emission dependency-orders the structs and, for cycles,
+forward-declares the classes in an eval (srb rejects reopening a
+T::Struct statically, but adding props at runtime works — the full
+bodies below the eval are all srb sees). Verified live against Hasura
+bool_exp.
+
+~~Connection reuse in Transport::HTTP~~ DONE 2026-07-12: persistent
+keep-alive connection behind a mutex, dropped on any failure; net/http's
+keep_alive_timeout handles idle expiry. Faraday remains the pooling
+answer.
+
+- Subscriptions; @defer/@stream (routers send multipart responses — the
+  transport classifies them as ServerError today, no incremental support).
+- Shared input-type structs across generated modules: one Hasura
+  bool_exp variable pulls its whole recursive closure into EVERY module
+  (~28k lines each) — correct but heavy in PRs; needs cross-module
+  sharing or selection-based pruning (field-test round 2).
+- Structured logging: log_tag pairs lines and names operations now, but
+  events are prose — an optional {event:, url:, ms:} payload contract, a
+  scrub_variables hook, and cache-age on hit lines (field-test round 2).
+- Cut 0.1.1 — RubyGems 0.1.0 is materially behind main (snake_case fix,
+  recursive inputs, keep-alive, logging, error umbrella).

data/README.md

@@ -19,49 +19,113 @@ query($id: ID!) {
 ```
 
 ```ruby
-result = PersonQuery.execute(id: "1")
+result = PersonQuery.execute!(id: "1")   # typed result, or raises on errors (execute returns an envelope)
 
 result.person&.name       # => "Daniel" (typed String)
 result.person&.birthday   # => Date (custom scalars deserialize)
 result.person&.nmae       # => srb tc: Method `nmae` does not exist
 ```
 
-####  Features
+New here? The **[getting started](docs/getting_started.md)** guide walks the
+production setup end to end — initializer, codegen, fakes, CI. Or run the
+**[examples](examples/)**, smallest first: `examples/countries.rb` (public
+API, no auth, all dynamic), `examples/rick_and_morty.rb` (filtering,
+pagination, a block-built type helper), and `examples/github/run.rb`
+(auth + checked-in generated modules; it stars this repo ⭐ and introduces
+you to your fellow stargazers).
 
-- **Queries and mutations** with typed variable kwargs — required vs optional falls out of nullability and defaults
-- **Fragments** (inline, named, interface conditions), **unions and interfaces** (member structs, `__typename` dispatch), **enums** (`T::Enum`), **custom scalars** (pluggable registry)
-- **Any schema source**: live schema class, introspection JSON, or SDL — including Apollo Federation supergraph SDL
-- **Any transport**: in-process schema execution (perfect for tests) or HTTP via the bundled executor — swap per call with `executor:`
-- **Dynamic mode** for development: `GraphWeaver::Codegen.load(...)` generates and evals on the fly, no build step
+#### Features
 
-####  Usage
+- **Queries and mutations** with typed variable kwargs — enums as `T::Enum`s, input objects as `T::Struct`s, required vs optional falling out of nullability and defaults
+- **Fragments** (inline, named, type conditions), **unions and interfaces** (member structs, `__typename` dispatch), **custom scalars** (pluggable registry), `@skip`/`@include` nullability
+- **Any schema source**: live schema class, introspection JSON, or SDL — including Apollo Federation supergraph SDL; introspect live endpoints with caching
+- **Any transport**: in-process schema execution, the zero-dependency HTTP executor, or Faraday with your own middleware — plus a composable `Retry` (exponential/linear/custom backoff, jitter, retry-by-error-class or GraphQL code) — swap per call with `executor:`
+- **Structured errors**: a typed response envelope (partial data + extensions survive), an error hierarchy split by failure site, field-level reports with entity ids, and `schema_stale?` detection — every error dual-surfaced as a human message plus JSON-ready `#to_h`
+- **Testing built in**: schema-correct fakes, failure simulation, record/replay cassettes with anonymization, rspec integration
+- **Dynamic mode** for development: `GraphWeaver.parse(...)` generates and evals on the fly, no build step
+
+#### Usage
+
+Three ways to run a query — pick by context:
+
+| Context | Use |
+|---------|-----|
+| Production | checked-in codegen (`rake graph_weaver:generate`) — reviewed, `srb tc`-checked |
+| Development, consoles | `client.parse` / `client.load_queries!` — no build step |
+| Scripts, one-offs | `client.execute!` — no module at all |
+
+The production path assembled is the [getting started](docs/getting_started.md);
+the pieces:
 
 ```ruby
 require "graph_weaver"
 
-# generate from any schema source
-schema = GraphWeaver::SchemaLoader.load("schema.json")   # or .graphql SDL, or a live class
+# a client for one server: transport (Faraday when loaded), auth, and a
+# lazily introspected schema. The first argument is a url or any schema
+# source — a live schema class, or a .json/.graphql dump
+api = GraphWeaver.new("https://api.example.com/graphql", auth: ENV["API_TOKEN"], cache: true)
 
-source = GraphWeaver::Codegen.new(
-  schema:,
-  executor_const: "MyApi::Executor",
+# make it the app default — generated modules execute through it
+GraphWeaver.client = api
+
+# generate checked-in typed modules (rake graph_weaver:generate, or directly)
+source = GraphWeaver::Codegen.generate(
+  schema: api.schema,
   query: File.read("queries/person.graphql"),
   module_name: "PersonQuery",
-).generate
-
+)
 File.write("app/queries/person_query.rb", source)
 
 # at runtime
-executor = GraphWeaver::HttpExecutor.new("https://api.example.com/graphql")
-PersonQuery.execute(id: "1", executor:)
+PersonQuery.execute(id: "1")                        # via GraphWeaver.client
+PersonQuery.execute(other_client, id: "1")          # or per call
 ```
 
-In development, skip the build step:
+Module names derive from the operation name (`query GetPerson` →
+`GetPerson`) or, for `parse` on a `.graphql` file, from the file name;
+pass `module_name:`/`name:` to override. Pass `client:` (a constant) to
+bake a default client into the generated module. Prefer Faraday? It's
+opt-in (`gem "faraday"`), and the client picks it up when loaded —
+middleware blocks and ready connections in [transports](docs/transports.md).
+
+In development, skip the build step entirely — modules from `client.parse`
+carry the client's transport, no global wiring needed:
 
 ```ruby
-PersonQuery = GraphWeaver::Codegen.load(schema:, executor_const: "...", query:, module_name: "PersonQuery")
+# parse a query into a typed module on the fly — a .graphql path or a raw string
+PersonQuery = api.parse("queries/person.graphql")
+PersonQuery.execute(id: "1")
+
+# or every query file at once (queries_path convention), named like generation would
+api.load_queries!
+
+# or one-shot, no module at all — variables are plain kwargs
+api.execute!("query($id: ID!) { person(id: $id) { name } }", id: "1")
 ```
 
+
+#### Dig deeper
+
+- **[Getting started](docs/getting_started.md)** — the production path in Rails,
+  step by step: initializer, rake tasks, fakes, CI, Sorbet or not
+- **[Generated modules](docs/generated_modules.md)** — module anatomy, typed
+  variables (enums, input objects), fragments/unions/interfaces,
+  `@skip`/`@include`, naming, clients, dynamic mode
+- **[Against a real API](docs/real_world.md)** — the exploratory tour:
+  introspect a live endpoint (GitHub end to end), dynamic mode, schema caching
+- **[Transports](docs/transports.md)** — clients, the execute contract,
+  Faraday, retries and backoff
+- **[Custom scalars](docs/scalars.md)** — the registry: codec inference,
+  requires, input coercion
+- **[Errors](docs/errors.md)** — the Response envelope, the error hierarchy,
+  field-level reports with entity ids, stale-schema detection
+- **[Logging](docs/logging.md)** — point `GraphWeaver.logger` at any Logger:
+  wire traffic at debug, introspection/cache/codegen at info, errors at warn
+- **[Testing](docs/testing.md)** — schema-correct fakes, failure simulation,
+  rspec integration
+- **[Cassettes](docs/cassettes.md)** — capture and replay real API
+  responses; anonymized recording (`GRAPHWEAVER_RECORD=1`, rake tasks)
+
 ----
 ## Installation
 
@@ -72,16 +136,12 @@ gem "graph_weaver"
 
 or
 
-```
+```sh
 gem install graph_weaver
 ```
 
 ----
 ## Development
 
-`make check` — regenerate spec fixtures, run specs, typecheck.
-
-See `PLAN.md` for roadmap and `NOTES.md` for the research notebook this
-gem grew out of (an exploration of graphql-client internals — GraphWeaver
-is a standalone client, not an extension; it depends only on `graphql`
-and `sorbet-runtime`).
+- `make check` — regenerate spec fixtures, run specs, typecheck
+- `make integration` — one-off checks against the live GitHub and Countries APIs

data/docs/cassettes.md

@@ -0,0 +1,76 @@
+# Cassettes: capture and replay
+
+Cassettes record real API responses and replay them in tests — above the
+transport (a client wrapping a client), so there's no HTTP
+interception and they work identically over HTTP, Faraday, or in-process
+execution. A cassette is a YAML file of `{query, variables, response}`
+entries, matched on the normalized query + variables.
+
+## The workflow
+
+```ruby
+# spec: replay when the cassette exists, record against `live` when not
+cassette = GraphWeaver::Testing::Cassette.use("github", client: live)
+result = RepoQuery.execute!(cassette, owner: "dpep", name: "graph_weaver")
+```
+
+1. **Record** — first run hits the live API and writes
+   `spec/cassettes/github.yml` (`Testing.config.cassette_dir` resolves bare
+   names).
+2. **Anonymize** — cassettes hold real data; scrub before committing (below).
+3. **Commit** — tests now run offline, fast, deterministic.
+4. **Re-record** when the API's real behavior changes:
+
+   ```sh
+   GRAPHWEAVER_RECORD=1 bundle exec rspec   # every Cassette.use records afresh
+   ```
+
+   (`Testing.config.record = true` is the programmatic equivalent.)
+
+Replaying an unrecorded request raises `MissingRecording` with the query
+and the path — no silent fabrication.
+
+## Anonymization
+
+Anonymizing rewrites recorded values through the same engine
+[FakeClient](testing.md) uses, while preserving everything that makes
+the recording faithful:
+
+| preserved | replaced |
+|-----------|----------|
+| shape: keys, list lengths, null positions | strings (semantically: emails look like emails) |
+| enums, booleans, `__typename` | numbers, dates |
+| id *relationships* (same original id → same fake id) | the id values themselves |
+
+Three ways to run it:
+
+```ruby
+# 1. as recordings happen — assertions you write against the recording
+#    run hold on replay, and real data never touches disk
+GraphWeaver::Testing.configure do |config|
+  config.schema = MySchema
+  config.anonymize = true
+end
+
+# 2. after the fact, per cassette
+GraphWeaver::Testing::Cassette.new("spec/cassettes/github.yml").anonymize!(schema:)
+```
+
+```sh
+# 3. the whole cassette_dir at once
+rake graph_weaver:cassettes:anonymize
+```
+
+Anonymization needs the schema (it walks each recorded query's selections
+to know which values are enums, dates, ids...). Variables are NOT
+anonymized — they're the replay matching key; don't record with secret
+variables.
+
+## When to use what
+
+- **FakeClient** — no recording needed; schema-correct random data.
+  Best default for unit tests.
+- **Cassettes** — real response *shapes* from a real API (pagination
+  quirks, actual union members, servers' null habits). Best for
+  integration-ish tests and regression pinning.
+- **Anonymized cassettes** — cassette fidelity, committable without PII.

data/docs/errors.md

@@ -0,0 +1,134 @@
+# Errors
+
+`execute` returns a typed **`Response` envelope** rather than raising on GraphQL
+errors — so partial data and top-level `extensions` (cost, throttle) survive.
+`execute!` is the shortcut when you just want the result:
+
+```ruby
+PersonQuery.execute!(id: "1")   # => Result, or raises QueryError  (== execute(...).data!)
+
+response = PersonQuery.execute(id: "1")   # => GraphWeaver::Response[Result]
+response.data           # T.nilable(Result) — typed, present even on partial success
+response.errors         # Array[GraphWeaver::GraphQLError]
+response.errors?        # any top-level errors?
+response.extensions     # { "cost" => … } — rides on success too
+response.data!          # the Result, or raise GraphWeaver::QueryError
+```
+
+The envelope is a single generic `GraphWeaver::Response[Result]` — `response.data`
+stays fully typed to *this* query's result, no per-query wrapper class.
+
+Every `GraphQLError` exposes `#message`, `#locations`, `#path`, `#extensions`,
+and `#code` (`extensions["code"]`) — match on the **code**, not the message
+string (`response.errors.first.code == "THROTTLED"`).
+
+Everything GraphWeaver raises descends from `GraphWeaver::Error`, split by where
+it failed:
+
+| Class | When |
+|-------|------|
+| `TransportError` | never reached the server — DNS, connection refused, TLS, timeout |
+| `ServerError` | reached it, non-2xx HTTP — `#status`, `#body` |
+| `QueryError` | 200 body with top-level GraphQL errors — `#errors`, `#data`, `#extensions`, `#codes` |
+| `TypeError` | the response wouldn't cast into the generated structs — `#struct`, `#cause` |
+| `ValidationError` | build time: the query didn't validate against the schema |
+
+```ruby
+begin
+  person = PersonQuery.execute!(id: "1").person
+rescue GraphWeaver::TransportError
+  retry                                   # network blip
+rescue GraphWeaver::ServerError => e
+  e.status >= 500 ? backoff : raise       # retry 5xx; a 4xx is our bug
+rescue GraphWeaver::QueryError => e
+  e.codes.include?("THROTTLED") ? backoff : raise
+end
+```
+
+Or skip the hand-rolling — `Retry` wraps any transport with
+configurable retries:
+
+```ruby
+transport = GraphWeaver::Retry.new(
+  GraphWeaver::Transport::HTTP.new(url),
+  tries: 5,                        # total attempts
+  backoff: :exponential,           # or :linear, or ->(attempt) { seconds }
+  base: 0.5, max: 30,              # seconds, clamped at max:
+  jitter: true,                    # randomize each delay by 50-100%
+  retry_codes: ["THROTTLED"],      # also retry GraphQL errors by code
+)
+```
+
+Defaults match the rescue block above: transport failures always retry,
+`ServerError` only on 5xx (a 4xx is your bug — retrying won't fix it;
+override with `retry_if:`), and GraphQL-level codes only when listed in
+`retry_codes:`. Exhausting `tries:` re-raises the last error.
+
+Two deliberate exceptions live *outside* the hierarchy, because typed
+kwargs should fail like any Ruby method call: a wrong-typed variable
+raises sorbet-runtime's `TypeError` ("Parameter 'page': Expected type
+T.nilable(Integer), got type String"), and a missing required variable
+raises a plain `ArgumentError` ("missing keyword: :id").
+
+Business/validation failures returned *as data* (Shopify-style `userErrors { field
+message code }`) aren't errors here — they're just fields you selected, so they
+deserialize onto `response.data` like anything else and you inspect them there.
+
+The one-shot `GraphWeaver.execute` / `execute!` mirror this: `execute` returns
+the envelope, `execute!` the result-or-raise.
+
+## Extending TransportError
+
+What counts as a `TransportError` is an **extensible set** — each transport
+seeds its own network exceptions (`Errno::*`, `SocketError`, timeouts, TLS; the
+Faraday transport adds its own), and you can register more so a custom adapter's
+or connection pool's failure gets the same treatment:
+
+```ruby
+GraphWeaver.register_transport_error(ConnectionPool::TimeoutError)
+GraphWeaver.transport_errors << MyAdapter::ResetError   # it's just a Set
+```
+
+
+## Programmatic surfacing
+
+Every error is dual-surface: `#message` for humans, `#to_h` for machines — a
+JSON-ready hash (error class, per-error `path`/`code`/`locations`/`extensions`)
+you can nest straight into a log line or an API response.
+
+Field-level tooling lives on both `Response` and `QueryError`:
+
+```ruby
+response.errors_at("person.email")      # errors touching a path (prefix match)
+response.each_error do |field, errors|  # grouped by index-stripped field
+  form.add_error(field, errors.map(&:message))
+end
+
+response.report
+# { "person.pets.name" => {
+#     "messages" => ["name hidden"], "codes" => ["PRIVATE"],
+#     "entity_ids" => ["7", "9"],    # resolved by walking paths through partial data
+#     "errors" => [ ...full to_h detail... ] },
+#   nil => { "codes" => ["DOWN"], ... } }   # global errors under nil
+```
+
+`GraphQLError#field` strips list indices (`people.3.email` → `people.email`) —
+the stable grouping key; the raw `#path` keeps indices for exact location.
+
+## Stale schemas
+
+GraphQL has no schema-version signal, so a schema change surfaces as the
+server rejecting your query's shape. `response.schema_stale?` /
+`QueryError#schema_stale?` detect validation-shaped rejections (Apollo's
+`GRAPHQL_VALIDATION_FAILED` code, or the message patterns graphql-ruby and
+GitHub use), and the raised message says what to do: regenerate modules and/or
+refresh the schema cache.
+
+## Cast failures
+
+When wire data disagrees with the types the schema promised at generation time
+(a nil where non-null was declared, a malformed scalar, an unknown enum value),
+casting raises `GraphWeaver::TypeError` naming the failing generated struct,
+with the original exception as `#cause`. Simulate one in tests with
+`GraphWeaver::Testing::FakeClient.new(schema:, corrupt: "Person.birthday")` — see
+[testing](testing.md).