keiyaku 0.1.0

data/README.md

@@ -0,0 +1,174 @@
+# keiyaku
+
+[![ci](https://github.com/onyxblade/keiyaku/actions/workflows/ci.yml/badge.svg)](https://github.com/onyxblade/keiyaku/actions/workflows/ci.yml)
+
+契約 — an OpenAPI-to-Ruby client generator built on the premise that almost
+everything existing generators emit is boilerplate that belongs in a runtime
+library. A spec is a contract; the generated code should be the part of it that
+is actually specific to your API, and nothing else.
+
+Generating the [Swagger Petstore](examples/petstore.yaml) (OAS 3.0.4, 19
+operations, 6 schemas):
+
+| | `openapi-generator -g ruby` | this |
+| --- | --- | --- |
+| generated Ruby | 4,316 lines across 25 files | **67 lines across 2 files** |
+| RBS | none | 88 lines |
+| shared runtime | — | 939 lines, written once |
+
+## Install
+
+```ruby
+gem "keiyaku"
+```
+
+One gem, no dependencies. The runtime is what an application loads; the
+generator is a separate file that only the executable pulls in, so nothing
+that merely calls an API ever loads it.
+
+## Generate
+
+```
+keiyaku petstore.yaml --module Petstore --out lib/petstore
+```
+
+Three files land in the output directory: `types.rb`, `client.rb`, and an
+`.rbs` carrying the type detail. Check them in — they are source, not build
+output, and reading the diff is how you see what changed in an API.
+
+`client.rb` is a table of operations, one line each:
+
+```ruby
+module Petstore
+  class Client < Keiyaku::Client
+    server "https://petstore3.swagger.io/api/v3"
+    security({ petstore_auth: :bearer, api_key: { header: "api_key" } })
+
+    put    :update_pet, "/pet", body: Pet, into: Pet, security: :petstore_auth
+    post   :add_pet, "/pet", body: Pet, into: Pet, security: :petstore_auth
+    get    :find_pets_by_status, "/pet/findByStatus", query: %i[status], required: %i[status], into: [Pet], security: :petstore_auth
+    get    :get_pet_by_id, "/pet/{petId}", into: Pet, security: [[:api_key], [:petstore_auth]]
+    delete :delete_pet, "/pet/{petId}", header: { "api_key" => :api_key }, security: :petstore_auth
+    get    :get_inventory, "/store/inventory", into: { String => Integer }, security: :api_key
+    post   :create_users_with_list_input, "/user/createWithList", body: [User], into: User
+    # ...
+  end
+end
+```
+
+`types.rb` is a value type per schema:
+
+```ruby
+Pet = Keiyaku.model({
+  id: Integer, name: String, category: Category, photo_urls: [String],
+  tags: [Tag], status: String
+}, required: %i[name photo_urls])
+```
+
+and the RBS beside them carries the full signature, so terse Ruby costs nothing
+in tooling:
+
+```rbs
+def find_pets_by_status: (status: String) -> Array[Pet]
+def update_pet_with_form: (Integer pet_id, ?name: String?, ?status: String?) -> Pet
+def get_inventory: () -> Hash[String, Integer]
+```
+
+## Call
+
+```ruby
+client = Petstore::Client.new(auth: { api_key: ENV["PETSTORE_KEY"] })
+
+pet = client.get_pet_by_id(5)          # => Petstore::Pet
+pet.photo_urls                         # camelCase mapped to snake_case
+pet.with(name: "Nori")                 # models are frozen values; copies are cheap
+
+case client.find_pets_by_status(status: "available")
+in [{ name:, category: { name: "Dogs" } }, *] then ...
+end
+```
+
+Methods have real arity — `get_pet_by_id(5)` with the wrong number of arguments
+raises `ArgumentError` at the call, not somewhere inside the client.
+Credentials are given by the name the document gave the scheme, and each
+operation is authenticated the way the document says *that* operation is.
+
+## What it covers
+
+- **Models** — frozen value types, compared by value, copied with `#with`,
+  pattern-matched. Names are mapped in both directions; `additionalProperties`
+  are carried rather than dropped, and round-trip losslessly.
+- **Parameters** — `form` and `deepObject` query styles, `simple` for path and
+  header, with the specification's `explode` defaults. `required:` says which a
+  caller has to pass.
+- **Bodies** — JSON, a server's own `+json` media type, text, binary, and
+  `multipart/form-data` with file uploads.
+- **Responses** — cast per status, including by range; `Keiyaku::ByStatus` for
+  an operation whose statuses are different types; typed error bodies mapped to
+  `Keiyaku::ClientError` / `ServerError`.
+- **Unions** — a `oneOf`/`anyOf` with a `discriminator` becomes a
+  `Keiyaku::OneOf` that dispatches on it.
+- **Security** — compiled per operation: alternatives, schemes required
+  together, `security: []`. API key in a header, query or cookie; bearer,
+  basic, OAuth 2 and OpenID Connect tokens.
+- **Transport** — `net/http` by default, autoloaded only if used; adapters for
+  faraday and http.rb ship without either being a dependency. Retries back off
+  with jitter and honour `Retry-After`; `timeout:` takes `{ open:, read: }`; a
+  request that never happened is a `Keiyaku::ConnectionError` whatever is
+  underneath.
+
+## What it refuses
+
+The failure mode that matters is a generator emitting plausible code that is
+subtly wrong. When this one cannot translate a construct faithfully it says so
+at generation time and emits an operation that raises:
+
+```
+8/9 operations, 3 files
+
+refused to generate 1 operation(s):
+  - list_widgets: query parameter expand uses style=deepObject on a schema that is not an object
+
+Calling one raises Keiyaku::Unsupported. Write those by hand.
+```
+
+A name Ruby will not take, a `$ref` into a file it cannot see, an encoding it
+does not know, a security scheme it has no way to send, an operation pointing
+at a server the client does not declare — each is refused rather than guessed
+at. The line it draws is the specification's: `deepObject` on an object is
+described and is generated, `deepObject` on an array is written n/a in
+OpenAPI's own table and is not.
+
+## Not done yet
+
+- `explode: false` query parameters are still refused, and so is every query
+  style but `form` and `deepObject` — `spaceDelimited` and `pipeDelimited` are
+  described by the specification and simply have not been written yet, which is
+  not the same as `deepObject` on an array, which never will be
+- the document has to be one file. A `$ref` into another is refused rather
+  than resolved, so anything split up has to be bundled first — by something
+  else, since nothing here fetches a URL to find out what a type is
+- `readOnly` and `writeOnly` are not read, so one schema is one model in both
+  directions. Until they are, `required:` cannot be enforced when a model is
+  constructed — the Petstore's `Pet.id` is required in a response and has no
+  business being set on a request, and one flag cannot mean both
+- `enum` values are not carried: the type they restrict is generated and the
+  list itself is not checked in either direction
+- there is no pagination. OpenAPI describes none, and a parameter named `page`
+  is not evidence of anything, so walking an operation has to be declared —
+  which for a document you did not write means a hints file, and there is none.
+  Until there is, the loop over the pages belongs to the application
+- a client is built for one server. An operation that overrides it is refused
+  rather than sent somewhere else, and server variables have no substitution,
+  so both are for the application to supply through `base_url:`
+- regeneration overwrites wholesale; there is no merge strategy for hand edits,
+  which bites hardest on the operations it refused and left you to write
+
+## More
+
+[DESIGN.md](DESIGN.md) is the long version: how the generated code and the
+runtime split, what happens to unions, uploads and open models, where type
+names come from, why a construct gets refused, and what the test suite is
+pointed at. [CHANGELOG.md](CHANGELOG.md) is what has changed.
+
+MIT licensed.

data/exe/keiyaku

@@ -0,0 +1,56 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "fileutils"
+require "optparse"
+require "keiyaku/emitter"
+
+options = { out: "." }
+
+parser = OptionParser.new do |o|
+  o.banner = "usage: keiyaku SPEC --module NAME [--out DIR]"
+  o.separator ""
+  o.on("-m", "--module NAME", "Ruby module to nest the client and its types in")
+  o.on("-o", "--out DIR", "Directory to write into (default: the working directory)")
+  o.separator ""
+  o.on("-v", "--version", "Print the version and exit") { puts Keiyaku::VERSION; exit }
+  o.on("-h", "--help", "Print this message and exit") { puts o; exit }
+end
+parser.parse!(into: options)
+
+spec = ARGV.shift
+abort parser.to_s unless spec && options[:module]
+abort "keiyaku: #{spec}: no such file" unless File.file?(spec)
+
+FileUtils.mkdir_p(options[:out])
+emitter =
+  begin
+    Keiyaku::Emitter.new(spec, namespace: options[:module])
+  rescue Keiyaku::Impossible => e
+    abort "keiyaku: #{e.message}"
+  end
+operations = emitter.emit(options[:out])
+
+built = operations.count { !_1[:unsupported] }
+puts "#{built}/#{operations.size} operations, #{Dir[File.join(options[:out], "*")].size} files"
+
+unless emitter.notes.empty?
+  puts "\nnotes:"
+  emitter.notes.uniq.each { puts "  - #{_1}" }
+end
+
+unless emitter.refusals.empty?
+  puts "\nrefused to generate #{emitter.refusals.size} operation(s):"
+  emitter.refusals.each { puts "  - #{_1.operation}: #{_1.reason}" }
+  puts "\nCalling one raises Keiyaku::Unsupported. Write those by hand."
+end
+
+# The generator writing a file Ruby cannot read back is a defect in the
+# generator, not in the document, and it exits non-zero so that a build does
+# not carry on with a client nothing can require.
+if emitter.broken
+  puts "\nthe generated client does not load:"
+  puts emitter.broken.lines.map { "  #{_1}" }
+  puts "\nThis is a bug in keiyaku. The files are still there to look at."
+  exit 1
+end

data/lib/keiyaku/adapters/faraday.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "faraday"
+require_relative "../runtime"
+
+module Keiyaku
+  # Puts a generated client on Faraday, so it inherits the connection pooling,
+  # instrumentation and retry middleware an application has already set up.
+  #
+  #   require "keiyaku/adapters/faraday"
+  #
+  #   client = Petstore::Client.new(adapter: Keiyaku::FaradayAdapter.new(conn))
+  #
+  # Faraday is not a dependency of this gem, and will not become one: a client
+  # for one API has no business choosing an HTTP stack for the application
+  # around it. Requiring this file is how you opt in.
+  #
+  # With faraday-retry in the connection, build the client with `retries: 0`.
+  # The middleware handles Retry-After and jitter properly; the loop in the
+  # runtime is only there so the stdlib path is not defenceless.
+  class FaradayAdapter
+    def initialize(connection = nil, timeout: nil, **options)
+      if timeout
+        # Faraday has its own spelling for these, and a connection that was
+        # handed in already carries whatever it was built with. Accepting the
+        # option and dropping it would be the worst of the three.
+        raise ArgumentError, "timeout: has no effect on a connection you built; set it on that" if connection
+
+        open, read = Keiyaku.timeouts(timeout)
+        options[:request] = { open_timeout: open, timeout: read }.merge(options[:request] || {})
+      end
+
+      @connection = connection || Faraday.new(**options)
+    end
+
+    def call(verb, uri, headers, body)
+      response = @connection.run_request(verb, uri, body, headers)
+      [response.status, response.headers.to_h, response.body]
+    rescue Faraday::ConnectionFailed, Faraday::TimeoutError, Faraday::SSLError => e
+      raise ConnectionError, "#{verb.to_s.upcase} #{uri}: #{e.message}"
+    end
+  end
+end

data/lib/keiyaku/adapters/http.rb

@@ -0,0 +1,32 @@
+# frozen_string_literal: true
+
+require "http"
+require_relative "../runtime"
+
+module Keiyaku
+  # Puts a generated client on http.rb, which is faster than Net::HTTP and can
+  # hold persistent connections.
+  #
+  #   require "keiyaku/adapters/http"
+  #
+  #   client = Petstore::Client.new(
+  #     adapter: Keiyaku::HTTPAdapter.new(HTTP.persistent("https://petstore3.swagger.io"))
+  #   )
+  #
+  # Like the Faraday one, this is an opt-in file rather than a dependency.
+  # Named after the gem it requires, not after what it does — otherwise there
+  # is no telling it apart from NetHTTPAdapter.
+  class HTTPAdapter
+    def initialize(client = nil, timeout: 15)
+      open, read = Keiyaku.timeouts(timeout)
+      @client = client || HTTP.timeout(connect: open, read:)
+    end
+
+    def call(verb, uri, headers, body)
+      response = @client.headers(headers).request(verb, uri, body.nil? ? {} : { body: })
+      [response.code, response.headers.to_h, response.body.to_s]
+    rescue HTTP::ConnectionError, HTTP::TimeoutError, *TRANSPORT_ERRORS => e
+      raise ConnectionError, "#{verb.to_s.upcase} #{uri}: #{e.message}"
+    end
+  end
+end

data/lib/keiyaku/adapters/net_http.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+require "net/http"
+require_relative "../runtime"
+
+module Keiyaku
+  # The stdlib adapter, which is what a client builds for itself when it was
+  # given none. Nothing has to require this file: the runtime autoloads it on
+  # that first default, so an application that named an adapter of its own
+  # never pays for net/http.
+  #
+  # A connection per request, since Net::HTTP holding one is a state an
+  # adapter with no state cannot keep. Where that cost matters, http.rb's
+  # persistent client or a Faraday connection is the answer, which is what
+  # those adapters are for.
+  class NetHTTPAdapter
+    def initialize(timeout: 15)
+      @open_timeout, @read_timeout = Keiyaku.timeouts(timeout)
+    end
+
+    def call(verb, uri, headers, body)
+      http = Net::HTTP.new(uri.host, uri.port)
+      http.use_ssl = uri.scheme == "https"
+      http.open_timeout = @open_timeout
+      http.read_timeout = @read_timeout
+
+      request = Net::HTTP.const_get(verb.to_s.capitalize).new(uri)
+      headers.each { |k, v| request[k] = v }
+
+      if body
+        request.body = body
+      elsif request.request_body_permitted?
+        # No body is no body: an optional one left out means no bytes and no
+        # Content-Type claiming there are some, which is the runtime's promise
+        # and not this file's to give away. Net::HTTP hands a POST that was
+        # given none an empty body of its own, and net/http before 0.5 labels
+        # that application/x-www-form-urlencoded — a media type this client
+        # never chose, on a request that carries nothing. Saying so here is
+        # one method rather than a header to unpick later, and the length
+        # still goes out as the zero it is, since a POST without one is what
+        # some servers answer with 411.
+        request.content_length = 0
+        def request.request_body_permitted? = false
+      end
+
+      response = http.request(request)
+      [response.code.to_i, response.to_hash.transform_values(&:first), response.body]
+    rescue *TRANSPORT_ERRORS => e
+      raise ConnectionError, "#{verb.to_s.upcase} #{uri}: #{e.message}"
+    end
+  end
+end

data/lib/keiyaku/emitter/rbs.rb

@@ -0,0 +1,201 @@
+# frozen_string_literal: true
+
+require_relative "../names"
+
+module Keiyaku
+  class Emitter
+    # The signature file, which is a second translation of the same document
+    # into a language with its own rules: `|` means one thing after `attr_reader`
+    # and another after `->`, a name that is not a method cannot be an
+    # attr_reader at all, and a component that turned out not to be a class is
+    # referred to by a type alias rather than by its constant. None of that has
+    # anything to do with what the Ruby emitter is deciding, so it is kept
+    # where it can be read as the one subject it is.
+    #
+    # It is handed the model and union tables rather than a copy of them,
+    # because a union component's expansion is worked out while the schemas are
+    # still being collected — the tables go on growing behind it.
+    class RBS
+      SCALARS = {
+        ":bool" => "bool", ":any" => "untyped", ":binary" => "String", ":text" => "String",
+        ":upload" => "Keiyaku::Upload | IO"
+      }.freeze
+
+      def initialize(namespace:, models:, unions:)
+        @namespace = namespace
+        @models = models
+        @unions = unions
+      end
+
+      def type_for(source)
+        # A recursive type is written as a Proc in the source and as itself here:
+        # RBS has no trouble with a class that mentions its own name.
+        return type_for(source[/\A-> \{(.*)\}\z/m, 1].strip) if source.start_with?("-> {")
+        return "Array[#{type_for(source[1..-2].strip)}]" if source.start_with?("[")
+        return "Hash[String, #{type_for(source[/=>(.*)}/m, 1].strip)}]" if source.start_with?("{")
+
+        # A named union is referred to by its alias; an inline one is spelled out.
+        if (expansion = @unions[source])
+          return source.start_with?("Keiyaku::OneOf[") ? expand_union(expansion) : Keiyaku.snake(source)
+        end
+
+        SCALARS[source] || source
+      end
+
+      def source(order, operations)
+        models = order.map { model_rbs(_1) }
+        methods = operations.filter_map { method_rbs(_1) }
+
+        <<~RBS
+          #{HEADER}
+
+          module #{@namespace}
+          #{models.join("\n\n")}
+
+            class Client < Keiyaku::Client
+          #{methods.join("\n")}
+            end
+          end
+        RBS
+      end
+
+      private
+
+      # A union is written bare everywhere but one place: RBS reads `|` after a
+      # return type as the start of another overload, so `-> A | B` is a syntax
+      # error where `attr_reader a: A | B` is fine. Only a top-level `|` needs
+      # the parentheses — inside `Array[...]` the brackets already close it off.
+      def return_type(type)
+        depth = 0
+        type.each_char do |char|
+          case char
+          when "[" then depth += 1
+          when "]" then depth -= 1
+          when "|" then return "(#{type})" if depth.zero?
+          end
+        end
+        type
+      end
+
+      # How the operation's `into:` reads as a return type. `untyped` in a union
+      # says nothing the union did not already say and takes the rest of it down
+      # with it, so an operation with one response the document left open is
+      # that: untyped.
+      def into_type(into)
+        types = into.values.uniq.map { type_for(_1) }
+        types.include?("untyped") ? "untyped" : types.join(" | ")
+      end
+
+      # RBS has no constant that stands for a union, so a union component becomes
+      # a type alias for signatures to use, plus the constant itself for callers
+      # that reach for `Event.cast`.
+      def union_type(const)
+        source = @models[const]
+        constant =
+          if source.start_with?("Keiyaku::OneOf[") then "Keiyaku::OneOf"
+          elsif source == ":any" then "Symbol"
+          else "untyped" # a union that collapsed to one type, so the constant is that type
+          end
+
+        "  type #{Keiyaku.snake(const)} = #{expand_union(@unions[const])}\n  #{const}: #{constant}"
+      end
+
+      # A variant of a union may be a component that turned out not to be a
+      # class — an array, a scalar — and RBS refers to one of those by its type
+      # alias. The constant is still there, but it holds a value rather than
+      # naming a type, so a signature that used it would not resolve.
+      def expand_union(expansion)
+        expansion.split(" | ").map { |name| @models[name].is_a?(String) ? Keiyaku.snake(name) : name }.join(" | ")
+      end
+
+      def model_rbs(const)
+        model = @models[const]
+        return union_type(const) if model.is_a?(String)
+
+        # `attr_reader "+1":` is not RBS, in either spelling — the name is not
+        # a method, so it is typed where it is actually read instead.
+        bare, quoted = model.fields.partition { |field, _| Names.bare?(field) }
+        declared = lambda do |field, type|
+          "#{type_for(type)}#{"?" unless model.required.include?(field)}"
+        end
+
+        readers = bare.map { |field, type| "    attr_reader #{field}: #{declared.(field, type)}" }
+
+        # Only the names that cannot be an attr_reader are worth an overload:
+        # `[]` is already declared on the base class, and a subclass's `def`
+        # replaces that rather than adding to it, so anything left out here is
+        # left out of the type entirely. Hence the trailing overload — without
+        # it, typing `rollup["+1"]` would cost `rollup[:total]`, which the same
+        # method reads perfectly well.
+        unless quoted.empty?
+          overloads = quoted.map { |field, type| "(#{field.inspect}) -> #{declared.(field, type)}" }
+          readers << "    def []: #{(overloads + ["(String | Symbol) -> untyped"]).join("\n           | ")}"
+        end
+
+        # `to_json_hash` and `deconstruct_keys` are not emitted: the first is
+        # word for word what the base class declares, and the second would be
+        # a union of every type in the model, which a pattern match binding one
+        # field is not helped by. What is emitted is what is more precise here
+        # than it can be there.
+        <<~RBS.chomp
+            class #{const} < ::Keiyaku::Model
+          #{readers.join("\n")}
+              def self.cast: (untyped, ?String) -> #{const}
+          #{constructors(const, model)}
+            end
+        RBS
+      end
+
+      # `new` and `with`, spelled out. Inherited from Keiyaku::Model they are
+      # `(**untyped)`, which typechecks a call site that leaves out a required
+      # field and one that misspells an optional one — the two mistakes a
+      # signature for a request body exists to catch.
+      #
+      # An open model still takes anything beyond its fields, so **untyped is
+      # the truth there and the declared keywords are what is gained.
+      def constructors(const, model)
+        keywords = model.fields.filter_map do |field, type|
+          next unless Names.bare?(field)
+
+          [field, "#{type_for(type)}#{"?" unless model.required.include?(field)}"]
+        end
+
+        required, optional = keywords.partition { |field, _| model.required.include?(field) }
+        new_args = required.map { |field, type| "#{field}: #{type}" } +
+                   optional.map { |field, type| "?#{field}: #{type}" }
+        with_args = keywords.map { |field, type| "?#{field}: #{type}" }
+
+        # A field Ruby will not take as a keyword label cannot be declared as
+        # one, so a model with any is as open as one the document opened.
+        rest = model.open || keywords.size < model.fields.size ? ["**untyped"] : []
+
+        ["    def self.new: (#{(new_args + rest).join(", ")}) -> #{const}",
+         "    def with: (#{(with_args + rest).join(", ")}) -> #{const}"].join("\n")
+      end
+
+      def method_rbs(op)
+        return if op[:name].nil?
+        if op[:unsupported]
+          return "    def #{op[:name]}: (*untyped) -> bot  # not generated: #{Emitter.comment(op[:unsupported])}"
+        end
+
+        types = op[:types]
+        positional = op[:template].scan(/\{(\w+)\}/).flatten.map do |param|
+          "#{type_for(types[param] || ":any")} #{Keiyaku.snake(param)}"
+        end
+        payload = op[:body] || op[:form] || op[:multipart]
+        positional << "#{"?" unless op[:body_required]}#{type_for(payload)} body" if payload
+
+        keyword = lambda do |json_name, ruby_name|
+          required = op[:required].include?(json_name)
+          type = type_for(types[json_name] || ":any")
+          "#{"?" unless required}#{Keiyaku.snake(ruby_name)}: #{type}#{"?" unless required}"
+        end
+        keywords = op[:query].map { keyword.(_1, _1) } +
+                   op[:header].map { |json, ruby| keyword.(json, ruby) }
+        arguments = (positional + keywords).join(", ")
+        "    def #{op[:name]}: (#{arguments}) -> #{op[:into] ? return_type(into_type(op[:into])) : "untyped"}"
+      end
+    end
+  end
+end