plumb 0.0.1 → 0.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 573f2aeb395f26fdc392bf7741d9874fb0318fabc4b8af6f1d1f7a8991f50100
-  data.tar.gz: 464c7aaa1b6dcbae0195b2bf51103438f3cd8a7f91593af0640a55f6f568e011
+  metadata.gz: db1a6e5f70bf36e91d053ff465e9f566cd4371e4620dac88aea6f028918311a8
+  data.tar.gz: 13e986c5a7815c3ecbdf6f1f6cabb4d9341b88421d8048a7e7182d9b638e1632
 SHA512:
-  metadata.gz: 76fa4ec0bbed8a7e2c33537c7079ce47b7f4cdc693c2d61b4218980b36e17d2f57cd2cc80229c3bd91f2024dbcd706c4fd8846a3bc7dd7a730146f2417a6e882
-  data.tar.gz: '03048bc44d4d3f0d9fca72d312cca9b679a2ff61e1026593c63fcbe727f550867862f43cecbdf3de93de756a5b577ff9e5f004c2f8c11e42985f4c23f9b0963d'
+  metadata.gz: 540cb16d4ab114931dad7b278578428357341e5318f5371a40d22b6d53714fe71290cb66892725316faf2060e5a1ca8e4c318951e9dc8eeb7839eac2a38d4800
+  data.tar.gz: 6404ab512cb061af57be9d7b3e41dfb967e3e651819f4fb540a6e018f55a5ab18a976649fe891338181f7e769b9efb0a29df1e8dd0586471b14ea0b7b04a511d

data/.rubocop.yml

@@ -1,3 +1,5 @@
+AllCops:
+  TargetRubyVersion: 3.2.2
 Style/CaseEquality:
   Enabled: false
 Style/LambdaCall:

data/README.md

@@ -1,6 +1,14 @@
 # Plumb
 
-Composable data validation and coercion in Ruby. WiP.
+**This library is work in progress!**
+
+Composable data validation, coercion and processing in Ruby. Takes over from https://github.com/ismasan/parametric
+
+This library takes ideas from the excellent https://dry-rb.org ecosystem, with some of the features offered by Dry-Types, Dry-Schema, Dry-Struct. However, I'm aiming at a subset of the functionality with a (hopefully) smaller API surface and fewer concepts, focusing on lessons learned after using Parametric in production for many years.
+
+If you're after raw performance and versatility I strongly recommend you use the Dry gems.
+
+For a description of the core architecture you can read [this article](https://ismaelcelis.com/posts/composable-pipelines-in-ruby/).
 
 ## Installation
 
@@ -18,7 +26,7 @@ module Types
   include Plumb::Types
   
   # Define your own types
-  Email = String[/&/]
+  Email = String[/@/]
 end
 
 # Use them
@@ -32,7 +40,6 @@ result.errors # ""
 ```
 
 
-
 ### `#resolve(value) => Result`
 
 `#resolve` takes an input value and returns a `Result::Valid` or `Result::Invalid`
@@ -162,7 +169,42 @@ StringToInt = Types::String.transform(Integer, &:to_i)
 StringToInteger.parse('10') # => 10
 ```
 
+### `#invoke`
 
+`#invoke` builds a Step that will invoke one or more methods on the value.
+
+```ruby
+StringToInt = Types::String.invoke(:to_i)
+StringToInt.parse('100') # 100
+
+FilteredHash = Types::Hash.invoke(:except, :foo, :bar)
+FilteredHash.parse(foo: 1, bar: 2, name: 'Joe') # { name: 'Joe' }
+
+# It works with blocks
+Evens = Types::Array[Integer].invoke(:filter, &:even?)
+Evens.parse([1,2,3,4,5]) # [2, 4]
+
+# Same as
+Evens = Types::Array[Integer].transform(Array) {|arr| arr.filter(&:even?) }
+```
+
+Passing an array of Symbol method names will build a chain of invocations.
+
+```ruby
+UpcaseToSym = Types::String.invoke(%i[downcase to_sym])
+UpcaseToSym.parse('FOO_BAR') # :foo_bar
+```
+
+That that, as opposed to `#transform`, this modified does not register a type in `#metadata[:type]`, which can be valuable for introspection or documentation (ex. JSON Schema).
+
+Also, there's no definition-time checks that the method names are actually supported by the input values.
+
+```ruby
+type = Types::Array.invoke(:strip) # This is fine here
+type.parse([1, 2]) # raises NoMethodError because Array doesn't respond to #strip
+```
+
+Use with caution.
 
 ### `#default`
 
@@ -178,7 +220,7 @@ Note that this is syntax sugar for:
 
 ```ruby
 # A String, or if it's Undefined pipe to a static string value.
-str = Types::String | (Types::Undefined >> 'nope'.freeze)
+str = Types::String | (Types::Undefined >> Types::Static['nope'.freeze])
 ```
 
 Meaning that you can compose your own semantics for a "default" value.
@@ -186,7 +228,7 @@ Meaning that you can compose your own semantics for a "default" value.
 Example when you want to apply a default when the given value is `nil`.
 
 ```ruby
-str = Types::String | (Types::Nil >> 'nope'.freeze)
+str = Types::String | (Types::Nil >> Types::Static['nope'.freeze])
 
 str.parse(nil) # 'nope'
 str.parse('yup') # 'yup'
@@ -195,7 +237,7 @@ str.parse('yup') # 'yup'
 Same if you want to apply a default to several cases.
 
 ```ruby
-str = Types::String | ((Types::Nil | Types::Undefined) >> 'nope'.freeze)
+str = Types::String | ((Types::Nil | Types::Undefined) >> Types::Static['nope'.freeze])
 ```
 
 
@@ -251,25 +293,23 @@ UserType.parse('Joe') # #<data User name="Joe">
 It takes an argument for a custom factory method on the object constructor.
 
 ```ruby
-class User
-  def self.create(attrs)
-    new(attrs)
-  end
-end
+# https://github.com/RubyMoney/monetize
+require 'monetize'
 
-UserType = Types::String.build(User, :create)
+StringToMoney = Types::String.build(Monetize, :parse)
+money = StringToMoney.parse('£10,300.00') # #<Money fractional:1030000 currency:GBP>
 ```
 
 You can also pass a block
 
 ```ruby
-UserType = Types::String.build(User) { |name| User.new(name) }
+StringToMoney = Types::String.build(Money) { |value| Monetize.parse(value) }
 ```
 
 Note that this case is identical to `#transform` with a block.
 
 ```ruby
-UserType = Types::String.transform(User) { |name| User.new(name) }
+StringToMoney = Types::String.transform(Money) { |value| Monetize.parse(value) }
 ```
 
 
@@ -350,7 +390,7 @@ Company = Types::Hash[
 result = Company.resolve(
   name: 'ACME',
   employees: [
-  	{ name: 'Joe', age: 40, role: 'product' },
+    { name: 'Joe', age: 40, role: 'product' },
     { name: 'Joan', age: 38, role: 'engineer' }
   ]
 )
@@ -366,6 +406,66 @@ result.valid? # false
 result.errors[:employees][0][:age] # ["must be a Numeric"]
 ```
 
+Note that you can use primitives as hash field definitions.
+
+```ruby
+User = Types::Hash[name: String, age: Integer]
+```
+
+Or to validate specific values:
+
+```ruby
+Joe = Types::Hash[name: 'Joe', age: Integer]
+```
+
+Or to validate against any `#===` interface:
+
+```ruby
+Adult = Types::Hash[name: String, age: (18..)]
+# Same as
+Adult = Types::Hash[name: Types::String, age: Types::Integer[18..]]
+```
+
+If you want to validate literal values, pass a `Types::Value`
+
+```ruby
+Settings = Types::Hash[age_range: Types::Value[18..]]
+
+Settings.parse(age_range: (18..)) # Valid
+Settings.parse(age_range: (20..30)) # Invalid
+```
+
+A `Types::Static` value will always resolve successfully to that value, regardless of the original payload.
+
+```ruby
+User = Types::Hash[name: Types::Static['Joe'], age: Integer]
+User.parse(name: 'Rufus', age: 34) # Valid {name: 'Joe', age: 34}
+```
+
+#### Optional keys
+
+Keys suffixed with `?` are marked as optional and its values will only be validated and coerced if the key is present in the input hash.
+
+```ruby
+User = Types::Hash[
+  age?: Integer,
+  name: String
+]
+
+User.parse(age: 20, name: 'Joe') # => Valid { age: 20, name: 'Joe' }
+User.parse(age: '20', name: 'Joe') # => Invalid, :age is not an Integer
+User.parse(name: 'Joe') #=> Valid { name: 'Joe' }
+```
+
+Note that defaults are not applied to optional keys that are missing.
+
+```ruby
+Types::Hash[
+  age?: Types::Integer.default(10), # does not apply default if key is missing  
+  name: Types::String.default('Joe') # does apply default if key is missing.
+]
+```
+
 
 
 #### Merging hash definitions
@@ -394,9 +494,11 @@ intersection = User & Employee # Hash[:name]
 
 Use `#tagged_by` to resolve what definition to use based on the value of a common key.
 
+Key used as index must be a `Types::Static`
+
 ```ruby
-NameUpdatedEvent = Types::Hash[type: 'name_updated', name: Types::String]
-AgeUpdatedEvent = Types::Hash[type: 'age_updated', age: Types::Integer]
+NameUpdatedEvent = Types::Hash[type: Types::Static['name_updated'], name: Types::String]
+AgeUpdatedEvent = Types::Hash[type: Types::Static['age_updated'], age: Types::Integer]
 
 Events = Types::Hash.tagged_by(
   :type,
@@ -409,6 +511,48 @@ Events.parse(type: 'name_updated', name: 'Joe') # Uses NameUpdatedEvent definiti
 
 
 
+#### `Types::Hash#inclusive`
+
+Use `#inclusive` to preserve input keys not defined in the hash schema.
+
+```ruby
+hash = Types::Hash[age: Types::Lax::Integer].inclusive
+
+# Only :age, is coerced and validated, all other keys are preserved as-is
+hash.parse(age: '30', name: 'Joe', last_name: 'Bloggs') # { age: 30, name: 'Joe', last_name: 'Bloggs' }
+```
+
+This can be useful if you only care about validating some fields, or to assemble different front and back hashes. For example a client-facing one that validates JSON or form data, and a backend one that runs further coercions or domain validations on some keys.
+
+```ruby
+# Front-end definition does structural validation
+Front = Types::Hash[price: Integer, name: String, category: String]
+
+# Turn an Integer into a Money instance
+IntToMoney = Types::Integer.build(Money)
+
+# Backend definition turns :price into a Money object, leaves other keys as-is
+Back = Types::Hash[price: IntToMoney].inclusive
+
+# Compose the pipeline
+InputHandler = Front >> Back
+
+InputHandler.parse(price: 100_000, name: 'iPhone 15', category: 'smartphones')
+# => { price: #<Money fractional:100000 currency:GBP>, name: 'iPhone 15', category: 'smartphone' }
+```
+
+#### `Types::Hash#filtered`
+
+The `#filtered` modifier returns a valid Hash with the subset of values that were valid, instead of failing the entire result if one or more values are invalid.
+
+```ruby
+User = Types::Hash[name: String, age: Integer]
+User.parse(name: 'Joe', age: 40) # => { name: 'Joe', age: 40 }
+User.parse(name: 'Joe', age: 'nope') # => { name: 'Joe' }
+```
+
+
+
 ### Hash maps
 
 You can also use Hash syntax to define a hash map with specific types for all keys and values:
@@ -420,6 +564,37 @@ currencies.parse(usd: 'USD', gbp: 'GBP') # Ok
 currencies.parse('usd' => 'USD') # Error. Keys must be Symbols
 ```
 
+Like other types, hash maps accept primitive types as keys and values:
+
+```ruby
+currencies = Types::Hash[Symbol, String]
+```
+
+And any `#===` interface as values, too:
+
+```ruby
+names_and_emails = Types::Hash[String, /\w+@\w+/]
+
+names_and_emails.parse('Joe' => 'joe@server.com', 'Rufus' => 'rufus')
+```
+
+Use `Types::Value` to validate specific values (using `#==`)
+
+```ruby
+names_and_ones = Types::Hash[String, Types::Integer.value(1)]
+```
+
+#### `#filtered`
+
+Calling the `#filtered` modifier on a Hash Map makes it return a sub set of the keys and values that are valid as per the key and value type definitions.
+
+```ruby
+# Filter the ENV for all keys starting with S3_*
+S3Config = Types::Hash[/^S3_\w+/, Types::Any].filtered
+
+S3Config.parse(ENV.to_h) # { 'S3_BUCKET' => 'foo', 'S3_REGION' => 'us-east-1' }
+```
+
 
 
 ### `Types::Array`
@@ -429,19 +604,54 @@ names = Types::Array[Types::String.present]
 names_or_ages = Types::Array[Types::String.present | Types::Integer[21..]]
 ```
 
+Arrays support primitive classes, or any `#===` interface:
+
+```ruby
+strings = Types::Array[String]
+emails = Types::Array[/@/]
+# Similar to 
+emails = Types::Array[Types::String[/@/]]
+```
+
+Prefer the latter (`Types::Array[Types::String[/@/]]`), as that first validates that each element is a `String` before matching agains the regular expression.
+
 #### Concurrent arrays
 
 Use `Types::Array#concurrent` to process array elements concurrently (using Concurrent Ruby for now).
 
 ```ruby
-ImageDownload = Types::URL >> ->(result) { HTTP.get(result.value) }
+ImageDownload = Types::URL >> ->(result) { 
+  resp = HTTP.get(result.value)
+  if (200...300).include?(resp.status)
+    result.valid(resp.body)
+  else
+    result.invalid(error: resp.status)
+  end
+}
 Images = Types::Array[ImageDownload].concurrent
 
 # Images are downloaded concurrently and returned in order.
 Images.parse(['https://images.com/1.png', 'https://images.com/2.png'])
 ```
 
-TODO: pluggable concurrently engines (Async?)
+TODO: pluggable concurrency engines (Async?)
+
+#### `#stream`
+
+Turn an Array definition into an enumerator that yields each element wrapped in `Result::Valid` or `Result::Invalid`.
+
+See `Types::Stream` below for more.
+
+#### `#filtered`
+
+The `#filtered` modifier makes an array definition return a subset of the input array where the values are valid, as per the array's element type.
+
+```ruby
+j_names = Types::Array[Types::String[/^j/]].filtered
+j_names.parse(%w[james ismael joe toby joan isabel]) # ["james", "joe", "joan"]
+```
+
+
 
 ### `Types::Tuple`
 
@@ -461,7 +671,69 @@ Error = Types::Tuple[:error, Types::String.present]
 Status = Ok | Error
 ```
 
+... Or any `#===` interface
+
+```ruby
+NameAndEmail = Types::Tuple[String, /@/]
+```
+
+As before, use `Types::Value` to check against literal values using `#==`
 
+```ruby
+NameAndRegex = Types::Tuple[String, Types::Value[/@/]]
+```
+
+
+
+### `Types::Stream`
+
+`Types::Stream` defines an enumerator that validates/coerces each element as it iterates.
+
+This example streams a CSV file and validates rows as they are consumed.
+
+```ruby
+require 'csv'
+
+Row = Types::Tuple[Types::String.present, Types:Lax::Integer]
+Stream = Types::Stream[Row]
+
+data = CSV.new(File.new('./big-file.csv')).each # An Enumerator
+# stream is an Enumerator that yields rows wrapped in[Result::Valid] or [Result::Invalid]
+stream = Stream.parse(data)
+stream.each.with_index(1) do |result, line|
+  if result.valid?
+    p result.value
+  else
+    p ["row at line #{line} is invalid: ", result.errors]
+  end
+end
+```
+
+#### `Types::Stream#filtered`
+
+Use `#filtered` to turn a `Types::Stream` into a stream that only yields valid elements.
+
+```ruby
+ValidElements = Types::Stream[Row].filtered
+ValidElements.parse(data).each do |valid_row|
+  p valid_row
+end
+```
+
+#### `Types::Array#stream`
+
+A `Types::Array` definition can be turned into a stream.
+
+```ruby
+Arr = Types::Array[Integer]
+Str = Arr.stream
+
+Str.parse(data).each do |row|
+  row.valid?
+  row.errors
+  row.value
+end
+```
 
 ### Plumb::Schema
 

data/examples/command_objects.rb

@@ -0,0 +1,207 @@
+# frozen_string_literal: true
+
+require 'bundler'
+Bundler.setup(:examples)
+require 'plumb'
+require 'json'
+require 'fileutils'
+require 'money'
+
+Money.default_currency = Money::Currency.new('GBP')
+Money.locale_backend = nil
+Money.rounding_mode = BigDecimal::ROUND_HALF_EVEN
+
+# Different approaches to the Command Object pattern using composable Plumb types.
+module Types
+  include Plumb::Types
+
+  # Note that within this `Types` module, when we say String, Integer etc, we mean Types::String, Types::Integer etc.
+  # Use ::String to refer to Ruby's String class.
+  #
+  ###############################################################
+  # Define core types in the domain
+  # The task is to process, validate and store mortgage applications.
+  ###############################################################
+
+  # Turn integers into Money objects (requires the money gem)
+  Amount = Integer.build(Money)
+
+  # A naive email check
+  Email = String[/\w+@\w+\.\w+/]
+
+  # A valid customer type
+  Customer = Hash[
+    name: String.present,
+    age?: Integer[18..],
+    email: Email
+  ]
+
+  # A step to validate a Mortgage application payload
+  # including valid customer, mortgage type and minimum property value.
+  MortgagePayload = Hash[
+    customer: Customer,
+    type: String.options(%w[first-time switcher remortgage]).default('first-time'),
+    property_value: Integer[100_000..] >> Amount,
+    mortgage_amount: Integer[50_000..] >> Amount,
+    term: Integer[5..30],
+  ]
+
+  # A domain validation step: the mortgage amount must be less than the property value.
+  # This is just a Proc that implements the `#call(Result::Valid) => Result::Valid | Result::Invalid` interface.
+  # # Note that this can be anything that supports that interface, like a lambda, a method, a class etc.
+  ValidateMortgageAmount = proc do |result|
+    if result.value[:mortgage_amount] > result.value[:property_value]
+      result.invalid(errors: { mortgage_amount: 'Cannot exceed property value' })
+    else
+      result
+    end
+  end
+
+  # A step to create a mortgage application
+  # This could be backed by a database (ex. ActiveRecord), a service (ex. HTTP API), etc.
+  # For this example I just save JSON files to disk.
+  class MortgageApplicationsStore
+    def self.call(result) = new.call(result)
+
+    def initialize(dir = './examples/data/applications')
+      @dir = dir
+      FileUtils.mkdir_p(dir)
+    end
+
+    # The Plumb::Step interface to make these objects composable.
+    # @param result [Plumb::Result::Valid]
+    # @return [Plumb::Result::Valid, Plumb::Result::Invalid]
+    def call(result)
+      if save(result.value)
+        result
+      else
+        result.invalid(errors: 'Could not save application')
+      end
+    end
+
+    def save(payload)
+      file_name = File.join(@dir, "#{Time.now.to_i}.json")
+      File.write(file_name, JSON.pretty_generate(payload))
+    end
+  end
+
+  # Finally, a step to send a notificiation to the customer.
+  # This should only run if the previous steps were successful.
+  NotifyCustomer = proc do |result|
+    # Send an email here.
+    puts "Sending notification to #{result.value[:customer][:email]}"
+    result
+  end
+
+  ###############################################################
+  # Option 1: define standalone steps and then pipe them together
+  ###############################################################
+  CreateMortgageApplication1 = MortgagePayload \
+    >> ValidateMortgageAmount \
+    >> MortgageApplicationsStore \
+    >> NotifyCustomer
+
+  ###############################################################
+  # Option 2: compose steps into a Plumb::Pipeline
+  # This is just a wrapper around step1 >> step2 >> step3 ...
+  # But the procedural style can make sequential steps easier to read and manage.
+  # Also to add/remove debugging and tracing steps.
+  ###############################################################
+  CreateMortgageApplication2 = Any.pipeline do |pl|
+    # The input payload
+    pl.step MortgagePayload
+
+    # Some inline logging to demostrate inline steps
+    # This is also useful for debugging and tracing.
+    pl.step do |result|
+      p [:after_payload, result.value]
+      result
+    end
+
+    # Domain validation
+    pl.step ValidateMortgageAmount
+
+    # Save the application
+    pl.step MortgageApplicationsStore
+
+    # Notifications
+    pl.step NotifyCustomer
+  end
+
+  # Note that I could have also started the pipeline directly off the MortgagePayload type.
+  # ex. CreateMortageApplication2 = MortgagePayload.pipeline do |pl
+  # For super-tiny command objects you can do it all inline:
+  #
+  #   Types::Hash[
+  #     name: String,
+  #     age: Integer
+  #   ].pipeline do |pl|
+  #     pl.step do |result|
+  #       .. some validations
+  #       result
+  #     end
+  #   end
+  #
+  # Or you can use Method objects as steps
+  #
+  #   pl.step SomeObject.method(:create)
+
+  ###############################################################
+  # Option 3: use your own class
+  # Use Plumb internally for validation and composition of shared steps or method objects.
+  ###############################################################
+  class CreateMortgageApplication3
+    def initialize
+      @pipeline = Types::Any.pipeline do |pl|
+        pl.step MortgagePayload
+        pl.step method(:validate)
+        pl.step method(:save)
+        pl.step method(:notify)
+      end
+    end
+
+    def run(payload)
+      @pipeline.resolve(payload)
+    end
+
+    private
+
+    def validate(result)
+      # etc
+      result
+    end
+
+    def save(result)
+      # etc
+      result
+    end
+
+    def notify(result)
+      # etc
+      result
+    end
+  end
+end
+
+# Uncomment each case to run
+# p Types::CreateMortgageApplication1.resolve(
+#   customer: { name: 'John Doe', age: 30, email: 'john@doe.com' },
+#   property_value: 200_000,
+#   mortgage_amount: 150_000,
+#   term: 25
+# )
+
+# p Types::CreateMortgageApplication2.resolve(
+#   customer: { name: 'John Doe', age: 30, email: 'john@doe.com' },
+#   property_value: 200_000,
+#   mortgage_amount: 150_000,
+#   term: 25
+# )
+
+# Or, with invalid data
+# p Types::CreateMortgageApplication2.resolve(
+#   customer: { name: 'John Doe', age: 30, email: 'john@doe.com' },
+#   property_value: 200_000,
+#   mortgage_amount: 201_000,
+#   term: 25
+# )