form_input 1.2.0 → 1.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 4014e07ca32e3c7a76ee15db0dfe478a4103d123
-  data.tar.gz: 83224f22dc199d2a3f1bf4d46011f616195d33ca
+SHA256:
+  metadata.gz: f5f2db9b4c0442f601294eda78d17eae6fa07658d746123b34904f3b436305ec
+  data.tar.gz: 3895b0ddd7f21befbb375d337a485af8c533641faa1171c1f643b14efa306fdf
 SHA512:
-  metadata.gz: 82f8812fbd77215e7a058f1585bc7b2ee2a5082215d4247ae865b8ced6e2c739c39e836741482ebf4cb089fe57b6c330f8b19726d8998ed8cfca48910faf92ba
-  data.tar.gz: e2e6853f4eb58193d11aa22696164844130c5c653df6b872372a40bfa1d790a8b61cf45bd287046cf7c375c14326709c49d907ba51e5779f0d3a72ad9ff12152
+  metadata.gz: 6864fff3b9a98fab84caba8c5768fd7a4f56c43792e0b540b9d3a8187c5eea9290cce2d0807512c5e9839515b3b48999f1f3e78d560af81765950215c483c981
+  data.tar.gz: 1dd327e0efa4d7c940023a4eae1be1cfd357def6dfd4f432ba9f960e1f42513f5aaeb9ad13afebb6cad2a27625e15dd1abca3c597a9b205fb2bdfb28b250c9ca

data/CHANGELOG.md

@@ -1,12 +1,29 @@
+= 1.4.0
+
+ * Dependencies were updated to support latest gems, in particular both Rack 2.x and 3.x are now supported.
+
+ * The inputs containing characters from Unicode Cf (Format) and Co (Private Use) categories are rejected as was intended.
+
+ * More robust handling of inputs with invalid encoding:
+   * Strings returned by `form_value` are now scrubbed so they don't cause errors when used in templates.
+   * The hash keys with invalid encoding are now handled properly, too.
+
+= 1.3.0
+
+ * Further improvements for JSON payloads:
+   * Added set of methods for distinguishing between set and unset parameters.
+   * The `import` now properly handles parameters with `nil` and `false` values.
+   * The `to_data` method now returns all set parameters, regardless of their value.
+
 = 1.2.0
 
  * Few changes for easier import and export of JSON payloads:
-   * The import and the new from_data methods now support numeric, boolean, and nil values natively.
-   * Added to_data method which creates a hash of all non-nil parameters.
+   * The `import` and the new `from_data` methods now support numeric, boolean, and `nil` values natively.
+   * Added `to_data` method which creates a hash of all non-`nil` parameters.
 
 = 1.1.0
 
- * Added report! methods for late reporting of the most fundamental errors.
+ * Added `report!` methods for late reporting of the most fundamental errors.
 
  * Added support for multiple `check` and `test` validation callbacks.
 

data/README.md

@@ -1,6 +1,6 @@
 # Form input
 
-[![Gem Version](https://img.shields.io/gem/v/form_input.svg)](http://rubygems.org/gems/form_input) [![Build Status](https://travis-ci.org/raxoft/form_input.svg?branch=master)](http://travis-ci.org/raxoft/form_input) [![Dependency Status](https://img.shields.io/gemnasium/raxoft/form_input.svg)](https://gemnasium.com/raxoft/form_input) [![Code Climate](https://img.shields.io/codeclimate/github/raxoft/form_input.svg)](https://codeclimate.com/github/raxoft/form_input) [![Coverage](https://img.shields.io/codeclimate/coverage/github/raxoft/form_input.svg)](https://codeclimate.com/github/raxoft/form_input)
+[![Gem Version](https://img.shields.io/gem/v/form_input.svg)](http://rubygems.org/gems/form_input) [![Build Status](https://travis-ci.org/raxoft/form_input.svg?branch=master)](http://travis-ci.org/raxoft/form_input) [![Maintainability](https://api.codeclimate.com/v1/badges/ea19d227eb1285d59860/maintainability)](https://codeclimate.com/github/raxoft/form_input/maintainability) [![Coverage](https://api.codeclimate.com/v1/badges/ea19d227eb1285d59860/test_coverage)](https://codeclimate.com/github/raxoft/form_input/test_coverage)
 
 Form input is a gem which helps dealing with web request input and with the creation of HTML forms.
 
@@ -50,7 +50,7 @@ Using them in your templates is as simple as this:
   .panel-heading
     = @title = "Contact Form"
   .panel-body
-    form *form_attrs
+    form method='post' action=request.path
       fieldset
         == snippet :form_panel, params: @form.params
         button.btn.btn-default type='submit' Send
@@ -794,16 +794,19 @@ to include parts of the URL as the form input:
 
 ``` ruby
   get '/contact/:email' do
-    form = ContactForm.new( params )             # NEVER EVER DO THIS!
-    form = ContactForm.new.import( params )      # Do this instead if you need to.
+    form = ContactForm.new( params )            # NEVER EVER DO THIS!
+    form = ContactForm.new.import( params )     # Do this instead (or use from_params).
+    ...
   end
 ```
 
 Similarly, you want to use `import` when feeding form input with JSON data (see [JSON Helpers](#json-helpers) for details):
 
 ``` ruby
-  post '/api/v1/contact' do
-    form = ContactForm.new.import( json_data )
+  post '/api/v1/contacts' do
+    form = ContactForm.new( json_data )         # NEVER EVER DO THIS!
+    form = ContactForm.new.import( json_data )  # Do this instead (or use from_data).
+    ...
   end
 ```
 
@@ -829,6 +832,14 @@ You can either clear the entire form, named parameters, or parameter subsets (wh
   form.clear( form.disabled_params )
 ```
 
+The `unset` method works the same way, except that it always requires an argument:
+
+``` ruby
+  form.unset( :message )
+  form.unset( :name, :company )
+  form.unset( form.invalid_params )
+```
+
 Alternatively, you can create form copies with just a subset of parameters set:
 
 ``` ruby
@@ -1026,7 +1037,7 @@ The `validate!` method on the other hand always invokes the validation,
 wiping any previously reported errors first.
 
 In either case any errors collected will remain stored
-until you change any of the parameter values with `set`, `clear`, or `[]=` methods,
+until you change any of the parameter values with `set`, `unset`, `clear`, or `[]=` methods,
 or you explicitly call `validate!`.
 Copies created with `dup` (but not `clone`), `only`, and `except` methods
 also have any errors reported before cleared.
@@ -1165,6 +1176,9 @@ In fact, the parameter has a dozen of simple boolean getters like this which you
   p.empty?        # Is the value nil or empty?
   p.filled?       # Is the value neither nil nor empty?
 
+  p.set?          # Was the parameter value set to something?
+  p.unset?        # Was the parameter value not set to anything?
+
   p.required?     # Is the parameter required?
   p.optional?     # Is the parameter not required?
 
@@ -1193,6 +1207,8 @@ The following methods are available:
   form.blank_params       # Parameters with nil, empty, or blank value.
   form.empty_params       # Parameters with nil or empty value.
   form.filled_params      # Parameters with some non-empty value.
+  form.set_params         # Parameters whose value was set to something.
+  form.unset_params       # Parameters whose value was not set to anything.
   form.required_params    # Parameters which are required and have to be filled.
   form.optional_params    # Parameters which are not required and can be nil or empty.
   form.disabled_params    # Parameters which are disabled and shall be rendered as such.
@@ -1349,8 +1365,9 @@ The only difference is that the [input filter](#input-filter) is not run in such
 The [input transform](#input-transform) is run as usual,
 and so are all the [validation methods](#errors-and-validation).
 
-Among other things this means that you can declare a parameter as integer using the `INTEGER_ARGS` macro,
+Among other things this means that you can declare a parameter as integer using the `INTEGER_ARGS` macro
 and pass in the value as either string or integer, and you end up with integer in both cases, which is pretty convenient.
+Of course, this works similarly for `FLOAT_ARGS` and `BOOL_ARGS`, too:
 
 ``` ruby
   class NumericInput < FormInput
@@ -1363,10 +1380,11 @@ and pass in the value as either string or integer, and you end up with integer i
 ```
 
 Another difference when dealing with JSON data is that
-the empty strings have completely different significance than in case of web forms.
-For this reason, the result of the `to_data` method includes even empty strings, arrays, and hashes
+the empty strings have completely different significance than in the case of the web forms.
+For this reason, the result of the `to_data` method includes any set parameters,
+even if the values are empty strings, arrays, or hashes
 (unlike the `to_hash` and `to_params` methods).
-The `nil` values are still filtered out, though.
+Even the `nil` values are included for parameters which were explicitly set to `nil`:
 
 ``` ruby
   class OptionalInput < FormInput
@@ -1376,27 +1394,67 @@ The `nil` values are still filtered out, though.
   end
 
   OptionalInput.from_data( string: '' ).to_data     # { string: '' }
+  OptionalInput.from_data( string: nil ).to_data    # { string: nil }
   OptionalInput.from_data( array: [] ).to_data      # { array: [] }
   OptionalInput.from_data( hash: {} ).to_data       # { hash: {} }
 ```
 
+With regards to `nil` values, you may want to differentiate
+between which parameters may be set to `nil` and which not
+(in addition to parameters being required or not).
+In such case, you can extend your JSON processing class wrapping `FormInput` like this:
+
+``` ruby
+  # Like param, but the parameter may be nil (and other parameters now can't).
+  def self.param?( name, *args, &block )
+    param( name, *args, allow_nil: true, &block )
+  end
+
+  # Like normal validate, but makes sure the parameters are not nil unless allowed.
+  def validate
+    super
+    set_params.each do |p|
+      unless p[ :allow_nil ]
+        p.report( "%p is nil" ) if p.value.nil?
+      end
+    end
+    self
+  end
+```
+
+This allows you to explicitely distinguish the three most commonly used cases with ease,
+especially when combined with further parameter types and checks:
+
+``` ruby
+  param :a, INTEGER_ARGS    # When set, this must be an integer.
+  param? :b, INTEGER_ARGS   # When set, this must be nil or an integer.
+  param! :c, INTEGER_ARGS   # This must be set to an integer.
+
+  param :s                  # When set, this must be a string, albeit perhaps empty.
+  param? :t                 # When set, this must be nil or a string, including empty.
+  param! :n                 # This must be set to a non-empty string.
+```
+
 The whole JSON processing may in the end look something like this:
 
 ``` ruby
   require 'oj'
+
   def json_data
-    data = Oj.load( request.body, symbolize_keys: true )
+    data = Oj.load( request.body, mode: :strict, symbolize_keys: true )
     halt( 422, 'Invalid data' ) unless Hash === data
     data
   rescue Oj::Error
     halt( 422, 'Invalid JSON' )
   end
 
-  post '/api/v1/contact' do
+  post '/api/v1/contacts' do
     input = ContactForm.from_data( json_data )
     halt( 422, "Invalid data: #{input.error_messages.first}" ) unless input.valid?
     # Somehow use the input.
-    Contact.create( input.to_data )
+    entry = Contact.create( input.to_data )
+    content_type :json
+    [ 201, Oj.dump( entry.to_hash ) ]
   end
 ```
 
@@ -1445,6 +1503,19 @@ and keys are passed to `form_name` to create the actual name:
     input type=p.type name=p.form_name( key ) value=value
 ```
 
+Note that for your convenience,
+any strings returned by `form_value` are scrubbed of characters in invalid encoding,
+should there be any,
+which makes them guaranteed to coalesce with the form template without triggering encoding errors.
+Under normal circumstances you would never encounter such invalid inputs,
+but sometimes hackers use them to try to break the applications on purpose.
+The scrubbing thus takes care that you don't have to deal with this in templates specially if it ever happens.
+However,
+if you would for some reason ever need the `form_value` without the scrubbing applied,
+note that you can use the `url_value` method
+(used internally by `url_params` and `url_query`, see [URL Helpers](#url-helpers))
+instead.
+
 For parameters which require additional data,
 like select, multi-select, or multi-checkbox parameters,
 you can ask for the data using the `data` method.
@@ -1699,11 +1770,8 @@ For [Sinatra], the helper may look like this:
 
 ``` ruby
   # Get hash with default form attributes, optionally overriding them as needed.
-  def form_attrs( *args )
-    opts = args.last.is_a?( Hash ) ? args.pop : {}
-    url = args.shift.to_s unless args.empty?
-    fail( ArgumentError, "Invalid arguments #{args.inspect}" ) unless args.empty?
-    { action: url || request.path, method: :post }.merge( opts )
+  def form_attrs( url = request.path, **opts )
+    { action: url.to_s, method: :post }.merge( opts )
   end
 ```
 

data/example/example.rb

@@ -0,0 +1,97 @@
+#! /usr/bin/env ruby
+#
+# Simple but functional example FormInput application for Sinatra.
+#
+# You will need the following gems:
+#   gem install sinatra sinatra-partial slim form_input
+#
+# Then just run it like this:
+#  ruby example.rb
+
+require 'sinatra'
+require 'sinatra/partial'
+require 'slim/smart'
+require 'form_input'
+
+# The example form. Feel free to experiment with changing this section to whatever you want to test.
+
+class ExampleForm < FormInput
+  param! :email, 'Email address', EMAIL_ARGS
+  param! :name, 'Name'
+  param :company, 'Company'
+  param! :message, 'Message', 1000, type: :textarea, size: 6, filter: ->{ rstrip }
+end
+
+# Support for snippets.
+
+set :partial_template_engine, :slim
+helpers do
+  def snippet( name, opts = {}, **locals )
+    partial( "snippets/#{name}", opts.merge( locals: locals ) )
+  end
+end
+
+# The whole application itself.
+
+get '/' do
+  @form = ExampleForm.new( request )
+  slim :form
+end
+
+post '/' do
+  @form = ExampleForm.new( request )
+  return slim :form unless @form.valid? and params[:action] == 'post'
+  logger.info "These data were successfully posted: #{@form.to_hash.inspect}"
+  slim :post
+end
+
+# Inline templates follow.
+
+__END__
+
+@@ layout
+doctype html
+html
+  head
+    title = @title
+    meta charset='utf-8'
+    meta name='viewport' content='width=device-width, initial-scale=1, shrink-to-fit=no'
+    link rel='stylesheet' href='https://stackpath.bootstrapcdn.com/bootstrap/3.4.1/css/bootstrap.min.css' integrity='sha384-HSMxcRTRxnN+Bdg0JdbxYKrThecOKuH5zCYotlSAcp1+c8xmyTe9GYg1l9a69psu' crossorigin='anonymous'
+    css:
+      label { width: 100%; }
+  body.container == yield
+
+@@ form
+.panel.panel-default
+  .panel-heading
+    = @title = "Example Form"
+  .panel-body
+    form method='post' action=request.path
+      fieldset
+        == snippet :form_panel, params: @form.params
+        button.btn.btn-default type='submit' name='action' value='post' Submit
+  - unless @form.empty?
+    .panel-footer
+      == partial :dump
+
+@@ post
+.panel.panel-default
+  .panel-heading
+    = @title = "Congratulations!"
+    All inputs are valid.
+  .panel-body
+    == partial :dump
+    form method='post' action=request.path
+      == snippet :form_hidden, params: @form.params
+      button.btn.btn-default type='submit' Go back
+      a.btn.btn-default< href=request.path Start over
+
+@@ dump
+p These are the filled form values:
+dl.dl-horizontal
+  - for p in @form.filled_params
+    dt = p.form_title
+    dd
+      code = p.value.inspect
+
+// EOF //

data/form_input.gemspec

@@ -23,11 +23,13 @@ EOT
 
   s.files       = %w[ LICENSE README.md CHANGELOG.md Rakefile .yardopts form_input.gemspec ] + Dir[ '{lib,test,example}/**/*.{rb,yml,txt,slim}' ]
 
-  s.required_ruby_version = '>= 2.0.0'
-  s.add_runtime_dependency 'rack', '>= 1.5', '< 3.0'
+  s.required_ruby_version = '>= 2.1.0'
+  s.add_runtime_dependency 'rack', '>= 1.5', '< 4.0'
+  s.add_development_dependency 'rake', '~> 13.0'
   s.add_development_dependency 'bacon', '~> 1.2'
-  s.add_development_dependency 'rack-test', '~> 0.6'
-  s.add_development_dependency 'r18n-core', '~> 2.0'
+  s.add_development_dependency 'simplecov', '~> 0.21'
+  s.add_development_dependency 'rack-test', '>= 1.0', '< 3.0'
+  s.add_development_dependency 'r18n-core', '~> 5.0'
 end
 
 # EOF #

data/lib/form_input/core.rb

@@ -8,9 +8,20 @@ class FormInput
   # Default size limit applied to all input.
   DEFAULT_SIZE_LIMIT = 255
 
-  # Default input filter applied to all input.
+  # Default input filter applied to all input (unless replaced by user's filter).
   DEFAULT_FILTER = ->{ gsub( /\s+/, ' ' ).strip }
 
+  # Default match applied to all input (in addition to user's match(es)).
+  # Note that the Graph property, despite being defined as "Non-blank characters",
+  # currently includes Cf (Other: Format) and Co (Other: Private Use) categories,
+  # which include stuff like BOM or BIDI formatting and other invisible characters,
+  # which you may or may not want. To protect the innocent and prevent nasty surprises,
+  # DEFAULT_REJECT was introduced to avoid these by default, but make it possible to override.
+  DEFAULT_MATCH = /\A(\p{Graph}|[ \t\r\n])*\z/u
+
+  # Default reject applied to all input (in addition to user's reject(s)).
+  DEFAULT_REJECT = /\p{Cf}|\p{Co}/u
+
   # Minimum hash key value we allow by default.
   DEFAULT_MIN_KEY = 0
 
@@ -49,6 +60,9 @@ class FormInput
     match_msg: "%p like this is not valid",
   }
 
+  # Character used as default replacement for invalid characters when formatting values.
+  DEFAULT_REPLACEMENT_CHARACTER = '?'
+
   # Parameter options which can be merged together into an array when multiple option hashes are merged.
   MERGEABLE_OPTIONS = [ :check, :test ]
 
@@ -92,8 +106,17 @@ class FormInput
       form ? form[ name ] : nil
     end
 
+    # Make sure given string is in our default encoding and contains only valid characters.
+    def cleanup_string( string, replacement )
+      unless string.valid_encoding? && ( string.encoding == DEFAULT_ENCODING || string.ascii_only? )
+        string = string.dup.force_encoding( DEFAULT_ENCODING ).scrub( replacement )
+      end
+      string
+    end
+
     # Format given value for form/URL output, applying the formatting filter as necessary.
-    def format_value( value )
+    def format_value( value, replacement = DEFAULT_REPLACEMENT_CHARACTER )
+      value = cleanup_string( value, replacement ) if replacement and value.is_a?( String )
       if format.nil? or value.nil? or ( value.is_a?( String ) and type = self[ :class ] and type != String )
         value.to_s
       else
@@ -102,16 +125,26 @@ class FormInput
     end
 
     # Get value of this parameter for use in form/URL, with all scalar values converted to strings.
-    def form_value
+    def formatted_value( replacement = DEFAULT_REPLACEMENT_CHARACTER )
       if array?
-        [ *value ].map{ |x| format_value( x ) }
+        [ *value ].map{ |x| format_value( x, replacement ) }
       elsif hash?
-        Hash[ [ *value ].map{ |k, v| [ k.to_s, format_value( v ) ] } ]
+        Hash[ [ *value ].map{ |k, v| [ replacement ? cleanup_string( k.to_s, replacement ) : k.to_s, format_value( v, replacement ) ] } ]
       else
-        format_value( value )
+        format_value( value, replacement )
       end
     end
 
+    # Get value of this parameter for use in form, with all scalar values converted to strings.
+    def form_value
+      formatted_value
+    end
+
+    # Get value of this parameter for use in URL, with all scalar values converted to strings.
+    def url_value
+      formatted_value( nil )
+    end
+
     # Test if given parameter has value of correct type.
     def correct?
       case v = value
@@ -162,6 +195,16 @@ class FormInput
       not empty?
     end
 
+    # Test if value of given parameter was set.
+    def set?
+      form ? form.instance_variable_defined?( "@#{name}" ) : false
+    end
+
+    # Test if value of given parameter is not set.
+    def unset?
+      not set?
+    end
+
     # Get the proper name for use in form names, adding [] to array and [key] to hash parameters.
     def form_name( key = nil )
       if array?
@@ -442,7 +485,7 @@ class FormInput
       # If there is a key pattern specified, make sure the key matches.
 
       if patterns = self[ :match_key ]
-        unless [ *patterns ].all?{ |x| value.to_s =~ x }
+        unless value.to_s.valid_encoding? and [ *patterns ].all?{ |x| value.to_s =~ x }
           report( :match_key )
           return
         end
@@ -553,7 +596,7 @@ class FormInput
         return
       end
 
-      unless value =~ /\A(\p{Graph}|[ \t\r\n])*\z/u
+      unless value =~ DEFAULT_MATCH && value !~ DEFAULT_REJECT
         report( :invalid_characters )
         return
       end
@@ -835,8 +878,10 @@ class FormInput
   # Import parameter values from given request or hash. Applies parameter input filters and transforms as well.
   # Returns self for chaining.
   def import( request )
+    hash = request.respond_to?( :params ) ? request.params : request.to_hash
     for name, param in @params
-      if value = request[ param.code ]
+      value = hash.fetch( param.code ) { hash.fetch( param.code.to_s, self ) }
+      unless value == self
         value = sanitize_value( value, param.filter )
         if transform = param.transform
           value = value.instance_exec( &transform )
@@ -856,12 +901,20 @@ class FormInput
     self
   end
 
+  # Unset values of given parameters. Both names and parameters are accepted.
+  # Returns self for chaining.
+  def unset( name, *names )
+    clear( name, *names )
+  end
+
   # Clear all/given parameter values. Both names and parameters are accepted.
   # Returns self for chaining.
   def clear( *names )
     names = names.empty? ? params_names : validate_names( names )
     for name in names
+      # Set the value to nil first so it triggers anything necessary.
       self[ name ] = nil
+      remove_instance_variable( "@#{name}" )
     end
     self
   end
@@ -883,7 +936,7 @@ class FormInput
   end
 
   # Return all non-empty parameters as a hash.
-  # See also #to_data, which creates a hash of non-nil parameters,
+  # See also #to_data, which creates a hash of set parameters,
   # and #url_params, which creates a hash suitable for url output.
   def to_hash
     result = {}
@@ -892,13 +945,13 @@ class FormInput
   end
   alias to_h to_hash
 
-  # Return all non-nil parameters as a hash.
+  # Return all set parameters as a hash.
   # Note that the keys are external names of the parameters (should they differ),
   # so the keys created by `from_data(data).to_data` remain consistent.
   # See also #to_hash, which creates a hash of non-empty parameters.
   def to_data
     result = {}
-    params.each{ |x| result[ x.code ] = x.value unless x.value.nil? }
+    set_params.each{ |x| result[ x.code ] = x.value }
     result
   end
 
@@ -914,11 +967,7 @@ class FormInput
 
   # Create copy of itself, with given parameters unset. Both names and parameters are accepted.
   def except( *names )
-    result = dup
-    for name in validate_names( names )
-      result[ name ] = nil
-    end
-    result
+    dup.clear( names )
   end
 
   # Create copy of itself, with only given parameters set. Both names and parameters are accepted.
@@ -926,11 +975,7 @@ class FormInput
     # It would be easier to create new instance here and only copy selected values,
     # but we want to use dup instead of new here, as the derived form can use
     # different parameters in its construction.
-    result = dup
-    for name in params_names - validate_names( names )
-      result[ name ] = nil
-    end
-    result
+    dup.clear( params_names - validate_names( names ) )
   end
 
   # Parameter lists.
@@ -990,6 +1035,18 @@ class FormInput
   end
   alias filled_parameters filled_params
 
+  # Get list of parameters whose values were set.
+  def set_params
+    params.select{ |x| x.set? }
+  end
+  alias set_parameters set_params
+
+  # Get list of parameters whose values were not set.
+  def unset_params
+    params.select{ |x| x.unset? }
+  end
+  alias unset_parameters unset_params
+
   # Get list of required parameters.
   def required_params
     params.select{ |x| x.required? }
@@ -1089,15 +1146,32 @@ class FormInput
   # Get hash of all non-empty parameters for use in URL.
   def url_params
     result = {}
-    filled_params.each{ |x| result[ x.code ] = x.form_value }
+    filled_params.each{ |x| result[ x.code ] = x.url_value }
     result
   end
   alias url_parameters url_params
   alias to_params url_params
 
+  # Escape given string for use in URL query.
+  def url_escape( string )
+    URI.encode_www_form_component( string )
+  end
+
+  # Build URL query from given URL parameters.
+  def build_url_query( value, prefix = nil )
+    case value
+    when Hash
+      value.map{ |k, v| k = url_escape( k ) ; build_url_query( v, prefix ? "#{prefix}[#{k}]" : k ) }.join( '&' )
+    when Array
+      value.map{ |v| build_url_query( v, "#{prefix}[]" ) }.join( '&' )
+    else
+      "#{prefix}=#{url_escape( value )}"
+    end
+  end
+
   # Create string containing URL query from all current non-empty parameters.
   def url_query
-    Rack::Utils.build_nested_query( url_params )
+    build_url_query( url_params )
   end
 
   # Extend given URL with query created from all current non-empty parameters.

data/lib/form_input/version.rb

@@ -3,7 +3,7 @@
 class FormInput
   module Version
     MAJOR = 1
-    MINOR = 2
+    MINOR = 4
     PATCH = 0
     STRING = [ MAJOR, MINOR, PATCH ].join( '.' ).freeze
   end

data/test/helper.rb

@@ -14,7 +14,9 @@ end unless jruby?
 
 if ENV[ 'COVERAGE' ]
   require 'simplecov'
-  SimpleCov.start
+  SimpleCov.start do
+    add_filter 'bundler'
+  end
 end
 
 # EOF #

data/test/test_core.rb

@@ -115,7 +115,15 @@ describe FormInput do
     [ 'q is required', q: "\u{0000}" ],  # Because strip strips \0 as well.
     [ 'q may not contain invalid characters', q: "a\u{0000}b" ],
     [ 'q may not contain invalid characters', q: "\u{0001}" ],
-    [ 'q may not contain invalid characters', q: "\u{2029}" ],
+    [ 'q may not contain invalid characters', q: "\u{2029}" ],    # NBSP
+    [ 'q may not contain invalid characters', q: "\u{00AD}" ],    # Soft-hyphen
+    [ 'q may not contain invalid characters', q: "\u{FEFF}" ],    # BOM, ZWNBSP
+    [ 'q may not contain invalid characters', q: "\u{E000}" ],    # Private use
+    [ 'q may not contain invalid characters', q: "\u{F8FF}" ],    # Private use
+    [ 'q may not contain invalid characters', q: "\u{F0000}" ],   # Private use plane 15
+    [ 'q may not contain invalid characters', q: "\u{FFFFD}" ],   # Private use plane 15
+    [ 'q may not contain invalid characters', q: "\u{100000}" ],  # Private use plane 16
+    [ 'q may not contain invalid characters', q: "\u{10FFFD}" ],  # Private use plane 16
     [ 'email address like this is not valid', email: 'abc' ],
     [ 'email address may have at most 255 characters', email: 'a@' + 'a' * 254 ],
     [ 'email address may have at most 255 bytes', email: 'á@' + 'a' * 253 ],
@@ -782,6 +790,9 @@ describe FormInput do
     names( f.empty_params ).should == [ :age, :rate, :password, :opts, :on ]
     names( f.blank_params ).should == [ :email, :age, :rate, :password, :opts, :on ]
 
+    names( f.set_params ).should == [ :query, :email, :text, :password ]
+    names( f.unset_params ).should == [ :age, :rate, :opts, :on ]
+
     names( f.tagged_params ).should == [ :age, :rate ]
     names( f.untagged_params ).should == [ :query, :email, :text, :password, :opts, :on ]
 
@@ -857,6 +868,8 @@ describe FormInput do
     p.should.not.be.blank
     p.should.not.be.empty
     p.should.be.filled
+    p.should.be.set
+    p.should.not.be.unset
     p.should.be.valid
     p.should.not.be.invalid
     p.should.be.required
@@ -890,6 +903,8 @@ describe FormInput do
     p.should.be.blank
     p.should.not.be.empty
     p.should.be.filled
+    p.should.be.set
+    p.should.not.be.unset
     p.should.not.be.valid
     p.should.be.invalid
     p.should.not.be.required
@@ -914,6 +929,8 @@ describe FormInput do
     p.should.be.blank
     p.should.be.empty
     p.should.not.be.filled
+    p.should.not.be.set
+    p.should.be.unset
     p.should.be.disabled
     p.should.not.be.enabled
     p[ :tag ].should == :mix
@@ -942,6 +959,8 @@ describe FormInput do
     p.should.be.blank
     p.should.be.empty
     p.should.not.be.filled
+    p.should.not.be.set
+    p.should.be.unset
     p.should.be.array
     p.should.not.be.hash
     p.should.not.be.scalar
@@ -953,6 +972,8 @@ describe FormInput do
     p.should.be.blank
     p.should.be.empty
     p.should.not.be.filled
+    p.should.not.be.set
+    p.should.be.unset
     p.should.not.be.array
     p.should.be.hash
     p.should.not.be.scalar
@@ -990,6 +1011,8 @@ describe FormInput do
     f.except( [ :email, :query ] ).url_query.should == "text=foo"
     f.except( [] ).url_query.should == "q=x&email=a%40b&text=foo"
 
+    f.except( :email ).to_data.should == { q: 'x', text: 'foo' }
+
     f.only( :email ).url_query.should == "email=a%40b"
     f.only( :email, :query ).url_query.should == "q=x&email=a%40b"
     f.only().url_query.should == ""
@@ -997,15 +1020,34 @@ describe FormInput do
     f.only( [ :email, :query ] ).url_query.should == "q=x&email=a%40b"
     f.only( [] ).url_query.should == ""
 
+    f.only( :email ).to_data.should == { email: 'a@b' }
+
+    d = f.dup
+    d.unset( :text )
+    d.should.not.be.empty
+    d.url_query.should == "q=x&email=a%40b"
+    d.to_data.should == { q: 'x', email: 'a@b' }
+    d.unset( d.required_params )
+    d.should.not.be.empty
+    d.url_query.should == "email=a%40b"
+    d.to_data.should == { email: 'a@b' }
+    d.unset( [] )
+    d.should.not.be.empty
+    d.url_query.should == "email=a%40b"
+    d.to_data.should == { email: 'a@b' }
+
     f.clear( :text )
     f.should.not.be.empty
     f.url_query.should == "q=x&email=a%40b"
+    f.to_data.should == { q: 'x', email: 'a@b' }
     f.clear( f.required_params )
     f.should.not.be.empty
     f.url_query.should == "email=a%40b"
+    f.to_data.should == { email: 'a@b' }
     f.clear
     f.should.be.empty
     f.url_query.should == ""
+    f.to_data.should == {}
 
     f = TestForm.new( { age: 2, query: "x" }, { rate: 1, query: "y" } )
     f.url_query.should == "q=y&age=2&rate=1"
@@ -1052,6 +1094,8 @@ describe FormInput do
     f.named_params( :typo, :missing ).should == [ nil, nil ]
 
     ->{ f.set( typo: 10 ) }.should.raise( NoMethodError )
+    ->{ f.unset() }.should.raise( ArgumentError )
+    ->{ f.unset( :typo ) }.should.raise( ArgumentError )
     ->{ f.clear( :typo ) }.should.raise( ArgumentError )
     ->{ f.except( :typo ) }.should.raise( ArgumentError )
     ->{ f.except( [ :typo ] ) }.should.raise( ArgumentError )
@@ -1109,6 +1153,7 @@ describe FormInput do
     f = TestForm.new( query: true, age: 3, rate: 0.35, text: false, opts: [], on: {} )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { query: true, age: 3, rate: 0.35, text: false }
+    f.to_data.should == { q: true, age: 3, rate: 0.35, text: false, opts: [], on: {} }
     f.url_params.should == { q: "true", age: "3", rate: "0.35", text: "false" }
     f.url_query.should == "q=true&age=3&rate=0.35&text=false"
     names( f.incorrect_params ).should == [ :query, :rate, :text ]
@@ -1116,6 +1161,7 @@ describe FormInput do
     f = TestForm.new( opts: 1 )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { opts: 1 }
+    f.to_data.should == { opts: 1 }
     f.url_params.should == { opts: [ "1" ] }
     f.url_query.should == "opts[]=1"
     names( f.incorrect_params ).should == [ :opts ]
@@ -1123,6 +1169,7 @@ describe FormInput do
     f = TestForm.new( opts: [ 2.5, true ] )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { opts: [ 2.5, true ] }
+    f.to_data.should == { opts: [ 2.5, true ] }
     f.url_params.should == { opts: [ "2.5", "true" ] }
     f.url_query.should == "opts[]=2.5&opts[]=true"
     names( f.incorrect_params ).should == []
@@ -1130,6 +1177,7 @@ describe FormInput do
     f = TestForm.new( opts: { "foo" => 10, true => false } )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { opts: { "foo" => 10, true => false } }
+    f.to_data.should == { opts: { "foo" => 10, true => false } }
     f.url_params.should == { opts: [ '["foo", 10]', '[true, false]' ] }
     f.url_query.should == "opts[]=%5B%22foo%22%2C+10%5D&opts[]=%5Btrue%2C+false%5D"
     names( f.incorrect_params ).should == [ :opts ]
@@ -1137,6 +1185,7 @@ describe FormInput do
     f = TestForm.new( on: 1 )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { on: 1 }
+    f.to_data.should == { on: 1 }
     f.url_params.should == { on: { "1" => "" } }
     f.url_query.should == "on[1]="
     names( f.incorrect_params ).should == [ :on ]
@@ -1144,6 +1193,7 @@ describe FormInput do
     f = TestForm.new( on: { 0 => 1, 2 => 3.4 } )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { on: { 0 => 1, 2 => 3.4 } }
+    f.to_data.should == { on: { 0 => 1, 2 => 3.4 } }
     f.url_params.should == { on: { "0" => "1", "2" => "3.4" } }
     f.url_query.should == "on[0]=1&on[2]=3.4"
     names( f.incorrect_params ).should == []
@@ -1151,6 +1201,7 @@ describe FormInput do
     f = TestForm.new( on: [ [ 10, 20 ], [ true, false ] ] )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { on: [ [ 10, 20 ], [ true, false ] ] }
+    f.to_data.should == { on: [ [ 10, 20 ], [ true, false ] ] }
     f.url_params.should == { on: { "10" => "20", "true" => "false" } }
     f.url_query.should == "on[10]=20&on[true]=false"
     names( f.incorrect_params ).should == [ :on ]
@@ -1158,6 +1209,7 @@ describe FormInput do
     f = TestForm.new( on: [ 1, true, false ] )
     ->{ f.validate }.should.not.raise
     f.to_hash.should == { on: [ 1, true, false ] }
+    f.to_data.should == { on: [ 1, true, false ] }
     f.url_params.should == { on: { "1" => "", "true" => "", "false" => "" } }
     f.url_query.should == "on[1]=&on[true]=&on[false]="
     names( f.incorrect_params ).should == [ :on ]
@@ -1165,13 +1217,19 @@ describe FormInput do
 
   should 'handle invalid encoding gracefully' do
     s = 255.chr.force_encoding( 'UTF-8' )
+    b = s.dup.force_encoding( 'BINARY' )
 
     f = TestForm.new( query: s )
     ->{ f.validate }.should.not.raise
     f.should.not.be.valid
     f.error_messages.should == [ "q must use valid encoding" ]
-    f.param( :query ).should.not.be.blank
+    p = f.param( :query )
+    p.should.not.be.blank
+    p.should.be.invalid
+    p.form_value.should == '?'
+    p.url_value.should == s
     f.to_hash.should == { query: s }
+    f.to_data.should == { q: s }
     f.url_params.should == { q: s }
     f.url_query.should == "q=%FF"
 
@@ -1179,10 +1237,46 @@ describe FormInput do
     ->{ f.validate }.should.not.raise
     f.should.not.be.valid
     f.error_messages.should == [ "q must use valid encoding" ]
-    f.param( :query ).should.not.be.blank
-    f.to_hash.should == { query: s.dup.force_encoding( 'BINARY' ) }
-    f.url_params.should == { q: s.dup.force_encoding( 'BINARY' ) }
+    p = f.param( :query )
+    p.should.not.be.blank
+    p.should.be.invalid
+    p.form_value.should == '?'
+    p.url_value.should == b
+    f.to_hash.should == { query: b }
+    f.to_data.should == { q: b }
+    f.url_params.should == { q: b }
     f.url_query.should == "q=%FF"
+
+    c = Class.new( FormInput )
+    c.hash :hsh, :h, match_key: /\A\w+\z/
+
+    f = c.new( hsh: { s => 1, 2 => s } )
+    ->{ f.validate }.should.not.raise
+    f.should.not.be.valid
+    f.error_messages.should == [ "h contain invalid key" ]
+    p = f.param( :hsh )
+    p.should.not.be.blank
+    p.should.be.invalid
+    p.form_value.should == { '?' => '1', '2' => '?' }
+    p.url_value.should == { s => '1', '2' => s }
+    f.to_hash.should == { hsh: { s => 1, 2 => s } }
+    f.to_data.should == { h: { s => 1, 2 => s } }
+    f.url_params.should == { h: { s => '1', '2' => s } }
+    f.url_query.should == "h[%FF]=1&h[2]=%FF"
+
+    f = c.new( request( "?h[%ff]=1&h[2]=%ff" ) )
+    ->{ f.validate }.should.not.raise
+    f.should.not.be.valid
+    f.error_messages.should == [ "h contain invalid key" ]
+    p = f.param( :hsh )
+    p.should.not.be.blank
+    p.should.be.invalid
+    p.form_value.should == { '?' => '1', '2' => '?' }
+    p.url_value.should == { s => '1', '2' => b }
+    f.to_hash.should == { hsh: { s => '1', 2 => b } }
+    f.to_data.should == { h: { s => '1', 2 => b } }
+    f.url_params.should == { h: { s => '1', '2' => b } }
+    f.url_query.should == "h[%FF]=1&h[2]=%FF"
   end
 
   should 'reject unexpected values in request input' do
@@ -1208,10 +1302,11 @@ describe FormInput do
       opts: [],
       on: {},
     }
-    f = TestForm.from_data(data)
+    f = TestForm.from_data( data )
     f.to_data.should == data
 
-    f = TestForm.from_data(data.merge(password: nil))
+    data[ :password ] = nil
+    f = TestForm.from_data( data )
     f.to_data.should == data
 
     f.to_hash.should == {
@@ -1220,8 +1315,6 @@ describe FormInput do
       age: 10,
       rate: 0.5,
     }
-    f.valid?(:age).should.be.true	# has filter and correct class
-    f.valid?(:rate).should.be.false	# has no filter and class
   end
 
   should 'make it easy to create URLs' do
@@ -1370,6 +1463,13 @@ describe FormInput do
     f.dup.validate?.should.be.valid
     f.validate.should.be.valid
     f.validate!.should.be.valid
+
+    f.unset( :query ).should.equal f
+    f.should.be.invalid
+    f.validate?.should.be.invalid
+    f.dup.validate?.should.be.invalid
+    f.validate.should.be.invalid
+    f.validate!.should.be.invalid
   end
 
   should 'support some custom error messages' do

data/test/test_types.rb

@@ -7,41 +7,40 @@ require 'form_input/types'
 require 'rack/test'
 
 class TestBasicTypesForm < FormInput
-
   param :int, INTEGER_ARGS
   param :float, FLOAT_ARGS
   param :bool, BOOL_ARGS
   param :checkbox, CHECKBOX_ARGS
+end
 
+class TestJSONTypesForm < FormInput
+  param :str
+  param :int, INTEGER_ARGS
+  param :float, FLOAT_ARGS
+  param :bool, BOOL_ARGS
 end
 
 class TestAddressTypesForm < FormInput
-
   param :email, EMAIL_ARGS
   param :zip, ZIP_ARGS
   param :phone, PHONE_ARGS
-
 end
 
 class TestTimeTypesForm < FormInput
-
   param :time, TIME_ARGS
   param :us_date, US_DATE_ARGS
   param :uk_date, UK_DATE_ARGS
   param :eu_date, EU_DATE_ARGS
   param :hours, HOURS_ARGS
-
 end
 
 class TestPrunedTypesForm < FormInput
-
   param :str, PRUNED_ARGS
   param :int, INTEGER_ARGS, PRUNED_ARGS
   array :arr, PRUNED_ARGS
   array :int_arr, INTEGER_ARGS, PRUNED_ARGS
   hash :hsh, PRUNED_ARGS
   hash :int_hsh, INTEGER_ARGS, PRUNED_ARGS
-
 end
 
 class Bacon::Context
@@ -131,6 +130,33 @@ describe FormInput do
     f.url_query.should == "int=a&float=b&bool=false&checkbox=true"
   end
 
+  should 'provide presets for standard JSON types' do
+    f = TestJSONTypesForm.from_data( {} )
+    f.should.be.valid
+    f.to_data.should == {}
+
+    f = TestJSONTypesForm.from_data( str: "foo", int: "0123", float: "0123.456", bool: "true" )
+    f.should.be.valid
+    f.to_data.should == { str: 'foo', int: 123, float: 123.456, bool: true }
+
+    f = TestJSONTypesForm.from_data( str: nil, int: 123, float: 123.456, bool: true )
+    f.should.be.valid
+    f.to_data.should == { str: nil, int: 123, float: 123.456, bool: true }
+
+    f = TestJSONTypesForm.from_data( bool: "false" )
+    f.should.be.valid
+    f.to_data.should == { bool: false }
+
+    f = TestJSONTypesForm.from_data( bool: false )
+    f.should.be.valid
+    f.to_data.should == { bool: false }
+
+    f = TestJSONTypesForm.from_data( int: "a", float: "b", bool: "c" )
+    f.should.be.invalid
+    names( f.invalid_params ).should == [ :int, :float ]
+    f.to_data.should == { int: "a", float: "b", bool: false }
+  end
+
   should 'provide presets for address parameter types' do
     f = TestAddressTypesForm.new( request( "?email=me@1.com&zip=12345&phone=123%20456%20789" ) )
     f.should.be.valid

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: form_input
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.4.0
 platform: ruby
 authors:
 - Patrik Rak
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-06-21 00:00:00.000000000 Z
+date: 2025-02-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rack
@@ -19,7 +19,7 @@ dependencies:
         version: '1.5'
     - - "<"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -29,7 +29,21 @@ dependencies:
         version: '1.5'
     - - "<"
       - !ruby/object:Gem::Version
-        version: '3.0'
+        version: '4.0'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13.0'
 - !ruby/object:Gem::Dependency
   name: bacon
   requirement: !ruby/object:Gem::Requirement
@@ -45,33 +59,53 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '1.2'
 - !ruby/object:Gem::Dependency
-  name: rack-test
+  name: simplecov
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.6'
+        version: '0.21'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.6'
+        version: '0.21'
+- !ruby/object:Gem::Dependency
+  name: rack-test
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '3.0'
 - !ruby/object:Gem::Dependency
   name: r18n-core
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: '5.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.0'
+        version: '5.0'
 description: |
   This gem allows you to describe your forms using a simple DSL
   and then takes care of sanitizing, transforming, and validating the input for you,
@@ -94,6 +128,7 @@ files:
 - example/controllers/ramaze/profile.rb
 - example/controllers/sinatra/press_release.rb
 - example/controllers/sinatra/profile.rb
+- example/example.rb
 - example/forms/change_password_form.rb
 - example/forms/login_form.rb
 - example/forms/lost_password_form.rb
@@ -149,18 +184,16 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.0.0
+      version: 2.1.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.4.5.1
+rubygems_version: 3.0.3.1
 signing_key: 
 specification_version: 4
 summary: Form helper which sanitizes, transforms, validates and encapsulates web request
   input.
 test_files: []
-has_rdoc: