philtre 0.0.0 → 0.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4e7ebc6f6d2d25c12725e30ad189728f27a843da
-  data.tar.gz: 954a8ba030a5ed3c04e8e7f814ee242387c346d5
+  metadata.gz: 581e7addf0188f66a5a341f6dd205e3eb6b93a7e
+  data.tar.gz: 58a47bc162546a980faa4f17f5b28e75b0b86e9c
 SHA512:
-  metadata.gz: 5fe7d82fceac966a6803e9568dda95fc56bafe63106d183e63444591a9ce3511739a7cd1ddb82820573690ca0079194223503875fa1a4154df54151fa9c1a662
-  data.tar.gz: d54f471e8819fd0ae883182c703b88f509d6590ba06e6817e8030540df5e4b7fb7dc122cf22be1888ae4f4ee69d072b6d5ff6b92dcb77998309feed638c823e8
+  metadata.gz: 9a7cb668887750dfc955373d5cdc2a1f54ca06b56e9322217c29fbc3a7f4d0b598d142e7e25cfb91f80d8cb4e4642b3196b748d32a499c03edd4541bfcd2e41b
+  data.tar.gz: 86d1ef5d892b136cafbd8e0ac63ef45e799ee268fc93900e11c52409ac5e0cae5db221184915b6db70bba07125a60c39114ecf426467ef3cf9086ef930a1ebf8

data/.gitignore

@@ -15,8 +15,9 @@ spec/reports
 test/tmp
 test/version_tmp
 tmp
-*.bundle
-*.so
-*.o
-*.a
-mkmf.log
+.ruby-gemset
+.ruby-version
+*.sublime-project
+*.sublime-workspace
+.floo
+.flooignore

data/.travis.yml

@@ -0,0 +1,7 @@
+language: ruby
+rvm:
+  - 2.0.0
+  - 2.1.0
+  - 2.1.1
+  - 2.1.2
+script: bundle exec rspec spec

data/README.md

@@ -1,12 +1,9 @@
-# Philtre
+# philtre [![Gem Version](https://badge.fury.io/rb/philtre.png)](http://badge.fury.io/rb/philtre)
 
-Coming Soon...
+It's the [Sequel](http://sequel.jeremyevans.net) equivalent for Ransack, Metasearch, Searchlogic. If
+this doesn't make you fall in love, I don't know what will :-p
 
-It's the Sequel equivalent for Ransack, Metasearch, Searchlogic.
-If this doesn't make you fall in love, I don't know what will.
-
-Yeah, I know. Corny naming. Filter => Philtre. But it lets you mix things up
-and something awesome comes out the other side.
+See philtre-rails for rails integration.
 
 ## Installation
 
@@ -14,6 +11,10 @@ Add this line to your application's Gemfile:
 
     gem 'philtre'
 
+Or for all the rails integration goodies
+
+    gem 'philtre-rails'
+
 And then execute:
 
     $ bundle
@@ -22,15 +23,171 @@ Or install it yourself as:
 
     $ gem install philtre
 
-## Usage
+## Basic Usage
+
+Parse the predicates on the end of field names, and modify a Sequel::Dataset
+to retrieve matching rows.
+
+So, using a fairly standard rails-style parameter hash:
+
+``` ruby
+  filter_parameters = {
+    birth_year: ['2012', '2011'],
+    title: 'bar',
+    order: ['title', 'name_asc', 'birth_year_desc'],
+  }
+
+  # This would normally be a real Sequel::Dataset
+  personages_dataset = Sequel.mock[:personages]
+
+  philtre = Philtre.new( filter_parameters ).apply( personages_dataset ).sql
+```
+
+should result in (formatting added here for clarity)
+
+``` SQL
+  SELECT *
+  FROM "personages"
+  WHERE
+    (("birth_year" IN ('2012', '2011'))
+    AND
+    ("title" = 'bar'))
+  ORDER BY ("title" ASC, "name" ASC, "date" DESC)
+```
+
+## Predicates
+
+```{title: 'sir'}``` is fine when you want to match on string equality. But
+there are all kinds of other things you need to do. For example
+
+```{title_like: 'sir', age_gt: 10}``` is for a where clause ```title ~* 'sir' and age >  10```
+
+There are a range of predefined predicates, mostly borrowed from the other search gems:
+
+```
+  gt
+  gte, gteq
+  lt
+  lte, lteq
+  eq
+  not_eq
+  matches, like
+  not_blank
+  like_all
+  like_any
+```
+
+## Custom Predicates
+
+There are two ways:
+
+1) You can also define your own by creating a Filter with a block:
+
+``` ruby
+  philtre = Philtre.new filter_parameters do
+    def tagged_by_id(tag_ids)
+      Tag.db[:projects_tags]
+        .select(:personage_id)
+        .filter(tag_id: tag_ids, :project_id => :personage__id )
+        .exists
+    end
+
+    def really_fancy(tag_ids)
+      # do some really fancy SQL here
+    end
+
+    # etc...
+  end
+```
+
+Now you can pass the filter_parameter hash ```{tagged_by_id: 45}```.
+
+The result of a predicate block should be a ```Sequel::SQL::Expression``` (ie
+one of Sequel's hash expressions in the simplest case) which will work instead
+of its named placeholder. That is, if the placeholder is inside a SELECT
+clause it worked work to give in an ORDER BY.
+
+2) You could also inherit from ```Philtre::Filter``` and override
+```#predicates```. And optionally override ```Philtre.new``` (which is just a
+factory method on ```module Philtre```) to return the instance of your class.
+
+## Advanced usage
+
+There is also the ```Philtre::Grinder``` class which can insert placeholders into
+your ```Sequel::Dataset``` definition, and then substitute those once it has the
+parameter hash. Effectively this makes it a SQL macro engine.
+
+Why so complicated? Well, it's really handy when you need to use aggregate
+queries, and apply different values in the parameter hash to where clauses
+inside and the outside of the aggregation. For example, give me a list of all
+stores in a particular region who share of total sales was more than some
+percentage. Yes, you can also use window functions to deal with that
+particular query.
+
+``` ruby
+  # This would normally be a real Sequel::Dataset
+  stores_dataset = Sequel.mock[:stores]
+
+  # parameterise it with placeholders
+  parameterised_dataset = stores_dataset.filter( :region.lieu, :sales_gt.lieu, :manager.lieu )
+
+  filter_parameters = {
+    region: 'The Bundus',
+    sales_gt: 10,
+    order: ['store_name', 'sales_desc'],
+  }
+
+  # generate the SQL you need
+  parameterised_dataset.grind( Philtre.new( filter_parameters ) ).sql
+```
+
+will result in
+
+``` SQL
+  SELECT *
+  FROM stores
+  WHERE ((region = 'The Bundus') AND (sales > 10))
+```
+
+Notice that the manager part of the where clause is absent because
+filter_parameters didn't have a manager key.
+
+Look at the sql generated by parameterised_dataset and you'll see the placeholders
+marked by SQL comments, so you can debug the Giant SQL Statement more easily. You
+might also want to find a command-line SQL pretty printer (eg ``fsqlf```) and use it to produce
+readable SQL instead of a very long hard-to-read string.
+
+If you don't like the monkey-patching of Symbol with #lieu, you can use
+several other ways to generate the placeholders. ```Philtre::PlaceHolder.new```
+is canonical in that all the other possibilities use it.
+
+## Highly Advanced Usage
+
+Sometimes method chaining gets ugly. So you can say
+
+``` ruby
+  store_id_range = 20..90
+  parameterised_dataset = stores_dataset.rolled do
+    where :region.lieu, :sales_gt.lieu, :manager.lieu
+    where store_id: store_id_range
+    select_append db[:products].join(:stores, :store_id => :id ).select(:product_name)
+  end
+```
+
+Notice that values outside the block are accessible inside, _without_
+the need for a block parameter. This uses Ripar under the cover and indirects
+the binding lookup, so may result in errors that you won't expect.
+
+## Specs
+
+Nothing fancy. Just:
 
-filter = Philtre.new( hash )
-dataset = filter.apply YourModel.dataset
+    $ rspec spec
 
 ## Contributing
 
-1. Fork it ( https://github.com/djellemah/philtre/fork )
+1. Fork it ( http://github.com/djellemah/philtre/fork )
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
 4. Push to the branch (`git push origin my-new-feature`)
-5. Create a new Pull Request
+5. Create new Pull Request

data/Rakefile

@@ -1,2 +1,10 @@
 require "bundler/gem_tasks"
 
+require 'pathname'
+
+SELF_PATH = Pathname(__FILE__).parent
+
+$: << SELF_PATH + 'lib'
+load SELF_PATH + 'lib/philtre/version.rb'
+
+load SELF_PATH + 'tasks/console.rake'

data/lib/philtre.rb

@@ -1,5 +1,62 @@
-require "philtre/version"
+require 'philtre/filter.rb'
+require 'philtre/grinder.rb'
 
+# The high-level interface to Philtre. There are several ways
+# to use it:
+# 1. Philtre.new
+#     philtre = Philtre.new name: 'Moustafa'
+# 1. Philtre
+#     philtre = Philtre dataset: some_dataset, age_gt: 21
+#     philtre = Philtre dataset: some_dataset, with {age_gt: 21}
+# 1. Philtre.filter
+#     philtre = Philtre.filter dataset: some_dataset, name: 'Moustafa', age_gt: 21
+#     philtre = Philtre.filter dataset: some_dataset, with: {name: 'Moustafa', age_gt: 21}
 module Philtre
-  # Your code goes here...
+  # Just a factory method that calls Filter.new
+  #
+  #  philtre = Philtre.new params[:filter]
+  def self.new( *filter_parameters, &blk )
+    Filter.new *filter_parameters, &blk
+  end
+
+  # This is the high-level, easy-to-read smalltalk-style interface
+  # params:
+  # - dataset is a Sequel::Model or a Sequel::Dataset
+  # - with is the param hash (optional, or just use hash-style args)
+  #
+  # for x-ample, in rails you could do
+  #
+  #  @personages = Philtre.filter dataset: Personage, with: params[:filter]
+  #
+  # or even
+  #
+  #  @personages = Philtre.filter dataset: Personage, name: 'Dylan', age_gt: 21, age_lt: 67
+  #
+  def self.filter( dataset: nil, with: {}, **kwargs )
+    new(with.merge kwargs).apply(dataset)
+  end
+
+  # Create a grinder with the parameters, and
+  # use it on the dataset. Return the result.
+  #
+  # dataset should have placeholders, otherwise calling this
+  # method just warms your cpu.
+  def self.grind( dataset: nil, with: {}, **kwargs )
+    filter = new(with.merge kwargs)
+    Philtre::Grinder.new(filter).transform(dataset)
+  end
+end
+
+require 'philtre/core_extensions.rb'
+
+# And this is the even higher-level smalltalk-style interface
+#
+#  Philtre dataset: Personage, with: params[:filter]
+module Kernel
+private
+  def Philtre( dataset: nil, with: {}, **kwargs )
+    Philtre.filter dataset: dataset, with: with, **kwargs
+  end
+
+  alias philtre Philtre
 end

data/lib/philtre/core_extensions.rb

@@ -0,0 +1,31 @@
+# several ways to create placeholders in Sequel statements
+module Kernel
+private
+  def PlaceHolder( name, sql_field = nil, bt = caller )
+    Philtre::PlaceHolder.new name, sql_field, bt = caller
+  end
+
+  alias_method :Lieu, :PlaceHolder
+end
+
+class Symbol
+  def lieu( sql_field = nil )
+    Lieu self, sql_field, caller
+  end
+
+  def place_holder( sql_field = nil )
+    PlaceHolder self, sql_field, caller
+  end
+end
+
+unless Hash.instance_methods.include? :slice
+  class Hash
+    # return a hash containing only the specified keys
+    def slice( *other_keys )
+      other_keys.inject(Hash.new) do |hash, key|
+        hash[key] = self[key] if has_key?( key )
+        hash
+      end
+    end
+  end
+end

data/lib/philtre/empty_expression.rb

@@ -0,0 +1,9 @@
+module Philtre
+  # used when transforming to unaltered or partially
+  # altered datasets
+  class EmptyExpression < Sequel::SQL::Expression
+    # sometimes this is returned in place of an empty array
+    def empty?; true; end
+    def to_s_append( ds, s ); end
+  end
+end

data/lib/philtre/filter.rb

@@ -0,0 +1,232 @@
+require 'sequel'
+
+Sequel.extension :blank
+
+require 'philtre/predicate_splitter'
+require 'philtre/predicate_dsl'
+require 'philtre/predicates'
+
+module Philtre
+  # Parse the predicates on the end of field names, and round-trip the search fields
+  # between incoming params, controller and views.
+  # So,
+  #
+  #   filter_parameters = {
+  #     birth_year: ['2012', '2011'],
+  #     title_like: 'sir',
+  #     order: ['title', 'name_asc', 'birth_year_desc'],
+  #   }
+  #
+  #   Philtre.new( filter_parameters ).apply( Personage.dataset ).sql
+  #
+  # should result in
+  #
+  #   SELECT * FROM "personages" WHERE (("birth_year" IN ('2012', '2011')) AND ("title" ~* 'bar')) ORDER BY ("title" ASC, "name" ASC, "date" DESC)
+  #
+  # TODO pass a predicates: parameter in here to specify a predicates object.
+  class Filter
+    def initialize( filter_parameters = nil, &custom_predicate_block )
+      # This must be a new instance of Hash, because sometimes
+      # HashWithIndifferentAccess is passed in, which breaks things in here.
+      # Don't use symbolize_keys because that creates a dependency on ActiveSupport
+      @filter_parameters =
+      if filter_parameters
+        # preserve 2.0 compatibility
+        filter_parameters.inject({}){|ha,(k,v)| ha[k.to_sym] = v; ha}
+      else
+        {}
+      end
+
+      if block_given?
+        predicates.extend_with &custom_predicate_block
+      end
+    end
+
+    attr_reader :filter_parameters
+
+    def empty?; filter_parameters.empty? end
+
+    # return a modified dataset containing all the predicates
+    def call( dataset )
+      # mainly for Sequel::Model
+      dataset = dataset.dataset if dataset.respond_to? :dataset
+
+      # clone here so later order! calls don't mess with a Model's default dataset
+      dataset = expressions.inject(dataset.clone) do |dataset, filter_expr|
+        dataset.filter( filter_expr )
+      end
+
+      # preserve existing order if we don't have one.
+      if order_clause.empty?
+        dataset
+      else
+        # There might be multiple orderings in the order_clause
+        dataset.order *order_clause
+      end
+    end
+
+    alias apply call
+
+    # Values in the parameter list which are not blank, and not
+    # an ordering. That is, parameters which will be used to generate
+    # the filter expression.
+    def valued_parameters
+      filter_parameters.select do |key,value|
+        key.to_sym != :order && (value.is_a?(Array) || !value.blank?)
+      end
+    end
+
+    # The set of expressions from the filter_parameters with values.
+    def expressions
+      valued_parameters.map do |key, value|
+        to_expr(key, value)
+      end
+    end
+
+    def self.predicates
+      @predicates ||= Predicates.new
+    end
+
+    # Hash of predicate names to blocks. One way to get custom predicates is
+    # to subclass filter and override this.
+    def predicates
+      # don't mess with the class' minimal set
+      @predicates ||= self.class.predicates.clone
+    end
+
+    attr_writer :predicates
+
+    def order_expr( order_predicate )
+      return if order_predicate.blank?
+
+      splitter = PredicateSplitter.new( order_predicate, nil )
+      case
+      when splitter === :asc
+        Sequel.asc splitter.field
+      when splitter === :desc
+        Sequel.desc splitter.field
+      else
+        Sequel.asc splitter.field
+      end
+    end
+
+    def order_for( order_field )
+      order_hash[order_field]
+    end
+
+    # return a possibly empty array of Sequel order expressions
+    def order_clause
+      @order_clause ||= order_expressions.map{|e| e.last}
+    end
+
+    # Associative array (not a Hash) of names to order expressions
+    # TODO this should just be a hash
+    def order_expressions
+      @order_expressions ||=
+      [filter_parameters[:order]].flatten.map do |order_predicate|
+        next if order_predicate.blank?
+        expr = order_expr order_predicate
+        [expr.expression, expr]
+      end.compact
+    end
+
+    def order_hash
+      @order_hash ||= Hash[ order_expressions ]
+    end
+
+    # turn a filter_parameter key => value into a Sequel::SQL::Expression subclass
+    # field will be the field name ultimately used in the expression. Defaults to key.
+    def to_expr( key, value, field = nil )
+      Sequel.expr( predicates[key, value, field] )
+    end
+
+    # turn the expression at predicate into a Sequel expression with
+    # field, having the value for predicate. Will be nil if the
+    # predicate has no value in valued_parameters.
+    # Will always be a Sequel::SQL::Expression.
+    def expr_for( predicate, field = nil )
+      unless (value = valued_parameters[predicate]).blank?
+        to_expr( predicate, value, field )
+      end
+    end
+
+    # for use in forms
+    def to_h(all=false)
+      filter_parameters.select{|k,v| all || !v.blank?}
+    end
+
+    attr_writer :filter_parameters
+    protected :filter_parameters=
+
+    # deallocate any cached lazies
+    def initialize_copy( *args )
+      super
+      @order_expressions = nil
+      @order_hash = nil
+      @order_clause = nil
+    end
+
+    def clone( extra_parameters = {} )
+      new_filter = super()
+
+      # and explicitly clone these because they may well be modified
+      new_filter.filter_parameters = filter_parameters.clone
+      new_filter.predicates = predicates.clone
+
+      extra_parameters.each do |key,value|
+        new_filter[key] = value
+      end
+
+      new_filter
+    end
+
+    # return a new filter including only the specified filter parameters/predicates.
+    # NOTE predicates are not the same as field names.
+    # args to select_block are the same as to filter_parameters, ie it's a Hash
+    # TODO should use clone
+    def subset( *keys, &select_block )
+      subset_params =
+      if block_given?
+        filter_parameters.select &select_block
+      else
+        filter_parameters.slice( *keys )
+      end
+      subset = self.class.new( subset_params )
+      subset.predicates = predicates.clone
+      subset
+    end
+
+    # return a subset of filter parameters/predicates,
+    # but leave this object without the matching keys.
+    # NOTE does not operate on field names.
+    def extract!( *keys, &select_block )
+      rv = subset( *keys, &select_block )
+      rv.to_h.keys.each do |key|
+        filter_parameters.delete( key )
+      end
+      rv
+    end
+
+    # hash of keys to expressions, but only where
+    # there are values.
+    def expr_hash
+      vary = valued_parameters.map do |key, value|
+        [ key, to_expr(key, value) ]
+      end
+
+      Hash[ vary ]
+    end
+
+    # easier access for filter_parameters
+    # return nil for nil and '' and []
+    def []( key )
+      rv = filter_parameters[key]
+      rv unless rv.blank?
+    end
+
+    # easier access for filter_parameters
+    def []=(key, value)
+      filter_parameters[key] = value
+    end
+  end
+end