praxis 2.0.pre.29 → 2.0.pre.30

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e8f0514dd2a155733de0efe5586dd4b0409c70a9b385822b15e39934a7b83ea2
-  data.tar.gz: ff66561cdaadb84a9c97b83c044c423561b6506e10a31cf72d032dce73dcb50c
+  metadata.gz: a2ffb701843f77a4f33c4b2628cdc7b609c5bb8b1d50bf0df454a77eded14ca3
+  data.tar.gz: 34852fb86cd240a18701aa013a1d65508c17d9c8831ecf5176efa9c73f7b5339
 SHA512:
-  metadata.gz: 9bcb89e0426fb2e9e8ea51e7794a42f0a1ec6969fce436196da8c9d32697b232a02bddb0008400d51c6dc8b2d475e1122283da8644ed78bb448d759fbf4f2363
-  data.tar.gz: 5c0a71be2bb1043ccc4269ccfa258d01e93b66d4dfa2f69938f5a495d8a296a305d2b8051c84a6683343672a69cd01e20f717dc87f4bf5ca4049b587438a3194
+  metadata.gz: 6578cd3f42a3f5dde6f32f9a860169e90f68d20c5a58dddefee2511e51e18368ec81a8ee5c75e60581ee53134e44940cbdad079a9e4c460e1b65121bea613b5b
+  data.tar.gz: 59073268280c18c2a9e77e626c5c75503c45d5b64351bd67898861d68b1f952b1f9c8c124af813049fc4b6aaed87cc74860d6561744c654343f00a60a1c59e4f

data/.rubocop.yml

@@ -10,7 +10,7 @@ AllCops:
     - "pkg/**/*"
     - "vendor/cache/**/*"
     - "tasks/thor/templates/**/*"
-  TargetRubyVersion: 2.5
+  TargetRubyVersion: 2.7
   NewCops: disable
 
 Metrics/BlockLength:

data/.ruby-version

@@ -1 +1 @@
-2.7.1
+3.1.1

data/CHANGELOG.md

@@ -2,6 +2,30 @@
 
 ## next
 
+## 2.0.pre.30
+  * A few cleanup and robustness additions:
+    * OpenAPI: Disable overriding a description when the schema is a ref (there are known issues with UI browsers)
+    * Internal: use `_pk` in batch processor invocation instead of `id` (resources will now have a `_pk` method which defaults to `id`)
+    * Bumped gemspec Ruby dependency to >=2.7 (but note, that this is just a little relaxed for older codebase, we're fully building for 3.x)
+  * Backwards incompatible changes:
+    * Enforces property names are symbols (before strings were allowed)
+    * Resource properties, using the `as:` option, are now enforced to be real association names (will not accept other resource names and unroll their dependencies)
+    * Deprecated the `:through` option for a property. You can just use `as:` with a long, dot-separated association path directly.
+  * Enhanced ordering semantics for pagination to allow for sorting of deep associated fields:
+    * Right now, you can sort by fields such as `books.author.name` as one of the sorting components (with `+` or `-` still available)
+  * Introduced better attribute grouping concepts, that help in defining subgroups of attributes of the same object, and allow lazy loading of only partial subsets so that one can have expensive computations on some of them, but they will never be invoked unless necessary. See MediaType.`group` and Resoruce.`property_group` explanations below.
+  * Introduced a 'group' stanza in MediaTypes, to specify a structure of attributes that exist in the main object, but that we want to neatly expose as a subset (instead of having them unrolled at the top):
+    * You can now use things like `group subinfo do ... end` blocks, defining which attributes to group
+    * Internal: Underneath, the system will create a BlueprintAttributeGroup (instead of a Struct) as a way to ensure that only the individual attributes that need to be rendered, are accessed (and not load the whole struct at once). While the behavior, to the outside, is gonna be identical to a Struct (i.e., exposes attributes as methods), this distinct object implementation is very important as it allows you to have attributes in the subgroup that are expensive to compute, and can be rest assured that they will not be accessed/computed unless they are required for rendering.
+  * Introduced the `property_group` stanza in resources, to indicate that a property contains a substructure of attributes, each of which must be able to be loaded only when necessary. This commonly goes hand in hand with a `group` stanza in the resource's MediaType:
+    * Usage of property group requires the name of the substructure (a symbol), and the associated mediatype that contains the definition of the `group` struct, under the same name of the property.
+    * Internally, this stanza, will define a normal property, and include as dependencies all of the sub attributes read from the MediaType's property, but appending the name (and `_`) to them to avoid collisions.
+    * Also, it will define a method with the property name which will return a Forwarding object, which will delegate each of the attribute methods back to the original self objects. This allows the object to avoid being 'loaded' as a whole as it happens with Struct, therefore only materializing/calling the attribute that we actually need to use, selectively.
+    * For example, if we have the `Book` MediaType which has a group atrribute called `subinfo` with a few attributes (like `name` and `pages`), we can use `property_group :subinfo, Book` on its domain object, so that the system will:
+      * define a `subinfo` property which will depend on `subinfo_name` and `subinfo_pages`
+      * define a `subinfo` method that will return a Forwarding object, that will forward `name` and `pages` methods to `subinfo_name` and `subinfo_pages` methods of the self resource.
+      * with that, we just need to define our `subinfo_name` and `subinfo_page` methods in the resource (and also define property dependencies for them if we need to)
+
 ## 2.0.pre.29
   * Assorted set of fixes to generate cleaner and more compliant OpenApi documents.
     * Mostly in the area of multipart generation, and requirements and nullability for OpenApi 3.0

data/lib/praxis/application.rb

@@ -39,6 +39,10 @@ module Praxis
       @logger = Logger.new($stdout)
     end
 
+    def inspect
+      "<#{self.class}##{object_id} root: #{@root}>"
+    end
+
     def setup(root: '.')
       return self unless @app.nil?
 

data/lib/praxis/blueprint.rb

@@ -2,6 +2,18 @@
 
 module Praxis
   class Blueprint
+    class DSLCompiler < Attributor::HashDSLCompiler
+      # group DSL is meant to group a subset of attributes in the media type (instead of presenting all things flat)
+      # It will create a normal attribute, but as a BlueprintAttributeGroup, rather than a Struct, so that we can more easily
+      # pass in objects that respond to the right methods, and avoid a Struct loading them all in hash keys.
+      # For example, if there are computationally intensive attributes in the subset, we want to make sure those functions
+      # aren't invoked by just merely loading, and only really invoked when we've asked to render them
+      # It takes the name of the group, and passes the attributes block that needs to be a subset of the MediaType where the group resides
+      def group(name, **options, &block)
+        attribute(name, Praxis::BlueprintAttributeGroup.for(target.options[:reference]), **options, &block)
+      end
+    end
+
     # Simple helper class that can parse the `attribute :foobar` dsl into
     # an equivalent structure hash. Example:
     # do
@@ -91,7 +103,7 @@ module Praxis
 
         opts[:reference] = (reference || self)
 
-        @options.merge!(opts)
+        @options.merge!(opts.merge(dsl_compiler: DSLCompiler))
         @block = block
 
         return @attribute

data/lib/praxis/blueprint_attribute_group.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+module Praxis
+  class BlueprintAttributeGroup < Blueprint
+    def self.constructable?
+      true
+    end
+
+    # Construct a new subclass, using attribute_definition to define attributes.
+    def self.construct(attribute_definition, options = {})
+      return self if attribute_definition.nil?
+
+      reference_type = @media_type
+      # Construct a group-derived class with the given mediatype as the reference
+      ::Class.new(self) do
+        @reference = reference_type
+        attributes(**options, &attribute_definition)
+      end
+    end
+
+    def self.for(media_type)
+      return media_type::AttributeGrouping if defined?(media_type::AttributeGrouping) # Cache the grouping class
+
+      ::Class.new(self) do
+        @media_type = media_type
+      end
+    end
+  end
+end

data/lib/praxis/docs/open_api/schema_object.rb

@@ -31,11 +31,13 @@ module Praxis
 
         def dump_schema(shallow: false, allow_ref: false)
           # We will dump schemas for mediatypes by simply creating a reference to the components' section
-          if type < Attributor::Container && ! (type < Praxis::Types::MultipartArray)
+          if type < Attributor::Container && !(type < Praxis::Types::MultipartArray)
             if (type < Praxis::Blueprint || type < Attributor::Model) && allow_ref && !type.anonymous?
-              # TODO: Do we even need a description?
-              h = @attribute_options[:description] ? { 'description' => @attribute_options[:description] } : {}
-
+              # TODO: Technically OpenAPI/JSON schema support passing a description when pointing to a $ref (to override it)
+              # However, it seems that UI browsers like redoc or elements have bugs where if that's done, they get into a loop and crash
+              # so for now, we're gonna avoid overriding the description until that is solved
+              # h = @attribute_options[:description] ? { 'description' => @attribute_options[:description] } : {}
+              h = {}
               Praxis::Docs::OpenApiGenerator.instance.register_seen_component(type)
               h.merge!('$ref' => "#/components/schemas/#{type.id}")
             elsif @collection
@@ -44,12 +46,11 @@ module Praxis
               h.merge!(type: 'array', items: items)
             else # Attributor::Struct, etc
               required_attributes = (type.describe[:requirements] || []).filter { |r| r[:type] == :all }.map { |r| r[:attributes] }.flatten.compact.uniq
-              props = type.attributes.transform_values.with_index do |definition, index|
+              props = type.attributes.transform_values do |definition|
                 # if type has an attribute in its requirements all, then it should be marked as required here
-                field_name = type.attributes.keys[index]
                 OpenApi::SchemaObject.new(info: definition).dump_schema(allow_ref: true, shallow: shallow)
               end
-              h = { type: :object}
+              h = { type: :object }
               h[:properties] = props if props.presence
               h[:required] = required_attributes unless required_attributes.empty?
             end

data/lib/praxis/endpoint_definition.rb

@@ -232,7 +232,7 @@ module Praxis
               action.sister_post_action # Avoid appending prefix
             else
               # Make sure to cleanup the leading '/' if any, as we're always adding it below
-              cleaned_path = action.sister_post_action.start_with?('/') ? action.sister_post_action[1..-1] : action.sister_post_action
+              cleaned_path = action.sister_post_action.start_with?('/') ? action.sister_post_action[1..] : action.sister_post_action
               "#{action.route.prefixed_path}/#{cleaned_path}"
             end
 

data/lib/praxis/extensions/attribute_filtering/active_record_filter_query_builder.rb

@@ -74,7 +74,7 @@ module Praxis
             # Mark where clause referencing the appropriate alias IF it's not the root table, as there is no association to reference
             # If we added root table as a reference, we better make sure it is not quoted, as it actually makes AR to see it as an
             # unmatched reference and eager loads the whole association (it means eager load ALL the things). Not good.
-            associated_query = associated_query.references(build_reference_value(column_prefix, query: associated_query)) unless for_model.table_name == column_prefix
+            associated_query = associated_query.references(self.class.build_reference_value(column_prefix, query: associated_query)) unless for_model.table_name == column_prefix
             self.class.add_clause(
               query: associated_query,
               column_prefix: column_prefix,
@@ -158,7 +158,7 @@ module Praxis
           if association_op
             neg = association_op == :not_null
             qr = quote_right_part(query: query, value: nil, column_object: association_key_column, negative: neg)
-            return query.where("#{quote_column_path(query: query, prefix: column_prefix, column_object: association_key_column)} #{qr}")
+            return query.where("#{quote_column_path(query: query, prefix: column_prefix, column_name: association_key_column.name)} #{qr}")
           end
 
           # Add an AND along with the condition, which ensures the left outter join 'exists' for it
@@ -168,7 +168,7 @@ module Praxis
           # NOTE: we don't need to do it for conditions applying to the root of the tree (there isn't a join to it)
           if association_key_column
             qr = quote_right_part(query: query, value: nil, column_object: association_key_column, negative: true)
-            query = query.where("#{quote_column_path(query: query, prefix: column_prefix, column_object: association_key_column)} #{qr}")
+            query = query.where("#{quote_column_path(query: query, prefix: column_prefix, column_name: association_key_column.name)} #{qr}")
           end
 
           case op
@@ -177,14 +177,14 @@ module Praxis
               add_safe_where(query: query, tab: column_prefix, col: column_object, op: 'LIKE', value: likeval)
             else
               quoted_right = quote_right_part(query: query, value: value, column_object: column_object, negative: false)
-              query.where("#{quote_column_path(query: query, prefix: column_prefix, column_object: column_object)} #{quoted_right}")
+              query.where("#{quote_column_path(query: query, prefix: column_prefix, column_name: column_object.name)} #{quoted_right}")
             end
           when '!='
             if likeval
               add_safe_where(query: query, tab: column_prefix, col: column_object, op: 'NOT LIKE', value: likeval)
             else
               quoted_right = quote_right_part(query: query, value: value, column_object: column_object, negative: true)
-              query.where("#{quote_column_path(query: query, prefix: column_prefix, column_object: column_object)} #{quoted_right}")
+              query.where("#{quote_column_path(query: query, prefix: column_prefix, column_name: column_object.name)} #{quoted_right}")
             end
           when '>'
             add_safe_where(query: query, tab: column_prefix, col: column_object, op: '>', value: value)
@@ -203,13 +203,13 @@ module Praxis
         # rubocop:disable Naming/MethodParameterName
         def self.add_safe_where(query:, tab:, col:, op:, value:)
           quoted_value = query.connection.quote_default_expression(value, col)
-          query.where("#{quote_column_path(query: query, prefix: tab, column_object: col)} #{op} #{quoted_value}")
+          query.where("#{quote_column_path(query: query, prefix: tab, column_name: col.name)} #{op} #{quoted_value}")
         end
         # rubocop:enable Naming/MethodParameterName
 
-        def self.quote_column_path(query:, prefix:, column_object:)
+        def self.quote_column_path(query:, prefix:, column_name:)
           c = query.connection
-          quoted_column = c.quote_column_name(column_object.name)
+          quoted_column = c.quote_column_name(column_name)
           if prefix
             quoted_table = c.quote_table_name(prefix)
             "#{quoted_table}.#{quoted_column}"
@@ -337,7 +337,7 @@ module Praxis
             if ref && ['!', '!!'].include?(condition[:op])
               cp = (nodetree.path + [condition[:name].to_s]).join(REFERENCES_STRING_SEPARATOR)
               conditions += [condition.merge(column_prefix: cp, model: model, parent_reflection: ref)]
-              h[condition[:name]] = {}
+              h[condition[:name].to_s] = {}
             else
               # Save the parent reflection where the condition applies as well (used later to get assoc keys)
               conditions += [condition.merge(column_prefix: column_prefix, model: model, parent_reflection: parent_reflection)]
@@ -350,14 +350,14 @@ module Praxis
         maj, min, = ActiveRecord.gem_version.segments
         if maj == 5 || (maj == 6 && min.zero?)
           # In AR 6 (and 6.0) the references are simple strings
-          def build_reference_value(column_prefix, **_args)
+          def self.build_reference_value(column_prefix, **_args)
             column_prefix
           end
         else
           # The latest AR versions discard passing references to joins when they're not SqlLiterals ... so let's wrap it
           # with our class, so that it is a literal (already quoted), but that can still provide the expected "symbol" without quotes
           # so that our aliasing code can match it.
-          def build_reference_value(column_prefix, query:)
+          def self.build_reference_value(column_prefix, query:)
             QuasiSqlLiteral.new(quoted: query.connection.quote_table_name(column_prefix), symbolized: column_prefix.to_sym)
           end
         end

data/lib/praxis/extensions/attribute_filtering/filter_tree_node.rb

@@ -7,7 +7,6 @@ module Praxis
         attr_reader :path, :conditions, :children
 
         # Parsed_filters is an Array of {name: X, op: Y, value: Z} ... exactly the format of the FilteringParams.load method
-        # It can also contain a :node_object
         def initialize(parsed_filters, path: [])
           @path = path # Array that marks the tree 'path' to this node (with respect to the absolute root)
           @conditions = [] # Conditions to apply directly to this node

data/lib/praxis/extensions/attribute_filtering/filters_parser.rb

@@ -63,7 +63,7 @@ module Praxis
             newval, fuzzy = if starting && ending
                               [raw_val[1..-2], :start_end]
                             elsif starting
-                              [raw_val[1..-1], :start]
+                              [raw_val[1..], :start]
                             elsif ending
                               [raw_val[0..-2], :end]
                             else

data/lib/praxis/extensions/pagination/active_record_pagination_handler.rb

@@ -15,14 +15,36 @@ module Praxis
           query.where(query.table[attr].gt(value))
         end
 
-        def self.order(query, order)
+        def self.order(query, order, root_resource:)
           return query unless order
 
           query = query.reorder('')
-
           order.each do |spec_hash|
-            direction, name = spec_hash.first
-            query = query.order(name => direction)
+            direction, string = spec_hash.first
+            info = association_info_for(root_resource, string.to_s.split('.'))
+
+            # Convert the includes path (it is not a tree), to a column prefix
+            pointer = info[:includes]
+
+            dotted = []
+            loop do
+              break if pointer.empty?
+
+              key, pointer = pointer.first
+              dotted.push(key)
+            end
+            column_prefix = dotted.empty? ? root_resource.model.table_name : ([''] + dotted).join(AttributeFiltering::ActiveRecordFilterQueryBuilder::REFERENCES_STRING_SEPARATOR)
+
+            # If the sorting refers to a deeper association, make sure to add the join and the special reference
+            if column_prefix
+              refval = AttributeFiltering::ActiveRecordFilterQueryBuilder.build_reference_value(column_prefix, query: query)
+              # Outter join hash needs to be a string based hash format!! (if it's in symbols, it won't match it and we'll cause extra joins)
+              query = query.left_outer_joins(info[:includes]).references(refval)
+            end
+
+            quoted_prefix = AttributeFiltering::ActiveRecordFilterQueryBuilder.quote_column_path(query: query, prefix: column_prefix, column_name: info[:attribute])
+            order_clause = Arel.sql(ActiveRecord::Base.sanitize_sql_array("#{quoted_prefix} #{direction}"))
+            query = query.order(order_clause)
           end
           query
         end
@@ -38,6 +60,34 @@ module Praxis
         def self.limit(query, limit)
           query.limit(limit)
         end
+
+        # Based off of a root resource and an incoming path of dot-separated attributes...
+        # find the leaf attribute and its associated resource (including mapping names of associations/attributes along the way)
+        # as defined by the `order_mapping` stanzas of resources:
+        # resource: final resource the attribute is associated with
+        # includes: a hash in the shape of AR includes, where keys are strings (that is very important)
+        # column_name: final attribute name where this path leads to. Nil if the path ends at an association
+        def self.association_info_for(resource, path)
+          main, *rest = path
+          mapped_name = resource.order_mapping[main.to_sym] || main
+
+          if (association = resource.model.reflections[mapped_name])
+            related_resource = resource.model_map[association.klass]
+            if rest.presence
+              result = association_info_for(related_resource, rest)
+              { resource: result[:resource], includes: { mapped_name => result[:includes] }, attribute: result[:attribute] }
+            else # Ends with an association (i.e., for ! or !! attributes)
+              { resource: related_resource, includes: { mapped_name => {} }, attribute: nil }
+            end
+          elsif !rest.presence
+            # Since it is not an association, must be a column name
+            { resource: resource, includes: {}, attribute: mapped_name }
+          else
+            # Could not find an association and there are more components to cover...something's not right
+            raise 'Error trying to map ordering components to the order_mapping of the resources. ' \
+                  "Could not find a mapping for property: #{mapped_name} in resource #{resource.name}. Did you forget to add a mapping for it in the `order_mapping` stanza ?"
+          end
+        end
       end
     end
   end

data/lib/praxis/extensions/pagination/ordering_params.rb

@@ -25,12 +25,15 @@ module Praxis
         class DSLCompiler < Attributor::DSLCompiler
           def by_fields(*fields)
             requested = fields.map(&:to_sym)
-            non_matching = requested - target.media_type.attributes.keys
-            unless non_matching.empty?
-              raise "Error, you've requested to order by fields that do not exist in the mediatype!\n" \
-              "The following #{non_matching.size} field/s do not exist in media type #{target.media_type.name} :\n" +
-                    non_matching.join(',').to_s
+
+            errors = []
+            requested.each do |field|
+              if (failed_field = self.class.validate_field(target.media_type, field.to_s.split('.').map(&:to_sym)))
+                errors += ["Cannot order by field: '#{field}'. It seems that the '#{failed_field}' attribute is not defined in the current #{target.media_type} structure (or its subtree)."]
+              end
             end
+            raise errors.join('\n') unless errors.empty?
+
             target.fields_allowed = requested
           end
 
@@ -44,6 +47,17 @@ module Praxis
               raise "Error: unknown parameter for the 'enforce_for' : #{which}. Only :all or :first are allowed"
             end
           end
+
+          def self.validate_field(type, path)
+            main, rest = path
+            next_attribute = type.respond_to?(:member_attribute) ? type.member_type.attributes[main] : type.attributes[main]
+
+            return main unless next_attribute
+
+            return nil if rest.nil?
+
+            validate_field(next_attribute.type, rest)
+          end
         end
 
         # Configurable DEFAULTS
@@ -154,9 +168,9 @@ module Praxis
             parsed_order = order.split(',').each_with_object([]) do |order_string, arr|
               item = case order_string[0]
                      when '-'
-                       { desc: order_string[1..-1].to_s }
+                       { desc: order_string[1..].to_s }
                      when '+'
-                       { asc: order_string[1..-1].to_s }
+                       { asc: order_string[1..].to_s }
                      else
                        { asc: order_string.to_s }
                      end
@@ -199,11 +213,12 @@ module Praxis
               field = field.to_sym
               next if self.class.fields_allowed.include?(field)
 
-              errors << if self.class.media_type.attributes.key?(field)
-                          "Ordering by field \'#{field}\' is disallowed. Ordering is only allowed using the following fields: " +
+              field_path = field.to_s.split('.').map(&:to_sym)
+              errors << if valid_attribute_path?(self.class.media_type, field_path)
+                          "Ordering by field \'#{field}\' in media type #{self.class.media_type.name} is disallowed. Ordering is only allowed using the following fields: " +
                           self.class.fields_allowed.map { |f| "\'#{f}\'" }.join(', ').to_s
                         else
-                          "Ordering by field \'#{field}\' is not possible as this field does not exist in " \
+                          "Ordering by field \'#{field}\' is not possible as this field is not reachable from " \
                           "media type #{self.class.media_type.name}"
                         end
             end
@@ -226,6 +241,19 @@ module Praxis
         def each(&block)
           items.each(&block)
         end
+
+        # Looks up if the given path (with symbol attribute names at each component) is actually
+        # a valid path from the given mediatype
+        def valid_attribute_path?(media_type, path)
+          first, *rest = path
+          # Get the member type if this is a collection
+          media_type = media_type.member_type if media_type.respond_to?(:member_attribute)
+          if (attribute = media_type.attributes[first])
+            rest.empty? ? true : valid_attribute_path?(attribute.type, rest)
+          else
+            false
+          end
+        end
       end
     end
   end

data/lib/praxis/extensions/pagination/pagination_handler.rb

@@ -6,7 +6,7 @@ module Praxis
       class PaginationHandler
         class PaginationException < RuntimeError; end
 
-        def self.paginate(query, pagination)
+        def self.paginate(query, pagination, root_resource:)
           return query unless pagination.paginator
 
           paginator = pagination.paginator
@@ -18,7 +18,7 @@ module Praxis
                 # i.e., We can be smart about allowing the main sort field matching the pagination one (in case you want to sub-order in a custom way)
                 oclause = if pagination.order.nil? || pagination.order.empty? # No ordering specified => use ascending based on the "by" field
                             direction = :asc
-                            order(query, [{ asc: paginator.by }])
+                            order(query, [{ asc: paginator.by }], root_resource: root_resource)
                           else
                             first_ordering = pagination.order.items.first
                             direction = first_ordering.keys.first
@@ -32,7 +32,7 @@ module Praxis
                                     "When paginating by a field value, one cannot specify the 'order' clause " \
                                     "unless the clause's primary field matches the pagination field."
                             end
-                            order(query, pagination.order)
+                            order(query, pagination.order, root_resource: root_resource)
                           end
 
                 if paginator.from

data/lib/praxis/extensions/pagination/sequel_pagination_handler.rb

@@ -14,7 +14,7 @@ module Praxis
           query.where("#{attr} > ?", value)
         end
 
-        def self.order(query, order)
+        def self.order(query, order, _root_resource: nil)
           return query unless order
 
           order_clause = order.map do |spec_hash|