stretchy-model 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a85f7bffcc11c54366d4cae7fec47b91341425e06b7f1c8a6269e0b23d4e649b
-  data.tar.gz: cd0b9314fa734aa7af7179f12ed00626a17bc869186f82088158bfca87c8d915
+  metadata.gz: dd18b424c18bda352233d72af113e9bbc944277e03a02cfb950780f0fab9a405
+  data.tar.gz: 6eb86522e7bc91012cc4c743d0a9fccc54c7e75b14ff4d6f4fca5bdb9fa31626
 SHA512:
-  metadata.gz: e193c837695100177eac9888841b14243596dfc9694515696ab1a9934174287c821cf52b2ca165e3f3eda04de9ba16c7c7330cb187641d71229d322c06674977
-  data.tar.gz: 21ace7b64f85d505a05b61b89b5b85d9d2af34941b57672da45f70e6c8736e8b9451c59891fef7d0795f8b54076cffd88bfc141fcd4da1c1f8ffaddcf8276697
+  metadata.gz: 0fd2a0fb25439d79c799f43f43bb01df7c4dfe0ce9eb577d445b6d415eb9c38ecdc0f45c73d8255d2e079ff502e943f6491b874e3bb9c41fc9c853efed84d252
+  data.tar.gz: 8ea4cfa584ec825ac30aa4aa5dea88b04c709cbf7cc75089e28cd1e850318f150f1f68ead89bc571535739aa8ff8db777102d5dd05173087d4af499bac8d8bbb

data/.rspec

@@ -1 +1 @@
---require spec_helper
+--require spec_helper

data/README.md

@@ -1,6 +1,5 @@
 stretchy-model
 ===
-
 <p>
     <a href="https://stretchy.io/" target="_blank"><img src="./stretchy.logo.png" alt="Gum Image" width="450" /></a>
     <br><br>
@@ -9,114 +8,42 @@ stretchy-model
 
 </p>
 
-Stretchy provides Elasticsearch models in a Rails environment with an integrated ActiveRecord-like interface and features. 
 
 ## Features
 Stretchy simplifies the process of querying, aggregating, and managing Elasticsearch-backed models, allowing Rails developers to work with search indices as comfortably as they would with traditional Rails models.
 
-## Attributes
-
-```ruby
-class Post < Stretchy::Record
-
-    attribute :title,                   :string
-    attribute :body,                    :string
-    attribute :flagged,                 :boolean,  default: false  
-    attribute :author,                   :hash 
-    attribute :tags,                    :array, default: []
-
-end
-```
->[!NOTE]
->`created_at`, `:updated_at` and `:id` are automatically added to all `Stretchy::Records`
-
-
-## Query
-```ruby
-  Post.where('author.name': "Jadzia", flagged: true).first
-  #=> <Post id: aW02w3092, title: "Fun Cats", body: "...", flagged: true,
-  #         author: {name: "Jadzia", age: 20}, tags: ["cat", "amusing"]>
-```
-
-## Aggregations
-```ruby
-
-  result = Post.filter(:range, 'author.age': {gte: 18})
-    .aggregation(:post_frequency, date_histogram: {
-      field: :created_at,
-      calender_interval: :month
-    })
+* Model fully back by Elasticsearch/Opensearch
+* Chain queries, scopes and aggregations
+* Reduce Elasticsearch query complexity
+* Support for time-based indices and aliases
+* Associations to both ActiveRecord models and Stretchy::Record
+* Bulk Operations made easy
+* Validations, custom attributes, and more...
 
-  result.aggregations.post_frequency
-  #=> {buckets: [{key_as_string: "2024-01-01", doc_count: 20}, ...]}
-```
-
-## Scopes
-
-```ruby
-class Post < Stretchy::Record
-  # ...attributes
-
-  # Scopes
-  scope :flagged, -> { where(flagged: true) }
-  scope :top_links, lambda do |size=10, url=".com"| 
-    aggregation(:links, 
-      terms: {
-        field: :links, 
-        size: size, 
-        include: ".*#{url}.*"
-      })
-  end
-end
-
-# Returns flagged posts and includes the top 10 'youtube.com' 
-# links in results.aggregations.links
-result = Post.flagged.top_links(10, "youtube.com")
-
-```
+Follow the guides to learn more about:
 
-## Bulk Operations
-
-
-```ruby
-Model.bulk(records_as_bulk_operations)
-```
-
-#### Bulk helper
-Generates structure for the bulk operation
-```ruby
-record.to_bulk # default to_bulk(:index)
-record.to_bulk(:delete)
-record.to_bulk(:update)
-```
+* [Models](https://theablefew.github.io/stretchy/#/guides/models?id=models)
+* [Querying](https://theablefew.github.io/stretchy/#/guides/querying?id=querying)
+* [Aggregations](https://theablefew.github.io/stretchy/#/guides/aggregations?id=aggregations)
+* [Scopes](https://theablefew.github.io/stretchy/#/guides/scopes?id=scopes)
 
-#### In batches
-Run bulk operations in batches specified by `size`
-```ruby
-Model.bulk_in_batches(records, size: 100) do |batch|
-    batch.map! { |record| Model.new(record).to_bulk }
-end
-```
 
+[Read the Documentation](https://theablefew.github.io/stretchy/#/) or walk through of a simple [Data Analysis](https://theablefew.github.io/stretchy/#/examples/data_analysis?id=data-analysis) example.
 
-## Instrumentation
-```ruby
-Blanket.first
-```
 
-```sh
-Blanket (6.322ms) curl -X GET 'http://localhost:9200/blankets/_search?size=1' -d '{"sort":{"date":"desc"}}'
-```
 
 ## Installation
 
 Install the gem and add to the application's Gemfile by executing:
 
-    $ bundle add stretchy-model
+```sh
+ bundle add stretchy-model
+```
 
 If bundler is not being used to manage dependencies, install the gem by executing:
-
-    $ gem install stretchy-model
+```sh
+  gem install stretchy-model
+```
 
 <details>
 <summary>Rails Configuration</summary>
@@ -131,6 +58,12 @@ rails credentials:edit
 ```yaml
 elasticsearch:
    url: localhost:9200
+
+# or opensearch
+# opensearch:
+#    host: https://localhost:9200
+#    user: admin
+#    password: admin
 ```
 
 #### Create an initializer 
@@ -149,11 +82,24 @@ end
 After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 >[!TIP]
->This library is built on top of the excellent [elasticsearch-persistence](https://github.com/elastic/elasticsearch-rails/tree/main/elasticsearch-persistence) gem. 
->
 > Full documentation on [Elasticsearch Query DSL and Aggregation options](https://github.com/elastic/elasticsearch-rails/tree/main/elasticsearch-persistence)
 
 ## Testing
+<details>
+<summary>Act</summary>
+
+Run github action workflow locally
+
+```sh
+brew install act --HEAD
+```
+
+```sh
+act -P ubuntu-latest=ghcr.io/catthehacker/ubuntu:runner-latest
+```
+
+</details>
+
 <details>
 <summary>Elasticsearch</summary>
 

data/Rakefile

@@ -2,3 +2,95 @@
 
 require "bundler/gem_tasks"
 task default: %i[]
+
+require 'octokit' 
+require 'versionomy'
+require 'rainbow'
+
+def determine_current_version
+  # Load current version
+  load 'lib/stretchy/version.rb'
+  current_version = Versionomy.parse(Stretchy::VERSION)
+end
+
+def determine_new_version(version)
+  # Load current version
+  current_version = determine_current_version
+
+  # Determine new version
+  case version.to_sym
+  when :major
+    current_version.bump(:major)
+  when :minor
+    current_version.bump(:minor)
+  when :patch
+    current_version.bump(:tiny)
+  else
+    version =~ /\Av?\d+\.\d+\.\d+\z/ ? Versionomy.parse(version).to_s.gsub(/v/,'') : current_version
+  end
+end
+
+def create_release_branch(new_version, base_branch)
+  system("git stash save 'Changes before creating release branch'")
+  system("git fetch origin #{base_branch}")
+  branch_name = "release/v#{new_version}"
+  system("git checkout -b #{branch_name} #{base_branch}")
+  branch_name
+end
+
+def update_version_file(new_version)
+  # Update lib/stretchy/version.rb
+  File.open('lib/stretchy/version.rb', 'w') do |file|
+    file.puts "module Stretchy\n  VERSION = '#{new_version}'\nend"
+  end
+end
+
+def commit_and_push_changes(new_version, branch_name)
+  system("git add lib/stretchy/version.rb")
+  system("git commit -m 'Bump version to v#{new_version}'")
+  system("git tag v#{new_version}")
+  system("git push origin #{branch_name} --tags -f")
+end
+
+def create_pull_request(new_version, base_branch, branch_name)
+  # Create a pull request
+  client = Octokit::Client.new(access_token: ENV['GH_TOKEN'])
+  client.create_pull_request('theablefew/stretchy', base_branch, branch_name, "Release v#{new_version}")
+end
+
+namespace :publish do
+  desc "Create a release"
+  task :release, [:version, :base_branch] do |t, args|
+    args.with_defaults(version: :patch, base_branch: 'main')
+    version = args[:version]
+    base_branch = args[:base_branch]
+  
+    old_version = determine_current_version
+    new_version = determine_new_version(version)
+    puts Rainbow("Bumping version from #{old_version} to #{new_version}").green
+    branch_name = create_release_branch(new_version, base_branch)
+    begin
+    update_version_file(new_version)
+    commit_and_push_changes(new_version, branch_name)
+    create_pull_request(new_version, base_branch, branch_name)
+    rescue => e
+      puts "Error: #{e.message}"
+      puts "Rolling back changes"
+      system("git tag -d v#{new_version}")
+      system("git checkout #{base_branch}")
+      system("git branch -D #{branch_name}")
+    end
+  end
+
+  task :major do
+    Rake::Task['publish:release'].invoke('major')
+  end
+
+  task :minor do
+    Rake::Task['publish:release'].invoke('minor')
+  end
+
+  task :patch do
+    Rake::Task['publish:release'].invoke('patch')
+  end
+end

data/lib/stretchy/associations.rb

@@ -3,7 +3,11 @@ module Stretchy
     extend ActiveSupport::Concern
 
     def save!
+      if valid?
         self.save
+      else
+        raise "Record is invalid"
+      end
     end
 
     # Required for Elasticsearch < 7
@@ -34,7 +38,7 @@ module Stretchy
     end
 
     def association_reflection(association)
-      ElasticRelation.new @@_associations[association], (dirty[association.to_sym] || [])
+      Stretchy::Relation.new @@_associations[association], (dirty[association.to_sym] || [])
     end
 
     def _destroy=(bool)
@@ -48,6 +52,7 @@ module Stretchy
     def save_associations
       @_after_save_objects.each_pair do |association, collection|
         collection.each do |instance|
+          # TODO: bulk update 
           instance.send("#{@@_association_options[association.to_sym][:foreign_key]}=", self.id)
           instance.save
         end
@@ -59,59 +64,194 @@ module Stretchy
       @@_associations ||= {}
       @@_association_options ||= {}
 
+      # The belongs_to method is used to set up a one-to-one connection with another model.
+      # This indicates that this model has exactly one instance of another model.
+      # For example, if your application includes authors and books, and each book can be assigned exactly one author,
+      # you'd declare the book model to belong to the author model.
+      #
+      # association:: [Symbol] the name of the association
+      # options:: [Hash] a hash to set up options for the association
+      #           :foreign_key - the foreign key used for the association. Defaults to "#{association}_id"
+      #           :primary_key - the primary key used for the association. Defaults to "id"
+      #           :class_name - the name of the associated object's class. Defaults to the name of the association
+      #
+      # Example:
+      #   belongs_to :author
+      #
+      # This creates a book.author method that returns the author of the book.
+      # It also creates an author= method that allows you to assign the author of the book.
+      #
       def belongs_to(association, options = {})
         @@_association_options[association] = {
           foreign_key: "#{association}_id", 
           primary_key: "id",
           class_name: association
-        }.reverse_merge(options)
+        }.merge(options)
 
         klass = @@_association_options[association][:class_name].to_s.singularize.classify.constantize
         @@_associations[association] = klass
 
         define_method(association.to_sym) do
-          klass.where(_id: self.send(@@_association_options[association][:foreign_key].to_sym)).first
+          instance_variable_get("@#{association}") || 
+            klass.where(_id: self.send(@@_association_options[association][:foreign_key].to_sym)).first
         end
 
         define_method("#{association}=".to_sym) do |val|
           options = @@_association_options[association] 
-          instance_variable_set("@#{options[:foreign_key]}", val.send(options[:primary_key]))
+          self.send("#{options[:foreign_key]}=", val.send(options[:primary_key]))
+          instance_variable_set("@#{association}", val)
+        end
+
+        define_method("build_#{association}") do |*args|
+          associated_object = klass.new(*args)
+          instance_variable_set("@#{association}", associated_object)
+          associated_object
+        end
+
+        before_save do
+          associated_object = instance_variable_get("@#{association}")
+          if associated_object && associated_object.new_record?
+            if associated_object.save!
+              self.send("#{@@_association_options[association][:foreign_key]}=", associated_object.id)
+            end
+          end
         end
       end
 
-      def has_one(association, class_name: nil, foreign_key: nil, dependent: :destroy)
 
-        klass = association.to_s.singularize.classify.constantize unless class_name.present?
-        foreign_key = "#{self.name.downcase}_id" unless foreign_key.present?
+
+
+
+
+
+
+
+
+      # The has_one method is used to set up a one-to-one connection with another model.
+      # This indicates that this model contains the foreign key.
+      #
+      # association:: [Symbol] The name of the association.
+      # options:: [Hash] A hash to set up options for the association.
+      #           :class_name - The name of the associated model. If not provided, it's derived from +association+.
+      #           :foreign_key - The name of the foreign key on the associated model. If not provided, it's derived from the name of this model.
+      #           :dependent - If set to +:destroy+, the associated object will be destroyed when this object is destroyed. This is the default behavior.
+      #           :primary_key - The name of the primary key on the associated model. If not provided, it's assumed to be +id+.
+      #           
+      #
+      # Example:
+      #   has_one :profile
+      #
+      # This creates a user.profile method that returns the profile of the user.
+      # It also creates a profile= method that allows you to assign the profile of the user.
+      #
+      def has_one(association, options = {})
+
+        @@_association_options[association] = {
+          foreign_key: "#{self.name.underscore}_id", 
+          primary_key: "id",
+          class_name: association
+        }.merge(options)
+
+        klass = @@_association_options[association][:class_name].to_s.singularize.classify.constantize
         @@_associations[association] = klass
 
+        foreign_key = @@_association_options[association][:foreign_key]
+
         define_method(association.to_sym) do
-          klass.where("#{foreign_key}": self.id).first
+          instance_variable_get("@#{association}") || 
+            klass.where("#{foreign_key}": self.id).first
+        end
+
+        define_method("#{association}=".to_sym) do |val|
+          instance_variable_set("@#{association}", val)
+          save!
+        end
+
+        before_save do
+          associated_object = instance_variable_get("@#{association}")
+          if associated_object
+            associated_object.send("#{foreign_key}=", self.id)
+            associated_object.save!
+          end
         end
       end
 
-      def has_many(association, klass, options = {})
-        @@_associations[association] = klass
 
-        opt_fk = options.delete(:foreign_key)
-        foreign_key = opt_fk ? opt_fk : "#{self.name.split("::").last.tableize.singularize}_id"
 
-        @@_association_options[association] = { foreign_key: foreign_key }
+
+
+
+
+
+
+      # The has_many method is used to set up a one-to-many connection with another model.
+      # This indicates that this model can be matched with zero or more instances of another model.
+      # For example, if your application includes authors and books, and each author can have many books,
+      # you'd declare the author model to have many books.
+      #
+      # association:: [Symbol] the name of the association
+      # options:: [Hash] a hash to set up options for the association
+      #           :foreign_key - the foreign key used for the association. Defaults to "#{self.name.downcase}_id"
+      #           :primary_key - the primary key used for the association. Defaults to "id"
+      #           :class_name - the name of the associated object's class. Defaults to the name of the association
+      #           :dependent - if set to :destroy, the associated object will be destroyed when this object is destroyed. This is the default behavior.
+      #           
+      #
+      # Example:
+      #   has_many :books
+      #
+      # This creates an author.books method that returns a collection of books for the author.
+      # It also creates a books= method that allows you to assign the books for the author.
+      #
+      def has_many(association, options = {})
+        @@_association_options[association] = {
+          foreign_key: "#{self.name.underscore}_id", 
+          primary_key: "id",
+          class_name: association.to_s.singularize.to_sym
+        }.merge(options)
+
+        klass = @@_association_options[association][:class_name].to_s.classify.constantize
+        foreign_key = @@_association_options[association][:foreign_key]
+        primary_key = @@_association_options[association][:primary_key]
+        @@_associations[association] = klass
 
         define_method(association.to_sym) do
           args = {}
-          args[foreign_key] = self.id
+          args["_#{primary_key}"] = self.send("#{association.to_s.singularize}_ids")
           self.new_record? ? association_reflection(association) : klass.where(args)
         end
 
+        define_method("#{association.to_s.singularize}_ids") do 
+          instance_variable_get("@#{association.to_s.singularize}_ids".to_sym)
+        end
+
+        define_method("#{association.to_s.singularize}_ids=") do |val|
+          instance_variable_set("@#{association.to_s.singularize}_ids".to_sym, val)
+        end
+
+        define_method("#{association}=".to_sym) do |val|
+          val.each { |v| after_save_objects(v.attributes, association)}
+          self.send("#{association.to_s.singularize}_ids=", val.map(&:id))
+          dirty
+        end
+
         define_method("build_#{association}".to_sym) do |*args|
           opts = {}
           opts[foreign_key] = self.id
           args.first.merge! opts
           klass.new *args
         end
+
+        after_save do
+          save_associations
+        end
       end
 
+
+
+
+
+
       def validates_associated(*attr_names)
         validates_with AssociatedValidator, _merge_attributes(attr_names)
       end
@@ -131,7 +271,7 @@ module Stretchy
       end
 
       def reflect_on_association(association)
-        ElasticRelation.new @@_associations[association]
+        Stretchy::Relation.new @@_associations[association]
       end
 
       def update_all(records, **attributes)

data/lib/stretchy/attributes/transformers/keyword_transformer.rb

@@ -0,0 +1,85 @@
+module Stretchy
+  module Attributes
+    module Transformers
+      class KeywordTransformer
+
+          KEYWORD_AGGREGATION_KEYS = [:terms, :rare_terms, :significant_terms, :cardinality, :string_stats]
+
+          attr_reader :attribute_types
+
+          def initialize(attribute_types)
+            @attribute_types = attribute_types
+          end
+
+          def cast_value_keys
+            values.transform_values do |value|
+              case value
+              when Array
+                value.map { |item| transform_keys_for_item(item) }
+              when Hash
+                transform_keys_for_item(value)
+              else
+                value
+              end
+            end
+          end
+          
+          def keyword?(arg)
+            attr = @attribute_types[arg.to_s] 
+            return false unless attr
+            attr.is_a?(Stretchy::Attributes::Type::Keyword)
+          end
+
+          def protected?(arg)
+            return false if arg.nil?
+            Stretchy::Relations::AggregationMethods::AGGREGATION_METHODS.include?(arg.to_sym)
+          end
+          
+          def transform(item, *ignore) 
+            item.each_with_object({}) do |(k, v), new_item|
+              if ignore && ignore.include?(k)
+                new_item[k] = v
+                next
+              end
+              new_key = (!protected?(k) && keyword?(k)) ? "#{k}.keyword" : k
+
+              new_value = v
+          
+              if new_value.is_a?(Hash)
+                new_value = transform(new_value)
+              elsif new_value.is_a?(Array)
+                new_value = new_value.map { |i| i.is_a?(Hash) ? transform(i) : i }
+              elsif new_value.is_a?(String) || new_value.is_a?(Symbol) 
+                new_value = "#{new_value}.keyword" if keyword?(new_value)
+              end
+
+              new_item[new_key] = new_value
+            end
+          end
+
+          # If terms are used, we assume that the field is a keyword field
+          # and append .keyword to the field name
+          # {terms: {field: 'gender'}}
+          # or nested aggs
+          # {terms: {field: 'gender'}, aggs: {name: {terms: {field: 'position.name'}}}}
+          # should be converted to
+          # {terms: {field: 'gender.keyword'}, aggs: {name: {terms: {field: 'position.name.keyword'}}}}
+          # {date_histogram: {field: 'created_at', interval: 'day'}}
+          # TODO: There may be cases where we don't want to add .keyword to the field and there should be a way to override this
+          def assume_keyword_field(args={}, parent_match=false)
+            if args.is_a?(Hash)
+              args.each do |k, v|
+                if v.is_a?(Hash) 
+                  assume_keyword_field(v, KEYWORD_AGGREGATION_FIELDS.include?(k))
+                else
+                  next unless v.is_a?(String) || v.is_a?(Symbol)
+                  args[k] = ([:field, :fields].include?(k.to_sym) && v !~ /\.keyword$/ && parent_match) ? "#{v}.keyword" : v.to_s
+                end
+              end
+            end
+          end
+
+      end
+    end
+  end
+end

data/lib/stretchy/attributes/type/array.rb

@@ -0,0 +1,15 @@
+module Stretchy
+    module Attributes
+        module Type
+
+            class Array < ActiveModel::Type::Value # :nodoc:
+
+                def type
+                    :array
+                end
+
+            end
+
+        end
+    end
+end

data/lib/stretchy/attributes/type/hash.rb

@@ -0,0 +1,17 @@
+module Stretchy
+    module Attributes
+        module Type
+            class Hash < ActiveModel::Type::Value # :nodoc:
+                def type
+                    :hash
+                end
+
+                private
+
+                def cast_value(value)
+                    Elasticsearch::Model::HashWrapper[value]
+                end
+            end
+        end
+    end
+end

data/lib/stretchy/attributes/type/keyword.rb

@@ -0,0 +1,11 @@
+module Stretchy
+    module Attributes
+        module Type
+            class Keyword < ActiveModel::Type::String # :nodoc:
+                def type
+                    :keyword
+                end
+            end
+        end
+    end
+end

data/lib/stretchy/attributes/type/text.rb

@@ -0,0 +1,12 @@
+module Stretchy
+  module Attributes
+      module Type
+          # alias for ActiveModel::Type::String
+          class Text < ActiveModel::Type::String
+              def type
+                  :text
+              end
+          end
+      end
+  end
+end