solid_cache 1.0.5 → 1.0.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9a27107ddce2c64a1884881e7cd3385533c32c1c13e2eee486a8c856de233c00
-  data.tar.gz: a1eb035fd95c8cadfbcce86603ed4ec2dcc0a133c9516101baf0dca94f1baf7d
+  metadata.gz: 6c96752c871ba3eb453aca3f4c838b76afa0393486787109ef6e2f1ce1266a3d
+  data.tar.gz: 57cf865289e1c289ceed0b0103f234b61f5283c4a693ba213d60e4e81d21f9f2
 SHA512:
-  metadata.gz: 8d0633c3f33b02df9f18e0f77b1ca6b200333bd6961d74af4462311a2ede3f264f6bf6f845da48ed187049affa9984f9aa43cfa39529ddaf192715c3940445bc
-  data.tar.gz: 819e979a65563ec3ce5e16c09b8800a80e5764b63938f1e7e2b64d38ea453a06dec8f6ba63c8589cd4bbff05632b65d47c595de5c949997ef18f47305435cfd5
+  metadata.gz: be0dad91c5e93e775c0f7f528106b0d8ca69709d3d6d1fe4d95e834249abd267b86c5828d857e173fa1357da967362d290079df47f27bbcbcf56c3b7e9c80683
+  data.tar.gz: ebc91d895a6721f1a8acf2ea4ee611bc8619aa3a432be492f9a934c7eae5796c6539cba51ebba9396245b5a09ca197cfbe6c8abd9c10fda60957c0f477987d51

data/README.md

@@ -1,6 +1,6 @@
 # Solid Cache
 
-Solid Cache is a database-backed Active Support cache store that let's you keep a much larger cache than is typically possible with traditional memory-only Redis or Memcached stores. This is thanks to the speed of modern SSD drives, which make the access-time penalty of using disk vs RAM insignificant for most caching purposes. Simply put, you're now usually better off keeping a huge cache on disk rather than a small cache in memory.
+Solid Cache is a database-backed Active Support cache store that lets you keep a much larger cache than is typically possible with traditional memory-only Redis or Memcached stores. This is thanks to the speed of modern SSD drives, which make the access-time penalty of using disk vs RAM insignificant for most caching purposes. Simply put, you're now usually better off keeping a huge cache on disk rather than a small cache in memory.
 
 ## Installation
 
@@ -9,9 +9,9 @@ Solid Cache is configured by default in new Rails 8 applications. But if you're
 1. `bundle add solid_cache`
 2. `bin/rails solid_cache:install`
 
-This will configure Solid Cache as the production cache store, create `config/solid_cache.yml`, create `db/cache_schema.rb`, and attempt to alter `config/database.yml` to include the new cache database.
+This will configure Solid Cache as the production cache store, create `config/cache.yml`, and create `db/cache_schema.rb`.
 
-If you've already made material changes to your `config/database.yml` file, the installer may not be able to add the cache db configuration directly. If you need to do it yourself, this is what it should look like:
+You will then have to add the configuration for the cache database in `config/database.yml`. If you're using sqlite, it'll look like this:
 
 ```yaml
 production:
@@ -43,7 +43,7 @@ Then run `db:prepare` in production to ensure the database is created and the sc
 
 ## Configuration
 
-Configuration will be read from `config/solid_cache.yml`. You can change the location of the config file by setting the `SOLID_CACHE_CONFIG` env variable.
+Configuration will be read from `config/cache.yml` or `config/solid_cache.yml`. You can change the location of the config file by setting the `SOLID_CACHE_CONFIG` env variable.
 
 The format of the file is:
 
@@ -69,6 +69,13 @@ production: &production
 
 For the full list of keys for `store_options` see [Cache configuration](#cache-configuration). Any options passed to the cache lookup will overwrite those specified here.
 
+After running `solid_cache:install`, `environments/production.rb` will replace your cache store with Solid Cache, but you can also do this manually:
+
+```ruby
+# config/environments/production.rb
+config.cache_store = :solid_cache_store
+```
+
 ### Connection configuration
 
 You can set one of `database`, `databases` and `connects_to` in the config file. They will be used to configure the cache databases in `SolidCache::Record#connects_to`.
@@ -91,7 +98,7 @@ If none of these are set, Solid Cache will use the `ActiveRecord::Base` connecti
 
 ### Engine configuration
 
-There are three options that can be set on the engine:
+There are five options that can be set on the engine:
 
 - `executor` - the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap asynchronous operations, defaults to the app executor
 - `connects_to` - a custom connects to value for the abstract `SolidCache::Record` active record model. Required for sharding and/or using a separate cache database to the main app. This will overwrite any value set in `config/solid_cache.yml`
@@ -169,17 +176,17 @@ production:
 ```
 
 ```yaml
-# config/solid_cache.yml
+# config/cache.yml
 production:
   databases: [cache_shard1, cache_shard2, cache_shard3]
 ```
 
 ## Enabling encryption
 
-To encrypt the cache values, you can add set the encrypt property.
+To encrypt the cache values, you can add the encrypt property.
 
 ```yaml
-# config/solid_cache.yml
+# config/cache.yml
 production:
   encrypt: true
 ```
@@ -189,7 +196,7 @@ or
 config.solid_cache.encrypt = true
 ```
 
-You will need to set up your application to (use Active Record Encryption)[https://guides.rubyonrails.org/active_record_encryption.html].
+You will need to set up your application to [use Active Record Encryption](https://guides.rubyonrails.org/active_record_encryption.html).
 
 Solid Cache by default uses a custom encryptor and message serializer that are optimised for it.
 
@@ -260,7 +267,7 @@ This ensures that all the Rails versions dependencies are updated.
 
 ## Implementation
 
-Solid Cache is a FIFO (first in, first out) cache. While this is not as efficient as an LRU cache, it is mitigated by the longer cache lifespan.
+Solid Cache is a FIFO (first in, first out) cache. While this is not as efficient as an LRU (least recently used) cache, it is mitigated by the longer cache lifespan.
 
 A FIFO cache is much easier to manage:
 1. We don't need to track when items are read.

data/Rakefile

@@ -44,7 +44,7 @@ configs.each do |config|
       if config == :default
         sh("bin/rails test")
       else
-        sh("SOLID_CACHE_CONFIG=config/solid_cache_#{config}.yml bin/rails test")
+        sh("SOLID_CACHE_CONFIG=config/cache_#{config}.yml bin/rails test")
       end
     end
   end

data/app/models/solid_cache/entry/expiration.rb

@@ -40,6 +40,8 @@ module SolidCache
                 candidates.pluck(:id)
               else
                 min_created_at = max_age.seconds.ago
+                # We don't have an index on created_at, but we can select
+                # the records by id and they'll be in created_at order.
                 candidates.pluck(:id, :created_at)
                           .filter_map { |id, created_at| id if created_at < min_created_at }
               end

data/app/models/solid_cache/entry.rb

@@ -14,6 +14,8 @@ module SolidCache
 
     KEY_HASH_ID_RANGE = -(2**63)..(2**63 - 1)
 
+    MULTI_BATCH_SIZE = 1000
+
     class << self
       def write(key, value)
         write_multi([ { key: key, value: value } ])
@@ -21,9 +23,11 @@ module SolidCache
 
       def write_multi(payloads)
         without_query_cache do
-          upsert_all \
-            add_key_hash_and_byte_size(payloads),
-            unique_by: upsert_unique_by, on_duplicate: :update, update_only: [ :key, :value, :byte_size ]
+          payloads.each_slice(MULTI_BATCH_SIZE).each do |payload_batch|
+            upsert_all \
+              add_key_hash_and_byte_size(payload_batch),
+              unique_by: upsert_unique_by, on_duplicate: :update, update_only: [ :key, :value, :byte_size ]
+          end
         end
       end
 
@@ -33,9 +37,13 @@ module SolidCache
 
       def read_multi(keys)
         without_query_cache do
-          query = Arel.sql(select_sql(keys), *key_hashes_for(keys))
+          {}.tap do |results|
+            keys.each_slice(MULTI_BATCH_SIZE).each do |keys_batch|
+              query = Arel.sql(select_sql(keys_batch), *key_hashes_for(keys_batch))
 
-          connection.select_all(query, "SolidCache::Entry Load").cast_values(attribute_types).to_h
+              results.merge!(connection.select_all(query, "SolidCache::Entry Load").cast_values(attribute_types).to_h)
+            end
+          end
         end
       end
 

data/app/models/solid_cache/record.rb

@@ -10,7 +10,20 @@ module SolidCache
 
     class << self
       def disable_instrumentation(&block)
-        connection.with_instrumenter(NULL_INSTRUMENTER, &block)
+        with_instrumenter(NULL_INSTRUMENTER, &block)
+      end
+
+      def with_instrumenter(instrumenter, &block)
+        if connection.respond_to?(:with_instrumenter)
+          connection.with_instrumenter(instrumenter, &block)
+        else
+          begin
+            old_instrumenter, ActiveSupport::IsolatedExecutionState[:active_record_instrumenter] = ActiveSupport::IsolatedExecutionState[:active_record_instrumenter], instrumenter
+            block.call
+          ensure
+            ActiveSupport::IsolatedExecutionState[:active_record_instrumenter] = old_instrumenter
+          end
+        end
       end
 
       def with_shard(shard, &block)

data/lib/generators/solid_cache/install/install_generator.rb

@@ -3,70 +3,13 @@
 class SolidCache::InstallGenerator < Rails::Generators::Base
   source_root File.expand_path("templates", __dir__)
 
-  def add_rails_cache
-    gsub_file app_root.join("config/environments/production.rb"),
-      /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
-  end
-
-  def create_config_solid_cache_yml
-    template "config/solid_cache.yml"
-  end
-
-  def add_cache_db_to_database_yml
-    if app_is_using_sqlite?
-      gsub_file database_yml, /production:\s*<<: \*default.*/m, sqlite_database_config_with_cache
-    else
-      gsub_file database_yml, /production:\s*<<: \*default.*/m, generic_database_config_with_cache
-    end
-  end
-
-  def add_solid_cache_db_schema
+  def copy_files
+    template "config/cache.yml"
     template "db/cache_schema.rb"
   end
 
-  private
-    def app_root
-      Pathname.new(destination_root)
-    end
-
-    def database_yml
-      app_root.join("config/database.yml")
-    end
-
-    def app_is_using_sqlite?
-      database_yml.read.match?(/production:.*sqlite3/m)
-    end
-
-    def sqlite_database_config_with_cache
-      <<~YAML
-        production:
-          primary:
-            <<: *default
-            database: storage/production.sqlite3
-          cache:
-            <<: *default
-            database: storage/production_cache.sqlite3
-            migrations_paths: db/cache_migrate
-      YAML
-    end
-
-    def app_name_from_production_database_name
-      database_yml.read.scan(/database: (\w+)_production/).flatten.first
-    end
-
-    def generic_database_config_with_cache
-      app_name = app_name_from_production_database_name
-
-      <<~YAML
-production:
-  primary: &production_primary
-    <<: *default
-    database: #{app_name}_production
-    username: #{app_name}
-    password: <%= ENV["#{app_name.upcase}_DATABASE_PASSWORD"] %>
-  cache:
-    <<: *production_primary
-    database: #{app_name}_production_cache
-      YAML
-    end
+  def configure_cache_store_adapter
+    gsub_file Pathname.new(destination_root).join("config/environments/production.rb"),
+      /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
+  end
 end

data/lib/generators/solid_cache/install/templates/db/cache_schema.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-ActiveRecord::Schema[7.1].define(version: 1) do
+ActiveRecord::Schema[7.2].define(version: 1) do
   create_table "solid_cache_entries", force: :cascade do |t|
     t.binary "key", limit: 1024, null: false
     t.binary "value", limit: 536870912, null: false

data/lib/solid_cache/configuration.rb

@@ -35,7 +35,7 @@ module SolidCache
         @connects_to =
           case
           when database
-            { database: { writing: database.to_sym } }
+            { shards: { database.to_sym => { writing: database.to_sym } } }
           when databases
             { shards: databases.map(&:to_sym).index_with { |database| { writing: database } } }
           when connects_to

data/lib/solid_cache/engine.rb

@@ -10,13 +10,16 @@ module SolidCache
     config.solid_cache = ActiveSupport::OrderedOptions.new
 
     initializer "solid_cache.config", before: :initialize_cache do |app|
-      app.paths.add "config/solid_cache", with: ENV["SOLID_CACHE_CONFIG"] || "config/solid_cache.yml"
+      config_paths = %w[config/cache config/solid_cache]
 
-      options = {}
-      if (config_path = Pathname.new(app.config.paths["config/solid_cache"].first)).exist?
-        options = app.config_for(config_path).to_h.deep_symbolize_keys
+      config_paths.each do |path|
+        app.paths.add path, with: ENV["SOLID_CACHE_CONFIG"] || "#{path}.yml"
       end
 
+      config_pathname = config_paths.map { |path| Pathname.new(app.config.paths[path].first) }.find(&:exist?)
+
+      options = config_pathname ? app.config_for(config_pathname).to_h.deep_symbolize_keys : {}
+
       options[:connects_to] = config.solid_cache.connects_to if config.solid_cache.connects_to
       options[:size_estimate_samples] = config.solid_cache.size_estimate_samples if config.solid_cache.size_estimate_samples
       options[:encrypt] = config.solid_cache.encrypt if config.solid_cache.encrypt

data/lib/solid_cache/store/connections.rb

@@ -34,26 +34,6 @@ module SolidCache
         end
       end
 
-      def with_connection_for(key, async: false, &block)
-        connections.with_connection_for(key) do
-          execute(async, &block)
-        end
-      end
-
-      def with_connection(name, async: false, &block)
-        connections.with(name) do
-          execute(async, &block)
-        end
-      end
-
-      def group_by_connection(keys)
-        connections.assign(keys)
-      end
-
-      def connection_names
-        connections.names
-      end
-
       def connections
         @connections ||= SolidCache::Connections.from_config(@shard_options)
       end
@@ -63,6 +43,26 @@ module SolidCache
           connections
         end
 
+        def with_connection_for(key, async: false, &block)
+          connections.with_connection_for(key) do
+            execute(async, &block)
+          end
+        end
+
+        def with_connection(name, async: false, &block)
+          connections.with(name) do
+            execute(async, &block)
+          end
+        end
+
+        def group_by_connection(keys)
+          connections.assign(keys)
+        end
+
+        def connection_names
+          connections.names
+        end
+
         def reading_key(key, failsafe:, failsafe_returning: nil, &block)
           failsafe(failsafe, returning: failsafe_returning) do
             with_connection_for(key, &block)
@@ -70,16 +70,15 @@ module SolidCache
         end
 
         def reading_keys(keys, failsafe:, failsafe_returning: nil)
-          group_by_connection(keys).map do |connection, keys|
+          group_by_connection(keys).map do |connection, grouped_keys|
             failsafe(failsafe, returning: failsafe_returning) do
               with_connection(connection) do
-                yield keys
+                yield grouped_keys
               end
             end
           end
         end
 
-
         def writing_key(key, failsafe:, failsafe_returning: nil, &block)
           failsafe(failsafe, returning: failsafe_returning) do
             with_connection_for(key, &block)
@@ -87,10 +86,10 @@ module SolidCache
         end
 
         def writing_keys(entries, failsafe:, failsafe_returning: nil)
-          group_by_connection(entries).map do |connection, entries|
+          group_by_connection(entries).map do |connection, grouped_entries|
             failsafe(failsafe, returning: failsafe_returning) do
               with_connection(connection) do
-                yield entries
+                yield grouped_entries
               end
             end
           end

data/lib/solid_cache/store/expiry.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require "concurrent/atomic/atomic_fixnum"
-
 module SolidCache
   class Store
     module Expiry

data/lib/solid_cache/store/failsafe.rb

@@ -6,7 +6,6 @@ module SolidCache
       TRANSIENT_ACTIVE_RECORD_ERRORS = [
         ActiveRecord::AdapterTimeout,
         ActiveRecord::ConnectionNotEstablished,
-        ActiveRecord::ConnectionTimeoutError,
         ActiveRecord::Deadlocked,
         ActiveRecord::LockWaitTimeout,
         ActiveRecord::QueryCanceled,

data/lib/solid_cache/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module SolidCache
-  VERSION = "1.0.5"
+  VERSION = "1.0.7"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: solid_cache
 version: !ruby/object:Gem::Version
-  version: 1.0.5
+  version: 1.0.7
 platform: ruby
 authors:
 - Donal McBreen
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2024-09-05 00:00:00.000000000 Z
+date: 2025-02-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -115,7 +115,7 @@ files:
 - lib/active_support/cache/solid_cache_store.rb
 - lib/generators/solid_cache/install/USAGE
 - lib/generators/solid_cache/install/install_generator.rb
-- lib/generators/solid_cache/install/templates/config/solid_cache.yml.tt
+- lib/generators/solid_cache/install/templates/config/cache.yml.tt
 - lib/generators/solid_cache/install/templates/db/cache_schema.rb
 - lib/solid_cache.rb
 - lib/solid_cache/configuration.rb
@@ -158,7 +158,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.17
+rubygems_version: 3.5.22
 signing_key:
 specification_version: 4
 summary: A database backed ActiveSupport::Cache::Store

data/lib/generators/solid_cache/install/templates/config/{solid_cache.yml.tt → cache.yml.tt}

@@ -1,5 +1,4 @@
 default: &default
-  database: <%= ENV.fetch("DATABASE", "cache") %>
   store_options:
     # Cap age of oldest cache entry to fulfill retention policies
     # max_age: <%%= 60.days.to_i %>
@@ -13,4 +12,5 @@ test:
   <<: *default
 
 production:
+  database: <%= ENV.fetch("DATABASE", "cache") %>
   <<: *default