solid_cache 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: '0649423e526a66fa9554f73d215e7e49c6ff093c004c1641d72a670a33688468'
+  data.tar.gz: ad56d9f53b6b0c6f9dcc049771a187e95c5e2cceeabeaa07286f96212f5b0fb7
+SHA512:
+  metadata.gz: 900011771476bff6c3b3ec5abe087a0dcf00596f6e200dec961aa7e13f3ad4d978d9ac8858d3912b8bc0bc63a8742de4e925bd04b120d15badb890448946f4c2
+  data.tar.gz: c9fd66e658ec1e38f628c592053f330179571a7299a1d4e6bae88a959996c10c4d7df158c30c9827464e0c3af526a4b5c0100f55f51c5b3e563e73b5af93226a

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2023 37signals
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,268 @@
+# Solid Cache
+
+Solid Cache is a database-backed Active Support cache store implementation. 
+
+Using SQL databases backed by SSDs we can have caches that are much larger and cheaper than traditional memory only Redis or Memcached backed caches.
+
+Testing on [HEY](https://hey.com) shows that reads and writes are 25%-50% slower than with a Redis cache (1.2ms vs 0.8-1ms per single-key read), but this is not a significant percentage of the overall request time.
+
+If cache misses are expensive (up to 50x the cost of a hit on HEY), then there are big advantages to caches that can hold months rather than days of data.
+
+## Usage
+
+To set Solid Cache as your Rails cache, you should add this to your environment config:
+
+```ruby
+config.cache_store = :solid_cache_store
+```
+
+Solid Cache is a FIFO (first in, first out) cache. While this is not as efficient as an LRU cache, this is mitigated by the longer cache lifespans.
+
+A FIFO cache is much easier to manage:
+1. We don't need to track when items are read
+2. We can estimate and control the cache size by comparing the maximum and minimum IDs.
+3. By deleting from one end of the table and adding at the other end we can avoid fragmentation (on MySQL at least).
+
+### Installation
+Add this line to your application's Gemfile:
+
+```ruby
+gem "solid_cache"
+```
+
+And then execute:
+```bash
+$ bundle
+```
+
+Or install it yourself as:
+```bash
+$ gem install solid_cache
+```
+
+Add the migration to your app:
+
+```bash
+$ bin/rails solid_cache:install:migrations
+```
+
+Then run it:
+```bash
+$ bin/rails db:migrate
+```
+
+### Configuration
+
+#### Engine configuration
+
+There are two options that can be set on the engine:
+
+- `executor` - the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap asynchronous operations, defaults to the app executor
+- `connects_to` - a custom connects to value for the abstract `SolidCache::Record` active record model. Requires for sharding and/or using a separate cache database to the main app.
+
+These can be set in your Rails configuration:
+
+```ruby
+Rails.application.configure do
+  config.solid_cache.connects_to = {
+    shards: {
+      shard1: { writing: :cache_primary_shard1 },
+      shard2: { writing: :cache_primary_shard2 }
+    }
+  }
+end
+```
+
+#### Cache configuration
+
+Solid Cache supports these options in addition to the standard `ActiveSupport::Cache::Store` options.
+
+- `error_handler` - a Proc to call to handle any `ActiveRecord::ActiveRecordError`s that are raises (default: log errors as warnings)
+- `expiry_batch_size` - the batch size to use when deleting old records (default: `100`)
+- `expiry_method` - what expiry method to use `thread` or `job` (default: `thread`)
+- `max_age` - the maximum age of entries in the cache (default: `2.weeks.to_i`)
+- `max_entries` - the maximum number of entries allowed in the cache (default: `2.weeks.to_i`)
+- `cluster` - a Hash of options for the cache database cluster, e.g `{ shards: [:database1, :database2, :database3] }`
+- `clusters` - and Array of Hashes for multiple cache clusters (ignored if `:cluster` is set)
+- `active_record_instrumentation` - whether to instrument the cache's queries (default: `true`)
+- `clear_with` - clear the cache with `:truncate` or `:delete` (default `truncate`, except for when Rails.env.test? then `delete`)
+- `max_key_bytesize` - the maximum size of a normalized key in bytes (default `1024`)
+
+For more information on cache clusters see [Sharding the cache](#sharding-the-cache)
+
+### Cache expiry
+
+Solid Cache tracks writes to the cache. For every write it increments a counter by 1. Once the counter reaches 80% of the `expiry_batch_size` it add a task to run on a background thread. That task will:
+
+1. Check if we have exceeded the `max_entries` value (if set) by subtracting the max and min IDs from the `SolidCache::Entry` table (this is an estimate that ignores any gaps).
+2. If we have it will delete `expiry_batch_size` entries
+3. If not it will delete up to `expiry_batch_size` entries, provided they are all older than `max_age`.
+
+Expiring when we reach 80% of the batch size allows us to expire records from the cache faster than we write to it when we need to reduce the cache size.
+
+Only triggering expiry when we write means that the if the cache is idle, the background thread is also idle.
+
+If you want the cache expiry to be run in a background job instead of a thread, you can set `expiry_method` to `:job`. This will enqueue a `SolidCache::ExpiryJob`.
+
+### Using a dedicated cache database
+
+Add database configuration to database.yml, e.g.:
+
+```
+development
+  cache:
+    database: cache_development
+    host: 127.0.0.1
+    migrations_paths: "db/cache/migrate"
+```
+
+Create database:
+```
+$ bin/rails db:create
+```
+
+Install migrations:
+```
+$ bin/rails solid_cache:install:migrations
+```
+
+Move migrations to custom migrations folder:
+```
+$ mkdir -p db/cache/migrate
+$ mv db/migrate/*.solid_cache.rb db/cache/migrate
+```
+
+Set the engine configuration to point to the new database:
+```
+Rails.application.configure do
+  config.solid_cache.connects_to = { default: { writing: :cache } }
+end
+```
+
+Run migrations:
+```
+$ bin/rails db:migrate
+```
+
+### Sharding the cache
+
+Solid Cache uses the [Maglev](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44824.pdf) consistent hashing scheme to shard the cache across multiple databases.
+
+To shard:
+
+1. Add the configuration for the database shards to database.yml
+2. Configure the shards via `config.solid_cache.connects_to`
+3. Pass the shards for the cache to use via the cluster option
+
+For example:
+```ruby
+# config/database.yml
+production:
+  cache_shard1:
+    database: cache1_production
+    host: cache1-db
+  cache_shard2:
+    database: cache2_production
+    host: cache2-db
+  cache_shard3:
+    database: cache3_production
+    host: cache3-db
+
+
+# config/environment/production.rb
+Rails.application.configure do
+  config.solid_cache.connects_to = {
+    shards: {
+      cache_shard1: { writing: :cache_shard1 },
+      cache_shard2: { writing: :cache_shard2 },
+      cache_shard3: { writing: :cache_shard3 },
+    }
+  }
+
+  config.cache_store = [ :solid_cache_store, cluster: { shards: [ :cache_shard1, :cache_shard2, :cache_shard3 ] } ]
+end
+```
+
+### Secondary cache clusters
+
+You can add secondary cache clusters. Reads will only be sent to the primary cluster (i.e. the first one listed).
+
+Writes will go to all clusters. The writes to the primary cluster are synchronous, but asyncronous to the secondary clusters.
+
+To specific multiple clusters you can do:
+
+```ruby
+Rails.application.configure do
+  config.solid_cache.connects_to = {
+    shards: {
+      cache_primary_shard1: { writing: :cache_primary_shard1 },
+      cache_primary_shard2: { writing: :cache_primary_shard2 },
+      cache_secondary_shard1: { writing: :cache_secondary_shard1 },
+      cache_secondary_shard2: { writing: :cache_secondary_shard2 },
+    }
+  }
+
+  primary_cluster = { shards: [ :cache_primary_shard1, :cache_primary_shard2 ] }
+  secondary_cluster = { shards: [ :cache_primary_shard1, :cache_primary_shard2 ] }
+  config.cache_store = [ :solid_cache_store, clusters: [ primary_cluster, secondary_cluster ] ]
+end
+```
+
+### Named shard destinations
+
+By default, the node key used for sharding is the name of the database in `database.yml`.
+
+It is possible to add names for the shards in the cluster config. This will allow you to shuffle or remove shards without breaking consistent hashing.
+
+```ruby
+Rails.application.configure do
+  config.solid_cache.connects_to = {
+    shards: {
+      cache_primary_shard1: { writing: :cache_primary_shard1 },
+      cache_primary_shard2: { writing: :cache_primary_shard2 },
+      cache_secondary_shard1: { writing: :cache_secondary_shard1 },
+      cache_secondary_shard2: { writing: :cache_secondary_shard2 },
+    }
+  }
+
+  primary_cluster = { shards: { cache_primary_shard1: :node1, cache_primary_shard2: :node2 } }
+  secondary_cluster = { shards: { cache_primary_shard1: :node3, cache_primary_shard2: :node4 } }
+  config.cache_store = [ :solid_cache_store, clusters: [ primary_cluster, secondary_cluster ] ]
+end
+```
+
+
+### Enabling encryption
+
+Add this to an initializer:
+
+```ruby
+ActiveSupport.on_load(:solid_cache_entry) do
+  encrypts :value
+end
+```
+
+### Index size limits
+The Solid Cache migrations try to create an index with 1024 byte entries. If that is too big for your database, you should:
+
+1. Edit the index size in the migration
+2. Set `max_key_bytesize` on your cache to the new value
+
+## Development
+
+Run the tests with `bin/rails test`. These will run against SQLite.
+
+You can also run the tests against MySQL and Postgres. First start up the databases:
+
+```shell
+$ docker compose up -d
+```
+
+Then run the tests for the target database
+```
+$ TARGET_DB=mysql bin/rails test
+$ TARGET_DB=postgres bin/rails test
+```
+
+## License
+Solid Cache is licensed under MIT.

data/Rakefile

@@ -0,0 +1,8 @@
+require "bundler/setup"
+
+APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
+load "rails/tasks/engine.rake"
+
+load "rails/tasks/statistics.rake"
+
+require "bundler/gem_tasks"

data/app/jobs/solid_cache/expiry_job.rb

@@ -0,0 +1,9 @@
+module SolidCache
+  class ExpiryJob < ActiveJob::Base
+    def perform(count, shard: nil, max_age:, max_entries:)
+      Record.with_shard(shard) do
+        Entry.expire(count, max_age: max_age, max_entries: max_entries)
+      end
+    end
+  end
+end

data/app/models/solid_cache/entry.rb

@@ -0,0 +1,157 @@
+module SolidCache
+  class Entry < Record
+    # This is all quite awkward but it achieves a couple of performance aims
+    # 1. We skip the query cache
+    # 2. We avoid the overhead of building queries and active record objects
+    class << self
+      def write(key, value)
+        upsert_all_no_query_cache([ { key: key, value: value } ])
+      end
+
+      def write_multi(payloads)
+        upsert_all_no_query_cache(payloads)
+      end
+
+      def read(key)
+        select_all_no_query_cache(get_sql, to_binary(key)).first
+      end
+
+      def read_multi(keys)
+        serialized_keys = keys.map { |key| to_binary(key) }
+        select_all_no_query_cache(get_all_sql(serialized_keys), serialized_keys).to_h
+      end
+
+      def expire(ids)
+        delete_no_query_cache(:id, ids) if ids.any?
+      end
+
+      def delete_by_key(key)
+        delete_no_query_cache(:key, to_binary(key))
+      end
+
+      def delete_matched(matcher, batch_size:)
+        like_matcher = arel_table[:key].matches(matcher, nil, true)
+        where(like_matcher).select(:id).find_in_batches(batch_size: batch_size) do |entries|
+          delete_no_query_cache(:id, entries.map(&:id))
+        end
+      end
+
+      def clear_truncate
+        connection.truncate(table_name)
+      end
+
+      def clear_delete
+        in_batches.delete_all
+      end
+
+      def increment(key, amount)
+        transaction do
+          uncached do
+            amount += lock.where(key: key).pick(:value).to_i
+            write(key, amount)
+            amount
+          end
+        end
+      end
+
+      def decrement(key, amount)
+        increment(key, -amount)
+      end
+
+      def id_range
+        uncached do
+          pick(Arel.sql("max(id) - min(id) + 1")) || 0
+        end
+      end
+
+      def expire(count, max_age:, max_entries:)
+        if (ids = expiry_candidate_ids(count, max_age: max_age, max_entries: max_entries)).any?
+          delete(ids)
+        end
+      end
+
+      private
+        def upsert_all_no_query_cache(attributes)
+          insert_all = ActiveRecord::InsertAll.new(self, attributes, unique_by: upsert_unique_by, on_duplicate: :update, update_only: [ :value ])
+          sql = connection.build_insert_sql(ActiveRecord::InsertAll::Builder.new(insert_all))
+
+          message = +"#{self} "
+          message << "Bulk " if attributes.many?
+          message << "Upsert"
+          # exec_query does not clear the query cache, exec_insert_all does
+          connection.exec_query sql, message
+        end
+
+        def upsert_unique_by
+          connection.supports_insert_conflict_target? ? :key : nil
+        end
+
+        def get_sql
+          @get_sql ||= build_sql(where(key: "placeholder").select(:value))
+        end
+
+        def get_all_sql(keys)
+          if connection.prepared_statements?
+            @get_all_sql_binds ||= {}
+            @get_all_sql_binds[keys.count] ||= build_sql(where(key: keys).select(:key, :value))
+          else
+            @get_all_sql_no_binds ||= build_sql(where(key: [ "placeholder1", "placeholder2" ]).select(:key, :value)).gsub("?, ?", "?")
+          end
+        end
+
+        def build_sql(relation)
+          collector = Arel::Collectors::Composite.new(
+            Arel::Collectors::SQLString.new,
+            Arel::Collectors::Bind.new,
+          )
+
+          connection.visitor.compile(relation.arel.ast, collector)[0]
+        end
+
+        def select_all_no_query_cache(query, values)
+          uncached do
+            if connection.prepared_statements?
+              result = connection.select_all(sanitize_sql(query), "#{name} Load", Array(values), preparable: true)
+            else
+              result = connection.select_all(sanitize_sql([ query, values ]), "#{name} Load", nil, preparable: false)
+            end
+
+            result.cast_values(SolidCache::Entry.attribute_types)
+          end
+        end
+
+        def delete_no_query_cache(attribute, values)
+          uncached do
+            relation = where(attribute => values)
+            sql = connection.to_sql(relation.arel.compile_delete(relation.table[primary_key]))
+
+            # exec_delete does not clear the query cache
+            if connection.prepared_statements?
+              connection.exec_delete(sql, "#{name} Delete All", Array(values)).nonzero?
+            else
+              connection.exec_delete(sql, "#{name} Delete All").nonzero?
+            end
+          end
+        end
+
+        def to_binary(key)
+          ActiveModel::Type::Binary.new.serialize(key)
+        end
+
+        def expiry_candidate_ids(count, max_age:, max_entries:)
+          cache_full = max_entries && max_entries < id_range
+          min_created_at = max_age.seconds.ago
+
+          uncached do
+            order(:id)
+              .limit(count * 3)
+              .pluck(:id, :created_at)
+              .filter_map { |id, created_at| id if cache_full || created_at < min_created_at }
+              .sample(count)
+          end
+        end
+    end
+  end
+end
+
+ActiveSupport.run_load_hooks :solid_cache_entry, SolidCache::Entry

data/app/models/solid_cache/record.rb

@@ -0,0 +1,27 @@
+module SolidCache
+  class Record < ActiveRecord::Base
+    NULL_INSTRUMENTER = ActiveSupport::Notifications::Instrumenter.new(ActiveSupport::Notifications::Fanout.new)
+
+    self.abstract_class = true
+
+    connects_to **SolidCache.connects_to if SolidCache.connects_to
+
+    class << self
+      def disable_instrumentation
+        connection.with_instrumenter(NULL_INSTRUMENTER) do
+          yield
+        end
+      end
+
+      def with_shard(shard, &block)
+        if shard && shard != Record.current_shard
+          Record.connected_to(shard: shard, &block)
+        else
+          block.call
+        end
+      end
+    end
+  end
+end
+
+ActiveSupport.run_load_hooks :solid_cache, SolidCache::Record

data/db/migrate/20230724121448_create_solid_cache_entries.rb

@@ -0,0 +1,11 @@
+class CreateSolidCacheEntries < ActiveRecord::Migration[7.0]
+  def change
+    create_table :solid_cache_entries do |t|
+      t.binary   :key,        null: false,   limit: 1024
+      t.binary   :value,      null: false,   limit: 512.megabytes
+      t.datetime :created_at, null: false
+
+      t.index    :key,        unique: true
+    end
+  end
+end

data/lib/active_support/cache/solid_cache_store.rb

@@ -0,0 +1,5 @@
+module ActiveSupport
+  module Cache
+    SolidCacheStore = SolidCache::Store
+  end
+end

data/lib/generators/solid_cache/install/USAGE

@@ -0,0 +1,9 @@
+Description:
+    Installs solid_cache as the Rails cache store
+
+Example:
+    bin/rails generate solid_cache:install
+
+    This will create:
+        Installs the solid_cache migrations
+        Replaces the cache store in envionment configuration

data/lib/generators/solid_cache/install/install_generator.rb

@@ -0,0 +1,18 @@
+class SolidCache::InstallGenerator < Rails::Generators::Base
+  class_option :skip_migrations,    type: :boolean, default: nil,
+                                    desc: "Skip migrations"
+
+  def add_rails_cache
+    %w[development test production].each do |env_name|
+      if (env_config = Pathname(destination_root).join("config/environments/#{env_name}.rb")).exist?
+        gsub_file env_config, /(# )?config\.cache_store = (:(?!null_store).*)/, "config.cache_store = :solid_cache_store"
+      end
+    end
+  end
+
+  def create_migrations
+    unless options[:skip_migrations]
+      rails_command "railties:install:migrations FROM=solid_cache", inline: true
+    end
+  end
+end

data/lib/solid_cache/cluster/connections.rb

@@ -0,0 +1,51 @@
+module SolidCache
+  class Cluster
+    module Connections
+      def initialize(options = {})
+        super(options)
+        @shard_options = options.fetch(:shards, nil)
+
+        if [ Hash, Array, NilClass ].none? { |klass| @shard_options.is_a? klass }
+          raise ArgumentError, "`shards` is a `#{@shard_options.class.name}`, it should be one of Array, Hash or nil"
+        end
+      end
+
+      def with_each_connection(async: false, &block)
+        return enum_for(:with_each_connection) unless block_given?
+
+        connections.with_each do
+          execute(async, &block)
+        end
+      end
+
+      def with_connection_for(key, async: false, &block)
+        connections.with_connection_for(key) do
+          execute(async, &block)
+        end
+      end
+
+      def with_connection(name, async: false, &block)
+        connections.with(name) do
+          execute(async, &block)
+        end
+      end
+
+      def group_by_connection(keys)
+        connections.assign(keys)
+      end
+
+      def connection_names
+        connections.names
+      end
+
+      private
+        def setup!
+          connections
+        end
+
+        def connections
+          @connections ||= SolidCache::Connections.from_config(@shard_options)
+        end
+    end
+  end
+end

data/lib/solid_cache/cluster/execution.rb

@@ -0,0 +1,52 @@
+module SolidCache
+  class Cluster
+    module Execution
+      def initialize(options = {})
+        super(options)
+        @background = Concurrent::SingleThreadExecutor.new(max_queue: 100, fallback_policy: :discard)
+        @active_record_instrumentation = options.fetch(:active_record_instrumentation, true)
+      end
+
+      private
+        def async(&block)
+          # Need current shard right now, not when block is called
+          current_shard = Entry.current_shard
+          @background << ->() do
+            wrap_in_rails_executor do
+              connections.with(current_shard) do
+                instrument(&block)
+              end
+            end
+          end
+        end
+
+        def execute(async, &block)
+          if async
+            async(&block)
+          else
+            instrument(&block)
+          end
+        end
+
+        def wrap_in_rails_executor(&block)
+          if SolidCache.executor
+            SolidCache.executor.wrap(&block)
+          else
+            block.call
+          end
+        end
+
+        def active_record_instrumentation?
+          @active_record_instrumentation
+        end
+
+        def instrument(&block)
+          if active_record_instrumentation?
+            block.call
+          else
+            Record.disable_instrumentation(&block)
+          end
+        end
+    end
+  end
+end