litestack 0.4.2 → 0.4.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 98a82dcbe7e4fe0b215e254fac5791127142fcbfd4a156b2509434f97b58aea9
-  data.tar.gz: a914b64d6e031559b1bdca83a8198426a49005f56ca1d0b4da2a7c6cb2384ab7
+  metadata.gz: e2ac02213caf3525c40b676550a6daa24380f3012a5d4c8f8042cf39aa477593
+  data.tar.gz: 00d0f60e20bcba2cc22704d4e7159fe9b819920d313e7dc7d4e4e799420e95ca
 SHA512:
-  metadata.gz: 72935d15fcced99bcc0da9cfb6c1ffcd43478a5456c073a3d8ab468a3e49f7c681bbdec861cb7c1e2c16659e0d11429e16d855be38fd30735448078977205dc9
-  data.tar.gz: ac181213e66533926ce5d075e8256dcf2c5dfa727ff7becaa244bbf053f354c564f29a294f4b2b8c1436320d41e9dc4e4697ba9e3169324bc798c60d8bfc762a
+  metadata.gz: 7c9c726aecbfd99ad73d82c50c285178becefb4792ece0672b55dfede10a47f0563700a691398373f372ca829491c13e0332a81b9b075f9a0435ae55ac138a1c
+  data.tar.gz: b4a28527632a3065d0deb8eadb5c806d260edfd89aa9b3f00ebbc20a2aa63ebfb7df907e7bd0a500f9e368ecd91b8967789b1f056632ef5bc73502dbbe1524c9

data/CHANGELOG.md

@@ -1,7 +1,20 @@
 ## [Unreleased]
 
+## [0.4.3] - 2024-02-15
+
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.1...master)
+- Add "sequel" as a development dependency
+- Diff links in CHANGELOG (thanks Weston Ganger)
+- Fix deamonize type in liteboard (thanks Julian Rubisch)
+- Better Litecache schema (streamlined numeric value support)
+- Support for set_multi and get_multi in Litecache (read_multi and write_multi support for Rails Cache store)
+- More tests written for Litecache and Rails Litecache store
+- Experimenting with removing the Rails LocalCache as it doesn't show enough improvement in perfromance to compensate for the memory overhead 
+- Switch Litecache to a FIFO eviction model vs LRU (thanks Julian Rubisch and Stephen Margheim)
+
 ## [0.4.2] - 2023-11-11
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.1...v0.4.2)
 - Add similarity search support for Litesearch (works best for non-trigram indexes)
 - Enable similarity search for ActiveRecord and Sequel models
 - Fix Litesearch tests
@@ -13,16 +26,19 @@
 
 ## [0.4.1] - 2023-10-11
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.0...v0.4.1)
 - Add missing Litesearch::Model dependency
 
 ## [0.4.0] - 2023-10-11
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.3.0...v0.4.0)
 - Introduced Litesearch, dynamic & fast full text search capability for Litedb
 - ActiveRecord and Sequel integration for Litesearch
 - Slight improvement to the Sequel Litedb adapter for better Litesearch integration
 
 ## [0.3.0] - 2023-08-13
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.6...v0.3.0)
 - Reworked the Litecable thread safety model
 - Fixed multiple litejob bugs (thanks Stephen Margheim)
 - Fixed Railtie dependency (thanks Marco Roth)
@@ -32,6 +48,7 @@
 
 ## [0.2.6] - 2023-07-16
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.3...v0.2.6)
 - Much improved database location setting (thanks Brad Gessler)
 - A Rails generator for better Rails Litestack defaults (thanks Brad Gessler)
 - Revamped Litemetric, now much faster and more accurate (still experimental)
@@ -39,19 +56,23 @@
 
 ## [0.2.3] - 2023-05-20
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.2...v0.2.3)
 - Cut back on options defined in the Litejob Rails adapter
 
 ## [0.2.2] - 2023-05-18
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.1...v0.2.2)
 - Fix default queue location in Litejob
 
 
 ## [0.2.1] - 2023-05-08
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.0...v0.2.1)
 - Fix a race condition in Litecable
 
 ## [0.2.0] - 2023-05-08
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.8...v0.2.0)
 - Litecable, a SQLite driver for ActionCable
 - Litemetric for metrics collection support (experimental, disabled by default)
 - New schema for Litejob, old jobs are auto-migrated
@@ -61,6 +82,7 @@
 
 ## [0.1.8] - 2023-03-08
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.7...v0.1.8)
 - More code cleanups, more test coverage
 - Retry support for jobs in Litejob
 - Job storage and garbage collection for failed jobs
@@ -69,6 +91,7 @@
 
 ## [0.1.7] - 2023-03-05
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.6...v0.1.7)
 - Code cleanup, removal of references to older name
 - Fix for the litedb rake tasks (thanks: netmute)
 - More fixes for the new concurrency model
@@ -76,6 +99,7 @@
 
 ## [0.1.6] - 2023-03-03
 
+- [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.0...v0.1.6)
 - Revamped the locking model, more robust, minimal performance hit
 - Introduced a new resource pooling class
 - Litecache and Litejob now use the resource pool

data/README.md

@@ -1,5 +1,9 @@
 ![litestack](https://github.com/oldmoe/litestack/blob/master/assets/litestack_logo_teal_large.png?raw=true)
 
+<a href="https://badge.fury.io/rb/litestack" target="_blank"><img height="21" style='border:0px;height:21px;' border='0' src="https://badge.fury.io/rb/litestack.svg" alt="Gem Version"></a>
+<a href='https://github.com/oldmoe/litestack/actions' target='_blank'><img src="https://github.com/oldmoe/litestack/actions/workflows/ruby.yml/badge.svg?branch=master" style="max-width:100%;" height='21' style='border:0px;height:21px;' border='0' alt="CI Status"></a>
+<a href='https://rubygems.org/gems/litestack' target='_blank'><img height='21' style='border:0px;height:21px;' src='https://img.shields.io/gem/dt/litestack?color=brightgreen&label=Rubygems%20Downloads' border='0' alt='RubyGems Downloads' /></a>
+
 All your data infrastructure, in a gem!
 
 Litestack is a Ruby gem that provides both Ruby and  Ruby on Rails applications an all-in-one solution for web application data infrastructure. It exploits the power and embeddedness of SQLite to deliver a full-fledged SQL database, a fast cache , a robust job queue, a reliable message broker, a full text search engine and a metrics platform all in a single package.
@@ -24,7 +28,7 @@ With litestack you only need to add a single gem to your app which would replace
 - Cache Server (e.g. Redis, Memcached)
 - Job Processor (e.g. Sidekiq, Goodjob)
 - Pubsub Server (e.g. Redis, PostgreSQL)
-- Fulltext Search Server (e.g. Elasticsearch, Mielisearch)
+- Fulltext Search Server (e.g. Elasticsearch, Meilisearch)
 
 To make it even more efficient, litestack will detect the presence of Fiber based IO frameworks like Async (e.g. when you use the Falcon web server) or Polyphony. It will then switch its background workers for caches and queues to fibers (using the semantics of the existing framework). This is done transparently and will generally lead to lower CPU and memory utilization.
 
@@ -258,7 +262,7 @@ Book.search('author: writer').limit(1).all
 > ![litemetric](https://github.com/oldmoe/litestack/blob/master/assets/litemetric_logo_teal.png?raw=true)
 
 ### Litemetric
-Litestack comes with a mdoule that can collect useful metrics for its different components, in each component, you need to add the following to the respective .yml file (database.yml in case of Litedb)
+Litestack comes with a module that can collect useful metrics for its different components, in each component, you need to add the following to the respective .yml file (database.yml in case of Litedb)
 ```yml
     metrics: true # default is false
 ```

data/bench/bench_cache_rails.rb

@@ -2,7 +2,7 @@ require "active_support"
 require_relative "../lib/litestack"
 require_relative "./bench"
 
-cache = ActiveSupport::Cache::Litecache.new({path: "../db/rails_cache.db"})
+cache = ActiveSupport::Cache::Litecache.new
 
 # can only use the lookup method when the gem is installed
 # cache = ActiveSupport::Cache.lookup_store(:litecache, {path: '../db/rails_cache.db'})
@@ -12,8 +12,10 @@ redis = ActiveSupport::Cache.lookup_store(:redis_cache_store, {})
 values = []
 keys = []
 count = 1000
+payload_sizes = [10, 100, 1000, 10000]
+#payload_sizes = [100]
 
-[10, 100, 1000, 10000].each do |size|
+payload_sizes.each do |size|
   count.times do
     keys << random_str(10)
     values << random_str(size)
@@ -31,6 +33,21 @@ count = 1000
     redis.write(keys[i], values[i])
   end
 
+  puts "== Multi Writes =="
+  bench("litecache multi-writes", count/5) do |i|
+    idx = i * 5 
+    payload = {} 
+    5.times {|j| payload[keys[idx + j]] = values[idx + j] }
+    cache.write_multi(payload)
+  end
+
+  bench("Redis multi-writes", count/5) do |i|
+    idx = i * 5 
+    payload = {} 
+    5.times {|j| payload[keys[idx + j]] = values[idx + j] }
+    redis.write_multi(payload)
+  end
+
   puts "== Reads =="
   bench("litecache reads", count) do |i|
     cache.read(random_keys[i])
@@ -39,6 +56,23 @@ count = 1000
   bench("Redis reads", count) do |i|
     redis.read(random_keys[i])
   end
+
+  puts "== Multi Reads =="
+  bench("litecache multi-reads", count/5) do |i|
+    idx = i * 5
+    payload = []
+    5.times {|j| payload << random_keys[idx+j]}
+    cache.read_multi(*payload)
+  end
+
+  bench("Redis multi-reads", count/5) do |i|
+    idx = i * 5
+    payload = []
+    5.times {|j| payload << random_keys[idx+j]}
+    redis.read_multi(*payload)
+  end
+
+
   puts "=========================================================="
 
   keys = []

data/bench/bench_cache_raw.rb

@@ -11,7 +11,7 @@ Fiber.scheduler.run
 require_relative "../lib/litestack/litecache"
 # require 'litestack'
 
-cache = Litecache.new({path: "../db/cache.db"}) # default settings
+cache = Litecache.new #({path: "../db/cache.db"}) # default settings
 redis = Redis.new # default settings
 
 values = []
@@ -35,28 +35,49 @@ count.times { keys << random_str(10) }
     cache.set(keys[i], values[i])
   end
 
-  # bench("file writes", count) do |i|
-  #  f = File.open("../files/#{keys[i]}.data", 'w+')
-  #  f.write(values[i])
-  #  f.close
-  # end
-
   bench("Redis writes", count) do |i|
     redis.set(keys[i], values[i])
   end
 
+  puts "== Multi Writes =="
+  bench("litecache multi-writes", count/5) do |i|
+    idx = i * 5 
+    payload = {} 
+    5.times {|j| payload[keys[idx + j]] = values[idx + j] }
+    cache.set_multi(payload)
+  end 
+
+  bench("Redis multi-writes", count/5) do |i|
+    idx = i * 5 
+    payload = [] 
+    5.times {|j| payload << keys[idx + j]; payload <<  values[idx + j]}
+    redis.mset(*payload)
+  end
+
   puts "== Reads =="
   bench("litecache reads", count) do |i|
     cache.get(random_keys[i])
   end
 
-  # bench("file reads", count) do |i|
-  #  data = File.read("../files/#{keys[i]}.data")
-  # end
-
   bench("Redis reads", count) do |i|
     redis.get(random_keys[i])
   end
+
+  puts "== Multi Reads =="
+  bench("litecache multi-reads", count/5) do |i|
+    idx = i * 5
+    payload = []
+    5.times {|j| payload << random_keys[idx+j]}
+    cache.get_multi(*payload)
+  end
+
+  bench("Redis multi-reads", count/5) do |i|
+    idx = i * 5
+    payload = []
+    5.times {|j| payload << random_keys[idx+j]}
+    redis.mget(*payload)
+  end
+
   puts "=========================================================="
 
   values = []

data/bin/liteboard

@@ -15,6 +15,7 @@ options = {
   Port: 9292,
   Host: 'localhost',
   environment: 'production',
+  pid: 'tmp/pids/liteboard.pid',
   quiet: false
 }
 
@@ -25,7 +26,7 @@ OptionParser.new do |parser|
   parser.on("-s", "--server SERVER", "use SERVER (e.g. puma/falcon/iodine)") { |v| options[:port] = v }
   parser.on("-H", "--host HOST", "listen on HOST (default: #{options[:Host]})") { |v| options[:Host] = v }
   parser.on("-p", "--port PORT", "use PORT (default: #{options[:Port]})") { |v| options[:Port] = v.to_i rescue options[:Port] }
-  parser.on("-D", "--deamonize", "run in the background") { |v| options[:deamonize] = true }  
+  parser.on("-D", "--daemonize", "run in the background") { |v| options[:daemonize] = true }  
   parser.on("-E", "--env ENVIRONMENT", "which environment to use (default: #{options[:environment]})") { |v| options[:environment] = v }  
   parser.on("-q", "--quiet", "turn off logging") { |v| options[:quiet] = true }  
   parser.on("-h", "--help", "print this message") do

data/lib/active_record/connection_adapters/litedb_adapter.rb

@@ -41,18 +41,16 @@ module ActiveRecord
     class LitedbAdapter < SQLite3Adapter
       ADAPTER_NAME = "litedb"
 
-      class << self
-        def dbconsole(config, options = {})
-          args = []
+      def self.dbconsole(config, options = {})
+        args = []
 
-          args << "-#{options[:mode]}" if options[:mode]
-          args << "-header" if options[:header]
-          args << File.expand_path(config.database, Rails.respond_to?(:root) ? Rails.root : nil)
+        args << "-#{options[:mode]}" if options[:mode]
+        args << "-header" if options[:header]
+        args << File.expand_path(config.database, Rails.respond_to?(:root) ? Rails.root : nil)
 
-          find_cmd_and_exec("sqlite3", *args)
-        end
+        self.find_cmd_and_exec("sqlite3", *args)
       end
-
+      
       NATIVE_DATABASE_TYPES = {
         primary_key: "integer PRIMARY KEY NOT NULL",
         string: {name: "text"},

data/lib/active_support/cache/litecache.rb

@@ -8,7 +8,7 @@ require_relative "../../litestack/litecache"
 module ActiveSupport
   module Cache
     class Litecache < Store
-      prepend Strategy::LocalCache
+      #prepend Strategy::LocalCache
 
       def self.supports_cache_versioning?
         true
@@ -25,17 +25,22 @@ module ActiveSupport
         options = merged_options(options)
         # todo: fix me
         # this is currently a hack to avoid dealing with Rails cache encoding and decoding
-        # @cache.transaction(:immediate) do
+        # and it can result in a race condition as it stands
+        # @cache.transaction(:immediate) do 
+          # currently transactions are not compatible with acquiring connections 
+          # this needs fixing by storing the connection to the context once acquired
         if (value = read(key, options))
           value = value.to_i + amount
           write(key, value, options)
+        else
+          write(key, amount, options)
         end
         # end
       end
 
       def decrement(key, amount = 1, options = nil)
         options = merged_options(options)
-        increment(key, -1 * amount, options[:expires_in])
+        increment(key, -1 * amount, options)
       end
 
       def prune(limit = nil, time = nil)
@@ -46,7 +51,7 @@ module ActiveSupport
         @cache.prune(limit)
       end
 
-      def clear
+      def clear(options=nil)
         @cache.clear
       end
 
@@ -73,10 +78,28 @@ module ActiveSupport
         deserialize_entry(@cache.get(key))
       end
 
+      def read_multi_entries(names, **options)
+        results = {}
+        return results if names == []
+        rs = @cache.get_multi(*names.flatten)
+        rs.each_pair{|k, v| results[k] = deserialize_entry(v).value }
+        results
+      end
+      
       # Write an entry to the cache.
       def write_entry(key, entry, **options)
         write_serialized_entry(key, serialize_entry(entry, **options), **options)
       end
+      
+      def write_multi_entries(entries, **options)
+        return if entries.empty?
+        entries.each_pair {|k,v| entries[k] = serialize_entry(v, **options)}
+        expires_in = options[:expires_in].to_i
+        if options[:race_condition_ttl] && expires_in > 0 && !options[:raw]
+          expires_in += 5.minutes
+        end
+        @cache.set_multi(entries, expires_in)        
+      end
 
       def write_serialized_entry(key, payload, **options)
         expires_in = options[:expires_in].to_i

data/lib/generators/litestack/install/templates/cable.yml

@@ -4,8 +4,5 @@ development:
 test:
   adapter: test
 
-staging:
-  adapter: litecable
-
 production:
   adapter: litecable

data/lib/litestack/litecache.rb

@@ -26,7 +26,7 @@ class Litecache
   #   expiry: 60 * 60 * 24 * 30 -> one month default expiry if none is provided
   #   size: 128 * 1024 * 1024 -> 128MB
   #   mmap_size: 128 * 1024 * 1024 -> 128MB to be held in memory
-  #   min_size: 32 * 1024 -> 32MB
+  #   min_size: 32 * 1024 -> 32KB
   #   return_full_record: false -> only return the payload
   #   sleep_interval: 1 -> 1 second of sleep between cleanup runs
 
@@ -39,7 +39,7 @@ class Litecache
     mmap_size: 128 * 1024 * 1024, # 128MB
     min_size: 8 * 1024 * 1024, # 16MB
     return_full_record: false, # only return the payload
-    sleep_interval: 1, # 1 second
+    sleep_interval: 30, # 30 seconds
     metrics: false
   }
 
@@ -63,15 +63,15 @@ class Litecache
 
   def initialize(options = {})
     options[:size] = DEFAULT_OPTIONS[:min_size] if options[:size] && options[:size] < DEFAULT_OPTIONS[:min_size]
-    @last_visited = {}
     init(options)
+    @expires_in = @options[:expiry] || 60 * 60 * 24 * 30
     collect_metrics if @options[:metrics]
   end
 
   # add a key, value pair to the cache, with an optional expiry value (number of seconds)
   def set(key, value, expires_in = nil)
     key = key.to_s
-    expires_in = @options[:expires_in] if expires_in.nil? || expires_in.zero?
+    expires_in ||= @expires_in 
     @conn.acquire do |cache|
       cache.stmts[:setter].execute!(key, value, expires_in)
       capture(:set, key)
@@ -82,11 +82,30 @@ class Litecache
     end
     true
   end
+  
+  # set multiple keys and values in one shot set_multi({k1: v1, k2: v2, ... })
+  def set_multi(keys_and_values, expires_in = nil)
+    expires_in ||= @expires_in 
+    transaction do |conn|
+      keys_and_values.each_pair do |k, v|
+        begin
+          key = k.to_s
+          conn.stmts[:setter].execute!(key, v, expires_in)
+          capture(:set, key)
+        rescue SQLite3::FullException
+          conn.stmts[:extra_pruner].execute!(0.2)
+          conn.execute("vacuum")
+          retry
+        end
+      end
+    end
+    true    
+  end 
 
   # add a key, value pair to the cache, but only if the key doesn't exist, with an optional expiry value (number of seconds)
   def set_unless_exists(key, value, expires_in = nil)
     key = key.to_s
-    expires_in = @options[:expires_in] if expires_in.nil? || expires_in.zero?
+    expires_in ||= @expires_in 
     changes = 0
     @conn.acquire do |cache|
       cache.transaction(:immediate) do
@@ -107,13 +126,30 @@ class Litecache
   def get(key)
     key = key.to_s
     if (record = @conn.acquire { |cache| cache.stmts[:getter].execute!(key)[0] })
-      @last_visited[key] = true
       capture(:get, key, 1)
       return record[1]
     end
     capture(:get, key, 0)
     nil
   end
+  
+  # get multiple values by their keys, a hash with values corresponding to the keys
+  # is returned,
+  def get_multi(*keys)
+    results = {}
+    transaction(:deferred) do |conn|
+      keys.length.times do |i|
+        key = keys[i].to_s
+        if (record = conn.stmts[:getter].execute!(key)[0]) 
+          results[keys[i]] = record[1] # use the original key format
+          capture(:get, key, 1)
+        else
+          capture(:get, key, 0) 
+        end
+      end
+    end
+    results
+  end
 
   # delete a key, value pair from the cache
   def delete(key)
@@ -127,7 +163,7 @@ class Litecache
 
   # increment an integer value by amount, optionally add an expiry value (in seconds)
   def increment(key, amount, expires_in = nil)
-    expires_in ||= @expires_in
+    expires_in ||= @expires_in 
     @conn.acquire { |cache| cache.stmts[:incrementer].execute!(key.to_s, amount, expires_in) }
   end
 
@@ -189,11 +225,14 @@ class Litecache
   end
 
   # low level access to SQLite transactions, use with caution
-  def transaction(mode, acquire = true)
-    return cache.transaction(mode) { yield } unless acquire
+  def transaction(mode=:immediate)
     @conn.acquire do |cache|
-      cache.transaction(mode) do
+      if cache.transaction_active?
         yield
+      else
+        cache.transaction(mode) do
+          yield cache
+        end
       end
     end
   end
@@ -209,17 +248,12 @@ class Litecache
     Litescheduler.spawn do
       while @running
         @conn.acquire do |cache|
-          cache.transaction(:immediate) do
-            @last_visited.delete_if do |k| # there is a race condition here, but not a serious one
-              cache.stmts[:toucher].execute!(k) || true
-            end
-            cache.stmts[:pruner].execute!
-          end
+          cache.stmts[:pruner].execute!
         rescue SQLite3::BusyException
           retry
         rescue SQLite3::FullException
           cache.stmts[:extra_pruner].execute!(0.2)
-        rescue Exception # standard:disable Lint/RescueException
+        rescue Exception => e # standard:disable Lint/RescueException
           # database is closed
         end
         sleep @options[:sleep_interval]

data/lib/litestack/litecache.sql.yml

@@ -3,10 +3,10 @@ schema:
     create_table_data: >
       CREATE TABLE IF NOT EXISTS data(
         id TEXT PRIMARY KEY,
-        value TEXT,
+        value ANY,
         expires_in INTEGER,
         last_used INTEGER
-      );
+      ) STRICT;
     create_expiry_index: >
       CREATE INDEX IF NOT EXISTS expiry_index ON data (expires_in);
     create_last_used_index: >
@@ -14,7 +14,7 @@ schema:
 
 stmts:
   pruner: >
-    DELETE FROM data WHERE expires_in <= $1;
+    DELETE FROM data WHERE expires_in <= unixepoch('now');
 
   extra_pruner: >
     DELETE FROM data WHERE id IN (
@@ -49,7 +49,7 @@ stmts:
       value = EXCLUDED.value,
       last_used = EXCLUDED.last_used,
       expires_in = EXCLUDED.expires_in;
-
+  
   inserter: >
     INSERT INTO data (id, value, expires_in, last_used)
     VALUES ($1, $2, unixepoch('now') + $3, unixepoch('now'))
@@ -65,7 +65,7 @@ stmts:
     SELECT id FROM data WHERE id = $1;
   
   getter: >
-    SELECT id, value, expires_in FROM data WHERE id = $1;
+    SELECT id, value, expires_in FROM data WHERE id = $1 AND expires_in >= unixepoch('now');
   
   deleter: >
     delete FROM data WHERE id = $1 RETURNING value;
@@ -79,6 +79,8 @@ stmts:
       last_used = EXCLUDED.last_used,
       expires_in = EXCLUDED.expires_in;
   
+      
+  
   counter: >
     SELECT count(*) FROM data;
   

data/lib/litestack/litemetric.sql.yml

@@ -19,7 +19,7 @@ schema:
         created_at INTEGER DEFAULT((unixepoch()/300*300)) NOT NULL,
         resolution TEXT DEFAULT('minute') NOT NULL,
         PRIMARY KEY(resolution, created_at, topic, name, key)
-      ) STRICT;
+      );
 
     create_topic_index_on_events: >
       CREATE INDEX events_topic_index ON events (resolution, created_at, topic) WHERE name = '___';

data/lib/litestack/litemetric_collector.sql.yml

@@ -12,7 +12,7 @@ schema:
         created_at INTEGER DEFAULT((unixepoch()/300*300)) NOT NULL ON CONFLICT REPLACE,
         resolution TEXT DEFAULT('minute') NOT NULL,
         PRIMARY KEY(resolution, created_at, topic, name, key)
-      ) STRICT;
+      );
 
 stmts:
   capture_event: >

data/lib/litestack/litescheduler.rb

@@ -29,7 +29,7 @@ module Litescheduler
   end
 
   def self.storage
-    if backend == :fiber || backend == :poylphony
+    if fiber_backed?
       Fiber.current.storage
     else
       Thread.current
@@ -37,7 +37,7 @@ module Litescheduler
   end
 
   def self.current
-    if backend == :fiber || backend == :poylphony
+    if fiber_backed?
       Fiber.current
     else
       Thread.current
@@ -64,7 +64,7 @@ module Litescheduler
   # they must send (true) as a parameter to this method
   # else it is a no-op for fibers
   def self.synchronize(fiber_sync = false, &block)
-    if (backend == :fiber) || (backend == :polyphony)
+    if fiber_backed?
       yield # do nothing, just run the block as is
     else
       mutex.synchronize(&block)
@@ -72,7 +72,7 @@ module Litescheduler
   end
 
   def self.max_contexts
-    return 50 if backend == :fiber || backend == :polyphony
+    return 50 if fiber_backed?
     5
   end
 
@@ -81,4 +81,10 @@ module Litescheduler
     # a single mutex per process (is that ok?)
     @@mutex ||= Mutex.new
   end
+
+  private
+
+  def self.fiber_backed?
+    backend == :fiber || backend == :polyphony
+  end
 end

data/lib/litestack/litesupport.rb

@@ -196,7 +196,7 @@ module Litesupport
     end
 
     def setup
-      @conn = create_pooled_connection
+      @conn = create_pooled_connection(@options[:connection_count])
       @logger = create_logger
       @running = true
     end
@@ -231,7 +231,8 @@ module Litesupport
     end
 
     def create_pooled_connection(count = 1)
-      Litesupport::Pool.new(1) { create_connection }
+      count = 1 unless count and count.is_a? Integer
+      Litesupport::Pool.new(count) { create_connection }
     end
 
     # common db object options

data/lib/litestack/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Litestack
-  VERSION = "0.4.2"
+  VERSION = "0.4.3"
 end

data/lib/railties/rails/commands/dbconsole.rb

@@ -27,7 +27,7 @@ module Rails
 
         args << db_config.database
 
-        find_cmd_and_exec(["mysql", "mysql5"], *args)
+        ActiveRecord::Base.connection.class.find_cmd_and_exec(["mysql", "mysql5"], *args)
 
       when /^postgres|^postgis/
         ENV["PGUSER"] = config[:username] if config[:username]
@@ -38,7 +38,7 @@ module Rails
         ENV["PGSSLCERT"] = config[:sslcert].to_s if config[:sslcert]
         ENV["PGSSLKEY"] = config[:sslkey].to_s if config[:sslkey]
         ENV["PGSSLROOTCERT"] = config[:sslrootcert].to_s if config[:sslrootcert]
-        find_cmd_and_exec("psql", db_config.database)
+        ActiveRecord::Base.connection.class.find_cmd_and_exec("psql", db_config.database)
 
       when "sqlite3", "litedb"
         args = []
@@ -47,7 +47,7 @@ module Rails
         args << "-header" if @options[:header]
         args << File.expand_path(db_config.database, Rails.respond_to?(:root) ? Rails.root : nil)
 
-        find_cmd_and_exec("sqlite3", *args)
+        ActiveRecord::Base.connection.class.find_cmd_and_exec("sqlite3", *args)
 
       when "oracle", "oracle_enhanced"
         logon = ""
@@ -58,7 +58,7 @@ module Rails
           logon << "@#{db_config.database}" if db_config.database
         end
 
-        find_cmd_and_exec("sqlplus", logon)
+        ActiveRecord::Base.connection.class.find_cmd_and_exec("sqlplus", logon)
 
       when "sqlserver"
         args = []
@@ -73,7 +73,7 @@ module Rails
           args += ["-S", host_arg]
         end
 
-        find_cmd_and_exec("sqlcmd", *args)
+        ActiveRecord::Base.connection.class.find_cmd_and_exec("sqlcmd", *args)
 
       else
         abort "Unknown command-line client for #{db_config.database}."

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: litestack
 version: !ruby/object:Gem::Version
-  version: 0.4.2
+  version: 0.4.3
 platform: ruby
 authors:
 - Mohamed Hassan
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2023-11-11 00:00:00.000000000 Z
+date: 2024-02-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: sqlite3
@@ -178,6 +178,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: sequel
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: 
 email:
 - oldmoe@gmail.com
@@ -190,7 +204,6 @@ files:
 - BENCHMARKS.md
 - CHANGELOG.md
 - Gemfile
-- Gemfile.lock
 - LICENSE.txt
 - README.md
 - ROADMAP.md
@@ -296,7 +309,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.20
+rubygems_version: 3.5.3
 signing_key: 
 specification_version: 4
 summary: A SQLite based, lightning fast, super efficient and dead simple to setup

data/Gemfile.lock

@@ -1,92 +0,0 @@
-PATH
-  remote: .
-  specs:
-    litestack (0.4.1)
-      erubi
-      hanami-router
-      oj
-      rack
-      sqlite3
-      tilt
-
-GEM
-  remote: https://rubygems.org/
-  specs:
-    ast (2.4.2)
-    docile (1.4.0)
-    erubi (1.12.0)
-    hanami-router (0.6.2)
-      hanami-utils (~> 0.7)
-      http_router (~> 0.11)
-    hanami-utils (0.9.2)
-    http_router (0.11.2)
-      rack (>= 1.0.0)
-      url_mount (~> 0.2.1)
-    json (2.6.3)
-    language_server-protocol (3.17.0.3)
-    lint_roller (1.1.0)
-    minitest (5.19.0)
-    oj (3.15.1)
-    parallel (1.23.0)
-    parser (3.2.2.3)
-      ast (~> 2.4.1)
-      racc
-    racc (1.7.1)
-    rack (3.0.8)
-    rainbow (3.1.1)
-    rake (13.0.6)
-    regexp_parser (2.8.1)
-    rexml (3.2.6)
-    rubocop (1.52.1)
-      json (~> 2.3)
-      parallel (~> 1.10)
-      parser (>= 3.2.2.3)
-      rainbow (>= 2.2.2, < 4.0)
-      regexp_parser (>= 1.8, < 3.0)
-      rexml (>= 3.2.5, < 4.0)
-      rubocop-ast (>= 1.28.0, < 2.0)
-      ruby-progressbar (~> 1.7)
-      unicode-display_width (>= 2.4.0, < 3.0)
-    rubocop-ast (1.29.0)
-      parser (>= 3.2.1.0)
-    rubocop-performance (1.18.0)
-      rubocop (>= 1.7.0, < 2.0)
-      rubocop-ast (>= 0.4.0)
-    ruby-progressbar (1.13.0)
-    simplecov (0.22.0)
-      docile (~> 1.1)
-      simplecov-html (~> 0.11)
-      simplecov_json_formatter (~> 0.1)
-    simplecov-html (0.12.3)
-    simplecov_json_formatter (0.1.4)
-    sqlite3 (1.6.3-arm64-darwin)
-    standard (1.30.1)
-      language_server-protocol (~> 3.17.0.2)
-      lint_roller (~> 1.0)
-      rubocop (~> 1.52.0)
-      standard-custom (~> 1.0.0)
-      standard-performance (~> 1.1.0)
-    standard-custom (1.0.2)
-      lint_roller (~> 1.0)
-      rubocop (~> 1.50)
-    standard-performance (1.1.2)
-      lint_roller (~> 1.1)
-      rubocop-performance (~> 1.18.0)
-    tilt (2.2.0)
-    unicode-display_width (2.4.2)
-    url_mount (0.2.1)
-      rack
-
-PLATFORMS
-  arm64-darwin-21
-
-DEPENDENCIES
-  litestack!
-  minitest (~> 5.0)
-  rack (~> 3.0)
-  rake (~> 13.0)
-  simplecov
-  standard (~> 1.3)
-
-BUNDLED WITH
-   2.4.8