litestack 0.4.2 → 0.4.4

Sign up to get free protection for your applications and to get access to all the features.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/.standard.yml +3 -1
  3. data/BENCHMARKS.md +3 -3
  4. data/CAVEATS.md +20 -0
  5. data/CHANGELOG.md +40 -1
  6. data/FILESYSTEMS.md +55 -0
  7. data/Gemfile +2 -0
  8. data/README.md +8 -4
  9. data/ROADMAP.md +6 -6
  10. data/assets/litestack_advantage.png +0 -0
  11. data/bench/bench.rb +2 -0
  12. data/bench/bench_cache_rails.rb +33 -2
  13. data/bench/bench_cache_raw.rb +36 -12
  14. data/bench/bench_jobs_rails.rb +3 -3
  15. data/bench/bench_jobs_raw.rb +3 -3
  16. data/bin/liteboard +16 -13
  17. data/gemfiles/rails70.gemfile +5 -0
  18. data/gemfiles/rails71.gemfile +5 -0
  19. data/gemfiles/rails71.gemfile.lock +264 -0
  20. data/lib/active_job/queue_adapters/litejob_adapter.rb +11 -3
  21. data/lib/active_record/connection_adapters/litedb_adapter.rb +8 -0
  22. data/lib/active_support/cache/litecache.rb +40 -7
  23. data/lib/generators/litestack/install/install_generator.rb +2 -2
  24. data/lib/generators/litestack/install/templates/cable.yml +0 -3
  25. data/lib/litestack/liteboard/liteboard.rb +15 -19
  26. data/lib/litestack/liteboard/views/litecable.erb +1 -1
  27. data/lib/litestack/litecable.rb +1 -1
  28. data/lib/litestack/litecache.rb +51 -19
  29. data/lib/litestack/litecache.sql.yml +7 -5
  30. data/lib/litestack/litedb.rb +5 -1
  31. data/lib/litestack/litejob.rb +1 -1
  32. data/lib/litestack/litejobqueue.rb +24 -14
  33. data/lib/litestack/litemetric.rb +7 -6
  34. data/lib/litestack/litemetric.sql.yml +1 -1
  35. data/lib/litestack/litemetric_collector.sql.yml +1 -1
  36. data/lib/litestack/litequeue.rb +17 -2
  37. data/lib/litestack/litequeue.sql.yml +38 -5
  38. data/lib/litestack/litescheduler.rb +9 -4
  39. data/lib/litestack/litesearch/index.rb +11 -10
  40. data/lib/litestack/litesearch/model.rb +61 -3
  41. data/lib/litestack/litesearch/schema.rb +7 -2
  42. data/lib/litestack/litesearch/schema_adapters/backed_adapter.rb +69 -25
  43. data/lib/litestack/litesearch/schema_adapters.rb +4 -4
  44. data/lib/litestack/litesearch.rb +2 -2
  45. data/lib/litestack/litesupport.rb +9 -7
  46. data/lib/litestack/railtie.rb +4 -2
  47. data/lib/litestack/version.rb +1 -1
  48. data/lib/litestack.rb +15 -15
  49. data/lib/railties/rails/commands/dbconsole.rb +5 -5
  50. data/lib/sequel/adapters/litedb.rb +9 -1
  51. data/lib/sequel/adapters/shared/litedb.rb +2 -2
  52. data/scripts/build_metrics.rb +2 -2
  53. data/scripts/test_cable.rb +1 -1
  54. metadata +105 -56
  55. data/Gemfile.lock +0 -92
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 98a82dcbe7e4fe0b215e254fac5791127142fcbfd4a156b2509434f97b58aea9
4
- data.tar.gz: a914b64d6e031559b1bdca83a8198426a49005f56ca1d0b4da2a7c6cb2384ab7
3
+ metadata.gz: ddcc1a0b4d5a5547b3e3deec7ce5abec68e31ffce348913867368513276ac69a
4
+ data.tar.gz: 743a21e710d5351131a2cbe50a94de13653a034b88d08b82e470bb3aac2bdfbb
5
5
  SHA512:
6
- metadata.gz: 72935d15fcced99bcc0da9cfb6c1ffcd43478a5456c073a3d8ab468a3e49f7c681bbdec861cb7c1e2c16659e0d11429e16d855be38fd30735448078977205dc9
7
- data.tar.gz: ac181213e66533926ce5d075e8256dcf2c5dfa727ff7becaa244bbf053f354c564f29a294f4b2b8c1436320d41e9dc4e4697ba9e3169324bc798c60d8bfc762a
6
+ metadata.gz: f54bdc732995e8083791c6b214ec6058bb439548ffdcbb14f33fbbeadbca870f4941cff558cdcbfa26212e53080534c12bfacfdfa2ba8ca689f380679ef0504a
7
+ data.tar.gz: f0a1fe1c95a592e69175e55ad212e351d5bca907b43c4215a4eca919d8dc2dfda68d80fa0ceb74da5909bf822adf0b2ea45e9108fcf0cd995a0a4b337a908ad0
data/.standard.yml CHANGED
@@ -1,3 +1,5 @@
1
1
  ignore:
2
2
  - 'test/**/*':
3
- - Style/GlobalVars
3
+ - Style/GlobalVars
4
+
5
+ ruby_version: 3.0
data/BENCHMARKS.md CHANGED
@@ -125,11 +125,11 @@ On average, Litecable is quite faster than the Redis based version and offers be
125
125
 
126
126
  > ![litesearch](https://github.com/oldmoe/litestack/blob/master/assets/litesearch_logo_teal.png?raw=true)
127
127
 
128
- Litesearch was benchmarked against Mielsearch, both using their respective ActiveRecord integrations. Mielsearch was running on the same machine as the benchmark script and was using the default configuration options. The dataset used for testing was the infamous Enron email corpus. Redisearch was not benchmarked due to the clients being not Rails 7.1 compatible (yet), will probably bench Redisearch when they are.
128
+ Litesearch was benchmarked against Meilisearch, both using their respective ActiveRecord integrations. Meilisearch was running on the same machine as the benchmark script and was using the default configuration options. The dataset used for testing was the infamous Enron email corpus. Redisearch was not benchmarked due to the clients being not Rails 7.1 compatible (yet), will probably bench Redisearch when they are.
129
129
 
130
130
  ### Building the index
131
131
 
132
- ||MieliSearch|Litesearch|
132
+ ||Meilisearch|Litesearch|
133
133
  |-:|-:|-:|
134
134
  |Time to insert 10K docs|265.42 seconds|29.06 seconds|
135
135
  |Inserted docs/second|38|344|
@@ -137,4 +137,4 @@ Litesearch was benchmarked against Mielsearch, both using their respective Activ
137
137
  |Searches/second|133|19608|
138
138
  |Index rebuild|0.822|0.626|
139
139
 
140
- We only limited the test to 10K documents becuause MieliSearch was taking a long time to index, so we decided to stop at a reasonable sample size. The search numbers for litesearch were double checked, event against a 100K document set and they remained virtually the same. It is clear that litesearch is much faster than MieliSearch in both indexing and searching, this could be partially attributed to litesearch being a simpler text search engine, but still, the difference is huge! For rebuilding the index though, Litesearch is not that much faster than Mielisearch.
140
+ We only limited the test to 10K documents becuause Meilisearch was taking a long time to index, so we decided to stop at a reasonable sample size. The search numbers for litesearch were double checked, event against a 100K document set and they remained virtually the same. It is clear that litesearch is much faster than Meilisearch in both indexing and searching, this could be partially attributed to litesearch being a simpler text search engine, but still, the difference is huge! For rebuilding the index though, Litesearch is not that much faster than Meilisearch.
data/CAVEATS.md ADDED
@@ -0,0 +1,20 @@
1
+ # CAVEATS
2
+
3
+ This is a document that lists some caveats that you need to consider when dealing with Litestack and SQLite applications
4
+
5
+ ## Single host (server) only
6
+
7
+ By design, SQLite can be accessed only (in a performant and safe way) from a single host machine. That means that you cannot have things like auto-scaling of app servers since there is but a single one of them. This app server can be a huge one, with many processes, but only one app server at a time.
8
+
9
+ ## Containerization becomes a bit tricky
10
+
11
+ Container technology was originally designed for stateless applications. Based on this, one of the core aspects of containers is that they are immutable, deploying an app to container means destroying one container and creating another. With statefull applications, like database servers and similarly SQLite based applications, this will potentially mean a down time will be perceived, which in some approaches is mitigate but not perfectly (yet). Expect some quirks when dealing with SQLite and containers.
12
+
13
+ ## Single writer / multi reader
14
+
15
+ In its default configuration, Litestack allows one writer at a time to write to the database file, while simultaneously allowing any number of readers to access the database. Generally SQLite is pretty fast in writes, but you should be aware that a long running write operation will prevent any other writers from proceeding, for example creating an index on a very large table will stop other writers until the index creation is finished. There is no concurrent index creation facility in SQLite, yet.
16
+
17
+ ## Litestack does not release the GVL
18
+
19
+ This also applies to the Ruby SQLite3 gem, when it is performing a query, it will not allow any other threads in the Ruby process to resume until it returns. Care should be taken when writing low latency multi-threaded applications with Litestack, it is generally recommended to use fibers over threads in that case, since everything is serialized anyway it makes sense to avoid the ovreheads of mult-threaded environments
20
+
data/CHANGELOG.md CHANGED
@@ -1,28 +1,59 @@
1
1
  ## [Unreleased]
2
2
 
3
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.4...master)
4
+
5
+ ## [0.4.4] - 2024-05-21
6
+
7
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.3...v0.4.4)
8
+ - Fix a table not defined bug for Litesearch
9
+ - Add conditional mapping of external fields to Litesearch (backed index)
10
+ - As a consequence of the above, support AR polymorphic associations
11
+ - As a consequence of the above, support indexing and searching for ActionText attributes
12
+ - Remove 'hanami-router' as a dependency, rely on vanilla Rack for Liteboard
13
+ - All configuratin .yml files now support ERB rendering
14
+ - Fix adapter registration with Rails 7.2
15
+ - Ground work for more durable job persistence in case of a process crash
16
+ - Many minor fixes
17
+
18
+ ## [0.4.3] - 2024-02-15
19
+
20
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.2...v0.4.3)
21
+ - Add "sequel" as a development dependency
22
+ - Diff links in CHANGELOG (thanks Weston Ganger)
23
+ - Fix daemonize type in liteboard (thanks Julian Rubisch)
24
+ - Better Litecache schema (streamlined numeric value support)
25
+ - Support for set_multi and get_multi in Litecache (read_multi and write_multi support for Rails Cache store)
26
+ - More tests written for Litecache and Rails Litecache store
27
+ - Experimenting with removing the Rails LocalCache as it doesn't show enough improvement in performance to compensate for the memory overhead
28
+ - Switch Litecache to a FIFO eviction model vs LRU (thanks Julian Rubisch and Stephen Margheim)
29
+
3
30
  ## [0.4.2] - 2023-11-11
4
31
 
32
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.1...v0.4.2)
5
33
  - Add similarity search support for Litesearch (works best for non-trigram indexes)
6
34
  - Enable similarity search for ActiveRecord and Sequel models
7
35
  - Fix Litesearch tests
8
36
  - Suppress chatty Litejob exit detector when there are no jobs in flight
9
37
  - Tidy up the test folder
10
- - [#41](https://github.com/oldmoe/litestack/pull/41) - Fix bug in Litecable where the `connected` event was not getting propogated
38
+ - [#41](https://github.com/oldmoe/litestack/pull/41) - Fix bug in Litecable where the `connected` event was not getting propagated
11
39
  - Add Litemetric and Liteboard info to README.doc
12
40
  - Fix the testing rake task
13
41
 
14
42
  ## [0.4.1] - 2023-10-11
15
43
 
44
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.4.0...v0.4.1)
16
45
  - Add missing Litesearch::Model dependency
17
46
 
18
47
  ## [0.4.0] - 2023-10-11
19
48
 
49
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.3.0...v0.4.0)
20
50
  - Introduced Litesearch, dynamic & fast full text search capability for Litedb
21
51
  - ActiveRecord and Sequel integration for Litesearch
22
52
  - Slight improvement to the Sequel Litedb adapter for better Litesearch integration
23
53
 
24
54
  ## [0.3.0] - 2023-08-13
25
55
 
56
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.6...v0.3.0)
26
57
  - Reworked the Litecable thread safety model
27
58
  - Fixed multiple litejob bugs (thanks Stephen Margheim)
28
59
  - Fixed Railtie dependency (thanks Marco Roth)
@@ -32,6 +63,7 @@
32
63
 
33
64
  ## [0.2.6] - 2023-07-16
34
65
 
66
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.3...v0.2.6)
35
67
  - Much improved database location setting (thanks Brad Gessler)
36
68
  - A Rails generator for better Rails Litestack defaults (thanks Brad Gessler)
37
69
  - Revamped Litemetric, now much faster and more accurate (still experimental)
@@ -39,19 +71,23 @@
39
71
 
40
72
  ## [0.2.3] - 2023-05-20
41
73
 
74
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.2...v0.2.3)
42
75
  - Cut back on options defined in the Litejob Rails adapter
43
76
 
44
77
  ## [0.2.2] - 2023-05-18
45
78
 
79
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.1...v0.2.2)
46
80
  - Fix default queue location in Litejob
47
81
 
48
82
 
49
83
  ## [0.2.1] - 2023-05-08
50
84
 
85
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.2.0...v0.2.1)
51
86
  - Fix a race condition in Litecable
52
87
 
53
88
  ## [0.2.0] - 2023-05-08
54
89
 
90
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.8...v0.2.0)
55
91
  - Litecable, a SQLite driver for ActionCable
56
92
  - Litemetric for metrics collection support (experimental, disabled by default)
57
93
  - New schema for Litejob, old jobs are auto-migrated
@@ -61,6 +97,7 @@
61
97
 
62
98
  ## [0.1.8] - 2023-03-08
63
99
 
100
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.7...v0.1.8)
64
101
  - More code cleanups, more test coverage
65
102
  - Retry support for jobs in Litejob
66
103
  - Job storage and garbage collection for failed jobs
@@ -69,6 +106,7 @@
69
106
 
70
107
  ## [0.1.7] - 2023-03-05
71
108
 
109
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.6...v0.1.7)
72
110
  - Code cleanup, removal of references to older name
73
111
  - Fix for the litedb rake tasks (thanks: netmute)
74
112
  - More fixes for the new concurrency model
@@ -76,6 +114,7 @@
76
114
 
77
115
  ## [0.1.6] - 2023-03-03
78
116
 
117
+ - [View Diff](https://github.com/oldmoe/litestack/compare/v0.1.0...v0.1.6)
79
118
  - Revamped the locking model, more robust, minimal performance hit
80
119
  - Introduced a new resource pooling class
81
120
  - Litecache and Litejob now use the resource pool
data/FILESYSTEMS.md ADDED
@@ -0,0 +1,55 @@
1
+ # Filesystems & Litestack
2
+
3
+ In the containerized world we live in, many layers of the hardware/software stacks are far abstracted that we no longer know they exist. For the filesystem enthusiasts out there this is a quick overview of how Litestack (and hence SQLite) can benefit from different filesystem
4
+
5
+ ## XFS
6
+
7
+ A very stable and trusted filesystem with excellent performance characteristics
8
+
9
+ - Fast reads / writes
10
+
11
+ ## EXT4
12
+
13
+ Another stable and performant filesystem
14
+
15
+ - Fast reads / writes
16
+
17
+ ## F2FS
18
+
19
+ Specially built for solid state storage, has an atomic write mode that is supported by SQLite
20
+
21
+ - Fast reads
22
+ - Reasonably fast durable writes in synchronous mode
23
+ - Compression (not very useful)
24
+
25
+ ## ZFS
26
+
27
+ Copy-on-write filesystem with a nifty set of features, very fast snapshotting but only for the FS as a whole, can send incremental snapshots to another host for backup. Naturally a lot more pages are written on write operations thus it is a bit slower in writes.
28
+
29
+ - Fast reads
30
+ - Slower writes
31
+ - FS Snapshotting with send/recv
32
+ - fast device cache
33
+ - Compression
34
+
35
+ ## Btrfs
36
+
37
+ Another CoW filesystem that delivers snapshotting and incremental send/recv at a much granular level.
38
+
39
+ - Fast reads
40
+ - Slower writes
41
+ - Fast copies (free backups)
42
+ - Sub volume Snapshotting with send/recv
43
+ - Compression
44
+
45
+ ## Bcachefs
46
+
47
+ A new CoW filesystem built on the foundations of the bcache module. Improving rapidly but still lacks a few of the common CoW fs features
48
+
49
+ - Fast reads
50
+ - Slower writes
51
+ - fast device cache
52
+
53
+
54
+
55
+
data/Gemfile CHANGED
@@ -4,3 +4,5 @@ source "https://rubygems.org"
4
4
 
5
5
  # Specify your gem's dependencies in litestack.gemspec
6
6
  gemspec
7
+
8
+ # gem "rails"
data/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  ![litestack](https://github.com/oldmoe/litestack/blob/master/assets/litestack_logo_teal_large.png?raw=true)
2
2
 
3
+ <a href="https://badge.fury.io/rb/litestack" target="_blank"><img height="21" style='border:0px;height:21px;' border='0' src="https://badge.fury.io/rb/litestack.svg" alt="Gem Version"></a>
4
+ <a href='https://github.com/oldmoe/litestack/actions' target='_blank'><img src="https://github.com/oldmoe/litestack/actions/workflows/ruby.yml/badge.svg?branch=master" style="max-width:100%;" height='21' style='border:0px;height:21px;' border='0' alt="CI Status"></a>
5
+ <a href='https://rubygems.org/gems/litestack' target='_blank'><img height='21' style='border:0px;height:21px;' src='https://img.shields.io/gem/dt/litestack?color=brightgreen&label=Rubygems%20Downloads' border='0' alt='RubyGems Downloads' /></a>
6
+
3
7
  All your data infrastructure, in a gem!
4
8
 
5
9
  Litestack is a Ruby gem that provides both Ruby and Ruby on Rails applications an all-in-one solution for web application data infrastructure. It exploits the power and embeddedness of SQLite to deliver a full-fledged SQL database, a fast cache , a robust job queue, a reliable message broker, a full text search engine and a metrics platform all in a single package.
@@ -24,7 +28,7 @@ With litestack you only need to add a single gem to your app which would replace
24
28
  - Cache Server (e.g. Redis, Memcached)
25
29
  - Job Processor (e.g. Sidekiq, Goodjob)
26
30
  - Pubsub Server (e.g. Redis, PostgreSQL)
27
- - Fulltext Search Server (e.g. Elasticsearch, Mielisearch)
31
+ - Fulltext Search Server (e.g. Elasticsearch, Meilisearch)
28
32
 
29
33
  To make it even more efficient, litestack will detect the presence of Fiber based IO frameworks like Async (e.g. when you use the Falcon web server) or Polyphony. It will then switch its background workers for caches and queues to fibers (using the semantics of the existing framework). This is done transparently and will generally lead to lower CPU and memory utilization.
30
34
 
@@ -55,7 +59,7 @@ litedb is a wrapper around SQLite3, offering a better default configuration that
55
59
 
56
60
  #### Direct litedb usage
57
61
 
58
- litedb can be used exactly as the SQLite3 gem, since litedb iherits from SQLite3
62
+ litedb can be used exactly as the SQLite3 gem, since litedb inherits from SQLite3
59
63
 
60
64
  ```ruby
61
65
  require 'litestack'
@@ -205,7 +209,7 @@ idx.search('kamal')
205
209
  idx.search('subject: awa')
206
210
  ```
207
211
 
208
- Litesearch integrates tightly with ActiveRecord ans Sequel, here are integration examples
212
+ Litesearch integrates tightly with ActiveRecord and Sequel, here are integration examples
209
213
 
210
214
  #### ActiveRecord
211
215
 
@@ -258,7 +262,7 @@ Book.search('author: writer').limit(1).all
258
262
  > ![litemetric](https://github.com/oldmoe/litestack/blob/master/assets/litemetric_logo_teal.png?raw=true)
259
263
 
260
264
  ### Litemetric
261
- Litestack comes with a mdoule that can collect useful metrics for its different components, in each component, you need to add the following to the respective .yml file (database.yml in case of Litedb)
265
+ Litestack comes with a module that can collect useful metrics for its different components, in each component, you need to add the following to the respective .yml file (database.yml in case of Litedb)
262
266
  ```yml
263
267
  metrics: true # default is false
264
268
  ```
data/ROADMAP.md CHANGED
@@ -21,25 +21,25 @@ This is a list of functional/non-functional features that litestack targets to i
21
21
  - [ ] Automated testing matrix
22
22
  - [ ] Automated gem builds
23
23
  - [ ] Litedb improvements
24
- - [ ] ActiveRecord Sqlite3 Adapter compatibility mode
24
+ - [ ] ActiveRecord Sqlite3 Adapter compatibility mode
25
25
  - [ ] Sequel Sqlite3 Adapter compatibility mode
26
- - [ ] Extension bundling/loading
26
+ - [ ] Extension bundling/loading
27
27
  - [ ] Liteiob improvements
28
28
  - [ ] Persist jobs even during execution
29
29
  - [ ] Zombie job detection
30
- - [ ] Better process exit handling
30
+ - [ ] Better process exit handling
31
31
  - [ ] Faster job dispatch for the no delay case
32
32
  - [ ] Database maintenance scripts
33
33
  - [ ] Online backup
34
34
  - [ ] Restore
35
35
  - [ ] Litestream integration
36
- - [ ] CoW filesystems specific backup path
36
+ - [ ] CoW filesystem-specific backup path
37
37
  - [ ] Zero downtime deployment scripts
38
38
  - [ ] Rails
39
- - [ ] Genereic Rack Applications
39
+ - [ ] Generic Rack Applications
40
40
  - [ ] Litemetric improvements
41
41
  - [ ] Rails performance module
42
42
  - [ ] Ruby Memory/GC module
43
43
  - [ ] Restore generic module
44
44
  - [ ] Better HTML/CSS
45
- - [ ] Kredis replacement implementation?
45
+ - [ ] Kredis replacement implementation?
Binary file
data/bench/bench.rb CHANGED
@@ -1,5 +1,7 @@
1
1
  require "sqlite3"
2
2
 
3
+ #RubyVM::YJIT.enable
4
+
3
5
  def bench(msg, iterations = 1000)
4
6
  GC.start
5
7
  # GC.compact
@@ -1,8 +1,8 @@
1
1
  require "active_support"
2
2
  require_relative "../lib/litestack"
3
- require_relative "./bench"
3
+ require_relative "bench"
4
4
 
5
- cache = ActiveSupport::Cache::Litecache.new({path: "../db/rails_cache.db"})
5
+ cache = ActiveSupport::Cache::Litecache.new
6
6
 
7
7
  # can only use the lookup method when the gem is installed
8
8
  # cache = ActiveSupport::Cache.lookup_store(:litecache, {path: '../db/rails_cache.db'})
@@ -31,6 +31,21 @@ count = 1000
31
31
  redis.write(keys[i], values[i])
32
32
  end
33
33
 
34
+ puts "== Multi Writes =="
35
+ bench("litecache multi-writes", count / 5) do |i|
36
+ idx = i * 5
37
+ payload = {}
38
+ 5.times { |j| payload[keys[idx + j]] = values[idx + j] }
39
+ cache.write_multi(payload)
40
+ end
41
+
42
+ bench("Redis multi-writes", count / 5) do |i|
43
+ idx = i * 5
44
+ payload = {}
45
+ 5.times { |j| payload[keys[idx + j]] = values[idx + j] }
46
+ redis.write_multi(payload)
47
+ end
48
+
34
49
  puts "== Reads =="
35
50
  bench("litecache reads", count) do |i|
36
51
  cache.read(random_keys[i])
@@ -39,6 +54,22 @@ count = 1000
39
54
  bench("Redis reads", count) do |i|
40
55
  redis.read(random_keys[i])
41
56
  end
57
+
58
+ puts "== Multi Reads =="
59
+ bench("litecache multi-reads", count / 5) do |i|
60
+ idx = i * 5
61
+ payload = []
62
+ 5.times { |j| payload << random_keys[idx + j] }
63
+ cache.read_multi(*payload)
64
+ end
65
+
66
+ bench("Redis multi-reads", count / 5) do |i|
67
+ idx = i * 5
68
+ payload = []
69
+ 5.times { |j| payload << random_keys[idx + j] }
70
+ redis.read_multi(*payload)
71
+ end
72
+
42
73
  puts "=========================================================="
43
74
 
44
75
  keys = []
@@ -1,6 +1,6 @@
1
1
  require "redis"
2
2
  require "sqlite3"
3
- require_relative "./bench"
3
+ require_relative "bench"
4
4
 
5
5
  # require 'polyphony'
6
6
  require "async/scheduler"
@@ -11,7 +11,7 @@ Fiber.scheduler.run
11
11
  require_relative "../lib/litestack/litecache"
12
12
  # require 'litestack'
13
13
 
14
- cache = Litecache.new({path: "../db/cache.db"}) # default settings
14
+ cache = Litecache.new # ({path: "../db/cache.db"}) # default settings
15
15
  redis = Redis.new # default settings
16
16
 
17
17
  values = []
@@ -35,28 +35,52 @@ count.times { keys << random_str(10) }
35
35
  cache.set(keys[i], values[i])
36
36
  end
37
37
 
38
- # bench("file writes", count) do |i|
39
- # f = File.open("../files/#{keys[i]}.data", 'w+')
40
- # f.write(values[i])
41
- # f.close
42
- # end
43
-
44
38
  bench("Redis writes", count) do |i|
45
39
  redis.set(keys[i], values[i])
46
40
  end
47
41
 
42
+ puts "== Multi Writes =="
43
+ bench("litecache multi-writes", count / 5) do |i|
44
+ idx = i * 5
45
+ payload = {}
46
+ 5.times { |j| payload[keys[idx + j]] = values[idx + j] }
47
+ cache.set_multi(payload)
48
+ end
49
+
50
+ bench("Redis multi-writes", count / 5) do |i|
51
+ idx = i * 5
52
+ payload = []
53
+ 5.times { |j|
54
+ payload << keys[idx + j]
55
+ payload << values[idx + j]
56
+ }
57
+ redis.mset(*payload)
58
+ end
59
+
48
60
  puts "== Reads =="
49
61
  bench("litecache reads", count) do |i|
50
62
  cache.get(random_keys[i])
51
63
  end
52
64
 
53
- # bench("file reads", count) do |i|
54
- # data = File.read("../files/#{keys[i]}.data")
55
- # end
56
-
57
65
  bench("Redis reads", count) do |i|
58
66
  redis.get(random_keys[i])
59
67
  end
68
+
69
+ puts "== Multi Reads =="
70
+ bench("litecache multi-reads", count / 5) do |i|
71
+ idx = i * 5
72
+ payload = []
73
+ 5.times { |j| payload << random_keys[idx + j] }
74
+ cache.get_multi(*payload)
75
+ end
76
+
77
+ bench("Redis multi-reads", count / 5) do |i|
78
+ idx = i * 5
79
+ payload = []
80
+ 5.times { |j| payload << random_keys[idx + j] }
81
+ redis.mget(*payload)
82
+ end
83
+
60
84
  puts "=========================================================="
61
85
 
62
86
  values = []
@@ -19,9 +19,9 @@ require "./rails_job"
19
19
  RailsJob.queue_adapter = :sidekiq
20
20
  t = Time.now.to_f
21
21
  puts "Make sure sidekiq is started with -c ./rails_job.rb"
22
- bench("enqueuing sidekiq jobs", count) do
23
- RailsJob.perform_later(count, t)
24
- end
22
+ #bench("enqueuing sidekiq jobs", count) do
23
+ # RailsJob.perform_later(count, t)
24
+ #end
25
25
 
26
26
  puts "Don't forget to check the sidekiq log for processing time conclusion"
27
27
 
@@ -18,9 +18,9 @@ require "./skjob"
18
18
 
19
19
  t = Process.clock_gettime(Process::CLOCK_MONOTONIC)
20
20
  puts "make sure sidekiq is started with skjob.rb as the job"
21
- bench("enqueuing sidekiq jobs", count) do |i|
22
- SidekiqJob.perform_async(count, t, delay)
23
- end
21
+ #bench("enqueuing sidekiq jobs", count) do |i|
22
+ # SidekiqJob.perform_async(count, t, delay)
23
+ #end
24
24
 
25
25
  puts "Don't forget to check the sidekiq log for processing time conclusion"
26
26
 
data/bin/liteboard CHANGED
@@ -2,19 +2,23 @@
2
2
  # frozen_string_literal: true
3
3
 
4
4
  require 'optparse'
5
+ require 'erb'
6
+ require 'yaml'
7
+ require 'rackup'
5
8
  require_relative '../lib/litestack/liteboard/liteboard'
6
9
  DEFAULTS = {
7
10
  config_path: Litemetric::DEFAULT_OPTIONS[:config_path],
8
11
  path: Litemetric::DEFAULT_OPTIONS[:path]
9
12
  }
10
13
 
11
- options = {
14
+ options = {
12
15
  config_path: nil,
13
16
  path: nil,
14
- deamonize: false,
17
+ daemonize: false,
15
18
  Port: 9292,
16
19
  Host: 'localhost',
17
20
  environment: 'production',
21
+ pid: './liteboard.pid',
18
22
  quiet: false
19
23
  }
20
24
 
@@ -25,9 +29,9 @@ OptionParser.new do |parser|
25
29
  parser.on("-s", "--server SERVER", "use SERVER (e.g. puma/falcon/iodine)") { |v| options[:port] = v }
26
30
  parser.on("-H", "--host HOST", "listen on HOST (default: #{options[:Host]})") { |v| options[:Host] = v }
27
31
  parser.on("-p", "--port PORT", "use PORT (default: #{options[:Port]})") { |v| options[:Port] = v.to_i rescue options[:Port] }
28
- parser.on("-D", "--deamonize", "run in the background") { |v| options[:deamonize] = true }
29
- parser.on("-E", "--env ENVIRONMENT", "which environment to use (default: #{options[:environment]})") { |v| options[:environment] = v }
30
- parser.on("-q", "--quiet", "turn off logging") { |v| options[:quiet] = true }
32
+ parser.on("-D", "--daemonize", "run in the background") { |v| options[:daemonize] = true }
33
+ parser.on("-E", "--env ENVIRONMENT", "which environment to use (default: #{options[:environment]})") { |v| options[:environment] = v }
34
+ parser.on("-q", "--quiet", "turn off logging") { |v| options[:quiet] = true }
31
35
  parser.on("-h", "--help", "print this message") do
32
36
  puts parser
33
37
  exit
@@ -35,40 +39,39 @@ OptionParser.new do |parser|
35
39
  end.parse!
36
40
 
37
41
  def check_database(path)
38
- unless File.exist?(path)
42
+ unless File.exist?(path)
39
43
  puts "liteboard: missing database file, please ensure the db path is correct"
40
44
  puts "liteboard: exiting"
41
45
  exit
42
46
  end
43
47
  end
44
48
 
45
- check_database(options[:path]) if options[:path]
49
+ check_database(options[:path]) if options[:path]
46
50
 
47
51
  # if there is a config file then we need to check it
48
52
  config = nil
49
53
  if options[:config_path]
50
54
  begin
51
- config = Yaml.load_file(options[:config_path])
55
+ config = YAML.load(ERB.new(File.read(options[:config_path])).result)
52
56
  rescue
53
57
  puts "liteboard: missing or bad config file, please ensure the config file path is correct"
54
58
  puts "liteboard: exiting"
55
59
  exit
56
- end
60
+ end
57
61
  else # no config path! use the default
58
- config = Yaml.load_file(DEFAULTS[:config_path]) rescue nil
62
+ config = YAML.load(ERB.new(File.read(DEFAULTS[:config_path])).result) rescue nil
59
63
  end
60
64
 
61
65
  if config
62
66
  if options[:path].nil?
63
67
  path = config['path'] || config[options[:environment]]['path']
64
68
  options[:path] = path
65
- end
69
+ end
66
70
  end
67
71
 
68
72
  # if still no path we assume a default db path
69
73
  options[:path] = DEFAULTS[:path] if options[:path].nil?
70
74
 
71
-
72
75
  # check the validity of the path before starting the server
73
76
  check_database(options[:path])
74
77
  Litemetric.options = options
@@ -78,4 +81,4 @@ options[:app] = Liteboard.app
78
81
  require_relative '../lib/litestack'
79
82
  puts "Starting Liteboard version #{Litestack::VERSION}"
80
83
 
81
- Rack::Server.start(options)
84
+ Rackup::Server.start(options)
@@ -0,0 +1,5 @@
1
+ source "https://rubygems.org"
2
+
3
+ gemspec path: "../"
4
+
5
+ gem "rails", "~> 7.0"
@@ -0,0 +1,5 @@
1
+ source "https://rubygems.org"
2
+
3
+ gemspec path: "../"
4
+
5
+ gem "rails", "~> 7.1"