qwe 0.0.0 → 0.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a2a6e78eee154ac187e57a6f648989edbeb98bb6d09ee98118a386148f04d24d
-  data.tar.gz: a65ced572c633210b481e3779dce21524858687c42fd3667e71727e4f165ceb9
+  metadata.gz: 9163afb748f542c3cee0749fa5cb182100c2c46c5eb52c7834dd6a9e9928e569
+  data.tar.gz: f36a49f10487cae9d6c6caeb5774946355d072e832d4139f6e3f74af8b65b8dc
 SHA512:
-  metadata.gz: 428a88b7fd3e63e9236931cc65d0a123f272e71e503bd7cbb0a309e23440b57d4d8421ac442cf6be993f088978f39ef57563f246c4f44ecc49828f8475ce11b9
-  data.tar.gz: d5536e2f23c54ae7aaa8c09384ed72f3d94f7af4224a93bdfca35d74a7b11d67807d8f13602758a0bcfe2d230167e7c41ab2af839afb94bc7419ccae2e5570e9
+  metadata.gz: c11193ca71612d9433848ec7217da3d9b795be3f73b70fc1f0ce6a01254bd908d3a6472c33aff7d1d8f7e72b30ad91492b7be02261d02bac1a3c2391369ae6d1
+  data.tar.gz: e2147f520f70ec10510236d22e62a48b28eb5b1bb63b9db759d0dce62a8d491e63e90c0d452e83d5aca9251ef8fe0f63c3679f4ce682eea074600d2f3e34e665

data/DOCS.md

@@ -5,7 +5,7 @@
 Gem provides `qwe` executable, which starts the server by default. See `qwe -h` for details.
 
 `-r` parameter is required for entrypoint to load your classes.
-Int rails it's reasonable to have `config/qwe.rb` for that purpose:
+In rails it's reasonable to have `config/qwe.rb` for that purpose:
 ```ruby
 APP_PATH = File.realpath("../")
 
@@ -46,6 +46,8 @@ Configuration of server started from puma is done with `config/qwe.yml`. Valid c
 are last variants of cli argument names. For example, detach timeout can be set with
 `-s`, `--detach` and `--detach_timeout` cli options, or `detach_timeout` config key.
 
+`--watch` is not available in puma plugin.
+
 ## Production example
 
 config/puma.rb
@@ -219,7 +221,7 @@ multi-line, so it isn't equal to commits count.
 - `commit(str)` - write string to commits file. Attributes do that for you.
 - `object_at(line)` - returns historic object state, or wayback, by evaluating commits until `line` in context of initial object state.
 - `fork(line)` - same as object_at, but creates a new record. Returns newly created `Qwe::DB::Record`.
-- `archive!` - compress commits file with `zst`. Requires native `zstd` package, see [ruby zstd docs](https://github.com/andrew-aladev/ruby-zstds?tab=readme-ov-file#installation). If commit is written into archived record, file will get unpacked with a warning.
+- `archive!` - compress commits file with `zst`. If commit is written into archived record, file will get unpacked with a warning.
 - `dir` - path to the record directory in filesystem
 - `dump` - `Marshal.dump(object)`
 
@@ -266,7 +268,8 @@ attribute :zxc, init: -> { Time.now }
 attribute :qwe, min: 10
 attribute :asd, max: 20
 ```
-- `array` - `bool` - initializes attribute to `Qwe::Attribute::ArrayProxy`, which is like array, but commits changes, see [limitations](#Limitations) on why it matters.
+- `array` - `bool` - initializes attribute to `Qwe::Proxy::Array`, which is like array, but commits changes, see [limitations](#Limitations) on why it matters.
+- `hash` - `bool` - same as array, but for hashes. Attribute type is `Qwe::Proxy::Hash`.
 - `enum` - `Array` or any object responding to `include?` - raise on write if value is not included in provided enum.
 ```ruby
 attribute :weather, enum: [:hot, :cold, :normal]
@@ -433,7 +436,7 @@ As we expect, resulting `a.str` value would be "qxe". However, `sub!` modifies s
 a.str = a.str.sub "w", "x"
 ```
 
-For strings this is somewhat fine, but for large arrays things get worse. For arrays there is `Qwe::Attribute::ArrayProxy` class, which extends `Array` class and commits changes.
+For strings this is somewhat fine, but for large arrays things get worse. For arrays there is `Qwe::Proxy::Array` class, which extends `Array` class and commits changes.
 
 So the rule here is **don't use in-place modifications on attributes**, and if you need these - extend desired class and write committing logic. Be sure to benchmark results - sometimes replacing object entirely would be faster than doing complex checks.
 

data/Gemfile.lock

@@ -1,14 +1,13 @@
 PATH
   remote: .
   specs:
-    qwe (0.0.0)
+    qwe (0.0.1)
       drb (~> 2.2.1)
-      ruby-zstds (~> 1.3)
+      zstd-ruby (~> 1.5)
 
 GEM
   remote: https://rubygems.org/
   specs:
-    adsp (1.0.10)
     ast (2.4.2)
     cgi (0.4.1)
     drb (2.2.1)
@@ -51,8 +50,6 @@ GEM
       rubocop (>= 1.48.1, < 2.0)
       rubocop-ast (>= 1.30.0, < 2.0)
     ruby-progressbar (1.13.0)
-    ruby-zstds (1.3.1)
-      adsp (~> 1.0)
     standard (1.35.1)
       language_server-protocol (~> 3.17.0.2)
       lint_roller (~> 1.0)
@@ -68,6 +65,7 @@ GEM
     standardrb (1.0.1)
       standard
     unicode-display_width (2.5.0)
+    zstd-ruby (1.5.6.6)
 
 PLATFORMS
   x86_64-linux

data/README.md

@@ -1,10 +1,28 @@
 # Qwe - pure ruby framework for stuff
 
-The stuff is mostly complex ruby objects living in RAM for a long time, accessed over DRb. Qwe provides database-like interface for managing that with a few extras, while attempting to impose as little bounds
-as possible on the way you write code.
+It might be useful if you have:
+- Extra high loads
+- Extensive interaction
+- Complexity in fitting into database
+
+Qwe is built upon [DRb](https://docs.ruby-lang.org/en/3.3/DRb.html), which provides a way to directly access ruby objects from different processes, thus avoiding latency, JSON or XML conversions, and disk I/O limits.
+
+Qwe manages:
+- Serving and accessing objects over DRb
+- Persisting them on disk
+- Loading them back into RAM upon access
+- Tracing every change
 
 [Full Documentation](DOCS.md)
 
+[Multiplayer paint example](https://gitlab.com/goose3228/qwe-paint) - Qwe is fast enough to process mouse events on server side.
+
+## Installation
+
+Qwe requires native [zstd](https://github.com/facebook/zstd) support.
+
+Otherwise it's regular `bundle add qwe` or `gem install qwe`
+
 ## Features
 
 ### Zero-config startup
@@ -24,7 +42,7 @@ The hash is now accessible from any ruby process with `Qwe::DB[]`,
  considering you know that id.
 
 ```ruby
-Qwe::DB[id][:qwe] = 123
+Qwe::DB[id][:one_two_three] = 123
 ```
 
 After a period of inactivity, or manually if configured,
@@ -127,6 +145,8 @@ by running independent Qwe servers.
 DRb does not provide any access control, and opens separate port for each object it serves.
 It is safe to run Qwe in docker container or other isolated network, and unsafe to run on virtual or physical device with all ports open.
 
+In web applications Qwe may be accessed though websockets without providing direct access to ruby methods.
+
 See [DRb docs](https://docs.ruby-lang.org/en/3.3/DRb.html#module-DRb-label-Security) for a more detailed explanation.
 
 ## License

data/exe/qwe

@@ -39,7 +39,7 @@ opt_parser = OptionParser.new do |parser|
     args[:threads] = a.to_i
   end
 
-  parser.on("--gc=SECONDS", "--gc_interval=SECONDS", "Call garbage collector (GC.start) every x seconds. Off by default") do |a|
+  parser.on("--gc=SECONDS", "--gc_interval=SECONDS", "Call garbage collector (GC.start) every x seconds. Off by default. Ruby can ignore garbage collection at 100% CPU load, and --gc forces ruby to still do that.") do |a|
     args[:gc_interval] = a.to_i
   end
 
@@ -66,7 +66,7 @@ if args[:watch]
   begin
     require "listen"
   rescue LoadError
-    puts "Can't load 'listen' gem"
+    puts "'listen' gem is required to use -w/--watch option"
     exit 1
   end
   pid = spawn_server(args)
@@ -86,30 +86,8 @@ if args[:watch]
   end
   sleep
 elsif args[:interactive]
-  require "irb"
-
-  Qwe::DB.connect("druby://localhost:#{args[:port]}") if args[:port]
-
-  if args[:interactive] > 0
-    obj = Qwe::DB[args[:interactive]]
-    puts "\e[0;32mQwe interactive prompt\e[0m, record \e[1;32m##{args[:interactive]}\e[0m, class \e[1;36m#{obj.record.obj_eval("self.class.to_s")}\e[0m"
-  else
-    obj = Qwe::DB.server.pick_worker
-    puts "\e[0;32mQwe interactive prompt\e[0m, worker \e[1;36m#{obj.uri}\e[0m, require #{obj.requirements}"
-
-    require obj.requirements
-  end
-
-  (obj.instance_exec { binding }).instance_exec do
-    irb(show_code: false)
-  rescue
-    # Older irb versions always print source code around call point
-    IRB.setup(source_location[0], argv: [])
-    workspace = IRB::WorkSpace.new(self)
-    binding_irb = IRB::Irb.new(workspace)
-    binding_irb.context.irb_path = File.expand_path(source_location[0])
-    binding_irb.run(IRB.conf)
-  end
+  require_relative "../lib/qwe/console"
+  Qwe::Console.start(args[:interactive], args[:port])
 elsif args[:create]
   Qwe::DB.connect("druby://localhost:#{args[:port]}") if args[:port]
   puts Qwe::DB.create(args[:create].to_sym)

data/lib/qwe/attribute.rb

@@ -116,7 +116,7 @@ module Qwe
     end
 
     add(:default) do
-      reader.stage "self.#{name} = " + init_or_default_stage(:default) + " unless @#{name}"
+      reader.stage "self.#{name} = " + init_or_default_stage(:default) + " if @#{name}.nil?"
     end
 
     add(:convert) do
@@ -134,60 +134,19 @@ module Qwe
     end
 
     add(:array) do
-      writer.stage "value = Qwe::Attribute::ArrayProxy.from_array(self, :#{name}, value)"
+      writer.stage "value = Qwe::Proxy::Array.from_array(self, :#{name}, value)"
       klass.init.stage("self.#{name} = []") unless init
     end
 
+    add(:hash) do
+      writer.stage "value = Qwe::Proxy::Hash.from_hash(self, :#{name}, value)"
+      klass.init.stage("self.#{name} = {}") unless init
+    end
+
     add(:enum) do
       raise "#{klass}.#{name} - enum #{enum} doesn't respond to include?" unless enum.respond_to?(:include?)
       klass.class_variable_set(:"@@#{name}_enum", enum)
       writer.stage "raise \"Enum - unknown value \#{self}.#{name} = \#{value}\" unless @@#{name}_enum.include?(value)"
     end
-
-    class ArrayProxy < Array
-      attr_accessor :__qwe_thing
-      attr_accessor :__qwe_name
-
-      include DRb::DRbUndumped
-
-      undef _dump
-
-      def __qwe_commit(m, *a, **k)
-        __qwe_thing&.root&.record&.commit("#{__qwe_thing.to_rb}.#{__qwe_name}.method(:#{m}).super_method.call(*#{a.to_rb}, **#{k.to_rb})\n")
-      end
-
-      TRANSACTION_METHODS = [:<<, :[]=, :clear, :compact!, :concat, :delete_at, :flatten!, :insert, :pop, :push, :replace, :reverse!, :rotate!, :shift, :shuffle!, :slice!, :unshift]
-      REPLACE_METHODS = [:delete, :delete_if, :fill, :filter!, :select!, :keep_if, :map!, :reject!, :sort!, :sort_by!, :uniq!]
-
-      TRANSACTION_METHODS.each do |m|
-        define_method(m) do |*a, **k|
-          __qwe_commit(m, *a, **k)
-          super(*a, **k)
-        end
-      end
-
-      REPLACE_METHODS.each do |m|
-        define_method(m) do |*a, **k, &block|
-          r = super(*a, **k, &block)
-          if block
-            __qwe_thing&.root&.record&.commit "#{__qwe_thing.to_rb}.instance_variable_set(:@#{__qwe_name}, #{to_rb})\n"
-          else
-            __qwe_commit(m, *a, **k)
-          end
-          r
-        end
-      end
-
-      def self.from_array(thing, name, arr = [])
-        a = new(arr)
-        a.__qwe_thing = thing
-        a.__qwe_name = name
-        a
-      end
-
-      def to_rb
-        "Qwe::Attribute::ArrayProxy.from_array(#{__qwe_thing.to_rb}, :#{__qwe_name}, #{super})"
-      end
-    end
   end
 end

data/lib/qwe/console.rb

@@ -0,0 +1,66 @@
+require "irb"
+
+module Qwe
+  class Console
+    def self.start(...)
+      new(...).start
+    end
+
+    attr_reader :id, :port
+
+    def initialize(id = 0, port = nil)
+      @id = id.to_i
+      @port = if port
+        port.to_i
+      else
+        Qwe::DB::Server::DEFAULT_PORT
+      end
+
+      Qwe::DB.connect("druby://localhost:#{port}") if port
+      setup
+    end
+
+    def setup
+      if id > 0
+        IRB::WorkSpace.define_method(:evaluate, &record_proc)
+        req = Qwe::DB[id].record.evaluate("Worker.instance.requirements")
+        msg = "\e[1;35mQwe interactive prompt\e[0m, record \e[1;32m##{id}\e[0m"
+      else
+        IRB::WorkSpace.define_method(:evaluate, &worker_proc)
+        req = Qwe::DB.server.pick_worker.requirements
+        msg = "\e[1;35mQwe interactive prompt\e[0m, random worker"
+      end
+
+      puts msg
+      require req if req
+    end
+
+    def to_s
+      "Qwe id=#{id}"
+    end
+
+    def start
+      binding.irb(show_code: false)
+    rescue
+      # Older irb versions always print source code around call point
+      IRB.setup(source_location[0], argv: [])
+      workspace = IRB::WorkSpace.new(self)
+      binding_irb = IRB::Irb.new(workspace)
+      binding_irb.context.irb_path = File.expand_path(source_location[0])
+      binding_irb.run(IRB.conf)
+    end
+
+    def record_proc
+      i = id
+      proc do |statements, *args|
+        Qwe::DB[i].record.evaluate(statements)
+      end
+    end
+
+    def worker_proc
+      proc do |statements, *args|
+        Qwe::DB.server.pick_worker.evaluate(statements)
+      end
+    end
+  end
+end

data/lib/qwe/db/commits_file.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-require "zstds"
+require "zstd-ruby"
 require "fileutils"
 
 module Qwe::DB
@@ -12,17 +12,23 @@ module Qwe::DB
     def initialize(dir)
       @path = File.join(dir, "commits")
       @archive_path = File.join(dir, "commits.zst")
-      unless File.exist?(@archive_path)
-        @file = File.open(@path, "a+")
-      end
+      open
+    end
+
+    def open
+      return if @file && !@file.closed? || File.exist?(@archive_path)
+      @file = File.open(@path, "a+")
     end
 
     def write(str)
-      unless file
+      if !file
         log "Write #{str} to archived commits file #{path}, unpacking"
-        ZSTDS::File.decompress archive_path, path
-        FileUtils.rm_f(archive_path)
+        data = read_archive
         @file = File.open(@path, "a+")
+        @file.write(data)
+        FileUtils.rm_f(archive_path)
+      elsif file.closed?
+        open
       end
       file.write(str)
     rescue => e
@@ -31,7 +37,7 @@ module Qwe::DB
     end
 
     def read(line = nil)
-      if file
+      if file && !file.closed?
         file.flush
         file.rewind
         if line
@@ -41,10 +47,12 @@ module Qwe::DB
           file.read
         end
       else
-        cs = ZSTDS::Stream::Reader.open(archive_path) do |r|
-          r.set_encoding("UTF-8")
-          r.read
+        cs = if File.exist?(archive_path)
+          read_archive
+        else
+          File.read(path)
         end
+
         if line
           cs.lines[0, line].join("")
         else
@@ -53,9 +61,13 @@ module Qwe::DB
       end
     end
 
+    def read_archive
+      Zstd.decompress(File.binread(archive_path)).force_encoding("UTF-8")
+    end
+
     def archive!
-      file&.close
-      ZSTDS::File.compress path, archive_path
+      File.binwrite(archive_path, Zstd.compress(read))
+      close
       FileUtils.rm_f(path)
       @file = nil
     end
@@ -65,7 +77,7 @@ module Qwe::DB
     end
 
     def close
-      file&.close
+      file&.close unless file&.closed?
     end
   end
 end

data/lib/qwe/db/record.rb

@@ -161,7 +161,7 @@ module Qwe::DB
       rec
     end
 
-    def obj_eval(rb)
+    def evaluate(rb)
       object.instance_eval(rb)
     end
 

data/lib/qwe/db/worker.rb

@@ -114,7 +114,7 @@ module Qwe::DB
       @@instance.work
     end
 
-    def e(str)
+    def evaluate(str)
       instance_eval str
     end
   end

data/lib/qwe/proxy/array.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+class Qwe::Proxy::Array < Array
+  Qwe::Proxy.make self
+
+  Qwe::Proxy.transaction self,
+    :<<, :[]=, :clear, :compact!, :concat, :delete_at, :flatten!,
+    :insert, :pop, :push, :replace, :reverse!, :rotate!, :shift,
+    :shuffle!, :slice!, :unshift
+
+  Qwe::Proxy.block self,
+    :delete, :delete_if, :fill, :filter!, :select!, :keep_if,
+    :map!, :reject!, :sort!, :sort_by!, :uniq!
+
+  def self.from_array(thing, name, arr = [])
+    if arr.is_a?(Qwe::Proxy::Array)
+      arr
+    else
+      a = new(arr)
+      a.__qwe_thing = thing
+      a.__qwe_name = name
+      a
+    end
+  end
+
+  def to_rb
+    "Qwe::Proxy::Array.from_array(#{__qwe_thing.to_rb}, :#{__qwe_name}, #{super})"
+  end
+end

data/lib/qwe/proxy/hash.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+class Qwe::Proxy::Hash < Hash
+  Qwe::Proxy.make self
+
+  Qwe::Proxy.transaction self,
+    :[]=, :store, :clear, :compact!, :default=, :rehash, :replace, :shift
+
+  Qwe::Proxy.block self,
+    :delete, :delete_if, :filter!, :select!, :keep_if, :merge!,
+    :reject!, :transform_keys!, :transform_values!
+
+  def self.from_hash(thing, name, hash = {})
+    if hash.is_a?(Qwe::Proxy::Hash)
+      hash
+    else
+      h = new
+      h.replace(hash)
+      h.__qwe_thing = thing
+      h.__qwe_name = name
+      h
+    end
+  end
+
+  def to_rb
+    "Qwe::Proxy::Hash.from_hash(#{__qwe_thing.to_rb}, :#{__qwe_name}, #{super})"
+  end
+end

data/lib/qwe/proxy.rb

@@ -0,0 +1,43 @@
+module Qwe
+  module Proxy
+    def self.make(klass)
+      klass.module_exec do
+        attr_accessor :__qwe_thing, :__qwe_name
+
+        include DRb::DRbUndumped
+
+        undef _dump
+
+        def __qwe_commit(m, *a, **k)
+          __qwe_thing&.root&.record&.commit("#{__qwe_thing.to_rb}.#{__qwe_name}.method(:#{m}).super_method.call(*#{a.to_rb}, **#{k.to_rb})\n")
+        end
+      end
+    end
+
+    def self.transaction(klass, *methods)
+      methods.each do |m|
+        klass.define_method(m) do |*a, **k|
+          __qwe_commit(m, *a, **k)
+          super(*a, **k)
+        end
+      end
+    end
+
+    def self.block(klass, *methods)
+      methods.each do |m|
+        klass.define_method(m) do |*a, **k, &block|
+          r = super(*a, **k, &block)
+          if block
+            __qwe_thing&.root&.record&.commit "#{__qwe_thing.to_rb}.instance_variable_set(:@#{__qwe_name}, #{to_rb})\n"
+          else
+            __qwe_commit(m, *a, **k)
+          end
+          r
+        end
+      end
+    end
+  end
+end
+
+require_relative "proxy/array"
+require_relative "proxy/hash"

data/lib/qwe/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Qwe
-  VERSION = "0.0.0"
+  VERSION = "0.0.2"
 end

data/lib/qwe.rb

@@ -9,6 +9,7 @@ require_relative "qwe/version"
 require_relative "qwe/mixins"
 require_relative "qwe/db"
 require_relative "qwe/function"
+require_relative "qwe/proxy"
 require_relative "qwe/attribute"
 
 module Qwe

data/qwe.gemspec

@@ -28,5 +28,5 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_dependency "drb", "~> 2.2.1"
-  spec.add_dependency "ruby-zstds", "~> 1.3"
+  spec.add_dependency "zstd-ruby", "~> 1.5"
 end

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: qwe
 version: !ruby/object:Gem::Version
-  version: 0.0.0
+  version: 0.0.2
 platform: ruby
 authors:
 - goose3228
-autorequire:
 bindir: exe
 cert_chain: []
-date: 2024-12-25 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: drb
@@ -25,19 +24,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: 2.2.1
 - !ruby/object:Gem::Dependency
-  name: ruby-zstds
+  name: zstd-ruby
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.3'
+        version: '1.5'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.3'
+        version: '1.5'
 description: Object persistence with DRb access, manual or automatic RAM/disk switch,
   and transactions as ruby code.
 email:
@@ -58,6 +57,7 @@ files:
 - lib/puma/plugin/qwe.rb
 - lib/qwe.rb
 - lib/qwe/attribute.rb
+- lib/qwe/console.rb
 - lib/qwe/db.rb
 - lib/qwe/db/commits_file.rb
 - lib/qwe/db/record.rb
@@ -68,6 +68,9 @@ files:
 - lib/qwe/mixins/process.rb
 - lib/qwe/mixins/root.rb
 - lib/qwe/mixins/thing.rb
+- lib/qwe/proxy.rb
+- lib/qwe/proxy/array.rb
+- lib/qwe/proxy/hash.rb
 - lib/qwe/version.rb
 - lib/spawn_worker.rb
 - qwe.gemspec
@@ -76,7 +79,6 @@ licenses:
 - MIT
 metadata:
   source_code_uri: https://gitlab.com/goose3228/qwe
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -91,8 +93,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.16
-signing_key:
+rubygems_version: 3.6.9
 specification_version: 4
 summary: Framework for in-RAM ruby applications
 test_files: []