qwe 0.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: a2a6e78eee154ac187e57a6f648989edbeb98bb6d09ee98118a386148f04d24d
+  data.tar.gz: a65ced572c633210b481e3779dce21524858687c42fd3667e71727e4f165ceb9
+SHA512:
+  metadata.gz: 428a88b7fd3e63e9236931cc65d0a123f272e71e503bd7cbb0a309e23440b57d4d8421ac442cf6be993f088978f39ef57563f246c4f44ecc49828f8475ce11b9
+  data.tar.gz: d5536e2f23c54ae7aaa8c09384ed72f3d94f7af4224a93bdfca35d74a7b11d67807d8f13602758a0bcfe2d230167e7c41ab2af839afb94bc7419ccae2e5570e9

data/DOCS.md

@@ -0,0 +1,469 @@
+# Starting server
+
+## Command line
+
+Gem provides `qwe` executable, which starts the server by default. See `qwe -h` for details.
+
+`-r` parameter is required for entrypoint to load your classes.
+Int rails it's reasonable to have `config/qwe.rb` for that purpose:
+```ruby
+APP_PATH = File.realpath("../")
+
+# Load the Rails application.
+require_relative "application"
+
+# Initialize the Rails application.
+Rails.application.initialize!
+```
+And server startup would be
+```
+qwe -r config/qwe.rb
+```
+or
+```
+bundle exec qwe -r config/qwe.rb
+```
+depending on your setup.
+
+## Development server example
+
+Autoloaders will handle includes,
+but the behavior of qwe on constant reloading is uncertain.
+In development you can restart server on code changes with watch (`-w`) option.
+Also it may be reasonable to limit worker threads count `-t` to not use all your cores,
+and discard experiments by saving data to `/tmp`:
+
+```
+qwe -r config/requirements.rb -w app -w config -t 2 -d /tmp/qwe_db
+```
+
+## Puma plugin
+
+Qwe server can be started in the same process as rails or any other puma application.
+With `plugin :qwe` in `config/puma.rb` server will start alongside rails on `rails server`.
+
+Configuration of server started from puma is done with `config/qwe.yml`. Valid config keys
+are last variants of cli argument names. For example, detach timeout can be set with
+`-s`, `--detach` and `--detach_timeout` cli options, or `detach_timeout` config key.
+
+## Production example
+
+config/puma.rb
+```ruby
+# ...
+plugin :qwe if ENV["Qwe_IN_PUMA"]
+# ...
+```
+config/qwe.yml
+```yml
+dir: ./storage/qwe_db
+require_file: ./config/qwe.rb
+detach_timeout: 100
+```
+
+# Connecting to server
+
+Each time you invoke `Qwe::DB` methods, it tries to connect to localhost at default port.
+If the server is running differently, you can connect to it once per process with
+```ruby
+Qwe::DB.connect("druby://example.com:1234")
+```
+
+## Creating an object
+
+```ruby
+Qwe::DB.create(MyModule::MyClass)
+```
+
+You can also create an instance of class which is unknown locally,
+but known to server by passing symbol instead of constant:
+```ruby
+Qwe::DB.create(:MyClass)
+```
+
+Also an object can be created from a string dump:
+```ruby
+class A
+  ...
+end
+
+a = A.new
+...
+
+Qwe::DB.create(Marshal.dump(a))
+```
+
+Physically records are stored in `qwe_db/records/id/`. There would be a 4 files:
+- `0` - Marshal dump of an object right after creation.
+- `dump` - current marshal dump.
+- `commits` or `commits.zst` - ruby code of all the changes.
+- `meta` - JSON metadata.
+
+Usually you don't need to mess up with these files,
+but simple file-based storage makes it possible if needed.
+
+## Deleting objects
+
+```ruby
+Qwe::DB.destroy(id)
+```
+
+Or just rm -r record files, it's fine unless record is in use.
+
+## Accessing objects
+
+`Qwe::DB.create` returns integer id of the record.
+
+`Qwe::DB[id]` returns DRb reference to your object. Methods you invoke on that reference
+will be delegated to the object living on the server, and return value is transmitted back.
+
+To avoid confusions with DRb, prevent creation of unnecessary references, and reduce lag,
+the best practice is not to descend on method chains whenever possible.
+
+```ruby
+Qwe::DB[id].one.two.three(arg)
+```
+will ping server 4 times. Instead, you should
+```ruby
+class A
+  def :one_two_three(...)
+    one.two.three(...)
+  end
+end
+```
+```ruby
+Qwe::DB[id].one_two_three(arg)
+```
+
+So it is preferable, but not required, to interact remotely only with the root object.
+
+Return values are transmitted through network sockets, so ensuring you don't return large objects unless necessary also increases performance.
+
+## Managing object state
+
+Your object can be either in RAM, or dumped to disk.
+
+Whenever you access an object with `Qwe::DB[]`, it gets loaded in RAM unless it's already there.
+
+It can be put back on disk (detached) with `Qwe::DB[id].record.detach`.
+
+Unless the server is explicitly started with `-d 0`, object will detach on it's own after `detach_timeout`, default 5 min.
+`record.keep` will prevent this by scheduling detachment at now + configured timeout.
+
+# The root and things
+
+`Qwe::Mixins::Root` mix-in provides a way to mark your object as **root**.
+
+Everything that is stored in top-level `Qwe::DB` should include it.
+In other words, when you `Qwe::DB.create(MyClass)`, `MyClass` should include `Root`.
+To avoid loosing objects to garbage collector, everything else that should persist
+is attached to root object. So `Root` is like a container for smaller things.
+
+`Qwe::Mixins::Thing` makes object attachable to root and provides attributes API.
+`Root` includes it too. Therefore most of your classes should include `Thing`, and one should include `Root`.
+
+```ruby
+class Passenger
+  include Qwe::Mixins::Thing
+  attribute :name, convert: :to_s
+end
+
+class Bus
+  include Qwe::Mixins::Root
+  attribute :passengers, :array
+
+  def board(name)
+    passengers << create(:Passenger, name:)
+  end
+end
+```
+```ruby
+Qwe::DB.create(:Bus)
+```
+
+## `Root` instance methods
+
+
+- `create(type, **attrs)` - create a new `Thing`, and attach it to self.
+optional `**attrs` provide initial attribute key-values.
+- `self[id]`, `things[id]` - whenever you create a thing, root stores it in `@things` array.
+`[]` is shorthand for accessing them.
+- `record` - Reference to `Qwe::DB::Record` containing object, see [later](#Record)
+- `commit(thing, method, *args, **keywords)` - Write to commits command to call `method` on `thing` with respective arguments.
+
+## `Thing` instance methods
+
+- `root` - reference to `Root` object.
+- `id` - respective index in root collection, so `root[id]` is `self`
+
+## `Thing` class methods
+
+- `attribute(...)` - `attr_accessor` with extras, see [attributes](#Attributes)
+- `attributes` - hash of attributes associated with this class.
+- `init` - init function, see [functions](#Functions)
+
+# Record
+
+`Qwe::DB::Record` represents all the logic around storing and serving your root, and can be accessed as `.record` from there.
+
+## Record Instance methods:
+
+- `id` - the same id you use in `Qwe::DB[]`, represents folder name where the record is stored.
+- `object` - reference to contained object.
+- `keep` - delay detachment until `Time.now + detach_timeout`.
+- `detach` - detach immediately
+- `save` - save immediately. Object will be saved anyway on detach, but you can save twice just to be sure.
+- `commits(line = nil)` - commits until `line` as a string. All commits if `line` is omitted.
+- `commits_length` - Current amount of lines in commits file. Some commits may be
+multi-line, so it isn't equal to commits count.
+- `commit(str)` - write string to commits file. Attributes do that for you.
+- `object_at(line)` - returns historic object state, or wayback, by evaluating commits until `line` in context of initial object state.
+- `fork(line)` - same as object_at, but creates a new record. Returns newly created `Qwe::DB::Record`.
+- `archive!` - compress commits file with `zst`. Requires native `zstd` package, see [ruby zstd docs](https://github.com/andrew-aladev/ruby-zstds?tab=readme-ov-file#installation). If commit is written into archived record, file will get unpacked with a warning.
+- `dir` - path to the record directory in filesystem
+- `dump` - `Marshal.dump(object)`
+
+# Attributes
+
+`Qwe::Mixins::Thing` provides `attribute` method, which behaves like `:attr_accessor`,
+but writer commits every change.
+
+```ruby
+attribute(name, *booleans, writer: nil, reader: nil, **params)
+```
+
+Any positional argument after first (name) is shorthand for true in params.
+For example, these are equal invocations:
+```ruby
+attribute :one, array: true
+attribute :one, :array
+```
+
+Beyond committing, attributes have a few additional parameters:
+- `writer` - `Proc` - define custom writer, but keep responsible for commits part of native writer. Should return value instead of setting instance variable.
+```ruby
+attribute :qwe, writer: ->(value) { value**2 }
+```
+- `reader` - `Proc` - replace reader entirely with given block.
+```ruby
+attribute :qwe, reader: -> { @my_attr || self.class.my_attr_default || self.class.my_attr_fallback }
+```
+- `convert` - `Symbol` - call this method on argument, used mostly for type conversions.
+```ruby
+attribute :qwe, convert: :to_f
+```
+- `init` - `Proc`, `Class` or any object - on initialization set attribute to
+`Proc` return value, `Class.new` or any value.
+Initialization happens in root.create, not constructor.
+```ruby
+attribute :qwe, init: 123
+attribute :asd, :array, init: [1, 2, 3]
+attribute :zxc, init: -> { Time.now }
+```
+- `default` - same as init, but value is set at reader if it is nil, like `@value || 123`. Since reader checks whether the value is falsy at every call, it is a bit slower.
+- `min`, `max` - set value to min/max on write if it is lesser/greater.
+```ruby
+attribute :qwe, min: 10
+attribute :asd, max: 20
+```
+- `array` - `bool` - initializes attribute to `Qwe::Attribute::ArrayProxy`, which is like array, but commits changes, see [limitations](#Limitations) on why it matters.
+- `enum` - `Array` or any object responding to `include?` - raise on write if value is not included in provided enum.
+```ruby
+attribute :weather, enum: [:hot, :cold, :normal]
+```
+
+# Functions
+
+To achieve desired performance, attributes are eval-compiled.
+`Qwe::Function` provides a simple way to join a few lines to a method.
+Performance is gained, for example, by checking whether an attribute has
+min, max, convert, etc... at server startup instead of every method call.
+
+## Function instance methods
+- `new(klass, name)` - creates an instance. Despite being instances, functions are created at module level.
+- `arg(name, default)` - add positional argument
+- `args` - hash of all arguments
+- `keyword(name, default)` - add keyword argument
+- `keywords` - hash of all keywords
+- `stage(string)` - add block of code
+- `code` - source code to be evaluated
+- `compile` - eval code and define method
+
+Creating function for a class defines `compile` method on it, which compiles all functions for that class.
+
+On startup worker compiles all existing functions, but if you are lazy-loading constants
+(it's on by default in non-production rails environments),
+adding `compile` before closing class is necessary.
+Or just enable enable eager loading, in rails it's in `confg/environments`.
+
+```ruby
+class A
+  @func = Qwe::Function.new(self, :func)
+  @func.stage("do_something(1 + 1)")
+  @func.stage("do_something_else") if 1 == 1.0
+
+  # ...
+
+  compile
+end
+
+# Equivalent to:
+class A
+  def func
+    do_something(1 + 1)
+    do_something_else
+  end
+end
+```
+
+Using compile multiple times is safe - function won't be re-evaluated unless it's changed.
+
+## Init function
+
+Functions are used for attribute reader, writer, and `Thing` initialization.
+
+`Thing` and `Root` don't use initializers in any way.
+Instead, root calls `init` function when creating thing with `create` method.
+
+Init function is accessible as `init` class method, so you can extend it like that:
+
+```ruby
+class A
+  init.stage "do_something"
+
+  def do_something
+    # ...
+  end
+
+  compile
+end
+```
+
+This will `do_something` each time `A` instance is created.
+
+Standard ruby initializers are left for possible future hacking.
+
+# Extending attributes
+
+Instead of directly manipulating attribute functions,
+you can extend attributes API itself with `add(param = nil, &block)` method.
+If param is passed, block will be executed only if attribute
+was invoked with this parameter.
+
+Block is evaluated in `Qwe::Attribute` instance context.
+Attribute instance is created every time you invoke `attribute` class method, and represents specific attribute of specific class.
+
+## `Qwe::Attribute` instance methods
+
+- `klass` - class for which attribute is declared.
+- `name` - attribute name.
+- `writer` - writer function. It has single argument `value`.
+At the last stage sets instance variable corresponding to attribute name to this value,
+so you should modify `value` instead of directly setting instance variable.
+- `reader` - reader function with no arguments. Returns respective instance variable.
+- `[keyword]` - returns keyword or boolean passed to `attribute` method. Also available with respond_to, i.e. `self.min`
+
+```ruby
+Qwe::Attribute.add(:rand) do
+  init.stage "self.#{name} = rand"
+end
+
+Qwe::Attribute.add(:plus) do
+  klass.class_variable_set(:"@@#{name}_plus", plus)
+  writer.stage "value += @@#{name}_plus"
+end
+
+class A
+  include Qwe::Mixins::Thing
+
+  attribute :one, :rand
+  attribute :two, plus: 3
+end
+
+a = A.new
+a.one # Random value between 0 and 1
+a.two = 1
+a.two # 4
+```
+
+# Commits internals
+
+Commits work by implementing `to_rb` method on all classes of interest - Integer, Float, Time, etc.
+This method is implemented for most standard types, but if something is missing you'll get an exception on commit, complaining that `to_rb` method is undefined.
+
+In that case just define `to_rb` to return object as a valid ruby code string in context of root.
+Here are a few examples of bundled to_rb's:
+
+```ruby
+class Integer
+  alias_method :to_rb, :to_s
+end
+
+class Rational
+  def to_rb
+    to_s + "r"
+  end
+end
+
+class Time
+  def to_rb
+    "Time.at(#{to_r}r)"
+  end
+end
+```
+
+# Limitations
+
+## In-place modifications of attributes
+
+Let's consider this code block:
+
+```ruby
+class A
+  attribute :str
+end
+
+a = A.new
+a.str = "qwe"
+a.str.sub! "w", "x"
+```
+
+As we expect, resulting `a.str` value would be "qxe". However, `sub!` modifies string in place, therefore does not call attribute writer and does not commit the change. This problem currently is not solved, since you can just do
+```ruby
+a.str = a.str.sub "w", "x"
+```
+
+For strings this is somewhat fine, but for large arrays things get worse. For arrays there is `Qwe::Attribute::ArrayProxy` class, which extends `Array` class and commits changes.
+
+So the rule here is **don't use in-place modifications on attributes**, and if you need these - extend desired class and write committing logic. Be sure to benchmark results - sometimes replacing object entirely would be faster than doing complex checks.
+
+Commits integrity can be validated by comparing object with it's fork
+```ruby
+class R
+  include Qwe::Mixins::Root
+
+  def validate
+    if record.dump == record.fork.dump
+      puts "Commits are fine"
+    else
+      puts "Commits are broken, you are doing something wrong"
+    end
+  end
+end
+```
+
+## Thread safety
+
+As in ruby itself, much of thread safety is left up to you. Since DRb is multi-threaded, unsafety usually comes from DRb interactions, and providing single-threaded interaction layer is somewhat on roadmap.
+
+The two common pitfalls are `record.save` and `record.archive` - you should ensure that nothing is committed while record gets dumped or commits file gets archived.
+
+## Not everything can be dumped
+
+`Proc`, `Thread`, and a few other types can't be dumped with `Marshal`, and therefore can't be saved to disk.
+
+## Fail safety
+
+In case worker process is killed with INT or TERM, it saves all records before stopping, but record persistence is not resilient to power outage, drive corruption, etc.
+
+It's possible to recreate objects from commits on startup after unhandled shutdown, but precise implementation and real value are uncertain - fail-safe database has little use without corresponding application-side code.

data/Gemfile

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in om.gemspec
+gemspec
+
+gem "rake", "~> 13.0"
+gem "minitest", "~> 5.0"
+gem "standardrb"
+gem "erb" # For standardrb
+
+gem "listen", "~> 3.9"

data/Gemfile.lock

@@ -0,0 +1,84 @@
+PATH
+  remote: .
+  specs:
+    qwe (0.0.0)
+      drb (~> 2.2.1)
+      ruby-zstds (~> 1.3)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    adsp (1.0.10)
+    ast (2.4.2)
+    cgi (0.4.1)
+    drb (2.2.1)
+    erb (4.0.4)
+      cgi (>= 0.3.3)
+    ffi (1.16.3)
+    json (2.7.2)
+    language_server-protocol (3.17.0.3)
+    lint_roller (1.1.0)
+    listen (3.9.0)
+      rb-fsevent (~> 0.10, >= 0.10.3)
+      rb-inotify (~> 0.9, >= 0.9.10)
+    minitest (5.22.3)
+    parallel (1.24.0)
+    parser (3.3.0.5)
+      ast (~> 2.4.1)
+      racc
+    racc (1.7.3)
+    rainbow (3.1.1)
+    rake (13.2.0)
+    rb-fsevent (0.11.2)
+    rb-inotify (0.10.1)
+      ffi (~> 1.0)
+    regexp_parser (2.9.0)
+    rexml (3.2.6)
+    rubocop (1.62.1)
+      json (~> 2.3)
+      language_server-protocol (>= 3.17.0)
+      parallel (~> 1.10)
+      parser (>= 3.3.0.2)
+      rainbow (>= 2.2.2, < 4.0)
+      regexp_parser (>= 1.8, < 3.0)
+      rexml (>= 3.2.5, < 4.0)
+      rubocop-ast (>= 1.31.1, < 2.0)
+      ruby-progressbar (~> 1.7)
+      unicode-display_width (>= 2.4.0, < 3.0)
+    rubocop-ast (1.31.2)
+      parser (>= 3.3.0.4)
+    rubocop-performance (1.20.2)
+      rubocop (>= 1.48.1, < 2.0)
+      rubocop-ast (>= 1.30.0, < 2.0)
+    ruby-progressbar (1.13.0)
+    ruby-zstds (1.3.1)
+      adsp (~> 1.0)
+    standard (1.35.1)
+      language_server-protocol (~> 3.17.0.2)
+      lint_roller (~> 1.0)
+      rubocop (~> 1.62.0)
+      standard-custom (~> 1.0.0)
+      standard-performance (~> 1.3)
+    standard-custom (1.0.2)
+      lint_roller (~> 1.0)
+      rubocop (~> 1.50)
+    standard-performance (1.3.1)
+      lint_roller (~> 1.1)
+      rubocop-performance (~> 1.20.2)
+    standardrb (1.0.1)
+      standard
+    unicode-display_width (2.5.0)
+
+PLATFORMS
+  x86_64-linux
+
+DEPENDENCIES
+  erb
+  listen (~> 3.9)
+  minitest (~> 5.0)
+  qwe!
+  rake (~> 13.0)
+  standardrb
+
+BUNDLED WITH
+   2.5.23

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 goose3228
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.