lutaml-store 0.3.0 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 34e9babf679b2654a563c89d68552afd0b2d98d5725656eb9baa2f1a878a80f4
-  data.tar.gz: 599409d62bfe96b0e57147f620aef55ea88abe40b5183bdfb02511d3a91ea281
+  metadata.gz: 4481af10b7c80db97bffbe2e1ad5119d615370596c46c88c84e536411d1ffdda
+  data.tar.gz: 618ad367561533d0aa576ef713dfb384173e34ec1ec0047e3cbab400af9c4dce
 SHA512:
-  metadata.gz: 77f4022441c6b6ab2f63b5106a2ca3bba16abab82bf75adbe8569ec73a0eb6cd824891920e5571e7a2f7b39c9d56ac5bd2824418f24aa8577abf893645941062
-  data.tar.gz: dbe7804392b8b32135c77f3766b6976a8111a357b40bfdad563906d59205a0305dfd3f467b371ba423859a6ebbb8a7fa8d8baef905875a6097478763ed2d6b43
+  metadata.gz: 2e22cb07d0781f6aba2c2ebc043322e5a756519284740b74661906b6f00cbce8536291967868b86fd2a1908de46e6c1d4450f35c517d4919f73020d65890dc63
+  data.tar.gz: 26f2ef6b87a85e2c020571319640ceed71955ace41c97c2052857b97bd8d68e599cf466cd349f31c802a0fda7172cd857ce1bfbed952076b7425aea07c7d2de0

data/CLAUDE.md

@@ -49,6 +49,12 @@ All inherit from `Format::Base`. Six formats: `Yaml`, `Yamls`, `Json`, `Jsonl`,
 
 All inherit from `Adapter::Base`. Three backends: `Memory`, `FileSystem`, `SQLite`. Registered in `Adapter` module and resolved via `Adapter.resolve(:type, options)`. New adapters can be added with `Adapter.register(:custom, CustomClass)` without modifying existing code (OCP).
 
+`Base#each_key` defaults to `keys.each`; `Base#update(key) { |old| new }` defaults to get + set inside `transaction`. Memory, FileSystem and Sqlite override `update` to be atomic. `FileSystem` percent-encodes file names (digest + key in `.meta` for long names), writes non-String values as JSON (`format: "json"` in `.meta`), and locks `<root>/.lock` (`with_lock(:ex/:sh)`, re-entrant per thread; a shared lock cannot be upgraded).
+
+Inside `Lutaml::Store`, a bare `Monitor` resolves to `Lutaml::Store::Monitor` (the stats collector) — write `::Monitor` for Ruby's re-entrant lock.
+
+Specs that load `sqlite3` must sort after `autoload_spec.rb` (see `sqlite_scale_spec.rb`). Run specs with a UTF-8 locale (`LANG=C.UTF-8`); the anti-pattern guard fails under US-ASCII.
+
 ### PackageStore and transports
 
 `PackageStore` provides structured multi-model persistence. `PackageDefinition` declares which models, assets, and metadata the package contains. Transports (`DirectoryTransport`, `ZipTransport`) handle reading/writing to disk. Format handlers determine serialization per model entry.

data/README.adoc

@@ -228,7 +228,21 @@ with optional caching, monitoring, and event emission.
 === Storage adapters
 
 All adapters inherit from `Adapter::Base` and provide `get`, `set`, `delete`,
-`exists?`, `keys`, `all`, `clear`, `size`, `each_key`, and bulk operations.
+`exists?`, `keys`, `all`, `clear`, `size`, `each_key`, `update`, and bulk
+operations. `each_key` defaults to `keys.each`, so a custom adapter only needs
+`keys`.
+
+`update(key) { |old| new }` is a read-modify-write: the block gets the current
+value (`nil` when the key is missing) and its result is stored. It is atomic
+across threads for Memory, and across threads and processes for FileSystem and
+SQLite. `BasicStore#update` calls it and refreshes the read cache.
+
+[source,ruby]
+----
+store = Lutaml::Store::BasicStore.new(adapter_type: :filesystem,
+                                      adapter_options: { path: "./data" })
+store.update("counter") { |n| (n.to_i + 1).to_s }
+----
 
 [cols="1,1,3"]
 |===
@@ -641,6 +655,21 @@ store = Lutaml::Store.new(
 Persistent file-based storage with SHA-256 integrity checks. Files organized
 by key in subdirectories.
 
+* *File names.* Every byte outside `[A-Za-z0-9._-]` is percent-encoded, so
+  `ISO/19115` and `ISO:19115` get different files and `keys` returns the
+  original keys. A leading `.` is encoded too, so no file is hidden or outside
+  the root. A name longer than 200 bytes becomes `~` plus the SHA-256 of the
+  key; the `.meta` file beside it keeps the key. Files written by 0.3.0 (with
+  `_` in place of unsafe characters) are still read, and move to the new name
+  on the next write of that key.
+* *Values.* A `String` is written as is. Any other value (for example the
+  Hash that `DatabaseStore` saves) is written as JSON and read back as JSON.
+* *Locks.* Writes hold an exclusive `flock` on `<path>/.lock` and reads hold a
+  shared one. Data and `.meta` files are written to a unique temp file and
+  renamed into place, so readers never see a partial write.
+* *Case-insensitive file systems* (the macOS default) can still merge keys that
+  differ only in letter case.
+
 === SQLite
 
 [source,ruby]
@@ -681,8 +710,20 @@ Error hierarchy:
 
 == Thread safety
 
-All adapters use mutex-based synchronization. Safe for concurrent use across
-threads.
+* `Adapter::Memory`: writes are synchronized with a Monitor; reads use a
+  frozen snapshot.
+* `Adapter::FileSystem`: safe across threads and processes that share one
+  directory (see the FileSystem section above). `transaction { ... }` holds the exclusive
+  lock for the whole block.
+* `Adapter::Sqlite`: SQLite locking with a busy timeout; `update` uses
+  `BEGIN IMMEDIATE`.
+* `CacheStore`: all operations run under one lock per instance. The TTL and
+  LRU bookkeeping is per process.
+* `BasicStore`: the optional read cache is per process. Disable it
+  (`cache: { enabled: false }`) when other processes write the same store.
+
+A `get` followed by a `set` is never atomic. Use `update` for a
+read-modify-write.
 
 == Development
 

data/lib/lutaml/store/adapter/base.rb

@@ -31,7 +31,9 @@ module Lutaml
         end
 
         def each_key(&block)
-          raise NotImplementedError
+          return enum_for(:each_key) unless block
+
+          keys.each(&block)
         end
 
         def all
@@ -88,6 +90,17 @@ module Lutaml
           yield
         end
 
+        # Read-modify-write: yields the current value (nil when missing) and
+        # stores the block's result. Atomic only where the adapter's
+        # #transaction is; Memory, FileSystem and Sqlite override this.
+        def update(key)
+          transaction do
+            new_value = yield(get(key))
+            set(key, new_value)
+            new_value
+          end
+        end
+
         def stats
           { adapter: self.class.name }
         end

data/lib/lutaml/store/adapter/filesystem.rb

@@ -3,13 +3,46 @@
 require "fileutils"
 require "digest"
 require "json"
+require "monitor"
+require "securerandom"
 
 module Lutaml
   module Store
     module Adapter
+      # One data file per key, with an optional `.meta` file beside it.
+      #
+      # File names percent-encode every byte outside `[A-Za-z0-9._-]` and a
+      # leading ".", so distinct keys never share a file and no name is hidden
+      # or climbs out of the root. The empty key is named "%". A name longer
+      # than MAX_NAME_BYTES becomes "~" + SHA-256 of the key, and the `.meta`
+      # file keeps the key. Files written by 0.3.0 (lossy "_" names) are still
+      # read, and are moved to the new name on the next write.
+      #
+      # Writers hold an exclusive lock on `<root>/.lock` (plus a per-root
+      # Monitor for threads); readers hold a shared lock. Files are written
+      # to a unique temp file and renamed into place.
       class FileSystem < Base
         DEFAULT_EXTENSION = ".data"
         METADATA_EXTENSION = ".meta"
+        LOCK_FILE = ".lock"
+        MAX_NAME_BYTES = 200
+        DIGEST_PREFIX = "~"
+        JSON_FORMAT = "json"
+        UNSAFE_BYTE = /[^A-Za-z0-9._-]/n
+        LEGACY_UNSAFE_CHAR = /[^a-zA-Z0-9._-]/
+        EMPTY_KEY_NAME = "%"
+        HELD_LOCKS = :lutaml_store_filesystem_locks
+
+        @monitors = {}
+        @monitors_guard = Mutex.new
+
+        class << self
+          # One Monitor per lock file, shared by every adapter instance in
+          # the process, so two instances on one root exclude each other.
+          def monitor_for(lock_path)
+            @monitors_guard.synchronize { @monitors[lock_path] ||= ::Monitor.new }
+          end
+        end
 
         def initialize(config = {})
           super
@@ -18,137 +51,134 @@ module Lutaml
           @create_directories = @config.fetch(:create_directories, true)
           @integrity_enabled = @config.fetch(:integrity_checks, true)
           @integrity_algorithm = @config.fetch(:integrity_algorithm, "sha256")
+          @lock_path = File.expand_path(File.join(@root_path, LOCK_FILE))
+          @monitor = self.class.monitor_for(@lock_path)
 
           setup_directory_structure
         end
 
         def get(key)
-          file_path = path_for_key(key)
-          return nil unless File.exist?(file_path)
+          status, result = with_lock(:sh) { read_verified(key) }
+          return result unless status == :corrupt
 
-          data = file_safe_read(file_path)
-          metadata = read_metadata(key)
-
-          begin
-            verify_data_integrity(data, metadata)
-          rescue Integrity::IntegrityError => e
-            repaired_data = repair_corruption(key)
-            return repaired_data if repaired_data
+          # Repair writes, so it runs under the exclusive lock, never the shared one.
+          with_lock(:ex) do
+            status, result = read_verified(key)
+            next result unless status == :corrupt
 
-            raise e
+            repair_corruption(key) || raise(result)
           end
-
-          data
         end
 
         def set(key, value)
-          file_path = path_for_key(key)
-          ensure_directory_exists(File.dirname(file_path))
-
-          full_metadata = wrap_with_integrity(value)
-          file_safe_write(file_path, value)
-          write_metadata(key, full_metadata) if full_metadata.any?
+          content, format = encode_value(value)
+          with_lock(:ex) { write_entry(key, content, format) }
           value
         end
 
-        def delete(key)
-          file_path = path_for_key(key)
-          metadata_path = metadata_path_for_key(key)
+        # Atomic read-modify-write: the exclusive lock is held across the
+        # read, the block and the write, for threads and processes.
+        def update(key)
+          with_lock(:ex) do
+            new_value = yield(get(key))
+            set(key, new_value)
+            new_value
+          end
+        end
 
-          return nil unless File.exist?(file_path)
+        def transaction(&block)
+          with_lock(:ex, &block)
+        end
 
-          value = file_safe_read(file_path)
-          File.delete(file_path)
-          File.delete(metadata_path) if File.exist?(metadata_path)
+        def delete(key)
+          with_lock(:ex) do
+            file_path = existing_path_for_key(key)
+            next nil unless file_path
 
-          cleanup_empty_directories(File.dirname(file_path))
+            value = decode_value(file_safe_read(file_path), read_metadata(metadata_path_for(file_path))[:format])
+            remove_entry(file_path)
+            remove_legacy_entry(key)
 
-          value
+            value
+          end
         end
 
         def exists?(key)
-          File.exist?(path_for_key(key))
+          !existing_path_for_key(key).nil?
         end
 
         def keys
           return [] unless Dir.exist?(@root_path)
 
-          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).filter_map do |file_path|
-            key_from_path(file_path) if File.file?(file_path)
+          with_lock(:sh) do
+            data_files.filter_map { |file_path| key_from_path(file_path) }
           end
         end
 
         def all
-          result = {}
-
-          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).each do |file_path|
-            next unless File.file?(file_path)
-
-            key = key_from_path(file_path)
+          keys.each_with_object({}) do |key, result|
             value = get(key)
-            result[key] = value if value
+            result[key] = value unless value.nil?
           end
-
-          result
         end
 
         def clear
           return 0 unless Dir.exist?(@root_path)
 
-          count = 0
-          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).each do |file_path|
-            if File.file?(file_path)
+          with_lock(:ex) do
+            count = 0
+            data_files.each do |file_path|
               File.delete(file_path)
               count += 1
             end
-          end
 
-          Dir.glob(File.join(@root_path, "**", "*#{METADATA_EXTENSION}")).each do |file_path|
-            File.delete(file_path) if File.file?(file_path)
-          end
+            Dir.glob(File.join(@root_path, "**", "*#{METADATA_EXTENSION}")).each do |file_path|
+              File.delete(file_path) if File.file?(file_path)
+            end
 
-          cleanup_empty_directories(@root_path, preserve_root: true)
+            cleanup_empty_directories(@root_path, preserve_root: true)
 
-          count
+            count
+          end
         end
 
         def size
           return 0 unless Dir.exist?(@root_path)
 
-          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).count { |path| File.file?(path) }
+          data_files.size
         end
 
         def close; end
 
         def verify_integrity
           corrupted_keys = []
+          all_keys = keys
 
-          keys.each do |key|
+          all_keys.each do |key|
             get(key)
           rescue Integrity::IntegrityError
             corrupted_keys << key
           end
 
           {
-            total_keys: keys.size,
+            total_keys: all_keys.size,
             corrupted_keys: corrupted_keys,
             integrity_ok: corrupted_keys.empty?
           }
         end
 
         def repair_corruption(key, backup_data = nil)
-          file_path = path_for_key(key)
-          return nil unless File.exist?(file_path)
+          with_lock(:ex) do
+            file_path = existing_path_for_key(key)
+            next nil unless file_path
 
-          corrupted_data = file_safe_read(file_path)
-          repaired_data = Integrity.repair_data(corrupted_data, backup_data)
+            format = read_metadata(metadata_path_for(file_path))[:format]
+            repaired_data = Integrity.repair_data(file_safe_read(file_path), backup_data)
+            next nil unless Integrity.valid_data?(repaired_data)
 
-          if Integrity.valid_data?(repaired_data)
-            set(key, repaired_data)
-            return repaired_data
+            write_entry(key, repaired_data, format)
+            decode_value(repaired_data, format)
           end
-
-          nil
         end
 
         def stats
@@ -167,31 +197,197 @@ module Lutaml
           FileUtils.mkdir_p(@root_path) unless Dir.exist?(@root_path)
         end
 
+        # ── Locking ──
+
+        # Re-entrant for the thread that already holds a lock on this root.
+        # A shared lock cannot be upgraded: flock would deadlock on its own fd.
+        def with_lock(mode, &block)
+          held = held_lock_mode
+          if held
+            raise BackendError, "Cannot take an exclusive lock inside a shared lock" if mode == :ex && held == :sh
+
+            return yield
+          end
+
+          if mode == :ex
+            @monitor.synchronize { flock_lock_file(File::LOCK_EX, :ex, &block) }
+          else
+            flock_lock_file(File::LOCK_SH, :sh, &block)
+          end
+        end
+
+        def flock_lock_file(operation, mode)
+          # Nothing to read and nothing to protect before the root exists.
+          return yield if mode == :sh && !Dir.exist?(@root_path)
+
+          ensure_directory_exists(@root_path)
+          File.open(@lock_path, File::RDWR | File::CREAT, 0o644) do |lock_file|
+            lock_file.flock(operation)
+            held_locks[@lock_path] = { mode: mode, pid: Process.pid, file: lock_file }
+            yield
+          ensure
+            held_locks.delete(@lock_path)
+          end
+        end
+
+        # A lock entry copied into a forked child is not the child's lock.
+        # The child closes its copy of the lock file, so the parent's flock
+        # is released when the parent closes its own descriptor.
+        def held_lock_mode
+          entry = held_locks[@lock_path]
+          return nil unless entry
+          return entry[:mode] if entry[:pid] == Process.pid
+
+          held_locks.delete(@lock_path)
+          entry[:file].close unless entry[:file].closed?
+          nil
+        end
+
+        # Per thread, not per fiber: an Enumerator or Fiber that runs inside
+        # a locked block re-enters the lock instead of waiting on itself.
+        def held_locks
+          Thread.current.thread_variable_get(HELD_LOCKS) ||
+            Thread.current.thread_variable_set(HELD_LOCKS, {})
+        end
+
+        # ── Entries ──
+
+        # Returns [:ok, value], [:missing, nil] or [:corrupt, error].
+        def read_verified(key)
+          file_path = existing_path_for_key(key)
+          return [:missing, nil] unless file_path
+
+          data = file_safe_read(file_path)
+          metadata = read_metadata(metadata_path_for(file_path))
+
+          begin
+            verify_data_integrity(data, metadata)
+          rescue Integrity::IntegrityError => e
+            return [:corrupt, e]
+          end
+
+          [:ok, decode_value(data, metadata[:format])]
+        end
+
+        def write_entry(key, content, format)
+          file_path = path_for_key(key)
+          metadata_path = metadata_path_for(file_path)
+          ensure_directory_exists(File.dirname(file_path))
+
+          metadata = {}
+          metadata[:integrity] = create_integrity_metadata(content) if @integrity_enabled
+          metadata[:format] = format if format
+          metadata[:key] = key.to_s if File.basename(file_path).start_with?(DIGEST_PREFIX)
+
+          file_safe_write(file_path, content)
+          if metadata.any?
+            file_safe_write(metadata_path, JSON.generate(metadata))
+          elsif File.exist?(metadata_path)
+            File.delete(metadata_path)
+          end
+          remove_legacy_entry(key)
+        end
+
+        def remove_entry(file_path)
+          metadata_path = metadata_path_for(file_path)
+          File.delete(file_path) if File.exist?(file_path)
+          File.delete(metadata_path) if File.exist?(metadata_path)
+          cleanup_empty_directories(File.dirname(file_path))
+        end
+
+        def remove_legacy_entry(key)
+          legacy_path = legacy_path_for_key(key)
+          remove_entry(legacy_path) if legacy_path && File.exist?(legacy_path)
+        end
+
+        def encode_value(value)
+          return [value, nil] if value.is_a?(String)
+
+          [JSON.generate(value), JSON_FORMAT]
+        end
+
+        def decode_value(content, format)
+          return content unless format == JSON_FORMAT
+
+          JSON.parse(content)
+        rescue JSON::ParserError
+          content
+        end
+
+        # ── Paths ──
+
+        def data_files
+          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).select { |path| File.file?(path) }
+        end
+
         def path_for_key(key)
-          safe_key = sanitize_key(key)
+          shard_path(file_name_for_key(key))
+        end
 
-          if safe_key.length >= 2
-            subdir = safe_key[0, 2]
-            File.join(@root_path, subdir, "#{safe_key}#{@extension}")
+        def shard_path(name)
+          if name.length >= 2
+            File.join(@root_path, name[0, 2], "#{name}#{@extension}")
           else
-            File.join(@root_path, "#{safe_key}#{@extension}")
+            File.join(@root_path, "#{name}#{@extension}")
           end
         end
 
-        def metadata_path_for_key(key)
-          data_path = path_for_key(key)
-          data_path.sub(@extension, METADATA_EXTENSION)
+        # The current path, or the 0.3.0 path when only that one exists.
+        def existing_path_for_key(key)
+          file_path = path_for_key(key)
+          return file_path if File.exist?(file_path)
+
+          legacy_path = legacy_path_for_key(key)
+          legacy_path if legacy_path && File.exist?(legacy_path)
+        end
+
+        # 0.3.0 replaced unsafe characters with "_". Nil when that name is
+        # the current one, or when it would leave the root ("..").
+        def legacy_path_for_key(key)
+          name = key.to_s.gsub(LEGACY_UNSAFE_CHAR, "_")
+          return nil if name.empty? || name.start_with?(".")
+
+          legacy_path = shard_path(name)
+          legacy_path unless legacy_path == path_for_key(key)
+        end
+
+        def metadata_path_for(data_path)
+          "#{data_path.delete_suffix(@extension)}#{METADATA_EXTENSION}"
         end
 
         def key_from_path(file_path)
-          relative_path = file_path.sub(@root_path, "").sub(%r{^/}, "")
-          File.basename(relative_path, @extension)
+          name = File.basename(file_path, @extension)
+          return decode_key(name) unless name.start_with?(DIGEST_PREFIX)
+
+          read_metadata(metadata_path_for(file_path))[:key]
         end
 
-        def sanitize_key(key)
-          key.to_s.gsub(/[^a-zA-Z0-9._-]/, "_")
+        def file_name_for_key(key)
+          encoded = encode_key(key)
+          return encoded if encoded.bytesize <= MAX_NAME_BYTES
+
+          "#{DIGEST_PREFIX}#{Digest::SHA256.hexdigest(key.to_s)}"
+        end
+
+        def encode_key(key)
+          raw = key.to_s
+          return EMPTY_KEY_NAME if raw.empty?
+
+          encoded = raw.b.gsub(UNSAFE_BYTE) { |byte| format("%%%02X", byte.ord) }
+          encoded = "%2E#{encoded[1..]}" if encoded.start_with?(".")
+          encoded.force_encoding(Encoding::UTF_8)
+        end
+
+        def decode_key(name)
+          return "" if name == EMPTY_KEY_NAME
+
+          decoded = name.b.gsub(/%([0-9A-F]{2})/n) { Regexp.last_match(1).hex.chr }
+          decoded.force_encoding(Encoding::UTF_8)
+          decoded.valid_encoding? ? decoded : decoded.b
         end
 
+        # ── Integrity and metadata ──
+
         def create_integrity_metadata(data)
           return {} unless @integrity_enabled
 
@@ -205,32 +401,16 @@ module Lutaml
           Integrity.verify_integrity_metadata(data, metadata[:integrity])
         end
 
-        def wrap_with_integrity(data, user_metadata = {})
-          metadata = user_metadata.dup
-          metadata[:integrity] = create_integrity_metadata(data) if @integrity_enabled
-          metadata
-        end
-
-        def write_metadata(key, metadata)
-          return unless metadata.any?
-
-          metadata_path = metadata_path_for_key(key)
-          ensure_directory_exists(File.dirname(metadata_path))
-
-          File.write(metadata_path, JSON.generate(metadata))
-        end
-
-        def read_metadata(key)
-          metadata_path = metadata_path_for_key(key)
+        def read_metadata(metadata_path)
           return {} unless File.exist?(metadata_path)
 
-          begin
-            JSON.parse(File.read(metadata_path), symbolize_names: true)
-          rescue JSON::ParserError
-            {}
-          end
+          JSON.parse(File.read(metadata_path), symbolize_names: true)
+        rescue JSON::ParserError, Errno::ENOENT
+          {}
         end
 
+        # ── Files ──
+
         def ensure_directory_exists(dir_path)
           return if Dir.exist?(dir_path)
 
@@ -249,38 +429,33 @@ module Lutaml
         end
 
         def file_safe_read(file_path)
-          File.open(file_path, "rb") do |file|
-            file.flock(File::LOCK_SH)
-            file.read
-          end
+          content = File.binread(file_path)
+          utf8 = content.dup.force_encoding(Encoding::UTF_8)
+          utf8.valid_encoding? ? utf8 : content
         rescue StandardError => e
           raise BackendError, "Failed to read file #{file_path}: #{e.message}"
         end
 
+        # The temp name is unique per call, so concurrent writers never share
+        # a temp file; rename makes the new content visible atomically.
         def file_safe_write(file_path, content)
-          temp_path = "#{file_path}.tmp.#{Process.pid}"
+          temp_path = "#{file_path}.tmp.#{Process.pid}.#{SecureRandom.hex(6)}"
 
           File.open(temp_path, "wb") do |file|
-            file.flock(File::LOCK_EX)
             file.write(content)
             file.fsync
           end
 
           File.rename(temp_path, file_path)
         rescue StandardError => e
-          File.delete(temp_path) if File.exist?(temp_path)
+          FileUtils.rm_f(temp_path)
           raise BackendError, "Failed to write file #{file_path}: #{e.message}"
         end
 
         def calculate_disk_usage
           return 0 unless Dir.exist?(@root_path)
 
-          total_size = 0
-          Dir.glob(File.join(@root_path, "**", "*#{@extension}")).each do |file_path|
-            total_size += File.size(file_path) if File.file?(file_path)
-          end
-
-          total_size
+          data_files.sum { |file_path| File.size(file_path) }
         end
       end
     end

data/lib/lutaml/store/adapter/memory.rb

@@ -1,15 +1,18 @@
 # frozen_string_literal: true
 
+require "monitor"
+
 module Lutaml
   module Store
     module Adapter
       # In-memory key-value adapter optimized for read-heavy workloads.
       # Writes are synchronized; reads are lock-free using snapshot copies.
+      # The write lock is a re-entrant Monitor so #update can call #set.
       class Memory < Base
         def initialize(config = {})
           super
           @data = {}
-          @write_mutex = Mutex.new
+          @write_mutex = ::Monitor.new
           @read_snapshot = {}.freeze
           @snapshot_stale = true
           @ttl_enabled = @config.fetch(:ttl_enabled, false)
@@ -72,6 +75,18 @@ module Lutaml
           value
         end
 
+        # Atomic read-modify-write across threads. Keeps the key's expiry.
+        def update(key)
+          @write_mutex.synchronize do
+            old_value = get(key)
+            expires_at = @ttl_data&.[](key)
+            new_value = yield(old_value)
+            set(key, new_value)
+            @ttl_data[key] = expires_at if expires_at
+            new_value
+          end
+        end
+
         def delete(key)
           @write_mutex.synchronize do
             existed = @data.key?(key)

data/lib/lutaml/store/adapter/sqlite.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require "json"
+require "monitor"
 
 module Lutaml
   module Store
@@ -19,6 +20,7 @@ module Lutaml
           @db_path = @config[:path] || raise(ConfigurationError, "SQLite adapter requires :path config")
           @table_name = @config[:table_name] || DEFAULT_TABLE_NAME
           @timeout = @config[:timeout] || 30_000
+          @update_lock = ::Monitor.new
 
           setup_database
         end
@@ -178,14 +180,37 @@ module Lutaml
           results
         end
 
+        # Atomic read-modify-write. BEGIN IMMEDIATE takes the database write
+        # lock before the read, so no other process can write in between;
+        # the Monitor keeps threads that share this connection apart.
+        def update(key, &block)
+          @update_lock.synchronize do
+            if @db.transaction_active?
+              update_in_transaction(key, &block)
+            else
+              @db.transaction(:immediate) { update_in_transaction(key, &block) }
+            end
+          end
+        rescue SQLite3::Exception => e
+          raise BackendError, "Update failed: #{e.message}"
+        end
+
+        # Holds the update lock too, so a concurrent #update from another
+        # thread cannot run inside this transaction.
         def transaction(&block)
-          @db.transaction(&block)
+          @update_lock.synchronize { @db.transaction(&block) }
         rescue SQLite3::Exception => e
           raise BackendError, "Transaction failed: #{e.message}"
         end
 
         private
 
+        def update_in_transaction(key)
+          new_value = yield(get(key))
+          set(key, new_value)
+          new_value
+        end
+
         def setup_database
           @db = SQLite3::Database.new(@db_path)
           @db.busy_timeout = @timeout

data/lib/lutaml/store/basic_store.rb

@@ -57,6 +57,17 @@ module Lutaml
         end
       end
 
+      # Atomic read-modify-write through the adapter. The block gets the
+      # adapter's current value, not the cached one.
+      def update(key, &block)
+        with_monitoring(:set) do
+          value = @adapter.update(key, &block)
+          @cache&.set(key, value)
+          emit_event(:set, key: key, value: value)
+          value
+        end
+      end
+
       def delete(key)
         with_monitoring(:delete) do
           result = @adapter.delete(key)

data/lib/lutaml/store/cache_store.rb

@@ -1,11 +1,16 @@
 # frozen_string_literal: true
 
 require "json"
+require "monitor"
 require "time"
 
 module Lutaml
   module Store
     # TTL-aware cache store with LRU eviction. Wraps a storage adapter directly.
+    #
+    # All public operations run under one re-entrant lock, so the LRU and
+    # cleanup bookkeeping is safe across threads. Across processes only the
+    # adapter's own guarantees apply.
     class CacheStore
       class CacheEntry
         attr_reader :value, :created_at, :ttl, :metadata
@@ -58,66 +63,75 @@ module Lutaml
         @cleanup_interval = config[:cleanup_interval] || 300
         @last_cleanup = Time.now
         @access_times = {}
+        @lock = ::Monitor.new
       end
 
       def get(key)
-        cleanup_expired if should_cleanup?
+        @lock.synchronize do
+          cleanup_if_due
 
-        entry_data = @adapter.get(key)
-        return nil unless entry_data
+          entry_data = @adapter.get(key)
+          next nil unless entry_data
 
-        begin
-          entry = deserialize_entry(entry_data)
+          begin
+            entry = deserialize_entry(entry_data)
+
+            if entry.expired?
+              delete(key)
+              next nil
+            end
 
-          if entry.expired?
+            @access_times[key] = Time.now
+            entry.value
+          rescue StandardError
             delete(key)
-            return nil
+            nil
           end
-
-          @access_times[key] = Time.now
-          entry.value
-        rescue StandardError
-          delete(key)
-          nil
         end
       end
 
       def set(key, value, ttl: :default, metadata: {})
-        cleanup_expired if should_cleanup?
-        evict_if_needed
+        @lock.synchronize do
+          cleanup_if_due
+          evict_if_needed
 
-        effective_ttl = ttl == :default ? @default_ttl : ttl
-        entry = CacheEntry.new(value, ttl: effective_ttl, metadata: metadata)
+          effective_ttl = ttl == :default ? @default_ttl : ttl
+          entry = CacheEntry.new(value, ttl: effective_ttl, metadata: metadata)
 
-        serialized_entry = serialize_entry(entry)
-        @adapter.set(key, serialized_entry)
+          serialized_entry = serialize_entry(entry)
+          @adapter.set(key, serialized_entry)
 
-        @access_times[key] = Time.now
-        value
+          @access_times[key] = Time.now
+          value
+        end
       end
 
       def delete(key)
-        value = nil
-        entry_data = @adapter.get(key)
-        if entry_data
-          begin
-            entry = deserialize_entry(entry_data)
-            value = entry.value unless entry.expired?
-          rescue StandardError
-            # If we can't deserialize, treat as nil
+        @lock.synchronize do
+          value = nil
+          entry_data = @adapter.get(key)
+          if entry_data
+            begin
+              entry = deserialize_entry(entry_data)
+              value = entry.value unless entry.expired?
+            rescue StandardError
+              # If we can't deserialize, treat as nil
+            end
           end
-        end
 
-        @access_times.delete(key)
+          @access_times.delete(key)
 
-        deleted = @adapter.delete(key)
+          deleted = @adapter.delete(key)
 
-        deleted ? value : nil
+          deleted ? value : nil
+        end
       end
 
       def clear
-        @access_times.clear
-        @adapter.clear
+        @lock.synchronize do
+          @access_times.clear
+          @adapter.clear
+        end
       end
 
       def exists?(key)
@@ -135,12 +149,13 @@ module Lutaml
       end
 
       def keys
-        cleanup_expired if should_cleanup?
-        @adapter.keys.select { |key| exists?(key) }
+        @lock.synchronize do
+          cleanup_if_due
+          @adapter.keys.select { |key| exists?(key) }
+        end
       end
 
       def size
-        cleanup_expired if should_cleanup?
         keys.size
       end
 
@@ -169,22 +184,7 @@ module Lutaml
       end
 
       def cleanup_expired
-        expired_keys = []
-
-        @adapter.each_key do |key|
-          entry_data = @adapter.get(key)
-          next unless entry_data
-
-          entry = deserialize_entry(entry_data)
-          expired_keys << key if entry.expired?
-        rescue StandardError
-          expired_keys << key
-        end
-
-        expired_keys.each { |key| delete(key) }
-        @last_cleanup = Time.now
-
-        expired_keys.size
+        @lock.synchronize { cleanup_expired_entries }
       end
 
       def cache_info
@@ -203,23 +203,25 @@ module Lutaml
       end
 
       def touch(key, ttl: nil)
-        entry_data = @adapter.get(key)
-        return false unless entry_data
+        @lock.synchronize do
+          entry_data = @adapter.get(key)
+          next false unless entry_data
 
-        begin
-          entry = deserialize_entry(entry_data)
-          return false if entry.expired?
+          begin
+            entry = deserialize_entry(entry_data)
+            next false if entry.expired?
 
-          new_ttl = ttl || entry.ttl
-          new_entry = CacheEntry.new(entry.value, ttl: new_ttl, metadata: entry.metadata)
+            new_ttl = ttl || entry.ttl
+            new_entry = CacheEntry.new(entry.value, ttl: new_ttl, metadata: entry.metadata)
 
-          serialized_entry = serialize_entry(new_entry)
-          @adapter.set(key, serialized_entry)
+            serialized_entry = serialize_entry(new_entry)
+            @adapter.set(key, serialized_entry)
 
-          @access_times[key] = Time.now
-          true
-        rescue StandardError
-          false
+            @access_times[key] = Time.now
+            true
+          rescue StandardError
+            false
+          end
         end
       end
 
@@ -263,14 +265,45 @@ module Lutaml
         Time.now - @last_cleanup > @cleanup_interval
       end
 
+      # Caller holds @lock.
+      def cleanup_if_due
+        cleanup_expired_entries if should_cleanup?
+      end
+
+      # Caller holds @lock. @last_cleanup is set first, so a nested call
+      # (through #delete) does not start a second scan.
+      def cleanup_expired_entries
+        @last_cleanup = Time.now
+        expired_keys = []
+
+        @adapter.each_key do |key|
+          entry_data = @adapter.get(key)
+          next unless entry_data
+
+          entry = deserialize_entry(entry_data)
+          expired_keys << key if entry.expired?
+        rescue StandardError
+          expired_keys << key
+        end
+
+        expired_keys.each { |key| delete(key) }
+
+        expired_keys.size
+      end
+
+      # Caller holds @lock. Keys this process never touched count as the
+      # least recently used.
       def evict_if_needed
         return unless @max_size
-        return if size < @max_size
 
-        keys_by_access = @access_times.sort_by { |_, time| time }.map(&:first)
-        keys_to_evict = keys_by_access.first(size - @max_size + 1)
+        stored_keys = @adapter.keys
+        overflow = stored_keys.size - @max_size + 1
+        return unless overflow.positive?
+
+        untouched = stored_keys.reject { |key| @access_times.key?(key) }
+        by_access = @access_times.sort_by { |_, time| time }.map(&:first)
 
-        keys_to_evict.each { |key| delete(key) }
+        (untouched + by_access).first(overflow).each { |key| delete(key) }
       end
     end
   end

data/lib/lutaml/store/config.rb

@@ -14,7 +14,7 @@ module Lutaml
                      cache: {}, monitoring: {}, events: {},
                      compression: {}, serialization: {}, **)
         @adapter_type = normalize_adapter_type(adapter_type)
-        @adapter_options = symbolize_keys(adapter_options)
+        @adapter_options = inline_adapter_options(adapter_type).merge(symbolize_keys(adapter_options))
 
         cache_config = symbolize_keys(cache)
         @cache_enabled = cache_config.fetch(:enabled, true)
@@ -134,6 +134,18 @@ module Lutaml
         end
       end
 
+      # Options given inline with the type, as in
+      # `adapter: { type: :filesystem, path: "./data" }` or
+      # `adapter: { type: :sqlite, options: { path: "a.db" } }`.
+      def inline_adapter_options(adapter_type)
+        return {} unless adapter_type.is_a?(Hash)
+
+        inline = symbolize_keys(adapter_type)
+        nested = inline.delete(:options) || {}
+        inline.delete(:type)
+        inline.merge(nested)
+      end
+
       def validate_adapter_config
         valid_adapters = %i[memory filesystem sqlite]
         unless valid_adapters.include?(@adapter_type)

data/lib/lutaml/store/version.rb

@@ -2,6 +2,6 @@
 
 module Lutaml
   module Store
-    VERSION = "0.3.0"
+    VERSION = "0.3.1"
   end
 end

metadata

@@ -1,13 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lutaml-store
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.3.1
 platform: ruby
 authors:
 - Ronald Tse
+autorequire:
 bindir: exe
 cert_chain: []
-date: 1980-01-02 00:00:00.000000000 Z
+date: 2026-09-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: lutaml-model
@@ -139,6 +140,7 @@ metadata:
   source_code_uri: https://github.com/lutaml/lutaml-store
   changelog_uri: https://github.com/lutaml/lutaml-store
   rubygems_mfa_required: 'true'
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -153,7 +155,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.16
+rubygems_version: 3.5.22
+signing_key:
 specification_version: 4
 summary: Store-centric database-style API for Lutaml::Model with multi-backend persistence.
 test_files: []