relaton-index 0.2.19 → 0.2.20

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0e0ad0db979d523bfd05a561b5f81ba0aab46f41f02d524c804c4767fa50e57b
-  data.tar.gz: f9e2c9fc9bdc73f8ae55cc5badd54190fbb0d2bef812c16c42ff6a4ed126958f
+  metadata.gz: 84483e9b27f8e48c618e2b6b7d124e618deebba9fa2ffb34d234027c3738ad32
+  data.tar.gz: '084e50ecfa029fb812e9552189ef795a901d5683b2f79baf643d55da0af93a78'
 SHA512:
-  metadata.gz: d53ca914ea49bc69a1c80833ad7996a6bde8b5c583dfd7b8ac4aa858e5a3bb5031050271adafbbb58a511f987769e4133b5bbac0479e5455a2c36b0a708031a2
-  data.tar.gz: e01cdbea3d72c1c76d48a735adba13af8af34f9c3cdd994de8532a8c8eef815548396c9d254a7824a1d7a83ce3d38ab41ca61602f851479f6111c35b035bbacf
+  metadata.gz: 198b94038473955e9c03f9e21b2e5876e9d36e94f72e11aec80d78350d988be5332cbbaceacf3e4253281b107afc4be370b66b89000df9d42f5dbdb17e5d8016
+  data.tar.gz: 38690e3090d278e9204200afefceac74eef9a4b5b40509fa2bea89d5c24053256c0f97c9c678f7642ea398caeb88b9710b7d272b8e55afc1bc6c1ca9d2a23fa1

data/CLAUDE.md

@@ -0,0 +1,64 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+relaton-index is a Ruby gem that provides indexing and searching of Relaton document references. It maps document identifiers to file paths, supporting both local index creation (for publishing) and remote index consumption (downloading from URLs with 24-hour caching).
+
+## Commands
+
+```bash
+# Run all tests (default rake task)
+rake spec
+
+# Run linting
+rake rubocop
+
+# Run specific test file
+bundle exec rspec spec/relaton/type_spec.rb
+
+# Run specific test by name
+bundle exec rspec spec/relaton/file_io_spec.rb -e "fetch_and_save"
+
+# Install dependencies
+bin/setup
+
+# Interactive console
+bin/console
+```
+
+## Architecture
+
+### Core Classes (all under `Relaton::Index` module in `lib/relaton/index/`)
+
+- **`Relaton::Index`** (module, `lib/relaton/index.rb`) — Static API entry point. Delegates to Pool and Config. Main methods: `find_or_create`, `close`, `configure`.
+
+- **Pool** — Object pool that caches Type instances by document type (`:ISO`, `:IEC`, `:IHO`, etc.). Reuses existing indexes if parameters match, recreates if they change.
+
+- **Type** — Represents one index for a document type. Holds an array of `{id:, file:}` hashes. Provides `add_or_update`, `search` (string substring match or block), and `save`.
+
+- **FileIO** — Handles reading/writing/downloading index files. Three modes based on `@url`: string URL (download and cache to `~/.relaton/{type}/`), `true` (read local file from `~/.relaton/{type}/`), `nil` (read from current directory). Uses class-level Mutex for thread-safe downloads. Validates index format on load.
+
+- **FileStorage** — Storage abstraction module with `ctime`, `read`, `write`, `remove`. Can be replaced via `Config.storage=` for custom backends (e.g., S3).
+
+- **Config** — Global configuration: `storage`, `storage_dir`, `filename` (default: "index.yaml").
+
+### Data Flow
+
+1. `Relaton::Index.find_or_create(:TYPE, url:, file:, id_keys:, pubid_class:)` → Pool looks up or creates Type
+2. Type lazily loads index via FileIO on first access
+3. FileIO either reads local YAML or downloads ZIP from URL, extracts, validates format
+4. Search matches against `:id` field (string comparison via `include?` or custom block)
+5. `save` writes index as YAML to local file
+
+### Index Format
+
+YAML array of hashes with `:id` (string or structured hash) and `:file` (path string). Supports backward compatibility with old string-based format and newer pubid object format.
+
+### Key Design Decisions
+
+- Remote indexes cached for 24 hours at `~/.relaton/{type}/index.yaml`
+- Thread safety via `@@mutex` in FileIO prevents concurrent downloads of the same file
+- Pubid deserialization is optional — when `pubid_class` is provided, string IDs are converted to structured objects
+- Index format validation checks for required `:id` and `:file` keys, with automatic recovery (re-download or removal) on corruption

data/README.adoc

@@ -24,7 +24,7 @@ If bundler is not being used to manage dependencies, install the gem by executin
 
 === Creating an index object
 
-The gem provides the `Relaton::Index.find_or_create {type}, url: {url}, file: {filename}, id_keys: {keys}` method to create an index object. The first argument is the type of dataset (ISO, IEC, IHO, etc.). The second argument is the URL to the zipped remote index file. The third argument is the filename of the local index file. The fourth argument is an array of ID's parts names. The URL, filename, and keys are optional.
+The gem provides the `Relaton::Index.find_or_create {type}, url: {url}, file: {filename}, id_keys: {keys}, pubid_class: {class}` method to create an index object. The first argument is the type of dataset (ISO, IEC, IHO, etc.). The second argument is the URL to the zipped remote index file. The third argument is the filename of the local index file. The fourth argument is an array of ID's parts names. The fifth argument is a class that implements `Pubid::Core::Identifier` for deserializing ID hashes into structured identifier objects. The URL, filename, keys, and pubid_class are optional.
 
 If the URL is specified and the local file in a `/{home}/.relaton/{type}` dir doesn't exist or is outdated, the index file will be downloaded from the URL saved as a local file and an index object will be created from the file. If the file in the `/{home}/.relaton/{type}` exists and is actual, the index object will be created from the local file.
 
@@ -97,6 +97,30 @@ end
 # => [{ id: "B-4 2.19.0", file: "data/b-4_2_19_0.xml" }]
 ----
 
+=== Using pubid_class for structured identifiers
+
+The `pubid_class` option allows index entries to be deserialized into structured identifier objects instead of plain strings or hashes. The class must include `Pubid::Core::Identifier` and provide a `.create(**hash)` factory method.
+
+When `pubid_class` is specified, each `:id` hash in the index is converted into a pubid object, enabling structured access to identifier components (e.g., publisher, number, part, year).
+
+[source,ruby]
+----
+require 'relaton/index'
+require 'pubid-iso'
+
+# Create an index with pubid_class to deserialize IDs into Pubid::Iso::Identifier objects
+index = Relaton::Index.find_or_create :ISO,
+  url: "https://raw.githubusercontent.com/relaton/relaton-data-iso/main/index-v2.zip",
+  pubid_class: Pubid::Iso::Identifier
+
+# Search returns entries with structured pubid objects as IDs
+results = index.search "ISO 1"
+results.first[:id]
+# => #<Pubid::Iso::Identifier: ISO 1>
+results.first[:id].publisher
+# => "ISO"
+----
+
 === Remove all index records
 
 This method removes all records from the index object. The index file is not removed.

data/lib/relaton/index/file_io.rb

@@ -7,6 +7,7 @@ module Relaton
     #
     class FileIO
       attr_reader :url, :pubid_class
+      attr_accessor :sorted
 
       @@file_locks = {}
       @@file_locks_mutex = Mutex.new
@@ -28,6 +29,7 @@ module Relaton
         @filename = filename
         @id_keys = id_keys || []
         @pubid_class = pubid_class
+        @sorted = false
       end
 
       #
@@ -117,7 +119,15 @@ module Relaton
       def deserialize_pubid(index)
         return index unless @pubid_class
 
-        index.map { |r| { id: @pubid_class.create(**r[:id]), file: r[:file] } }
+        @sorted = true
+        prev_number = nil
+        index.map do |r|
+          id = @pubid_class.create(**r[:id])
+          num = get_id_number id
+          @sorted = false if prev_number && prev_number > num
+          prev_number = num
+          { id: id, file: r[:file] }
+        end
       end
 
       def warn_local_index_error(reason)
@@ -183,12 +193,24 @@ module Relaton
       # @return [void]
       #
       def save(index)
-        yaml = index.map do |item|
+        yaml = sort_structured_index(index).map do |item|
           item.transform_values { |value| value.is_a?(Pubid::Core::Identifier::Base) ? value.to_h : value }
         end.to_yaml
         Index.config.storage.write file, yaml
       end
 
+      def sort_structured_index(index)
+        if @pubid_class && index.first&.dig(:id).is_a?(Pubid::Core::Identifier::Base)
+          index.sort_by { |item| get_id_number item[:id] }
+        else
+          index
+        end
+      end
+
+      def get_id_number(id)
+        id.respond_to?(:base) && id.base ? id.base.number.to_s : id.number.to_s
+      end
+
       #
       # Remove index file from storage
       #

data/lib/relaton/index/type.rb

@@ -10,9 +10,11 @@ module Relaton
       # @param [String, Symbol] type type of index (ISO, IEC, etc.)
       # @param [String, nil] url external URL to index, used to fetch index for searching files
       # @param [String, nil] file output file name
-      # @param [Pubid::Core::Identifier::Base] pubid class for deserialization
+      # @param [Array<Symbol>] id_keys keys of identifier to be used for sorting index
+      #   format of index file is checked if id_keys all is provided at least in one of the IDs
+      # @param [Pubid::Core::Identifier::Base, nil] pubid class for deserialization
       #
-      def initialize(type, url = nil, file = nil, id_keys = nil, pubid_class = nil)
+      def initialize(type, url = nil, file = nil, id_keys = nil, pubid_class = nil) # rubocop:disable Metrics/ParameterLists
         @file = file
         filename = file || Index.config.filename
         @file_io = FileIO.new type.to_s.downcase, url, filename, id_keys, pubid_class
@@ -45,11 +47,15 @@ module Relaton
       # @return [void]
       #
       def add_or_update(id, file)
-        item = index.find { |i| i[:id] == id }
+        key = id.to_s
+        item = id_lookup[key]
         if item
           item[:file] = file
         else
-          index << { id: id, file: file }
+          new_item = { id: id, file: file }
+          index << new_item
+          id_lookup[key] = new_item
+          @file_io.sorted = false
         end
       end
 
@@ -60,18 +66,11 @@ module Relaton
       #
       # @return [Array<Hash>] search results
       #
-      def search(id = nil)
-        index.select do |i|
-          if block_given?
-            yield(i)
-          else
-            if i[:id].is_a?(String)
-              id.is_a?(String) ? i[:id].include?(id) : i[:id].include?(id.to_s)
-            else
-              id.is_a?(String) ? i[:id].to_s.include?(id) : i[:id] == id
-            end
-          end
-        end
+      def search(id = nil, &block)
+        items = search_candidates(id)
+        return items.select(&block) if block
+
+        items.select { |i| match_item(i, id) }
       end
 
       #
@@ -91,6 +90,7 @@ module Relaton
       def remove_file
         @file_io.remove
         @index = nil
+        @id_lookup = nil
       end
 
       #
@@ -100,6 +100,61 @@ module Relaton
       #
       def remove_all
         @index = []
+        @id_lookup = nil
+        @file_io.sorted = true
+      end
+
+      private
+
+      def id_lookup
+        @id_lookup ||= index.each_with_object({}) do |item, h|
+          h[item[:id].to_s] = item
+        end
+      end
+
+      def search_candidates(id)
+        # index needs to be created to check if sorted
+        idx = index
+        if @file_io.sorted && id && !id.is_a?(String)
+          candidates_by_number(id)
+        else
+          idx
+        end
+      end
+
+      def candidates_by_number(id)
+        target = get_id_number(id)
+        left = bsearch_left(target)
+        return [] unless left
+
+        right = bsearch_right(target)
+        index[left...right]
+      end
+
+      def get_id_number(id)
+        id.respond_to?(:base) && id.base ? id.base.number.to_s : id.number.to_s
+      end
+
+      def bsearch_left(target)
+        index.bsearch_index do |item|
+          get_id_number(item[:id]) >= target
+        end
+      end
+
+      def bsearch_right(target)
+        index.bsearch_index do |item|
+          get_id_number(item[:id]) > target
+        end || index.size
+      end
+
+      def match_item(item, id)
+        if item[:id].is_a?(String)
+          item[:id].include?(id.is_a?(String) ? id : id.to_s)
+        elsif id.is_a?(String)
+          item[:id].to_s.include?(id)
+        else
+          item[:id] == id
+        end
       end
     end
   end

data/lib/relaton/index/version.rb

@@ -2,6 +2,6 @@
 
 module Relaton
   module Index
-    VERSION = "0.2.19"
+    VERSION = "0.2.20"
   end
 end

data/relaton-index.gemspec

@@ -31,7 +31,7 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_dependency "openssl", "~> 3.3.2"
-  spec.add_dependency "pubid-core", "~> 1.15.0"
+  spec.add_dependency "pubid-core", "~> 1.15.6"
   spec.add_dependency "relaton-logger", "~> 0.2.0"
   spec.add_dependency "rubyzip", "~> 2.3.0"
 

metadata

@@ -1,13 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: relaton-index
 version: !ruby/object:Gem::Version
-  version: 0.2.19
+  version: 0.2.20
 platform: ruby
 authors:
 - Ribose Inc.
+autorequire:
 bindir: exe
 cert_chain: []
-date: 1980-01-02 00:00:00.000000000 Z
+date: 2026-02-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: openssl
@@ -29,14 +30,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.15.0
+        version: 1.15.6
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.15.0
+        version: 1.15.6
 - !ruby/object:Gem::Dependency
   name: relaton-logger
   requirement: !ruby/object:Gem::Requirement
@@ -65,6 +66,7 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 2.3.0
+description:
 email:
 - open.source@ribose.com
 executables: []
@@ -73,6 +75,7 @@ extra_rdoc_files: []
 files:
 - ".rspec"
 - ".rubocop.yml"
+- CLAUDE.md
 - Gemfile
 - LICENSE.txt
 - README.adoc
@@ -93,6 +96,7 @@ licenses:
 metadata:
   homepage_uri: https://github.com/relaton/relaton-index
   source_code_uri: https://github.com/relaton/relaton-index
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -107,7 +111,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.9
+rubygems_version: 3.5.22
+signing_key:
 specification_version: 4
 summary: Relaton Index is a library for indexing Relaton files.
 test_files: []