obp-access 0.1.5 → 0.1.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 728e6533423209e6fc2a17945ca7afc15f7b36c8bfe39451e1ad8f66d6554bb5
-  data.tar.gz: 92a04641a9ebc2774e60518e560871287c9855998848f2d3d2ffddcee95f0221
+  metadata.gz: b11820d71fdd40bfeeddd92447062fc728a2642a3bdd4b572fcb3162bd86230a
+  data.tar.gz: 4c061e02a1ead75b4db4828d7646d69052a44ed0369bb905c9a0aafb8f064a99
 SHA512:
-  metadata.gz: e4f9adc699152992207abfc9986d5e06e8db3a110f37eb2ed1e6cfd4f3dcd57d6f4ee63614ed44b26a40bcf70d355a3ebd2cf51041ba961040a5063a9ee88802
-  data.tar.gz: 883124569b65db99872fa2f12de85070ee72dea94fa113b6af87d94a8ecc2bb2c61f7eaf0a261eacdb69d817e032c9820810fb5973301043476f0f15768e5989
+  metadata.gz: 819be7f79f169115d5e0c486f09e8b061ac68fb85040e764832f0280e63094893c9d2da2428395f32179fff03625906f841ed4dea4020c7aa484873170603d49
+  data.tar.gz: ae43acfe12065482d2bfb872e59b0a221416664c43415a49078b206957e5190744b3ebafd60c7f78e5bb15821f04693f876f7de5127238cdabab29af95c125d4

data/CLAUDE.md

@@ -9,10 +9,11 @@ Ruby gem (`obp-access`) that fetches informative content (introduction, scope, t
 ## Commands
 
 - **Install dependencies:** `bundle install`
-- **Run tests:** `bundle exec rspec`
+- **Run tests:** `bundle exec rspec` (note: retriever/access-fetch specs hit the live OBP API)
 - **Lint:** `bundle exec rubocop`
 - **Run a single test:** `bundle exec rspec spec/obp/access/grammar_parser_spec.rb`
 - **Console:** `bin/console` (loads the gem for interactive use)
+- **CLI:** `bundle exec exe/obp-access` (fetch a document or all languages from OBP)
 
 ## Critical rules
 
@@ -25,13 +26,15 @@ Ruby gem (`obp-access`) that fetches informative content (introduction, scope, t
 
 Data flows through these classes in `lib/obp/access/`:
 
-1. **`Access`** (entry point) — orchestrates the pipeline. `Obp::Access.fetch(urn)` returns an instance per language. `Access.fetch_all(urn, languages:)` returns separate instances per language.
+1. **`Access`** (entry point) — orchestrates the pipeline. `Obp::Access.fetch(urn)` returns an instance per language. `Access.fetch_all(urn, languages:)` returns separate instances per language. `Access.from_html(urn:, html:, caption:, titles:)` converts an already-downloaded OBP HTML fragment with no network access (the offline path used by fixture-based tests).
 
-2. **`Parser`** — fetches content from the ISO OBP API (`https://www.iso.org/obp/ui`) via HTTP POST with a URN payload. Parses the JSON response to extract HTML content, titles, and images.
+2. **`Parser`** — fetches content from the ISO OBP API (`https://www.iso.org/obp/ui`) via HTTP POST with a URN payload. Parses the JSON response to extract HTML content, titles, and images. **`RawHtmlParser`** subclasses it and duck-types the API for pre-captured HTML.
 
-3. **`Converter`** — wraps the HTML source, normalizes whitespace, parses it with Nokogiri, and passes DOM nodes to the Renderer.
+3. **`Urn`** — parses URN identity metadata (`originator`, `doc_number` incl. part numbers like `80000:-12` → "80000-12", `doc_type` IS/TS/TR/PAS/Guide/IWA, `edition`, `version`). `Elements::Root` builds `<std-ident>` from these — never parse URN segments positionally elsewhere.
 
-4. **`Renderer`** — recursively walks DOM nodes and dispatches them to element classes registered in `ElementRegistry`. Each element class matches against CSS classes and builds NISO STS XML.
+4. **`Converter`** — wraps the HTML source, normalizes whitespace, parses it with Nokogiri, and passes DOM nodes to the Renderer.
+
+5. **`Renderer`** — recursively walks DOM nodes and dispatches them to element classes registered in `ElementRegistry`. Each element class matches against CSS classes and builds NISO STS XML.
 
 ### Element system
 
@@ -43,11 +46,15 @@ Data flows through these classes in `lib/obp/access/`:
 
 ### Supporting classes
 
-- **`GrammarParser`** — extracts part-of-speech and gender from bold term markup.
+- **`GrammarParser`** — extracts part-of-speech and gender from bold term markup (POS markers `adj.`/`verb`, gender markers `m`/`f`/`n`, 〈bracketed〉 subject fields). Gender/POS only occur in non-English term entries (fr/de/ru); examples live in `spec/obp/access/grammar_parser_spec.rb`.
 - **`DomainExtractor`** — extracts subject-field domains from definition text.
 - **`InlineRenderer`** — renders inline HTML elements (links, xrefs, italic, bold, entailed terms) to STS XML.
 - **`Imager`** — downloads images from OBP. Uses `Parallel` for concurrent downloads.
 
+### Testing conventions
+
+- Captured OBP HTML fixtures live in `obp-output/` at the repo root (gitignored — download them via the CLI or a browser). Conversion specs read them through `Access.from_html` and **skip** when the fixture is absent, so the offline suite never touches the network.
+
 ## Key dependencies
 
 - **sts** — NISO STS gem for generating STS objects from XML

data/README.adoc

@@ -54,6 +54,27 @@ obp = Obp::Access.fetch("iso:std:iso:5598:ed-3:v1:en", languages: :all)
 obp.to_xml(pretty: true) # => NISO STS XML with all available language langSets
 ----
 
+==== Already-downloaded HTML
+
+When the OBP page was fetched out of band (e.g. through a browser or
+waffle-punch), build an Access instance from the raw HTML fragment and
+convert it without any further network calls:
+
+[source,ruby]
+----
+obp = Obp::Access.from_html(
+  urn:     "iso:std:iso:80000-12:ed-2:v2:en",
+  html:    File.read("iso-80000-12-en.html"),
+  caption: "ISO 80000-12:2019",
+  titles:  { "en" => "Quantities and units — Part 12: Solid state physics" },
+)
+obp.to_xml(pretty: true) # => NISO STS XML
+obp.to_terms             # => YAML string of table-carried terms
+----
+
+`caption` is used to build the STS metadata (std-ref, copyright year); it
+should match the document reference. `titles` is optional.
+
 ==== CLI
 
 Fetch a single document (English only):

data/lib/obp/access/elements/base.rb

@@ -47,7 +47,7 @@ module Obp
           end
 
           def id
-            @id ||= node.attr("id").split("_").last
+            @id ||= node.attr("id")&.split("_")&.last
           end
 
           def to_xml

data/lib/obp/access/elements/footnotes.rb

@@ -25,18 +25,25 @@ module Obp
           end
 
           def render_fn(xml, note)
-            xml.fn(id: "fn_#{fn_id(note)}") do
-              xml.label fn_id(note)
+            id = fn_id(note)
+            xml.fn(id: "fn_#{id}") do
+              xml.label id
               xml.p fn_text(note)
             end
           end
 
           def fn_id(note)
-            note.attr("id").split("_").last
+            note.attr("id")&.split("_")&.last || note.object_id.to_s
           end
 
           def fn_text(note)
-            note.at_css("div").text.strip.sub(/\A\d+\s*/, "")
+            div = note.at_css("div")
+            if div
+              div.text.strip.sub(/\A\d+\s*/,
+                                 "")
+            else
+              note.text.strip.sub(/\A\d+\s*/, "")
+            end
           end
         end
       end

data/lib/obp/access/elements/root.rb

@@ -42,7 +42,7 @@ module Obp
           def std_meta_content(xml)
             render_titles(xml)
             xml.public_send(:"proj-id", ref_undated)
-            xml.public_send(:"release-version", doc_type)
+            xml.public_send(:"release-version", urn.doc_type)
             render_std_ident(xml)
             xml.public_send(:"content-language", metas["language"])
             xml.public_send(:"std-ref", ref_dated, type: "dated")
@@ -55,10 +55,10 @@ module Obp
           def render_std_ident(xml)
             xml.public_send(:"std-ident") do
               xml.originator holder
-              xml.public_send(:"doc-type", doc_type)
-              xml.public_send(:"doc-number", urn_parts[3])
-              xml.edition urn_parts[4].delete_prefix("ed-")
-              xml.version urn_parts[5].delete_prefix("v")
+              xml.public_send(:"doc-type", urn.doc_type)
+              xml.public_send(:"doc-number", urn.doc_number)
+              xml.edition urn.edition
+              xml.version urn.version
             end
           end
 
@@ -85,35 +85,27 @@ module Obp
           end
 
           def holder
-            metas["caption"].split.first
+            caption&.split(/[[:space:]]/)&.first || urn.originator.upcase
           end
 
           def ref
-            metas["caption"]
+            caption || urn.to_s
           end
 
           def ref_dated
-            metas["caption"].gsub(/\(.*?\)/, "")
+            caption&.gsub(/\([^()]*\)/, "") || ref
           end
 
           def ref_undated
-            @ref_undated ||= metas["caption"].split(":").first
+            @ref_undated ||= caption&.split(":")&.first || urn.doc_number
           end
 
           def copyright_year
-            metas["caption"][/:(\d{4})/, 1]
+            caption&.[](/:(\d{4})/, 1)
           end
 
-          def doc_type
-            case urn_parts[3]
-            when "ts" then "TS"
-            when "tr" then "TR"
-            else "IS"
-            end
-          end
-
-          def urn_parts
-            @urn_parts ||= urn.raw.split(":")
+          def caption
+            metas["caption"]
           end
         end
       end

data/lib/obp/access/elements/table_wrap.rb

@@ -36,7 +36,10 @@ module Obp
           # following rows when the builder re-parses the fragment. OBP's
           # "sts-unknown-element" placeholders are stripped first.
           def table_markup
-            table = node.at_css("table").dup
+            table = node.at_css("table")
+            return "" unless table
+
+            table = table.dup
             table.css(".sts-unknown-element").each(&:remove)
             table.children.map(&:to_xml).join
           end

data/lib/obp/access/fetcher.rb

@@ -48,7 +48,10 @@ module Obp
           "v-loc" => "#{API_URL}##{@urn}",
         )
 
-        Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
+        Net::HTTP.start(
+          uri.hostname, uri.port, use_ssl: true,
+                                  open_timeout: 30, read_timeout: 60
+        ) do |http|
           http.request(request)
         end
       end

data/lib/obp/access/parser.rb

@@ -55,7 +55,7 @@ module Obp
       end
 
       def tab_data
-        @tab_data ||= state.filter_map { |attr| attr["tabs"] }.first.last
+        @tab_data ||= state.filter_map { |attr| attr["tabs"] }.first&.last || {}
       end
 
       def titles

data/lib/obp/access/raw_html_parser.rb

@@ -0,0 +1,41 @@
+module Obp
+  class Access
+    # Parser that converts an already-downloaded OBP HTML fragment (e.g.
+    # captured via a browser / waffle-punch) without touching the network.
+    # Duck-types with Parser for the Access pipeline.
+    class RawHtmlParser < Parser
+      def initialize(urn:, directory:, html:, caption: nil, titles: nil)
+        super(urn:, directory:)
+        @raw_html = html
+        @caption = caption
+        @raw_titles = titles
+      end
+
+      def html
+        @raw_html
+      end
+
+      def available_languages
+        [urn.language]
+      end
+
+      private
+
+      def state
+        raise "RawHtmlParser does not fetch OBP state"
+      end
+
+      def tab_data
+        { "caption" => @caption, "description" => title }
+      end
+
+      def title
+        @caption
+      end
+
+      def titles
+        @raw_titles || { urn.language => title }
+      end
+    end
+  end
+end

data/lib/obp/access/urn.rb

@@ -1,6 +1,8 @@
 module Obp
   class Access
     class Urn
+      DOC_TYPE_SEGMENTS = %w[ts tr pas guide iwa].freeze
+
       attr_reader :raw, :language, :base
 
       def initialize(raw)
@@ -26,6 +28,59 @@ module Obp
       def hash
         raw.hash
       end
+
+      def parts
+        @parts ||= raw.split(":")
+      end
+
+      # Originator segment: "iso", "iec", "itu", ...
+      def originator
+        parts[2]
+      end
+
+      # Document type segment if present ("ts", "tr", ...), else nil
+      def doc_type_segment
+        segments = identifier_segments
+        segments.find { |s| DOC_TYPE_SEGMENTS.include?(s) }
+      end
+
+      # Document number, including part number when present.
+      # "iso:std:iso:80000:-12:ed-2:v1:en" → "80000-12"
+      def doc_number
+        identifier_segments
+          .reject { |s| DOC_TYPE_SEGMENTS.include?(s) }
+          .join
+      end
+
+      def edition
+        segment = parts.find { |p| p.start_with?("ed-") }
+        segment&.delete_prefix("ed-")
+      end
+
+      def version
+        segment = parts.find { |p| p.match?(/\Av\d+\z/) }
+        segment&.delete_prefix("v")
+      end
+
+      def doc_type
+        case doc_type_segment
+        when "ts" then "TS"
+        when "tr" then "TR"
+        when "pas" then "PAS"
+        when "guide" then "Guide"
+        when "iwa" then "IWA"
+        else "IS"
+        end
+      end
+
+      private
+
+      # Segments between the originator and the edition segment.
+      def identifier_segments
+        start_index = 3
+        end_index = parts.index { |p| p.start_with?("ed-") } || parts.size
+        parts[start_index...end_index]
+      end
     end
   end
 end

data/lib/obp/access/version.rb

@@ -1,5 +1,5 @@
 module Obp
   class Access
-    VERSION = "0.1.5".freeze
+    VERSION = "0.1.6".freeze
   end
 end

data/lib/obp/access.rb

@@ -16,6 +16,7 @@ require_relative "access/retriever"
 require_relative "access/fetcher"
 require_relative "access/element_registry"
 require_relative "access/parser"
+require_relative "access/raw_html_parser"
 require_relative "access/converter"
 require_relative "access/imager"
 require_relative "access/renderer"
@@ -76,6 +77,15 @@ module Obp
       resolved.map { |lang| new(Urn.new("#{urn.base}:#{lang}")) }
     end
 
+    # Build an Access from an already-downloaded OBP HTML fragment (e.g.
+    # captured via a browser or waffle-punch). No network requests are made.
+    def self.from_html(urn:, html:, caption: nil, titles: nil)
+      raise ArgumentError, "URN is required" unless urn
+      raise ArgumentError, "HTML is required" unless html
+
+      new(Urn.new(urn), html: html, caption: caption, titles: titles)
+    end
+
     def self.resolve_languages(primary, requested, available)
       case requested
       when :all then [primary] | available
@@ -84,8 +94,11 @@ module Obp
       end
     end
 
-    def initialize(urn)
+    def initialize(urn, html: nil, caption: nil, titles: nil)
       @urn = urn
+      @raw_html = html
+      @caption = caption
+      @raw_titles = titles
     end
 
     def to_xml(pretty: false)
@@ -119,10 +132,20 @@ module Obp
       parser.available_languages
     end
 
+    # Raw sts-standard HTML fragment as served by the OBP.
+    def html
+      parser.html
+    end
+
     private
 
     def parser
-      @parser ||= Parser.new(urn:, directory: tmpdir)
+      @parser ||= if @raw_html
+                    RawHtmlParser.new(urn:, directory: tmpdir, html: @raw_html,
+                                      caption: @caption, titles: @raw_titles)
+                  else
+                    Parser.new(urn:, directory: tmpdir)
+                  end
     end
 
     # Shares Parser#html (memoized) with the STS conversion path, so

data/obp-access.gemspec

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require_relative "lib/obp/access/version"
+
+Gem::Specification.new do |spec|
+  spec.name = "obp-access"
+  spec.version = Obp::Access::VERSION
+  spec.authors = ["Ribose Inc."]
+  spec.email = ["open.source@ribose.com"]
+
+  spec.summary = ""
+  spec.description = ""
+  spec.homepage = ""
+  spec.required_ruby_version = ">= 3.1"
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files = Dir.chdir(__dir__) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      (File.expand_path(f) == __FILE__) ||
+        f.start_with?(*%w[bin/ test/ spec/ features/ .git .github appveyor
+                          Gemfile])
+    end
+  end
+  spec.bindir = "exe"
+  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+  spec.add_dependency "nokogiri"
+  spec.add_dependency "parallel"
+  spec.add_dependency "sts"
+  spec.add_dependency "thor"
+  spec.metadata["rubygems_mfa_required"] = "true"
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: obp-access
 version: !ruby/object:Gem::Version
-  version: 0.1.5
+  version: 0.1.6
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2026-07-18 00:00:00.000000000 Z
+date: 2026-09-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -125,12 +125,14 @@ files:
 - lib/obp/access/imager.rb
 - lib/obp/access/inline_renderer.rb
 - lib/obp/access/parser.rb
+- lib/obp/access/raw_html_parser.rb
 - lib/obp/access/renderer.rb
 - lib/obp/access/retriever.rb
 - lib/obp/access/table_mapper.rb
 - lib/obp/access/table_term_extractor.rb
 - lib/obp/access/urn.rb
 - lib/obp/access/version.rb
+- obp-access.gemspec
 homepage: ''
 licenses: []
 metadata: