suma 0.2.5 → 0.3.0

data/README.adoc

@@ -39,7 +39,8 @@ $ suma
  Commands:
   suma build METANORMA_SITE_MANIFEST                                  # Build collection specified in site manifest (`metanorma*.yml`)
   suma convert-jsdai XML_FILE IMAGE_FILE OUTPUT_DIR                   # Convert JSDAI XML and image files to SVG and EXP files
-  suma extract-terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH      # Extract terms from SCHEMA_MANIFEST_FILE into Glossarist v2 format
+  suma extract-terms SCHEMA_MANIFEST_FILE GLOSSARIST_OUTPUT_PATH      # Extract EXPRESS entity concepts into Glossarist v3 format
+  suma generate-register SCHEMA_MANIFEST_FILE OUTPUT_PATH            # Generate hierarchical register.yaml from schema manifest
   suma generate-schemas METANORMA_MANIFEST_FILE SCHEMA_MANIFEST_FILE  # Generate EXPRESS schema manifest file from Metanorma site manifest
   suma help [COMMAND]                                                 # Describe available commands or one specific command
   suma reformat EXPRESS_FILE_PATH                                     # Reformat EXPRESS files
@@ -301,6 +302,61 @@ $ bundle exec suma extract-terms schemas-activity-modules.yml terms_output
 ====
 
 
+=== Generate register command
+
+The `suma generate-register` command reads an EXPRESS schema manifest and
+emits a Glossarist v3 `register.yaml` with hierarchical sections. The
+output is used by the concept browser to render a navigable sidebar of
+sections and to resolve concept cross-references.
+
+[source,sh]
+----
+$ suma generate-register SCHEMA_MANIFEST_FILE OUTPUT_PATH -u URN --id ID --ref REF
+----
+
+Parameters:
+
+`SCHEMA_MANIFEST_FILE`:: Path to the schema manifest (e.g., "schemas-smrl-part-2.yml")
+
+`OUTPUT_PATH`:: Directory where `register.yaml` will be written
+
+Options:
+
+`--urn`, `-u`:: Base URN prefix for the dataset (e.g. `urn:iso:std:iso:10303:-2:ed-2:en:tech:*`).
+The wildcard is stripped for the base URN and kept in `urnAliases`.
+*Required*.
+
+The URN is fully configurable — every component can be changed to match the
+dataset being exported. The general structure is:
+
+----
+urn:iso:std:iso:<part>:ed-<edition>:<lang>:tech:*
+----
+
+For example, to export edition 3 in French:
+
+----
+--urn urn:iso:std:iso:10303:-2:ed-3:fr:tech:*
+----
+
+The current ISO 10303-2 SMRL data is edition 2; use `ed-2` unless you are
+re-publishing an earlier edition.
+
+`--id`:: Dataset identifier (e.g. `iso10303-2-express`). *Required*.
+
+`--ref`:: Human-readable dataset reference label (e.g. `ISO 10303-2 EXPRESS Concepts`). *Required*.
+
+`--language_code`, `-l`:: Language code for section names (default: "eng")
+
+The generated register contains two top-level groups:
+
+* **Resources** — integrated resource schemas (path contains `schemas/resources/`)
+* **Application Modules** — ARM and MIM schemas (path contains `schemas/modules/`)
+
+Each schema gets a human-readable name via the `SchemaNaming` module
+(e.g. `topology_schema` → "Resource: Topology", `Activity_method_assignment_mim` → "Module: Activity Method Assignment (MIM)"). Acronyms (AIC, AEC, BREP, 2D, 3D) are preserved; function words (as, of, the) are lowercased per ISO title-case convention.
+
+
 === Convert JSDAI image outputs to SVG and Annotated EXPRESS
 
 The `suma convert-jsdai` command converts JSDAI EXPRESS-G diagram outputs
@@ -864,6 +920,137 @@ For complete documentation of all Expressir commands and options, see the
 https://github.com/lutaml/expressir[Expressir documentation].
 
 
+=== Check SVG quality command
+
+==== General
+
+The `check_svg_quality` command validates SVG files for conformance and quality
+using the https://github.com/claricle/svg_conform[svg_conform] library. It supports
+both batch directory scanning and single file analysis.
+
+By default, it uses the `metanorma` profile which allows raster images
+(PNG, JPEG) and other features specific to Metanorma documentation.
+
+[source,sh]
+----
+$ suma check_svg_quality [PATH] [options]
+----
+
+Parameters:
+
+`PATH`:: Path to directory containing SVG files or a single SVG file.
+Defaults to `schemas` directory.
+
+==== Batch directory scanning
+
+When given a directory, it scans all SVG files and produces a quality report
+sorted by error count (most problematic files first).
+
+[example]
+.Check all SVG files in schemas directory
+[source,sh]
+----
+$ suma check_svg_quality ../iso-10303/schemas
+----
+
+.Output
+----
+🔍 Scanning 3878 SVG files...
+
+╔══════════════════════════════════════════════════════════════════════════════╗
+║  🔍 SVG Quality Report    Sorted by error count (most first)             ║
+╚══════════════════════════════════════════════════════════════════════════════╝
+
+  📊 OVERVIEW
+
+    ● Total Files   : 3878
+    ● Valid         : 3845 ✅
+    ● Invalid      : 33 ❌
+    ...
+
+  💥 CRITICAL QUALITY (2 files)
+
+    ✗   0/100    813 errors  schemas/resources/action_schema/action_schemaexpg2.svg
+    ✗  25/100    156 errors  schemas/modules/activity/mimexpg1.svg
+----
+
+==== Single file analysis
+
+When given a path to a single SVG file, it shows detailed error breakdown
+grouped by requirement type.
+
+[example]
+.Analyze a single SVG file
+[source,sh]
+----
+$ suma check_svg_quality schemas/resources/action_schema/action_schemaexpg2.svg
+----
+
+.Output
+----
+📄 SVG Quality Report: schemas/resources/action_schema/action_schemaexpg2.svg
+
+  Valid: NO ❌
+  Errors: 813
+
+  📋 Error Details
+
+  style (56 occurrences)
+    - Style property 'enable-background' removed
+    - Style property 'overflow' removed
+    ...
+
+  allowed_elements (109 occurrences)
+    - The element 'image' is not allowed as a child of 'svg'
+    ...
+
+  color_restrictions (108 occurrences)
+    - Color 'rgb(33, 128, 255)' in style property 'fill' is not allowed
+    ...
+----
+
+Options:
+
+`--profile=PROFILE`:: Validation profile to use (default: `metanorma`).
+Other options: `svg_1_2_rfc`, `svg_1_2_rfc_with_rdf`, `base`, `lucid_fix`
+
+`--sort=MODE`:: Sort order: `errors` (most errors first, default) or
+`quality` (lowest quality scores first)
+
+`--format=FORMAT`:: Output format: `terminal` (default), `json`, or `yaml`
+
+`--output=PATH`, `-o PATH`:: Write output to file instead of stdout
+
+`--min_errors=N`:: Filter to show only files with at least N errors
+
+`--limit=N`:: Limit output to top N files
+
+`--progress`:: Show progress during batch scanning
+
+`--summary-only`:: Show only summary, not individual file details
+
+[example]
+.Check with custom profile
+[source,sh]
+----
+$ suma check_svg_quality schemas/ --profile svg_1_2_rfc
+----
+
+[example]
+.Output as JSON
+[source,sh]
+----
+$ suma check_svg_quality schemas/ --format json --output quality_report.json
+----
+
+[example]
+.Show only files with 100+ errors
+[source,sh]
+----
+$ suma check_svg_quality schemas/ --min_errors 100
+----
+
+
 == Usage: Ruby
 
 === General

data/exe/suma

@@ -1,6 +1,6 @@
 #!/usr/bin/env ruby
 # frozen_string_literal: true
 
-require "suma/cli"
+require "suma"
 
 Suma::Cli::Core.start(ARGV)

data/lib/suma/cli/build.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require "thor"
-require_relative "../thor_ext"
 
 module Suma
   module Cli
@@ -15,10 +14,6 @@ module Suma
                                 desc: "Generate file that contains all schemas in the collection."
 
       def build(metanorma_site_manifest)
-        # Lazy-load dependencies only when this command is actually used
-        require_relative "../processor"
-        require_relative "../utils"
-
         unless File.exist?(metanorma_site_manifest)
           raise Errno::ENOENT, "Specified Metanorma site manifest file " \
                                "`#{metanorma_site_manifest}` not found."
@@ -31,16 +26,15 @@ module Suma
       private
 
       def run(manifest, options)
-        # Set schemas_all_path to match metanorma_yaml_path
         schemas_all_path = options[:schemas_all_path] ||
           manifest.gsub("metanorma", "schemas")
 
-        Processor.run(
+        Processor.new(
           metanorma_yaml_path: manifest,
           schemas_all_path: schemas_all_path,
           compile: options[:compile],
           output_directory: "_site",
-        )
+        ).run
       end
 
       def log_error(error)

data/lib/suma/cli/check_svg_quality.rb

@@ -0,0 +1,172 @@
+# frozen_string_literal: true
+
+require "pathname"
+
+module Suma
+  module Cli
+    # Check SVG quality using svg_conform Validator API - thin CLI wrapper
+    class CheckSvgQuality
+      DATA_PATH = "schemas"
+      DEFAULT_PATTERN = "**/*.svg"
+      DEFAULT_PROFILE = :metanorma
+
+      def initialize(pattern: DEFAULT_PATTERN, profile: DEFAULT_PROFILE,
+                    format: "terminal", output: nil, min_errors: nil,
+                    summary_only: false, progress: false, limit: nil,
+                    sort: "errors")
+        @options = {
+          pattern: pattern,
+          profile: profile,
+          format: format,
+          output: output,
+          min_errors: min_errors,
+          summary_only: summary_only,
+          progress: progress,
+          limit: limit,
+          sort: sort.to_sym,
+        }
+      end
+
+      def run(path = DATA_PATH)
+        require "svg_conform"
+
+        path_obj = Pathname.new(path).expand_path
+
+        # Enable progress by default when outputting to terminal
+        show_progress = options[:progress] || ($stdout.tty? && !options[:output])
+        if show_progress
+          $stdout.sync = true
+          $stderr.sync = true
+        end
+
+        if path_obj.file?
+          # Single file mode - show detailed errors
+          analyze_single_file(path_obj)
+        else
+          # Directory mode - show batch report
+          svg_files = find_svg_files(path_obj)
+
+          if svg_files.empty?
+            puts "No SVG files found in #{path}"
+            return
+          end
+
+          puts "🔍 Scanning #{svg_files.size} SVG files..."
+          puts
+
+          reports = analyze_files_one_by_one(svg_files, show_progress)
+          batch_report = SvgQuality::BatchReport.new(reports)
+          sorted_report = sort_report(batch_report)
+          output_report(sorted_report)
+        end
+      end
+
+      def analyze_single_file(path)
+        validator = SvgConform::Validator.new
+        result = validator.validate_file(path.to_s, profile: options[:profile])
+
+        puts "📄 SVG Quality Report: #{path}"
+        puts ""
+        puts "  Valid: #{result.valid? ? 'YES ✅' : 'NO ❌'}"
+        puts "  Errors: #{result.error_count}"
+        puts ""
+
+        if result.errors.any?
+          puts "  📋 Error Details"
+          puts ""
+
+          # Group errors by requirement_id
+          by_req = result.errors.group_by(&:requirement_id)
+
+          by_req.each do |req_id, errors|
+            puts "  #{req_id} (#{errors.size} occurrences)"
+            errors.first(5).each do |e|
+              puts "    - #{e.message}"
+            end
+            if errors.size > 5
+              puts "    ... and #{errors.size - 5} more"
+            end
+            puts ""
+          end
+        end
+      end
+
+      private
+
+      attr_reader :options
+
+      def find_svg_files(path)
+        if path.directory?
+          Pathname.glob(path.join(options[:pattern])).select(&:file?)
+        elsif path.file? && path.extname == ".svg"
+          [path]
+        else
+          []
+        end
+      end
+
+      def analyze_files_one_by_one(files, show_progress = false)
+        validator = SvgConform::Validator.new
+        reports = []
+
+        files.each_with_index do |file, index|
+          result = validator.validate_file(file.to_s,
+                                           profile: options[:profile])
+          report = SvgQuality::Report.new(file.to_s, result)
+          reports << report
+
+          if show_progress
+            tier = report.quality_tier
+            status = report.valid? ? "✅" : "❌"
+            msg = "  [#{index + 1}/#{files.size}] #{tier[:emoji]} #{report.error_count} errors #{status} #{shorten_path(file)}\n"
+            $stderr.print msg
+            $stderr.flush
+          end
+        end
+
+        reports
+      end
+
+      def sort_report(batch_report)
+        case options[:sort]
+        when :quality
+          batch_report.sort_by_quality
+        else
+          batch_report.sort_by_errors
+        end
+      end
+
+      def output_report(batch_report)
+        filtered = batch_report.filter_by_min_errors(options[:min_errors])
+        limited = filtered.limit(options[:limit])
+
+        formatter = case options[:format].to_sym
+                    when :json
+                      SvgQuality::Formatters::JsonFormatter.new(limited,
+                                                                output: options[:output])
+                    when :yaml
+                      SvgQuality::Formatters::YamlFormatter.new(limited,
+                                                                output: options[:output])
+                    else
+                      SvgQuality::Formatters::TerminalFormatter.new(limited,
+                                                                    output: options[:output], sort: options[:sort])
+                    end
+
+        puts formatter.format
+      end
+
+      def shorten_path(path)
+        p = Pathname.new(path)
+        if p.absolute?
+          begin
+            p.relative_path_from(Pathname.pwd)
+          rescue StandardError
+            p
+          end
+        else
+          p
+        end.to_s
+      end
+    end
+  end
+end

data/lib/suma/cli/compare.rb

@@ -1,12 +1,9 @@
 # frozen_string_literal: true
 
 require "thor"
-require_relative "../eengine/wrapper"
-require_relative "../eengine_converter"
 
 module Suma
   module Cli
-    # Command to compare EXPRESS schemas using eengine
     class Compare < Thor
       desc "compare TRIAL_SCHEMA REFERENCE_SCHEMA",
            "Compare EXPRESS schemas using eengine and generate Change YAML"
@@ -44,166 +41,17 @@ module Suma
                        desc: "Enable verbose output"
 
       def compare(trial_schema, reference_schema)
-        # Validate schema files exist
-        unless File.exist?(trial_schema)
-          say "Error: Trial schema not found: #{trial_schema}", :red
-          exit 1
-        end
-
-        unless File.exist?(reference_schema)
-          say "Error: Reference schema not found: #{reference_schema}", :red
-          exit 1
-        end
-
-        # Check eengine availability
-        unless Eengine::Wrapper.available?
-          say "Error: eengine not found in PATH", :red
-          say "Install eengine following instructions at:"
-          say "  macOS: https://github.com/expresslang/homebrew-eengine"
-          say "  Linux: https://github.com/expresslang/eengine-releases"
-          exit 1
-        end
+        comparer = SchemaComparer.new(trial_schema, reference_schema, options)
 
-        # Auto-detect repo roots
-        trial_stepmod = options[:trial_stepmod] ||
-          detect_repo_root(trial_schema)
-        reference_stepmod = options[:reference_stepmod] ||
-          detect_repo_root(reference_schema)
+        result = comparer.compare
 
-        if options[:verbose]
-          say "Using eengine version: #{Eengine::Wrapper.version}", :green
-          say "Trial repo root: #{trial_stepmod}", :cyan
-          say "Reference repo root: #{reference_stepmod}", :cyan
-        end
-
-        # Create a temporary directory for eengine output
-        require "tmpdir"
-        out_dir = nil
-        out_dir = Dir.mktmpdir("eengine-compare-")
-
-        # Run comparison
-        result = Eengine::Wrapper.compare(
-          trial_schema,
-          reference_schema,
-          mode: options[:mode],
-          trial_stepmod: trial_stepmod,
-          reference_stepmod: reference_stepmod,
-          out_dir: out_dir,
-        )
-
-        unless result[:has_changes]
+        if result.nil?
           say "No changes detected between schemas", :yellow
-          # Clean up temp directory
-          FileUtils.rm_rf(out_dir) if out_dir && File.directory?(out_dir)
-          return
-        end
-
-        unless result[:xml_path]
-          say "Error: XML output not found", :red
-          exit 1
-        end
-
-        if options[:verbose]
-          say "Comparison XML generated: #{result[:xml_path]}", :green
-        end
-
-        # Convert to Change YAML
-        convert_to_change_yaml(result[:xml_path], trial_schema, out_dir)
-      rescue Eengine::EengineError => e
-        # Clean up temp directory
-        FileUtils.rm_rf(out_dir) if out_dir && File.directory?(out_dir)
-        say "Error: #{e.message}", :red
-        say e.stderr if e.respond_to?(:stderr) && options[:verbose]
-        exit 1
-      end
-
-      private
-
-      def detect_repo_root(schema_path)
-        # Walk up from schema path to find .git directory
-        current = File.expand_path(File.dirname(schema_path))
-
-        loop do
-          if File.directory?(File.join(current, ".git"))
-            return current
-          end
-
-          parent = File.dirname(current)
-          break if parent == current # reached root
-
-          current = parent
-        end
-
-        # If no .git found, use the directory containing the schema
-        # (for non-git workflows)
-        File.dirname(schema_path)
-      end
-
-      def convert_to_change_yaml(xml_path, trial_schema, out_dir)
-        schema_name = extract_schema_name(trial_schema)
-        output_path = determine_output_path(trial_schema)
-
-        # Load existing ChangeSchema if it exists
-        existing_schema = if File.exist?(output_path)
-                            if options[:verbose]
-                              say "Loading existing change schema: " \
-                                  "#{output_path}", :cyan
-                            end
-                            require "expressir/changes"
-                            Expressir::Changes::SchemaChange.from_file(output_path)
-                          end
-
-        # Convert using Suma's converter
-        converter = EengineConverter.new(xml_path, schema_name)
-        change_schema = converter.convert(
-          version: options[:version],
-          existing_change_schema: existing_schema,
-        )
-
-        # Save using Expressir model
-        change_schema.to_file(output_path)
-
-        # Determine what action was taken
-        if existing_schema
-          existing_version = existing_schema.versions.find do |ed|
-            ed.version == options[:version]
-          end
-
-          say "Change YAML file updated: #{output_path}", :green
-          if existing_version
-            say "  Replaced existing version #{options[:version]}", :green
-          else
-            say "  Added version #{options[:version]} to change versions",
-                :green
-          end
-        else
-          say "Change YAML file created: #{output_path}", :green
-        end
-
-        if options[:verbose]
-          say "\nGenerated change schema content:", :cyan
-          say File.read(output_path)
-        end
-
-        # Clean up temp directory and XML file
-        FileUtils.rm_rf(out_dir) if out_dir && File.directory?(out_dir)
-      end
-
-      def extract_schema_name(path)
-        # Remove version suffix if present (e.g., schema_1.exp -> schema)
-        basename = File.basename(path, ".exp")
-        basename.sub(/_\d+$/, "")
-      end
-
-      def determine_output_path(trial_schema)
-        if options[:output]
-          options[:output]
         else
-          # Place .changes.yaml next to the trial schema in the NEW repo
-          base = extract_schema_name(trial_schema)
-          dir = File.dirname(trial_schema)
-          File.join(dir, "#{base}.changes.yaml")
+          say "Change YAML file: #{result}", :green
         end
+      rescue Suma::Error => e
+        raise Thor::Error, e.message
       end
     end
   end

data/lib/suma/cli/convert_jsdai.rb

@@ -1,8 +1,6 @@
 # frozen_string_literal: true
 
 require "thor"
-require_relative "../thor_ext"
-require_relative "../jsdai"
 require "fileutils"
 
 module Suma