expressir 2.1.31 → 2.2.1

data/docs/_guides/formatter/formatter-architecture.adoc

@@ -0,0 +1,401 @@
+== Formatter architecture
+
+=== General
+
+The Expressir formatter uses a modular architecture that separates formatting
+concerns into focused, composable modules. This design improves maintainability,
+testability, and extensibility while preserving full backward compatibility with
+existing code.
+
+The formatter architecture follows object-oriented design principles with clear
+separation of concerns. Each module handles a specific aspect of EXPRESS
+formatting, and they work together through Ruby's module inclusion mechanism.
+
+The concept of Profiles is used to define different formatting styles or
+conventions. The base `Formatter` class provides standard formatting, while
+specialized formatters like `PrettyFormatter` extend it with additional features
+and ELF compliance.
+
+=== Feature
+
+==== Remark preservation
+
+Expressir fully preserves EXPRESS remarks (comments) during parsing and formatting, maintaining them in their original positions:
+
+==== Preamble remarks
+
+Remarks between a scope declaration and its first child are preserved as preamble remarks:
+
+[source,express]
+----
+SCHEMA example;
+  -- This is a preamble remark
+  -- It appears after SCHEMA but before declarations
+
+  ENTITY person;
+    -- Entity preamble remark
+    name : STRING;
+  END_ENTITY;
+
+END_SCHEMA;
+----
+
+==== Inline tail remarks
+
+Remarks on the same line as attribute or enumeration item declarations:
+
+[source,express]
+----
+ENTITY person;
+  name : STRING; -- Inline remark for name attribute
+  age : INTEGER; -- Inline remark for age attribute
+END_ENTITY;
+
+TYPE status = ENUMERATION OF
+  (active,   -- Active status
+   inactive, -- Inactive status
+   pending); -- Pending status
+END_TYPE;
+----
+
+==== END_* scope remarks
+
+Remarks on END_TYPE, END_ENTITY, END_SCHEMA, etc. lines:
+
+[source,express]
+----
+TYPE status = ENUMERATION OF
+  (active,
+   inactive);
+END_TYPE; -- Status enumeration type
+
+ENTITY person;
+  name : STRING;
+END_ENTITY; -- Person entity
+
+END_SCHEMA; -- schema_name
+----
+
+==== Unicode support
+
+All remark types support full Unicode content:
+
+[source,express]
+----
+SCHEMA test;
+  -- 日本語、中文、한글 in remarks
+
+  ENTITY person;
+    name : STRING; -- Name in Japanese: 名前
+  END_ENTITY;
+
+END_SCHEMA; -- test
+----
+
+For implementation details, see link:docs/ARCHITECTURE.md#remark-attachment-system[Remark Attachment System].
+
+
+=== Module organization
+
+The formatter consists of a main `Formatter` class that includes specialized
+formatting modules, each responsible for a distinct category of EXPRESS language
+constructs.
+
+==== Formatter modules
+
+RemarkFormatter:: Handles formatting of remarks (comments) in all forms
++
+[example]
+====
+* Embedded remarks: `(* comment *)`
+* Tail remarks: `-- comment`
+* Tagged remarks with identifiers
+* Preamble remarks before first declarations
+* END_* scope remarks on closing statements
+====
+
+RemarkItemFormatter:: Formats individual remark items and remark metadata
++
+[example]
+====
+Handles the internal structure of remark items, including tags, format specification, and text content.
+====
+
+LiteralsFormatter:: Formats literal values (strings, numbers, booleans, binary)
++
+[example]
+====
+[source,express]
+----
+'string literal'
+123
+3.14
+TRUE
+%10101011
+----
+====
+
+ReferencesFormatter:: Formats references to entities, attributes, and other elements
++
+[example]
+====
+[source,express]
+----
+entity_ref
+entity_ref.attribute_ref
+entity_ref[index]
+----
+====
+
+SupertypeExpressionsFormatter:: Formats supertype constraint expressions
++
+[example]
+====
+[source,express]
+----
+SUPERTYPE OF (ONEOF(subtype1, subtype2))
+ABSTRACT SUPERTYPE OF (subtype1 AND subtype2)
+----
+====
+
+StatementsFormatter:: Formats procedural statements (assignment, if, case, repeat, etc.)
++
+[example]
+====
+[source,express]
+----
+IF condition THEN
+  statement;
+END_IF;
+----
+====
+
+ExpressionsFormatter:: Formats expressions (binary, unary, function calls, queries)
++
+[example]
+====
+[source,express]
+----
+a + b * c
+QUERY(x <* entity | condition)
+entity_constructor(arg1, arg2)
+----
+====
+
+DataTypesFormatter:: Formats data type declarations (INTEGER, STRING, ENUMERATION, SELECT, etc.)
++
+[example]
+====
+[source,express]
+----
+STRING(255)
+ENUMERATION OF (red, green, blue)
+SELECT (type1, type2, type3)
+----
+====
+
+DeclarationsFormatter:: Formats declarations (ENTITY, TYPE, FUNCTION, SCHEMA, etc.)
++
+[example]
+====
+[source,express]
+----
+ENTITY person;
+  name : STRING;
+END_ENTITY;
+----
+====
+
+=== RemarkInfo model
+
+==== General
+
+Remarks were previously represented as plain strings, which lost important formatting information. The [`RemarkInfo`](lib/expressir/model/remark_info.rb:6) class properly models remarks with their complete metadata.
+
+==== Attributes
+
+The `RemarkInfo` class has three attributes:
+
+`text`:: The remark content (String)
+
+`format`:: The remark format: 'tail' or 'embedded' (String)
+
+`tag`:: Optional tag for associating the remark with specific items (String or nil)
+
+==== Methods
+
+`tail?`:: Returns true if the remark uses tail format (`-- comment`)
+
+`embedded?`:: Returns true if the remark uses embedded format (`(* comment *)`)
+
+`tagged?`:: Returns true if the remark has an associated tag
+
+`to_s`:: Returns the remark text for backward compatibility
+
+==== Benefits over plain strings
+
+Type safety:: Explicit format information prevents format confusion
+
+Preservation:: Original format is maintained through parse/format cycles
+
+Extensibility:: Easy to add metadata (tags, positions, etc.) without breaking existing code
+
+Clarity:: Code explicitly shows whether a remark is tail or embedded
+
+
+=== Using the base formatter
+
+The base [`Formatter`](lib/expressir/express/formatter.rb:13) class provides
+standard EXPRESS formatting with fixed 2-space indentation.
+
+.Format a repository
+[example]
+====
+[source,ruby]
+----
+# Parse an EXPRESS schema
+repository = Expressir::Express::Parser.from_file("schema.exp")
+
+# Format to string
+formatted = Expressir::Express::Formatter.format(repository)
+puts formatted
+
+# Or create instance for custom options
+formatter = Expressir::Express::Formatter.new(no_remarks: true)
+formatted = formatter.format(repository)
+----
+====
+
+.Format without remarks
+[example]
+====
+[source,ruby]
+----
+# Useful for generating clean schemas without documentation
+formatter = Expressir::Express::Formatter.new(no_remarks: true)
+clean_schema = formatter.format(repository)
+----
+====
+
+=== Using ELF PrettyFormatter
+
+The [`PrettyFormatter`](lib/expressir/express/pretty_formatter.rb:7) extends the
+base `Formatter` with ELF (EXPRESS Language Foundation) compliance and
+additional features.
+
+See the <<_pretty_print_with_elf_compliance,Pretty print with ELF compliance>>
+section for detailed usage examples and configuration options.
+
+
+=== Extending the formatter
+
+==== Creating a custom formatter
+
+You can create custom formatters by extending `Formatter` or any class that
+inherits from it.
+
+.Custom formatter with specific behavior
+[example]
+====
+[source,ruby]
+----
+class MyCustomFormatter < Expressir::Express::Formatter
+  # Override specific formatting methods
+  def format_declarations_entity(node)
+    # Custom entity formatting logic
+    super(node)  # Or completely custom implementation
+  end
+
+  # Override indentation
+  def indent(str)
+    return if str.nil?
+
+    # Use 3 spaces instead of 2
+    indent_str = "   "
+    str.split("\n").map { |x| "#{indent_str}#{x}" }.join("\n")
+  end
+end
+
+# Use the custom formatter
+formatter = MyCustomFormatter.new
+formatted = formatter.format(repository)
+----
+====
+
+==== Adding new formatter modules
+
+To add a new formatting module:
+
+. Create module in `lib/expressir/express/formatters/`
+. Define private formatting methods
+. Include module in `Formatter` class
+. Add tests in `spec/expressir/express/formatters/`
+
+.Example: Creating a new formatter module
+[example]
+====
+[source,ruby]
+----
+# lib/expressir/express/formatters/my_formatter.rb
+module Expressir
+  module Express
+    module MyFormatter
+      private
+
+      def format_my_construct(node)
+        # Formatting logic here
+      end
+    end
+  end
+end
+
+# In lib/expressir/express/formatter.rb
+require_relative "formatters/my_formatter"
+
+class Formatter
+  include MyFormatter
+  # ... other includes ...
+end
+----
+====
+
+=== Design decisions
+
+==== Why modules instead of inheritance
+
+The formatter uses module composition instead of class inheritance because:
+
+Separation of concerns:: Each module handles one category of formatting
++
+Each formatter module is focused on a single responsibility (remarks, literals,
+expressions, etc.), making the code easier to understand and maintain.
+
+Composability:: Modules can be mixed and matched as needed
++
+Different formatters can include only the modules they need, or override
+specific modules without affecting others.
+
+Testability:: Each module can be tested independently
++
+Unit tests can focus on individual modules without needing to set up the entire
+formatter.
+
+Maintainability:: Changes to one area don't affect others
++
+Bug fixes or enhancements to one formatter module don't risk breaking other
+formatting logic.
+
+==== Why RemarkInfo instead of strings
+
+The `RemarkInfo` model was introduced to:
+
+Preserve format information:: Tail vs embedded format is crucial for round-trip
+formatting
+
+Support tags:: Tags associate remarks with specific schema elements
+
+Enable future extensions:: Easy to add line numbers, positions, or other
+metadata
+
+Improve type safety:: Explicit object type prevents formatting errors
+

data/docs/_guides/ruby-api/parsing-files.adoc

@@ -250,7 +250,7 @@ end
 The exception provides:
 
 `filename`:: The file that failed to parse
-`parse_failure_cause`:: The underlying Parslet parse error
+`parse_failure_cause`:: The underlying Parsanol parse error
 `message`:: Human-readable error description
 
 === Other common errors

data/docs/_pages/parsers.adoc

@@ -38,8 +38,8 @@ Expressir uses a multi-stage parsing pipeline:
          │
          ▼
    ┌──────────┐
-   │  Parslet │ (PEG parser)
-   │  Grammar │
+   │  Parsanol │ (PEG parser)
+   │  Grammar  │
    └────┬─────┘
         │
         ▼
@@ -71,9 +71,35 @@ Expressir uses a multi-stage parsing pipeline:
 └────────────────────┘
 ----
 
+==== Backend: Parsanol
+
+Expressir uses the **Parsanol** parser backend:
+
+**Parsanol (High-Performance Rust)**::
+* High-performance Rust backend
+* 18-44x faster parsing
+* 99.5% fewer allocations
+* Supports source position tracking (Slice)
+* Available automatically when Parsanol gem is installed
+
+Expressir automatically uses Parsanol when available:
+
+[source,ruby]
+----
+# Automatically uses native Rust parser (Parsanol)
+repo = Expressir::Express::Parser.from_file("geometry.exp")
+
+# Check if native parser is available
+if defined?(Parsanol::Native) && Parsanol::Native.available?
+  # Using Rust parser - 20-30x faster
+end
+----
+
+See the link:https://github.com/parsanol/parsanol-ruby[Parsanol documentation] for performance details.
+
 ==== Stage 1: Lexical Analysis and Parsing
 
-Expressir uses **Parslet**, a Parsing Expression Grammar (PEG) parser for Ruby:
+Expressir uses **Parsanol**, a high-performance Parsing Expression Grammar (PEG) parser:
 
 [source,ruby]
 ----
@@ -87,7 +113,7 @@ rule(:entityHead) do
 end
 ----
 
-**Parslet advantages**:
+**Parsanol advantages**:
 
 * **Pure Ruby**: No external dependencies
 * **Composable rules**: Complex grammars from simple parts
@@ -677,7 +703,7 @@ link:../guides/cli/benchmark-performance.html[Benchmark and optimize]
 
 === Bibliography
 
-* https://github.com/kschiess/parslet[Parslet] - PEG parser for Ruby
+* https://github.com/parsanon/parsanon-ruby[Parsanol] - High-performance PEG parser for Ruby
 * https://en.wikipedia.org/wiki/Parsing_expression_grammar[PEG on Wikipedia] - Understanding PEG parsers
 * link:express-language.html[EXPRESS Language] - Understanding what is being parsed
 * link:data-model.html[Data Model] - Understanding the parsing result

data/docs/lychee.toml

@@ -52,6 +52,9 @@ exclude = [
   "https://twitter\\.com/.*",
   "https://x\\.com/.*",
 
+  # Self-referential documentation links (site under development)
+  "https://www\\.lutaml\\.org/expressir/.*",
+
   # External sites with broken or moved URLs
   "https://www\\.nist\\.gov/.*",
   "https://www\\.steptools\\.com/.*",

data/expressir.gemspec

@@ -23,7 +23,7 @@ Gem::Specification.new do |spec|
   spec.metadata["changelog_uri"] = "https://github.com/lutaml/expressir/releases"
   spec.metadata["rubygems_mfa_required"] = "true"
 
-  spec.required_ruby_version = Gem::Requirement.new(">= 3.0.0")
+  spec.required_ruby_version = Gem::Requirement.new(">= 3.2.0")
 
   spec.files = Dir.chdir(File.expand_path(__dir__)) do
     `git ls-files -z`.split("\x0").reject do |f|
@@ -32,13 +32,14 @@ Gem::Specification.new do |spec|
   end
 
   spec.add_dependency "base64"
+  spec.add_dependency "benchmark"
   spec.add_dependency "benchmark-ips"
   spec.add_dependency "csv"
   spec.add_dependency "liquid"
   spec.add_dependency "lutaml-model"
   spec.add_dependency "moxml"
   spec.add_dependency "paint"
-  spec.add_dependency "parslet", "~> 2.0"
+  spec.add_dependency "parsanol", "~> 1.1"
   spec.add_dependency "ruby-progressbar", "~> 1.11"
   spec.add_dependency "rubyzip", "~> 2.3"
   spec.add_dependency "table_tennis"

data/lib/expressir/benchmark.rb

@@ -30,7 +30,7 @@ module Expressir
           end
 
           # Calculate objects per second
-          if result.respond_to?(:schemas)
+          if result.is_a?(Model::Repository)
             objects_per_second = calculate_objects_per_second(result)
             puts "Objects per second: #{objects_per_second}" if Expressir.configuration.benchmark_verbose?
           end
@@ -42,7 +42,7 @@ module Expressir
             result = yield
           end
 
-          if result.respond_to?(:schemas)
+          if result.is_a?(Model::Repository)
             objects_per_second = calculate_objects_per_second(result, time.real)
             output_benchmark_result(file, time.real, objects_per_second)
           else
@@ -157,7 +157,7 @@ module Expressir
           puts "Cache write time:  #{results[:cache_write_time].round(4)}s"
           puts "Cache read time:   #{results[:cache_read_time].round(4)}s"
 
-          if results[:repository].respond_to?(:schemas)
+          if results[:repository].is_a?(Model::Repository)
             objects = count_objects(results[:repository])
             puts "Total objects:     #{objects}"
             puts "Objects per second (parsing): #{(objects / results[:parsing_time]).round(2)}"
@@ -189,7 +189,7 @@ module Expressir
       # @param repository [Object] The repository object
       # @return [Integer] Number of objects
       def count_objects(repository)
-        return 0 unless repository.respond_to?(:schemas)
+        return 0 unless repository.is_a?(Model::Repository)
 
         count = repository.schemas.size
 
@@ -261,12 +261,12 @@ module Expressir
 
         # Try to count total objects and schemas
         results.each do |result|
-          if result.respond_to?(:schemas)
+          if result.is_a?(Model::Repository)
             schema_count += result.schemas.size
             total_objects += count_objects(result)
           elsif result.is_a?(Array)
             result.each do |r|
-              if r.respond_to?(:id)  # Likely a schema
+              if r.is_a?(Model::Declarations::Schema)
                 schema_count += 1
                 total_objects += 1   # Count the schema itself
               end

data/lib/expressir/cli.rb

@@ -23,6 +23,15 @@ module Expressir
     end
 
     desc "format PATH", "pretty print EXPRESS schema located at PATH"
+    method_option :profile, type: :string,
+                            desc: "Formatting profile: 'iso' (ISO/TC 184/SC 4) or 'elf' (ELF Pretty Print)",
+                            default: "iso"
+    method_option :indent, type: :numeric,
+                           desc: "Indentation width (ELF profile only)",
+                           default: 4
+    method_option :provenance, type: :boolean,
+                               desc: "Include provenance information (ELF profile only)",
+                               default: true
     def format(path)
       Commands::Format.new(options).run(path)
     end

data/lib/expressir/commands/base.rb

@@ -17,15 +17,8 @@ module Expressir
         end
       end
 
-      def exit_with_error(message, exit_code = 1)
-        say message
-        # In test mode, raise an exception instead of exiting
-        # This makes it easier to test error cases
-        if defined?(@test_mode) && @test_mode
-          raise message # Just raise the message as an exception
-        else
-          exit exit_code
-        end
+      def exit_with_error(message, _exit_code = 1)
+        raise Expressir::CommandError.new(message)
       end
     end
   end

data/lib/expressir/commands/format.rb

@@ -3,6 +3,36 @@ module Expressir
     class Format < Base
       def run(path)
         repository = Expressir::Express::Parser.from_file(path)
+
+        profile = options[:profile] || "iso"
+
+        case profile.downcase
+        when "elf"
+          format_with_elf_profile(repository)
+        when "iso"
+          format_with_iso_profile(repository)
+        else
+          raise Expressir::InvalidOptionError.new(
+            "profile",
+            profile,
+            valid_options: ["iso", "elf"]
+          )
+        end
+      end
+
+      private
+
+      def format_with_elf_profile(repository)
+        formatter_options = {
+          indent: options[:indent] || 4,
+          provenance: options.fetch(:provenance, true),
+        }
+
+        formatter = Expressir::Express::PrettyFormatter.new(formatter_options)
+        say formatter.format(repository)
+      end
+
+      def format_with_iso_profile(repository)
         repository.schemas.each do |schema|
           say "\n(* Expressir formatted schema: #{schema.id} *)\n"
           say schema.to_s(no_remarks: true)