expressir 1.3.0.pre.1-x86_64-linux-gnu

data/Rakefile

@@ -0,0 +1,17 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require "rake/extensiontask"
+require "rubocop/rake_task"
+require "yard"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: %i[compile spec]
+
+GEMSPEC = Gem::Specification.load("expressir.gemspec")
+
+RuboCop::RakeTask.new
+
+Gem::PackageTask.new(GEMSPEC).define
+
+YARD::Rake::YardocTask.new

data/bin/console

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "reeper"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+require "pry"
+Pry.start

data/bin/rspec

@@ -0,0 +1,29 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'rspec' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require "pathname"
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
+                                           Pathname.new(__FILE__).realpath)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300).include?("This file was generated by Bundler")
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("rspec-core", "rspec")

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/docs/development.md

@@ -0,0 +1,90 @@
+## Development Plan
+
+This document is a draft to understand the basic of the original library, and
+how the whole tool chain is working at the moment. Once, we've an understanding
+then let's look at the potential area of improvement, and plan out a high level
+plan to start the development.
+
+### What is it, expressir?
+
+* Expressir is a ruby tool to harvest iso express data model
+* Express is one of the language to represent data model
+* Expressir tool is used to manipulate iso express data model
+* The input for the expressir is a standard xml document
+* XML Spec: http://stepmod.sourceforge.net/express_model_spec/
+* Expressir does the job of representing express model as ruby class
+* REXML-based parser that reads EXPRESS as XML and creates the dictionary (i.e.
+  instances of the EXPRESS Ruby Classes) and then calls a mapper, that may be
+  specified as an option at runtime
+* Mappers that read the EXPRESS dictionary and perform the conversion to the
+  desired language or format
+* Express to UML2 Mapper - Convert the express model to UML2
+* Express to OWL Structure - Convert the express model to OWL Structure
+* Ruby API generator for express language
+* Current Ruby API is super slow, it takes 2 hours to generate 400 entity
+
+### How does the original version work?
+
+At the core the `expressir.rb` is responsible for everything at moment, it takes
+an Express XML file as an input and then it parses that XML file and build the
+correct ruby interface for further processing.
+
+This `expressir.rb` also expect one mapping file `deafult to mapping.rb`, and it
+also expect the file to specify a custom method `map_from_express` to take the
+`Ruby Express Representation` and convert this ruby representation to the
+desire output format.
+
+The library also provides couple mapping for UM2, OWL and SysML, so anyone can
+export their data to any of those format if necessary, and the use cases for
+those library are as following:
+
+```ruby
+ruby expressir.rb expxml=<schema.xml> map=<mapping_owl.rb>
+```
+
+### What are potential improvement area?
+
+* The interface is not well defined, someone needs to dig deeper to understand
+  just the basic of the library and how to use for their use cases. This is
+  something that could be improved as initial step.
+* At the moment, these are some ruby files, so potentially we could group those
+  together as a CLI tool and group some of the common functionality, and provide
+  a straight forward interface for users.
+* The good part about this library is author had the extensibility in mind from
+  the beginning, so we should keep that functionality as flexible as possible.
+* There are lot of boilerplate code in the library, but it's understandable as
+  it was written quite long time ago, so maybe most of the tool was not even
+  available back then, so this is the another area we could improve.
+* Another improvement we could do in terms of actual codebase, we should be able
+  to refactor this library and cleanup as much as possible.
+
+### What are the initial changes?
+
+Initially, we can start by creating a ruby cli gem, group these functionality
+together. This could be named as `expressir`, and this gem will be shipped with an
+executable called `expressir`.
+
+Second of all, let's add some dedicated interface for the default type, so user
+does not need to know all the details but the name of the interface, for example
+we could expose the transformation as follows:
+
+```sh
+# Convert to owl
+expressir express2owl file=[express_xml_file] [**options]
+
+# Convert to UML
+expressir express2uml file=[express_xml_file] [**options]
+
+# Convert to SysML
+expressir express2sysml file=[express_xml_file] [**options]
+
+# Custom conversion
+expressir express2custom file=[express_xml_file] mapping=[custom_mapping.rb]
+```
+
+Third of all once this library is functional as the original version then we
+will go through each of the existing types, and refactor those as necessary.
+
+### References
+
+* https://martinfowler.com/bliki/MovingToNokogiri.html

data/exe/expressir

@@ -0,0 +1,22 @@
+#!/usr/bin/env ruby
+# encoding: UTF-8
+
+# resolve bin path, ignoring symlinks
+require "pathname"
+bin_file = Pathname.new(__FILE__).realpath
+
+# add self to libpath
+$:.unshift File.expand_path("../../lib", bin_file)
+
+# Fixes https://github.com/rubygems/rubygems/issues/1420
+require "rubygems/specification"
+
+module Gem
+  class Specification
+    def this; self; end
+  end
+end
+
+# start up the CLI
+require "expressir"
+Expressir::Cli.start(ARGV)

data/exe/format

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+require "yaml"
+require "expressir/express/parser"
+require "expressir/express/formatter"
+require "expressir/express/schema_head_formatter"
+require "expressir/express/hyperlink_formatter"
+
+exp_files = ARGV
+
+repository = Expressir::Express::Parser.from_files(exp_files)
+formatter = Class.new(Expressir::Express::Formatter) do
+  include Expressir::Express::SchemaHeadFormatter
+  include Expressir::Express::HyperlinkFormatter
+end
+result = repository.to_hash(formatter: formatter, skip_empty: true)
+
+puts YAML.dump(result)

data/exe/format-test

@@ -0,0 +1,81 @@
+#!/usr/bin/env ruby
+
+require "yaml"
+require "tempfile"
+require "expressir/express/parser"
+require "expressir/express/formatter"
+require "expressir/express/schema_head_formatter"
+require "expressir/express/hyperlink_formatter"
+require "expressir/express/cache"
+
+exp_files = [
+  # basic test
+  # '../iso-10303-stepmod/data/resources/action_schema/action_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/basic_attribute_schema/basic_attribute_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/support_resource_schema/support_resource_schema_annotated.exp',
+  # cyclic reference test
+  # '../iso-10303-stepmod/data/modules/analysis/mim_annotated.exp',
+  # '../iso-10303-stepmod/data/modules/analysis_product_relationships/mim_annotated.exp',
+  # cyclic reference test 2
+  # '../iso-10303-stepmod/data/resources/product_property_definition_schema/product_property_definition_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/product_property_representation_schema/product_property_representation_schema_annotated.exp',
+  # renamed reference test (36s)
+  # '../iso-10303-stepmod/ballots/ballots/ap210_wg12/express/resources/mathematical_functions_schema.exp',
+  # '../iso-10303-stepmod/data/resources/iso13584_expressions_schema/iso13584_expressions_schema.exp',
+  # annotated-express test (12s)
+  # see https://github.com/metanorma/annotated-express/blob/master/data/documents/resources/fundamentals_of_product_description_and_support/sections/04-schemas.adoc
+  "../iso-10303-stepmod/data/resources/action_schema/action_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/application_context_schema/application_context_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/approval_schema/approval_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/basic_attribute_schema/basic_attribute_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/certification_schema/certification_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/contract_schema/contract_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/date_time_schema/date_time_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/document_schema/document_schema.exp",
+  "../iso-10303-stepmod/data/resources/effectivity_schema/effectivity_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/experience_schema/experience_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/external_reference_schema/external_reference_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/group_schema/group_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/language_schema/language_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/location_schema/location_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/management_resources_schema/management_resources_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/measure_schema/measure_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/person_organization_schema/person_organization_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/process_property_schema/process_property_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_definition_schema/product_definition_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_property_definition_schema/product_property_definition_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_property_representation_schema/product_property_representation_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/qualifications_schema/qualifications_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/security_classification_schema/security_classification_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/support_resource_schema/support_resource_schema_annotated.exp",
+  # full test (6m18s + 8s = 6m26s)
+  # *`bundle exec ../stepmod-utils/exe/stepmod-find-express-files ../iso-10303-stepmod`.strip.split("\n").map{|file| File.exists?(file.sub(/\.exp$/, '_annotated.exp')) ? file.sub(/\.exp$/, '_annotated.exp') : file}
+]
+
+start = Time.now
+repository = Expressir::Express::Parser.from_files(exp_files)
+puts "Parser.from_files time:  #{(Time.now - start).round(2)}s"
+
+temp_file = Tempfile.new
+begin
+  start = Time.now
+  Expressir::Express::Cache.to_file(temp_file, repository)
+  puts "Cache.to_file time:      #{(Time.now - start).round(2)}s"
+
+  start = Time.now
+  repository = Expressir::Express::Cache.from_file(temp_file)
+  puts "Cache.from_file time:    #{(Time.now - start).round(2)}s"
+ensure
+  temp_file.close
+  temp_file.unlink
+end
+
+start = Time.now
+formatter = Class.new(Expressir::Express::Formatter) do
+  include Expressir::Express::SchemaHeadFormatter
+  include Expressir::Express::HyperlinkFormatter
+end
+result = repository.to_hash(formatter: formatter, skip_empty: true)
+puts "Repository.to_hash time: #{(Time.now - start).round(2)}s"
+
+# puts YAML.dump(result)

data/exe/generate-parser

@@ -0,0 +1,51 @@
+#!/usr/bin/env ruby
+
+require "fileutils"
+require "antlr4-native"
+
+grammar_file = ARGV.shift
+
+# ANTLR does weird things if the grammar file isn't in the current working directory
+temp_grammar_file = File.join(FileUtils.pwd, File.basename(grammar_file))
+FileUtils.cp(grammar_file, temp_grammar_file)
+
+# generate parser
+generator = Antlr4Native::Generator.new(
+  grammar_files: [File.basename(temp_grammar_file)],
+  output_dir: "ext",
+  parser_root_method: "syntax",
+)
+generator.generate
+
+# fix issues with generated parser
+parser_source_file = File.join("ext", "express-parser", "express_parser.cpp")
+parser_source_lines = File.read(parser_source_file).split("\n")
+
+# - add ParserProxy tokens method, simple compensation for missing exposed BufferedTokenStream
+i = parser_source_lines.index { |x| x == "  Object syntax() {" }
+parser_source_lines[i + 6] += <<~CPP.split("\n").map { |x| x == "" ? x : "  #{x}" }.join("\n")
+
+  Array getTokens() {
+    Array a;
+
+    std::vector<Token*> tokens = this -> tokens -> getTokens();
+
+    for (auto &token : tokens) {
+      a.push(token);
+    }
+
+    return a;
+  }
+
+CPP
+i = parser_source_lines.index { |x| x == '    .define_method("syntax", &ParserProxy::syntax, Return().keepAlive())' }
+parser_source_lines[i] += <<~CPP.split("\n").map { |x| x == "" ? x : "    #{x}" }.join("\n")
+
+  .define_method("tokens", &ParserProxy::getTokens)
+CPP
+
+# write fixed parser file
+File.write(parser_source_file, "#{parser_source_lines.join("\n")}\n")
+
+# cleanup
+FileUtils.rm(temp_grammar_file)

data/expressir.gemspec

@@ -0,0 +1,48 @@
+lib = File.expand_path("lib", __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "expressir/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "expressir"
+  spec.version       = Expressir::VERSION
+  spec.authors       = ["Ribose Inc."]
+  spec.email         = ["open.source@ribose.com"]
+
+  spec.summary       = "ISO EXPRESS parser and tools in Ruby."
+  spec.description   = "Expressir (“EXPRESS in Ruby”) is a Ruby parser for EXPRESS and a set of Ruby tools for accessing ISO EXPRESS data models."
+  spec.homepage      = "https://github.com/lutaml/expressir"
+  spec.license       = "MIT"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = spec.homepage
+  spec.metadata["changelog_uri"] = "https://github.com/lutaml/expressir/releases"
+
+  spec.required_ruby_version = Gem::Requirement.new(">= 2.7.0")
+
+  spec.files         = `git ls-files`.split("\n")\
+    + Dir.glob("ext/express-parser/antlr4-upstream/runtime/Cpp/runtime/**/*")
+
+  spec.test_files    = `git ls-files -- {spec}/*`.split("\n")
+
+  spec.bindir        = "exe"
+  spec.require_paths = ["lib"]
+  spec.executables   = %w[expressir]
+
+  spec.extensions = File.join(*%w(ext express-parser extconf.rb))
+
+  spec.add_runtime_dependency "rice", "~> 4.1"
+  spec.add_runtime_dependency "thor", "~> 1.0"
+  spec.add_development_dependency "antlr4-native", "~> 2.1.0"
+  spec.add_development_dependency "asciidoctor", "~> 2.0.13"
+  spec.add_development_dependency "bundler", "~> 2.3"
+  spec.add_development_dependency "byebug", "~> 11.1"
+  spec.add_development_dependency "pry", "~> 0.12.2"
+  spec.add_development_dependency "rake", "~> 13.0"
+  spec.add_development_dependency "rake-compiler", "~> 1.2"
+  spec.add_development_dependency "rake-compiler-dock", "~> 1.3"
+  spec.add_development_dependency "rspec", "~> 3.11"
+  spec.add_development_dependency "rubocop", "1.58"
+  spec.add_development_dependency "rubocop-performance", "~> 1.19"
+  spec.add_development_dependency "webrick", "~> 1.7.0"
+  spec.add_development_dependency "yard", "~> 0.9.26"
+end

data/lib/expressir/cli/ui.rb

@@ -0,0 +1,36 @@
+require "thor"
+
+module Expressir
+  module Cli
+    class UI < Thor
+      def self.ask(message)
+        new.ask(message)
+      end
+
+      def self.say(message)
+        new.say(message)
+      end
+
+      def self.error(message)
+        if log_types.include?("error")
+          new.error(message)
+        end
+      end
+
+      def self.info(message)
+        if log_types.include?("info")
+          new.say(message)
+        end
+      end
+
+      def self.run(command)
+        require "open3"
+        Open3.capture3(command)
+      end
+
+      def self.log_types
+        Expressir.configuration.logs.map(&:to_s) || []
+      end
+    end
+  end
+end

data/lib/expressir/cli.rb

@@ -0,0 +1,21 @@
+require "thor"
+require "expressir/cli/ui"
+
+module Expressir
+  module Cli
+    def self.ui
+      Expressir::Cli::UI
+    end
+
+    def self.start(args)
+      Base.start(args)
+    end
+
+    class Base < Thor
+      desc "version", "The Expressir Version"
+      def version
+        Cli.ui.say("Version #{Expressir::VERSION}")
+      end
+    end
+  end
+end

data/lib/expressir/config.rb

@@ -0,0 +1,23 @@
+module Expressir
+  module Config
+    def configure
+      if block_given?
+        yield configuration
+      end
+    end
+
+    def configuration
+      @configuration ||= Configuration.new
+    end
+  end
+
+  class Configuration
+    attr_accessor :logs
+
+    def initialize
+      @logs = %i(error)
+    end
+  end
+
+  extend Config
+end

data/lib/expressir/express/cache.rb

@@ -0,0 +1,51 @@
+require 'yaml'
+require 'zlib'
+require 'expressir/model'
+
+module Expressir
+  module Express
+    class Cache
+      # Save Express model into a cache file
+      # @param file [String] cache file path
+      # @param content [Model::ModelElement] Express model
+      # @param root_path [String] Express repository root path, to be stripped from Express file paths to create a portable cache file
+      # @param test_overwrite_version [String] don't use, only for tests
+      # @return [nil]
+      def self.to_file(file, content, root_path: nil, test_overwrite_version: nil)
+        version = test_overwrite_version || VERSION
+
+        cache = Model::Cache.new(
+          version: version,
+          content: content
+        )
+
+        hash = cache.to_hash(root_path: root_path)
+        yaml = YAML.dump(hash)
+        yaml_compressed = Zlib::Deflate.deflate(yaml)
+
+        File.binwrite(file, yaml_compressed)
+        nil
+      end
+
+      # Load Express model from a cache file
+      # @param file [String] cache file path
+      # @param root_path [String] Express repository root path, to be prepended to Express file paths if loading a portable cache file
+      # @param test_overwrite_version [String] don't use, only for tests
+      # @return [Model::ModelElement] Express model
+      def self.from_file(file, root_path: nil, test_overwrite_version: nil)
+        version = test_overwrite_version || VERSION
+
+        yaml_compressed = File.binread(file)
+        yaml = Zlib::Inflate.inflate(yaml_compressed)
+        hash = YAML.load(yaml)
+        cache = Model::ModelElement.from_hash(hash, root_path: root_path)
+
+        if cache.version != version
+          raise Error.new("Cache version mismatch, cache version is #{cache.version}, Expressir version is #{version}")
+        end
+
+        cache.content
+      end
+    end
+  end
+end

data/lib/expressir/express/extension.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+begin
+    # native precompiled gems package shared libraries in <gem_dir>/lib/nokogiri/<ruby_version>
+    ::RUBY_VERSION =~ /(\d+\.\d+)/
+    require_relative "#{Regexp.last_match(1)}/express_parser"
+  rescue LoadError => e
+#    if /musl/.match?(RUBY_PLATFORM)
+#      warn(<<~EOM)
+#  
+#        ERROR: It looks like you're trying to use Expressir as a precompiled native gem on a musl system.
+#  
+#          #{e.message}
+#  
+#          If that's the case, then please install Expressir via the `ruby` platform gem:
+#              gem install expressir --platform=ruby
+#          or:
+#              bundle config set force_ruby_platform true
+#  
+#  
+#      EOM
+#      raise e
+#    end
+  
+    # use "require" instead of "require_relative" because non-native gems will place C extension files
+    # in Gem::BasicSpecification#extension_dir after compilation (during normal installation), which
+    # is in $LOAD_PATH but not necessarily relative to this file
+    require "expressir/express/express_parser"
+  end
+