expressir 1.3.0-aarch64-linux-gnu

data/README.adoc

@@ -0,0 +1,155 @@
+= Expressir: EXPRESS in Ruby
+
+image:https://img.shields.io/gem/v/expressir.svg["Gem Version", link="https://rubygems.org/gems/expressir"]
+// image:https://codeclimate.com/github/lutaml/expressir/badges/gpa.svg["Code Climate", link="https://codeclimate.com/github/lutaml/expressir"]
+image:https://github.com/lutaml/expressir/workflows/rake/badge.svg["Build Status", link="https://github.com/lutaml/expressir/actions?workflow=rake"]
+
+== Purpose
+
+Expressir ("`EXPRESS in Ruby`") is a Ruby parser for EXPRESS and
+a set of Ruby tools for accessing ISO EXPRESS data models.
+
+== Architecture
+
+Expressir consists of 3 parts:
+
+. Parsers. A parser allows Expressir to read EXPRESS files, including:
+
+** EXPRESS data modelling language (ISO 10303-11:2007)
+** EXPRESS data modelling language in XML (STEPmod)
+** EXPRESS XML (ISO 10303-28:2007)
+"`Industrial automation systems and integration — Product data representation and exchange — Part 28: Implementation methods: XML representations of EXPRESS schemas and data, using XML schemas`")
+
+. Data model. The data model (`lib/expressir/express`) is the Ruby data model that fully represents an EXPRESS data model.
+
+. Converters. A converter transforms the EXPRESS Ruby data model into an interoperable export format, including:
+** EXPRESS data modelling language (ISO 10303-11:2007)
+** W3C OWL
+** OMG SysML (XMI 2.1, XMI 2.5)
+** OMG UML 2 (XMI 2.1)
+** OMG UML 2 for Eclipse (XMI 2.1)
+
+
+== Usage
+
+This gem ships with a CLI tool. To check what's available you can simply run
+the script directly from `exe/expressir`, by default it will display some usage
+instructions.
+
+[source, sh]
+----
+./exe/expressir
+
+Commands:
+  expressir help [COMMAND]  # Describe available commands or one specific command
+  expressir version         # The Expressir Version
+----
+
+== Development
+
+We are following Sandi Metz's Rules for this gem, you can read
+the http://robots.thoughtbot.com/post/50655960596/sandi-metz-rules-for-developers[description of the rules here] All new code should follow these rules.
+If you make changes in a pre-existing file that violates these rules you should
+fix the violations as part of your contribution.
+
+=== Setup
+
+Clone the repository.
+
+[source, sh]
+----
+git clone https://github.com/metanorma/expressir
+----
+
+Setup your environment.
+
+[source, sh]
+----
+bin/setup
+----
+
+Run the test suite
+
+[source, sh]
+----
+bin/rspec
+----
+
+=== Grammar updates
+EXPRESS grammar is lined as git submodule to ext/express-grammar
+Shoudl you update it, run ```rake generate```. This command will generate source code for updated native extension using antlr4-native gem.
+Please note that we create several classes on top of antlr4-native output so using embedded rake task is a real requirement.
+
+When new extension is gnerated and tested plase check in updated C++ files to git (```rake generate``` is NOT a CI step, 
+extension source files are pulled from the repo).
+
+== Installation
+
+Add this line to your application's `Gemfile`:
+
+[source, sh]
+----
+gem "expressir"
+----
+
+And then execute:
+
+[source, sh]
+----
+$ bundle install
+----
+
+Or install it yourself as:
+
+[source, sh]
+----
+$ gem install expressir
+----
+
+
+== Contributing
+
+First, thank you for contributing! We love pull requests from everyone. By
+participating in this project, you hereby grant
+https://www.ribose.com[Ribose Inc.] the right to grant or transfer an unlimited
+number of non exclusive licenses or sub-licenses to third parties, under the
+copyright covering the contribution to use the contribution by all means.
+
+Here are a few technical guidelines to follow:
+
+* Open an https://github.com/lutaml/expressir/issues[issues] to discuss a new
+  feature.
+* Write tests to support your new feature.
+* Make sure the entire test suite passes locally and on CI.
+* Open a Pull Request.
+* https://github.com/thoughtbot/guides/tree/master/protocol/git#write-a-feature[Squash your commits] after receiving feedback.
+* Party!
+
+
+== License
+
+Expressir is distributed under the BSD 2-clause license.
+
+Expressir is built on code originally from the NIST Reeper project.
+
+The https://www.nist.gov/services-resources/software/reeper[NIST Reeper license]
+is reproduced below:
+
+[quote]
+____
+This software was funded by NIST and developed by EuroSTEP.
+Pursuant to title 17 Section 105 of the United States Code this
+software is not subject to copyright protection and is in the public
+domain.
+
+We would appreciate acknowledgment if the software is used. Links to
+non-Federal Government Web sites do not imply NIST endorsement of any
+particular product, service, organization, company, information
+provider, or content.
+____
+
+
+== Credits
+
+Expressir is created using the structure and examples provided by
+the NIST Reeper software on https://sourceforge.net/p/reeper/[Sourceforge].

data/Rakefile

@@ -0,0 +1,17 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+require "rake/extensiontask"
+require "rubocop/rake_task"
+require "yard"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec
+
+GEMSPEC = Gem::Specification.load("expressir.gemspec")
+
+RuboCop::RakeTask.new
+
+Gem::PackageTask.new(GEMSPEC).define
+
+YARD::Rake::YardocTask.new

data/bin/console

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "reeper"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+require "pry"
+Pry.start

data/bin/rspec

@@ -0,0 +1,29 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+#
+# This file was generated by Bundler.
+#
+# The application 'rspec' is installed as part of a gem, and
+# this file is here to facilitate running it.
+#
+
+require "pathname"
+ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile",
+                                           Pathname.new(__FILE__).realpath)
+
+bundle_binstub = File.expand_path("bundle", __dir__)
+
+if File.file?(bundle_binstub)
+  if File.read(bundle_binstub, 300).include?("This file was generated by Bundler")
+    load(bundle_binstub)
+  else
+    abort("Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
+Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.")
+  end
+end
+
+require "rubygems"
+require "bundler/setup"
+
+load Gem.bin_path("rspec-core", "rspec")

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/docs/development.md

@@ -0,0 +1,90 @@
+## Development Plan
+
+This document is a draft to understand the basic of the original library, and
+how the whole tool chain is working at the moment. Once, we've an understanding
+then let's look at the potential area of improvement, and plan out a high level
+plan to start the development.
+
+### What is it, expressir?
+
+* Expressir is a ruby tool to harvest iso express data model
+* Express is one of the language to represent data model
+* Expressir tool is used to manipulate iso express data model
+* The input for the expressir is a standard xml document
+* XML Spec: http://stepmod.sourceforge.net/express_model_spec/
+* Expressir does the job of representing express model as ruby class
+* REXML-based parser that reads EXPRESS as XML and creates the dictionary (i.e.
+  instances of the EXPRESS Ruby Classes) and then calls a mapper, that may be
+  specified as an option at runtime
+* Mappers that read the EXPRESS dictionary and perform the conversion to the
+  desired language or format
+* Express to UML2 Mapper - Convert the express model to UML2
+* Express to OWL Structure - Convert the express model to OWL Structure
+* Ruby API generator for express language
+* Current Ruby API is super slow, it takes 2 hours to generate 400 entity
+
+### How does the original version work?
+
+At the core the `expressir.rb` is responsible for everything at moment, it takes
+an Express XML file as an input and then it parses that XML file and build the
+correct ruby interface for further processing.
+
+This `expressir.rb` also expect one mapping file `deafult to mapping.rb`, and it
+also expect the file to specify a custom method `map_from_express` to take the
+`Ruby Express Representation` and convert this ruby representation to the
+desire output format.
+
+The library also provides couple mapping for UM2, OWL and SysML, so anyone can
+export their data to any of those format if necessary, and the use cases for
+those library are as following:
+
+```ruby
+ruby expressir.rb expxml=<schema.xml> map=<mapping_owl.rb>
+```
+
+### What are potential improvement area?
+
+* The interface is not well defined, someone needs to dig deeper to understand
+  just the basic of the library and how to use for their use cases. This is
+  something that could be improved as initial step.
+* At the moment, these are some ruby files, so potentially we could group those
+  together as a CLI tool and group some of the common functionality, and provide
+  a straight forward interface for users.
+* The good part about this library is author had the extensibility in mind from
+  the beginning, so we should keep that functionality as flexible as possible.
+* There are lot of boilerplate code in the library, but it's understandable as
+  it was written quite long time ago, so maybe most of the tool was not even
+  available back then, so this is the another area we could improve.
+* Another improvement we could do in terms of actual codebase, we should be able
+  to refactor this library and cleanup as much as possible.
+
+### What are the initial changes?
+
+Initially, we can start by creating a ruby cli gem, group these functionality
+together. This could be named as `expressir`, and this gem will be shipped with an
+executable called `expressir`.
+
+Second of all, let's add some dedicated interface for the default type, so user
+does not need to know all the details but the name of the interface, for example
+we could expose the transformation as follows:
+
+```sh
+# Convert to owl
+expressir express2owl file=[express_xml_file] [**options]
+
+# Convert to UML
+expressir express2uml file=[express_xml_file] [**options]
+
+# Convert to SysML
+expressir express2sysml file=[express_xml_file] [**options]
+
+# Custom conversion
+expressir express2custom file=[express_xml_file] mapping=[custom_mapping.rb]
+```
+
+Third of all once this library is functional as the original version then we
+will go through each of the existing types, and refactor those as necessary.
+
+### References
+
+* https://martinfowler.com/bliki/MovingToNokogiri.html

data/exe/expressir

@@ -0,0 +1,22 @@
+#!/usr/bin/env ruby
+# encoding: UTF-8
+
+# resolve bin path, ignoring symlinks
+require "pathname"
+bin_file = Pathname.new(__FILE__).realpath
+
+# add self to libpath
+$:.unshift File.expand_path("../../lib", bin_file)
+
+# Fixes https://github.com/rubygems/rubygems/issues/1420
+require "rubygems/specification"
+
+module Gem
+  class Specification
+    def this; self; end
+  end
+end
+
+# start up the CLI
+require "expressir"
+Expressir::Cli.start(ARGV)

data/exe/format

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+require "yaml"
+require "expressir/express/parser"
+require "expressir/express/formatter"
+require "expressir/express/schema_head_formatter"
+require "expressir/express/hyperlink_formatter"
+
+exp_files = ARGV
+
+repository = Expressir::Express::Parser.from_files(exp_files)
+formatter = Class.new(Expressir::Express::Formatter) do
+  include Expressir::Express::SchemaHeadFormatter
+  include Expressir::Express::HyperlinkFormatter
+end
+result = repository.to_hash(formatter: formatter, skip_empty: true)
+
+puts YAML.dump(result)

data/exe/format-test

@@ -0,0 +1,81 @@
+#!/usr/bin/env ruby
+
+require "yaml"
+require "tempfile"
+require "expressir/express/parser"
+require "expressir/express/formatter"
+require "expressir/express/schema_head_formatter"
+require "expressir/express/hyperlink_formatter"
+require "expressir/express/cache"
+
+exp_files = [
+  # basic test
+  # '../iso-10303-stepmod/data/resources/action_schema/action_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/basic_attribute_schema/basic_attribute_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/support_resource_schema/support_resource_schema_annotated.exp',
+  # cyclic reference test
+  # '../iso-10303-stepmod/data/modules/analysis/mim_annotated.exp',
+  # '../iso-10303-stepmod/data/modules/analysis_product_relationships/mim_annotated.exp',
+  # cyclic reference test 2
+  # '../iso-10303-stepmod/data/resources/product_property_definition_schema/product_property_definition_schema_annotated.exp',
+  # '../iso-10303-stepmod/data/resources/product_property_representation_schema/product_property_representation_schema_annotated.exp',
+  # renamed reference test (36s)
+  # '../iso-10303-stepmod/ballots/ballots/ap210_wg12/express/resources/mathematical_functions_schema.exp',
+  # '../iso-10303-stepmod/data/resources/iso13584_expressions_schema/iso13584_expressions_schema.exp',
+  # annotated-express test (12s)
+  # see https://github.com/metanorma/annotated-express/blob/master/data/documents/resources/fundamentals_of_product_description_and_support/sections/04-schemas.adoc
+  "../iso-10303-stepmod/data/resources/action_schema/action_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/application_context_schema/application_context_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/approval_schema/approval_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/basic_attribute_schema/basic_attribute_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/certification_schema/certification_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/contract_schema/contract_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/date_time_schema/date_time_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/document_schema/document_schema.exp",
+  "../iso-10303-stepmod/data/resources/effectivity_schema/effectivity_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/experience_schema/experience_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/external_reference_schema/external_reference_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/group_schema/group_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/language_schema/language_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/location_schema/location_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/management_resources_schema/management_resources_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/measure_schema/measure_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/person_organization_schema/person_organization_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/process_property_schema/process_property_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_definition_schema/product_definition_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_property_definition_schema/product_property_definition_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/product_property_representation_schema/product_property_representation_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/qualifications_schema/qualifications_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/security_classification_schema/security_classification_schema_annotated.exp",
+  "../iso-10303-stepmod/data/resources/support_resource_schema/support_resource_schema_annotated.exp",
+  # full test (6m18s + 8s = 6m26s)
+  # *`bundle exec ../stepmod-utils/exe/stepmod-find-express-files ../iso-10303-stepmod`.strip.split("\n").map{|file| File.exists?(file.sub(/\.exp$/, '_annotated.exp')) ? file.sub(/\.exp$/, '_annotated.exp') : file}
+]
+
+start = Time.now
+repository = Expressir::Express::Parser.from_files(exp_files)
+puts "Parser.from_files time:  #{(Time.now - start).round(2)}s"
+
+temp_file = Tempfile.new
+begin
+  start = Time.now
+  Expressir::Express::Cache.to_file(temp_file, repository)
+  puts "Cache.to_file time:      #{(Time.now - start).round(2)}s"
+
+  start = Time.now
+  repository = Expressir::Express::Cache.from_file(temp_file)
+  puts "Cache.from_file time:    #{(Time.now - start).round(2)}s"
+ensure
+  temp_file.close
+  temp_file.unlink
+end
+
+start = Time.now
+formatter = Class.new(Expressir::Express::Formatter) do
+  include Expressir::Express::SchemaHeadFormatter
+  include Expressir::Express::HyperlinkFormatter
+end
+result = repository.to_hash(formatter: formatter, skip_empty: true)
+puts "Repository.to_hash time: #{(Time.now - start).round(2)}s"
+
+# puts YAML.dump(result)

data/exe/generate-parser

@@ -0,0 +1,51 @@
+#!/usr/bin/env ruby
+
+require "fileutils"
+require "antlr4-native"
+
+grammar_file = ARGV.shift
+
+# ANTLR does weird things if the grammar file isn't in the current working directory
+temp_grammar_file = File.join(FileUtils.pwd, File.basename(grammar_file))
+FileUtils.cp(grammar_file, temp_grammar_file)
+
+# generate parser
+generator = Antlr4Native::Generator.new(
+  grammar_files: [File.basename(temp_grammar_file)],
+  output_dir: "ext",
+  parser_root_method: "syntax",
+)
+generator.generate
+
+# fix issues with generated parser
+parser_source_file = File.join("ext", "express-parser", "express_parser.cpp")
+parser_source_lines = File.read(parser_source_file).split("\n")
+
+# - add ParserProxy tokens method, simple compensation for missing exposed BufferedTokenStream
+i = parser_source_lines.index { |x| x == "  Object syntax() {" }
+parser_source_lines[i + 6] += <<~CPP.split("\n").map { |x| x == "" ? x : "  #{x}" }.join("\n")
+
+  Array getTokens() {
+    Array a;
+
+    std::vector<Token*> tokens = this -> tokens -> getTokens();
+
+    for (auto &token : tokens) {
+      a.push(token);
+    }
+
+    return a;
+  }
+
+CPP
+i = parser_source_lines.index { |x| x == '    .define_method("syntax", &ParserProxy::syntax, Return().keepAlive())' }
+parser_source_lines[i] += <<~CPP.split("\n").map { |x| x == "" ? x : "    #{x}" }.join("\n")
+
+  .define_method("tokens", &ParserProxy::getTokens)
+CPP
+
+# write fixed parser file
+File.write(parser_source_file, "#{parser_source_lines.join("\n")}\n")
+
+# cleanup
+FileUtils.rm(temp_grammar_file)

data/expressir.gemspec

@@ -0,0 +1,48 @@
+lib = File.expand_path("lib", __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "expressir/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "expressir"
+  spec.version       = Expressir::VERSION
+  spec.authors       = ["Ribose Inc."]
+  spec.email         = ["open.source@ribose.com"]
+
+  spec.summary       = "ISO EXPRESS parser and tools in Ruby."
+  spec.description   = "Expressir (“EXPRESS in Ruby”) is a Ruby parser for EXPRESS and a set of Ruby tools for accessing ISO EXPRESS data models."
+  spec.homepage      = "https://github.com/lutaml/expressir"
+  spec.license       = "MIT"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = spec.homepage
+  spec.metadata["changelog_uri"] = "https://github.com/lutaml/expressir/releases"
+
+  spec.required_ruby_version = Gem::Requirement.new(">= 2.7.0")
+
+  spec.files         = `git ls-files`.split("\n")\
+    + Dir.glob("ext/express-parser/antlr4-upstream/runtime/Cpp/runtime/**/*")
+
+  spec.test_files    = `git ls-files -- {spec}/*`.split("\n")
+
+  spec.bindir        = "exe"
+  spec.require_paths = ["lib"]
+  spec.executables   = %w[expressir]
+
+  spec.extensions = File.join(*%w(ext express-parser extconf.rb))
+
+  spec.add_runtime_dependency "rice", "~> 4.1"
+  spec.add_runtime_dependency "thor", "~> 1.0"
+  spec.add_development_dependency "antlr4-native", "~> 2.1.0"
+  spec.add_development_dependency "asciidoctor", "~> 2.0.13"
+  spec.add_development_dependency "bundler", "~> 2.3"
+  spec.add_development_dependency "byebug", "~> 11.1"
+  spec.add_development_dependency "pry", "~> 0.12.2"
+  spec.add_development_dependency "rake", "~> 13.0"
+  spec.add_development_dependency "rake-compiler", "~> 1.2"
+  spec.add_development_dependency "rake-compiler-dock", "~> 1.3"
+  spec.add_development_dependency "rspec", "~> 3.11"
+  spec.add_development_dependency "rubocop", "1.58"
+  spec.add_development_dependency "rubocop-performance", "~> 1.19"
+  spec.add_development_dependency "webrick", "~> 1.7.0"
+  spec.add_development_dependency "yard", "~> 0.9.26"
+end

data/lib/expressir/cli/ui.rb

@@ -0,0 +1,36 @@
+require "thor"
+
+module Expressir
+  module Cli
+    class UI < Thor
+      def self.ask(message)
+        new.ask(message)
+      end
+
+      def self.say(message)
+        new.say(message)
+      end
+
+      def self.error(message)
+        if log_types.include?("error")
+          new.error(message)
+        end
+      end
+
+      def self.info(message)
+        if log_types.include?("info")
+          new.say(message)
+        end
+      end
+
+      def self.run(command)
+        require "open3"
+        Open3.capture3(command)
+      end
+
+      def self.log_types
+        Expressir.configuration.logs.map(&:to_s) || []
+      end
+    end
+  end
+end

data/lib/expressir/cli.rb

@@ -0,0 +1,21 @@
+require "thor"
+require "expressir/cli/ui"
+
+module Expressir
+  module Cli
+    def self.ui
+      Expressir::Cli::UI
+    end
+
+    def self.start(args)
+      Base.start(args)
+    end
+
+    class Base < Thor
+      desc "version", "The Expressir Version"
+      def version
+        Cli.ui.say("Version #{Expressir::VERSION}")
+      end
+    end
+  end
+end

data/lib/expressir/config.rb

@@ -0,0 +1,23 @@
+module Expressir
+  module Config
+    def configure
+      if block_given?
+        yield configuration
+      end
+    end
+
+    def configuration
+      @configuration ||= Configuration.new
+    end
+  end
+
+  class Configuration
+    attr_accessor :logs
+
+    def initialize
+      @logs = %i(error)
+    end
+  end
+
+  extend Config
+end