expressir 0.2.1 → 0.2.6

data/docs/development.md

@@ -1,90 +1,90 @@
-## Development Plan
-
-This document is a draft to understand the basic of the original library, and
-how the whole tool chain is working at the moment. Once, we've an understanding
-then let's look at the potential area of improvement, and plan out a high level
-plan to start the development.
-
-### What is it, expressir?
-
-* Expressir is a ruby tool to harvest iso express data model
-* Express is one of the language to represent data model
-* Expressir tool is used to manipulate iso express data model
-* The input for the expressir is a standard xml document
-* XML Spec: http://stepmod.sourceforge.net/express_model_spec/
-* Expressir does the job of representing express model as ruby class
-* REXML-based parser that reads EXPRESS as XML and creates the dictionary (i.e.
-  instances of the EXPRESS Ruby Classes) and then calls a mapper, that may be
-  specified as an option at runtime
-* Mappers that read the EXPRESS dictionary and perform the conversion to the
-  desired language or format
-* Express to UML2 Mapper - Convert the express model to UML2
-* Express to OWL Structure - Convert the express model to OWL Structure
-* Ruby API generator for express language
-* Current Ruby API is super slow, it takes 2 hours to generate 400 entity
-
-### How does the original version work?
-
-At the core the `expressir.rb` is responsible for everything at moment, it takes
-an Express XML file as an input and then it parses that XML file and build the
-correct ruby interface for further processing.
-
-This `expressir.rb` also expect one mapping file `deafult to mapping.rb`, and it
-also expect the file to specify a custom method `map_from_express` to take the
-`Ruby Express Representation` and convert this ruby representation to the
-desire output format.
-
-The library also provides couple mapping for UM2, OWL and SysML, so anyone can
-export their data to any of those format if necessary, and the use cases for
-those library are as following:
-
-```ruby
-ruby expressir.rb expxml=<schema.xml> map=<mapping_owl.rb>
-```
-
-### What are potential improvement area?
-
-* The interface is not well defined, someone needs to dig deeper to understand
-  just the basic of the library and how to use for their use cases. This is
-  something that could be improved as initial step.
-* At the moment, these are some ruby files, so potentially we could group those
-  together as a CLI tool and group some of the common functionality, and provide
-  a straight forward interface for users.
-* The good part about this library is author had the extensibility in mind from
-  the beginning, so we should keep that functionality as flexible as possible.
-* There are lot of boilerplate code in the library, but it's understandable as
-  it was written quite long time ago, so maybe most of the tool was not even
-  available back then, so this is the another area we could improve.
-* Another improvement we could do in terms of actual codebase, we should be able
-  to refactor this library and cleanup as much as possible.
-
-### What are the initial changes?
-
-Initially, we can start by creating a ruby cli gem, group these functionality
-together. This could be named as `expressir`, and this gem will be shipped with an
-executable called `expressir`.
-
-Second of all, let's add some dedicated interface for the default type, so user
-does not need to know all the details but the name of the interface, for example
-we could expose the transformation as follows:
-
-```sh
-# Convert to owl
-expressir express2owl file=[express_xml_file] [**options]
-
-# Convert to UML
-expressir express2uml file=[express_xml_file] [**options]
-
-# Convert to SysML
-expressir express2sysml file=[express_xml_file] [**options]
-
-# Custom conversion
-expressir express2custom file=[express_xml_file] mapping=[custom_mapping.rb]
-```
-
-Third of all once this library is functional as the original version then we
-will go through each of the existing types, and refactor those as necessary.
-
-### References
-
-* https://martinfowler.com/bliki/MovingToNokogiri.html
+## Development Plan
+
+This document is a draft to understand the basic of the original library, and
+how the whole tool chain is working at the moment. Once, we've an understanding
+then let's look at the potential area of improvement, and plan out a high level
+plan to start the development.
+
+### What is it, expressir?
+
+* Expressir is a ruby tool to harvest iso express data model
+* Express is one of the language to represent data model
+* Expressir tool is used to manipulate iso express data model
+* The input for the expressir is a standard xml document
+* XML Spec: http://stepmod.sourceforge.net/express_model_spec/
+* Expressir does the job of representing express model as ruby class
+* REXML-based parser that reads EXPRESS as XML and creates the dictionary (i.e.
+  instances of the EXPRESS Ruby Classes) and then calls a mapper, that may be
+  specified as an option at runtime
+* Mappers that read the EXPRESS dictionary and perform the conversion to the
+  desired language or format
+* Express to UML2 Mapper - Convert the express model to UML2
+* Express to OWL Structure - Convert the express model to OWL Structure
+* Ruby API generator for express language
+* Current Ruby API is super slow, it takes 2 hours to generate 400 entity
+
+### How does the original version work?
+
+At the core the `expressir.rb` is responsible for everything at moment, it takes
+an Express XML file as an input and then it parses that XML file and build the
+correct ruby interface for further processing.
+
+This `expressir.rb` also expect one mapping file `deafult to mapping.rb`, and it
+also expect the file to specify a custom method `map_from_express` to take the
+`Ruby Express Representation` and convert this ruby representation to the
+desire output format.
+
+The library also provides couple mapping for UM2, OWL and SysML, so anyone can
+export their data to any of those format if necessary, and the use cases for
+those library are as following:
+
+```ruby
+ruby expressir.rb expxml=<schema.xml> map=<mapping_owl.rb>
+```
+
+### What are potential improvement area?
+
+* The interface is not well defined, someone needs to dig deeper to understand
+  just the basic of the library and how to use for their use cases. This is
+  something that could be improved as initial step.
+* At the moment, these are some ruby files, so potentially we could group those
+  together as a CLI tool and group some of the common functionality, and provide
+  a straight forward interface for users.
+* The good part about this library is author had the extensibility in mind from
+  the beginning, so we should keep that functionality as flexible as possible.
+* There are lot of boilerplate code in the library, but it's understandable as
+  it was written quite long time ago, so maybe most of the tool was not even
+  available back then, so this is the another area we could improve.
+* Another improvement we could do in terms of actual codebase, we should be able
+  to refactor this library and cleanup as much as possible.
+
+### What are the initial changes?
+
+Initially, we can start by creating a ruby cli gem, group these functionality
+together. This could be named as `expressir`, and this gem will be shipped with an
+executable called `expressir`.
+
+Second of all, let's add some dedicated interface for the default type, so user
+does not need to know all the details but the name of the interface, for example
+we could expose the transformation as follows:
+
+```sh
+# Convert to owl
+expressir express2owl file=[express_xml_file] [**options]
+
+# Convert to UML
+expressir express2uml file=[express_xml_file] [**options]
+
+# Convert to SysML
+expressir express2sysml file=[express_xml_file] [**options]
+
+# Custom conversion
+expressir express2custom file=[express_xml_file] mapping=[custom_mapping.rb]
+```
+
+Third of all once this library is functional as the original version then we
+will go through each of the existing types, and refactor those as necessary.
+
+### References
+
+* https://martinfowler.com/bliki/MovingToNokogiri.html

data/exe/expressir

@@ -1,20 +1,20 @@
-#!/usr/bin/env ruby
-# encoding: UTF-8
-
-# resolve bin path, ignoring symlinks
-require "pathname"
-bin_file = Pathname.new(__FILE__).realpath
-
-# add self to libpath
-$:.unshift File.expand_path("../../lib", bin_file)
-
-# Fixes https://github.com/rubygems/rubygems/issues/1420
-require "rubygems/specification"
-
-class Gem::Specification
-  def this; self; end
-end
-
-# start up the CLI
-require "expressir"
-Expressir::Cli.start(ARGV)
+#!/usr/bin/env ruby
+# encoding: UTF-8
+
+# resolve bin path, ignoring symlinks
+require "pathname"
+bin_file = Pathname.new(__FILE__).realpath
+
+# add self to libpath
+$:.unshift File.expand_path("../../lib", bin_file)
+
+# Fixes https://github.com/rubygems/rubygems/issues/1420
+require "rubygems/specification"
+
+class Gem::Specification
+  def this; self; end
+end
+
+# start up the CLI
+require "expressir"
+Expressir::Cli.start(ARGV)

data/exe/generate-parser

@@ -0,0 +1,48 @@
+#!/usr/bin/env ruby
+
+require 'fileutils'
+require 'antlr4-native'
+
+grammar_file = ARGV.shift
+
+# ANTLR does weird things if the grammar file isn't in the current working directory
+temp_grammar_file = File.join(FileUtils.pwd(), File.basename(grammar_file))
+FileUtils.cp(grammar_file, temp_grammar_file)
+
+# generate parser
+generator = Antlr4Native::Generator.new(
+  grammar_files: [File.basename(temp_grammar_file)],
+  output_dir: 'ext',
+  parser_root_method: 'syntax'
+)
+generator.generate
+
+# fix issues with generated parser
+parser_source_file = File.join(*%w(ext express-parser express_parser.cpp))
+parser_source_lines = File.read(parser_source_file).split(/\n/)
+
+# - add ParserProxy tokens method, simple compensation for missing exposed BufferedTokenStream
+i = parser_source_lines.index{|x| x == '  Object syntax() {'}
+parser_source_lines[i + 6] += "\n" + <<~CPP.split(/\n/).map{|x| x != "" ? "  " + x : x}.join("\n") + "\n"
+  Array getTokens() {
+    Array a;
+
+    std::vector<Token*> tokens = this -> tokens -> getTokens();
+
+    for (auto &token : tokens) {
+      a.push(token);
+    }
+
+    return a;
+  }
+CPP
+i = parser_source_lines.index{|x| x == '    .define_method("syntax", &ParserProxy::syntax)'}
+parser_source_lines[i] += "\n" + <<~CPP.split(/\n/).map{|x| x != "" ? "    " + x : x}.join("\n")
+  .define_method("tokens", &ParserProxy::getTokens)
+CPP
+
+# write fixed parser file
+File.write(parser_source_file, parser_source_lines.join("\n") + "\n")
+
+# cleanup
+FileUtils.rm(temp_grammar_file)

data/expressir.gemspec

@@ -1,36 +1,42 @@
-lib = File.expand_path("lib", __dir__)
-$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
-require "expressir/version"
-
-Gem::Specification.new do |spec|
-  spec.name          = "expressir"
-  spec.version       = Expressir::VERSION
-  spec.authors       = ["Ribose Inc."]
-  spec.email         = ["open.source@ribose.com'"]
-
-  spec.summary       = "ISO EXPRESS parser in Ruby, tools for accessing EXPRESS data models."
-  spec.description   = "ISO EXPRESS parser in Ruby, tools for accessing EXPRESS data models."
-  spec.homepage      = "https://github.com/lutaml/expressir"
-  spec.license       = "MIT"
-
-  spec.metadata["homepage_uri"] = spec.homepage
-  spec.metadata["source_code_uri"] = spec.homepage
-  spec.metadata["changelog_uri"] = "https://github.com/lutaml/expressir/releases"
-
-  spec.files         = `git ls-files`.split("\n")
-  spec.test_files    = `git ls-files -- {spec}/*`.split("\n")
-
-  spec.bindir        = "exe"
-  spec.require_paths = ["lib"]
-  spec.executables   = %w[expressir]
-
-  spec.add_runtime_dependency "thor", "~> 1.0"
-  spec.add_runtime_dependency "activesupport", "~> 5.0"
-  spec.add_runtime_dependency "antlr4-runtime", "~> 0.2.12"
-  spec.add_development_dependency "nokogiri", "~> 1.10"
-
-  spec.add_development_dependency "bundler", "~> 2.0"
-  spec.add_development_dependency "pry", "~> 0.12.2"
-  spec.add_development_dependency "rake", "~> 10.0"
-  spec.add_development_dependency "rspec", "~> 3.0"
-end
+lib = File.expand_path("lib", __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "expressir/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "expressir"
+  spec.version       = Expressir::VERSION
+  spec.authors       = ["Ribose Inc."]
+  spec.email         = ["open.source@ribose.com'"]
+
+  spec.summary       = "ISO EXPRESS parser in Ruby, tools for accessing EXPRESS data models."
+  spec.description   = "ISO EXPRESS parser in Ruby, tools for accessing EXPRESS data models."
+  spec.homepage      = "https://github.com/lutaml/expressir"
+  spec.license       = "MIT"
+
+  spec.metadata["homepage_uri"] = spec.homepage
+  spec.metadata["source_code_uri"] = spec.homepage
+  spec.metadata["changelog_uri"] = "https://github.com/lutaml/expressir/releases"
+
+  spec.required_ruby_version = Gem::Requirement.new(">= 2.4.0")
+
+  spec.files         = `git ls-files`.split("\n")\
+    + Dir.glob('ext/express-parser/antlr4-upstream/runtime/Cpp/runtime/**/*')
+
+  spec.test_files    = `git ls-files -- {spec}/*`.split("\n")
+
+  spec.bindir        = "exe"
+  spec.require_paths = ["lib"]
+  spec.executables   = %w[expressir]
+
+  spec.extensions = File.join(*%w(ext express-parser extconf.rb))
+
+  spec.add_runtime_dependency "thor", "~> 1.0"
+  spec.add_runtime_dependency "rice", "~> 2.2.0"
+  spec.add_development_dependency "antlr4-native", "~> 1.1.0"
+  spec.add_development_dependency "nokogiri", "~> 1.10"
+  spec.add_development_dependency "bundler", "~> 2.0"
+  spec.add_development_dependency "pry", "~> 0.12.2"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rake-compiler", "~> 1.1.1"
+  spec.add_development_dependency "rspec", "~> 3.0"
+end

data/ext/express-parser/antlr4-upstream/runtime/Cpp/runtime/CMakeLists.txt

@@ -0,0 +1,118 @@
+
+include_directories(
+  ${PROJECT_SOURCE_DIR}/runtime/src
+  ${PROJECT_SOURCE_DIR}/runtime/src/atn
+  ${PROJECT_SOURCE_DIR}/runtime/src/dfa
+  ${PROJECT_SOURCE_DIR}/runtime/src/misc
+  ${PROJECT_SOURCE_DIR}/runtime/src/support
+  ${PROJECT_SOURCE_DIR}/runtime/src/tree
+  ${PROJECT_SOURCE_DIR}/runtime/src/tree/pattern
+  ${PROJECT_SOURCE_DIR}/runtime/src/tree/xpath
+)
+
+
+file(GLOB libantlrcpp_SRC
+  "${PROJECT_SOURCE_DIR}/runtime/src/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/atn/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/dfa/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/misc/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/support/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/tree/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/tree/pattern/*.cpp"
+  "${PROJECT_SOURCE_DIR}/runtime/src/tree/xpath/*.cpp"
+)
+
+add_library(antlr4_shared SHARED ${libantlrcpp_SRC})
+add_library(antlr4_static STATIC ${libantlrcpp_SRC})
+
+set(LIB_OUTPUT_DIR "${CMAKE_HOME_DIRECTORY}/dist") # put generated libraries here.
+message(STATUS "Output libraries to ${LIB_OUTPUT_DIR}")
+
+# make sure 'make' works fine even if ${LIB_OUTPUT_DIR} is deleted.
+add_custom_target(make_lib_output_dir ALL
+    COMMAND ${CMAKE_COMMAND} -E make_directory ${LIB_OUTPUT_DIR}
+    )
+
+add_dependencies(antlr4_shared make_lib_output_dir)
+add_dependencies(antlr4_static make_lib_output_dir)
+
+if(CMAKE_SYSTEM_NAME MATCHES "Linux")
+  target_link_libraries(antlr4_shared ${UUID_LIBRARIES})
+  target_link_libraries(antlr4_static ${UUID_LIBRARIES})
+elseif(APPLE)
+  target_link_libraries(antlr4_shared ${COREFOUNDATION_LIBRARY})
+  target_link_libraries(antlr4_static ${COREFOUNDATION_LIBRARY})
+endif()
+
+if(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
+  set(disabled_compile_warnings "/wd4251")
+else()
+  set(disabled_compile_warnings "-Wno-overloaded-virtual")
+endif()
+
+
+if("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
+  set(disabled_compile_warnings "${disabled_compile_warnings} -Wno-dollar-in-identifier-extension -Wno-four-char-constants")
+elseif("${CMAKE_CXX_COMPILER_ID}" MATCHES "GNU" OR "${CMAKE_CXX_COMPILER_ID}" MATCHES "Intel")
+  set(disabled_compile_warnings "${disabled_compile_warnings} -Wno-multichar")
+endif()
+
+set(extra_share_compile_flags "")
+set(extra_static_compile_flags "")
+if(WIN32)
+  set(extra_share_compile_flags "-DANTLR4CPP_EXPORTS")
+  set(extra_static_compile_flags "-DANTLR4CPP_STATIC")
+endif(WIN32)
+if(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
+  if(WITH_STATIC_CRT)
+    target_compile_options(antlr4_shared PRIVATE "/MT$<$<CONFIG:Debug>:d>")
+    target_compile_options(antlr4_static PRIVATE "/MT$<$<CONFIG:Debug>:d>")
+  else()
+    target_compile_options(antlr4_shared PRIVATE "/MD$<$<CONFIG:Debug>:d>")
+    target_compile_options(antlr4_static PRIVATE "/MD$<$<CONFIG:Debug>:d>")
+  endif()
+endif()
+
+set(static_lib_suffix "")
+if(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
+  set(static_lib_suffix "-static")
+endif()
+
+if(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
+  set(extra_share_compile_flags "-DANTLR4CPP_EXPORTS -MP /wd4251")
+  set(extra_static_compile_flags "-DANTLR4CPP_STATIC -MP")
+endif()
+
+set_target_properties(antlr4_shared
+                      PROPERTIES VERSION   ${ANTLR_VERSION}
+                                 SOVERSION ${ANTLR_VERSION}
+                                 OUTPUT_NAME antlr4-runtime
+                                 LIBRARY_OUTPUT_DIRECTORY ${LIB_OUTPUT_DIR}
+                                 # TODO: test in windows. DLL is treated as runtime.
+                                 # see https://cmake.org/cmake/help/v3.0/prop_tgt/LIBRARY_OUTPUT_DIRECTORY.html
+                                 RUNTIME_OUTPUT_DIRECTORY ${LIB_OUTPUT_DIR}
+                                 ARCHIVE_OUTPUT_DIRECTORY ${LIB_OUTPUT_DIR}
+                                 COMPILE_FLAGS "${disabled_compile_warnings} ${extra_share_compile_flags}")
+
+set_target_properties(antlr4_static
+                      PROPERTIES VERSION   ${ANTLR_VERSION}
+                                 SOVERSION ${ANTLR_VERSION}
+                                 OUTPUT_NAME "antlr4-runtime${static_lib_suffix}"
+                                 ARCHIVE_OUTPUT_DIRECTORY ${LIB_OUTPUT_DIR}
+                                 COMPILE_FLAGS "${disabled_compile_warnings} ${extra_static_compile_flags}")
+
+install(TARGETS antlr4_shared
+        DESTINATION lib 
+        EXPORT antlr4-targets)
+install(TARGETS antlr4_static
+        DESTINATION lib
+        EXPORT antlr4-targets)
+
+install(DIRECTORY "${PROJECT_SOURCE_DIR}/runtime/src/" 
+        DESTINATION "include/antlr4-runtime"
+        COMPONENT dev 
+        FILES_MATCHING PATTERN "*.h"
+        )
+
+
+