chem_scanner 0.1.3

data/README.md

@@ -0,0 +1,177 @@
+
+# Introduction
+
+The `ChemScanner` library attempts to extract and interpret reactions/molecules information from ChemDraw-related files format: CDX, CDXML, embedded CDX within DOC and DOCX, [Perkin Elmer ELN](http://www.perkinelmer.com/category/notebook).
+
+# Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'chem_scanner'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install chem_scanner
+
+# UI for ChemScanner
+You can try the `ChemScanner` at https://eln.chemotion.net/ or https://eln.chemotion.net/chemscanner. The UI is more user-friendly which some additional features:
+
+ - Export to Excel and CML.
+ - Preview of the original scheme.
+ - Import directly to [Chemotion ELN](https://eln.chemotion.net)
+ - Add comment for each extracted scheme. These comments would also appear in the export and Chemotion ELN imported molecules/reactions.
+ - ...
+
+# Usage
+
+To scan/extract a single CDX file
+
+```ruby
+require 'chem_scanner'
+
+cdx = ChemScanner::Cdx.new
+cdx.read('/path/to/cdx/file')
+# Get array of scanned Canonical SMILES
+cdx.molecules.map(&:get_cano_smiles)
+# Get array of scanned Reactions in SMILES
+cdx.reactions.map(&:reaction_smiles)
+```
+There are 5 classes correspond to 5 supported file formats: CDX, CDXML, DOC, DOCX, PerkinELN.
+
+# API
+
+## Molecule
+
+ - Access "scanned" molecules
+
+  ```ruby
+# Molecules - array of scanned molecules
+cdx.molecules
+# Get array of scanned Canonical SMILES
+cdx.molecules.map(&:get_cano_smiles)
+# Get one  molecule
+molecule = cdx.molecules.first
+# Number of scanned molecules
+cdx.molecules.count
+```
+
+- Molecule class: 
+
+```ruby
+# Canonical SMILES
+molecule.get_cano_smiles
+# Molfile
+molecule.get_mdl
+# RDKIT RWMol (https://www.rdkit.org/docs/cppapi/classRDKit_1_1RWMol.html)
+molecule.rw_mol
+# Molecule label (bold text near molecule)
+molecule.label
+# Molecule text (molecule description)
+molecule.text
+# Molecule details (additional information from Perkin Elmer ELN)
+molecule.details
+```
+We are using a [ruby-binding version](https://github.com/CamAnNguyen/rdkit_chem) of `RDKit` as  a dependency of `ChemScanner`.
+
+## Reaction
+
+Reaction consist of 3 groups of molecules: `reactants`, `reagents` and `products`. Each group is and array of molecules, which each element is an object of `Molecule` class. In addition, some abbreviations belong to the reaction are represented by SMILES. Those could be access via `reagent_smiles`
+
+```ruby
+reaction = cdx.reactions.first
+# Access extracted structure group
+reactants = reaction.reactants
+reagents = reaction.reagents
+products = reaction.products
+reagent_smiles = reaction.reagent_smiles
+```
+
+Further manipulation of each group would be similar to `Molecule` class.
+
+ - **Reaction properties**
+
+Reaction itself has `description`, `yield`, `time`, `temperature` and `details` properties. All these properties are extracted from the ChemDraw scheme, excep `details` field are additional information from `PerkinELN`.
+
+ - **Reaction step**
+
+Some multi-step reactions can also be recognized. If a reaction is a multi-step reaction, the "steps" could be accessed via:
+
+```ruby
+# Get first scanned reaction
+reaction = cdx.reactions.first
+# Access first step
+step = reaction.steps.first
+step.number # Should be 1 
+step.description
+step.time
+step.temperature
+# List reagents SMILES
+step.reagents
+```
+
+Each step has these following properties: `description`, `time`, `temperature`, and `reagents`
+
+## Supported File Formats
+
+CDX, CDXML, PerkinELN usage and API are described above. Their outputs are simple `molecules` and `reactions`.
+
+DOC and DOCX classes are little bit different. Since DOC and DOCX file can contain more than 1 embedded ChemDraw schemes, which each embedded scheme is 1 CDX scheme. 
+`ChemScanner` attempts to extract all of them and put into one `Hash` map, called `cdx_map`.
+
+```ruby
+require 'chem_scanner'
+
+doc = ChemScanner::Doc.new
+doc.read('/path/to/doc/file')
+doc.cdx_map.each do |key, cdx|
+  puts cdx.reactions.map(&:reaction_smiles)
+end
+
+# Access all molecules in all CDXs
+doc.molecules.map(&:get_cano_smiles)
+# Access all reactions in all CDXs
+doc.reactions.map(&:get_cano_smiles)
+```
+
+DOCX is a bit different, `ChemScanner` can extract the CDX together with its preview image within the documents.
+
+```ruby
+require 'chem_scanner'
+
+docx = ChemScanner::Docx.new
+docx.read('/path/to/docx/file')
+docx.cdx_map.each do |key, cdx_info|
+  # Get the CDX scheme
+  cdx = cdx_info[:cdx]
+  puts cdx.reactions.map(&:reaction_smiles)
+  
+  # Preview images, used for ChemScanner UI
+  img_ext = cdx_info[:img_ext] # Could be '.png', '.emf'
+  img_b64 = cdx_info[:img_b64] # Base64 encoded of image
+end
+
+# Access all molecules in all CDXs
+docx.molecules.map(&:get_cano_smiles)
+# Access all reactions in all CDXs
+docx.reactions.map(&:get_cano_smiles)
+```
+
+# Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+# Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/CamAnNguyen/chem_scanner. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+# License
+
+The gem is available as open source under the terms of the [GNU AGPLv3 License](https://www.gnu.org/licenses/agpl-3.0.en.html).

data/Rakefile

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "chem_scanner"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/chem_scanner.gemspec

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+lib = File.expand_path("lib", __dir__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require "chem_scanner/version"
+
+# rubocop:disable Metrics/BlockLength
+Gem::Specification.new do |spec|
+  spec.name          = "chem_scanner"
+  spec.version       = ChemScanner::VERSION
+  spec.authors       = ["an.nguyen"]
+  spec.email         = ["an.nguyen@kit.edu"]
+
+  spec.summary       = "Extraction of chemical information"
+  spec.description   = "ChemScanner is a chemical utiliy to extract " \
+                       "chemical information from various scientific formats"
+  spec.homepage      = "https://chemotion.net"
+  spec.license       = "MIT"
+
+  spec.files = Dir.chdir(File.expand_path(__dir__)) do
+    `git ls-files -z`.split("\x0").reject do |f|
+      f.match(%r{^(test|spec|features)/})
+    end
+  end
+
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.required_ruby_version = ">= 2.3"
+
+  spec.add_development_dependency "bundler", ">= 1.16"
+  spec.add_development_dependency "rake", ">= 10.0"
+  spec.add_development_dependency "rspec", ">= 3.0"
+
+  spec.add_dependency "chronic_duration", ">= 0.10"
+  spec.add_dependency "nokogiri", ">= 1.8"
+  spec.add_dependency "rdkit_chem"
+  spec.add_dependency "ruby-geometry", ">= 0.0.6"
+  spec.add_dependency "ruby-ole", ">= 1.2"
+end
+# rubocop:enable Metrics/BlockLength

data/lib/chem_scanner.rb

@@ -0,0 +1,79 @@
+require "yaml"
+require "rdkit_chem"
+require "ostruct"
+require "forwardable"
+
+# ChemScanner main module
+module ChemScanner
+  (
+    Gem.find_files("chem_scanner/extension/*/*.rb") +
+    Gem.find_files("chem_scanner/extension/*.rb") +
+    Gem.find_files("chem_scanner/configuration/*.rb")
+  ).each { |file| require file }
+
+  @superatom = Superatom.instance
+  @abbreviation = Abbreviation.instance
+
+  def self.sync_custom_superatom
+    @superatom.sync_custom
+  end
+
+  def self.all_superatoms
+    @superatom.all
+  end
+
+  def self.predefined_superatoms
+    @superatom.predefined
+  end
+
+  def self.custom_superatoms
+    @superatom.custom
+  end
+
+  def self.get_superatom(superatom)
+    @superatom.get_superatom(superatom)
+  end
+
+  def self.add_superatom(satom, smi)
+    @superatom.add(satom, smi)
+  end
+
+  def self.remove_superatom(satom)
+    @superatom.remove(satom)
+  end
+
+  def self.predefined_abbreviations
+    @abbreviation.predefined
+  end
+
+  def self.solvents
+    @abbreviation.solvents
+  end
+
+  def self.all_abbreviations
+    @abbreviation.all
+  end
+
+  def self.get_abbreviation(abb)
+    if @superatom.get_superatom(abb).empty?
+      @abbreviation.get_abbreviation(abb)
+    else
+      ""
+    end
+  end
+
+  def self.add_abbreviation(abb, smi)
+    @abbreviation.add(abb, smi)
+  end
+
+  def self.add_abbreviation_hash(hash)
+    @abbreviation.add_hash(hash)
+  end
+
+  def self.remove_abbreviation(abb)
+    @abbreviation.remove(abb)
+  end
+end
+
+Gem.find_files("chem_scanner/*.rb").each { |file| require file }
+Gem.find_files("chem_scanner/export/*.rb").each { |file| require file }

data/lib/chem_scanner/cdx.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+# Top module
+module ChemScanner
+  require "chem_scanner/chem_draw/parser"
+  require "chem_scanner/chem_draw/cdx_reader"
+
+  # Class which traverse the tree in CDX binary files and parse
+  class Cdx < ChemDraw::Parser
+    attr_accessor :version
+
+    CREATIONPROGRAM = 0x0003
+    COLORTABLE = 0x0300
+    FONTTABLE = 0x0100
+
+    def initialize
+      super
+
+      @type = "cdx"
+    end
+
+    def read(file, is_path = true)
+      @reader = ChemDraw::CdxReader.new(file, is_path)
+      return false unless @reader.valid
+
+      read_global
+      read_objects until @reader.end?
+      rebuild_objects_map
+
+      @scheme = Interpreter::Scheme.new(self)
+      @scheme.interpret
+
+      @molecules = @scheme.molecules
+      @reactions = @scheme.reactions
+
+      true
+    end
+
+    def raw_data
+      @reader.bin
+    end
+
+    private
+
+    def read_global
+      tag = @reader.read_next until tag == CREATIONPROGRAM
+      @version = @reader.data.split(" ").last
+
+      tag = @reader.read_next until tag == COLORTABLE
+      @color_table = read_colortable(@reader.data, "cdx")
+
+      tag = @reader.read_next until tag == FONTTABLE
+      @font_table = read_fonttable(@reader.data, "cdx")
+    end
+
+    def read_objects
+      tag = @reader.read_next(true)
+
+      while tag.positive?
+        cid = @reader.current_id
+        parse_object(tag, cid)
+
+        tag = @reader.read_next(true)
+      end
+    end
+  end
+end

data/lib/chem_scanner/cdxml.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+# Main module
+module ChemScanner
+  require "nokogiri"
+  require "chem_scanner/chem_draw/parser"
+
+  # Read and Parse CDXML
+  class Cdxml < ChemDraw::Parser
+    attr_accessor :version, :reader
+
+    CDXML_DOCTYPE = "http://www.cambridgesoft.com/xml/cdxml.dtd"
+
+    def initialize
+      super
+
+      @type = "cdxml"
+    end
+
+    def read(file, is_path = true)
+      fs = is_path ? File.open(file) : file
+      @cdxml = Nokogiri::XML(fs)
+      return false if @cdxml.internal_subset.system_id != CDXML_DOCTYPE
+
+      read_global
+
+      @cdxml.xpath("//page").each do |page|
+        @reader = page
+        read_objects
+      end
+
+      rebuild_objects_map
+
+      @scheme = Interpreter::Scheme.new(self)
+      @scheme.interpret
+
+      @molecules = @scheme.molecules
+      @reactions = @scheme.reactions
+
+      true
+    end
+
+    def raw_data
+      @cdxml.to_xml
+    end
+
+    def read_global
+      @version = @cdxml.xpath("//CDXML/@CreationProgram").text.split(" ").last
+
+      ct = @cdxml.xpath("//CDXML/colortable").first
+      @color_table = read_colortable(ct, "cdxml")
+
+      ft = @cdxml.xpath("//CDXML/fonttable").first
+      @font_table = read_fonttable(ft, "cdxml")
+    end
+
+    def read_objects
+      nodes = @reader.element_children
+
+      nodes.each do |node|
+        @reader = node
+        nid = (node.attr("id") || 0).to_i
+
+        if ChemDraw::CDXML_OBJ[node.name] == "Group"
+          read_objects
+        else
+          parse_object(node.name, nid)
+        end
+      end
+    end
+  end
+end