menoh 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: ffe56775dcbde74f64fdeed4fa59bf6fcc539803
+  data.tar.gz: cc676fe5e5d2f2a2e591125fc6dcdfc9c4054962
+SHA512:
+  metadata.gz: 6b1fb84c56bf469d76f97c86e0a0b0b0a832f146ff3ca52224d4983c429720c6dcb3c0191ca87fc1fb925234ea2d04bda2960a280265e52b7d790e270d73df4d
+  data.tar.gz: 62f09585697c3d536d55fef32f0477b289cb0593786e97ae7820759b87fc749d72446d8434abf23550c626e5737fc5e999c4375a4fa46807c31d85ddeeea3b14

data/.gitignore

@@ -0,0 +1,55 @@
+*.gem
+*.rbc
+/.vagrant/
+/.config
+/coverage/
+/external/
+/InstalledFiles
+/pkg/
+/spec/reports/
+/spec/examples.txt
+/test/tmp/
+/test/version_tmp/
+/tmp/
+
+# Used by dotenv library to load environment variables.
+# .env
+
+## Specific to RubyMotion:
+.dat*
+.repl_history
+build/
+*.bridgesupport
+build-iPhoneOS/
+build-iPhoneSimulator/
+
+## Specific to RubyMotion (use of CocoaPods):
+#
+# We recommend against adding the Pods directory to your .gitignore. However
+# you should judge for yourself, the pros and cons are mentioned at:
+# https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control
+#
+# vendor/Pods/
+
+## Documentation cache and generated files:
+/.yardoc/
+/_yardoc/
+/doc/
+/rdoc/
+
+## Environment normalization:
+/.bundle/
+/vendor/bundle
+/lib/bundler/man/
+
+# for a library or gem, you might want to ignore these files since the code is
+# intended to run in multiple environments; otherwise, check them in:
+# Gemfile.lock
+# .ruby-version
+# .ruby-gemset
+
+# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:
+.rvmrc
+
+*.so
+*.log

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,74 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, gender identity and expression, level of experience,
+nationality, personal appearance, race, religion, or sexual identity and
+orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at miyoshik@preferred.jp. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at [http://contributor-covenant.org/version/1/4][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/4/

data/Dockerfile

@@ -0,0 +1,49 @@
+FROM ubuntu:xenial-20180228
+
+LABEL maintainer "Kunihiko Miyoshi <miyoshik@preferred.jp>"
+LABEL OBJECT="Menoh Ruby Extension Reference Environment"
+
+ENV INSTALL_PREFIX /usr/local
+ENV LD_LIBRARY_PATH ${INSTALL_PREFIX}/lib
+
+RUN apt-get update && apt-get install -y \
+  git \
+  gcc \
+  g++ \
+  cmake \
+  cmake-data \
+  libopencv-dev \
+  protobuf-compiler \
+  libprotobuf-dev \
+  ruby-dev \
+  ruby-rmagick \
+  ruby-bundler && \
+  rm -rf /var/lib/apt/lists/*
+
+# MKL-DNN
+RUN mkdir /opt/mkl-dnn
+WORKDIR /opt/mkl-dnn
+RUN git clone https://github.com/01org/mkl-dnn.git && \
+    cd mkl-dnn/scripts && bash ./prepare_mkl.sh && cd .. && \
+    sed -i 's/add_subdirectory(examples)//g' CMakeLists.txt && \
+    sed -i 's/add_subdirectory(tests)//g' CMakeLists.txt && \
+    mkdir -p build && cd build && cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX .. && make && \
+    make install
+
+# Menoh
+WORKDIR /opt/
+RUN git clone https://github.com/pfnet-research/menoh.git && \
+    cd menoh && \
+    sed -i 's/add_subdirectory(example)//g' CMakeLists.txt && \
+    sed -i 's/add_subdirectory(test)//g' CMakeLists.txt && \
+    mkdir build && \
+    cd build && \
+    cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX .. && \
+    make install
+
+# menoh-ruby
+RUN gem install rake-compiler
+RUN mkdir /opt/menoh-ruby
+ADD . /opt/menoh-ruby
+WORKDIR /opt/menoh-ruby
+RUN rake && rake install

data/Gemfile

@@ -0,0 +1,5 @@
+source 'https://rubygems.org'
+
+git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
+
+gemspec

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2018 Preferred Networks, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,80 @@
+# Menoh Ruby Extension 
+
+This is a Ruby extension of [Menoh](https://github.com/pfnet-research/menoh); an ONNX runtime engine developed by [@okdshin](https://github.com/okdshin) and their team [@pfnet-research](https://github.com/pfnet-research).
+
+## Installation
+
+You need `ruby-dev`, `bundler`, `rake-compiler` to install this extension.
+
+```bash
+$ sudo apt install ruby-dev
+$ sudo gem install bundler
+$ sudo gem install rake-compiler
+```
+
+And add this line to your application's Gemfile:
+
+```ruby
+gem 'menoh'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install menoh
+
+## Usage
+
+Please see [Menoh tutorial](https://github.com/pfnet-research/menoh/blob/master/docs/tutorial.md) and [menoh-ruby tutorial](https://github.com/pfnet-research/menoh-ruby/blob/master/docs/tutorial.md).
+And we have [some examples](https://github.com/pfnet-research/menoh-ruby/blob/master/example/) on this repository.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install ruby dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+### Docker
+
+You can develop on docker. For details, please refer to [Dockerfile](Dockerfile).
+
+```bash
+$ export IMAGE_VERSION=0.0.0 # please specify version
+$ sudo -E docker build -t menoh-ruby:$IMAGE_VERSION `pwd`
+$ sudo -E docker run -it --name menoh-ruby-test -v $(pwd):/opt/menoh-ruby --entrypoint /bin/bash menoh-ruby:$IMAGE_VERSION
+＄ cd /opt/menoh-ruby
+＄ rake && rake install
+
+```
+
+#### attach after stop
+
+```bash
+sudo docker start menoh-ruby-test bash
+sudo docker attach menoh-ruby-test
+```
+
+### Vagrant
+
+You can also set up the development environment by using Vagrant. The details are available on [Vagrantfile](Vagrantfile).
+
+```bash
+$ vagrant up
+$ vagrant ssh
+$ cd /vagrant
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/pfnet-research/menoh-ruby. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+
+## Code of Conduct
+
+Everyone interacting in the OnnxInstant project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/pfnet-research/menoh-ruby/blob/master/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,18 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'test'
+  t.libs << 'lib'
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+require 'rake/extensiontask'
+
+task build: :compile
+
+Rake::ExtensionTask.new('menoh_native') do |ext|
+  ext.lib_dir = 'lib/menoh'
+end
+
+task default: %i[clobber compile test]

data/Vagrantfile

@@ -0,0 +1,46 @@
+# -*- mode: ruby -*-
+# vi: set ft=ruby :
+
+Vagrant.configure('2') do |config|
+  config.vm.box = 'ubuntu/xenial64'
+
+  config.vm.provider 'virtualbox' do |vb|
+    vb.memory = '4096'
+    vb.cpus = 2
+  end
+
+  config.vm.provision 'shell', inline: <<-SHELL
+    export INSTALL_PREFIX=/usr/local
+    sudo apt update
+    sudo apt install -y gcc g++ cmake cmake-data libopencv-dev
+    sudo apt install -y ruby-dev ruby-rmagick
+    sudo gem install bundler rake-compiler
+
+    # protobuf
+    sudo apt install -y protobuf-compiler libprotobuf-dev
+
+    # mkl-dnn
+    cd
+    git clone https://github.com/01org/mkl-dnn.git
+    cd mkl-dnn/scripts && bash ./prepare_mkl.sh && cd ..
+    sed -i 's/add_subdirectory(examples)//g' CMakeLists.txt
+    sed -i 's/add_subdirectory(tests)//g' CMakeLists.txt
+    mkdir -p build && cd build && cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX .. && make
+    sudo make install
+
+    # Menoh
+    cd
+    git clone https://github.com/pfnet-research/menoh.git
+    cd menoh
+    sed -i 's/add_subdirectory(example)//g' CMakeLists.txt
+    sed -i 's/add_subdirectory(test)//g' CMakeLists.txt
+    mkdir -p build && cd build && cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX .. && make
+    sudo make install
+
+    # menoh-ruby
+    export LD_LIBRARY_PATH=${INSTALL_PREFIX}/lib:$LD_LIBRARY_PATH
+    cd /vagrant
+    rake
+    sudo rake install
+  SHELL
+end

data/bin/console

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "menoh"
+
+require "pry"
+Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install

data/docs/tutorial.md

@@ -0,0 +1,121 @@
+# Tutorial
+
+In this tutorial, we are going to make a CNN model inference software.
+This tutorial is based on [Menoh](https://github.com/pfnet-research/menoh)'s [original tutorial](https://github.com/pfnet-research/menoh/blob/master/docs/tutorial.md).
+
+This script loads `data/VGG16.onnx` and takes input image, then outputs classification result.
+
+## Setup model
+
+For gettinig ONNX model's named variables, please refer to [Menoh Tutorial](https://github.com/pfnet-research/menoh/blob/master/docs/tutorial.md).
+
+VGG16 has one input and one output. So now we can check that the input name is *140326425860192* (input of 0:Conv) and the output name is *140326200803680* (output of 39:Softmax).
+
+Some of we are interested the feature vector of input image. So in addition, we are going to take the output of 32:FC(fc6, which is the first FC layer after CNNs) named *140326200777584*.
+
+We define name aliases for convenience:
+
+```ruby
+CONV1_1_IN_NAME = '140326425860192'.freeze
+FC6_OUT_NAME = '140326200777584'.freeze
+SOFTMAX_OUT_NAME = '140326200803680'.freeze
+```
+
+To build model, we load model data from ONNX file:
+
+```ruby
+onnx_obj = Menoh::Menoh.new './data/VGG16.onnx'
+```
+
+Now let's build the model.
+
+```ruby
+# data shape of input images
+input_shape = {
+  channel_num: 3,
+  width: 224,
+  height: 224
+}
+# model options for model
+model_opt = {
+  backend: 'mkldnn',
+  input_layers: [
+    {
+      name: CONV1_1_IN_NAME,
+      dims: [
+        image_list.length,
+        input_shape[:channel_num],
+        input_shape[:width],
+        input_shape[:height]
+      ]
+    }
+  ],
+  output_layers: [FC6_OUT_NAME, SOFTMAX_OUT_NAME]
+}
+# make model for inference under 'model_opt'
+model = onnx_obj.make_model model_opt
+```
+
+## Preprocessing dataset
+
+Before running the inference, the preprocessing of input dataset is required. `data/VGG16.onnx` takes 3 channels 224 x 224 sized image but input image is not always sized 224x224. So we use Imagemagick's `resize_to_fill` method for resizing.
+
+`VGG16.onnx`'s input layer *140326425860192* takes images as NCHW format (N x Channels x Height x Width). But RMagick's image array has alternately flatten values for each channel. So next we call `export_pixels` method for each channels `['B', 'G', 'R']`, then `flatten` arrays.
+
+```ruby
+image_list = [
+  './data/Light_sussex_hen.jpg',
+  './data/honda_nsx.jpg',
+]
+image_set = [
+  {
+    name: CONV1_1_IN_NAME,
+    data: image_list.map do |image_filepath|
+      image = Magick::Image.read(image_filepath).first
+      image = image.resize_to_fill(input_shape[:width], input_shape[:height])
+      'BGR'.split('').map do |color|
+        image.export_pixels(0, 0, image.columns, image.rows, color).map { |pix| pix / 256 }
+      end.flatten
+    end.flatten
+  }
+]
+```
+
+In current case, the range of pixel value `data/VGG16.onnx` taking is [0, 256]. On the other hand RMagick's image array takes [0, 65536]. So we have to scale the values by dividing 256.
+
+And sometimes model takes values scaled in range [0, 1] or something. In that case, we can scale values here:
+
+```ruby
+image_set = [
+  {
+    name: CONV1_1_IN_NAME,
+    data: image_list.map do |image_filepath|
+      image = Magick::Image.read(image_filepath).first
+      image = image.resize_to_fill(input_shape[:width], input_shape[:height])
+      'BGR'.split('').map do |color|
+        image.export_pixels(0, 0, image.columns, image.rows, color).map { |pix| pix / 65536 }
+      end.flatten
+    end.flatten
+  }
+]
+```
+
+## Run inference and get results
+
+Now we can run the inference.
+
+```ruby
+# execute inference
+inferenced_results = model.run image_set
+```
+
+The `inferenced_results` is the array that contains the hash of results of `output_layers`. So you can get each value as follows.
+
+```ruby
+fc6_out = inferenced_results.find { |x| x[:name] == FC6_OUT_NAME }
+softmax_out = inferenced_results.find { |x| x[:name] == SOFTMAX_OUT_NAME }
+```
+
+That's it.
+
+The full code is available at [VGG16 example](https://github.com/pfnet-research/menoh-ruby/blob/master/example/example_vgg16.rb).