treyja 0.1.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 9ad4ca8e455a01ba5b5c44cf4314011f89e10f7ab48ee3c2ba88c08e7ab31105
4
+ data.tar.gz: c3bb03cf7e18d52ca9e6036903db316e9c7a672cafe346b42875d5f77deec31a
5
+ SHA512:
6
+ metadata.gz: 7220d6f7efb6c80d3ab9bcd2778261b6095cf56dd72601fea2b5e9828456bd5ccf6c1fff843f0feb60e4701f639b459144b8523b6f3afaee0672afa344acab94
7
+ data.tar.gz: 172a497f871142f62aa488e08c5a3d210e0774dd1d5285d49ae8136f8b261235c0c4c058527f4f162b926e18daa1d3c1ba0d89ea4afc86b90eb4573921c8191c
data/.gitignore ADDED
@@ -0,0 +1,16 @@
1
+
2
+ # nix result soft links
3
+ /result
4
+
5
+ # gem
6
+ /.bundle/
7
+ /.yardoc
8
+ /_yardoc/
9
+ /coverage/
10
+ /doc/
11
+ /pkg/
12
+ /spec/reports/
13
+ /tmp/
14
+
15
+ # rspec failure tracking
16
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format documentation
2
+ --color
3
+ --require spec_helper
data/.travis.yml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ sudo: false
3
+ language: ruby
4
+ cache: bundler
5
+ rvm:
6
+ - 2.5.3
7
+ before_install: gem install bundler -v 1.17.1
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at kayhide+xc@gmail.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,6 @@
1
+ source "https://rubygems.org"
2
+
3
+ git_source(:github) {|repo_name| "https://github.com/#{repo_name}" }
4
+
5
+ # Specify your gem's dependencies in treyja.gemspec
6
+ gemspec
data/Gemfile.lock ADDED
@@ -0,0 +1,43 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ treyja (0.1.2)
5
+ google-protobuf (~> 3.6)
6
+
7
+ GEM
8
+ remote: https://rubygems.org/
9
+ specs:
10
+ coderay (1.1.2)
11
+ diff-lcs (1.3)
12
+ google-protobuf (3.7.1)
13
+ method_source (0.9.2)
14
+ pry (0.12.0)
15
+ coderay (~> 1.1.0)
16
+ method_source (~> 0.9.0)
17
+ rake (10.5.0)
18
+ rspec (3.8.0)
19
+ rspec-core (~> 3.8.0)
20
+ rspec-expectations (~> 3.8.0)
21
+ rspec-mocks (~> 3.8.0)
22
+ rspec-core (3.8.0)
23
+ rspec-support (~> 3.8.0)
24
+ rspec-expectations (3.8.3)
25
+ diff-lcs (>= 1.2.0, < 2.0)
26
+ rspec-support (~> 3.8.0)
27
+ rspec-mocks (3.8.0)
28
+ diff-lcs (>= 1.2.0, < 2.0)
29
+ rspec-support (~> 3.8.0)
30
+ rspec-support (3.8.0)
31
+
32
+ PLATFORMS
33
+ ruby
34
+
35
+ DEPENDENCIES
36
+ bundler (~> 1.17)
37
+ pry
38
+ rake (~> 10.0)
39
+ rspec (~> 3.0)
40
+ treyja!
41
+
42
+ BUNDLED WITH
43
+ 1.17.1
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2019 Cross Compass Ltd.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,151 @@
1
+ # Treyja
2
+
3
+ Treyja is a Ruby script which takes `Tensors` protobin and outputs in human readable way.
4
+
5
+ ## Install
6
+
7
+ Make sure you have `ruby` and `gem` commands installed.
8
+
9
+ ```console
10
+ $ ruby --version
11
+ ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin17]
12
+ $ gem --version
13
+ 2.7.6
14
+ ```
15
+
16
+ Using `specific_install` gem, you can install `treyja` gem directly from the directlyGitHub repo.
17
+
18
+ The following command will do it:
19
+
20
+ ```console
21
+ $ gem install specific_install
22
+ $ gem specific_install https://github.com/xc-jp/treyja-ruby.git
23
+ ```
24
+
25
+ You can check if `treyja` is properly installed by:
26
+
27
+ ```console
28
+ % gem list treyja
29
+
30
+ *** LOCAL GEMS ***
31
+
32
+ treyja (0.1.0)
33
+ ```
34
+
35
+
36
+ ## How to use
37
+
38
+ Assume that we are in the root dir of `alderaan` and the path to the project root of treyja-ruby is set to `$TREYJA_RUBY_ROOT`.
39
+
40
+ ### To show in JSON format
41
+
42
+ ```console
43
+ $ draupnir get draupnir/test/fixtures/configuration/literal.yaml | $TREYJA_RUBY_ROOT/bin/treyja json
44
+ {"tensors":[{"dims":[3,2],"dataType":"FLOAT","int32Data":[],"int64Data":[],"uint32Data":[],"uint64Data":[],"floatData":[0.2,0.30000001,1,2,3,5],"doubleData":[]}]}
45
+ {"tensors":[{"dims":[3,2],"dataType":"FLOAT","int32Data":[],"int64Data":[],"uint32Data":[],"uint64Data":[],"floatData":[0,0,0.5,0.60000002,0.69999999,0.80000001],"doubleData":[]}]}
46
+ ```
47
+
48
+ #### Using jq (optional)
49
+
50
+ [jq](https://stedolan.github.io/jq/) command is handy to pretty print JSON.
51
+
52
+ ```console
53
+ $ draupnir get draupnir/test/fixtures/configuration/literal.yaml | $TREYJA_RUBY_ROOT/bin/treyja json | jq
54
+ {
55
+ "tensors": [
56
+ {
57
+ "dims": [
58
+ 3,
59
+ 2
60
+ ],
61
+ "dataType": "FLOAT",
62
+ "int32Data": [],
63
+ "int64Data": [],
64
+ "uint32Data": [],
65
+ "uint64Data": [],
66
+ "floatData": [
67
+ 0.2,
68
+ 0.30000001,
69
+ 1,
70
+ 2,
71
+ 3,
72
+ 5
73
+ ],
74
+ "doubleData": []
75
+ }
76
+ ]
77
+ }
78
+ {
79
+ "tensors": [
80
+ {
81
+ "dims": [
82
+ 3,
83
+ 2
84
+ ],
85
+ "dataType": "FLOAT",
86
+ "int32Data": [],
87
+ "int64Data": [],
88
+ "uint32Data": [],
89
+ "uint64Data": [],
90
+ "floatData": [
91
+ 0,
92
+ 0,
93
+ 0.5,
94
+ 0.60000002,
95
+ 0.69999999,
96
+ 0.80000001
97
+ ],
98
+ "doubleData": []
99
+ }
100
+ ]
101
+ }
102
+ ```
103
+
104
+
105
+ ### To output png image
106
+
107
+ Output png image files into the directory specified by `--output` option.
108
+
109
+ Shape of input tensors must be 3 dimensional and channel count must be 1 (grayscale), 2 (grayscale and alpha), 3 (RGB) or 4 (RGB and alpha).
110
+
111
+ Element type of input tensors must be uint8, int8, float or double.
112
+
113
+ ```console
114
+ $ draupnir get mnist --take 100 | $TREYJA_RUBY_ROOT/bin/treyja image --output tmp/mnist-images
115
+ ```
116
+
117
+ ![mnits-0](https://user-images.githubusercontent.com/1730718/48336250-3b13a100-e6a3-11e8-9f39-558ab792ac01.png)
118
+ ![mnist-1](https://user-images.githubusercontent.com/1730718/48352360-df134180-e6cf-11e8-8de0-7d083315a55d.png)
119
+ ![mnist-2](https://user-images.githubusercontent.com/1730718/48352418-f8b48900-e6cf-11e8-8bf5-0714c152911c.png)
120
+ ![mnist-3](https://user-images.githubusercontent.com/1730718/48352488-21d51980-e6d0-11e8-9af7-907441dfe540.png)
121
+
122
+
123
+ ## Development
124
+
125
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
126
+
127
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
128
+
129
+ ### Setup Ruby
130
+
131
+ If `rake` or `bundle` failed, you may not have `ruby` installed on your system.
132
+
133
+ Setup ruby in your favorite way.
134
+
135
+ With `nix-env`, you can install ruby-2.5.x by:
136
+
137
+ ```console
138
+ $ nix-env -i $(nix-env -qa | grep ruby-2.5)
139
+ ```
140
+
141
+ ## Contributing
142
+
143
+ Bug reports and pull requests are welcome on GitHub at https://github.com/xc-jp/treyja-ruby. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
144
+
145
+ ## License
146
+
147
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
148
+
149
+ ## Code of Conduct
150
+
151
+ Everyone interacting in the Treyja project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/xc-jp/treyja-ruby/blob/master/CODE_OF_CONDUCT.md).
data/Rakefile ADDED
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
data/bin/console ADDED
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "treyja"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start(__FILE__)
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
data/default.nix ADDED
@@ -0,0 +1,53 @@
1
+ { nixpkgs ? null }:
2
+
3
+ let
4
+ src = builtins.fetchTarball {
5
+ # nixpkgs-19.03 as of 2019/03/11.
6
+ url = "https://github.com/NixOS/nixpkgs/archive/50876481a0127ad885fcbfd48ab24bbacbc26395.tar.gz";
7
+ sha256 = "063q2jhi9lf6azbhlrn3cygpaa3n65n3d8g7c1s0vvsj8rxv8b80";
8
+ };
9
+
10
+ pkgs = if isNull nixpkgs then src else nixpkgs;
11
+ in
12
+
13
+ with (import pkgs {});
14
+
15
+ let
16
+ # This needs to be ruby-2.5.
17
+ treyjaRuby = ruby_2_5;
18
+
19
+ gems = bundlerEnv {
20
+ name = "treyja-env";
21
+
22
+ ruby = treyjaRuby;
23
+
24
+ # This specifies the directory with the `Gemfile`, `Gemfile.lock`,
25
+ # and `gemset.nix` file.
26
+ #
27
+ # The `../gemset.nix` file can be regenerated with the
28
+ # following command. This needs to be done whenever the
29
+ # `Gemfile.lock` file changes.
30
+ #
31
+ # $ $(nix-build /some/path/to/nixpkgs -A bundix --no-out-link)/bin/bundix --magic
32
+ gemdir = ./.;
33
+ };
34
+
35
+ in
36
+
37
+ stdenv.mkDerivation {
38
+ name = "treyja-bin";
39
+ src = ./.;
40
+ buildInputs = [ gems.wrappedRuby ];
41
+ nativeBuildInputs = [ makeWrapper ];
42
+ installPhase = ''
43
+ mkdir -p $out
44
+ install -D -m755 $src/bin/treyja $out/bin/treyja
45
+ makeWrapper $src/bin/treyja $out/bin/treyja \
46
+ --set BUNDLE_GEMFILE ${gems.confFiles}/Gemfile \
47
+ --set BUNDLE_PATH ${gems}/${treyjaRuby.gemPath} \
48
+ --set BUNDLE_FROZEN 1 \
49
+ --set GEM_HOME ${gems}/${treyjaRuby.gemPath} \
50
+ --set GEM_PATH ${gems}/${treyjaRuby.gemPath} \
51
+ --set PATH "${gems.wrappedRuby}/bin:$PATH"
52
+ '';
53
+ }
data/exe/treyja ADDED
@@ -0,0 +1,211 @@
1
+ #! /usr/bin/env ruby
2
+
3
+ $LOAD_PATH << File.join(__dir__, "../lib")
4
+ require 'tensors_pb.rb'
5
+
6
+ $unabbrev = {
7
+ 'h' => 'help',
8
+ 'help' => 'help',
9
+ 'output' => 'output',
10
+ 'normalize' => 'normalize',
11
+ 'round' => 'round'
12
+ }
13
+
14
+ $options = [
15
+ *ARGV.map { |k| (ms = k.match(/^--?(\w+)/)) ? [$unabbrev[ms[1]], true] : nil },
16
+ *ARGV.each_cons(2).map { |k, v| (v !~ /^-/ && ms = k.match(/^--?(\w+)/)) ? [$unabbrev[ms[1]], v] : nil }
17
+ ].compact.to_h
18
+
19
+ $command, $file = ARGV.take_while { |s| s !~ /^-/ }
20
+ if $command.nil? || $options["help"]
21
+ puts <<EOS
22
+ Usage: treyja COMMAND [FILE] [OPTIONS]
23
+
24
+ If FILE is not given, it reads from STDIN.
25
+
26
+ Available options:
27
+ -h,--help Show this help text
28
+ --output DIR Directory to output images
29
+ --normalize Normalize floating values (image only)
30
+ --round INT Digit to round floating values (default: 4)
31
+
32
+ Available commands:
33
+ json Output tensors in JSON format
34
+ csv Output tensors in CSV format
35
+ image Create png images and output to DIR
36
+ dump Dump in the format of inner expression
37
+ EOS
38
+ exit
39
+ end
40
+
41
+ $round = $options.fetch('round', 4).to_f
42
+ if $round > 0
43
+ class Float
44
+ def to_s_with_round
45
+ self.round($round).to_s_without_round
46
+ end
47
+ alias_method :to_s_without_round, :to_s
48
+ alias_method :to_s, :to_s_with_round
49
+ end
50
+
51
+ class Double
52
+ def to_s_with_round
53
+ self.round($round).to_s_without_round
54
+ end
55
+ alias_method :to_s_without_round, :to_s
56
+ alias_method :to_s, :to_s_with_round
57
+ end
58
+ end
59
+
60
+ def reader
61
+ io = $file ? open($file) : STDIN
62
+ io.binmode
63
+
64
+ Enumerator.new do |y|
65
+ while magic = io.read(4)
66
+ raise "Incorrect magic bytes" unless magic == 'XCIX'
67
+ length = io.read(8).reverse.unpack("Q").first.to_i
68
+ y << Tensors::TensorsProto.decode(io.read(length))
69
+ end
70
+ end
71
+ end
72
+
73
+
74
+ def write_image tensor, k, file
75
+ width, height, channel = tensor.dims
76
+ width ||= 1
77
+ height ||= 1
78
+ channel ||= 1
79
+ offset = width * height * channel * k
80
+ range = offset ... (width * height * channel + offset)
81
+
82
+ normalizer =
83
+ if $options['normalize']
84
+ -> (xs) {
85
+ min, max = xs.minmax
86
+ xs.map { |f| ((f - min) / (max - min) * 255).to_i }
87
+ }
88
+ else
89
+ -> (xs) { xs.map { |f| (f * 255).to_i } }
90
+ end
91
+
92
+ depth, org_data =
93
+ case tensor.data_type
94
+ when :UINT8
95
+ [8, tensor.byte_data.unpack('C*')[range]]
96
+ when :INT8
97
+ [8, tensor.byte_data.unpack('C*')[range]]
98
+ when :FLOAT
99
+ [8, normalizer.call(tensor.float_data[range])]
100
+ when :DOUBLE
101
+ [8, normalizer.call(tensor.double_data[range])]
102
+ else
103
+ raise "unsupported data type: #{tensor.data_type}"
104
+ end
105
+
106
+ color_type =
107
+ case channel
108
+ when 1
109
+ 0 # grayscale
110
+ when 2
111
+ 4 # grayscale and alpha
112
+ when 3
113
+ 2 # rgb
114
+ when 4
115
+ 6 # rgba
116
+ else
117
+ raise "unsupported channel: #{channel}"
118
+ end
119
+
120
+ raw_data = (0...height).map do |y|
121
+ (0...width).map do |x|
122
+ (0...channel).map do |ch|
123
+ org_data[ch * height * width + y * width + x]
124
+ end
125
+ end
126
+ end
127
+
128
+ def chunk(type, data)
129
+ [data.bytesize, type, data, Zlib.crc32(type + data)].pack("NA4A*N")
130
+ end
131
+
132
+ open(file, 'w') do |io|
133
+ io.print "\x89PNG\r\n\x1a\n"
134
+ io.print chunk("IHDR", [width, height, depth, color_type, 0, 0, 0].pack("NNCCCCC"))
135
+ img_data = raw_data.map {|line| ([0] + line.flatten).pack("C*") }.join
136
+ io.print chunk("IDAT", Zlib::Deflate.deflate(img_data))
137
+ io.print chunk("IEND", "")
138
+ end
139
+ end
140
+
141
+ def dims_to_indices dims
142
+ if dims.empty?
143
+ [[]]
144
+ else
145
+ x = dims.first
146
+ xs = dims[1..-1]
147
+ Enumerator.new do |y|
148
+ dims_to_indices(xs).each do |is|
149
+ x.times.each do |i|
150
+ y << [i, *is]
151
+ end
152
+ end
153
+ end
154
+ end
155
+ end
156
+
157
+ def raw_data tensor
158
+ case tensor.data_type
159
+ when :INT8, :UINT8
160
+ tensor.byte_data.unpack('C*')
161
+ when :FLOAT
162
+ tensor.float_data
163
+ when :DOUBLE
164
+ tensor.double_data
165
+ else
166
+ raise "unsupported data type: #{tensor.data_type}"
167
+ end
168
+ end
169
+
170
+ case $command
171
+ when "dump"
172
+ reader.each do |ts|
173
+ p ts
174
+ STDOUT.flush
175
+ end
176
+ when "json"
177
+ reader.each do |ts|
178
+ puts ts.to_json
179
+ STDOUT.flush
180
+ end
181
+ when "image"
182
+ dir = $options["output"]
183
+ raise "--output option required" unless dir
184
+
185
+ FileUtils.mkdir_p dir
186
+
187
+ require "zlib"
188
+ reader.each_with_index do |tensors, i|
189
+ tensors.tensors.each_with_index do |tensor, j|
190
+ n = tensor.dims.drop(3).inject(1, :*) # Drop width, height and channel and fold the rest
191
+ n.times.each do |k|
192
+ postfix = ([i, j] + (n > 1 ? [k] : [])).map(&:to_s).join('-')
193
+ dst = File.join(dir, "image-#{postfix}.png")
194
+ write_image tensor, k, dst
195
+ end
196
+ end
197
+ end
198
+ when "csv"
199
+ if head = reader.first
200
+ headers = head.tensors.each_with_index.flat_map do |t, i|
201
+ dims_to_indices(t.dims).map do |ix|
202
+ [i.to_s, ix.empty? ? nil : ix.reverse.map(&:to_s).join('_')].compact.join('-')
203
+ end
204
+ end
205
+ puts headers.join(',')
206
+ puts head.tensors.flat_map { |t| raw_data(t) }.map(&:to_s).join(',')
207
+ reader.each do |ts|
208
+ puts ts.tensors.flat_map { |t| raw_data(t) }.map(&:to_s).join(',')
209
+ end
210
+ end
211
+ end
data/gemset.nix ADDED
@@ -0,0 +1,35 @@
1
+ {
2
+ coderay = {
3
+ source = {
4
+ remotes = ["https://rubygems.org"];
5
+ sha256 = "15vav4bhcc2x3jmi3izb11l4d9f3xv8hp2fszb7iqmpsccv1pz4y";
6
+ type = "gem";
7
+ };
8
+ version = "1.1.2";
9
+ };
10
+ google-protobuf = {
11
+ source = {
12
+ remotes = ["https://rubygems.org"];
13
+ sha256 = "134d3ini9ymdwxpz445m28ss9x0m6vcpijcdkzvgk4n538wdmppf";
14
+ type = "gem";
15
+ };
16
+ version = "3.6.1";
17
+ };
18
+ method_source = {
19
+ source = {
20
+ remotes = ["https://rubygems.org"];
21
+ sha256 = "1pviwzvdqd90gn6y7illcdd9adapw8fczml933p5vl739dkvl3lq";
22
+ type = "gem";
23
+ };
24
+ version = "0.9.2";
25
+ };
26
+ pry = {
27
+ dependencies = ["coderay" "method_source"];
28
+ source = {
29
+ remotes = ["https://rubygems.org"];
30
+ sha256 = "1417109nmp7sp8blbdhjx3ckkygm94x1fsfdqn3n7s6dgmc5c35y";
31
+ type = "gem";
32
+ };
33
+ version = "0.12.0";
34
+ };
35
+ }
data/lib/tensor_pb.rb ADDED
@@ -0,0 +1,35 @@
1
+ # Generated by the protocol buffer compiler. DO NOT EDIT!
2
+ # source: tensor.proto
3
+
4
+ require 'google/protobuf'
5
+
6
+ Google::Protobuf::DescriptorPool.generated_pool.build do
7
+ add_message "tensor.TensorProto" do
8
+ repeated :dims, :int64, 1
9
+ optional :data_type, :enum, 2, "tensor.TensorProto.DataType"
10
+ repeated :int32_data, :int32, 3
11
+ repeated :int64_data, :int64, 4
12
+ repeated :uint32_data, :uint32, 5
13
+ repeated :uint64_data, :uint64, 6
14
+ repeated :float_data, :float, 7
15
+ repeated :double_data, :double, 8
16
+ optional :byte_data, :bytes, 9
17
+ end
18
+ add_enum "tensor.TensorProto.DataType" do
19
+ value :INT8, 0
20
+ value :INT16, 1
21
+ value :INT32, 2
22
+ value :INT64, 3
23
+ value :UINT8, 4
24
+ value :UINT16, 5
25
+ value :UINT32, 6
26
+ value :UINT64, 7
27
+ value :FLOAT, 8
28
+ value :DOUBLE, 9
29
+ end
30
+ end
31
+
32
+ module Tensor
33
+ TensorProto = Google::Protobuf::DescriptorPool.generated_pool.lookup("tensor.TensorProto").msgclass
34
+ TensorProto::DataType = Google::Protobuf::DescriptorPool.generated_pool.lookup("tensor.TensorProto.DataType").enummodule
35
+ end
data/lib/tensors_pb.rb ADDED
@@ -0,0 +1,15 @@
1
+ # Generated by the protocol buffer compiler. DO NOT EDIT!
2
+ # source: tensors.proto
3
+
4
+ require 'google/protobuf'
5
+
6
+ require 'tensor_pb'
7
+ Google::Protobuf::DescriptorPool.generated_pool.build do
8
+ add_message "tensors.TensorsProto" do
9
+ repeated :tensors, :message, 1, "tensor.TensorProto"
10
+ end
11
+ end
12
+
13
+ module Tensors
14
+ TensorsProto = Google::Protobuf::DescriptorPool.generated_pool.lookup("tensors.TensorsProto").msgclass
15
+ end
data/lib/treyja.rb ADDED
@@ -0,0 +1,6 @@
1
+ require "treyja/version"
2
+
3
+ module Treyja
4
+ class Error < StandardError; end
5
+ # Your code goes here...
6
+ end
@@ -0,0 +1,3 @@
1
+ module Treyja
2
+ VERSION = "0.1.2"
3
+ end
data/treyja.gemspec ADDED
@@ -0,0 +1,44 @@
1
+
2
+ lib = File.expand_path("../lib", __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require "treyja/version"
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "treyja"
8
+ spec.version = Treyja::VERSION
9
+ spec.authors = ["Cross Compass"]
10
+ spec.email = ["greenia-dev@cross-compass.com"]
11
+
12
+ spec.summary = %q{Treyja: parses and displays tensors.}
13
+ spec.description = %q{Treyja: parses and displays tensors.}
14
+ spec.homepage = "http://github.com/xc-jp/treyja-ruby"
15
+ spec.license = "MIT"
16
+
17
+ # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
18
+ # to allow pushing to a single host or delete this section to allow pushing to any host.
19
+ if spec.respond_to?(:metadata)
20
+ # spec.metadata["allowed_push_host"] = "TODO: Set to 'http://mygemserver.com'"
21
+
22
+ spec.metadata["homepage_uri"] = spec.homepage
23
+ spec.metadata["source_code_uri"] = "http://github.com/xc-jp/treyja-ruby"
24
+ # spec.metadata["changelog_uri"] = "TODO: Put your gem's CHANGELOG.md URL here."
25
+ else
26
+ raise "RubyGems 2.0 or newer is required to protect against " \
27
+ "public gem pushes."
28
+ end
29
+
30
+ # Specify which files should be added to the gem when it is released.
31
+ # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
32
+ spec.files = Dir.chdir(File.expand_path('..', __FILE__)) do
33
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
34
+ end
35
+ spec.bindir = "exe"
36
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
37
+ spec.require_paths = ["lib"]
38
+
39
+ spec.add_runtime_dependency "google-protobuf", "~> 3.6"
40
+ spec.add_development_dependency "bundler", "~> 1.17"
41
+ spec.add_development_dependency "rake", "~> 10.0"
42
+ spec.add_development_dependency "rspec", "~> 3.0"
43
+ spec.add_development_dependency "pry"
44
+ end
data/treyja/.gitignore ADDED
@@ -0,0 +1,11 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+
10
+ # rspec failure tracking
11
+ .rspec_status
metadata ADDED
@@ -0,0 +1,137 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: treyja
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.2
5
+ platform: ruby
6
+ authors:
7
+ - Cross Compass
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2019-04-26 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: google-protobuf
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '3.6'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '3.6'
27
+ - !ruby/object:Gem::Dependency
28
+ name: bundler
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '1.17'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '1.17'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rake
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '10.0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '10.0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '3.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '3.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: pry
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ description: 'Treyja: parses and displays tensors.'
84
+ email:
85
+ - greenia-dev@cross-compass.com
86
+ executables:
87
+ - treyja
88
+ extensions: []
89
+ extra_rdoc_files: []
90
+ files:
91
+ - ".gitignore"
92
+ - ".rspec"
93
+ - ".travis.yml"
94
+ - CODE_OF_CONDUCT.md
95
+ - Gemfile
96
+ - Gemfile.lock
97
+ - LICENSE.txt
98
+ - README.md
99
+ - Rakefile
100
+ - bin/console
101
+ - bin/setup
102
+ - default.nix
103
+ - exe/treyja
104
+ - gemset.nix
105
+ - lib/tensor_pb.rb
106
+ - lib/tensors_pb.rb
107
+ - lib/treyja.rb
108
+ - lib/treyja/version.rb
109
+ - treyja.gemspec
110
+ - treyja/.gitignore
111
+ homepage: http://github.com/xc-jp/treyja-ruby
112
+ licenses:
113
+ - MIT
114
+ metadata:
115
+ homepage_uri: http://github.com/xc-jp/treyja-ruby
116
+ source_code_uri: http://github.com/xc-jp/treyja-ruby
117
+ post_install_message:
118
+ rdoc_options: []
119
+ require_paths:
120
+ - lib
121
+ required_ruby_version: !ruby/object:Gem::Requirement
122
+ requirements:
123
+ - - ">="
124
+ - !ruby/object:Gem::Version
125
+ version: '0'
126
+ required_rubygems_version: !ruby/object:Gem::Requirement
127
+ requirements:
128
+ - - ">="
129
+ - !ruby/object:Gem::Version
130
+ version: '0'
131
+ requirements: []
132
+ rubyforge_project:
133
+ rubygems_version: 2.7.6
134
+ signing_key:
135
+ specification_version: 4
136
+ summary: 'Treyja: parses and displays tensors.'
137
+ test_files: []