ruby-dnn 1.1.4 → 1.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d33f1d5472229d184630d37063d4c42888230044a9e1b069035a144e8aae0964
-  data.tar.gz: 530d31b5fc5073fa02253eb0f7d7b78a007ccaed16ea4a49aba56a87174afb59
+  metadata.gz: 2aa12b717ef532b8afe44de7cb388c7d87cb271bf38a25adaad2c335c8817d4b
+  data.tar.gz: e1322f86b06c11ac3728e948e18469ccc6e454eeefb180108a0a9bfc2dbd8143
 SHA512:
-  metadata.gz: acf104aec7d661da52623930dbd8a60e105aff6b0c7fd4bdc675f507a50f55372451d7c4066992d13a390629d89ec7ba42d7c316376797e439e6b399d702e5aa
-  data.tar.gz: 1e338be765295d9c9827dc82c70e7af79473386be1574ae647ef2568c24be9f9142d94104b62726bda800aff064c37ede794066cfb098a40744dd24a2e7addb4
+  metadata.gz: 6c9c53ca73a5ab7fc53935f37e0804d761e36735aa06972f63e151ce76a44f01878a2c01dfe0622bf39baa5c98b16ee8d06db212356fcc329a9dd40ae2d78f1c
+  data.tar.gz: 5a195c0afd677127afad2433df2fda3a09c711464c46cb174713d00d695026d2b86faa64f82cc96f2b417b468fc372c04efee22761800c900e2f6b0ca05ac57d

data/.gitignore

@@ -6,3 +6,4 @@
 /pkg/
 /spec/reports/
 /tmp/
+*.lock

data/.travis.yml

@@ -1,6 +1,7 @@
 sudo: false
 language: ruby
 rvm:
-  - 2.6.1
+  - 2.6
+  - 2.7
 script:
   - rake test

data/README.md

@@ -1,6 +1,7 @@
 # ruby-dnn
-[![Gem Version](https://badge.fury.io/rb/ruby-dnn.svg)](https://badge.fury.io/rb/ruby-dnn)  
+[![Gem Version](https://badge.fury.io/rb/ruby-dnn.svg)](https://badge.fury.io/rb/ruby-dnn)
 [![Build Status](https://travis-ci.org/unagiootoro/ruby-dnn.svg?branch=master)](https://travis-ci.org/unagiootoro/ruby-dnn)
+[![Docs Latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://rubydoc.info/gems/ruby-dnn)
 
 ruby-dnn is a ruby deep learning library. This library supports full connected neural network and convolution neural network
 and recurrent neural network.
@@ -54,18 +55,18 @@ When create a model with 'define by run' style:
 class MLP < Model
   def initialize
     super
-    @l1 = Dense.new(256)
-    @l2 = Dense.new(256)
-    @l3 = Dense.new(10)
+    @d1 = Dense.new(256)
+    @d2 = Dense.new(256)
+    @d3 = Dense.new(10)
   end
 
   def forward(x)
     x = InputLayer.new(784).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = ReLU.(x)
-    x = @l2.(x)
+    x = @d2.(x)
     x = ReLU.(x)
-    x = @l3.(x)
+    x = @d3.(x)
     x
   end
 end
@@ -84,6 +85,23 @@ puts "loss: #{loss}"
 Please refer to examples for basic usage.  
 If you want to know more detailed information, please refer to the source code.
 
+## Sample
+
+* Pix2pix  
+Convert an abstract image into a building image.  
+[https://github.com/unagiootoro/facade-pix2pix](https://github.com/unagiootoro/facade-pix2pix)
+![facade-pix2pix.png](img/facade-pix2pix.png)
+
+* Cycle-GAN  
+Convert apples to oranges and oranges to apples.  
+[https://github.com/unagiootoro/apple2orange-cyclegan](https://github.com/unagiootoro/apple2orange-cyclegan)
+![cycle-gan.PNG](img/cycle-gan.PNG)
+
+* DQN  
+Learn the game so that the pole on the cart does not fall.  
+[https://github.com/unagiootoro/ruby-rl](https://github.com/unagiootoro/ruby-rl)
+![cart-pole.gif](img/cart-pole.gif)
+
 ## Implemented
 || Implemented classes |
 |:-----------|------------:|
@@ -95,28 +113,27 @@ If you want to know more detailed information, please refer to the source code.
 | Losses | MeanSquaredError, MeanAbsoluteError, Hinge, HuberLoss, SoftmaxCrossEntropy, SigmoidCrossEntropy |
 
 ## Datasets
-By setting the environment variable "RUBY_DNN_DOWNLOADS_PATH", you can specify the path to read dataset.
+By setting the environment variable `RUBY_DNN_DOWNLOADS_PATH`, you can specify the path to read dataset.
 
-● Iris  
-● MNIST  
-● Fashion-MNIST  
-● CIFAR-10  
-● CIFAR-100  
-● STL-10
+* Iris  
+* MNIST  
+* Fashion-MNIST  
+* CIFAR-10  
+* CIFAR-100  
+* STL-10
 
-## Examples
-● VAE  
-● DCGAN  
-● Pix2pix
+## Use GPU
+If you do `require "cumo/narray"` before `require "dnn"`, you can run it on GPU.
+Or, set the environment variable `RUBY_DNN_USE_CUMO` to `ENABLE` to force the GPU to be used.
 
 ## TODO
-● Write a test.  
-● Write a document.  
-● Support to GPU.  
+* Write a test.  
+* Write a document.  
+* Improve performance when using GPU.  
 
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake "spec"` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake "test"` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 

data/examples/api-examples/early_stopping_example.rb

@@ -27,22 +27,22 @@ y_test = DNN::Utils.to_categorical(y_test, 10, Numo::SFloat)
 class MLP < Model
   def initialize
     super
-    @l1 = Dense.new(256)
-    @l2 = Dense.new(256)
-    @l3 = Dense.new(10)
+    @d1 = Dense.new(256)
+    @d2 = Dense.new(256)
+    @d3 = Dense.new(10)
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
   end
 
   def forward(x)
     x = InputLayer.new(784).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = @bn1.(x)
     x = ReLU.(x)
-    x = @l2.(x)
+    x = @d2.(x)
     x = @bn2.(x)
     x = ReLU.(x)
-    x = @l3.(x)
+    x = @d3.(x)
     x
   end
 end

data/examples/api-examples/initializer_example.rb

@@ -28,22 +28,22 @@ class MLP < Model
   def initialize
     super
     # Set the initial values of weight and bias to the initial values of He.
-    @l1 = Dense.new(256, weight_initializer: He.new, bias_initializer: He.new)
-    @l2 = Dense.new(256, weight_initializer: He.new, bias_initializer: He.new)
-    @l3 = Dense.new(10, weight_initializer: He.new, bias_initializer: He.new)
+    @d1 = Dense.new(256, weight_initializer: He.new, bias_initializer: He.new)
+    @d2 = Dense.new(256, weight_initializer: He.new, bias_initializer: He.new)
+    @d3 = Dense.new(10, weight_initializer: He.new, bias_initializer: He.new)
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
   end
 
   def forward(x)
     x = InputLayer.new(784).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = @bn1.(x)
     x = ReLU.(x)
-    x = @l2.(x)
+    x = @d2.(x)
     x = @bn2.(x)
     x = ReLU.(x)
-    x = @l3.(x)
+    x = @d3.(x)
     x
   end
 end

data/examples/api-examples/regularizer_example.rb

@@ -29,22 +29,22 @@ class MLP < Model
   def initialize
     super
     # Set L2 regularizer(weight decay) for weight and bias.
-    @l1 = Dense.new(256, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
-    @l2 = Dense.new(256, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
-    @l3 = Dense.new(10, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
+    @d1 = Dense.new(256, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
+    @d2 = Dense.new(256, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
+    @d3 = Dense.new(10, weight_regularizer: L2.new(L2_LAMBDA), bias_regularizer: L2.new(L2_LAMBDA))
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
   end
 
   def forward(x)
     x = InputLayer.new(784).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = @bn1.(x)
     x = ReLU.(x)
-    x = @l2.(x)
+    x = @d2.(x)
     x = @bn2.(x)
     x = ReLU.(x)
-    x = @l3.(x)
+    x = @d3.(x)
     x
   end
 end

data/examples/api-examples/save_example.rb

@@ -38,22 +38,22 @@ y_test = DNN::Utils.to_categorical(y_test, 10, Numo::SFloat)
 class MLP < Model
   def initialize
     super
-    @l1 = Dense.new(256)
-    @l2 = Dense.new(256)
-    @l3 = Dense.new(10)
+    @d1 = Dense.new(256)
+    @d2 = Dense.new(256)
+    @d3 = Dense.new(10)
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
   end
 
   def forward(x)
     x = InputLayer.new(784).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = @bn1.(x)
     x = ReLU.(x)
-    x = @l2.(x)
+    x = @d2.(x)
     x = @bn2.(x)
     x = ReLU.(x)
-    x = @l3.(x)
+    x = @d3.(x)
     x
   end
 end

data/examples/dcgan/dcgan.rb

@@ -4,13 +4,13 @@ include DNN::Layers
 class Generator < Model
   def initialize
     super
-    @l1 = Dense.new(1024)
-    @l2 = Dense.new(7 * 7 * 64)
-    @l3 = Conv2DTranspose.new(64, 4, strides: 2, padding: true)
-    @l4 = Conv2D.new(64, 4, padding: true)
-    @l5 = Conv2DTranspose.new(32, 4, strides: 2, padding: true)
-    @l6 = Conv2D.new(32, 4, padding: true)
-    @l7 = Conv2D.new(1, 4, padding: true)
+    @d1 = Dense.new(1024)
+    @d2 = Dense.new(7 * 7 * 64)
+    @cv1 = Conv2D.new(64, 4, padding: true)
+    @cvt1 = Conv2DTranspose.new(64, 4, strides: 2, padding: true)
+    @cvt2 = Conv2DTranspose.new(32, 4, strides: 2, padding: true)
+    @cv2 = Conv2D.new(32, 4, padding: true)
+    @cv3 = Conv2D.new(1, 4, padding: true)
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
     @bn3 = BatchNormalization.new
@@ -21,32 +21,32 @@ class Generator < Model
 
   def forward(x)
     x = InputLayer.new(20).(x)
-    x = @l1.(x)
+    x = @d1.(x)
     x = @bn1.(x)
     x = ReLU.(x)
 
-    x = @l2.(x)
+    x = @d2.(x)
     x = @bn2.(x)
     x = ReLU.(x)
 
     x = Reshape.(x, [7, 7, 64])
-    x = @l3.(x)
+    x = @cvt1.(x)
     x = @bn3.(x)
     x = ReLU.(x)
 
-    x = @l4.(x)
+    x = @cv1.(x)
     x = @bn4.(x)
     x = ReLU.(x)
 
-    x = @l5.(x)
+    x = @cvt2.(x)
     x = @bn5.(x)
     x = ReLU.(x)
 
-    x = @l6.(x)
+    x = @cv2.(x)
     x = @bn6.(x)
     x = ReLU.(x)
 
-    x = @l7.(x)
+    x = @cv3.(x)
     x = Tanh.(x)
     x
   end
@@ -55,12 +55,12 @@ end
 class Discriminator < Model
   def initialize
     super
-    @l1 = Conv2D.new(32, 4, strides: 2, padding: true)
-    @l2 = Conv2D.new(32, 4, padding: true)
-    @l3 = Conv2D.new(64, 4, strides: 2, padding: true)
-    @l4 = Conv2D.new(64, 4, padding: true)
-    @l5 = Dense.new(1024)
-    @l6 = Dense.new(1)
+    @cv1 = Conv2D.new(32, 4, strides: 2, padding: true)
+    @cv2 = Conv2D.new(32, 4, padding: true)
+    @cv3 = Conv2D.new(64, 4, strides: 2, padding: true)
+    @cv4 = Conv2D.new(64, 4, padding: true)
+    @d1 = Dense.new(1024)
+    @d2 = Dense.new(1)
     @bn1 = BatchNormalization.new
     @bn2 = BatchNormalization.new
     @bn3 = BatchNormalization.new
@@ -68,26 +68,26 @@ class Discriminator < Model
 
   def forward(x)
     x = InputLayer.new([28, 28, 1]).(x)
-    x = @l1.(x)
+    x = @cv1.(x)
     x = LeakyReLU.(x, 0.2)
 
-    x = @l2.(x)
+    x = @cv2.(x)
     x = @bn1.(x)
     x = LeakyReLU.(x, 0.2)
 
-    x = @l3.(x)
+    x = @cv3.(x)
     x = @bn2.(x)
     x = LeakyReLU.(x, 0.2)
 
-    x = @l4.(x)
+    x = @cv4.(x)
     x = @bn3.(x)
     x = LeakyReLU.(x, 0.2)
 
     x = Flatten.(x)
-    x = @l5.(x)
+    x = @d1.(x)
     x = LeakyReLU.(x, 0.2)
 
-    x = @l6.(x)
+    x = @d2.(x)
     x
   end
 
@@ -129,7 +129,7 @@ class DCGAN < Model
     y_fake = Numo::SFloat.zeros(batch_size, 1)
     @dis.enable_training
     dis_loss = @dis.train_on_batch(x_batch, y_real)
-    dis_loss + @dis.train_on_batch(images, y_fake)
+    dis_loss += @dis.train_on_batch(images, y_fake)
 
     noise = Numo::SFloat.new(batch_size, 20).rand(-1, 1)
     label = Numo::SFloat.cast([1] * batch_size).reshape(batch_size, 1)

data/examples/judge-number/README.md

@@ -0,0 +1,29 @@
+# Prepare
+This example use to sinatra.
+
+```
+$ gem install sinatra
+$ gem install sinatra-contrib
+```
+
+# Let's try
+This example prepared weights that have already been trained.
+If you want to try it immediately, skip steps (1) and (2).
+
+### (1) Training MNIST
+```
+$ ruby mnist_train.rb
+```
+
+### (2) Make weights
+```
+$ ruby make_weights.rb
+```
+
+### (3) Launch sinatra server
+```
+$ ruby server.rb
+```
+
+### (4) Access 127.0.0.1:4567 with your browser
+![](capture.PNG)

data/examples/judge-number/convnet8.rb

@@ -0,0 +1,70 @@
+require "dnn"
+require "numo/linalg/autoloader"
+
+include DNN::Models
+include DNN::Layers
+include DNN::Optimizers
+include DNN::Losses
+
+class ConvNet < Model
+  def self.create(input_shape)
+    convnet = ConvNet.new(input_shape, 32)
+    convnet.setup(Adam.new, SoftmaxCrossEntropy.new)
+    convnet
+  end
+
+  def initialize(input_shape, base_filter_size)
+    super()
+    @input_shape = input_shape
+    @cv1 = Conv2D.new(base_filter_size, 3, padding: true)
+    @cv2 = Conv2D.new(base_filter_size, 3, padding: true)
+    @cv3 = Conv2D.new(base_filter_size * 2, 3, padding: true)
+    @cv4 = Conv2D.new(base_filter_size * 2, 3, padding: true)
+    @cv5 = Conv2D.new(base_filter_size * 4, 3, padding: true)
+    @cv6 = Conv2D.new(base_filter_size * 4, 3, padding: true)
+    @bn1 = BatchNormalization.new
+    @bn2 = BatchNormalization.new
+    @bn3 = BatchNormalization.new
+    @bn4 = BatchNormalization.new
+    @d1 = Dense.new(512)
+    @d2 = Dense.new(10)
+  end
+
+  def forward(x)
+    x = InputLayer.new(@input_shape).(x)
+
+    x = @cv1.(x)
+    x = ReLU.(x)
+    x = Dropout.(x, 0.25)
+
+    x = @cv2.(x)
+    x = @bn1.(x)
+    x = ReLU.(x)
+    x = MaxPool2D.(x, 2)
+
+    x = @cv3.(x)
+    x = ReLU.(x)
+    x = Dropout.(x, 0.25)
+
+    x = @cv4.(x)
+    x = @bn2.(x)
+    x = ReLU.(x)
+    x = MaxPool2D.(x, 2)
+
+    x = @cv5.(x)
+    x = ReLU.(x)
+    x = Dropout.(x, 0.25)
+
+    x = @cv6.(x)
+    x = @bn3.(x)
+    x = ReLU.(x)
+    x = MaxPool2D.(x, 2)
+
+    x = Flatten.(x)
+    x = @d1.(x)
+    x = @bn4.(x)
+    x = ReLU.(x)
+    x = @d2.(x)
+    x
+  end
+end