torch-rb 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fdc46d7eba97851841de58e63b35badbd579ce1bc2a3e311998db03956cb84ad
-  data.tar.gz: 944520049bd683d303e4587d18394eeb13102aa9f4addeca87875ee214806de4
+  metadata.gz: '09221364dad232f1b76129fe9dc9407675cc2afbd03bd1339c736d4eec752df7'
+  data.tar.gz: a37a0584aed809009ebd74e7c0da9430481ccabf1ac915d2468ee7511c249588
 SHA512:
-  metadata.gz: 9026cbd9b0cdf286e79e7b296f2fae95712bd553c2cd58a34044feeed482f1ed2630b7b1e852a82e87d1011478b7e24d442d634f07de67b2c0a6a8e1cf6ade9a
-  data.tar.gz: 193e25fc5065d42160b0584f3e53512606c1b4329c6d872b90a972d7c4d46a4be81eef8c709e3060339c96758fe504dc8618da9e69ff411d9351d528983da72b
+  metadata.gz: c220d35971b9ce3e5a7a80f6a5d1ae4324f3524d0e0171c680deced66fe2f29342a46eecb1a4447d84a401a677c7bb1ef910a0c7ee6c925ea4b578b7e5712772
+  data.tar.gz: 807fe2907de1caac92da6dddb0154b7971dda3aa0ee2c53f6b3046732f4bf3c02310e59a6441efa0fdf1ad3d0ddb5dcd7a3c1a946adbfbea73b6b30f10a71487

data/CHANGELOG.md

@@ -1,3 +1,25 @@
+## 0.9.0 (2021-10-23)
+
+- Updated LibTorch to 1.10.0
+- Added `real` and `imag` methods to tensors
+
+## 0.8.3 (2021-10-17)
+
+- Fixed `dup` method for tensors and parameters
+- Fixed issues with transformers
+
+## 0.8.2 (2021-10-03)
+
+- Added transformers
+- Added left shift and right shift
+
+## 0.8.1 (2021-06-15)
+
+- Added `Backends` module
+- Added `FFT` module
+- Added `Linalg` module
+- Added `Special` module
+
 ## 0.8.0 (2021-06-15)
 
 - Updated LibTorch to 1.9.0

data/README.md

@@ -28,15 +28,19 @@ It can take a few minutes to compile the extension.
 
 ## Getting Started
 
-Deep learning is significantly faster with a GPU. If you don’t have an NVIDIA GPU, we recommend using a cloud service. [Paperspace](https://www.paperspace.com/) has a great free plan.
+A good place to start is [Deep Learning with Torch.rb: A 60 Minute Blitz](tutorials/blitz/README.md).
 
-We’ve put together a [Docker image](https://github.com/ankane/ml-stack) to make it easy to get started. On Paperspace, create a notebook with a custom container. Under advanced options, set the container name to:
+## Tutorials
 
-```text
-ankane/ml-stack:torch-gpu
-```
+- [Transfer learning](tutorials/transfer_learning/README.md)
+- [Sequence models](tutorials/nlp/sequence_models.md)
+- [Word embeddings](tutorials/nlp/word_embeddings.md)
 
-And leave the other fields in that section blank. Once the notebook is running, you can run the [MNIST example](https://github.com/ankane/ml-stack/blob/master/torch-gpu/MNIST.ipynb).
+## Examples
+
+- [Image classification with MNIST](examples/mnist) ([日本語版](https://qiita.com/kojix2/items/c19c36dc1bf73ea93409))
+- [Collaborative filtering with MovieLens](examples/movielens)
+- [Generative adversarial networks](examples/gan)
 
 ## API
 
@@ -48,7 +52,7 @@ This library follows the [PyTorch API](https://pytorch.org/docs/stable/torch.htm
 
 You can follow PyTorch tutorials and convert the code to Ruby in many cases. Feel free to open an issue if you run into problems.
 
-## Tutorial
+## Overview
 
 Some examples below are from [Deep Learning with PyTorch: A 60 Minutes Blitz](https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html)
 
@@ -214,7 +218,7 @@ Define a neural network
 ```ruby
 class MyNet < Torch::NN::Module
   def initialize
-    super
+    super()
     @conv1 = Torch::NN::Conv2d.new(1, 6, 3)
     @conv2 = Torch::NN::Conv2d.new(6, 16, 3)
     @fc1 = Torch::NN::Linear.new(16 * 6 * 6, 120)
@@ -225,20 +229,10 @@ class MyNet < Torch::NN::Module
   def forward(x)
     x = Torch::NN::F.max_pool2d(Torch::NN::F.relu(@conv1.call(x)), [2, 2])
     x = Torch::NN::F.max_pool2d(Torch::NN::F.relu(@conv2.call(x)), 2)
-    x = x.view(-1, num_flat_features(x))
+    x = Torch.flatten(x, 1)
     x = Torch::NN::F.relu(@fc1.call(x))
     x = Torch::NN::F.relu(@fc2.call(x))
-    x = @fc3.call(x)
-    x
-  end
-
-  def num_flat_features(x)
-    size = x.size[1..-1]
-    num_features = 1
-    size.each do |s|
-      num_features *= s
-    end
-    num_features
+    @fc3.call(x)
   end
 end
 ```
@@ -402,19 +396,9 @@ Here’s a list of functions to create tensors (descriptions from the [C++ docs]
   Torch.zeros(3) # tensor([0, 0, 0])
   ```
 
-## Examples
-
-Here are a few full examples:
-
-- [Image classification with MNIST](examples/mnist) ([日本語版](https://qiita.com/kojix2/items/c19c36dc1bf73ea93409))
-- [Collaborative filtering with MovieLens](examples/movielens)
-- [Sequence models and word embeddings](examples/nlp)
-- [Generative adversarial networks](examples/gan)
-- [Transfer learning](examples/transfer-learning)
-
 ## LibTorch Installation
 
-[Download LibTorch](https://pytorch.org/). For Linux, use the `cxx11 ABI` version. Then run:
+[Download LibTorch](https://pytorch.org/) (for Linux, use the `cxx11 ABI` version). Then run:
 
 ```sh
 bundle config build.torch-rb --with-torch-dir=/path/to/libtorch
@@ -424,7 +408,8 @@ Here’s the list of compatible versions.
 
 Torch.rb | LibTorch
 --- | ---
-0.8.0+ | 1.9.0+
+0.9.0+ | 1.10.0+
+0.8.0-0.8.3 | 1.9.0-1.9.1
 0.6.0-0.7.0 | 1.8.0-1.8.1
 0.5.0-0.5.3 | 1.7.0-1.7.1
 0.3.0-0.4.2 | 1.6.0
@@ -444,9 +429,7 @@ Then install the gem (no need for `bundle config`).
 
 ## Performance
 
-### Linux
-
-Deep learning is significantly faster on a GPU. Install [CUDA](https://developer.nvidia.com/cuda-downloads) and [cuDNN](https://developer.nvidia.com/cudnn) and reinstall the gem.
+Deep learning is significantly faster on a GPU. With Linux, install [CUDA](https://developer.nvidia.com/cuda-downloads) and [cuDNN](https://developer.nvidia.com/cudnn) and reinstall the gem.
 
 Check if CUDA is available
 
@@ -460,15 +443,14 @@ Move a neural network to a GPU
 net.cuda
 ```
 
-## rbenv
+If you don’t have a GPU that supports CUDA, we recommend using a cloud service. [Paperspace](https://www.paperspace.com/) has a great free plan. We’ve put together a [Docker image](https://github.com/ankane/ml-stack) to make it easy to get started. On Paperspace, create a notebook with a custom container. Under advanced options, set the container name to:
 
-This library uses [Rice](https://github.com/jasonroelofs/rice) to interface with LibTorch. Rice and earlier versions of rbenv don’t play nicely together. If you encounter an error during installation, upgrade ruby-build and reinstall your Ruby version.
-
-```sh
-brew upgrade ruby-build
-rbenv install [version]
+```text
+ankane/ml-stack:torch-gpu
 ```
 
+And leave the other fields in that section blank. Once the notebook is running, you can run the [MNIST example](https://github.com/ankane/ml-stack/blob/master/torch-gpu/MNIST.ipynb).
+
 ## History
 
 View the [changelog](https://github.com/ankane/torch.rb/blob/master/CHANGELOG.md)

data/codegen/generate_functions.rb

@@ -11,6 +11,9 @@ def generate_functions
   generate_files("torch", :define_singleton_method, functions[:torch])
   generate_files("tensor", :define_method, functions[:tensor])
   generate_files("nn", :define_singleton_method, functions[:nn])
+  generate_files("fft", :define_singleton_method, functions[:fft])
+  generate_files("linalg", :define_singleton_method, functions[:linalg])
+  generate_files("special", :define_singleton_method, functions[:special])
 end
 
 def load_functions
@@ -20,11 +23,14 @@ end
 
 def skip_functions(functions)
   functions.reject do |f|
-    f.base_name.start_with?("_") ||
+    (f.base_name.start_with?("_") && f.base_name != "__lshift__" && f.base_name != "__rshift__") ||
     f.base_name.include?("_backward") ||
     f.base_name.include?("_forward") ||
     f.base_name == "to" ||
     f.base_name == "record_stream" ||
+    f.base_name == "is_pinned" ||
+    f.base_name == "pin_memory" ||
+    f.base_name == "fused_moving_avg_obs_fake_quant" ||
     # in ext.cpp
     f.base_name == "index" ||
     f.base_name == "index_put_" ||
@@ -38,10 +44,26 @@ end
 
 def group_functions(functions)
   nn_functions, other_functions = functions.partition { |f| f.python_module == "nn" }
+  linalg_functions, other_functions = other_functions.partition { |f| f.python_module == "linalg" }
+  fft_functions, other_functions = other_functions.partition { |f| f.python_module == "fft" }
+  special_functions, other_functions = other_functions.partition { |f| f.python_module == "special" }
+  unexpected_functions, other_functions = other_functions.partition { |f| f.python_module }
   torch_functions = other_functions.select { |f| f.variants.include?("function") }
   tensor_functions = other_functions.select { |f| f.variants.include?("method") }
 
-  {torch: torch_functions, tensor: tensor_functions, nn: nn_functions}
+  if unexpected_functions.any?
+    unexpected_modules = unexpected_functions.map(&:python_module).uniq
+    raise "Unexpected modules: #{unexpected_modules.join(", ")}"
+  end
+
+  {
+    torch: torch_functions,
+    tensor: tensor_functions,
+    nn: nn_functions,
+    linalg: linalg_functions,
+    fft: fft_functions,
+    special: special_functions
+  }
 end
 
 def generate_files(type, def_method, functions)
@@ -111,11 +133,15 @@ def generate_attach_def(name, type, def_method)
     end
 
   ruby_name = "_#{ruby_name}" if ["size", "stride", "random!", "stft"].include?(ruby_name)
+  ruby_name = ruby_name.sub(/\Afft_/, "") if type == "fft"
+  ruby_name = ruby_name.sub(/\Alinalg_/, "") if type == "linalg"
+  ruby_name = ruby_name.sub(/\Aspecial_/, "") if type == "special"
+  ruby_name = name if name.start_with?("__")
 
   # cast for Ruby < 2.7 https://github.com/thisMagpie/fftw/issues/22#issuecomment-49508900
   cast = RUBY_VERSION.to_f > 2.7 ? "" : "(VALUE (*)(...)) "
 
-  "rb_#{def_method}(m, \"#{ruby_name}\", #{cast}#{type}_#{name}, -1);"
+  "rb_#{def_method}(m, \"#{ruby_name}\", #{cast}#{full_name(name, type)}, -1);"
 end
 
 def generate_method_def(name, functions, type, def_method)
@@ -128,7 +154,7 @@ def generate_method_def(name, functions, type, def_method)
 
   template = <<~EOS
     // #{name}
-    static VALUE #{type}_#{name}(int argc, VALUE* argv, VALUE self_)
+    static VALUE #{full_name(name, type)}(int argc, VALUE* argv, VALUE self_)
     {
       HANDLE_TH_ERRORS#{assign_self}
       static RubyArgParser parser({
@@ -364,6 +390,8 @@ def generate_function_params(function, params, remove_self)
           end
         when "generator", "tensorlist", "intlist"
           func
+        when "string"
+          "stringViewOptional"
         else
           "#{func}Optional"
         end
@@ -401,9 +429,7 @@ def generate_dispatch_params(function, params)
           if function.out?
             "const Tensor &"
           else
-            # TODO
-            # "const c10::optional<at::Tensor> &"
-            "const OptionalTensor &"
+            "const c10::optional<at::Tensor> &"
           end
         elsif param[:modifier]
           if param[:modifier].include?("!") && function.retvals.size > 1
@@ -427,7 +453,11 @@ def generate_dispatch_params(function, params)
       when "float[]"
         "ArrayRef<double>"
       when "str"
-        "std::string"
+        if param[:optional]
+          "c10::string_view"
+        else
+          "std::string"
+        end
       when "Scalar", "bool", "ScalarType", "Layout", "Device", "Storage", "Generator", "MemoryFormat", "Storage"
         param[:type]
       else
@@ -560,3 +590,11 @@ def signature_type(param)
   type += "?" if param[:optional]
   type
 end
+
+def full_name(name, type)
+  if %w(fft linalg special).include?(type) && name.start_with?("#{type}_")
+    name
+  else
+    "#{type}_#{name}"
+  end
+end