red-arrow 0.8.2 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: d5c8c88ef78b27019b54dca4de8c914f17b80d9e
-  data.tar.gz: 85a527d72ea710c9f4c55e4c2db36110693a9f2c
+SHA256:
+  metadata.gz: e7ad82aea6e936a77e584a46b230a543f6f790a0e340e8587e82db6c17a07161
+  data.tar.gz: 78500925eb85fc529e3293e3690a3a5aaf19d4208354729174e09b1adfa45988
 SHA512:
-  metadata.gz: 7b4635c2517a7b750c7f46341bd659504e4d2cbbba6f1d2ef4f32c47c1870de891a7bf009fcd4783284f72b515eaeb2e4846121bf0b66e8eeb017d9ab5f1727c
-  data.tar.gz: 21a10ffe298a91ec56b5354bcc6693348fdb815db04861fc04124849b111ffadc5ddbac2582cf6d87d8f96ec2060179fb2e19866f8f7be2699d547a2e2d861dc
+  metadata.gz: 5947773a60fe43c92451aa53451212cc3dd3c1fb0f29206fcf0e39c5ca8f83a42314970a42c96dbbdef2d34635f5b0cb89edf6b208b4d0de72079a4628ef9179
+  data.tar.gz: 660eaaf51764cfe5f2d05616ea1b6db77a9fb293f1716cfbaa0e7cf1d812a8d98381740ea378ebc222f1524fb7c18bebcc187c808de4b45244121e1675119a02

data/Gemfile

@@ -1,18 +1,21 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 source "https://rubygems.org/"
 

data/NOTICE.txt

@@ -0,0 +1,2 @@
+Apache Arrow
+Copyright 2016 The Apache Software Foundation

data/README.md

@@ -1,28 +1,43 @@
-# README
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
 
-## Name
+    http://www.apache.org/licenses/LICENSE-2.0
 
-Red Arrow
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
 
-## Description
+# Red Arrow - Apache Arrow Ruby
 
-Red Arrow is a Ruby bindings of Apache Arrow. Red Arrow is based on GObject Introspection.
+Red Arrow is the Ruby bindings of Apache Arrow. Red Arrow is based on GObject Introspection.
 
 [Apache Arrow](https://arrow.apache.org/) is an in-memory columnar data store. It's used by many products for data analytics.
 
 [GObject Introspection](https://wiki.gnome.org/action/show/Projects/GObjectIntrospection) is a middleware for language bindings of C library. GObject Introspection can generate language bindings automatically at runtime.
 
-Red Arrow uses [Arrow GLib](https://github.com/apache/arrow/tree/master/c_glib) and [gobject-introspection gem](https://rubygems.org/gems/gobject-introspection) to generate Ruby bindings of Apache Arrow.
+Red Arrow uses [Apache Arrow GLib](https://github.com/apache/arrow/tree/master/c_glib) and [gobject-introspection gem](https://rubygems.org/gems/gobject-introspection) to generate Ruby bindings of Apache Arrow.
 
-Arrow GLib is a C wrapper for [Arrow C++](https://github.com/apache/arrow/tree/master/cpp). GObject Introspection can't use Arrow C++ directly. Arrow GLib is a bridge between Arrow C++ and GObject Introspection.
+Apache Arrow GLib is a C wrapper for [Apache Arrow C++](https://github.com/apache/arrow/tree/master/cpp). GObject Introspection can't use Apache Arrow C++ directly. Apache Arrow GLib is a bridge between Apache Arrow C++ and GObject Introspection.
 
 gobject-introspection gem is a Ruby bindings of GObject Introspection. Red Arrow uses GObject Introspection via gobject-introspection gem.
 
 ## Install
 
-Install Arrow GLib before install Red Arrow. Use [packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org) for installing Arrow GLib.
+Install Apache Arrow GLib before install Red Arrow. Use [packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org) for installing Apache Arrow GLib.
 
-Install Red Arrow after you install Arrow GLib:
+Note that the Apache Arrow GLib packages are "unofficial". "Official" packages will be released in the future.
+
+Install Red Arrow after you install Apache Arrow GLib:
 
 ```text
 % gem install red-arrow
@@ -33,25 +48,7 @@ Install Red Arrow after you install Arrow GLib:
 ```ruby
 require "arrow"
 
-# TODO
+table = Arrow::Table.load("/dev/shm/data.arrow")
+# Process data in table
+table.save("/dev/shm/data-processed.arrow")
 ```
-
-## Dependencies
-
-* [Apache Arrow](https://arrow.apache.org/)
-
-* [Arrow GLib](https://github.com/apache/arrow/tree/master/c_glib)
-
-* [gobject-introspection gem](https://rubygems.org/gems/gobject-introspection)
-
-## Authors
-
-* Kouhei Sutou \<kou@clear-code.com\>
-
-## License
-
-Apache License 2.0. See `doc/text/apache-2.0.txt` and `NOTICE` for
-details.
-
-(Kouhei Sutou has a right to change the license including contributed
-patches.)

data/Rakefile

@@ -1,24 +1,26 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "rubygems"
 require "bundler/gem_helper"
-require "packnga"
 
-base_dir = File.join(File.dirname(__FILE__))
+base_dir = File.join(__dir__)
 
 helper = Bundler::GemHelper.new(base_dir)
 def helper.version_tag
@@ -28,14 +30,6 @@ end
 helper.install
 spec = helper.gemspec
 
-Packnga::DocumentTask.new(spec) do |task|
-  task.original_language = "en"
-  task.translate_language = "ja"
-end
-
-Packnga::ReleaseTask.new(spec) do
-end
-
 desc "Run tests"
 task :test do
   cd("dependency-check") do

data/dependency-check/Rakefile

@@ -1,18 +1,21 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "pkg-config"
 require "native-package-installer"
@@ -30,7 +33,7 @@ end
 namespace :dependency do
   desc "Check dependency"
   task :check do
-    unless PKGConfig.check_version?("arrow-glib", 0, 8, 0)
+    unless PKGConfig.check_version?("arrow-glib", 0, 9, 0)
       unless NativePackageInstaller.install(:debian => "libarrow-glib-dev",
                                             :redhat => "arrow-glib-devel")
         exit(false)

data/doc/text/development.md

@@ -1,3 +1,22 @@
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
+
 # Development
 
 ## Naming convention

data/lib/arrow.rb

@@ -1,16 +1,19 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "gobject-introspection"
 

data/lib/arrow/array-builder.rb

@@ -1,16 +1,19 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "date"
 
@@ -35,7 +38,7 @@ module Arrow
           when Float
             return DoubleArray.new(values)
           when Integer
-            if value.negative?
+            if value < 0
               builder = IntArrayBuilder.new
               return builder.build(values)
             else
@@ -63,62 +66,59 @@ module Arrow
 
     def build(values)
       value_convertable = respond_to?(:convert_to_arrow_value, true)
-      if respond_to?(:append_values)
-        start_index = 0
-        current_index = 0
-        status = :value
-        values.each do |value|
-          if value.nil?
-            if status == :value
-              if start_index != current_index
-                target_values = values[start_index...current_index]
-                if value_convertable
-                  target_values = target_values.collect do |v|
-                    convert_to_arrow_value(v)
-                  end
+      start_index = 0
+      current_index = 0
+      status = :value
+
+      values.each do |value|
+        if value.nil?
+          if status == :value
+            if start_index != current_index
+              target_values = values[start_index...current_index]
+              if value_convertable
+                target_values = target_values.collect do |v|
+                  convert_to_arrow_value(v)
                 end
-                append_values(target_values)
-                start_index = current_index
               end
-              status = :null
-            end
-          else
-            if status == :null
-              append_nulls(current_index - start_index)
+              append_values(target_values, nil)
               start_index = current_index
-              status = :value
             end
+            status = :null
           end
-          current_index += 1
-        end
-        if start_index != current_index
-          if status == :value
-            if start_index == 0 and current_index == values.size
-              target_values = values
-            else
-              target_values = values[start_index...current_index]
-            end
-            if value_convertable
-              target_values = target_values.collect do |v|
-                convert_to_arrow_value(v)
-              end
-            end
-            append_values(target_values)
-          else
+        else
+          if status == :null
             append_nulls(current_index - start_index)
+            start_index = current_index
+            status = :value
           end
         end
-      else
-        values.each do |value|
-          if value.nil?
-            append_null
+        current_index += 1
+      end
+      if start_index != current_index
+        if status == :value
+          if start_index == 0 and current_index == values.size
+            target_values = values
           else
-            value = convert_to_arrow_value(value) if value_convertable
-            append(value)
+            target_values = values[start_index...current_index]
+          end
+          if value_convertable
+            target_values = target_values.collect do |v|
+              convert_to_arrow_value(v)
+            end
           end
+          append_values(target_values, nil)
+        else
+          append_nulls(current_index - start_index)
         end
       end
+
       finish
     end
+
+    def append_nulls(n)
+      n.times do
+        append_null
+      end
+    end
   end
 end

data/lib/arrow/array.rb

@@ -1,16 +1,19 @@
-# Copyright 2017-2018 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 module Arrow
   class Array
@@ -29,6 +32,7 @@ module Arrow
     end
 
     def [](i)
+      i += length if i < 0
       if null?(i)
         nil
       else