red-parquet 0.0.2 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: f182f603bf41836b43670bbb064b30570ed88270
-  data.tar.gz: 3707be9a6e4415441e91a5bb200225487c41123f
+SHA256:
+  metadata.gz: bc5d46efc9fc652cd9c8e52345d3be7a17fe5bf22629efb0b5710baa678e8168
+  data.tar.gz: 185c1aadebfa2f0c38224fb2b9a925856736ea13f2084e6c6240c8e8c184a892
 SHA512:
-  metadata.gz: 977fa271681371c5e0a827ac55287d7dd3431649f417b61993852fcac0fadda05c97dc9283933be993b98f3c36c3486f457e292b7f35fcccc25ce1d8f3b1ab89
-  data.tar.gz: 9e44068de2e857804b6ea3d3d25e0c1f2c17c920bc26271092818f53bb61e88c0f70775a866b8f0d31503a17f40e1e549bbbe322b94a45eb408ba4875b1b886d
+  metadata.gz: 653075c4bccdf6c800d0ff52e8a37fcaf4477b59dda7dcc1b1a291f91536e0731892afda623c5c74ebf5e7358f8621a822266cc534c26e3c2ccd9a0751f5cc5f
+  data.tar.gz: ff57c715fd2d25cfe08f2fd6eae5f8a05d3a6a291f1e8ed505591d971768ff260cac825fccb767ab04193facb7154b6b137b9a30fea6449ffce53c5aecc638b9

data/Gemfile

@@ -1,18 +1,21 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 source "https://rubygems.org/"
 

data/NOTICE.txt

@@ -0,0 +1,2 @@
+Apache Arrow
+Copyright 2016 The Apache Software Foundation

data/README.md

@@ -1,28 +1,43 @@
-# README
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one
+  or more contributor license agreements.  See the NOTICE file
+  distributed with this work for additional information
+  regarding copyright ownership.  The ASF licenses this file
+  to you under the Apache License, Version 2.0 (the
+  "License"); you may not use this file except in compliance
+  with the License.  You may obtain a copy of the License at
 
-## Name
+    http://www.apache.org/licenses/LICENSE-2.0
 
-Red Parquet
+  Unless required by applicable law or agreed to in writing,
+  software distributed under the License is distributed on an
+  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+  KIND, either express or implied.  See the License for the
+  specific language governing permissions and limitations
+  under the License.
+-->
 
-## Description
+# Red Parquet - Apache Parquet Ruby
 
-Red Parquet is a Ruby bindings of Apache Parquet. Red Parquet is based on GObject Introspection.
+Red Parquet is the Ruby bindings of Apache Parquet. Red Parquet is based on GObject Introspection.
 
 [Apache Parquet](https://parquet.apache.org/) is a columnar storage format.
 
 [GObject Introspection](https://wiki.gnome.org/action/show/Projects/GObjectIntrospection) is a middleware for language bindings of C library. GObject Introspection can generate language bindings automatically at runtime.
 
-Red Parquet uses [Parquet GLib](https://github.com/red-data-tools/parquet-glib) and [gobject-introspection gem](https://rubygems.org/gems/gobject-introspection) to generate Ruby bindings of Apache Parquet.
+Red Parquet uses [Apache Parquet GLib](https://github.com/apache/arrow/tree/master/c_glib/parquet-glib) and [gobject-introspection gem](https://rubygems.org/gems/gobject-introspection) to generate Ruby bindings of Apache Parquet.
 
-Parquet GLib is a C wrapper for [Parquet C++](https://github.com/apache/parquet-cpp). GObject Introspection can't use Parquet C++ directly. Parquet GLib is a bridge between Parquet C++ and GObject Introspection.
+Apache Parquet GLib is a C wrapper for [Apache Parquet C++](https://github.com/apache/arrow/tree/master/cpp/parquet). GObject Introspection can't use Apache Parquet C++ directly. Apache Parquet GLib is a bridge between Apache Parquet C++ and GObject Introspection.
 
 gobject-introspection gem is a Ruby bindings of GObject Introspection. Red Parquet uses GObject Introspection via gobject-introspection gem.
 
 ## Install
 
-Install Parquet GLib before install Red Parquet. Use [Apache Arrow packages](https://github.com/red-data-tools/arrow-packages) for installing Parquet GLib.
+Install Apache Parquet GLib before install Red Parquet. Use [packages.red-data-tools.org](https://github.com/red-data-tools/packages.red-data-tools.org) for installing Apache Parquet GLib.
 
-Install Red Parquet after you install Parquet GLib:
+Note that the Apache Parquet GLib packages are "unofficial". "Official" packages will be released in the future.
+
+Install Red Parquet after you install Apache Parquet GLib:
 
 ```text
 % gem install red-parquet
@@ -33,24 +48,7 @@ Install Red Parquet after you install Parquet GLib:
 ```ruby
 require "parquet"
 
-# TODO
+table = Arrow::Table.load("/dev/shm/data.parquet")
+# Process data in table
+table.save("/dev/shm/data-processed.parquet")
 ```
-
-## Dependencies
-
-* [Apache Parquet C++](https://github.com/apache/parquet-cpp) with Apache Arrow support
-
-* [Parquet GLib](https://github.com/red-data-tools/parquet-glib)
-
-* [Red Arrow](https://rubygems.org/gems/red-arrow)
-
-## Authors
-
-* Kouhei Sutou \<kou@clear-code.com\>
-
-## License
-
-Apache License 2.0. See doc/text/apache-2.0.txt for details.
-
-(Kouhei Sutou has a right to change the license including contributed
-patches.)

data/Rakefile

@@ -1,40 +1,32 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "rubygems"
 require "bundler/gem_helper"
-require "packnga"
 
-base_dir = File.join(File.dirname(__FILE__))
+base_dir = File.join(__dir__)
 
 helper = Bundler::GemHelper.new(base_dir)
-def helper.version_tag
-  version
-end
-
 helper.install
-spec = helper.gemspec
 
-Packnga::DocumentTask.new(spec) do |task|
-  task.original_language = "en"
-  task.translate_language = "ja"
-end
-
-Packnga::ReleaseTask.new(spec) do
-end
+release_task = Rake::Task["release"]
+release_task.prerequisites.replace(["build", "release:rubygem_push"])
 
 desc "Run tests"
 task :test do

data/dependency-check/Rakefile

@@ -1,18 +1,21 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "pkg-config"
 require "native-package-installer"

data/lib/parquet.rb

@@ -1,16 +1,19 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "arrow"
 

data/lib/parquet/arrow-table-loadable.rb

@@ -1,24 +1,30 @@
-# Copyright 2018 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 module Parquet
   module ArrowTableLoadable
     private
-    def load_as_parquet(path)
-      reader = Parquet::ArrowFileReader.new(path)
-      reader.n_threads = @options[:n_threads] || 4 # TODO
-      reader.read_table
+    def load_as_parquet
+      input = open_input_stream
+      reader = Parquet::ArrowFileReader.new(input)
+      reader.use_threads = (@options[:use_threads] != false)
+      table = reader.read_table
+      table.instance_variable_set(:@input, input)
+      table
     end
   end
 end

data/lib/parquet/arrow-table-savable.rb

@@ -1,24 +1,29 @@
-# Copyright 2018 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 module Parquet
   module ArrowTableSavable
     private
-    def save_as_parquet(path)
+    def save_as_parquet
       chunk_size = @options[:chunk_size] || 1024 # TODO
-      Parquet::ArrowFileWriter.open(@table.schema, path) do |writer|
-        writer.write_table(@table, chunk_size)
+      open_output_stream do |output|
+        Parquet::ArrowFileWriter.open(@table.schema, output) do |writer|
+          writer.write_table(@table, chunk_size)
+        end
       end
     end
   end

data/lib/parquet/loader.rb

@@ -1,16 +1,19 @@
-# Copyright 2017-2018 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 module Parquet
   class Loader < GObjectIntrospection::Loader

data/lib/parquet/version.rb

@@ -1,17 +1,26 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 module Parquet
-  VERSION = "0.0.2"
+  VERSION = "0.12.0"
+
+  module Version
+    numbers, TAG = VERSION.split("-")
+    MAJOR, MINOR, MICRO = numbers.split(".").collect(&:to_i)
+    STRING = VERSION
+  end
 end

data/red-parquet.gemspec

@@ -1,52 +1,49 @@
 # -*- ruby -*-
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-clean_white_space = lambda do |entry|
-  entry.gsub(/(\A\n+|\n+\z)/, '') + "\n"
-end
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
-$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), "lib"))
-require "parquet/version"
+require_relative "lib/parquet/version"
 
 Gem::Specification.new do |spec|
   spec.name = "red-parquet"
-  spec.version = Parquet::VERSION
-  spec.homepage = "https://github.com/red-data-tools/red-parquet"
-  spec.authors = ["Kouhei Sutou"]
-  spec.email = ["kou@clear-code.com"]
+  version_components = [
+    Parquet::Version::MAJOR.to_s,
+    Parquet::Version::MINOR.to_s,
+    Parquet::Version::MICRO.to_s,
+    Parquet::Version::TAG,
+  ]
+  spec.version = version_components.compact.join(".")
+  spec.homepage = "https://arrow.apache.org/"
+  spec.authors = ["Apache Arrow Developers"]
+  spec.email = ["dev@arrow.apache.org"]
 
-  readme = File.read("README.md")
-  readme.force_encoding("UTF-8")
-  entries = readme.split(/^\#\#\s(.*)$/)
-  clean_white_space.call(entries[entries.index("Description") + 1])
-  description = clean_white_space.call(entries[entries.index("Description") + 1])
-  spec.summary, spec.description, = description.split(/\n\n+/, 3)
+  spec.summary = "Red Parquet is the Ruby bindings of Apache Parquet"
+  spec.description = "Apache Parquet is a columnar storage format."
   spec.license = "Apache-2.0"
   spec.files = ["README.md", "Rakefile", "Gemfile", "#{spec.name}.gemspec"]
-  spec.files += [".yardopts"]
+  spec.files += ["LICENSE.txt", "NOTICE.txt"]
   spec.files += Dir.glob("lib/**/*.rb")
-  spec.files += Dir.glob("doc/text/*")
   spec.test_files += Dir.glob("test/**/*")
   spec.extensions = ["dependency-check/Rakefile"]
 
-  spec.add_runtime_dependency("red-arrow", ">= 0.8.1")
+  spec.add_runtime_dependency("red-arrow")
 
   spec.add_development_dependency("bundler")
   spec.add_development_dependency("rake")
   spec.add_development_dependency("test-unit")
-  spec.add_development_dependency("packnga")
-  spec.add_development_dependency("kramdown")
 end

data/test/helper.rb

@@ -1,16 +1,19 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 require "parquet"
 

data/test/run-test.rb

@@ -1,28 +1,35 @@
 #!/usr/bin/env ruby
 #
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 $VERBOSE = true
 
 require "pathname"
 
-base_dir = Pathname.new(__FILE__).dirname.parent.expand_path
+base_dir = Pathname.new(__dir__).parent.expand_path
+arrow_base_dir = base_dir.parent + "red-arrow"
 
 lib_dir = base_dir + "lib"
 test_dir = base_dir + "test"
 
+arrow_lib_dir = arrow_base_dir + "lib"
+
+$LOAD_PATH.unshift(arrow_lib_dir.to_s)
 $LOAD_PATH.unshift(lib_dir.to_s)
 
 require_relative "helper"

data/test/test-arrow-table.rb

@@ -1,16 +1,19 @@
-# Copyright 2018 Kouhei Sutou <kou@clear-code.com>
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
 #
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
+#   http://www.apache.org/licenses/LICENSE-2.0
 #
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
 
 class TestArrowTableReader < Test::Unit::TestCase
   def setup
@@ -37,9 +40,15 @@ class TestArrowTableReader < Test::Unit::TestCase
     @table = Arrow::Table.new(schema, [@count_column, @visible_column])
   end
 
-  def test_save_load
+  def test_save_load_path
     tempfile = Tempfile.open(["red-parquet", ".parquet"])
     @table.save(tempfile.path)
     assert_equal(@table, Arrow::Table.load(tempfile.path))
   end
+
+  def test_save_load_buffer
+    buffer = Arrow::ResizableBuffer.new(1024)
+    @table.save(buffer, format: :parquet)
+    assert_equal(@table, Arrow::Table.load(buffer, format: :parquet))
+  end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: red-parquet
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.12.0
 platform: ruby
 authors:
-- Kouhei Sutou
+- Apache Arrow Developers
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-01-05 00:00:00.000000000 Z
+date: 2019-01-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: red-arrow
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.8.1
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: 0.8.1
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -66,50 +66,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-- !ruby/object:Gem::Dependency
-  name: packnga
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: kramdown
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-description: "[Apache Parquet](https://parquet.apache.org/) is a columnar storage
-  format."
+description: Apache Parquet is a columnar storage format.
 email:
-- kou@clear-code.com
+- dev@arrow.apache.org
 executables: []
 extensions:
 - dependency-check/Rakefile
 extra_rdoc_files: []
 files:
-- ".yardopts"
 - Gemfile
+- LICENSE.txt
+- NOTICE.txt
 - README.md
 - Rakefile
 - dependency-check/Rakefile
-- doc/text/apache-2.0.txt
-- doc/text/news.md
 - lib/parquet.rb
 - lib/parquet/arrow-table-loadable.rb
 - lib/parquet/arrow-table-savable.rb
@@ -118,9 +88,8 @@ files:
 - red-parquet.gemspec
 - test/helper.rb
 - test/run-test.rb
-- test/test-arrow-file-reader.rb
 - test/test-arrow-table.rb
-homepage: https://github.com/red-data-tools/red-parquet
+homepage: https://arrow.apache.org/
 licenses:
 - Apache-2.0
 metadata: {}
@@ -139,14 +108,11 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.5.2.2
+rubygems_version: 3.0.2
 signing_key: 
 specification_version: 4
-summary: Red Parquet is a Ruby bindings of Apache Parquet. Red Parquet is based on
-  GObject Introspection.
+summary: Red Parquet is the Ruby bindings of Apache Parquet
 test_files:
 - test/helper.rb
-- test/run-test.rb
-- test/test-arrow-file-reader.rb
 - test/test-arrow-table.rb
+- test/run-test.rb

data/.yardopts

@@ -1,6 +0,0 @@
---output-dir doc/reference/en
---markup markdown
---markup-provider kramdown
-lib/**/*.rb
--
-doc/text/**/*

data/doc/text/news.md

@@ -1,13 +0,0 @@
-# News
-
-## 0.0.2 - 2018-01-05
-
-### Improvements
-
-  * `Arrow::Table#save`: Added Parquet save support.
-
-  * `Arrow::Table.load`: Added Parquet load support.
-
-## 0.0.1 - 2017-05-17
-
-Initial release!!!

data/test/test-arrow-file-reader.rb

@@ -1,41 +0,0 @@
-# Copyright 2017 Kouhei Sutou <kou@clear-code.com>
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-class TestArrowFileReader < Test::Unit::TestCase
-  def test_read_write
-    tempfile = Tempfile.open(["data", ".parquet"])
-
-    values = [true, nil, false, true]
-    chunk_size = 2
-
-    field = Arrow::Field.new("enabled", Arrow::BooleanDataType.new)
-    schema = Arrow::Schema.new([field])
-    Parquet::ArrowFileWriter.open(schema, tempfile.path) do |writer|
-      columns = [
-        Arrow::Column.new(field, Arrow::BooleanArray.new(values)),
-      ]
-      table = Arrow::Table.new(schema, columns)
-      writer.write_table(table, chunk_size)
-    end
-
-    reader = Parquet::ArrowFileReader.new(tempfile.path)
-    reader.n_threads = 4
-    assert_equal(chunk_size, reader.n_row_groups)
-    table = reader.read_table
-    table_data = table.each_column.collect do |column|
-      [column.name, column.to_a]
-    end
-    assert_equal([["enabled", values]], table_data)
-  end
-end