hbase-jruby 0.2.1-java → 0.2.2-java

data/CHANGELOG.md

@@ -1,6 +1,14 @@
 Changelog
 =========
 
+0.2.2
+-----
+- Added `HBase::Table#delete_row` method
+- Dependency profiles as prefixes
+  - Supported prefixes: `cdh4.1`, `cdh3`, `0.94`, `0.92`
+  - e.g. `HBase.resolve_dependency! 'cdh4.1.3'`
+- Advanced data access with `Scoped#with_java_scan` and `Scoped#with_java_get`
+
 0.2.1
 -----
 - Fix: NameError even when appropriate CLASSPATH is set

data/README.md

@@ -1,11 +1,6 @@
 # hbase-jruby
 
-*hbase-jruby* is a Ruby-esque interface for accessing HBase from JRuby.
-
-With JRuby, you can of course just use the native Java APIs of HBase,
-but doing so requires lots of keystrokes even for the most basic operations and
-can lead to having overly verbose code that will be frowned upon by Rubyists.
-Anyhow, JRuby is Ruby, not Java, right?
+*hbase-jruby* is a simple JRuby binding for HBase.
 
 *hbase-jruby* provides the followings:
 - Easy, Ruby-esque interface for the fundamental HBase operations
@@ -17,7 +12,7 @@ Anyhow, JRuby is Ruby, not Java, right?
 ```ruby
 require 'hbase-jruby'
 
-HBase.resolve_dependency! 'cdh4.1'
+HBase.resolve_dependency! 'cdh4.1.3'
 
 hbase = HBase.new
 table = hbase[:test_table]
@@ -56,7 +51,7 @@ table.delete(:rowkey9)
     git clone -b devel https://github.com/junegunn/hbase-jruby.git
     cd hbase-jruby
     rake build
-    gem install pkg/hbase-jruby-0.2.1-java.gem
+    gem install pkg/hbase-jruby-0.2.2-java.gem
 
 ## Setting up
 
@@ -69,27 +64,29 @@ or by `require`ing relevant JAR files after launching JRuby.
 ### `HBase.resolve_dependency!`
 
 Well, there's an easier way.
-You can call `HBase.resolve_dependency!` helper method passing one of the arguments listed below.
+Call `HBase.resolve_dependency!` helper method passing one of the arguments listed below.
 
-| Argument   | Description                                              | Required executable |
-|------------|----------------------------------------------------------|---------------------|
-| 'cdh4.1'   | Predefined Maven profile for Cloudera CDH4.1             | mvn                 |
-| 'cdh3'     | Predefined Maven profile for Cloudera CDH3               | mvn                 |
-| '0.94'     | Predefined Maven profile for Apache HBase 0.94           | mvn                 |
-| '0.92'     | Predefined Maven profile for Apache HBase 0.92           | mvn                 |
-| *POM PATH* | Follow dependency described in the given POM file        | mvn                 |
-| *:local*   | Resolve HBase dependency using `hbase classpath` command | hbase               |
+| Argument   | Dependency               | Required executable |
+|------------|--------------------------|---------------------|
+| cdh4.1[.*] | Cloudera CDH4.1          | mvn                 |
+| cdh3[u*]   | Cloudera CDH3            | mvn                 |
+| 0.94[.*]   | Apache HBase 0.94        | mvn                 |
+| 0.92[.*]   | Apache HBase 0.92        | mvn                 |
+| *POM PATH* | Custom Maven POM file    | mvn                 |
+| `:local`   | Local HBase installation | hbase               |
 
-```ruby
-# Examples
+#### Examples
 
-# Load JAR files from CDH4.1 distribution of HBase using Maven
-HBase.resolve_dependency! 'cdh4.1'
+```ruby
+# Load JAR files from CDH4.1.x using Maven
+HBase.resolve_dependency! 'cdh4.1.3'
+HBase.resolve_dependency! 'cdh4.1.1'
 
-# Load JAR files for HBase 0.94 using Maven
-HBase.resolve_dependency! '0.94', :verbose => true
+# Load JAR files of HBase 0.94.x using Maven
+HBase.resolve_dependency! '0.94.1'
+HBase.resolve_dependency! '0.94.2', :verbose => true
 
-# Dependency resolution with your own POM file
+# Dependency resolution with custom POM file
 HBase.resolve_dependency! '/path/to/my/pom.xml'
 HBase.resolve_dependency! '/path/to/my/pom.xml', :profile => 'trunk'
 
@@ -375,10 +372,16 @@ table.delete('rowkey1', 'cf1:col1', 1352978648642)
 table.delete('rowkey1', 'cf1:col1', 1352978648642, 1352978649642)
 
 # Batch delete
-table.delete(['rowkey1'], ['rowkey2'], ['rowkey3', 'cf1:col1'])
+table.delete(['rowkey1'], ['rowkey2'], ['rowkey3', 'cf1:col1', 1352978648642, 135297864964])
+```
+
+However, the last syntax seems a bit unwieldy when you just wish to delete a few rows.
+In that case, use simpler `delete_row` method.
+
+```ruby
+table.delete_row 'rowkey1'
 
-# Truncate table
-table.truncate!
+table.delete_row 'rowkey1', 'rowkey2', 'rowkey3'
 ```
 
 ### Atomic increment of column values
@@ -409,18 +412,20 @@ end
 You can control how you retrieve data by chaining
 the following methods of `HBase::Table` (or `HBase::Scoped`).
 
-| Method       | Description                                                     |
-|--------------|-----------------------------------------------------------------|
-| `range`      | Specifies the rowkey range of scan                              |
-| `project`    | To retrieve only a subset of columns                            |
-| `filter`     | Filtering conditions of scan                                    |
-| `while`      | Allows early termination of scan (server-side)                  |
-| `at`         | Only retrieve data with the specified timestamp                 |
-| `time_range` | Only retrieve data within the specified time range              |
-| `limit`      | Limits the number of rows                                       |
-| `versions`   | Limits the number of versions of each column                    |
-| `caching`    | Sets the number of rows for caching during scan                 |
-| `batch`      | Limits the maximum number of values returned for each iteration |
+| Method           | Description                                                     |
+|------------------|-----------------------------------------------------------------|
+| `range`          | Specifies the rowkey range of scan                              |
+| `project`        | To retrieve only a subset of columns                            |
+| `filter`         | Filtering conditions of scan                                    |
+| `while`          | Allows early termination of scan (server-side)                  |
+| `at`             | Only retrieve data with the specified timestamp                 |
+| `time_range`     | Only retrieve data within the specified time range              |
+| `limit`          | Limits the number of rows                                       |
+| `versions`       | Limits the number of versions of each column                    |
+| `caching`        | Sets the number of rows for caching during scan                 |
+| `batch`          | Limits the maximum number of values returned for each iteration |
+| `with_java_scan` | *(ADVANCED)* Access Java Scan object in the given block         |
+| `with_java_get`  | *(ADVANCED)* Access Java Get object in the given block          |
 
 Each invocation to these methods returns an `HBase::Scoped` instance with which
 you can retrieve data with the following methods.
@@ -450,6 +455,9 @@ table.range('A'..'Z').                      # Row key range,
       versions(2).                          # Only fetches 2 versions for each value
       batch(100).                           # Batch size for scan set to 100
       caching(1000).                        # Caching 1000 rows
+      with_java_scan { |scan|               # Directly access Java Scan object
+        scan.setCacheBlocks false
+      }.
       to_a                                  # To Array
 ```
 

data/hbase-jruby.gemspec

@@ -8,8 +8,8 @@ Gem::Specification.new do |gem|
   gem.version       = HBase::JRuby::VERSION
   gem.authors       = ["Junegunn Choi"]
   gem.email         = ["junegunn.c@gmail.com"]
-  gem.description   = %q{Ruby-esque interface for accessing HBase from JRuby}
-  gem.summary       = %q{Ruby-esque interface for accessing HBase from JRuby}
+  gem.description   = %q{A JRuby binding for HBase}
+  gem.summary       = %q{A JRuby binding for HBase}
   gem.homepage      = "https://github.com/junegunn/hbase-jruby"
   gem.platform      = 'java'
   gem.license       = 'MIT'

data/lib/hbase-jruby/byte_array.rb

@@ -3,7 +3,7 @@ class << self
   # Shortcut method to HBase::ByteArray.new
   # @param [*Object] values
   def ByteArray *values
-    ByteArray.new *values
+    ByteArray.new(*values)
   end
 end
 # Boxed class for Java byte arrays

data/lib/hbase-jruby/dependency.rb

@@ -1,9 +1,20 @@
 require 'java'
 require 'open-uri'
 require 'tempfile'
+require 'erb'
 
 # HBase connection
 class HBase
+
+  # @private
+  SUPPORTED_PROFILES = {
+    # Prefix => Latest version
+    'cdh4.1' => 'cdh4.1.3',
+    'cdh3'   => 'cdh3u5',
+    '0.94'   => '0.94.3',
+    '0.92'   => '0.92.1',
+  }
+
   class << self
     # @overload resolve_dependency!(dist, options)
     #   Resolve Hadoop and HBase dependency with a predefined Maven profile
@@ -40,12 +51,25 @@ class HBase
           mvn = `which mvn`
           raise RuntimeError, "Cannot find `mvn` executable" if mvn.empty?
 
+          # POM file path given (with optional profile)
           if File.exists?(dist)
             path = dist
             profile = options[:profile] && "-P #{options[:profile]}"
+          # Predefined dependencies
           else
-            path = File.expand_path("../pom/pom.xml", __FILE__)
-            profile = "-P #{dist}"
+            matched_profiles = SUPPORTED_PROFILES.keys.select { |pf| dist.start_with? pf }
+            if matched_profiles.length != 1
+              raise ArgumentError, "Invalid profile: #{dist}"
+            end
+            matched_profile = matched_profiles.first
+            profiles = SUPPORTED_PROFILES.dup
+            profiles[matched_profile] = dist if dist != matched_profile
+            tempfiles << tf = Tempfile.new('hbase-jruby-pom')
+            erb = ERB.new(File.read File.expand_path("../pom/pom.xml.erb", __FILE__))
+            tf << erb.result(binding)
+            tf.close(false)
+            path = tf.path
+            profile = "-P #{matched_profile}"
           end
 
           # Download dependent JAR files and build classpath string

data/lib/hbase-jruby/pom/{pom.xml → pom.xml.erb}

@@ -14,8 +14,8 @@
       <id>cdh4.1</id>
       <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <hadoop.version>2.0.0-cdh4.1.2</hadoop.version>
-        <hbase.version>0.92.1-cdh4.1.2</hbase.version>
+        <hadoop.version>2.0.0-<%= profiles['cdh4.1'] %></hadoop.version>
+        <hbase.version>0.92.1-<%= profiles['cdh4.1'] %></hbase.version>
       </properties>
       <repositories>
         <repository>
@@ -51,8 +51,8 @@
       <id>cdh3</id>
       <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <hbase.version>0.90.6-cdh3u5</hbase.version>
-        <hadoop.version>0.20.2-cdh3u5</hadoop.version>
+        <hbase.version>0.90.6-<%= profiles['cdh3'] %></hbase.version>
+        <hadoop.version>0.20.2-<%= profiles['cdh3'] %></hadoop.version>
       </properties>
 
       <repositories>
@@ -83,7 +83,7 @@
       <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <hadoop.version>1.1.1</hadoop.version>
-        <hbase.version>0.92.2</hbase.version>
+        <hbase.version><%= profiles['0.92'] %></hbase.version>
       </properties>
 
       <dependencies>
@@ -108,7 +108,7 @@
       <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <hadoop.version>1.1.1</hadoop.version>
-        <hbase.version>0.94.3</hbase.version>
+        <hbase.version><%= profiles['0.94'] %></hbase.version>
       </properties>
 
       <dependencies>
@@ -127,75 +127,6 @@
         </dependency>
       </dependencies>
     </profile>
-
-    <!-- For backward compatibility -->
-    <profile>
-      <id>cdh4.1.2</id>
-      <properties>
-        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <hadoop.version>2.0.0-cdh4.1.2</hadoop.version>
-        <hbase.version>0.92.1-cdh4.1.2</hbase.version>
-      </properties>
-      <repositories>
-        <repository>
-          <id>cloudera-releases</id>
-          <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
-          <releases>
-            <enabled>true</enabled>
-          </releases>
-          <snapshots>
-            <enabled>false</enabled>
-          </snapshots>
-        </repository>
-      </repositories>
-
-      <dependencies>
-        <dependency>
-          <groupId>org.apache.hadoop</groupId>
-          <artifactId>hadoop-common</artifactId>
-          <version>${hadoop.version}</version>
-          <scope>compile</scope>
-        </dependency>
-
-        <dependency>
-          <groupId>org.apache.hbase</groupId>
-          <artifactId>hbase</artifactId>
-          <version>${hbase.version}</version>
-          <scope>compile</scope>
-        </dependency>
-      </dependencies>
-    </profile>
-
-    <profile>
-      <id>cdh3u5</id>
-      <properties>
-        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <hbase.version>0.90.6-cdh3u5</hbase.version>
-        <hadoop.version>0.20.2-cdh3u5</hadoop.version>
-      </properties>
-
-      <repositories>
-        <repository>
-          <id>cloudera-releases</id>
-          <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
-          <releases>
-            <enabled>true</enabled>
-          </releases>
-          <snapshots>
-            <enabled>false</enabled>
-          </snapshots>
-        </repository>
-      </repositories>
-
-      <dependencies>
-        <dependency>
-          <groupId>org.apache.hbase</groupId>
-          <artifactId>hbase</artifactId>
-          <version>${hbase.version}</version>
-          <scope>compile</scope>
-        </dependency>
-      </dependencies>
-    </profile>
   </profiles>
 </project>
 

data/lib/hbase-jruby/scoped.rb

@@ -217,6 +217,28 @@ class Scoped
     spawn :@batch, b
   end
 
+  # Returns an HBase::Scoped object with the Scan-customization block added.
+  # The given block will be evaluated just before an actual scan operation.
+  # With method-chaining, multiple blocks can be registered to be evaluated sequentially.
+  # @return [HBase::Scoped]
+  # @yield [org.apache.hadoop.hbase.client.Scan]
+  def with_java_scan &block
+    raise ArgumentError, "Block not given" if block.nil?
+    raise ArgumentError, "Invalid arity: should be 1" unless block.arity == 1
+    spawn :@scan_cbs, @scan_cbs + [block]
+  end
+
+  # Returns an HBase::Scoped object with the Get-customization block added
+  # The given block will be evaluated just before an actual get operation.
+  # With method-chaining, multiple blocks can be registered to be evaluated sequentially.
+  # @return [HBase::Scoped]
+  # @yield [org.apache.hadoop.hbase.client.Get]
+  def with_java_get &block
+    raise ArgumentError, "Block not given" if block.nil?
+    raise ArgumentError, "Invalid arity: should be 1" unless block.arity == 1
+    spawn :@get_cbs, @get_cbs + [block]
+  end
+
 private
   # @param [HBase::Table] table
   def initialize table
@@ -230,6 +252,8 @@ private
     @caching  = nil
     @limit    = nil
     @trange   = nil
+    @scan_cbs = []
+    @get_cbs  = []
   end
 
   def spawn *args
@@ -365,6 +389,11 @@ private
       when Time, Fixnum
         get.setTimeStamp @trange
       end
+
+      # Customization
+      @get_cbs.each do |prc|
+        prc.call get
+      end
     }
   end
 
@@ -494,6 +523,11 @@ private
 
       # Batch
       scan.setBatch @batch if @batch
+
+      # Customization
+      @scan_cbs.each do |prc|
+        prc.call scan
+      end
     }
   end
 

data/lib/hbase-jruby/table.rb

@@ -40,6 +40,14 @@ class Table
     end
   end
 
+  def with_java_scan &block
+    self.each.with_java_scan(&block)
+  end
+
+  def with_java_get &block
+    self.each.with_java_get(&block)
+  end
+
   # Performs PUT operations
   # @overload put(rowkey, data)
   #   Put operation on a rowkey
@@ -119,6 +127,13 @@ class Table
     }
   end
 
+  # Delete rows.
+  # @param [*Object] rowkeys List of rowkeys of rows to delete
+  # @return [nil]
+  def delete_row *rowkeys
+    htable.delete rowkeys.map { |rk| Delete.new(Util.to_bytes rk) }
+  end
+
   # Atomically increase numeric values
   # @overload increment(rowkey, column, by)
   #   Atomically increase column value by the specified amount

data/lib/hbase-jruby/version.rb

@@ -1,5 +1,5 @@
 class HBase
 module JRuby
-  VERSION = "0.2.1"
+  VERSION = "0.2.2"
 end
 end

data/test/test_scoped.rb

@@ -390,6 +390,8 @@ class TestScoped < TestHBaseJRubyBase
     @table.put :rowkey3 => { 'cf1:a' => { t3 => 3 } }
     @table.put :rowkey4 => { 'cf1:a' => { t4 => 4 } }
 
+    assert_equal 4, @table.count
+
     assert_equal 1, @table.time_range(t2, t3).count
     assert_equal 2, @table.time_range(t2, t3 + 1).count
     assert_equal 2, @table.time_range(t2, t4).count
@@ -413,5 +415,43 @@ class TestScoped < TestHBaseJRubyBase
     # according to current hbase impl, later call overrides the previous time ranges. but, why do this?
     assert_equal 2, @table.time_range(t2, t3).at(t1).count
   end
+
+  def test_with_java_scan
+    ('a'..'z').each do |rk|
+      @table.put rk, 'cf1:a' => 1
+    end
+
+    assert_equal 2, @table.with_java_scan { |scan|
+      scan.setStartRow HBase::Util.to_bytes 'a'
+      scan.setStopRow HBase::Util.to_bytes 'd'
+    }.with_java_scan { |scan|
+      scan.setStartRow HBase::Util.to_bytes 'b'
+    }.count
+  end
+
+  def test_with_java_get
+    t1, t2, t3, t4 =
+      Time.now - 4000,
+      Time.now - 3000,
+      Time.now - 2000,
+      Time.now - 1000
+    @table.put :r1 => { 'cf1:a' => { t1 => 1 } }
+    @table.put :r2 => { 'cf1:a' => { t2 => 2 } }
+    @table.put :r3 => { 'cf1:a' => { t3 => 3 } }
+    @table.put :r4 => { 'cf1:a' => { t4 => 4 } }
+
+    assert_equal 4, @table.count
+
+    rks = [:r1, :r2, :r3, :r4]
+    assert_equal 4, @table.get(rks).compact.count
+
+    scoped = @table.with_java_get { |get|
+      get.setTimeRange(t1.to_i * 1000, t4.to_i * 1000)
+    }
+    assert_equal 3, scoped.get(rks).compact.count
+    assert_equal 2, scoped.with_java_get { |get|
+      get.setTimeRange(t2.to_i * 1000, t4.to_i * 1000)
+    }.get(rks).compact.count
+  end
 end
 

data/test/test_table.rb

@@ -266,5 +266,21 @@ class TestTable < TestHBaseJRubyBase
     assert_nil @table.get('row3').to_hash['cf1:a']
     assert_equal 2, @table.get('row3').fixnum('cf1:b')
   end
+
+  def test_delete_row
+    @table.put(1 => { 'cf1:a' => 1 }, 's' => { 'cf1:a' => 2 }, { :short => 3 } => { 'cf1:a' => 3 })
+
+    assert_equal 1, @table.get(1).fixnum('cf1:a')
+    assert_equal 2, @table.get('s').fixnum('cf1:a')
+    assert_equal 3, @table.get({ :short => 3 }).fixnum('cf1:a')
+    assert_equal 3, @table.count
+
+    @table.delete_row 1, { :short => 3 }
+
+    assert_equal nil, @table.get(1)
+    assert_equal 2,   @table.get('s').fixnum('cf1:a')
+    assert_equal nil, @table.get({ :short => 3 })
+    assert_equal 1,   @table.count
+  end
 end
 

metadata

@@ -2,14 +2,14 @@
 name: hbase-jruby
 version: !ruby/object:Gem::Version
   prerelease:
-  version: 0.2.1
+  version: 0.2.2
 platform: java
 authors:
 - Junegunn Choi
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2013-01-29 00:00:00.000000000 Z
+date: 2013-02-13 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: test-unit
@@ -47,7 +47,7 @@ dependencies:
     none: false
   prerelease: false
   type: :development
-description: Ruby-esque interface for accessing HBase from JRuby
+description: A JRuby binding for HBase
 email:
 - junegunn.c@gmail.com
 executables: []
@@ -68,7 +68,7 @@ files:
 - lib/hbase-jruby/column_key.rb
 - lib/hbase-jruby/dependency.rb
 - lib/hbase-jruby/hbase.rb
-- lib/hbase-jruby/pom/pom.xml
+- lib/hbase-jruby/pom/pom.xml.erb
 - lib/hbase-jruby/result.rb
 - lib/hbase-jruby/scoped.rb
 - lib/hbase-jruby/scoped/aggregation.rb
@@ -113,7 +113,7 @@ rubyforge_project:
 rubygems_version: 1.8.24
 signing_key:
 specification_version: 3
-summary: Ruby-esque interface for accessing HBase from JRuby
+summary: A JRuby binding for HBase
 test_files:
 - test/helper.rb
 - test/test_aggregation.rb