aerospike 2.9.0 → 2.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b4a6204e2eedbfcbd9a755abdc89da26d978ec5cddbc401b9085863e953c9a5a
-  data.tar.gz: 9ef1ad5e358a98a30da46cf6aded8bc8958f62e16f239523f313a7c2ddd169b2
+  metadata.gz: 7bef523a7e7500d8e39cb80f3b9ee6831a5ddc0bd6d782685181cae6a6c6ada5
+  data.tar.gz: 760d626bffac6ed69bcdda048daf39b07a12a85264b67be0677ad5d552c5473c
 SHA512:
-  metadata.gz: 6ca26471e04a8999b081699e2b68f51eaa00e28cc1e8e8fc4d7e0f1fa2830de9dbb5725b888420cab85dbd5f34af2dae675b81963a9b7c88320e3393e2b0c1a1
-  data.tar.gz: c854595481004116f1c0fd90600dcf0ba94dfe9b371a8a310a21afae601dd491ecd960658cfe7fe3f08859259af421b19453aeec2cfda68484af328f7c98657c
+  metadata.gz: df19a0c2900463a35b73ba52bfae3bdb2c5f6cbdaba3ab17172219e96ad3de3878bbc1ff57403fbbffe0fb48cc147ee642297e2fe47c829d768e76b1e4dee0fa
+  data.tar.gz: 528ade6f9abfb9b9e266439fefa6606f2623aec2503ec6c5466a5c3f9ffd9656debea49ff434f715f042c3ae319546e80352aa36cdc925ed351cab2633c75c4b

data/CHANGELOG.md

@@ -2,7 +2,54 @@
 
 All notable changes to this project will be documented in this file.
 
-## [Unreleased]
+## [2.13.0] - 2019-07-17
+
+* **New Features**
+  * Adds support for replica policies.
+
+* **Improvements**
+  * Remove support for "old" partition tokenizer.
+  * Refactor how partition parser is initialized and called.
+  * Adds support for 'replicas' and remove the old partition table queries from the server.
+
+## [2.12.0] - 2019-04-21
+
+* **New Features**
+  * Support for predicate expressions in all transaction.
+  * Support for `operation.delete` in `client#operate`.
+
+* **Improvements**
+  * Optimize serialization for nested structures. Thanks to [@Kacper Madej](https://github.com/madejejej)! [[#94](https://github.com/aerospike/aerospike-client-ruby/pull/94)]
+  * Remove `Thread#abort_on_exception` from `batch_index_command`. Thanks to [@Kacper Madej](https://github.com/madejejej)! [[#94](https://github.com/aerospike/aerospike-client-ruby/pull/92)]
+  * Does not allow values other than Integer, Float, String, Symbol and nil to be used as keys in Maps.
+
+* **Bug Fixes**
+  * Fixes tests that weren't using ENV variables for connections. This will allow the tests to be run on any server.
+
+
+## [2.11.0] - 2019-05-17
+
+* **New Features**
+  * Support for predicate expressions in queries. Thanks to [@Minus10Degrees](https://github.com/Minus10Degrees)! [[#78](https://github.com/aerospike/aerospike-client-ruby/issues/78)]
+
+* **Bug Fixes**
+  * Client#execute\_udf\_on\_query should not modify the statement argument. [[#79](https://github.com/aerospike/aerospike-client-ruby/issues/79)]
+  * Encoding::UndefinedConversionError when reading blob data from CDT list/map bin. [[#84](https://github.com/aerospike/aerospike-client-ruby/issues/84)]
+
+## [2.10.0] - 2019-05-10
+
+* **New Features**
+  * Add support for LB discovery / seeding. Thanks to [@filiptepper](https://github.com/filiptepper)! [[#80](https://github.com/aerospike/aerospike-client-ruby/issues/80)]
+
+## [2.9.1] - 2019-04-03
+
+* **Bug Fixes**
+  * Query fails if one or more cluster nodes do not have records in the set [[#77](https://github.com/aerospike/aerospike-client-ruby/issues/77)]
+
+* **Updates**
+  * Change admin message version to 2 (from 0)
+  * Remove unused BIN_EXISTS_ERROR (6) and BIN_NOT_FOUND (17) error codes
+  * Tests: Support setting user/password when running specs
 
 ## [2.9.0] - 2018-11-09
 
@@ -10,10 +57,10 @@ All notable changes to this project will be documented in this file.
   * Add INFINITY and WILDCARD values for use in CDT map/list comparators. [AER-5945]
 
 * **Bug Fixes**
-  * Default policies set on Client instance do not get applied [[#74](https://github.com/aerospike/aerospike-client-ruby/issue/74)]
+  * Default policies set on Client instance do not get applied [[#74](https://github.com/aerospike/aerospike-client-ruby/issues/74)]
 
 * **Updates**
-  * *BREAKING CHANGE*: Change default for send_key write policy to false [[#73](https://github.com/aerospike/aerospike-client-ruby/issue/73)]
+  * *BREAKING CHANGE*: Change default for send_key write policy to false [[#73](https://github.com/aerospike/aerospike-client-ruby/issues/73)]
   * Support truncate info command argument "lut=now" for servers that require it. [AER-5955]
 
 ## [2.8.0] - 2018-08-06
@@ -140,7 +187,7 @@ All notable changes to this project will be documented in this file.
 
 ## [2.0.0] - 2016-05-27
 
-* **Breaking Changes** - Please refer to detailed list of [API changes](docs/api-changes.md#v2.0.0) for further details.
+* **Breaking Changes** - Please refer to detailed list of [API changes](https://www.aerospike.com/docs/client/ruby/usage/incompatible.html#version-2-0-0) for further details.
   * Incompatible integer key digests: digests for integer keys computed by v2 and v1 are different; the Aerospike server uses the key digest to retrieve records. This will impact your ability to read records with integer keys that were created by a v1 client version.
   * Backward incompatible changes to the `Aerospike::Client.new` initializer.
   * The `Aerospike::Client.new_many` initializer has been removed; use `Aerospike::Client.new` instead.

data/README.md

@@ -97,7 +97,7 @@ This library is packaged with a number of tests.
 
 To run all the test cases:
 
-    $ bundle exec rspec
+    $ AEROSPIKE_HOSTS="<host:port>[,<hoist:port>]" AEROSPIKE_USER="<user>" AEROSPIKE_PASSWORD="<pass>" bundle exec rspec
 
 <a name="Examples"></a>
 ## Examples

data/lib/aerospike.rb

@@ -1,4 +1,4 @@
-# Copyright 2014-2018 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.
@@ -101,11 +101,10 @@ require 'aerospike/connection/create'
 
 require 'aerospike/cluster'
 require 'aerospike/cluster/create_connection'
-require 'aerospike/cluster/partition'
 require 'aerospike/cluster/find_nodes_to_remove'
 require 'aerospike/cluster/find_node'
-require 'aerospike/cluster/partition_tokenizer_new'
-require 'aerospike/cluster/partition_tokenizer_old'
+require 'aerospike/cluster/partition'
+require 'aerospike/cluster/partition_parser'
 require 'aerospike/node'
 require 'aerospike/node/generation'
 require 'aerospike/node/refresh/failed'
@@ -142,6 +141,15 @@ require 'aerospike/query/stream_command'
 require 'aerospike/query/query_command'
 require 'aerospike/query/scan_command'
 require 'aerospike/query/statement'
+require 'aerospike/query/pred_exp'
+
+require 'aerospike/query/pred_exp/and_or'
+require 'aerospike/query/pred_exp/geo_json_value'
+require 'aerospike/query/pred_exp/integer_value'
+require 'aerospike/query/pred_exp/op'
+require 'aerospike/query/pred_exp/regex'
+require 'aerospike/query/pred_exp/regex_flags'
+require 'aerospike/query/pred_exp/string_value'
 
 module Aerospike
   extend Loggable

data/lib/aerospike/aerospike_exception.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2014-2018 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.
@@ -90,5 +90,11 @@ module Aerospike
         super(ResultCode::COMMAND_REJECTED, msg)
       end
     end
+
+    class InvalidNamespace < Aerospike
+      def initialize(msg=nil)
+        super(ResultCode::INVALID_NAMESPACE, msg)
+      end
+    end
   end
 end

data/lib/aerospike/atomic/atomic.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2014-2017 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

data/lib/aerospike/bin.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2014-2017 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/list_operation.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-# Copyright 2016-2018 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/map_operation.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2016-2018 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/map_order.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2016-2017 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/map_policy.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2016-2018 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/map_return_type.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2016-2017 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/cdt/map_write_mode.rb

@@ -1,5 +1,5 @@
 # encoding: utf-8
-# Copyright 2016-2018 Aerospike, Inc.
+# Copyright 2016-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.

data/lib/aerospike/client.rb

@@ -1,4 +1,4 @@
-# Copyright 2014-2018 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.
@@ -322,11 +322,11 @@ module Aerospike
 
       if policy.use_batch_direct
         key_map = BatchItem.generate_map(keys)
-        execute_batch_direct_commands(keys) do |node, batch|
+        execute_batch_direct_commands(policy, keys) do |node, batch|
           BatchDirectCommand.new(node, batch, policy, key_map, bin_names, results, info_flags)
         end
       else
-        execute_batch_index_commands(keys) do |node, batch|
+        execute_batch_index_commands(policy, keys) do |node, batch|
           BatchIndexCommand.new(node, batch, policy, bin_names, results, info_flags)
         end
       end
@@ -351,11 +351,11 @@ module Aerospike
 
       if policy.use_batch_direct
         key_map = BatchItem.generate_map(keys)
-        execute_batch_direct_commands(keys) do |node, batch|
+        execute_batch_direct_commands(policy, keys) do |node, batch|
           BatchDirectExistsCommand.new(node, batch, policy, key_map, results)
         end
       else
-        execute_batch_index_commands(keys) do |node, batch|
+        execute_batch_index_commands(policy, keys) do |node, batch|
           BatchIndexExistsCommand.new(node, batch, policy, results)
         end
       end
@@ -526,7 +526,7 @@ module Aerospike
         raise Aerospike::Exceptions::Aerospike.new(Aerospike::ResultCode::SERVER_NOT_AVAILABLE, "Executing UDF failed because cluster is empty.")
       end
 
-      # TODO: wait until all migrations are finished
+      statement = statement.clone
       statement.set_aggregate_function(package_name, function_name, function_args, false)
 
       # Use a thread per node
@@ -884,17 +884,16 @@ module Aerospike
       command.execute
     end
 
-    def execute_batch_index_commands(keys)
+    def execute_batch_index_commands(policy, keys)
       if @cluster.nodes.empty?
         raise Aerospike::Exceptions::Aerospike.new(Aerospike::ResultCode::SERVER_NOT_AVAILABLE, "Executing Batch Index command failed because cluster is empty.")
       end
 
-      batch_nodes = BatchIndexNode.generate_list(@cluster, keys)
+      batch_nodes = BatchIndexNode.generate_list(@cluster, policy.replica, keys)
       threads = []
 
       batch_nodes.each do |batch|
         threads << Thread.new do
-          Thread.current.abort_on_exception = true
           command = yield batch.node, batch
           execute_command(command)
         end
@@ -903,12 +902,12 @@ module Aerospike
       threads.each(&:join)
     end
 
-    def execute_batch_direct_commands(keys)
+    def execute_batch_direct_commands(policy, keys)
       if @cluster.nodes.empty?
         raise Aerospike::Exceptions::Aerospike.new(Aerospike::ResultCode::SERVER_NOT_AVAILABLE, "Executing Batch Direct command failed because cluster is empty.")
       end
 
-      batch_nodes = BatchDirectNode.generate_list(@cluster, keys)
+      batch_nodes = BatchDirectNode.generate_list(@cluster, policy.replica, keys)
       threads = []
 
       # Use a thread per namespace per node

data/lib/aerospike/cluster.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: true
 
-# Copyright 2014-2018 Aerospike, Inc.
+# Copyright 2014-2020 Aerospike, Inc.
 #
 # Portions may be licensed to Aerospike, Inc. under one or more contributor
 # license agreements.
@@ -38,6 +38,8 @@ module Aerospike
       @cluster_name = policy.cluster_name
       @tls_options = policy.tls
 
+      @replica_index = Atomic.new(0)
+
       @aliases = {}
       @cluster_nodes = []
       @partition_write_map = {}
@@ -102,18 +104,92 @@ module Aerospike
       (node_array.length > 0) && !@closed.value
     end
 
-    def get_node_for_key(key)
-      partition = Partition.new_by_key(key)
+    # Returns a node on the cluster for read operations
+    def batch_read_node(partition, replica_policy)
+      case replica_policy
+        when Aerospike::Replica::MASTER, Aerospike::Replica::SEQUENCE
+          return master_node(partition)
+        when Aerospike::Replica::MASTER_PROLES
+          return master_proles_node(partition)
+        when Aerospike::Replica::RANDOM
+          return random_node
+        else
+          raise Aerospike::Exceptions::InvalidNode("invalid policy.replica value")
+      end
+    end
+
+    # Returns a node on the cluster for read operations
+    def read_node(partition, replica_policy, seq)
+      case replica_policy
+        when Aerospike::Replica::MASTER
+          return master_node(partition)
+        when Aerospike::Replica::MASTER_PROLES
+          return master_proles_node(partition)
+        when Aerospike::Replica::SEQUENCE
+          return sequence_node(partition, seq)
+        when Aerospike::Replica::RANDOM
+          return random_node
+        else
+          raise Aerospike::Exceptions::InvalidNode("invalid policy.replica value")
+      end
+    end
+
+    # Returns a node on the cluster for read operations
+    def master_node(partition)
+      partition_map = partitions
+      replica_array = partition_map[partition.namespace]
+      raise Aerospike::Exceptions::InvalidNamespace("namespace not found in the partition map") if !replica_array
+
+      node_array = (replica_array.get)[0]
+      raise Aerospike::Exceptions::InvalidNamespace("namespace not found in the partition map") if !node_array
+
+      node = (node_array.get)[partition.partition_id]
+      raise Aerospike::Exceptions::InvalidNode if !node || !node.active?
+
+      node
+    end
+
+    # Returns a node on the cluster for read operations
+    def master_proles_node(partition)
+      partition_map = partitions
+      replica_array = partition_map[partition.namespace]
+      raise Aerospike::Exceptions::InvalidNamespace("namespace not found in the partition map") if !replica_array
+
+      replica_array = replica_array.get
 
-      # Must copy hashmap reference for copy on write semantics to work.
-      nmap = partitions
-      if node_array = nmap[partition.namespace]
-        node = node_array.value[partition.partition_id]
+      node = nil
+      for replica in replica_array
+        idx = (@replica_index.update{|v| v.succ} % replica_array.size).abs
+        node = (replica_array[idx].get)[partition.partition_id]
 
         return node if node && node.active?
       end
 
-      random_node
+      raise Aerospike::Exceptions::InvalidNode
+    end
+
+    # Returns a random node on the cluster
+    def sequence_node(partition, seq)
+      partition_map = partitions
+      replica_array = partition_map[partition.namespace]
+      raise Aerospike::Exceptions::InvalidNamespace("namespace not found in the partition map") if !replica_array
+
+      replica_array = replica_array.get
+
+      node = nil
+      for replica in replica_array
+        idx = (seq.update{|v| v.succ} % replica_array.size).abs
+        node = (replica_array[idx].get)[partition.partition_id]
+
+        return node if node && node.active?
+      end
+
+      raise Aerospike::Exceptions::InvalidNode
+    end
+
+    def get_node_for_key(replica_policy, key)
+      partition = Partition.new_by_key(key)
+      batch_read_node(partition, replica_policy)
     end
 
     # Returns a random node on the cluster
@@ -124,8 +200,8 @@ module Aerospike
       i = 0
       while i < length
         # Must handle concurrency with other non-tending threads, so node_index is consistent.
-        index = (@node_index.update{ |v| v+1 } % node_array.length).abs
-        node = node_array[index]
+        idx = (@node_index.update{ |v| v.succ } % node_array.length).abs
+        node = node_array[idx]
 
         return node if node.active?
 
@@ -167,12 +243,9 @@ module Aerospike
       end
     end
 
-    def update_partitions(tokens, node)
-      nmap = tokens.update_partition(partitions, node)
-      # update partition write map
+    def update_partitions(parser)
+      nmap = parser.update_partitions(partitions)
       set_partitions(nmap) if nmap
-
-      Aerospike.logger.info("Partitions for node #{node.name} updated")
     end
 
     def request_info(policy, *commands)
@@ -441,8 +514,10 @@ module Aerospike
     def find_node_in_partition_map(filter)
       partitions_list = partitions
 
-      partitions_list.values.each do |node_array|
-        return true if node_array.value.any? { |node| node == filter }
+      partitions_list.values.each do |replica_array|
+        replica_array.get.each do |node_array|
+          return true if node_array.value.any? { |node| node == filter }
+        end
       end
       false
     end