khiva 0.1.1 → 0.1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b7e270535ccb0436961b438ccf0d6effd1976933e1c5a2f98920d207816d6c21
-  data.tar.gz: 8fa258b1c954a3cdd5ad02c0b1da57f6d175fc844e0fdabe2ae82f902a18d431
+  metadata.gz: 5bf2169ce0f0464a8c9d6e07e962a3d00cc8cf1403178f562b2dc6b694915460
+  data.tar.gz: eb67390404557a984c16a7bcdf72cf3b943cf9b85e4f57b9ed2ae22fd55a1eab
 SHA512:
-  metadata.gz: a54ff69b81b37c7e206a46babc733ed1987e16ae84dd40a19eae0fb688b08e245a8df1b212debc5342f1e7da1a934aa972b8a936faf58b1300aa3cdecb519a70
-  data.tar.gz: aac9b7764789f171d5f7b33834148117a7423b2182303b4d45bd1307d9a57c187fa492499810692846f25b6f13e20f3243b8cba4b28c82a0faa4b3ce949fb478
+  metadata.gz: 60d803c2a01cfcc47fb05e5663fc5c884b77ba20b73aaed88bc8042912546c846d57265800409eb41fccfc2d5a5720e81c4a5f3f13c987ca73ac429d801d6ce7
+  data.tar.gz: c5e223048d0736927ec5b1440f7b3d993bc71a411f0e08f6a0ee26d1407a275e3a7da2e6e22b5515deb5f45e701ce899334fd72b6ea8e7e8963a983a9db0b545

data/CHANGELOG.md

@@ -1,3 +1,9 @@
+## 0.1.2 (2020-12-19)
+
+- Added `find_best_n_occurrences` and `mass` methods
+- Added more dimensionality methods
+- Added more feature methods
+
 ## 0.1.1 (2020-12-18)
 
 - Added many more methods

data/README.md

@@ -2,7 +2,7 @@
 
 [Khiva](https://github.com/shapelets/khiva) - high-performance time series algorithms - for Ruby
 
-:fire: Runs on GPUs and CPUs
+:fire: Runs on GPUs (even on Mac) and CPUs
 
 [![Build Status](https://github.com/ankane/khiva-ruby/workflows/build/badge.svg?branch=master)](https://github.com/ankane/khiva-ruby/actions)
 
@@ -22,24 +22,85 @@ gem 'khiva'
 
 ## Getting Started
 
-Calculate a matrix profile
+Calculate the [matrix profile](https://stumpy.readthedocs.io/en/latest/Tutorial_The_Matrix_Profile.html) between two time series
 
 ```ruby
-a = Khiva::Array.new([11, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11])
-b = Khiva::Array.new([9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 9])
-profile, index = Khiva::Matrix.stomp(a, b, 3)
+a = Khiva::Array.new([11, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 11], type: :f32)
+b = Khiva::Array.new([9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 9], type: :f32)
+m = 3 # subsequence length
+profile, index = Khiva::Matrix.stomp(a, b, m)
 ```
 
-Find discords
+Find motifs (repeated patterns)
 
 ```ruby
-distances, indices, subsequences = Khiva::Matrix.find_best_n_discords(profile, index, 2, 2)
+n = 2 # number of motifs to extract
+distances, indices, subsequences = Khiva::Matrix.find_best_n_motifs(profile, index, m, n)
 ```
 
-Find motifs
+Find discords (anomalies)
+
+```ruby
+n = 2 # number of discords to extract
+distances, indices, subsequences = Khiva::Matrix.find_best_n_discords(profile, index, m, n)
+```
+
+## Examples
+
+- [Anomaly Detection](#anomaly-detection)
+- [Similarity Search](#similarity-search)
+
+### Anomaly Detection
+
+Detect anomalies in a time series
+
+```ruby
+# generate a random time series with anomalies from position 100-109
+series = 1000.times.map { |i| i >= 100 && i <= 109 ? 0.5 : rand }
+
+# calculate the matrix profile with subsequence length 10
+a = Khiva::Array.new(series, type: :f32)
+m = 10
+profile, index = Khiva::Matrix.stomp_self_join(a, m)
+
+# find and print the position of the most anomalous subsequence
+_, _, subsequences = Khiva::Matrix.find_best_n_discords(profile, index, m, 1)
+pos = subsequences.to_a.first
+p pos
+```
+
+Use [matplotlib.rb](https://github.com/mrkn/matplotlib.rb) for visualization
+
+```ruby
+require "matplotlib/pyplot"
+plt = Matplotlib::Pyplot
+
+# series
+plt.figure(0)
+plt.title("Series")
+plt.plot(series)
+
+# matrix profile
+plt.figure(1)
+plt.title("Matrix Profile")
+plt.plot(profile.to_a)
+
+# most anomalous subsequence and its closest subsequence
+plt.figure(2)
+plt.title("Subsequences")
+plt.plot(series[pos, m], label: "Anomalous")
+plt.plot(series[index.to_a[pos], m], label: "Closest")
+plt.legend
+```
+
+### Similarity Search
+
+Find a similar pattern in time series
 
 ```ruby
-distances, indices, subsequences = Khiva::Matrix.find_best_n_motifs(profile, index, 2, 2)
+series = Khiva::Array.new([1, 2, 3, 4, 3, 4, 5, 6], type: :f32)
+query = Khiva::Array.new([1, 2, 1], type: :f32)
+distances, indices = Khiva::Matrix.find_best_n_occurrences(query, series, 1)
 ```
 
 ## Modules
@@ -117,12 +178,48 @@ centroids, labels = Khiva::Clustering.k_shape(tss, k)
 
 ## Dimensionality
 
-Piecewise aggregate approximate (PAA)
+Piecewise Aggregate Approximation (PAA)
 
 ```ruby
 Khiva::Dimensionality.paa(a, bins)
 ```
 
+Perceptually Important Points (PIP)
+
+```ruby
+Khiva::Dimensionality.pip(a, number_ips)
+```
+
+Piecewise Linear Approximation (PLA Bottom Up)
+
+```ruby
+Khiva::Dimensionality.pla_bottom_up(a, max_error)
+```
+
+Piecewise Linear Approximation (PLA Sliding Window)
+
+```ruby
+Khiva::Dimensionality.pla_sliding_window(a, max_error)
+```
+
+Ramer-Douglas-Peucker (RDP)
+
+```ruby
+Khiva::Dimensionality.ramer_douglas_peucker(a, epsilon)
+```
+
+Symbolic Aggregate ApproXimation (SAX)
+
+```ruby
+Khiva::Dimensionality.sax(a, alphabet_size)
+```
+
+Visvalingam
+
+```ruby
+Khiva::Dimensionality.visvalingam(a, num_points)
+```
+
 ## Distances
 
 Dynamic time warping (DTW) distance
@@ -175,6 +272,48 @@ Absolute sum of changes
 Khiva::Features.absolute_sum_of_changes(tss)
 ```
 
+Aggregated autocorrelation
+
+```ruby
+Khiva::Features.aggregated_autocorrelation(tss, aggregation_function)
+```
+
+Approximate entropy
+
+```ruby
+Khiva::Features.approximate_entropy(tss, m, r)
+```
+
+Autocorrelation
+
+```ruby
+Khiva::Features.auto_correlation(tss, max_lag, unbiased)
+```
+
+Auto-covariance
+
+```ruby
+Khiva::Features.auto_covariance(tss, unbiased: false)
+```
+
+Binned entropy
+
+```ruby
+Khiva::Features.binned_entropy(tss, max_bins)
+```
+
+Schreiber, T. and Schmitz, A. (1997) measure of non-linearity
+
+```ruby
+Khiva::Features.c3(tss, lag)
+```
+
+Estimate of complexity defined by Batista, Gustavo EAPA, et al (2014)
+
+```ruby
+Khiva::Features.cid_ce(tss, z_normalize)
+```
+
 Number of values above the mean
 
 ```ruby
@@ -187,6 +326,24 @@ Number of values below the mean
 Khiva::Features.count_below_mean(tss)
 ```
 
+Cross-correlation
+
+```ruby
+Khiva::Features.cross_correlation(xss, yss, unbiased)
+```
+
+Cross-covariance
+
+```ruby
+Khiva::Features.cross_covariance(xss, yss, unbiased)
+```
+
+Energy ratio by chunks
+
+```ruby
+Khiva::Features.energy_ratio_by_chunks(arr, num_segments, segment_focus)
+```
+
 The spectral centroid (mean), variance, skew, and kurtosis of the absolute fourier transform spectrum
 
 ```ruby
@@ -205,6 +362,42 @@ First location of the minimum value
 Khiva::Features.first_location_of_minimum(tss)
 ```
 
+Maximum is duplicated
+
+```ruby
+Khiva::Features.has_duplicate_max(tss)
+```
+
+Minimum is duplicated
+
+```ruby
+Khiva::Features.has_duplicate_min(tss)
+```
+
+Any elements are duplicated
+
+```ruby
+Khiva::Features.has_duplicates(tss)
+```
+
+Index of the mass quantile
+
+```ruby
+Khiva::Features.index_mass_quantile(tss, q)
+```
+
+Kurtosis
+
+```ruby
+Khiva::Features.kurtosis(tss)
+```
+
+Standard deviation above threshold
+
+```ruby
+Khiva::Features.large_standard_deviation(tss, r)
+```
+
 Last location of the maximum value
 
 ```ruby
@@ -283,6 +476,54 @@ Minimum
 Khiva::Features.minimum(tss)
 ```
 
+Number of m-crossings
+
+```ruby
+Khiva::Features.number_crossing_m(tss, m)
+```
+
+Number of peaks of at least support `n`
+
+```ruby
+Khiva::Features.number_peaks(tss, n)
+```
+
+Partial autocorrelation
+
+```ruby
+Khiva::Features.partial_autocorrelation(tss, lags)
+```
+
+Percentage of unique values present more than once
+
+```ruby
+Khiva::Features.percentage_of_reoccurring_datapoints_to_all_datapoints(tss, sorted)
+```
+
+Percentage of values present more than once
+
+```ruby
+Khiva::Features.percentage_of_reoccurring_values_to_all_values(tss, sorted)
+```
+
+Quantile
+
+```ruby
+Khiva::Features.quantile(tss, q, precision: 100000000)
+```
+
+Count of values within the interval [min, max]
+
+```ruby
+Khiva::Features.range_count(tss, min, max)
+```
+
+Ratio of values more than `r` sigma away from the mean
+
+```ruby
+Khiva::Features.ratio_beyond_r_sigma(tss, coeff)
+```
+
 Ratio of unique values
 
 ```ruby
@@ -301,18 +542,48 @@ Skewness
 Khiva::Features.skewness(tss)
 ```
 
+Cross power spectral density at different frequencies
+
+```ruby
+Khiva::Features.spkt_welch_density(tss, coeff)
+```
+
 Standard deviation
 
 ```ruby
 Khiva::Features.standard_deviation(tss)
 ```
 
+Sum of all data points present more than once
+
+```ruby
+Khiva::Features.sum_of_reoccurring_datapoints(tss, sorted: false)
+```
+
+Sum of all values present more than once
+
+```ruby
+Khiva::Features.sum_of_reoccurring_values(tss, sorted: false)
+```
+
 Sum of values
 
 ```ruby
 Khiva::Features.sum_values(tss)
 ```
 
+If looks symmetric
+
+```ruby
+Khiva::Features.symmetry_looking(tss, r)
+```
+
+Time reversal asymmetry
+
+```ruby
+Khiva::Features.time_reversal_asymmetry_statistic(tss, lag)
+```
+
 Number of occurrences of a value
 
 ```ruby
@@ -395,6 +666,30 @@ Khiva::Linalg.lls(a, b)
 
 ## Matrix
 
+Find discords
+
+```ruby
+distances, indices, subsequences = Khiva::Matrix.find_best_n_discords(profile, index, m, n)
+```
+
+Find motifs
+
+```ruby
+distances, indices, subsequences = Khiva::Matrix.find_best_n_motifs(profile, index, m, n)
+```
+
+Find best occurences
+
+```ruby
+distances, indices = Khiva::Matrix.find_best_n_occurrences(q, t, n)
+```
+
+Mueen’s Algorithm for Similarity Search (MASS)
+
+```ruby
+distances = Khiva::Matrix.mass(q, t)
+```
+
 Calculate the matrix profile between `ta` and `tb` using a subsequence length of `m` with the STOMP algorithm
 
 ```ruby

data/lib/khiva/dimensionality.rb

@@ -6,6 +6,42 @@ module Khiva
         FFI.call(:paa, a, Utils.int_ptr(bins), result)
         Array.new(result)
       end
+
+      def pip(a, number_ips)
+        result = Utils.create_ptr
+        FFI.call(:pip, a, Utils.int_ptr(number_ips), result)
+        Array.new(result)
+      end
+
+      def pla_bottom_up(a, max_error)
+        result = Utils.create_ptr
+        FFI.call(:pla_bottom_up, a, Utils.float_ptr(max_error), result)
+        Array.new(result)
+      end
+
+      def pla_sliding_window(a, max_error)
+        result = Utils.create_ptr
+        FFI.call(:pla_sliding_window, a, Utils.float_ptr(max_error), result)
+        Array.new(result)
+      end
+
+      def ramer_douglas_peucker(a, epsilon)
+        result = Utils.create_ptr
+        FFI.call(:ramer_douglas_peucker, a, Utils.double_ptr(epsilon), result)
+        Array.new(result)
+      end
+
+      def sax(a, alphabet_size)
+        result = Utils.create_ptr
+        FFI.call(:sax, a, Utils.int_ptr(alphabet_size), result)
+        Array.new(result)
+      end
+
+      def visvalingam(a, num_points)
+        result = Utils.create_ptr
+        FFI.call(:visvalingam, a, Utils.int_ptr(num_points), result)
+        Array.new(result)
+      end
     end
   end
 end

data/lib/khiva/features.rb

@@ -13,6 +13,50 @@ module Khiva
         Array.new(result)
       end
 
+      def aggregated_autocorrelation(arr, aggregation_function)
+        result = Utils.create_ptr
+        FFI.call(:aggregated_autocorrelation, arr, Utils.int_ptr(aggregation_function), result)
+        Array.new(result)
+      end
+
+      # TODO aggregated_linear_trend
+
+      def approximate_entropy(arr, m, r)
+        result = Utils.create_ptr
+        FFI.call(:approximate_entropy, arr, Utils.int_ptr(m), Utils.float_ptr(r), result)
+        Array.new(result)
+      end
+
+      def auto_correlation(arr, max_lag, unbiased)
+        result = Utils.create_ptr
+        FFI.call(:auto_correlation, arr, Utils.long_ptr(max_lag), Utils.bool_ptr(unbiased), result)
+        Array.new(result)
+      end
+
+      def auto_covariance(arr, unbiased: false)
+        result = Utils.create_ptr
+        FFI.call(:auto_covariance, arr, Utils.bool_ptr(unbiased), result)
+        Array.new(result)
+      end
+
+      def binned_entropy(arr, max_bins)
+        result = Utils.create_ptr
+        FFI.call(:binned_entropy, arr, Utils.int_ptr(max_bins), result)
+        Array.new(result)
+      end
+
+      def c3(arr, lag)
+        result = Utils.create_ptr
+        FFI.call(:c3, arr, Utils.long_ptr(lag), result)
+        Array.new(result)
+      end
+
+      def cid_ce(arr, z_normalize)
+        result = Utils.create_ptr
+        FFI.call(:cid_ce, arr, Utils.bool_ptr(z_normalize), result)
+        Array.new(result)
+      end
+
       def count_above_mean(arr)
         result = Utils.create_ptr
         FFI.call(:count_above_mean, arr, result)
@@ -25,12 +69,34 @@ module Khiva
         Array.new(result)
       end
 
+      def cross_correlation(xss, yss, unbiased)
+        result = Utils.create_ptr
+        FFI.call(:cross_correlation, xss, yss, Utils.bool_ptr(unbiased), result)
+        Array.new(result)
+      end
+
+      def cross_covariance(xss, yss, unbiased)
+        result = Utils.create_ptr
+        FFI.call(:cross_covariance, xss, yss, Utils.bool_ptr(unbiased), result)
+        Array.new(result)
+      end
+
+      # TODO cwt_coefficients
+
+      def energy_ratio_by_chunks(arr, num_segments, segment_focus)
+        result = Utils.create_ptr
+        FFI.call(:energy_ratio_by_chunks, arr, Utils.long_ptr(num_segments), Utils.long_ptr(segment_focus), result)
+        Array.new(result)
+      end
+
       def fft_aggregated(arr)
         result = Utils.create_ptr
         FFI.call(:fft_aggregated, arr, result)
         Array.new(result)
       end
 
+      # TODO fft_coefficient
+
       def first_location_of_maximum(arr)
         result = Utils.create_ptr
         FFI.call(:first_location_of_maximum, arr, result)
@@ -43,6 +109,42 @@ module Khiva
         Array.new(result)
       end
 
+      def has_duplicate_max(arr)
+        result = Utils.create_ptr
+        FFI.call(:has_duplicate_max, arr, result)
+        Array.new(result)
+      end
+
+      def has_duplicate_min(arr)
+        result = Utils.create_ptr
+        FFI.call(:has_duplicate_min, arr, result)
+        Array.new(result)
+      end
+
+      def has_duplicates(arr)
+        result = Utils.create_ptr
+        FFI.call(:has_duplicates, arr, result)
+        Array.new(result)
+      end
+
+      def index_mass_quantile(arr, q)
+        result = Utils.create_ptr
+        FFI.call(:index_mass_quantile, arr, Utils.float_ptr(q), result)
+        Array.new(result)
+      end
+
+      def kurtosis(arr)
+        result = Utils.create_ptr
+        FFI.call(:kurtosis, arr, result)
+        Array.new(result)
+      end
+
+      def large_standard_deviation(arr, r)
+        result = Utils.create_ptr
+        FFI.call(:large_standard_deviation, arr, Utils.float_ptr(r), result)
+        Array.new(result)
+      end
+
       def last_location_of_maximum(arr)
         result = Utils.create_ptr
         FFI.call(:last_location_of_maximum, arr, result)
@@ -121,6 +223,56 @@ module Khiva
         Array.new(result)
       end
 
+      def number_crossing_m(arr, m)
+        result = Utils.create_ptr
+        FFI.call(:number_crossing_m, arr, Utils.int_ptr(m), result)
+        Array.new(result)
+      end
+
+      # TODO number_cwt_peaks
+
+      def number_peaks(arr, n)
+        result = Utils.create_ptr
+        FFI.call(:number_peaks, arr, Utils.int_ptr(n), result)
+        Array.new(result)
+      end
+
+      def partial_autocorrelation(arr, lags)
+        result = Utils.create_ptr
+        FFI.call(:partial_autocorrelation, arr, lags, result)
+        Array.new(result)
+      end
+
+      def percentage_of_reoccurring_datapoints_to_all_datapoints(arr, sorted)
+        result = Utils.create_ptr
+        FFI.call(:percentage_of_reoccurring_datapoints_to_all_datapoints, arr, Utils.bool_ptr(sorted), result)
+        Array.new(result)
+      end
+
+      def percentage_of_reoccurring_values_to_all_values(arr, sorted)
+        result = Utils.create_ptr
+        FFI.call(:percentage_of_reoccurring_values_to_all_values, arr, Utils.bool_ptr(sorted), result)
+        Array.new(result)
+      end
+
+      def quantile(arr, q, precision: 100000000)
+        result = Utils.create_ptr
+        FFI.call(:quantile, arr, q, Utils.float_ptr(precision), result)
+        Array.new(result)
+      end
+
+      def range_count(arr, min, max)
+        result = Utils.create_ptr
+        FFI.call(:range_count, arr, Utils.float_ptr(min), Utils.float_ptr(max), result)
+        Array.new(result)
+      end
+
+      def ratio_beyond_r_sigma(arr, r)
+        result = Utils.create_ptr
+        FFI.call(:ratio_beyond_r_sigma, arr, Utils.float_ptr(r), result)
+        Array.new(result)
+      end
+
       def ratio_value_number_to_time_series_length(arr)
         result = Utils.create_ptr
         FFI.call(:ratio_value_number_to_time_series_length, arr, result)
@@ -139,18 +291,48 @@ module Khiva
         Array.new(result)
       end
 
+      def spkt_welch_density(arr, coeff)
+        result = Utils.create_ptr
+        FFI.call(:spkt_welch_density, arr, Utils.int_ptr(coeff), result)
+        Array.new(result)
+      end
+
       def standard_deviation(arr)
         result = Utils.create_ptr
         FFI.call(:standard_deviation, arr, result)
         Array.new(result)
       end
 
+      def sum_of_reoccurring_datapoints(arr, sorted: false)
+        result = Utils.create_ptr
+        FFI.call(:sum_of_reoccurring_datapoints, arr, Utils.bool_ptr(sorted), result)
+        Array.new(result)
+      end
+
+      def sum_of_reoccurring_values(arr, sorted: false)
+        result = Utils.create_ptr
+        FFI.call(:sum_of_reoccurring_values, arr, Utils.bool_ptr(sorted), result)
+        Array.new(result)
+      end
+
       def sum_values(arr)
         result = Utils.create_ptr
         FFI.call(:sum_values, arr, result)
         Array.new(result)
       end
 
+      def symmetry_looking(arr, r)
+        result = Utils.create_ptr
+        FFI.call(:symmetry_looking, arr, Utils.float_ptr(r), result)
+        Array.new(result)
+      end
+
+      def time_reversal_asymmetry_statistic(arr, lag)
+        result = Utils.create_ptr
+        FFI.call(:time_reversal_asymmetry_statistic, arr, Utils.int_ptr(lag), result)
+        Array.new(result)
+      end
+
       def value_count(arr, v)
         result = Utils.create_ptr
         FFI.call(:value_count, arr, Utils.float_ptr(v), result)

data/lib/khiva/matrix.rb

@@ -17,6 +17,19 @@ module Khiva
         [Array.new(motif_distances), Array.new(motif_indices), Array.new(subsequence_indices)]
       end
 
+      def find_best_n_occurrences(q, t, n)
+        distances = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP)
+        indices = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP)
+        FFI.call(:find_best_n_occurrences, q, t, n, distances, indices)
+        [Array.new(distances), Array.new(indices)]
+      end
+
+      def mass(q, t)
+        distances = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP)
+        FFI.call(:mass, q, t, distances)
+        Array.new(distances)
+      end
+
       def stomp(tssa, tssb, m)
         profile = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP)
         index = Fiddle::Pointer.malloc(Fiddle::SIZEOF_VOIDP)

data/lib/khiva/version.rb

@@ -1,3 +1,3 @@
 module Khiva
-  VERSION = "0.1.1"
+  VERSION = "0.1.2"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: khiva
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.1.2
 platform: ruby
 authors:
 - Andrew Kane
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-12-18 00:00:00.000000000 Z
+date: 2020-12-20 00:00:00.000000000 Z
 dependencies: []
 description:
 email: andrew@chartkick.com