cotcube-level 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: c1c9fc6eb075ccc486c29c13d9989e52b5e659330cba2b900bdd4f420d085bb8
+  data.tar.gz: 6303babcc58fd8def53c747174c0feaa669c848787c0c32eaa811e0f76aad61f
+SHA512:
+  metadata.gz: a5bc9e406caecd100ad2aebaf5934f9c2b6f0b3c43dc51aa76b63575bb52bd77fec7b06f5b183801149f6b35fb367462261e93bb2c174ef8b10b29bb61ee2370
+  data.tar.gz: b1de063e02b133b76aea5ffe01b357fad9b9ed99461225cf8780b7cc592ea00bd043afef6d957a3a2f78db02e75291bfac775aa2c56c3cd36886378fbd7f7767

data/.irbrc.rb

@@ -0,0 +1,12 @@
+def verbose_toggle
+  irb_context.echo ? irb_context.echo = false : irb_context.echo = true
+end
+
+alias vt verbose_toggle
+
+$debug = true
+IRB.conf[:USE_MULTILINE] = false
+# require 'bundler'
+# Bundler.require
+
+require_relative 'lib/cotcube-level'

data/CHANGELOG.md

@@ -0,0 +1,14 @@
+## 0.2.0 (August 17, 2021)
+  - cotcube-level: added new module_functions, added restrictive constants for intervals and swaptypes
+  - adding new features to so-called 'test suite'
+  - triangulate: added saving/caching, added rejection of base data older than abs_peak, utilized new output helpers, added analyzation lambda
+  - helpers: added output helpers and save / load
+  - fixed License definition
+  - added current stuff to main module loader 'cotcube-level.rb'
+  - added (copied from legacy swapseeker) and adapted module functions for swap detection (shearing, detect_slope, triangulate)
+  - added (copied from legacy swapseeker) and adapted stencil model
+  - added trivial testsuite for days, which contains loading of example data, stencil generation and slope detection
+
+## 0.1.0 (May 07, 2021)
+  - initial commit
+

data/Gemfile

@@ -0,0 +1,5 @@
+source "https://rubygems.org"
+
+# Specify your gem's dependencies in bitangent.gemspec
+gemspec
+

data/README.md

@@ -0,0 +1,49 @@
+## Cotcube::Level
+
+When naming this gem, I had in mind the German level a.k.a spirit level. But naming was not quite in the beginning
+of the development process. All in the beginning was the wish to create an algorithm to automatically draw trend
+lines. So for like 2 year I was whirling my head on how to put an algorithmic ruler on a chart and rotate it until
+one or a set of trend lines are concise outpout. 
+
+As most hard do accomplish thing turn out to be much easier when you put them up side down, same happend to me in this
+matter. I found rotating the ruler is too hard for, but instead transforming (shearing) the chart itself while keeping
+the ruler at its _level_ is much more eligible.
+
+The idea and the development of the algorithm had taken place within another Cotcube project named 'SwapSeeker'. At
+some point the SwapSeeker has become too complex so I decided to decouple the Level and the Stencil as independent
+functional unit, that can be used on arbitrary time series.
+
+### The shear mapping
+
+There is really no magic in it. The timeseries (or basically an interval of a timeseries) needs to be prepared to locate
+in Cartesian Quadrant I with fitting x==0 to y==0, and then a binary search on shearing angles determines the resulting 
+muiltitangent of which as the origin ('now') one point already is given. As limitation only shearing between 0 and 90
+deg is supported. 
+
+The result contains 
+
+- only the origin, if it is the absolute high (resp. low) within the provided interval of the time series. This happens
+to happen if _deg -> 0_.
+- the origin and one arbitrary point, the usual result.
+- the origin and two or more points residing on the same level--what is the desired result showing mathematical accuracy where 
+human eye cannot detect it in time. 
+
+### The stencil
+
+The shearing transformation is based on _x_ and _y_ values, where y obviously are the values of the series while _x_
+refers to the time. It turned out a much bigger challenge to create an expedient mapping from DateTime to Integer,
+where, as you foresee, 'now' should result in _x.zero?_. 
+
+## Usage
+
+TODO: Write usage instructions here
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/BSD-3-Clause).

data/VERSION

@@ -0,0 +1 @@
+0.2.0

data/cotcube-level.gemspec

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+Gem::Specification.new do |spec|
+  spec.name          = 'cotcube-level'
+  spec.version       = File.read("#{__dir__}/VERSION")
+  spec.authors       = ['Benjamin L. Tischendorf']
+  spec.email         = ['donkeybridge@jtown.eu']
+
+  spec.summary       = 'A gem to shear a time series '
+  spec.description   = 'A gem to shear a time series, basically serving as a yet unssen class of indicators.'
+
+  spec.homepage      = 'https://github.com/donkeybridge/' + spec.name
+  spec.license       = 'BSD-3-Clause'
+  spec.required_ruby_version = Gem::Requirement.new('~> 2.7')
+
+  spec.metadata['homepage_uri'] = spec.homepage
+  spec.metadata['source_code_uri'] = spec.homepage
+  spec.metadata['changelog_uri'] = spec.homepage + '/CHANGELOG.md'
+
+  # Specify which files should be added to the gem when it is released.
+  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
+  spec.files = Dir.chdir(File.expand_path(__dir__)) do
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  end
+  spec.bindir        = 'bin'
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ['lib']
+
+  spec.add_dependency 'activesupport',      '~> 6'
+  spec.add_dependency 'colorize',           '~> 0.8'
+  spec.add_dependency 'cotcube-helpers',    '~> 0.1'
+  spec.add_dependency 'yaml',               '~> 0.1'
+
+  spec.add_development_dependency 'rake',   '~> 13'
+  spec.add_development_dependency 'rspec', '~>3.6'
+  spec.add_development_dependency 'yard', '~>0.9'
+end

data/lib/cotcube-level.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+#
+
+require 'active_support'
+require 'active_support/core_ext/time'
+require 'active_support/core_ext/numeric'
+require 'colorize'
+require 'date'    unless defined?(DateTime)
+require 'csv'     unless defined?(CSV)
+require 'yaml'    unless defined?(YAML)
+require 'json'    unless defined?(JSON)
+require 'digest'  unless defined?(Digest)
+require 'cotcube-helpers'
+
+# require_relative 'cotcube-level/filename
+
+%w[ stencil detect_slope triangulate helpers].each do |part| 
+  require_relative "cotcube-level/#{part}"
+end
+
+
+
+module Cotcube
+  module Level
+
+    PRECISION = 7
+    INTERVALS = %i[ daily ]
+    SWAPTYPES = %i[ full ]
+    #module_function :init, # checks whether environment is prepared and returns the config hash
+    module_function :detect_slope,
+                    :shear_to_deg,
+                    :shear_to_rad,
+                    :rad2deg,
+                    :deg2rad,
+                    :puts_swaps,
+                    :save_swaps,
+                    :get_jsonl_name,
+                    :load_swaps,
+                    :member_to_human,
+                    :triangulate
+    
+    # please note that module_functions of sources provided in private files must slso be published within these
+  end
+end
+

data/lib/cotcube-level/detect_slope.rb

@@ -0,0 +1,117 @@
+module Cotcube
+  module Level
+    #
+      # TODO: add support for slopes not only exactly matching but also allow 'dip' of n x ticksize
+      def detect_slope(base:, max: 90, debug: false, format: '% 5.2f', calculus: false, ticksize: nil, max_dev: 200)
+        raise ArgumentError, "'0 < max < 90, but got '#{max}'" unless max.is_a? Numeric and 0 < max and max <= 90
+        #
+        # aiming for a shearing angle, all but those in a line below the abscissa
+        #
+        # doing a binary search starting at part = 45 degrees
+        # on each iteration,
+        #   part is halved and added or substracted based on current success
+        #   if more than the mandatory result is found, all negative results are removed and degrees are increased by part
+        #
+        raise ArgumentError, 'detect_slope needs param Array :base' unless base.is_a? Array
+
+        # from given base, choose non-negative stencil containing values
+        old_base = base.dup.select{|b| b[:x] >= 0 and not b[:y].nil? }
+
+        # some debug output
+        old_base.each {|x| p x} if old_base.size < 50 and debug
+
+        # set initial shearing angle if not given as param
+        deg ||= -max / 2.0 
+
+        # create first sheering. please note how selection working with d[:yy]
+        new_base = shear_to_deg(base: old_base, deg: deg).select { |d| d[:yy] >= 0 } #-ticksize }
+
+        # debug output
+        puts "Iterating slope:\t#{format '% 7.5f',deg
+                           }\t\t#{new_base.size
+                           } || #{new_base.values_at(*[0]).map{|f| "'#{f[:x]
+                                                                    } | #{format format,f[:y]
+                                                                    } | #{format format,f[:yy]}'"}.join(" || ") }" if debug
+        # set initial part to deg
+        part = deg.abs
+        #
+        # the loop, that runs until either
+        #   - only two points are left on the slope
+        #   - the slope has even angle
+        #   - several points are on the slope in quite a good approximation ('round(7)')
+        #
+        until deg.round(PRECISION).zero? || part.round(PRECISION).zero? ||
+            ((new_base.size >= 2) && (new_base.map { |f| f[:yy].round(PRECISION).zero? }.uniq.size == 1))
+
+          part /= 2.0
+          if new_base.size == 1
+            # the graph was sheared too far, reuse old_base
+            deg = deg + part
+          else
+            # the graph was sheared too short, continue with new base
+            deg = deg - part
+            old_base = new_base.dup unless deg.round(PRECISION).zero?
+          end
+
+          # the actual sheering operation
+          # note that this basically maps old_base with yy = y + (dx||x * tan(deg) )
+          #
+          new_base = shear_to_deg(base: old_base, deg: deg).select { |d| d[:yy] >= 0 } #-ticksize }
+          new_base.last[:dx] = 0.0 
+          if debug
+            print " #{format '% 8.5f',deg}"
+            puts "Iterating slope:\t#{format '% 8.5f',deg
+                                 }\t#{new_base.size
+                               } || #{new_base.values_at(*[0]).map{|f| "'#{f[:x]
+                                                                     } | #{format '%4.5f', part
+                                                                     } | #{format format,f[:y]
+                                                                     } | #{format format,f[:yy]}'"}.join(" || ") }"
+          end
+        end
+        puts ' done.' if debug
+
+        ### Sheering ends here
+
+        # define the approximited result as (also) 0.0
+        new_base.each{|x| x[:yy] = 0.0}
+        if debug
+          puts "RESULT: #{deg} #{deg2rad(deg)}"
+          new_base.each {|f| puts "\t#{f.inspect}" }
+        end
+        # there is speacial treatment for even slopes
+        if deg.round(PRECISION).zero?
+          #puts "found even slope"
+          # this is intentionally voided as evenness is calculated somewhere else
+          # even_base = base.dup.select{|b| b[:x] >= 0 and not b[:y].nil? }[-2..-1].map{|x| x.dup}
+          # last_barrier is the last bar, that exceeds
+          #binding.irb
+          #last_barrier = even_base.select{|bar| (bar[:y] - even_base.last[:y]).abs > evenness * ticksize}.last
+          #even_base.select!{|bar| (bar[:y] - even_base.last[:y]).abs <= evenness * ticksize}
+          # could be, that no last barrier exists, when there is a top or bottom plateau
+          #even_base.select!{|bar| bar[:x] < last_barrier[:x]} unless last_barrier.nil?
+          # TODO
+          return { deg: 0,  slope: 0, members: [] } #, members: even_base.map { |x| xx = x.dup; %i[y yy].map { |z|  xx[z]=nil }; xx } })
+        end
+
+
+        #####################################################################################
+        # Calculate the slope bsaed on the angle that resulted above
+        #     y = m x + n -->
+        #                     m = delta-y / delta-x
+        #                     n = y0 - m * x0
+        #
+        slope     = (new_base.first[:y] - new_base.last[:y]) / (
+                        (new_base.first[:dx].nil? ? new_base.first[:x] : new_base.first[:dx]).to_f - 
+                        (new_base. last[:dx].nil? ? new_base. last[:x] : new_base. last[:dx]).to_f 
+                    )
+        # the result
+        {
+          deg:        deg,
+          slope:      slope,
+          members:   new_base.map { |x| x.dup }
+        }
+      end
+  end
+end
+
+

data/lib/cotcube-level/helpers.rb

@@ -0,0 +1,100 @@
+# frozen_string_literal: true
+
+module Cotcube
+  module Level
+      def rad2deg(deg)
+        deg * 180 / Math::PI
+      end
+
+      def deg2rad(rad)
+        rad * Math::PI / 180
+      end
+
+      def shear_to_deg(base:, deg:)
+        shear_to_rad(base: base, rad: deg2rad(deg))
+      end
+
+      def shear_to_rad(base: , rad:)
+        tan = Math.tan(rad)
+        base.map { |bar|
+          # separating lines for easier debugging
+          bar[:yy] =
+            bar[:y] +
+            (bar[:dx].nil? ? bar[:x] : bar[:dx]) * tan
+          bar
+        }
+      end
+
+      def member_to_human(member,side: ,format:)
+        high = side == :high
+             "#{member[:datetime].strftime("%a, %Y-%m-%d %H:%M")
+        }  x: #{format '%-4d', member[:x]
+        } dx: #{format '%-8.3f', (member[:dx].nil? ? member[:x] : member[:dx].round(3))
+            } #{high ? "high" : "low"
+           }: #{format format, member[high ? :high : :low]
+         } i: #{(format '%4d', member[:i]) unless member[:i].nil?
+            } #{member[:miss].nil? ? '' : "miss: #{member[:miss]}" }"
+      end
+
+      def puts_swaps(swaps, format: )
+        swaps = [ swaps ] unless swaps.is_a? Array
+        swaps.each do |swap|
+          puts "side: #{swap[:side] }\tlen: #{swap[:length]}  \trating: #{swap[:rating]}".colorize(swap[:color] || :white )
+          puts "diff: #{swap[:ticks]}\tdif: #{swap[:diff].round(7)}\tdepth: #{swap[:depth]}".colorize(swap[:color] || :white )
+          puts "tpi:  #{swap[:tpi]  }\tppi: #{swap[:ppi]}".colorize(swap[:color] || :white )
+          swap[:members].each {|x| puts member_to_human(x, side: swap[:side], format: format) }
+        end
+      end
+
+      def get_jsonl_name(interval:, swap_type:, contract:, sym: nil)
+        raise "Interval #{interval } is not supported, please choose from #{INTERVALS}" unless INTERVALS.include? interval
+        raise "Swaptype #{swap_type} is not supported, please choose from #{SWAPTYPES}" unless SWAPTYPES.include? swap_type
+        sym ||= Cotcube::Helpers.get_id_set(contract: contract)
+        root = '/var/cotcube/level'
+        dir     = "#{root}/#{sym[:id]}"
+        symlink = "#{root}/#{sym[:symbol]}"
+        `mkdir -p #{dir}`         unless File.exist?(dir)
+        `ln -s #{dir} #{symlink}` unless File.exist?(symlink)
+        file = "#{dir}/#{contract}_#{interval.to_s}_#{swap_type.to_s}.jsonl"
+        `touch #{file}`
+        file
+      end
+
+      def save_swaps(swaps, interval:, swap_type:, contract:, sym: nil)
+        file = get_jsonl_name(interval: interval, swap_type: swap_type, contract: contract, sym: sym)
+        swaps.each do |swap|
+          swap_json = swap.to_json
+          digest = Digest::SHA256.hexdigest swap_json
+          res = `cat #{file} | grep #{digest}`.strip
+          unless res.empty?
+            puts "Cannot save swap, it is already in #{file}:"
+            p swap
+          else
+            swap[:digest] = digest
+            File.open(file, 'a+'){|f| f.write(swap.to_json + "\n") }
+          end
+        end
+      end
+
+      def load_swaps(interval:, swap_type:, contract:, sym: nil)
+        file = get_jsonl_name(interval: interval, swap_type: swap_type, contract: contract, sym: sym)
+        jsonl = File.read(file)
+        jsonl.
+          each_line.
+          map do |x|
+          JSON.parse(x).
+            deep_transform_keys(&:to_sym).
+            tap do |sw|
+            sw[:datetime] = DateTime.parse(sw[:datetime]) rescue nil
+            sw[:side]     = sw[:side].to_sym
+            unless sw[:empty]
+              sw[:color]    = sw[:color].to_sym
+              sw[:members].map{|mem| mem[:datetime] = DateTime.parse(mem[:datetime]) }
+            end
+          end
+        end
+      end
+  end
+
+end
+

data/lib/cotcube-level/stencil.rb

@@ -0,0 +1,150 @@
+# frozen_string_literal: true
+
+module Cotcube
+  module Level
+
+    class Stencil
+      attr_accessor :base
+      attr_reader   :interval
+
+
+      # Class method that loads the (latest) obligatory stencil for given interval and type.
+      # These raw stencils are located in /var/cotcube/level/stencils
+      #
+      # Current daily stencils contain dates from 2020-01-01 to 2023-12-31
+      #
+      def self.provide_raw_stencil(type:, interval: :daily, version: nil)
+        loading = lambda do |typ|
+          file_base = "/var/cotcube/level/stencils/stencil_#{interval.to_s}_#{typ.to_s}.csv_"
+          if Dir["#{file_base}?*"].empty?
+            raise ArgumentError, "Could not find any stencil matching interval #{interval} and type #{typ}. Check #{file_base} manually!"
+          end
+          if version.nil? # use latest available version if not given
+            file = Dir["#{file_base}?*"].sort.last
+          else
+            file = "#{file_base}#{version}"
+            unless File.exist? file
+              raise ArgumentError, "Cannot open stencil from non-existant file #{file}."
+            end
+          end
+          CSV.read(file).map{|x| { datetime: CHICAGO.parse(x.first).freeze, x: x.last.to_i.freeze } }
+        end
+        unless const_defined? :RAW_STENCILS
+          const_set :RAW_STENCILS, { daily:
+                                     { full: loading.call( :full).freeze,
+                                       rtc:  loading.call( :rtc).freeze
+            }.freeze
+          }.freeze
+        end
+        RAW_STENCILS[interval][type].map{|x| x.dup}
+      end
+
+      def initialize(
+        range: nil,                 # used to shrink the stencil size, accepts String or Date
+        interval:,
+        swap_type:,
+        ranges: nil,                # currently not used, prepared to be used in connection intraday
+        contract: nil,
+        date: nil,
+        debug: false,
+        version: nil,               # when referring to a specicic version of the stencil
+        timezone: CHICAGO,
+        stencil: nil,               # instead of loading, use this data
+        #config: init,
+        warnings: true
+      )
+        @debug     = debug
+        @interval  = interval
+        @swap_type = swap_type
+        @swaps     = []
+        @contract  = contract
+        @warnings = warnings
+        step =  case @interval
+                when :hours, :hour; 1.hour
+                when :quarters, :quarter; 15.minutes
+                else; 1.day
+                end
+
+        case @interval
+        when :day, :days, :daily, :dailies, :synth, :synthetic #, :week, :weeks, :month, :months
+          unless range.nil?
+            starter = range.begin.is_a?(String) ? timezone.parse(range.begin) : range.begin
+            ender   = range.  end.is_a?(String) ? timezone.parse(range.  end) : range.  end
+          end
+
+          stencil_type = case swap_type
+                       when :rth
+                         :full
+                       when :rthc
+                         :rtc
+                       else
+                         swap_type
+                       end
+          # TODO: Check / warn / raise whether stencil (if provided) is a proper data type
+          raise ArgumentError, "Stencil should be nil or Array" unless [NilClass, Array].include? stencil.class
+          raise ArgumentError, "Each stencil members should contain at least :datetime and :x" unless stencil.nil? or
+            stencil.map{|x| ([:datetime, :x] - x.keys).empty? and [ActiveSupport::TimeWithZone, Day].include?( x[:datetime] ) and x[:x].is_a?(Integer)}.reduce(:&)
+
+          base = stencil || Stencil.provide_raw_stencil(type: stencil_type, interval: :daily, version: version)
+
+          # fast forward to prev trading day
+          date = CHICAGO.parse(date) unless [NilClass, Date, ActiveSupport::TimeWithZone].include? date.class
+          @date = date || Date.today
+          best_match = base.select{|x| x[:datetime].to_date <= @date}.last[:datetime]
+          @date  = best_match
+
+          offset = base.map{|x| x[:datetime]}.index(@date)
+
+          # apply offset to stencil, so zero will match today (or what was provided as 'date')
+          @base = base.map.
+            each_with_index{|d,i| d[:x] = (offset - i).freeze; d }
+          # if range was given, shrink stencil to specified range
+          @base.select!{|d| (d[:datetime] >= starter and d[:datetime] <= ender) } unless range.nil?
+        else
+          raise RuntimeError, "'interval: #{interval}' was provided, what does not match anything this tool can handle (currently :days, :dailies, :synthetic)."
+        end
+      end
+
+      def dup
+        Stencil.new(
+          debug:      @debug,
+          interval:   @interval,
+          swap_type:  @swap_type,
+          date:       @date,
+          contract:   @contract,
+          stencil:    @base.map{|x| x.dup}
+        )
+      end
+
+      def zero
+        @zero ||=  @base.find{|b| b[:x].zero? }
+      end
+
+      def apply(to: )
+        offset = 0 
+        @base.each_index do |i| 
+          begin
+            offset += 1 while to[i+offset][:datetime].to_date < @base[i][:datetime].to_date
+          rescue
+            # appending
+            to << @base[i]
+            next
+          end
+          if to[i+offset][:datetime].to_date > @base[i][:datetime].to_date
+            # skipping 
+            offset -= 1
+            next
+          end
+          # merging
+          to[i+offset][:x] = @base[i][:x]
+        end
+        # finally remove all bars that do not belong to the stencil (i.e. holidays)
+        to.reject!{|x| x[:x].nil? }
+      end
+
+    end
+
+  end
+
+end
+

data/lib/cotcube-level/triangulate.rb

@@ -0,0 +1,238 @@
+module Cotcube
+  module Level
+      def triangulate(
+          contract: nil,        # contract actually isnt needed to triangulation, but allows much more convenient output
+          side:,                # :upper or :lower
+          base:,                # the base of a readily injected stencil
+          range: (0..-1),       # range is relative to base
+          max: 90,              # the range which to scan for swaps goes from deg 0 to max
+          debug: false,
+          format: '% 5.2f',
+          min_members: 3,       # this param should not be changed manually, it is used for the guess operation
+          min_rating: 3,        # swaps having a lower rating are discarded
+          allow_sub: true,      # this param determines whether guess can be called or not
+          save: true,           # allow saving of results
+          cached: true,         # allow loading of yet  cached intervals
+          interval: nil,        # interval and swap_type are only needed if saving / caching of swaps is desired
+          swap_type: nil,       #      if not given, a warning is printed and swaps are not saved
+          deviation: 2)
+
+        raise ArgumentError, "'0 < max < 90, but got '#{max}'" unless max.is_a? Numeric and 0 < max and max <= 90
+        raise ArgumentError, 'need :side either :upper or :lower for dots' unless [:upper, :lower].include? side
+
+        ###########################################################################################################################
+        # init some helpers
+        #
+        high       = side == :upper
+        first      = base.to_a.find{|x|  not x[:high].nil? }
+        zero       = base.select{|x| x[:x].zero? }
+        raise ArgumentError, "Inappropriate base, it should contain ONE :x.zero, but contains #{zero.size}." unless zero.size==1
+        zero       = zero.first
+        contract ||= zero.contract
+        sym = Cotcube::Helpers.get_id_set(contract: contract)
+
+        if cached
+          if interval.nil? or swap_type.nil?
+            puts "Warning: Cannot use cache as both :interval and :swap_type must be given".light_yellow
+          else
+            cache = load_swaps(interval: interval, swap_type: swap_type, contract: contract, sym: sym)
+            selected = cache.select{|sw| sw[:datetime] == zero[:datetime] and sw[:side] == side}
+            unless selected.empty?
+              puts 'cache_hit'.light_white if debug
+              return (selected.first[:empty] ? [] : selected )
+            end
+          end
+        end
+        ticksize = sym[:ticksize]  / sym[:bcf] # need to adjust, as we are working on barchart data, not on exchange data !!
+
+
+        ###########################################################################################################################
+        # prepare base (i.e. dupe the original, create proper :y, and reject unneeded items)
+        #
+        base = base.
+          map { |x|
+            y = x.dup
+            y[:y] = (high ?
+                     (y[:high] - zero[:high]).round(8) :
+                     (zero[:low] - y[:low]).round(8)
+                    ) unless y[:high].nil?
+            y
+          }.
+          reject{|b| b.nil? or b[:datetime] < first[:datetime] or b[:x] < 0 or b[:y].nil?}[range]
+        abs_peak = base.send(high ? :max_by : :min_by){|x| x[high ? :high : :low] }[:datetime]
+
+        base.reject!{|x| x[:datetime] < abs_peak}
+
+        ###########################################################################################################################z
+        # only if (and only if) the range portion above change the underlying base
+        #   the offset has to be fixed for :x and :y
+
+        unless range == (0..-1)
+          puts "adjusting range to '#{range}'".light_yellow if debug
+          offset_x = base.last[:x]
+          offset_y = base.last[:y]
+          base.map!{|b| b[:x] -= offset_x; b[:y] -= offset_y  ; b}
+        end
+        base.each_index.map{|i| base[i][:i] = -base.size + i }
+
+
+        ###########################################################################################################################
+        # LAMBDA no1: simplifying DEBUG output
+        #
+        present = lambda {|z| swap_to_human(z) }
+
+
+
+        ###########################################################################################################################
+        # LAMBDA no2: all members except the pivot itself now most probably are too far to the left
+        #             finalizing tries to get the proper dx value for them
+        #
+        finalize = lambda do |res|
+          res.map do |r|
+            r[:members].each  do |m|
+              next if m[:yy].nil? or m[:yy].zero?
+
+              diff = (m[:x] - m[:dx]).abs / 2.0
+              m[:dx] = m[:x] + diff
+              # it employs another binary-search
+              while m[:yy].round(PRECISION) != 0 # or m[:yy].round(PRECISION) < -ticksize
+                print '.' if debug
+                m[:yy] = shear_to_deg(deg: r[:deg], base: [ m ] ).first[:yy]
+                diff /= 2.0
+                if m[:yy] > 0
+                  m[:dx] += diff
+                else
+                  m[:dx] -= diff
+                end
+              end
+              m[:yy] = m[:yy].abs.round(8)
+            end # r.members
+            puts 'done!'.light_yellow if debug
+
+            r[:members].each {|x| puts "finalizing #{x}".magenta } if debug
+            ## The transforming part
+
+            r
+	  end    # res
+	end      # lambda
+
+        ###########################################################################################################################
+	# LAMDBA no3:  the actual 'function' to retrieve the slope
+        #
+	get_slope = lambda do |b|
+	  if debug
+            puts "in get_slope ... SETTING BASE: ".light_green
+	    puts "Last:\t#{present.call  b.last}"
+	    puts "First:\t#{present.call b.first}"
+	  end
+	  members = [ b.last[:i] ]
+	  loop do
+	    current_slope   = detect_slope(base: b, ticksize: ticksize, format: sym[:format], debug: debug)
+            current_members = current_slope[:members]
+	      .map{|dot| dot[:i]}
+	    new_members = current_members - members
+	    puts "New members: #{new_members} as of #{current_members} - #{members}" if debug
+	    # the return condition is if no new members are found in slope
+	    # except lowest members are neighbours, what causes a re-run
+	    if new_members.empty? 
+	      mem_sorted=members.sort
+	      if mem_sorted[1] == mem_sorted[0] + 1
+                b2 = b[mem_sorted[1]..mem_sorted[-1]].map{|x| x.dup; x[:dx] = nil; x}
+                puts 'starting rerun' if debug
+                alternative_slope = get_slope.call(b2)
+                alternative = alternative_slope[:members].map{|bar| bar[:i]}
+		if (mem_sorted[1..-1] - alternative).empty?
+		  current_slope = alternative_slope
+		  members = alternative
+		end
+	      end
+              if min_members >= 3 and members.size >= 3
+                current_slope[:raw]    = members.map{|x| x.abs }.sort
+                current_slope[:length] = current_slope[:raw][-1] - current_slope[:raw][0]
+                current_slope[:rating] = current_slope[:raw][1..-2].map{|dot| [ dot - current_slope[:raw][0], current_slope[:raw][-1] - dot].min }.max
+              end
+              members.sort_by{|i| -i}.each do |x|
+                puts "#{range}\t#{present.call(b[x])}" if debug
+                current_slope[:members] << b[x] unless current_slope[:members].map{|x| x[:datetime]}.include? b[x][:datetime]
+                current_slope[:members].sort_by!{|x| x[:datetime]}
+              end
+              return current_slope
+
+            end
+            new_members.each do |mem|
+              current_deviation = (0.1 * b[mem][:x])
+              current_deviation =  1                  if current_deviation < 1
+              current_deviation =  deviation          if current_deviation > deviation
+              b[mem][:dx] = b[mem][:x] + current_deviation
+            end
+            members += new_members
+          end
+        end # of lambda
+
+        analyze = lambda do |swaps|
+          swaps.each do |swap|
+            swap[:datetime] = swap[:members].last[:datetime]
+            swap[:side]   = side
+            rat = swap[:rating]
+            swap[:color ] = (rat > 75) ? :light_blue : (rat > 30) ? :magenta : (rat > 15) ? :light_magenta : (rat > 7) ? (high ? :light_green : :light_red) : high ? :green : :red
+            swap[:diff]   = swap[:members].last[ high ? :high : :low ] - swap[:members].first[ high ? :high : :low ]
+            swap[:ticks]  = (swap[:diff] / sym[:ticksize]).to_i
+            swap[:tpi]    = (swap[:ticks].to_f / swap[:length]).round(3)
+            swap[:ppi]    = (swap[:tpi] * sym[:power]).round(3)
+            swap_base     = shear_to_deg(base: base[swap[:members].first[:i]..], deg: swap[:deg]).map{|x| x[:dev] = (x[:yy] / sym[:ticksize]).abs.floor; x}
+            swap[:depth]  = swap_base.max_by{|x| x[:dev]}[:dev]
+            swap[:avg_dev]= (swap_base.reject{|x| x[:dev].zero?}.map{|x| x[:dev]}.reduce(:+) / (swap_base.size - swap[:members].size).to_f).ceil rescue 0
+            # a miss is considered a point that is less than 10% of the average deviation away of the slope
+            unless swap[:avg_dev].zero?
+              misses        = swap_base.select{|x| x[:dev] <= swap[:avg_dev] / 10.to_f and x[:dev] > 0}.map{|x| x[:miss] = x[:dev]; x}
+              # misses are sorted among members, but stay marked
+              swap[:members]= (swap[:members] + misses).sort_by{|x| x[:datetime] }
+            end
+          end # swap
+        end # of lambda
+
+        ###########################################################################################################################
+        # after declaring lambdas, the rest is quite few code
+        #
+        current_range = (0..-1)                                                                                         # RANGE   set
+        current_slope = { members: [] }                                                                                 # SLOPE   reset
+        current_base = base[current_range]                                                                              # BASE    set
+        current_results = [ ]                                                                                           # RESULTS reset
+        while current_base.size >= 5                                                                                    # LOOP
+
+          puts '-------------------------------------------------------------------------------------' if debug
+
+          while current_base.size >= 5 and current_slope[:members].size < min_members
+            puts "---- #{current_base.size} #{current_range.to_s.light_yellow} ------" if debug
+            current_slope = get_slope.call(current_base)                                                                # SLOPE   call
+            next_i  = current_slope[:members][-2]
+            current_range = ((next_i.nil? ? -2 : next_i[:i])+1..-1)                                                     # RANGE   adjust
+            current_base = base[current_range]                                                                          # BASE    adjust
+            if debug
+              print 'Hit <enter> to continue...'
+              STDIN.gets
+            end
+          end
+          puts "Current slope: ".light_yellow + "#{current_slope}" if debug
+          current_results << current_slope if current_slope                                                             # RESULTS add
+          current_slope = { members: [] }                                                                               # SLOPE   reset
+        end
+        current_results.select!{|x|  x[:members].size >= min_members }
+
+        # Adjust all members (except pivot) to fit the actual dx-value
+        finalize.call(current_results)
+        analyze.call(current_results)
+        current_results.reject!{|swap| swap[:rating] < min_rating}
+        if save
+          if interval.nil? or swap_type.nil?
+            puts "WARNING: Cannot save swaps, as both :interval and :swap_type must be given".colorize(:light_yellow)
+          else
+            to_save = current_results.empty? ? [ { datetime: zero[:datetime], side: side, empty: true } ] : current_results
+            save_swaps(to_save, interval: interval, swap_type: swap_type, contract: contract, sym: sym)
+          end
+        end
+        current_results
+      end
+  end
+end
+

metadata

@@ -0,0 +1,156 @@
+--- !ruby/object:Gem::Specification
+name: cotcube-level
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- Benjamin L. Tischendorf
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2021-08-17 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6'
+- !ruby/object:Gem::Dependency
+  name: colorize
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+- !ruby/object:Gem::Dependency
+  name: cotcube-helpers
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+- !ruby/object:Gem::Dependency
+  name: yaml
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.1'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '13'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.6'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.6'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.9'
+description: A gem to shear a time series, basically serving as a yet unssen class
+  of indicators.
+email:
+- donkeybridge@jtown.eu
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".irbrc.rb"
+- CHANGELOG.md
+- Gemfile
+- README.md
+- VERSION
+- cotcube-level.gemspec
+- lib/cotcube-level.rb
+- lib/cotcube-level/detect_slope.rb
+- lib/cotcube-level/helpers.rb
+- lib/cotcube-level/stencil.rb
+- lib/cotcube-level/triangulate.rb
+homepage: https://github.com/donkeybridge/cotcube-level
+licenses:
+- BSD-3-Clause
+metadata:
+  homepage_uri: https://github.com/donkeybridge/cotcube-level
+  source_code_uri: https://github.com/donkeybridge/cotcube-level
+  changelog_uri: https://github.com/donkeybridge/cotcube-level/CHANGELOG.md
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - "~>"
+    - !ruby/object:Gem::Version
+      version: '2.7'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.1.2
+signing_key: 
+specification_version: 4
+summary: A gem to shear a time series
+test_files: []