cotcube-bardata 0.1.3 → 0.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e879208e6163ff7f56d5b4135e62e67e4054426e98d446fd13742fe34bbb6935
-  data.tar.gz: 8bafe1a22673d497a55c201870b94b9561d3a01cb523606900e8592c0d0b1c6d
+  metadata.gz: ad166746ea75e103e71ce235710267f98fc6a1b13705b73dc978bd0001b282ed
+  data.tar.gz: 94b63c1eab7181655f99484ffcb27a29eadef2e2befa03e0301e1f82dc5e2b42
 SHA512:
-  metadata.gz: e8bbe5746c9c07435a80b647d222ea6420b5057c1b26e4370f0e1dccedae71028fb54fb25f45ef47719501f0b297c928a6d589ca7a0dfb67a02e40836d2ff1d4
-  data.tar.gz: 6cb37bdbc50a0c4fd2f9867dd983c6348df5a8f9372e0195e6c0f4dfd90463057e993b5a3c57a39b7f932166ad09f325dbff77311473513575cdd874d728a136
+  metadata.gz: 07a2d4ffc5291c1474f14c49ce1d9d42804b2a5bb5ce9fabfef048904a7d8455a106381e76ec6c8bbe05feeda50afb9db9b6f56087c572b704bbce2e7ab313c9
+  data.tar.gz: d3e83e577ad407550ab590dcd8b5c2880c05939197b80d67a07de117d2ffba918adc852f1de87e16b7262e22d71c53a1bd789d700b6ae3ef82be645adc532b52

data/.irbrc.rb

@@ -1,5 +1,7 @@
+# frozen_string_literal: true
+
 def verbose_toggle
-  irb_context.echo ? irb_context.echo = false : irb_context.echo = true
+  irb_context.echo = (irb_context.echo ? false : true)
 end
 
 alias vt verbose_toggle

data/CHANGELOG.md

@@ -1,3 +1,25 @@
+## 0.1.4 (January 02, 2021)
+  - two minor fixes (cached.rb, daily.rb)
+  - adding first (shy) specs ... to be continued
+  - cotcube-bardata.rb: added dependency parallel, added new module files and functions
+  - provide.rb: writing provide, the central accessor to actual bardata
+  - cached.rb: implementing 'provide_cached', which manages reduced and dimished subsets of 'quarters'
+  - helpers.rb: added get_id_set
+  - daily.rb: applied cops
+  - quarters.rb: applied cops, used new get_id_set, slimmed down content in favor of 'provide'
+  - eods.rb: renamed get_id_from to get_id_set
+  - added explanation to range_matrix.rb
+  - added 'holidays' to trade_dates.csv, depending on according CSV
+  - applied cops to init.rb
+  - changed name from get_range to trading_hours
+  - minor change in gemspec
+  - fixed typos in README
+  - added bounded versions to gemspec
+  - applied cops
+  - new file trading_hours.rb providing get_range(). Based on CSV data it provides a list of ranges depicting seconds since Sunday 0:00am, which in turn can be used with the helper Array.new.to_time_interval.
+  - new file and method 'range_matrix', investigating high-low ranges of entire daily
+  - Too bad, found copied README...fixing something quite embarassing.
+
 ## 0.1.3 (December 23, 2020)
   - added .provide_most_liquids_by_eod which supports :age, filtering out files that have been updated more recently
   - added 'type' to symbols to filter for e.g. currencies

data/Gemfile

@@ -1,5 +1,6 @@
-source "https://rubygems.org"
+# frozen_string_literal: true
+
+source 'https://rubygems.org'
 
 # Specify your gem's dependencies in bitangent.gemspec
 gemspec
-

data/README.md

@@ -1,36 +1,19 @@
-# Bitangent
+# cotcube-bardata
 
-This gem provides a class (and will maybe later provide a commandline tool) to process 
-time series data into bitangents.
+This gem is a versatile provider of bardata. It relies on a directory structure most probably saved to `/var/cotcube/bardata/`. The following directories and files contain data others might expect to be delivered by a database:
 
-The underlying algorithm starts with the entire range of data and shears it (starting at 
-90 or -90 degrees) along the x axis until a bitangent is found parallel to the x axis 
-(i.e. __y == 0__ resp. __y == y.last__), resulting in an angle and a group of at least 
-2 measurements. Please note here: 
-- Measurements can be clustered to 1 finding in case there is no significant distance to 
-  the bitangent in regard to the :fuzziness, which is at least 1 'tick'.
-- Ocassionally a bitangent becomes an N-tangent, when multiple findings or clusters are 
-  in one line resp. the fuzzied ranged on the sheared graph.
-- Shearing is limited by reaching 0 degrees, so everything below the horizont (or above resp.)
-  is discarded. 
-
-After identifiying the angle of Z degress delivering N findings( actually N - 1, as the 
-last finding always is the last member of the series), the entire time series then is split  
-into N subranges, where each subrange is processed again until it reaches minimum size
-(which defaults to 3 items).
-
-Except for the very first range the challenge is to trim away the beginning of each sub
-range.
-
-The result of such a search is a tree, where it might be considerable to walk and change
-this tree by adding elements to the time series instead of recalculating it completely. 
+1. `eods`: within *eods/<id or symbol>/<date>.csv*, for each date a list of contracts is located, to be applied with the list of headers `%i[ contract date open high low close volume oi ]`.
+2. `daily`: within *daily/<id or symbol>/<contract>.csv*, for each contract all eods are provided. The list of headers is expected as `%i[ contract date open high low close volume oi ]`. Please note that it is not obvious whether `close` contains settlement or actual closing price, depending on the exchange and the broker providing the source data.
+3. `quarters`: within *quarters/<id or symbol>/<contract>.csv*, for each contract a list of quarters (15 minute intervals) is provided, depending on the first occurrence of the contract within the topN volume segment. Note the different headers here: `%i[ contract date_alike day open high low close volume ]`.
+4. `trading_hours`: within *trading_hours/<symbol or type>_<set>.csv* a list of intervals is provided, with the headers `%i[ interval_start interval_end ]` for each interval described by seconds since Sunday 0:00p.m. (as defaulted by Ruby's *DateTime.new.wday)*.
+5. `trade_dates.csv`: A growing list of trade\_dates as provided by the CME.  
 
 ## Installation
 
 Add this line to your application's Gemfile:
 
 ```ruby
-gem 'bitangent'
+gem 'cotcube-bardata'
 ```
 
 And then execute:
@@ -39,11 +22,60 @@ And then execute:
 
 Or install it yourself as:
 
-    $ gem install bitangent
+    $ gem install cotcube-bardata
 
 ## Usage
 
-TODO: Write usage instructions here
+### Configuration
+
+The gem expects a configfile 'bardata.yml', located in '/etc/cotcube/' on linux platform and '/usr/local/etc/cotcube/' on FreeBSD. The location of the configfile can be overwritten by passing the according parameter to `init`.
+
+### daily.rb
+
+Provides
+
+* `provide_daily(symbol: nil, id: nil, contract:, timezone: Time.find_zone('America/Chicago'), config: init)`
+* `continuous(symbol: nil, id: nil, config: init, date: nil)`: Loads all dailies for given *id* and groups by date, hence providing a list of eods. 
+* `continuous_ml(symbol: nil, id: nil, base: nil)`: Provides a list of contracts, containing a list of most liquid by volume contracts as `{ date: , ml: }`
+* `continuous_actual_ml(symbol: nil, id: nil)`: Same as above, but providing the succeeding trading day (as the signaled 'ML' is yet one day old before it can be used). 
+* `continuous_overview(symbol: nil, id: nil, config: init, selector: :volume, human: false, filter: nil)`: Several purposes, but most noticeable providing the range of first and last occurrence within the top N% by volume within eods.
+
+### eods.rb
+
+Provides
+
+* `most_liquid_for(symbol: nil, id: nil, date: last_trade_date, config: init, quiet: false)`
+* `provide_most_liquids_by_eod(config: init, date: last_trade_date, filter: :volume_part, age: 1.hour)`
+* `provide_eods(**args)`
+* `provide_quarters(**args)`
+
+### quarters.rb
+
+Provides `provide_quarters(**args)`.
+
+### provide.rb
+
+Provides `provide(**args)`.
+
+### range\_matrix.rb
+
+Provides `range_matrix`, a simple method processing data based on `Bardata.continuous_actual_ml` to return a statistical overview of daily high-low ranges (not True Ranges). It contains sets for 
+
+* all available data,
+* a data subset containing the recent 12 months
+* data diminished by `:dim` (top and bottom) based on all available data
+
+and contains  *max*, *avg*, *lower*, *median*, *upper* and *max* (where 'upper' and 'lower' are like the median but at the 25percentile and 75percentile resp.).
+
+As a third dimension (sorry!) all of the above is applied to days, weeks as well as months. 
+
+### trade\_dates.rb
+
+Provides `last_trade_date`, simple fetches the current 5 trade dates from CME and returns the very last.
+
+### trading\_hours.rb
+
+Provides `get_range(symbol: nil, set: :full, force_set: false, config: init, debug: false)`, loading a set of intervals. The sets are defaulting to :full when the requested set is not found--unless :force\_set is enabled. Furthermore, if symbol is not found, the type-based version is returned. Eventually, if neither could be returned, the 24x7 interval is returned.
 
 ## Development
 
@@ -53,9 +85,10 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/bitangent.
+Bug reports and pull requests are welcome on GitHub at https://github.com/donkeybridge/bitangent.
 
 
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+

data/Rakefile

@@ -1,6 +1,8 @@
-require "bundler/gem_tasks"
-require "rspec/core/rake_task"
+# frozen_string_literal: true
+
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
 
 RSpec::Core::RakeTask.new(:spec)
 
-task :default => :spec
+task default: :spec

data/VERSION

@@ -1 +1 @@
-0.1.3
+0.1.4

data/cotcube-bardata.gemspec

@@ -9,13 +9,13 @@ Gem::Specification.new do |spec|
   spec.summary       = 'Functions to provide bardata; and some simple time series aggregations'
   spec.description   = 'Functions to provide bardata; and some simple time series aggregations '
 
-  spec.homepage      = 'https://github.com/donkeybridge/'+ spec.name
+  spec.homepage      = "https://github.com/donkeybridge/#{spec.name}"
   spec.license       = 'BSD-4-Clause'
   spec.required_ruby_version = Gem::Requirement.new('~> 2.7')
 
   spec.metadata['homepage_uri'] = spec.homepage
   spec.metadata['source_code_uri'] = spec.homepage
-  spec.metadata['changelog_uri'] = spec.homepage + '/CHANGELOG.md'
+  spec.metadata['changelog_uri'] = "#{spec.homepage}/CHANGELOG.md"
 
   # Specify which files should be added to the gem when it is released.
   # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
@@ -26,16 +26,15 @@ Gem::Specification.new do |spec|
   spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ['lib']
 
-  spec.add_dependency 'cotcube-indicators'
-  spec.add_dependency 'cotcube-helpers'
-  spec.add_dependency 'yaml'
-  spec.add_dependency 'activesupport'
-  spec.add_dependency 'colorize'
-  spec.add_dependency 'httparty'
-  spec.add_dependency 'rubyzip'
+  spec.add_dependency 'activesupport',      '~> 6'
+  spec.add_dependency 'colorize',           '~> 0.8'
+  spec.add_dependency 'cotcube-helpers',    '~> 0.1'
+  spec.add_dependency 'cotcube-indicators', '~> 0.1'
+  spec.add_dependency 'httparty',           '~> 0.18'
+  spec.add_dependency 'parallel',           '~> 1'
+  spec.add_dependency 'yaml',               '~> 0.1'
 
-
-  spec.add_development_dependency 'rake'
+  spec.add_development_dependency 'rake',  '>= 12'
   spec.add_development_dependency 'rspec', '~>3.6'
   spec.add_development_dependency 'yard', '~>0.9'
 end

data/lib/cotcube-bardata.rb

@@ -1,3 +1,5 @@
+# rubocop:disable Naming/FileName
+# rubocop:enable Naming/FileName
 # frozen_string_literal: true
 
 require 'active_support'
@@ -9,35 +11,59 @@ require 'date' unless defined?(DateTime)
 require 'csv'  unless defined?(CSV)
 require 'yaml' unless defined?(YAML)
 require 'cotcube-helpers'
-
-
+require 'parallel'
 
 require_relative 'cotcube-bardata/constants'
+require_relative 'cotcube-bardata/helpers'
 require_relative 'cotcube-bardata/init'
 require_relative 'cotcube-bardata/trade_dates'
 require_relative 'cotcube-bardata/daily'
 require_relative 'cotcube-bardata/quarters'
 require_relative 'cotcube-bardata/eods'
+require_relative 'cotcube-bardata/cached'
 require_relative 'cotcube-bardata/provide'
+require_relative 'cotcube-bardata/range_matrix'
+require_relative 'cotcube-bardata/trading_hours'
 
 module Cotcube
   module Bardata
-
     module_function :config_path, # provides the path of configuration directory
-      :config_prefix,             # provides the prefix of the configuration directory according to OS-specific FSH
-      :init,                      # checks whether environment is prepared and returns the config hash
-      :last_trade_date,           # Provides the most recent trade date (today or maybe last friday before weekend)
-      :provide,                   # 
-      :most_liquid_for,           # the most_liquid contract for a given symbol or id, based on date or last_trade_date
-      :provide_eods,              # provides the list of eods, either with data or just the contracts, filtered for liquidity threshold
-      :provide_most_liquids_by_eod,
-      :provide_daily,             # provides the list of dailies for a given symbol, which include OI. Note that the close is settlement price.
-      :continuous,                # for a given date or range, provide all contracts that exceed a given threshold of volume share
-      :continuous_overview,       # based on continuous, create list of when which contract was most liquid
-      :provide_quarters,          # provide the list of quarters, possibly as hours or days.
-      :symbols                    # reads and provides the symbols file
-    
-    # please not that module_functions of source provided in private files must be published there
+                    # provides the prefix of the configuration directory according to OS-specific FSH
+                    :config_prefix,
+                    # checks whether environment is prepared and returns the config hash
+                    :init,
+                    # Provides the most recent trade date (today or maybe last friday before weekend)
+                    :last_trade_date,
+                    :provide,
+                    # the most_liquid contract for a given symbol or id, based on date or last_trade_date
+                    :most_liquid_for,
+                    # provides the list of eods, either with data or just the contracts,
+                    # filtered for liquidity threshold
+                    :provide_eods,
+                    :provide_most_liquids_by_eod,
+                    # provides the list of dailies for a given symbol, which include OI.
+                    # Note that the close is most probably settlement price.
+                    :provide_daily,
+                    # for a given date or range, provide all contracts that exceed a given threshold of volume share
+                    :continuous,
+                    # the list of most liquid contracts (by each days volume share)
+                    :continuous_ml,
+                    # same list but riped one day each
+                    :continuous_actual_ml,
+                    # based on continuous, create list of when which contract was most liquid
+                    :continuous_overview,
+                    # provide the list of quarters, possibly as hours or days.
+                    :provide_quarters,
+                    # some statistics to estimate daily volatility of specific contract
+                    :range_matrix,
+                    # create an array of ranges based on specified source data
+                    :trading_hours,
+                    # receive id / symbol information on an uncertain set of parameters
+                    :get_id_set,
+                    :compare,
+                    :holidays,
+                    :symbols # reads and provides the symbols file
+
+    # please note that module_functions of source provided in private files must be published there
   end
 end
-

data/lib/cotcube-bardata/cached.rb

@@ -0,0 +1,80 @@
+# frozen_string_literal: true
+
+module Cotcube
+  # missing top level documentation
+  module Bardata
+
+    # send pre-created days based on quarters
+    def provide_cached(contract:,
+                       symbol: nil, id: nil,
+                       config: init,
+                       timezone: Time.find_zone('America/Chicago'),
+                       set: :full,             # most probably either :full or :rth
+                       force_update: false,    # force reloading via provide_quarters
+                       force_recent: false)    # 
+
+      headers = %i[contract datetime open high low close volume]
+      sym      = get_id_set(symbol: symbol, id: id, contract: contract)
+      contract = contract[-3..-1]
+      dir      = "#{config[:data_path]}/cached/#{sym[:id]}_#{set.to_s.downcase}"
+      symlink  = "#{config[:data_path]}/cached/#{sym[:symbol]}_#{set.to_s.downcase}"
+      `mkdir -p #{dir}` unless Dir.exist? dir
+      `ln -s #{dir} #{symlink}` unless File.exist? symlink
+      file = "#{dir}/#{contract}.csv"
+      quarters_file = "#{config[:data_path]}/quarters/#{sym[:id]}/#{contract[-3..-1]}.csv"
+      if File.exist?(file) and not force_update
+        base = CSV.read(file, headers: headers).map{|x| 
+          x = x.to_h
+          x[:datetime] = timezone.parse(x[:datetime])
+          %i[open high low close].each{|z| x[z] = x[z].to_f.round(9)}
+          x[:volume] = x[:volume].to_i
+          x[:type]   = "#{set.to_s.downcase}_day".to_sym
+          x
+        }
+        if base.last[:high].zero? 
+          # contract exists but is closed (has the CLOSED marker)
+          base.pop
+          return base
+        elsif File.mtime(file) < File.mtime(quarters_file)
+          return base
+        else
+          puts "File #{file} exists, but is neither closed nor current. Running update.".colorize(:light_green)
+        end
+      end
+      begin 
+        data = provide_quarters(contract: contract, id: sym[:id], keep_marker: true)
+      rescue
+        puts "Cannot provide quarters for requested contract #{sym[:symbol]}:#{contract}, returning '[ ]'".colorize(:light_red)
+        return [] 
+      end
+
+      # removing marker if existing
+      contract_is_marked = data.last[:high].zero?
+      data.pop if contract_is_marked
+      unless set == :full or data.size < 3
+        requested_set = trading_hours(symbol: sym[:symbol], set: set)
+        data = data.select_within(ranges: requested_set, attr: :datetime) {|x| x.to_datetime.to_sssm }
+      end
+
+      base = Cotcube::Helpers.reduce(bars: data, to: :days)
+
+      # remove last day of result if not marked
+      base.pop unless contract_is_marked or force_recent
+
+      base.map do |x| 
+        x[:datetime] = x[:datetime].to_date
+        x[:type]= "#{set}_day".to_sym
+        x.delete(:day)
+      end
+      CSV.open(file, 'w') do |csv|
+        base.each{|b| csv << b.values_at(*headers) }
+        if contract_is_marked
+          marker = [ "#{sym[:symbol]}#{contract}",base.last[:datetime]+1.day,0,0,0,0,0 ]
+          csv << marker
+        end
+      end
+      base
+    end
+
+  end
+end

data/lib/cotcube-bardata/constants.rb

@@ -2,16 +2,16 @@
 
 module Cotcube
   module Bardata
-
     SYMBOL_EXAMPLES = [
-      { id: "13874U", symbol: "ET", ticksize: 0.25, power: 1.25, months: "HMUZ", bcf: 1.0, reports: "LF", name: "S&P 500 MICRO" },
-      { id: "209747", symbol: "NM", ticksize: 0.25, power: 0.5,  monhts: "HMUZ", bcf: 1.0, reports: "LF", name: "NASDAQ 100 MICRO" }
+      { id: '13874U', symbol: 'ET', ticksize: 0.25, power: 1.25, months: 'HMUZ', bcf: 1.0, reports: 'LF',
+        name: 'S&P 500 MICRO' },
+      { id: '209747', symbol: 'NM', ticksize: 0.25, power: 0.5,  months: 'HMUZ', bcf: 1.0, reports: 'LF',
+        name: 'NASDAQ 100 MICRO' }
     ].freeze
 
-    MONTH_COLOURS  = { 'F' => :cyan,  'G' => :green,   'H' => :light_green,
-                       'J' => :blue,  'K' => :yellow,  'M' => :light_yellow,
-                       'N' => :cyan,  'Q' => :magenta, 'U' => :light_magenta,
-                       'V' => :blue,  'X' => :red,     'Z' => :light_red }.freeze
-
+    MONTH_COLOURS = { 'F' => :cyan, 'G' => :green, 'H' => :light_green,
+                      'J' => :blue,  'K' => :yellow,  'M' => :light_yellow,
+                      'N' => :cyan,  'Q' => :magenta, 'U' => :light_magenta,
+                      'V' => :blue,  'X' => :red,     'Z' => :light_red }.freeze
   end
 end

data/lib/cotcube-bardata/daily.rb

@@ -1,92 +1,127 @@
 # frozen_string_literal: true
 
 module Cotcube
+  # Missing top level documentation comment
   module Bardata
-
     # just reads bardata/daily/<id>/<contract>.csv
-    def provide_daily(symbol: nil, id: nil, contract:, timezone: Time.find_zone('America/Chicago'), config: init)
+    def provide_daily(contract:, symbol: nil, id: nil, timezone: Time.find_zone('America/Chicago'), config: init)
       contract = contract.to_s.upcase
-      raise ArgumentError, "Contract '#{contract}' is bogus, should be like 'M21' or 'ESM21'" unless contract.is_a? String and [3,5].include? contract.size
-      if contract.to_s.size == 5
-        symbol   = contract[0..1]
-        contract = contract[2..4] 
-      end
-      unless symbol.nil?
-        symbol_id = symbols.select{|s| s[:symbol] == symbol.to_s.upcase}.first[:id]
-        raise ArgumentError, "Could not find match in #{config[:symbols_file]} for given symbol #{symbol}" if symbol_id.nil?
-        raise ArgumentError, "Mismatching symbol #{symbol} and given id #{id}" if not id.nil? and symbol_id != id
-        id = symbol_id
+      unless contract.is_a?(String) && [3, 5].include?(contract.size)
+        raise ArgumentError, "Contract '#{contract}' is bogus, should be like 'M21' or 'ESM21'"
       end
-      raise ArgumentError, "Could not guess :id or :symbol from 'contract: #{contract}', please clarify." if id.nil?
+
+      sym = get_id_set(symbol: symbol, id: id, contract: contract)
+      contract = contract[2..4] if contract.to_s.size == 5
+      id = sym[:id]
       id_path   = "#{config[:data_path]}/daily/#{id}"
       data_file = "#{id_path}/#{contract}.csv"
-      raise RuntimeError, "No data found for requested :id (#{id_path} does not exist)" unless Dir.exist?(id_path)
-      raise RuntimeError, "No data found for requested contract #{symbol}:#{contract} in #{id_path}." unless File.exist?(data_file)
-      data = CSV.read(data_file, headers: %i[contract date open high low close volume oi] ).map do |row|
+      raise "No data found for requested :id (#{id_path} does not exist)" unless Dir.exist?(id_path)
+
+      raise "No data found for requested contract #{symbol}:#{contract} in #{id_path}." unless File.exist?(data_file)
+
+      data = CSV.read(data_file, headers: %i[contract date open high low close volume oi]).map do |row|
         row = row.to_h
-        row.each do |k, _| 
-          row[k] = row[k].to_f if [:open, :high, :low, :close].include? k
-          row[k] = row[k].to_i if [:volume, :oi].include? k
+        row.each do |k, _|
+          row[k] = row[k].to_f if %i[open high low close].include? k
+          row[k] = row[k].to_i if %i[volume oi].include? k
         end
-        row[:datetime] = timezone.parse(row[:date])
+        row[:datetime] = timezone.parse(row[:date]).to_date
+        row[:type]     = :daily
         row
       end
+      data.pop if data.last[:high].zero?
       data
     end
-   
-    # reads all files in  bardata/daily/<id> and aggregates by date (what is a pre-stage of a continuous based on daily bars)
+
+    # reads all files in  bardata/daily/<id> and aggregates by date
+    # (what is a pre-stage of a continuous based on daily bars)
     def continuous(symbol: nil, id: nil, config: init, date: nil)
-      unless symbol.nil?
-        symbol_id = symbols.select{|s| s[:symbol] == symbol.to_s.upcase}.first[:id]
-        raise ArgumentError, "Could not find match in #{config[:symbols_file]} for given symbol #{symbol}" if symbol_id.nil?
-        raise ArgumentError, "Mismatching symbol #{symbol} and given id #{id}" if not id.nil? and symbol_id != id
-        id = symbol_id
-      end
-      raise ArgumentError, "Could not guess :id or :symbol, please clarify." if id.nil?
-      id_path    = "#{config[:data_path]}/daily/#{id}"
-      available_contracts = Dir[id_path + '/*.csv'].map{|x| x.split('/').last.split('.').first}.sort_by{|x| x[-7]}.sort_by{|x| x[-6..-5]}
-      data = [] 
-      available_contracts.each do |c| 
-        provide_daily(id: id, config: config, contract: c).each do |x| 
+      sym = get_id_set(symbol: symbol, id: id)
+      id  = sym[:id]
+      id_path = "#{config[:data_path]}/daily/#{id}"
+      available_contracts = Dir["#{id_path}/*.csv"].map { |x| x.split('/').last.split('.').first }
+      available_contracts.sort_by! { |x| x[-7] }.sort_by! { |x| x[-6..-5] }
+      data = []
+      available_contracts.each do |c|
+        provide_daily(id: id, config: config, contract: c).each do |x|
           data << x
         end
       end
-      result = [] 
-      data.sort_by{|x| x[:date]}.group_by{|x| x[:date]}.map{|k,v| 
-        v.map{|x| x.delete(:date)}
+      result = []
+      data.sort_by { |x| x[:date] }.group_by { |x| x[:date] }.map do |k, v|
+        v.map { |x| x.delete(:date) }
         result << {
           date: k,
-          volume: v.map{|x| x[:volume]}.reduce(:+),
-          oi:     v.map{|x| x[:oi    ]}.reduce(:+)
+          volume: v.map { |x| x[:volume] }.reduce(:+),
+          oi: v.map { |x| x[:oi] }.reduce(:+)
         }
         result.last[:contracts] = v
-      } 
-      date.nil? ? result : result.select{|x| x[:date] == date}.first
+      end
+      date.nil? ? result : result.select { |x| x[:date] == date }.first
     end
 
-    # based on .continuous, this methods sorts the prepared dailies continuous for each date on either :volume (default) or :oi
+    def continuous_ml(symbol: nil, id: nil, base: nil)
+      (base.nil? ? Cotcube::Bardata.continuous(symbol: symbol, id: id) : base).map do |x|
+        x[:ml] = x[:contracts].max_by { |z| z[:volume] }[:contract]
+        { date: x[:date], ml: x[:ml] }
+      end
+    end
+
+    # the method above delivers the most_liquid as it is found at the end of the day. D
+    # during trading, the work is done with data
+    # that is already one day old. This is is fixed here:
+    def continuous_actual_ml(symbol: nil, id: nil)
+      continuous =    Cotcube::Bardata.continuous    symbol: symbol, id: id
+      continuous_ml = Cotcube::Bardata.continuous_ml base: continuous
+      continuous_hash = continuous.to_h { |x| [x[:date], x[:contracts]] }
+      actual_ml = continuous_ml.pairwise { |a, b| { date: b[:date], ml: a[:ml] } }
+      actual_ml.map do |x|
+        r = continuous_hash[x[:date]].select { |z| x[:ml] == z[:contract] }.first
+        r = continuous_hash[x[:date]].min_by { |z| -z[:volume] } if r.nil?
+        r
+      end
+    end
+
+    # based on .continuous, this methods sorts the prepared dailies continuous for each date
+    # on either :volume (default) or :oi
     # with this job done, it can provide the period for which a past contract was the most liquid
     #
-    def continuous_overview(symbol: nil, id: nil, config: init, selector: :volume, human: false, filter: nil)
-      raise ArgumentError, "Selector must be either :volume or :oi" unless selector.is_a? Symbol and [:volume, :oi].include? selector
-      
-      unless symbol.nil?
-        symbol_id = symbols.select{|s| s[:symbol] == symbol.to_s.upcase}.first[:id]
-        raise ArgumentError, "Could not find match in #{config[:symbols_file]} for given symbol #{symbol}" if symbol_id.nil?
-        raise ArgumentError, "Mismatching symbol #{symbol} and given id #{id}" if not id.nil? and symbol_id != id
-        id = symbol_id
-      end
-      raise ArgumentError, "Could not guess :id or :symbol, please clarify." if id.nil?
-      data = continuous(id: id, config: config).map{|x| 
+    def continuous_overview(symbol: nil, id: nil, # rubocop:disable Metrics/ParameterLists
+                            config: init,
+                            selector: :volume,
+                            human: false,
+                            filter: nil)
+      raise ArgumentError, 'Selector must be either :volume or :oi' unless selector.is_a?(Symbol) &&
+                                                                           %i[volume oi].include?(selector)
+
+      sym = get_id_set(symbol: symbol, id: id)
+      id  = sym[:id]
+      data = continuous(id: id, config: config).map do |x|
         {
-          date:   x[:date],
-          volume: x[:contracts].sort_by{|x| - x[:volume]}[0..4].compact.select{|x| not x[:volume].zero?},
-          oi:     x[:contracts].sort_by{|x| - x[:oi]}[0..4].compact.select{|x| not x[:oi].zero?}
+          date: x[:date],
+          volume: x[:contracts].sort_by { |z| - z[:volume] }[0..4].compact.reject { |z| z[:volume].zero? },
+          oi: x[:contracts].sort_by { |z| - z[:oi] }[0..4].compact.reject { |z| z[:oi].zero? }
         }
-      }.select{|x| not x[selector].empty? }
-      result = data.group_by{|x| x[selector].first[:contract]}
+      end
+      data.reject! { |x| x[selector].empty? }
+      result = data.group_by { |x| x[selector].first[:contract] }
       if human
-        result.each {|k,v| puts "#{k}\t#{v.first[:date]}\t#{v.last[:date]}" if filter.nil? or v.first[selector].first[:contract][2..4]=~ /#{filter}/ }
+        result.each do |k, v|
+          next unless filter.nil? || v.first[selector].first[:contract][2..4] =~ (/#{filter}/)
+
+          # rubocop:disable Layout/ClosingParenthesisIndentation
+          puts "#{k
+             }\t#{v.first[:date]
+             }\t#{v.last[:date]
+             }\t#{format('%4d', (Date.parse(v.last[:date]) - Date.parse(v.first[:date])))
+             }\t#{result[k].map do |x|
+                    x[:volume].select do
+                      x[:contract] == k
+                    end
+                  end.size
+             }"
+          # rubocop:enable Layout/ClosingParenthesisIndentation
+        end
       end
       result
     end