eco-helpers 2.0.21 → 2.0.26

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5525ed41d4f4b42d96eb0d71f02911e7cfb9112890aff984fb872a60d7515976
-  data.tar.gz: d6dacbda91325cea0867c253b99458ae18b16e6450431ead852d686dd6e56859
+  metadata.gz: 722e1dc695f9d9fae5fbceca8a2269f4836393ee4dc50fc0f850e43fefc88789
+  data.tar.gz: bc34444cdf33bff51895fa5ed6bb7a89b89074e5bd11852c0a1a4f4bf6bcc574
 SHA512:
-  metadata.gz: 1f4a164e153ac8e3d75bfc7024b2ee7c6c101a1703fb79b58a0d8c8fc31fc9978b6353c2a178a419c94ca22e6b49bf440364706642e7b087a4601f4cd7da3b51
-  data.tar.gz: 9fdd7340a79b853b0dd51676e9ea2ba773928c68928b62d7781a525b2c27c1032ec7da04cae1f57afb7b290649350280d0aa5d0f2e7b91ba8a46cfe685bbeadb
+  metadata.gz: 9a18aaa7abf012872f21209907909f1c309ddb530586bc9e27d595ace57377c3c4dba2b0a7bacc902e160cf2f51d4171a5de3e0536b6eda4dff7d018307744ca
+  data.tar.gz: efcaa1832677c0cf13d5dfb0ba6a3abaa8bff3fec4e1d2639b2b832358a1e51b849e594da64827bed096eb4f8f03a5664a6a8245c58128acf3478eaf0e94d38a

data/CHANGELOG.md

@@ -1,7 +1,107 @@
 # Change Log
 All notable changes to this project will be documented in this file.
 
-## [2.0.21] - 2021-06-0x
+## [2.0.26] - 2021-06-xx
+
+### Added
+  - `Eco::API::MicroCases#set_supervisor`, tries to keep in sync the `#subordinates` **count** of previous and new supervisor
+  - new **option** `-run-postlaunch` to run post launch cases, even when we run in `dry-run` mode
+    * when in `dry-run` it will **not** reload the people base of the session
+  - new **option** to append **new** entries to the `People` object
+    * **invokable** on **cli** via `-append-starters` (`{people: {append_created: true}}`)
+    * the following use cases include this option:
+      * `Eco::API::UseCases::DefaultCases::UpsertCase`
+      * `Eco::API::UseCases::DefaultCases::HrisCase`
+      * `Eco::API::UseCases::DefaultCases::CreateDetailsWithSupervisorCase`
+      * `Eco::API::UseCases::DefaultCases::CreateDetailsCase`
+      * `Eco::API::UseCases::DefaultCases::CreateCase`
+    * the option involves a new keyed argument `:append_created` in a couple of `MicroCases`
+      * `Eco::API::MicroCases#with_each`: where internally the search is performed against a copy of the `People` object.
+      * `Eco::API::MicroCases#with_each_starter`
+    * when `--help` is invoked option to filter the shown `-options`, `-usecases` and `filters` by a word contained in the option.
+  - **added** `csv` **header checks** for feed files, which entailed some changes:
+    * `Eco::API::Common::Loaders::Parser`
+      - new **subclass** `RequiredAttrs`, creatred when calling `.active_when_all` and `.active_when_any`
+      - **dependency** injection via `.dependencies` as `{required_attrs: RequiredAttrs}`
+    * **added** `Eco::API::Common::People::PersonParser#required_attrs` to offer all the `RequiredAttrs`, where defined
+      - the **new method** `#required_attrs` to expose the injected `RequiredAttrs`
+    * **new** keyed argument `check_headers:` in `Eco::API::Common::People::EntryFactory#entries`
+      - subsequent changes to accommodate the new param in `Eco::API::Session#csv_entries`
+      - `eco/cli/config/default/input` calls using this param to `true`
+    * `Eco::API::Common::People::DefaultParsers::CSVParser`
+      - added option `check_headers` via `dependencies` that enables the headers check
+      - it will now offer detailed warning messages on what can happen with the **missing headers**
+        * it will also list the **unknown header** names
+
+### Changed
+  - `Eco::API::MicroCases#set_supervisor`, the order of the 2 first parameters
+  - `Eco::API::Organization::People`: internally `@by_id` cache Hash included `nil` values => **not** any more.
+  - removed **unused** methods on **patches** for `Ecoportal::API::V1::Person` and `Ecoportal::API::Internal::Person`
+    * specifically `#reset_account!` and `#consolidate_account!` as well as `#reset_details!` and `#consolidate_details!`
+  - internal changes in `Eco::API::Common::People::Entries#entry`
+    * **added** option to trigger `MultipleSearchResults` StandardError when multiple candiates are found.
+    * **removed** `nil` values from the `caches` (the Hashes to optimize the search)
+  - slight structure refactor of `Eco::API::Common::Loaders`
+    * moved base class to subfolder/namespace
+    * decoupled pure `Loader` logics to `Loaders::Base` and use case inheriance chain loader to `Loaders::CaseBase`
+  - `Eco::API::Session::BatchJob` the `post_launch`:
+    * sets the `id` to the `person` if it was **created** successfully
+    * when in `dry-run` it fakes the `id` with a counter
+  - `Eco::API::Common::People::PersonFactory` gets `subordinates` initialized to `0` (when **creating** a `new` person)
+
+### Fixed
+
+
+## [2.0.25] - 2021-06-23
+
+### Added
+  - `Eco::API::UseCases::DefaultCases::HrisCase` validation error to require `-schema-id` command line when there are people in schemas other than the active one
+
+### Changed
+  - `Eco::API::Session::Batch::Job`
+    * for backwards compatibility `-include-only-excluded` should bring an options structure compatible with `-include-excluded`
+
+### Fixed
+  - `Eco::API::Session::Batch` fixed typo that would prevent `prompt_user` to work
+
+## [2.0.24] - 2021-06-22
+
+### Added
+  - `Eco::API::Session::Batch::Job` made **native** `-include-excluded`
+    * also added new option `-include-only-excluded` to be able to only target people HRIS excluded
+
+## [2.0.23] - 2021-06-22
+
+### Added
+  - `Eco::API::Session::Batch::Job` **new** option (`-save-requests`) to save requests even if in `dry-run` (`-simulate`)
+### Changed
+  - `Eco::API::Session::Batch::Job` new people won't create updates unless they have either details or account
+    * because that entry is not supposed to be created unless has account or details
+
+## [2.0.22] - 2021-06-18
+
+### Added
+  - exposed `logger` in `BaseLoader` and
+  - support for multiple input files
+    * `Eco::API::Common::People::EntryFactory#entries`:
+      - refactored to allow multiple input files parsing
+      - moreover to `idx`, hash entries will get their `source_file`
+    * Input callback at `lib/eco/cli/config/default/input` refactored format detection and enabled folder input
+    * `SCR.get_file` language extended to also mention folder (not just file)
+  - support for `.xls` and `.xlsx` files
+    * `Eco::API::Common::People::DefaultParsers::XLSParser` the Excel files **parser**
+    * `Eco::API::Common::People::PersonParser` added `:xls` as an accepted format
+    * `Eco::API::Session#fields_mapper` exposed mapper through a method to allow **headers detection**
+      - The external names of the fields are the column headers of the input file
+    * `Eco::API::UseCases::BaseIO` when arguments validation rails, now it raises with specific `MissingParameter` error
+
+### Changed
+  - dry out `BaseLoader` (only session is set as instance variable)
+  - removed `creek` **dependency** (it was not used anywhere in the gem)
+    * we just kept `roo` and `roo-xls`
+  - custom `Error` classes now all inherit from `StandardError` (rather than `Exception`)
+
+## [2.0.21] - 2021-06-04
 
 ### Added
   - `Eco::CSV::Table`, support to create the table out of an `Array<Hash>`
@@ -11,9 +111,6 @@ All notable changes to this project will be documented in this file.
 ### Changed
   - `Eco::API::Common::People::EntryFactory` slight **refactor** to boost better support for multiple input formats
 
-### Fixed
-
-
 ## [2.0.20] - 2021-05-31
 
 ### Added

data/eco-helpers.gemspec

@@ -42,5 +42,4 @@ Gem::Specification.new do |spec|
   spec.add_dependency 'jaro_winkler',  '>= 1.5.4', '< 1.6'
   spec.add_dependency 'roo',           '>= 2.8.3', '< 2.9'
   spec.add_dependency 'roo-xls',       '>= 1.2.0', '< 1.3'
-  spec.add_dependency 'creek',         '>= 2.5.2', '< 2.6'
 end

data/lib/eco/api/common.rb

@@ -10,7 +10,6 @@ require_relative 'common/class_helpers'
 require_relative 'common/class_auto_loader'
 require_relative 'common/class_hierarchy'
 require_relative 'common/class_meta_basics'
-require_relative 'common/base_loader'
 require_relative 'common/loaders'
 require_relative 'common/session'
 require_relative 'common/people'

data/lib/eco/api/common/loaders.rb

@@ -7,6 +7,8 @@ module Eco
   end
 end
 
+require_relative 'loaders/base'
+require_relative 'loaders/case_base'
 require_relative 'loaders/use_case'
 require_relative 'loaders/policy'
 require_relative 'loaders/error_handler'

data/lib/eco/api/common/loaders/base.rb

@@ -0,0 +1,58 @@
+module Eco
+  module API
+    module Common
+      module Loaders
+        class Base
+          extend Eco::API::Common::ClassHelpers
+
+          class << self              
+            # Sort order
+            def <=>(other)
+              created_at <=> other.created_at
+            end
+
+            # If still not set, it sets the `created_at` class timestamp.
+            def set_created_at!
+              @created_at = Time.now unless @created_at
+            end
+
+            # Class creation timestamp, to be able to load them in the order they were declared.
+            def created_at
+              @created_at ||= Time.now
+            end
+          end
+
+          # This method will be called when the BaseLoader is created
+          # @note
+          #   - this method should implement the loading logics for the given `Children` class.
+          def initialize
+            raise "You should implement this method"
+          end
+
+          def name
+            self.class.name
+          end
+
+          private
+
+          def session
+            ASSETS.session
+          end
+
+          def config
+            session.config
+          end
+
+          def logger
+            session.logger
+          end
+
+          def micro
+            session.micro
+          end
+
+        end
+      end
+    end
+  end
+end

data/lib/eco/api/common/loaders/case_base.rb

@@ -0,0 +1,33 @@
+module Eco
+  module API
+    module Common
+      module Loaders
+        class CaseBase < Loaders::Base
+
+          class << self
+            attr_writer :name, :type
+
+            # The name that this case, policy or error handler will have.
+            def name(value = nil)
+              name_only_once! if value
+              set_created_at!
+              return @name ||= self.to_s unless value
+              @name = value
+            end
+
+            # Prevent the same class to be re-opened/re-named
+            def name_only_once!
+              raise "You have already declared #{self} or you are trying to give it a name twice" if @name
+            end
+
+          end
+
+          def name
+            self.class.name
+          end
+
+        end
+      end
+    end
+  end
+end

data/lib/eco/api/common/loaders/error_handler.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class ErrorHandler < Eco::API::Common::BaseLoader
+        class ErrorHandler < Eco::API::Common::Loaders::CaseBase
 
           class << self
             attr_writer :error
@@ -17,7 +17,7 @@ module Eco
           end
 
           inheritable_class_vars :error
-          
+
           def initialize(handlers)
             raise "Expected Eco::API::Policies. Given #{handlers.class}" unless handlers.is_a?(Eco::API::Error::Handlers)
             handlers.on(self.error, &self.method(:main))

data/lib/eco/api/common/loaders/parser.rb

@@ -2,7 +2,28 @@ module Eco
   module API
     module Common
       module Loaders
-        class Parser < Eco::API::Common::BaseLoader
+        class Parser < Eco::API::Common::Loaders::CaseBase
+
+          # Helper class to scope what required attributes it depends on
+          class RequiredAttrs < Struct.new(:attr, :type, :attrs)
+            def active?(*input_attrs)
+              missing(*input_attrs).empty?
+            end
+
+            def dependant?(attr)
+              attrs.include?(attr)
+            end
+
+            def missing(*input_attrs)
+              return [] if input_attrs.include?(attr)
+              match = input_attrs & attrs
+              miss  = attrs - match
+              return []    if miss.empty?
+              return attrs if match.empty?
+              return miss  if type == :all
+              []
+            end
+          end
 
           class << self
             attr_reader :active_when
@@ -19,12 +40,14 @@ module Eco
               @attribute = value
             end
 
-            # TODO: it migh rather merge?
             # Some parsers require dependencies to do their job.
-            def dependencies(value = nil)
+            def dependencies(**value)
               @dependencies ||= {}
-              return @dependencies unless value
-              @dependencies = value
+              return @dependencies.merge({
+                required_attrs: @active_when_attrs
+              }) unless !value.empty?
+              raise "Expected Hash. Given: '#{value.class}'" unless value.is_a?(Hash)
+              @dependencies.merge!(value)
             end
 
             # Define or get the `phase` that the `parser` kicks in.
@@ -47,6 +70,7 @@ module Eco
 
             # Helper to build the `active_when` condition.
             def active_when_any(*attrs)
+              @active_when_attrs = RequiredAttrs.new(attribute, :any, attrs)
               @active_when = Proc.new do |source_data|
                 keys = data_keys(source_data)
                 attrs.any? {|key| keys.include?(key)}
@@ -55,6 +79,7 @@ module Eco
 
             # Helper to build the `active_when` condition.
             def active_when_all(*attrs)
+              @active_when_attrs = RequiredAttrs.new(attribute, :all, attrs)
               @active_when = Proc.new do |source_data|
                 keys = data_keys(source_data)
                 attrs.all? {|key| keys.include?(key)}

data/lib/eco/api/common/loaders/policy.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class Policy < Eco::API::Common::BaseLoader
+        class Policy < Eco::API::Common::Loaders::CaseBase
 
           def initialize(policies)
             raise "Expected Eco::API::Policies. Given #{policies.class}" unless policies.is_a?(Eco::API::Policies)

data/lib/eco/api/common/loaders/use_case.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class UseCase < Eco::API::Common::BaseLoader
+        class UseCase < Eco::API::Common::Loaders::CaseBase
 
           class << self
             # @return [Symbol] the `type` of usecase (i.e. `:sync`, `:transform`, `:import`, `:other`)

data/lib/eco/api/common/people/default_parsers.rb

@@ -44,3 +44,4 @@ require_relative 'default_parsers/freemium_parser'
 require_relative 'default_parsers/policy_groups_parser'
 require_relative 'default_parsers/login_providers_parser'
 require_relative 'default_parsers/csv_parser'
+require_relative 'default_parsers/xls_parser'

data/lib/eco/api/common/people/default_parsers/csv_parser.rb

@@ -2,7 +2,9 @@ class Eco::API::Common::People::DefaultParsers::CSVParser < Eco::API::Common::Lo
   attribute :csv
 
   def parser(data, deps)
-    Eco::CSV.parse(data, headers: true, skip_blanks: true).each_with_object([]) do |row, arr_hash|
+    Eco::CSV.parse(data, headers: true, skip_blanks: true).tap do |table|
+      check_headers(table) if deps[:check_headers]
+    end.each_with_object([]) do |row, arr_hash|
       row_hash = row.headers.uniq.each_with_object({}) do |attr, hash|
         next if attr.to_s.strip.empty?
         hash[attr.strip] = parse_string(row[attr])
@@ -36,4 +38,94 @@ class Eco::API::Common::People::DefaultParsers::CSVParser < Eco::API::Common::Lo
     ["NULL"].any? {|token| str == token}
   end
 
+  def check_headers(table)
+    headers = table.headers
+    missing = missing_headers(headers)
+    unknown = unknown_headers(headers)
+    unless missing.empty? && unknown.empty?
+      msg  = "Detected possible HEADER ISSUES !!!\n"
+      msg << "There might be Missing or Wrong HEADER names in the CSV file:\n"
+      msg << "  * UNKNOWN (or not used?): #{unknown}\n" unless unknown.empty?
+      msg << "  * MISSING DIRECT: #{missing[:direct]}\n" unless (missing[:direct] || []).empty?
+      unless (data =  missing[:indirect] || []).empty?
+        msg << "  * MISSING INDIRECT:\n"
+        data.each do |ext, info|
+          msg <<  "    - '#{ext}' => "
+          msg << (info[:attrs] || {}).map do |status, attrs|
+            if status == :inactive
+              "makes inactive: #{attrs}"
+            elsif status == :active
+              "there could be missing info in: #{attrs}"
+            end
+          end.compact.join("; ") + "\n"
+        end
+      end
+      logger.warn(msg)
+      sleep(2)
+    end
+  end
+
+  def unknown_headers(headers)
+    (headers - known_headers) - all_internal_attrs
+  end
+
+  def missing_headers(headers)
+    hint = headers & all_internal_attrs
+    hext = headers - hint
+    int_head = hint + hext.map {|e| fields_mapper.to_internal(e)}.compact
+    known_as_int = known_headers.select do |e|
+      i = fields_mapper.to_internal(e)
+      int_head.include?(i)
+    end
+    ext  = headers.select do |e|
+      i = fields_mapper.to_internal(e)
+      int_head.include?(i)
+    end
+    ext_present = known_as_int | ext
+    ext_miss    = known_headers - ext_present
+    #int_miss  = ext_miss.map {|ext| fields_mapper.to_internal(ext)}
+    ext_miss.each_with_object({}) do |ext, missing|
+      next unless int = fields_mapper.to_internal(ext)
+      if all_internal_attrs.include?(int)
+        missing[:direct] ||= []
+        missing[:direct] << ext
+      end
+      related_attrs_requirements = required_attrs.values.select do |req|
+        req.dependant?(int) && !int_head.include?(req.attr)
+      end
+      next if related_attrs_requirements.empty?
+      missing[:indirect] ||= {}
+      data = missing[:indirect][ext] = {}
+      data[:int]   = int
+      data[:attrs] = {}
+      related_attrs_requirements.each_with_object(data[:attrs]) do |req, attrs|
+        status = req.active?(*int_head) ? :active : :inactive
+        attrs[status] ||= []
+        attrs[status] << req.attr
+      end
+    end
+  end
+
+  def known_headers
+    @known_headers ||= fields_mapper.list(:external).compact
+  end
+
+  def fields_mapper
+    session.fields_mapper
+  end
+
+  def required_attrs
+    @required_attrs ||= person_parser.required_attrs.each_with_object({}) do |ra, out|
+      out[ra.attr] = ra
+    end
+  end
+
+  def all_internal_attrs
+    person_parser.all_attrs(include_defined_parsers: true)
+  end
+
+  def person_parser
+    session.entry_factory.person_parser
+  end
+
 end