eco-helpers 2.0.22 → 2.0.27

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ba43099c7902b3e2623640a196abd86f39215403d28e7a614da78b2422629323
-  data.tar.gz: 7e3701cbeb7b3a0a65453dc002d6f4c5cb6e6ec4103e6cd0178c1af333d29d7d
+  metadata.gz: 144eb5a5b4c0069c07c1af3bac6c23c43b975643f45b8b7d8e5edd05daa8defd
+  data.tar.gz: 6f2c5e8f1744380cb543d11087b3d5bbab822697e876c21c8a4ca1984ea5f3a9
 SHA512:
-  metadata.gz: f14ab296c2cff2dc694ef9182593329ca1bc86cf08ca32ba879cd5e74d658d1d123add425bc62a981182b9e4da06b33d225d613cd1a9d9cbfd7284e6b6d24ea5
-  data.tar.gz: f020822ad4ae2e18f1f0424bb3a47417ff9a6bc78a439517cee3cd9212548efe4c89256579d73ccde27709dc5d2ea3d3e74bd072de23d4c089ccf3c7a107fc07
+  metadata.gz: 8bb382dbc126f15308cdd51f3ec8d520a154450080388e329243102e0fa2c4d9dcbba395ec91f59ea645ea734f222a5af8fe25a1000a39b2474aba2c89a725dd
+  data.tar.gz: 11874a475a744e673e8741777079d54081c0b6be855a609cf7ed6e8a032f29c6e074cefe7e8a94d57944d70be4b8bebbcb5d6a7ff81e25d6e2798ce4259779b1

data/CHANGELOG.md

@@ -1,6 +1,101 @@
 # Change Log
 All notable changes to this project will be documented in this file.
 
+## [2.0.27] - 2021-06-xx
+
+### Added
+
+### Changed
+  -  Parent class `Eco::API::UseCases::DefaultCases::ToCsvCase` shouldn't know header maps necessary for `Eco::API::UseCases::DefaultCases::ToCsvDetailedCase`
+    * Moved related header maps to `ToCsvDetailedCase`
+
+### Fixed
+  - `--help` should work fine now
+  - Attribute parsers that depended on indirect parser attributes were **not** included
+    - **changed** a couple of classes
+      - `Eco::API::Common::People::PersonEntryAttributeMapper`, where methods should receive `data` to re-scope if the data is available (most specifically `#internal_attrs`)
+      - `Eco::API::Common::People::PersonEntry`, where when setting the final values on `Person`, it should include any internal attribute that is present in the `final_entry` (the result of all the parsing process)
+  - `Eco::API::Common::People::DefaultParsers::CSVParser`
+    - indirect attributes that depended on other indirect attributes would show as missing even if they were active
+      - i.e. `name` depends on `first-name` & `surname`, **AND** `details-name` depends on `name`
+
+## [2.0.26] - 2021-06-25
+
+### Added
+  - `Eco::API::MicroCases#set_supervisor`, tries to keep in sync the `#subordinates` **count** of previous and new supervisor
+  - new **option** `-run-postlaunch` to run post launch cases, even when we run in `dry-run` mode
+    * when in `dry-run` it will **not** reload the people base of the session
+  - new **option** to append **new** entries to the `People` object
+    * **invokable** on **cli** via `-append-starters` (`{people: {append_created: true}}`)
+    * the following use cases include this option:
+      * `Eco::API::UseCases::DefaultCases::UpsertCase`
+      * `Eco::API::UseCases::DefaultCases::HrisCase`
+      * `Eco::API::UseCases::DefaultCases::CreateDetailsWithSupervisorCase`
+      * `Eco::API::UseCases::DefaultCases::CreateDetailsCase`
+      * `Eco::API::UseCases::DefaultCases::CreateCase`
+    * the option involves a new keyed argument `:append_created` in a couple of `MicroCases`
+      * `Eco::API::MicroCases#with_each`: where internally the search is performed against a copy of the `People` object.
+      * `Eco::API::MicroCases#with_each_starter`
+    * when `--help` is invoked option to filter the shown `-options`, `-usecases` and `filters` by a word contained in the option.
+  - **added** `csv` **header checks** for feed files, which entailed some changes:
+    * `Eco::API::Common::Loaders::Parser`
+      - new **subclass** `RequiredAttrs`, creatred when calling `.active_when_all` and `.active_when_any`
+      - **dependency** injection via `.dependencies` as `{required_attrs: RequiredAttrs}`
+    * **added** `Eco::API::Common::People::PersonParser#required_attrs` to offer all the `RequiredAttrs`, where defined
+      - the **new method** `#required_attrs` to expose the injected `RequiredAttrs`
+    * **new** keyed argument `check_headers:` in `Eco::API::Common::People::EntryFactory#entries`
+      - subsequent changes to accommodate the new param in `Eco::API::Session#csv_entries`
+      - `eco/cli/config/default/input` calls using this param to `true`
+    * `Eco::API::Common::People::DefaultParsers::CSVParser`
+      - added option `check_headers` via `dependencies` that enables the headers check
+      - it will now offer detailed warning messages on what can happen with the **missing headers**
+        * it will also list the **unknown header** names
+
+### Changed
+  - `Eco::API::MicroCases#set_supervisor`, the order of the 2 first parameters
+  - `Eco::API::Organization::People`: internally `@by_id` cache Hash included `nil` values => **not** any more.
+  - removed **unused** methods on **patches** for `Ecoportal::API::V1::Person` and `Ecoportal::API::Internal::Person`
+    * specifically `#reset_account!` and `#consolidate_account!` as well as `#reset_details!` and `#consolidate_details!`
+  - internal changes in `Eco::API::Common::People::Entries#entry`
+    * **added** option to trigger `MultipleSearchResults` StandardError when multiple candiates are found.
+    * **removed** `nil` values from the `caches` (the Hashes to optimize the search)
+  - slight structure refactor of `Eco::API::Common::Loaders`
+    * moved base class to subfolder/namespace
+    * decoupled pure `Loader` logics to `Loaders::Base` and use case inheriance chain loader to `Loaders::CaseBase`
+  - `Eco::API::Session::BatchJob` the `post_launch`:
+    * sets the `id` to the `person` if it was **created** successfully
+    * when in `dry-run` it fakes the `id` with a counter
+  - `Eco::API::Common::People::PersonFactory` gets `subordinates` initialized to `0` (when **creating** a `new` person)
+
+### Fixed
+
+
+## [2.0.25] - 2021-06-23
+
+### Added
+  - `Eco::API::UseCases::DefaultCases::HrisCase` validation error to require `-schema-id` command line when there are people in schemas other than the active one
+
+### Changed
+  - `Eco::API::Session::Batch::Job`
+    * for backwards compatibility `-include-only-excluded` should bring an options structure compatible with `-include-excluded`
+
+### Fixed
+  - `Eco::API::Session::Batch` fixed typo that would prevent `prompt_user` to work
+
+## [2.0.24] - 2021-06-22
+
+### Added
+  - `Eco::API::Session::Batch::Job` made **native** `-include-excluded`
+    * also added new option `-include-only-excluded` to be able to only target people HRIS excluded
+
+## [2.0.23] - 2021-06-22
+
+### Added
+  - `Eco::API::Session::Batch::Job` **new** option (`-save-requests`) to save requests even if in `dry-run` (`-simulate`)
+### Changed
+  - `Eco::API::Session::Batch::Job` new people won't create updates unless they have either details or account
+    * because that entry is not supposed to be created unless has account or details
+
 ## [2.0.22] - 2021-06-18
 
 ### Added
@@ -24,10 +119,6 @@ All notable changes to this project will be documented in this file.
     * we just kept `roo` and `roo-xls`
   - custom `Error` classes now all inherit from `StandardError` (rather than `Exception`)
 
-
-### Fixed
-
-
 ## [2.0.21] - 2021-06-04
 
 ### Added
@@ -38,7 +129,6 @@ All notable changes to this project will be documented in this file.
 ### Changed
   - `Eco::API::Common::People::EntryFactory` slight **refactor** to boost better support for multiple input formats
 
-
 ## [2.0.20] - 2021-05-31
 
 ### Added

data/lib/eco/api/common.rb

@@ -10,7 +10,6 @@ require_relative 'common/class_helpers'
 require_relative 'common/class_auto_loader'
 require_relative 'common/class_hierarchy'
 require_relative 'common/class_meta_basics'
-require_relative 'common/base_loader'
 require_relative 'common/loaders'
 require_relative 'common/session'
 require_relative 'common/people'

data/lib/eco/api/common/loaders.rb

@@ -7,6 +7,8 @@ module Eco
   end
 end
 
+require_relative 'loaders/base'
+require_relative 'loaders/case_base'
 require_relative 'loaders/use_case'
 require_relative 'loaders/policy'
 require_relative 'loaders/error_handler'

data/lib/eco/api/common/loaders/base.rb

@@ -0,0 +1,58 @@
+module Eco
+  module API
+    module Common
+      module Loaders
+        class Base
+          extend Eco::API::Common::ClassHelpers
+
+          class << self              
+            # Sort order
+            def <=>(other)
+              created_at <=> other.created_at
+            end
+
+            # If still not set, it sets the `created_at` class timestamp.
+            def set_created_at!
+              @created_at = Time.now unless @created_at
+            end
+
+            # Class creation timestamp, to be able to load them in the order they were declared.
+            def created_at
+              @created_at ||= Time.now
+            end
+          end
+
+          # This method will be called when the BaseLoader is created
+          # @note
+          #   - this method should implement the loading logics for the given `Children` class.
+          def initialize
+            raise "You should implement this method"
+          end
+
+          def name
+            self.class.name
+          end
+
+          private
+
+          def session
+            ASSETS.session
+          end
+
+          def config
+            session.config
+          end
+
+          def logger
+            session.logger
+          end
+
+          def micro
+            session.micro
+          end
+
+        end
+      end
+    end
+  end
+end

data/lib/eco/api/common/loaders/case_base.rb

@@ -0,0 +1,33 @@
+module Eco
+  module API
+    module Common
+      module Loaders
+        class CaseBase < Loaders::Base
+
+          class << self
+            attr_writer :name, :type
+
+            # The name that this case, policy or error handler will have.
+            def name(value = nil)
+              name_only_once! if value
+              set_created_at!
+              return @name ||= self.to_s unless value
+              @name = value
+            end
+
+            # Prevent the same class to be re-opened/re-named
+            def name_only_once!
+              raise "You have already declared #{self} or you are trying to give it a name twice" if @name
+            end
+
+          end
+
+          def name
+            self.class.name
+          end
+
+        end
+      end
+    end
+  end
+end

data/lib/eco/api/common/loaders/error_handler.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class ErrorHandler < Eco::API::Common::BaseLoader
+        class ErrorHandler < Eco::API::Common::Loaders::CaseBase
 
           class << self
             attr_writer :error
@@ -17,7 +17,7 @@ module Eco
           end
 
           inheritable_class_vars :error
-          
+
           def initialize(handlers)
             raise "Expected Eco::API::Policies. Given #{handlers.class}" unless handlers.is_a?(Eco::API::Error::Handlers)
             handlers.on(self.error, &self.method(:main))

data/lib/eco/api/common/loaders/parser.rb

@@ -2,7 +2,28 @@ module Eco
   module API
     module Common
       module Loaders
-        class Parser < Eco::API::Common::BaseLoader
+        class Parser < Eco::API::Common::Loaders::CaseBase
+
+          # Helper class to scope what required attributes it depends on
+          class RequiredAttrs < Struct.new(:attr, :type, :attrs)
+            def active?(*input_attrs)
+              missing(*input_attrs).empty?
+            end
+
+            def dependant?(attr)
+              attrs.include?(attr)
+            end
+
+            def missing(*input_attrs)
+              return [] if input_attrs.include?(attr)
+              match = input_attrs & attrs
+              miss  = attrs - match
+              return []    if miss.empty?
+              return attrs if match.empty?
+              return miss  if type == :all
+              []
+            end
+          end
 
           class << self
             attr_reader :active_when
@@ -19,12 +40,14 @@ module Eco
               @attribute = value
             end
 
-            # TODO: it migh rather merge?
             # Some parsers require dependencies to do their job.
-            def dependencies(value = nil)
+            def dependencies(**value)
               @dependencies ||= {}
-              return @dependencies unless value
-              @dependencies = value
+              return @dependencies.merge({
+                required_attrs: @active_when_attrs
+              }) unless !value.empty?
+              raise "Expected Hash. Given: '#{value.class}'" unless value.is_a?(Hash)
+              @dependencies.merge!(value)
             end
 
             # Define or get the `phase` that the `parser` kicks in.
@@ -47,6 +70,7 @@ module Eco
 
             # Helper to build the `active_when` condition.
             def active_when_any(*attrs)
+              @active_when_attrs = RequiredAttrs.new(attribute, :any, attrs)
               @active_when = Proc.new do |source_data|
                 keys = data_keys(source_data)
                 attrs.any? {|key| keys.include?(key)}
@@ -55,6 +79,7 @@ module Eco
 
             # Helper to build the `active_when` condition.
             def active_when_all(*attrs)
+              @active_when_attrs = RequiredAttrs.new(attribute, :all, attrs)
               @active_when = Proc.new do |source_data|
                 keys = data_keys(source_data)
                 attrs.all? {|key| keys.include?(key)}

data/lib/eco/api/common/loaders/policy.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class Policy < Eco::API::Common::BaseLoader
+        class Policy < Eco::API::Common::Loaders::CaseBase
 
           def initialize(policies)
             raise "Expected Eco::API::Policies. Given #{policies.class}" unless policies.is_a?(Eco::API::Policies)

data/lib/eco/api/common/loaders/use_case.rb

@@ -2,7 +2,7 @@ module Eco
   module API
     module Common
       module Loaders
-        class UseCase < Eco::API::Common::BaseLoader
+        class UseCase < Eco::API::Common::Loaders::CaseBase
 
           class << self
             # @return [Symbol] the `type` of usecase (i.e. `:sync`, `:transform`, `:import`, `:other`)

data/lib/eco/api/common/people/default_parsers/csv_parser.rb

@@ -2,7 +2,9 @@ class Eco::API::Common::People::DefaultParsers::CSVParser < Eco::API::Common::Lo
   attribute :csv
 
   def parser(data, deps)
-    Eco::CSV.parse(data, headers: true, skip_blanks: true).each_with_object([]) do |row, arr_hash|
+    Eco::CSV.parse(data, headers: true, skip_blanks: true).tap do |table|
+      check_headers(table) if deps[:check_headers]
+    end.each_with_object([]) do |row, arr_hash|
       row_hash = row.headers.uniq.each_with_object({}) do |attr, hash|
         next if attr.to_s.strip.empty?
         hash[attr.strip] = parse_string(row[attr])
@@ -36,4 +38,130 @@ class Eco::API::Common::People::DefaultParsers::CSVParser < Eco::API::Common::Lo
     ["NULL"].any? {|token| str == token}
   end
 
+  def check_headers(table)
+    headers = table.headers
+    missing = missing_headers(headers)
+    unknown = unknown_headers(headers)
+    unless missing.empty? && unknown.empty?
+      msg  = "Detected possible HEADER ISSUES !!!\n"
+      msg << "There might be Missing or Wrong HEADER names in the CSV file:\n"
+      msg << "  * UNKNOWN (or not used?): #{unknown}\n" unless unknown.empty?
+      msg << "  * MISSING DIRECT: #{missing[:direct]}\n" unless (missing[:direct] || []).empty?
+      unless (data =  missing[:indirect] || []).empty?
+        msg << "  * MISSING INDIRECT:\n"
+        data.each do |ext, info|
+          msg <<  "    - '#{ext}' => "
+          msg << (info[:attrs] || {}).map do |status, attrs|
+            if status == :inactive
+              "makes inactive: #{attrs}"
+            elsif status == :active
+              "there could be missing info in: #{attrs}"
+            end
+          end.compact.join("; ") + "\n"
+        end
+      end
+      logger.warn(msg)
+      sleep(2)
+    end
+  end
+
+  def unknown_headers(headers)
+    (headers - known_headers) - all_internal_attrs
+  end
+
+  def missing_headers(headers)
+    int_head = internal_present_or_active(headers)
+    ext  = headers.select do |e|
+      i = fields_mapper.to_internal(e)
+      int_head.include?(i)
+    end
+    ext_present = known_headers_present(int_head) | ext
+    ext_miss    = known_headers - ext_present
+
+    ext_miss.each_with_object({}) do |ext, missing|
+      next unless int = fields_mapper.to_internal(ext)
+      if all_internal_attrs.include?(int)
+        missing[:direct] ||= []
+        missing[:direct] << ext
+      end
+      related_attrs_requirements = required_attrs.values.select do |req|
+        dep       = req.dependant?(int)
+        affects   = dep && !int_head.include?(int)
+        in_header = int_head.include?(req.attr)
+        affects || (dep && !in_header)
+      end
+      next if related_attrs_requirements.empty?
+      missing[:indirect] ||= {}
+      data = missing[:indirect][ext] = {}
+      data[:int]   = int
+      data[:attrs] = {}
+      related_attrs_requirements.each_with_object(data[:attrs]) do |req, attrs|
+        status = req.active?(*int_head) ? :active : :inactive
+        attrs[status] ||= []
+        attrs[status] << req.attr
+      end
+    end
+  end
+
+  def known_headers_present(headers_internal)
+    @known_internal ||= known_headers.select do |ext|
+      int = fields_mapper.to_internal(ext)
+      headers_internal.include?(int)
+    end
+  end
+
+  def internal_present_or_active(headers, inactive_requirements = {})
+    hint = headers & all_internal_attrs
+    hext = headers - hint
+    int_present = hint + hext.map {|e| fields_mapper.to_internal(e)}.compact
+
+    update_inactive = Proc.new do
+      inactive_requirements.dup.each do |attr, req|
+        if req.active?(*int_present)
+          inactive_requirements.delete(attr)
+          int_present << attr
+          update_inactive.call
+        end
+      end
+    end
+
+    required_attrs.values.each do |req|
+      next if int_present.include?(req)
+      if req.active?(*int_present)
+        inactive_requirements.delete(req.attr)
+        int_present << req.attr
+        update_inactive.call
+      else
+        inactive_requirements[req.attr] = req
+      end
+    end
+    int_present
+  end
+
+  def known_headers
+    @known_headers ||= fields_mapper.list(:external).compact
+  end
+
+  def fields_mapper
+    session.fields_mapper
+  end
+
+  def required_attrs
+    @required_attrs ||= person_parser.required_attrs.each_with_object({}) do |ra, out|
+      out[ra.attr] = ra
+    end
+  end
+
+  def all_internal_attrs
+    @all_internal_attrs ||= [].tap do |int_attrs|
+      known_int_attrs  = person_parser.all_attrs(include_defined_parsers: true)
+      known_int_attrs |= fields_mapper.list(:internal).compact
+      int_attrs.concat(known_int_attrs)
+    end
+  end
+
+  def person_parser
+    session.entry_factory.person_parser
+  end
+
 end