crm_formatter 1.0.5.pre.rc.1 → 1.0.6.pre.rc.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bdb2067ec41f29fd4b3b9b0b6a839ee0af6b8a905c39afb66ebbe910c6fda6b4
-  data.tar.gz: 4a3041f79b59b2b37321d7dafda6f169c39e42e7f0c17e6dc67a5adb6f6ed4a9
+  metadata.gz: '031873afa102ba537d240f6c86c828bd29c75d85bf5063af6f6f5b715398203f'
+  data.tar.gz: a4c08972f40519a94ac636b5c5f287c68bb485bb6ae3c1126ffcc80a61dd3b2a
 SHA512:
-  metadata.gz: 05b474f9a9d2b1b638b28bfe92a29396e266339f3c81a2d9effa54a27b9896f47d871da35a0d7b0f94de754b241398c69c3b46c84b5f2158d82c56f9f2daf32e
-  data.tar.gz: c9aee22d23856645887ddc7f4388c99565f4d89ef72b93b41993109407990ad089e5f2a185a20de068778efd1455585ae3e49b45f392ad1825a79bcb53416876
+  metadata.gz: 6a37c47288003b2593422fc199d6875dfb47a0104d48d53b1bb2366a59c7038229082cb17d5b1adc14b14f4227d704c07cfd6672adb5678c7b7744a0c6fb8169
+  data.tar.gz: 4ca91da1f810d5d93451fd6d36d82878185fc33df5f32e7318537a4fa5367eb9cf91ac22c038261bbb38340cf2e207424ae9f68175ba7c80174a34e3f0662cbd

data/README.md

@@ -0,0 +1,296 @@
+# CRMFormatter
+
+Reformat and Normalize CRM Contact Data, Addresses, Phones, Emails and URLs.  
+Please note, that this gem is a rapid work in process.  It is from a collection of modules currently being used on a production app, but decided to open them up.  The tests have not yet been written for the gem and there will still be changes in the near future.  Documentation is limited, but is coming.  Here are some basic points below to help you get started.
+
+## Getting Started
+
+In your Gemfile add:
+
+```
+gem 'crm_formatter', '~> 1.0.5.pre.rc.1'
+```
+
+Or to install locally:
+
+```
+gem install crm_formatter --pre
+```
+
+### Prerequisites
+
+CRMFormatter is optimized for Rails 5.2 and Ruby 2.5, but has worked well in older versions too.
+
+### Architecture
+
+CRMFormatter is the top level Module wrapper comprising of 3 Classes, Address, Phone and Web.  There is also a Helpers Module for shared tasks.
+
+
+### Usage
+
+CRMFormatter could be used anywhere in an app for various reasons.  Most commonly from an app's helper methods, controllers or model.  If you wanted to ensure all new data is properly formatted before saving, methods could be accessed from the model via before_save, callbacks, or validations.  In addition to CRMFormatter formatting CRM data, it also provides detailed reports on what if anything has changed and it always preserves the original data submitted.  Here are a few examples below of how it could be used.
+
+The gem can be used both en mass to clean up an entire database, or fully-integrated into an app's regular environment.
+They were however, designed for an app that Harvests business data for sales and marketing teams, so they work perfectly with NokoGiri and Mechanize!
+
+** These are just examples below, not strict usage guides ...
+
+### Usage by Class & Methods
+* The examples at the bottom of the page are rather verbose, so first, here is a list of methods available to you.
+
+# Address
+
+* 'get_full_address' takes a hash of address parts then runs each through their respective formatters, then also adds an additional feature of combining them into a long full address string, and indicates if there were any changes from the original version and newly formatted.
+
+```
+  addr_formatter = CRMFormatter::Address.new
+
+  addr_formatter.get_full_address(full_address_hsh)
+  full_address_hsh = {street: street, city: city, state: state, zip: zip}
+
+  addr_formatter.format_street(street)
+
+  addr_formatter.format_city(city)
+
+  addr_formatter.format_state(state)
+
+  addr_formatter.format_zip(zip)
+
+  addr_formatter.format_full_address(adr = {})
+
+  addr_formatter.compare_versions(original, formatted)
+
+
+```
+
+# Phone
+
+* Subtle but important distinction between 'format_phone' which simply puts a phone in any format, like 555-123-4567 into normalized (555) 123-4567, and 'validate_phone' which also uses 'format_phone' to normalize its output, but is mainly tasked with determining if the phone number seem legitimate. If you know for sure that it is a phone number, but just want to normalize then first try format_phone.  If you are doing web scraping or throwing in strings of text mixed with phones, then validate_phone might work better.
+
+```
+  ph_formatter = CRMFormatter::Phone.new
+
+  ph_formatter.validate_phone(phone)
+
+  ph_formatter.format_phone(phone)
+
+```
+
+# Web
+
+
+
+```
+  web_formatter = CRMFormatter::Web.new
+
+  web_formatter.format_url(url)
+
+  web_formatter.extract_link(url_path)
+
+  web_formatter.remove_invalid_links(link)
+
+  web_formatter.remove_invalid_hrefs(href)
+
+  web_formatter.convert_to_scheme_host(url)
+
+```
+
+
+
+
+
+
+
+
+
+
+* Data will always be returned as hashes, with your original, modified, and details about what has changed.
+* You may pass optional arguments at initialization to provide lists of data to match against, for example a list of words that if in a URL, it would automatically report as junk (but still keeping your original in tact.)
+
+Typically, you will want to create your own method as a wrapper for the Gem methods, like below...
+
+## Address
+** The examples below are rather verbose, so you can make them much more compact of course.
+
+```
+def self.run_adrs
+
+  crm_address_formatter = CRMFormatter::Address.new
+
+  contacts = Contact.where.not(full_address: nil)
+
+  contacts.each do |contact|
+
+    cont_adr_hsh = { street: contact.street, city: contact.city,
+                    state: contact.state, zip: contact.zip }
+
+    formatted_address_hsh = crm_address_formatter.format_full_address(cont_adr_hsh)
+
+  end
+
+end
+
+```
+
+
+
+## Phone
+
+In the phone example, format_all_phone_in_my_db could be a custom wrapper method, which when called by Rails C or from a front end GUI process, could grab all phones in db meeting certain criteria to be scrubbed. The results will always be in hash format, such as below.... phone_hash  
+
+```
+@crm_phone = CRMFormatter::Phone.new
+
+def self.format_all_phone_in_my_db
+  phones_from_contacts = Contacts.where.not(phone: nil)
+
+  phones_from_contacts.each do |contact|
+    phone_hash = @crm_phone.validate_phone(contact.phone)  
+  end
+
+end
+
+phone_hash = { phone: 555-123-4567, valid_phone: (555) 123-4567, phone_edit: true }
+```
+
+
+## Web
+
+In the example below, you might write a wrapper method named anything you like, such as 'format_a_url' and 'clean_many_websites' to pass in urls to be formatted.
+
+```
+  @web = CRMFormatter::Web.new
+
+  def format_a_url(url)
+    hsh = @web.format_url(url)
+  end
+
+  def clean_many_websites(array_of_urls)
+
+    hashes = array_of_urls.map do |url|
+                @web.format_url(url)
+              end
+
+  end
+```
+
+
+### Additional Usage Examples
+
+### Webs
+* Another example below.
+
+```
+def self.run_webs
+
+  url_flags = %w(approv avis budget business collis eat enterprise facebook financ food google gourmet hertz hotel hyatt insur invest loan lube mobility motel motorola parts quick rent repair restaur rv ryder service softwar travel twitter webhost yellowpages yelp youtube)
+
+  link_flags = %w(: .biz .co .edu .gov .jpg .net // anounc book business buy bye call cash cheap click collis cont distrib download drop event face feature feed financ find fleet form gas generat graphic hello home hospi hour hours http info insta)
+
+  href_flags = %w(? .com .jpg @ * after anounc apply approved blog book business buy call care career cash charit cheap check click)
+
+  extension_flags = %w(au ca edu es gov in ru uk us)
+
+  args = { url_flags: url_flags, link_flags: link_flags, href_flags: href_flags, extension_flags: extension_flags }
+  web = CRMFormatter::Web.new(args)
+
+  urls = Accounts.where.not(url: nil).pluck(:url)
+
+  validated_url_hashes = urls.map { |url| web.format_url(url) }
+  valid_urls = validated_url_hashes.map { |hsh| hsh[:valid_url] }.compact
+  extracted_link_hashes = urls.map { |url| web.extract_link(url) }
+  links = extracted_link_hashes.map { |hsh| hsh[:link] }.compact
+  validated_link_hashes = links.map { |link| web.remove_invalid_links(link) }
+  hrefs = ["Hot Inventory", "Join Our Sale!", "Don't Wait till Later", "Apply Today!", "No Cash Down!"]
+  validated_href_hashes = hrefs.map { |href| web.remove_invalid_hrefs(href) }
+
+end
+
+```
+
+### Fully Integrated into an App Example
+** The gem is currently being used within another app in the following way...
+
+```
+  @web_formatter.convert_to_scheme_host(url)
+  @web_formatter.format_url(url)
+```
+ The two methods above, which are many available to you in the gem are being used below.
+```
+  curl_result[:response_code] = result&.response_code.to_s
+  web_hsh = @web_formatter.format_url(result&.last_effective_url)
+
+  if web_hsh[:formatted_url].present?
+    curl_result[:verified_url] = @web_formatter.convert_to_scheme_host(web_hsh[:formatted_url])
+  end
+
+```
+
+# Above is an isolated sliver of the larger environment shown below...
+
+```
+  def start_curl(url, timeout)
+
+    curl_result = { verified_url: nil, response_code: nil, curl_err: nil }
+    if url.present?
+      result = nil
+
+      begin # Curl Exception Handling
+        begin # Timeout Exception Handling
+          Timeout.timeout(timeout) do
+            puts "\n\n=== WAITING FOR CURL RESPONSE ==="
+            result = Curl::Easy.perform(url) do |curl|
+              curl.follow_location = true
+              curl.useragent = "curb"
+              curl.connect_timeout = timeout
+              curl.enable_cookies = true
+              curl.head = true #testing - new
+            end # result
+
+            curl_result[:response_code] = result&.response_code.to_s
+            web_hsh = @web_formatter.format_url(result&.last_effective_url)
+
+            if web_hsh[:formatted_url].present?
+              curl_result[:verified_url] = @web_formatter.convert_to_scheme_host(web_hsh[:formatted_url])
+            end
+          end
+
+        rescue Timeout::Error # Timeout Exception Handling
+          curl_result[:curl_err] = "Error: Timeout"
+        end
+
+      rescue LoadError => e  # Curl Exception Handling
+        curl_err = error_parser("Error: #{$!.message}")
+        # CheckInt.new.check_int if curl_err.include?('TCP')
+        curl_result[:curl_err] = curl_err
+      end
+    else ## If no url present?
+      curl_result[:curl_err] = 'URL Nil'
+    end
+
+    print_result(curl_result)
+    curl_result
+  end
+```
+
+
+### Another Example from a Production Environment...
+
+```
+  def format_url(url)
+    url_hash = @web_formatter.format_url(url)
+    url_hash.merge!({ verified_url: nil, url_redirected: false, response_code: nil, url_sts: nil, url_date: Time.now, wx_date: nil, timeout: nil })
+    url_hash = evaluate_formatted_url(url_hash)
+  end
+
+```
+
+
+
+## Author
+
+Adam J Booth  - [4rlm](https://github.com/4rlm)
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE.md](LICENSE.md) file for details

data/gem_notes_crm_formatter.txt

@@ -0,0 +1,109 @@
+==== FIRST CREATE APP TO TEST GEM ====
+-----------------------------------
+rails new gem_tester
+cd gem_tester
+rails g scaffold Article
+rails g controller Home index
+-----------------------------------
+get 'home/index'
+root 'home#index'
+-----------------------------------
+
+
+==== CREATE BASIC GEM ====
+-----------------------------------
+create crm_formatter.gemspec
+-----------------------------------
+mkdir bin lib test lib/crm_formatter
+-----------------------------------
+touch Rakefile
+touch lib/crm_formatter.rb
+touch lib/crm_formatter/version.rb
+touch lib/crm_formatter/translator.rb
+touch bin/crm_formatter
+touch test/test_crm_formatter.rb
+-----------------------------------
+$ git init .
+$ ls -la
+$ rm .DS_Store
+$ git status
+$ git add .
+$ git commit -m "Initial Commit"
+$ git log
+$ git status
+$ git ls-files -z
+---- Create Repo on GitHub, then push.
+-----------------------------------
+$ gem build crm_formatter.gemspec
+  Successfully built RubyGem
+  Name: crm_formatter
+  Version: 1.0.5.pre.rc.1
+  File: crm_formatter-1.0.5.pre.rc.1.gem
+-----------------------------------
+$ gem install crm_formatter-1.0.5.pre.rc.1.gem
+  Successfully installed crm_formatter-1.0.5.pre.rc.1
+  Parsing documentation for crm_formatter-1.0.5.pre.rc.1
+  Installing ri documentation for crm_formatter-1.0.5.pre.rc.1
+  Done installing documentation for crm_formatter after 0 seconds
+  1 gem installed
+  -----------------------------------
+$ irb
+  >> require 'crm_formatter'
+  => true
+  >> CRMFormatter.hi 'english'
+  => "hello world"
+  -----------------------------------
+$ curl -u adamjbooth https://rubygems.org/api/v1/api_key.yaml > ~/.gem/credentials; chmod 0600 ~/.gem/credentials
+  Enter host password for user 'adamjbooth': RG<usual>rg
+  -----------------------------------
+$ gem push crm_formatter-1.0.5.pre.rc.1.gem
+  Pushing gem to https://rubygems.org...
+  Successfully registered gem: crm_formatter (1.0.4.pre.rc.1)
+  -----------------------------------
+
+
+============= after each change: =======================
+gem build crm_formatter.gemspec
+gem install crm_formatter-1.0.3.pre.rc.1.gem
+irb
+require 'crm_formatter'
+CRMFormatter.hi('english')
+
+Translator.new.greeter
+
+CRMFormatter
+=========================
+
+CRMFormatter::Address.new
+CRMFormatter::Address.new.state
+CRMFormatter::Address.self_state
+
+/Users/Adam/Desktop/gems/crm_formatter/lib/crm_formatter.rb
+====================================================================
+
+
+require '/Users/Adam/Desktop/gems/crm_formatter/lib/crm_formatter/address.rb'
+crm_phone = CRMFormatter::Phone.new
+crm_phone.validate_phone("229-954-6172")
+
+=> [{:phone=>"229-954-6172", :valid_phone=>"(229) 954-6172", :change=>true},
+ {:phone=>"4.142285305606521", :valid_phone=>nil, :change=>nil},
+ {:phone=>"23870 Addie Green", :valid_phone=>nil, :change=>nil},
+ {:phone=>"73408-0942", :valid_phone=>nil, :change=>nil},
+ {:phone=>"8853 Hank Course", :valid_phone=>nil, :change=>nil},
+ {:phone=>"910-202-3756 x827", :valid_phone=>"(910) 202-3756", :change=>true},
+ {:phone=>"119-305-4317", :valid_phone=>"119-305-4317", :change=>false},
+ {:phone=>"197-294-8073 x2164", :valid_phone=>"(972) 948-0732", :change=>true},
+ {:phone=>"53.866282605787006", :valid_phone=>nil, :change=>nil},
+ {:phone=>"91783", :valid_phone=>nil, :change=>nil}]
+
+
+
+
+
+
+address = CRMFormatter::Address.new
+address.state
+
+address = CRMFormatter::Address.new.state
+address.gh

data/lib/crm_formatter/version.rb

@@ -1,3 +1,3 @@
 module CRMFormatter
-  VERSION = "1.0.5-rc.1"
+  VERSION = "1.0.6-rc.1"
 end

data/lib/crm_formatter/web.rb

@@ -5,7 +5,7 @@ module CRMFormatter
         @url_flags = args.fetch(:url_flags, [])
         @link_flags = args.fetch(:link_flags, [])
         @href_flags = args.fetch(:href_flags, [])
-        @extension_flags = args.fetch(:ext_flags, [])
+        @extension_flags = args.fetch(:extension_flags, [])
         @length_min = args.fetch(:length_min, 2)
         @length_max = args.fetch(:length_max, 100)
       end
@@ -28,14 +28,15 @@ module CRMFormatter
             url = url[0..-2] if url[-1] == '/'
 
             symbs = ['(', ')', '[', ']', '{', '}', '*', '@', '^', '$', '+', '!', '<', '>', '~', ',', "'"]
+
             return url_hsh if symbs.any? {|symb| url&.include?(symb) }
 
             uri = URI(url)
             if uri.present?
               host_parts = uri.host&.split(".")
 
-              if @ext_flags.any?
-                bad_host_sts = host_parts&.map { |part| TRUE if @ext_flags.any? {|ext| part == ext } }&.compact&.first
+              if @extension_flags.any?
+                bad_host_sts = host_parts&.map { |part| TRUE if @extension_flags.any? {|ext| part == ext } }&.compact&.first
                 return url_hsh if bad_host_sts
               end
 
@@ -44,7 +45,9 @@ module CRMFormatter
               url = "#{scheme}://#{host}" if host.present? && scheme.present?
               url = "http://#{url}" if url[0..3] != "http"
               url = url.gsub("//", "//www.") if !url.include?("www.")
+
               return url_hsh if @url_flags.any? { |bad_text| url&.include?(bad_text) }
+
               url_hsh[:formatted_url] = convert_to_scheme_host(url) if url.present?
               url_hsh[:url_edit] = url_hsh[:formatted_url] != url_hsh[:url_path]
             end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: crm_formatter
 version: !ruby/object:Gem::Version
-  version: 1.0.5.pre.rc.1
+  version: 1.0.6.pre.rc.1
 platform: ruby
 authors:
 - Adam Booth
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2018-05-13 00:00:00.000000000 Z
+date: 2018-05-14 00:00:00.000000000 Z
 dependencies: []
 description: Reformat and Normalize CRM Contact Data, Addresses, Phones, Emails and
   URLs.  Originally developed for proprietary use in an enterprise software suite.  Recently
@@ -23,9 +23,11 @@ extensions: []
 extra_rdoc_files: []
 files:
 - ".gitignore"
+- README.md
 - Rakefile
 - bin/crm_formatter
 - crm_formatter.gemspec
+- gem_notes_crm_formatter.txt
 - lib/crm_formatter.rb
 - lib/crm_formatter/address.rb
 - lib/crm_formatter/helpers.rb