wgit 0.10.8 → 0.12.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 66e8b435303d07b2f81d260badc96662936599c9782916f7f014b74a7c617499
-  data.tar.gz: 7b55890c66ec09efd8d5749bd66605a4cb43d5091416f072f8fcc5aaaa85fbe7
+  metadata.gz: 4dee43af6274102c9bc6ad4f32c8811f57c5dc2833e923e038aac8f7f2072385
+  data.tar.gz: 9463768a40c78ab9c91ac34dd1c3a0fd1e7b990440b70766feebb9e2f0f99bd4
 SHA512:
-  metadata.gz: fe1b605224f6682ac504f17b55ab83518556f1320f0410741af8f95bf3a669918c69b48832fb413ca1f78482fdbb7e0d2e7d6f57841c6a562b7f926f7511cdd7
-  data.tar.gz: 856be2111709bc96488b7d43abbc49c563a9a56330344adb4b9ec40fc263cb91e63465c3c3dab317c0d8930965a609a43102d53d80bbc2001e6165a15cb905fa
+  metadata.gz: 5c94fcae3a56254a6c0d9d67597f1a2125439c1ee3d7d68e22fb70fa59298735d76b0fb8e77bc44b0850f6ba561fe11df3a867973d5f0533adddda9d2c6f2002
+  data.tar.gz: 779bf20dc1eaa29cc926a836d5a5a155c2270db1be0d22bc52ba9893cbd8d3aaec2cee7810b660fb9014020f5de2290af2b23680cc040a9c682ca82431d6f50d

data/CHANGELOG.md

@@ -1,6 +1,6 @@
 # Wgit Change Log
 
-## v0.0.0 (TEMPLATE - DO NOT EDIT)
+## v0.0.0 [- BREAKING CHANGES] (TEMPLATE - DO NOT EDIT)
 ### Added
 - ...
 ### Changed/Removed
@@ -9,6 +9,77 @@
 - ...
 ---
 
+## v0.12.0 - BREAKING CHANGES
+A big release with several breaking changes, not all of which can be listed below. The headline features for this release are the introduction of a database adapter, allowing Wgit to work with practically any underlying database system; and a custom in-house text extractor.
+### Added
+- `Wgit::Database::DatabaseAdapter` class to allow the easy development of other database adapters that work with Wgit.
+- `Wgit::Database::InMemory` database adapter class and logic for quick and easy usage of Wgit on the fly (without needing a MongoDB instance to be running). This also serves as an example of how to write your own database adapter class.
+- `Wgit::HTMLToText` class and logic for extracting text from a HTML string. This is now how a `Wgit::Document` extracts its text for searching etc. This class is different in that it not only focuses on the elements we specify, but also applies those elements display in how it delimits where one sentence ends and the next starts e.g. `<span>` is `:inline` etc.
+- `Wgit::Document#nearest_fragment` method which allows you to search for the nearest fragement (#blah) to a piece of text and/or element type.
+- `Wgit::Model.search_fields` and `.set_search_fields` for setting which search fields are used in document and database adapter searches.
+- `Wgit::Model.include_doc_html` and `include_doc_score` methods to set in one place if html/score is included in the corresponding document model output.
+- `Wgit::Assertable#common_arr_types` method.
+- `Wgit::Utils.pprint(display:)` param to turn off all logs easily (by setting from ENV etc).
+- `Wgit::Crawler#crawl_site(max_pages:)` param to quit the crawl early.
+- `Wgit::Utils.pprint_search_results(include_score:)` param to print the score of each result.
+- `Wgit::Utils.pprint_all_search_results` method to print all matches for each result.
+- `Wgit::DSL#search` params `include_score:` and `top_result_only:`.
+### Changed/Removed
+- Update Wgit to work with ruby v3 and above, removed support for older versions.
+- Removed default extractors `meta_robots` and `meta_wgit` without losing any functionality.
+- `Wgit::Database::Model` is now moved up a level to become `Wgit::Model`.
+- `Wgit::Database` is now `Wgit::Database::MongoDB` and is alised to `Wgit::Database.adapter_class`.
+- Renamed any `Wgit::Database` and `Wgit::DSL` methods called `#clear_*` to be `#empty_*`.
+- Moved `Wgit::MongoDB#search!` to `Wgit::DSL#search`.
+- Renamed `Wgit::MongoDB#search_text` to `Wgit::Database::MongoDB#search!`.
+- Reworked `Wgit::DSL` methods to accept a database param instead of a connection_string. This offers better flexibility.
+- Updated the Wgit `Dockerfile` to pull from `mongo:latest`.
+- `Wgit::Document#search` now searches more than just it's text. See `Wgit::Model.search_fields` for more information.
+- `Wgit::Document` now has the following search methods: `#search`, `#search_text`, `#search_text!`. See their documentation and source code for more information.
+- `Wgit::Document.define_extractor` now accepts a `nil` xpath parameter which omits the xpath search.
+- `Wgit::Indexer#index_*` methods now use `:skip` block return value to avoid DB upserts.
+- Updated `Wgit::Crawler.new` to accept `ferrum_opts:` which are passed to directly ferrum on init.
+- `Wgit::Document#search` now accepts a `&block` which yields the results Hash.
+- Updated `Wgit::Indexer#index_*` methods to accept `ENV["WGIT_IGNORE_ROBOTS_TXT"]` ignoring a site's robot.txt file.
+### Fixed
+- Issue [Document#search only searches the text](https://github.com/michaeltelford/wgit/issues/2)
+- Issue [Document#search doesn't find certain text](https://github.com/michaeltelford/wgit/issues/20)
+- Issue with `Wgit::Document#search(whole_sentence:)` search capability by updating the regex used.
+- Issue with `Wgit::Document#search` which now totals the text score for duplicated results.
+---
+
+## v0.11.0 - BREAKING CHANGES
+This release is a biggie with the main headline being the introduction of robots.txt support (see below). This release introduces several breaking changes so take care when updating your current version of Wgit.
+### Added
+- Ability to prevent indexing via `robots.txt` and `noindex` values in HTML `meta` elements and HTTP response header `X-Robots-Tag`. See new class `Wgit::RobotsParser` and the updated `Wgit::Indexer#index_*` methods. Also see the [wiki article](https://github.com/michaeltelford/wgit/wiki/How-To-Prevent-Indexing) on the subject.
+- `Wgit::RobotsParser` class for parsing `robots.txt` files.
+- `Wgit::Response#no_index?` and `Wgit::Document#no_index?` methods (see wiki article above).
+- Added two new default extractors which extract robots meta elements for use in `Wgit::Document#no_index?`.
+- Added `Wgit::Document.to_h_ignore_vars` Array for user manipulation.
+- Added `Wgit::Utils.pprint` method to aid debugging.
+- Added `Wgit::Utils.sanitize_url` method.
+- Added `Wgit::Indexer#index_www(max_urls_per_iteration:, ...)` param.
+- Added `Wgit::Url#redirects` and `#redirects=` methods.
+- Added `Wgit::Url#redirects_journey` used by `Wgit::Indexer` to insert a Url and it's redirects.
+- Added `Wgit::Database#bulk_upsert` which `Wgit::Indexer` now uses where possible. This reduces the total database calls made during an index operation.
+### Changed/Removed
+- Updated `Wgit::Indexer#index_*` methods to honour index prevention methods (see the [wiki article](https://github.com/michaeltelford/wgit/wiki/How-To-Prevent-Indexing)).
+- Updated `Wgit::Utils.sanitize*` methods so they no longer modify the receiver.
+- Updated `Wgit::Crawler#crawl_url` to always return the crawled `Wgit::Document`. If relying on `nil` in your code, you should now use `doc.empty?` instead.
+- Updated `Wgit::Indexer` method logs.
+- Updated/added custom class `#inspect` methods.
+- Renamed `Wgit::Utils.printf_search_results` to `pprint_search_results`.
+- Renamed `Wgit::Url#concat` to `#join`. The `#concat` method is now `String#concat`.
+- Updated `Wgit::Indexer` methods to now write external Urls to the Database as: `doc.external_urls.map(&:to_origin)` meaning `http://example.com/about` becomes `http://example.com`.
+- Updated the following methods to no longer omit trailing slashes from Urls: `Wgit::Url` - `#to_path`, `#omit_base`, `#omit_origin` and `Wgit::Document` - `#internal_links`, `#internal_absolute_links`, `#external_links`. For an average website, this results in ~30% less network requests when crawling.
+- Updated Ruby version to `3.3.0`.
+- Updated all bundle dependencies to latest versions, see `Gemfile.lock` for exact versions.
+### Fixed
+- `Wgit::Crawler#crawl_site` now internally records all redirects for a given Url.
+- `Wgit::Crawler#crawl_site` infinite loop when using Wgit on a Ruby version > `3.0.2`.
+- Various other minor fixes/improvements throughout the code base.
+---
+
 ## v0.10.8
 ### Added
 - Custom `#inspect` methods to `Wgit::Url` and `Wgit::Document` classes.

data/CODE_OF_CONDUCT.md

@@ -55,7 +55,7 @@ further defined and clarified by project maintainers.
 ## Enforcement
 
 Instances of abusive, harassing, or otherwise unacceptable behavior may be
-reported by contacting the project team at michael.telford@live.com. All
+reported by contacting the project team on Github. All
 complaints will be reviewed and investigated and will result in a response that
 is deemed necessary and appropriate to the circumstances. The project team is
 obligated to maintain confidentiality with regard to the reporter of an incident.

data/CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 ## Consult
 
-Before you make a contribution, reach out to michael.telford@live.com about what changes need made. Otherwise, your time spent might be wasted. Once you're clear on what needs done follow the technical steps below.
+Before you make a contribution, reach out on Github about what changes need made. Otherwise, your time spent might be wasted. Once you're clear on what needs done follow the technical steps below.
 
 ## Technical Steps
 
@@ -12,7 +12,7 @@ Before you make a contribution, reach out to michael.telford@live.com about what
 - Write some code
 - Re-run the tests (which now hopefully pass)
 - Push your branch to your `origin` remote
-- Open a GitHub Pull Request (with the target branch being wgit's `origin/master`)
+- Open a GitHub Pull Request (with the target branch as wgit's (upstream) `master`)
 - Apply any requested changes
 - Wait for your PR to be merged
 

data/README.md

@@ -2,7 +2,7 @@
 
 [![Inline gem version](https://badge.fury.io/rb/wgit.svg)](https://rubygems.org/gems/wgit)
 [![Inline downloads](https://img.shields.io/gem/dt/wgit)](https://rubygems.org/gems/wgit)
-[![Inline build](https://travis-ci.org/michaeltelford/wgit.svg?branch=master)](https://travis-ci.org/michaeltelford/wgit)
+[![Inline build](https://github.com/michaeltelford/wgit/actions/workflows/wgit.yaml/badge.svg?branch=master)](https://github.com/michaeltelford/wgit/actions)
 [![Inline docs](http://inch-ci.org/github/michaeltelford/wgit.svg?branch=master)](http://inch-ci.org/github/michaeltelford/wgit)
 [![Inline code quality](https://api.codacy.com/project/badge/Grade/d5a0de62e78b460997cb8ce1127cea9e)](https://www.codacy.com/app/michaeltelford/wgit?utm_source=github.com&utm_medium=referral&utm_content=michaeltelford/wgit&utm_campaign=Badge_Grade)
 
@@ -13,12 +13,12 @@ Wgit is a HTML web crawler, written in Ruby, that allows you to programmatically
 Wgit was primarily designed to crawl static HTML websites to index and  search their content - providing the basis of any search engine; but Wgit is suitable for many application domains including:
 
 - URL parsing
-- Document content extraction (data mining)
-- Crawling entire websites (statistical analysis)
+- Document content extraction (data mining etc)
+- Recursive website crawling (indexing, statistical analysis etc)
 
 Wgit provides a high level, easy-to-use API and DSL that you can use in your own applications and scripts.
 
-Check out this [demo search engine](https://wgit-search-engine.fly.dev) - [built](https://github.com/michaeltelford/search_engine) using Wgit and Sinatra - deployed to [fly.io](https://fly.io). Try searching for something that's Ruby related like "Matz" or "Rails".
+Check out this [demo search engine](https://wgit-search-engine.fly.dev) - [built](https://github.com/michaeltelford/search_engine) using Wgit, Sinatra and MongoDB - deployed to [fly.io](https://fly.io). Try searching for something that's Ruby related like "Matz" or "Rails".
 
 ## Table Of Contents
 
@@ -78,7 +78,7 @@ Which outputs:
 ]
 ```
 
-Great! But what if we want to crawl and store the content in a database, so that it can be searched? Wgit makes it easy to index and search HTML using [MongoDB](https://www.mongodb.com/):
+Great! But what if we want to crawl and store the content in a database, so that it can be searched? Wgit makes it easy to index and search HTML using [MongoDB](https://www.mongodb.com/) (by default):
 
 ```ruby
 require 'wgit'
@@ -87,7 +87,7 @@ include Wgit::DSL
 
 Wgit.logger.level = Logger::WARN
 
-connection_string 'mongodb://user:password@localhost/crawler'
+ENV['WGIT_CONNECTION_STRING'] = 'mongodb://user:password@localhost/crawler'
 
 start  'http://quotes.toscrape.com/tag/humor/'
 follow "//li[@class='next']/a/@href"
@@ -109,11 +109,11 @@ http://quotes.toscrape.com/tag/humor/page/2/
 ...
 ```
 
-Using a MongoDB [client](https://robomongo.org/), we can see that the two web pages have been indexed, along with their extracted *quotes* and *authors*:
+Using a database client, we can see that the two web pages have been indexed, along with their extracted *quotes* and *authors*:
 
 ![MongoDBClient](https://raw.githubusercontent.com/michaeltelford/wgit/assets/assets/wgit_mongo_index.png)
 
-The [DSL](https://github.com/michaeltelford/wgit/wiki/How-To-Use-The-DSL) makes it easy to write scripts for experimenting with. Wgit's DSL is simply a wrapper around the underlying classes however. For comparison, here is the above example written using the Wgit API *instead of* the DSL:
+The [DSL](https://github.com/michaeltelford/wgit/wiki/How-To-Use-The-DSL) makes it easy to write scripts for experimenting with. Wgit's DSL is simply a wrapper around the underlying classes. For comparison, here is the quote example re-written using the Wgit API *instead of* the DSL:
 
 ```ruby
 require 'wgit'
@@ -143,9 +143,9 @@ puts JSON.generate(quotes)
 There are many [other HTML crawlers](https://awesome-ruby.com/#-web-crawling) out there so why use Wgit?
 
 - Wgit has excellent unit testing, 100% documentation coverage and follows [semantic versioning](https://semver.org/) rules.
-- Wgit excels at crawling an entire website's HTML out of the box. Many alternative crawlers require you to provide the `xpath` needed to *follow* the next URLs to crawl. Wgit by default, crawls the entire site by extracting its internal links pointing to the same host.
+- Wgit excels at crawling an entire website's HTML out of the box. Many alternative crawlers require you to provide the `xpath` needed to *follow* the next URLs to crawl. Wgit by default, crawls the entire site by extracting its internal links pointing to the same host - no `xpath` needed.
 - Wgit allows you to define content *extractors* that will fire on every subsequent crawl; be it a single URL or an entire website. This enables you to focus on the content you want.
-- Wgit can index (crawl and store) HTML to a database making it a breeze to build custom search engines. You can also specify which page content gets searched, making the search more meaningful. For example, here's a script that will index the Wgit [wiki](https://github.com/michaeltelford/wgit/wiki) articles:
+- Wgit can index (crawl and save) HTML to a database making it a breeze to build custom search engines. You can also specify which page content gets searched, making the search more meaningful. For example, here's a script that will index the Wgit [wiki](https://github.com/michaeltelford/wgit/wiki) articles:
 
 ```ruby
 require 'wgit'
@@ -164,6 +164,9 @@ indexer = Wgit::Indexer.new
 indexer.index_site(wiki, **opts)
 ```
 
+- Wgit supports different databases through the use of "adapter" classes, which you can write to support your own database of choice.
+- Wgit's built in indexing methods will by default, honour a site's `robots.txt` rules. There's also a handy `robots.txt` parser that you can use in your own code.
+
 ## Why Not Wgit?
 
 So why might you not use Wgit, I hear you ask?
@@ -176,9 +179,9 @@ So why might you not use Wgit, I hear you ask?
 
 Only MRI Ruby is tested and supported, but Wgit may work with other Ruby implementations.
 
-Currently, the required MRI Ruby version is:
+Currently, the supported range of MRI Ruby versions is:
 
-`ruby '>= 2.6', '< 4'`
+`ruby '~> 3.0'` a.k.a. between Ruby 3.0 and up to but not including Ruby 4.0. Wgit will probably work fine with older versions but best to upgrade if possible.
 
 ### Using Bundler
 
@@ -208,10 +211,11 @@ Installing the Wgit gem adds a `wgit` executable to your `$PATH`. The executable
 The `wgit` executable does the following things (in order):
 
 1. `require wgit`
-2. `eval`'s a `.wgit.rb` file (if one exists in either the local or home directory, which ever is found first)
-3. Starts an interactive shell (using `pry` if it's installed, or `irb` if not)
+2. Loads an `.env` file (if one exists in either the local or home directory, which ever is found first)
+3. `eval`'s a `.wgit.rb` file (if one exists in either the local or home directory, which ever is found first)
+4. Starts an interactive shell (using `pry` if it's installed, or `irb` if not)
 
-The `.wgit.rb` file can be used to seed fixture data or define helper functions for the session. For example, you could define a function which indexes your website for quick and easy searching everytime you start a new session.
+The `.wgit.rb` file can be used to seed fixture data or define helper functions for the session. For example, you could define a function which indexes your website for quick and easy searching everytime you start `wgit`.
 
 ## License
 
@@ -237,14 +241,14 @@ And you're good to go!
 
 ### Tooling
 
-Wgit uses the [`toys`](https://github.com/dazuma/toys) gem (instead of Rake) for task invocation. For a full list of available tasks a.k.a. tools, run `toys --tools`. You can search for a tool using `toys -s tool_name`. The most commonly used tools are listed below...
+Wgit uses the [toys](https://github.com/dazuma/toys) gem (instead of Rake) for task invocation. Always run `toys` as `bundle exec toys`. For a full list of available tasks a.k.a. tools, run `bundle exec toys --tools`. You can search for a tool using `bundle exec toys -s tool_name`. The most commonly used tools are listed below...
 
-Run `toys db` to see a list of database related tools, enabling you to run a Mongo DB instance locally using Docker. Run `toys test` to execute the tests.
+Run `bundle exec toys db` to see a list of database related tools, enabling you to run a Mongo DB instance locally using Docker. Run `bundle exec toys test` to execute the tests.
 
-To generate code documentation locally, run `toys yardoc`. To browse the docs in a browser run `toys yardoc --serve`. You can also use the `yri` command line tool e.g. `yri Wgit::Crawler#crawl_site` etc.
+To generate code documentation locally, run `bundle exec toys yardoc`. To browse the docs in a browser run `bundle exec toys yardoc --serve`. You can also use the `yri` command line tool e.g. `yri Wgit::Crawler#crawl_site` etc.
 
-To install this gem onto your local machine, run `toys install` and follow the prompt.
+To install this gem onto your local machine, run `bundle exec toys install` and follow the prompt.
 
 ### Console
 
-You can run `toys console` for an interactive shell using the `./bin/wgit` executable. The `toys setup` task will have created an `.env` and `.wgit.rb` file which get loaded by the executable. You can use the contents of this [gist](https://gist.github.com/michaeltelford/b90d5e062da383be503ca2c3a16e9164) to turn the executable into a development console. It defines some useful functions, fixtures and connects to the database etc. Don't forget to set the `WGIT_CONNECTION_STRING` in the `.env` file.
+You can run `bundle exec toys console` for an interactive shell using the `./bin/wgit` executable. The `bundle exec toys setup` task will have created an `.env` and `.wgit.rb` file which get loaded by the executable. You can use the contents of this [gist](https://gist.github.com/michaeltelford/b90d5e062da383be503ca2c3a16e9164) to turn the executable into a development console. It defines some useful functions, fixtures and connects to the database etc. Don't forget to set the `WGIT_CONNECTION_STRING` in the `.env` file.

data/bin/wgit

@@ -1,39 +1,95 @@
 #!/usr/bin/env ruby
 
-require 'wgit'
+require "wgit"
+
+# Shorten the filepath for nicer output to the user.
+def format_path(dir, path)
+  return path.gsub(dir, ".") if dir == Dir.pwd
+  return path.gsub(dir, "~") if dir == Dir.home
+
+  path
+end
+
+# Load .env file (if it exists somewhere).
+def load_env
+  begin
+    require "dotenv"
+  rescue LoadError
+    puts "Skipping .env load because 'dotenv' isn't installed"
+    puts
+    return false
+  end
+
+  puts "Searching for .env file in local and home directories..."
+  success = false
+
+  [Dir.pwd, Dir.home].each do |dir|
+    path = "#{dir}/.env"
+    next unless File.exist?(path)
+
+    puts "Loading #{format_path(dir, path)}"
+    puts "Call `load_env` after changes to re-load the environment variables"
+
+    Dotenv.load(path)
+    success = true
+
+    break
+  end
+
+  puts
+
+  success
+end
 
 # Eval .wgit.rb file (if it exists somewhere).
-def eval_wgit(filepath = nil)
-  puts 'Searching for .wgit.rb file in local and home directories...'
+def eval_wgit
+  puts "Searching for .wgit.rb file in local and home directories..."
+  success = false
 
-  [filepath, Dir.pwd, Dir.home].each do |dir|
+  [Dir.pwd, Dir.home].each do |dir|
     path = "#{dir}/.wgit.rb"
     next unless File.exist?(path)
 
-    puts "Eval'ing #{path}"
-    puts 'Call `eval_wgit` after changes to re-eval the file'
+    puts "Eval'ing #{format_path(dir, path)}"
+    puts "Call `eval_wgit` after changes to re-eval the file"
+
     eval(File.read(path))
+    success = true
 
     break
   end
 
-  nil
-end
+  puts
 
-eval_wgit
-puts "\n#{Wgit.version_str}\n\n"
+  success
+end
 
+# Choose and return which REPL class to use.
 # Use Pry if installed or fall back to IRB.
-begin
-  require 'pry'
-  klass = Pry
-rescue LoadError
-  require 'irb'
-  klass = IRB
-
-  puts "Starting IRB because Pry isn't installed."
+def repl_class
+  begin
+    require "pry"
+    klass = Pry
+  rescue LoadError
+    require "irb"
+    klass = IRB
+
+    puts "Using 'irb' REPL because 'pry' isn't installed"
+    puts
+  end
+
+  klass
 end
 
+### START OF EXECUTABLE ###
+
+load_env
+eval_wgit
+klass = repl_class
+
+puts Wgit.version_str
+puts "#{'-' * Wgit.version_str.size}\n\n"
+
 klass.start
 
-puts 'Interactive session complete.'
+puts "Interactive wgit session complete"

data/lib/wgit/assertable.rb

@@ -4,11 +4,18 @@ module Wgit
   # Module containing assertion methods including type checking and duck typing.
   module Assertable
     # Default type fail message.
-    DEFAULT_TYPE_FAIL_MSG = 'Expected: %s, Actual: %s'
+    DEFAULT_TYPE_FAIL_MSG = "Expected: %s, Actual: %s"
+
     # Wrong method message.
-    NON_ENUMERABLE_MSG = 'Expected an Enumerable responding to #each, not: %s'
+    NON_ENUMERABLE_MSG = "Expected an Enumerable responding to #each, not: %s"
+
+    # Enumerable with more than one type across it's elements.
+    MIXED_ENUMERABLE_MSG = "Expected an Enumerable with elements of a single \
+common type"
+
     # Default duck fail message.
     DEFAULT_DUCK_FAIL_MSG = "%s doesn't respond_to? %s"
+
     # Default required keys message.
     DEFAULT_REQUIRED_KEYS_MSG = "Some or all of the required keys are not \
 present: %s"
@@ -47,6 +54,25 @@ present: %s"
       arr.each { |obj| assert_types(obj, type_or_types, msg) }
     end
 
+    # All objects within arr must match one of the types listed in
+    # type_or_types; or an exception is raised using msg, if provided.
+    # Ancestors of the same type are allowed and considered common.
+    #
+    # @param arr [Enumerable#each] Enumerable of objects to type check.
+    # @param type_or_types [Type, Array<Type>] The allowed type(s).
+    # @param msg [String] The raised StandardError message, if provided.
+    # @raise [StandardError] If the assertion fails.
+    # @return [Object] The given arr on successful assertion.
+    def assert_common_arr_types(arr, type_or_types, msg = nil)
+      raise format(NON_ENUMERABLE_MSG, arr.class) unless arr.respond_to?(:each)
+
+      type = arr.first.class
+      type_match = arr.all? { |obj| type.ancestors.include?(obj.class) }
+      raise MIXED_ENUMERABLE_MSG unless type_match
+
+      assert_arr_types(arr, type_or_types, msg)
+    end
+
     # The obj_or_objs must respond_to? all of the given methods or an
     # Exception is raised using msg, if provided.
     #
@@ -75,7 +101,7 @@ present: %s"
     # @raise [KeyError] If the assertion fails.
     # @return [Hash] The given hash on successful assertion.
     def assert_required_keys(hash, keys, msg = nil)
-      msg ||= format(DEFAULT_REQUIRED_KEYS_MSG, keys.join(', '))
+      msg ||= format(DEFAULT_REQUIRED_KEYS_MSG, keys.join(", "))
       all_present = keys.all? { |key| hash.keys.include? key }
       raise KeyError, msg unless all_present
 
@@ -86,7 +112,7 @@ present: %s"
 
     # obj must respond_to? all methods or an exception is raised.
     def _assert_respond_to(obj, methods, msg = nil)
-      raise 'methods must respond_to? :all?' unless methods.respond_to?(:all?)
+      raise "methods must respond_to? :all?" unless methods.respond_to?(:all?)
 
       msg ||= format(DEFAULT_DUCK_FAIL_MSG, "#{obj.class} (#{obj})", methods)
       match = methods.all? { |method| obj.respond_to?(method) }
@@ -95,7 +121,8 @@ present: %s"
       obj
     end
 
-    alias assert_type     assert_types
-    alias assert_arr_type assert_arr_types
+    alias_method :assert_type,            :assert_types
+    alias_method :assert_arr_type,        :assert_arr_types
+    alias_method :assert_common_arr_type, :assert_common_arr_types
   end
 end

data/lib/wgit/core_ext.rb

@@ -3,7 +3,7 @@
 # Script which extends Ruby's core functionality when parsed.
 # Needs to be required separately to 'wgit' using `require 'wgit/core_ext'`.
 
-require_relative 'url'
+require_relative "url"
 
 # Extend the standard String functionality.
 class String