html-proofer 2.3.0 → 3.0.0

data/README.md

@@ -1,265 +0,0 @@
-# HTML::Proofer
-
-If you generate HTML files, _then this tool might be for you_.
-
-`HTML::Proofer` is a set of tests to validate your HTML output. These tests check if your image references are legitimate, if they have alt tags, if your internal links are working, and so on. It's intended to be an all-in-one checker for your output.
-
-[![Build Status](https://travis-ci.org/gjtorikian/html-proofer.svg?branch=master)](https://travis-ci.org/gjtorikian/html-proofer) [![Gem Version](https://badge.fury.io/rb/html-proofer.svg)](http://badge.fury.io/rb/html-proofer)
-
-## Installation
-
-Add this line to your application's Gemfile:
-
-    gem 'html-proofer'
-
-And then execute:
-
-    $ bundle
-
-Or install it yourself as:
-
-    $ gem install html-proofer
-
-**NOTE:** When installation speed matters, set `NOKOGIRI_USE_SYSTEM_LIBRARIES` to `true` in your environment. This is useful for increasing the speed of your Continuous Integration builds.
-
-### Real-life examples
-
-Project | Repository
-:--- | :---
-[Raspberry Pi documentation](http://www.raspberrypi.org/documentation/) | [raspberrypi/documentation]( https://github.com/raspberrypi/documentation)
-[Open Whisper Systems website](https://whispersystems.org/) | [WhisperSystems/whispersystems.org](https://github.com/WhisperSystems/whispersystems.org)
-[Jekyll website](http://jekyllrb.com/) | [jekyll/jekyll](https://github.com/jekyll/jekyll)
-
-## What's Tested?
-
-### Images
-
-`img` elements:
-
-* Whether all your images have alt tags
-* Whether your internal image references are not broken
-* Whether external images are showing
-
-### Links
-
-`a`, `link` elements:
-
-* Whether your internal links are not broken; this includes hash references (`#linkToMe`)
-* Whether external links are working
-
-### Scripts
-
-`script` elements:
-
-* Whether your internal script references are not broken
-* Whether external scripts are loading
-
-### Favicon
-
-Checks if your favicons are valid. This is an optional feature, set the `check_favicon` option to turn it on.
-
-### HTML
-
-Nokogiri looks at the markup and [provides errors](http://www.nokogiri.org/tutorials/ensuring_well_formed_markup.html) when parsing your document.
-This is an optional feature, set the `check_html` option to enable validation errors from Nokogiri.
-
-## Usage
-
-### Using in a script
-
-Require the gem; generate some HTML; create a new instance of the `HTML::Proofer` on
-your output folder; then `run` it. Here's a simple example:
-
-```ruby
-require 'html/proofer'
-require 'html/pipeline'
-require 'find'
-
-# make an out dir
-Dir.mkdir("out") unless File.exist?("out")
-
-pipeline = HTML::Pipeline.new [
-  HTML::Pipeline::MarkdownFilter,
-  HTML::Pipeline::TableOfContentsFilter
-], :gfm => true
-
-# iterate over files, and generate HTML from Markdown
-Find.find("./docs") do |path|
-  if File.extname(path) == ".md"
-    contents = File.read(path)
-    result = pipeline.call(contents)
-
-    File.open("out/#{path.split("/").pop.sub('.md', '.html')}", 'w') { |file| file.write(result[:output].to_s) }
-  end
-end
-
-# test your out dir!
-HTML::Proofer.new("./out").run
-```
-
-### Using on the command-line
-
-You'll get a new program called `htmlproof` with this gem. Terrific!
-
-Use it like you'd expect to:
-
-``` bash
-htmlproof ./out --href-swap wow:cow,mow:doh --ext .html.erb --href-ignore www.github.com
-```
-
-Note: since `href_swap` is a bit special, you'll pass in a pair of `RegEx:String` values.
-`htmlproof` will figure out what you mean.
-
-### Using with Jekyll
-
-Want to use HTML Proofer with your Jekyll site? Awesome. Simply add `gem 'html-proofer'`
-to your `Gemfile` as described above, and add the following to your `Rakefile`,
-using `rake test` to execute:
-
-```ruby
-require 'html/proofer'
-
-task :test do
-  sh "bundle exec jekyll build"
-  HTML::Proofer.new("./_site").run
-end
-```
-
-Don't have or want a `Rakefile`? You _could_ also do something like the following:
-
-```bash
-htmlproof ./_site
-```
-
-## Configuration
-
-The `HTML::Proofer` constructor takes an optional hash of additional options:
-
-| Option | Description | Default |
-| :----- | :---------- | :------ |
-| `alt_ignore` | An array of Strings or RegExps containing `img`s whose missing `alt` tags are safe to ignore. | `[]` |
-| `empty_alt_ignore` | If `true`, ignores images with empty alt tags. | `false` |
-| `check_external_hash` | Checks whether external hashes exist (even if the website exists). This slows the checker down. | `false` |
-|`checks_to_ignore`| An array of Strings indicating which checks you'd like to not perform. | `[]`
-| `directory_index_file` | Sets the file to look for when a link refers to a directory. | `index.html` |
-| `disable_external` | If `true`, does not run the external link checker, which can take a lot of time. | `false` |
-| `error_sort` | Defines the sort order for error output. Can be `:path`, `:desc`, or `:status`. | `:path`
-| `ext` | The extension of your HTML files including the dot. | `.html`
-| `file_ignore` | An array of Strings or RegExps containing file paths that are safe to ignore. | `[]` |
-| `href_ignore` | An array of Strings or RegExps containing `href`s that are safe to ignore. Note that non-HTTP(S) URIs are always ignored. | `[]` |
-| `href_swap` | A hash containing key-value pairs of `RegExp => String`. It transforms links that match `RegExp` into `String` via `gsub`. | `{}` |
-| `only_4xx` | Only reports errors for links that fall within the 4xx status code range. | `false` |
-| `url_ignore` | An array of Strings or RegExps containing URLs that are safe to ignore. It affects all HTML attributes. Note that non-HTTP(S) URIs are always ignored. | `[]` |
-| `check_favicon` | Enables the favicon checker. | `false` |
-| `check_html` | Enables HTML validation errors from Nokogiri | `false` |
-| `verbose` | If `true`, outputs extra information as the checking happens. Useful for debugging. | `false` |
-
-### Configuring Typhoeus and Hydra
-
-[Typhoeus](https://github.com/typhoeus/typhoeus) is used to make fast, parallel requests to external URLs. You can pass in any of Typhoeus' options for the external link checks with the options namespace of `:typhoeus`. For example:
-
-``` ruby
-HTML::Proofer.new("out/", {:ext => ".htm", :typhoeus => { :verbose => true, :ssl_verifyhost => 2 } })
-```
-
-This sets `HTML::Proofer`'s extensions to use _.htm_, and gives Typhoeus a configuration for it to be verbose, and use specific SSL settings. Check [the Typhoeus documentation](https://github.com/typhoeus/typhoeus#other-curl-options) for more information on what options it can receive.
-
-You can similarly pass in a `:hydra` option with a hash configuration for Hydra.
-
-The default value is `typhoeus => { :followlocation => true }`.
-
-### Configuring Parallel
-
-[Parallel](https://github.com/grosser/parallel) is being used to speed internal file checks. You can pass in any of its options with the options "namespace" `:parallel`. For example:
-
-``` ruby
-HTML::Proofer.new("out/", {:ext => ".htm", :parallel => { :in_processes => 3} })
-```
-
-`:in_processes => 3` will be passed into Parallel as a configuration option.
-
-### Array of links
-
-Instead of a directory as the first argument, you can also pass in an array of links:
-
-``` ruby
-HTML::Proofer.new(["http://github.com", "http://jekyllrb.com"])
-```
-
-This configures Proofer to just test those links to ensure they are valid. Note that for the command-line, you'll need to pass a special `--as-links` argument:
-
-``` bash
-bin/htmlproof www.google.com,www.github.com --as-links
-```
-
-## Ignoring content
-
-Add the `data-proofer-ignore` attribute to any tag to ignore it from the checks.
-
-
-``` html
-<a href="http://notareallink" data-proofer-ignore>Not checked.</a>
-```
-
-## Custom tests
-
-Want to write your own test? Sure! Just create two classes--one that inherits from `HTML::Proofer::CheckRunner`, and another that inherits from `HTML::Proofer::Checkable`.
-
-The `CheckRunner` subclass must define one method called `run`. This is called on your content, and is responsible for performing the validation on whatever elements you like. When you catch a broken issue, call `add_issue(message)` to explain the error.
-
-The `Checkable` subclass defines various helper methods you can use as part of your test. Usually, you'll want to instantiate it within `run`. You have access to all of your element's attributes.
-
-Here's an example custom test that protects against `mailto` links that point to `octocat@github.com`:
-
-``` ruby
-class OctocatLinkCheck < ::HTML::Proofer::Checkable
-
-  def mailto?
-    return false if @data_ignore_proofer || @href.nil? || @href.empty?
-    return @href.match /^mailto\:/
-  end
-
-  def octocat?
-    return @href.match /\:octocat@github.com\Z/
-  end
-
-end
-
-class MailToOctocat < ::HTML::Proofer::CheckRunner
-
-  def run
-    @html.css('a').each do |l|
-      link = OctocatLinkCheck.new l, self
-
-      if link.mailto? && link.octocat?
-        return add_issue("Don't email the Octocat directly!")
-      end
-    end
-  end
-end
-```
-
-## Troubleshooting
-
-### Certificates
-
-To ignore certificates turn off the Typhoeus SSL verification:
-
-``` ruby
-HTML::Proofer.new("out/", {
-  :typhoeus => {
-    :ssl_verifypeer => false,
-    :ssl_verifyhost => 0}
-}).run
-```
-
-### User-Agent
-
-To change the User-Agent used by Typhoeus:
-
-``` ruby
-HTML::Proofer.new("out/", {
-  :typhoeus => {
-    :headers => { "User-Agent" => "Mozilla/5.0 (compatible; My New User-Agent)" }
-}}).run
-```

data/Rakefile

@@ -1,21 +0,0 @@
-require 'bundler'
-Bundler::GemHelper.install_tasks
-
-require 'rspec/core/rake_task'
-
-RSpec::Core::RakeTask.new(:spec)
-
-task :default => [:spec, :proof_readme]
-
-task :proof_readme do
-  require 'html/proofer'
-  require 'redcarpet'
-
-  redcarpet = Redcarpet::Markdown.new Redcarpet::Render::HTML.new({}), {}
-  html = redcarpet.render File.read('README.md')
-
-  mkdir_p 'out'
-  File.write('out/README.html', html)
-
-  HTML::Proofer.new('./out').run
-end

data/bin/htmlproof

@@ -1,71 +0,0 @@
-#!/usr/bin/env ruby
-STDOUT.sync = true
-
-$LOAD_PATH.unshift File.join(File.dirname(__FILE__), *%w( .. lib ))
-
-require 'html/proofer'
-require 'mercenary'
-require 'rubygems'
-
-def to_regex?(item)
-  if item.start_with? '/' and item.end_with? '/'
-    Regexp.new item[1...-1]
-  else
-    item
-  end
-end
-
-Mercenary.program(:htmlproof) do |p|
-  p.version Gem::Specification.load(File.join(File.dirname(__FILE__), '..', 'html-proofer.gemspec')).version
-  p.description %(Test your rendered HTML files to make sure they're accurate.)
-  p.syntax 'htmlproof PATH [options]'
-
-  p.description 'Runs the HTML-Proofer suite on the files in PATH. For more details, see the README.'
-
-  p.option 'as_links', '--as-links', 'Assumes that `PATH` is a comma-separated array of links to check.'
-  p.option 'alt_ignore', '--alt-ignore image1,[image2,...]', Array, 'Comma-separated list of Strings or RegExps containing `img`s whose missing `alt` tags are safe to ignore'
-  p.option 'empty_alt_ignore', '--empty-alt-ignore', 'Ignores images with empty alt tags.'
-  p.option 'checks_to_ignore', '--checks-to-ignore check1,[check2,...]', Array, ' An array of Strings indicating which checks you\'d like to not perform.'
-  p.option 'check_external_hash', '--check-external-hash', 'Checks whether external hashes exist (even if the website exists). This slows the checker down (default: `false`).'
-  p.option 'directory_index_file', '--directory-index-file', String, 'Sets the file to look for when a link refers to a directory. (default: `index.html`)'
-  p.option 'disable_external', '--disable-external', 'Disables the external link checker (default: `false`)'
-  p.option 'error_sort', '--error-sort SORT', 'Defines the sort order for error output. Can be `path`, `desc`, or `status` (default: `path`).'
-  p.option 'ext', '--ext EXT', String, 'The extension of your HTML files (default: `.html`)'
-  p.option 'file_ignore', '--file-ignore file1,[file2,...]', Array, 'Comma-separated list of Strings or RegExps containing file paths that are safe to ignore'
-  p.option 'href_ignore', '--href-ignore link1,[link2,...]', Array, 'Comma-separated list of Strings or RegExps containing `href`s that are safe to ignore.'
-  p.option 'href_swap', '--href-swap re:string,[re:string,...]', Array, 'Comma-separated list of key-value pairs of `RegExp:String`. Transforms links matching `RegExp` into `String`'
-  p.option 'only_4xx', '--only-4xx', 'Only reports errors for links that fall within the 4x status code range.'
-  p.option 'check_favicon', '--check-favicon', 'Enables the favicon checker (default: `false`).'
-  p.option 'check_html', '--check-html', 'Enables HTML validation errors from Nokogiri (default: `false`).'
-  p.option 'verbose', '--verbose', 'Enables more verbose logging.'
-
-  p.action do |args, opts|
-    args = ['.'] if args.empty?
-    path = args.first
-
-    options = {}
-
-    # prepare every to go to proofer
-    p.options.select { |o| !opts[o.config_key].nil? }.each do |option|
-      if option.return_type.to_s == 'Array' # TODO: is_a? doesn't work here?
-        opts[option.config_key] = opts[option.config_key].map { |i| to_regex?(i) }
-      end
-      options[option.config_key.to_sym] = opts[option.config_key]
-    end
-
-    # some minor manipulation of a special option
-    unless opts['href_swap'].nil?
-      options[:href_swap] = {}
-      opts['href_swap'].each do |s|
-        pair = s.split(':', 2)
-        options[:href_swap][Regexp.new(pair[0])] = pair[1]
-      end
-    end
-
-    options[:error_sort] = opts['error-sort'].to_sym unless opts['error-sort'].nil?
-
-    path = path.delete(' ').split(',') if opts['as_links']
-
-    HTML::Proofer.new(path, options).run
-  end
-end

data/html-proofer.gemspec

@@ -1,31 +0,0 @@
-# -*- encoding: utf-8 -*-
-$LOAD_PATH.push File.expand_path('../lib', __FILE__)
-require 'html/proofer/version'
-
-Gem::Specification.new do |gem|
-  gem.name          = 'html-proofer'
-  gem.version       = HTML::Proofer::VERSION
-  gem.authors       = ['Garen Torikian']
-  gem.email         = ['gjtorikian@gmail.com']
-  gem.description   = %(Test your rendered HTML files to make sure they're accurate.)
-  gem.summary       = %(A set of tests to validate your HTML output. These tests check if your image references are legitimate, if they have alt tags, if your internal links are working, and so on. It's intended to be an all-in-one checker for your documentation output.)
-  gem.homepage      = 'https://github.com/gjtorikian/html-proofer'
-  gem.license       = 'MIT'
-  gem.executables   = ['htmlproof']
-  gem.files         = `git ls-files -z`.split("\x0")
-  gem.test_files    = gem.files.grep(%r{^(spec)/})
-  gem.require_paths = ['lib']
-
-  gem.add_dependency 'mercenary',       '~> 0.3.2'
-  gem.add_dependency 'nokogiri',        '~> 1.5'
-  gem.add_dependency 'colored',         '~> 1.2'
-  gem.add_dependency 'typhoeus',        '~> 0.7'
-  gem.add_dependency 'yell',            '~> 2.0'
-  gem.add_dependency 'parallel',        '~> 1.3'
-  gem.add_dependency 'addressable',     '~> 2.3'
-
-  gem.add_development_dependency 'redcarpet'
-  gem.add_development_dependency 'rspec', '~> 3.1'
-  gem.add_development_dependency 'rake'
-  gem.add_development_dependency 'awesome_print'
-end

data/lib/html/proofer/check_runner.rb

@@ -1,62 +0,0 @@
-# encoding: utf-8
-
-module HTML
-  class Proofer
-    # Mostly handles issue management and collecting of external URLs.
-    class CheckRunner
-
-      attr_reader :issues, :src, :path, :options, :typhoeus_opts, :hydra_opts, :parallel_opts, \
-                  :external_urls, :href_ignores, :url_ignores, :alt_ignores, :empty_alt_ignore
-
-      def initialize(src, path, html, options, typhoeus_opts, hydra_opts, parallel_opts)
-        @src    = src
-        @path   = path
-        @html   = remove_ignored(html)
-        @options = options
-        @typhoeus_opts = typhoeus_opts
-        @hydra_opts = hydra_opts
-        @parallel_opts = parallel_opts
-        @issues = []
-        @href_ignores = @options[:href_ignore]
-        @url_ignores = @options[:url_ignore]
-        @alt_ignores = @options[:alt_ignore]
-        @empty_alt_ignore = @options[:empty_alt_ignore]
-        @external_urls = {}
-      end
-
-      def run
-        fail NotImplementedError, 'HTML::Proofer::CheckRunner subclasses must implement #run'
-      end
-
-      def add_issue(desc, line_number = nil, status = -1)
-        @issues << Issue.new(@path, desc, line_number, status)
-      end
-
-      def add_to_external_urls(href)
-        if @external_urls[href]
-          @external_urls[href] << @path
-        else
-          @external_urls[href] = [@path]
-        end
-      end
-
-      def self.checks
-        classes = []
-
-        ObjectSpace.each_object(Class) do |c|
-          next unless c.superclass == self
-          classes << c
-        end
-
-        classes
-      end
-
-    private
-
-      def remove_ignored(html)
-        html.css('code, pre').each(&:unlink)
-        html
-      end
-    end
-  end
-end

data/lib/html/proofer/checkable.rb

@@ -1,164 +0,0 @@
-require 'addressable/uri'
-require_relative './utils'
-
-module HTML
-  class Proofer
-    # Represents the superclass from which all checks derive.
-    class Checkable
-      include HTML::Utils
-      attr_reader :line
-
-      def initialize(obj, check)
-        obj.attributes.each_pair do |attribute, value|
-          instance_variable_set("@#{attribute.tr('-:', '_')}".to_sym, value.value)
-        end
-
-        @text = obj.content
-        @check = check
-        @checked_paths = {}
-        @type = self.class.name
-        @line = obj.line
-
-        if @href && @check.options[:href_swap]
-          @href = swap(@href, @check.options[:href_swap])
-        end
-
-        # fix up missing protocols
-        @href.insert 0, 'http:' if @href =~ %r{^//}
-        @src.insert 0, 'http:' if @src =~ %r{^//}
-      end
-
-      def url
-        @src || @srcset || @href || ''
-      end
-
-      def valid?
-        !parts.nil?
-      end
-
-      def parts
-        @parts ||= Addressable::URI.parse url
-      rescue URI::Error
-        @parts = nil
-      end
-
-      def path
-        Addressable::URI.unencode parts.path unless parts.nil?
-      end
-
-      def hash
-        parts.fragment unless parts.nil?
-      end
-
-      def scheme
-        parts.scheme unless parts.nil?
-      end
-
-      # path is to an external server
-      def remote?
-        %w( http https ).include? scheme
-      end
-
-      def non_http_remote?
-        !scheme.nil? && !remote?
-      end
-
-      def ignore?
-        return true if @data_proofer_ignore
-
-        # ignore base64 encoded images
-        if %w(ImageCheckable FaviconCheckable).include? @type
-          return true if url.match(/^data:image/)
-        end
-
-        # ignore user defined URLs
-        return true if ignores_pattern_check(@check.url_ignores)
-
-        # ignore user defined hrefs
-        if 'LinkCheckable' === @type
-          return true if ignores_pattern_check(@check.href_ignores)
-        end
-
-        # ignore user defined alts
-        if 'ImageCheckable' === @type
-          return true if ignores_pattern_check(@check.alt_ignores)
-        end
-      end
-
-      def ignore_empty_alt?
-        @check.empty_alt_ignore
-      end
-
-      # path is external to the file
-      def external?
-        !internal?
-      end
-
-      # path is an anchor or a query
-      def internal?
-        url.start_with? '#', '?'
-      end
-
-      def file_path
-        return if path.nil?
-
-        if path =~ /^\// # path relative to root
-          base = File.directory?(@check.src) ? @check.src : File.dirname(@check.src)
-        elsif File.exist?(File.expand_path path, @check.src) # relative links, path is a file
-          base = File.dirname @check.path
-        elsif File.exist?(File.join(File.dirname(@check.path), path)) # relative links in nested dir, path is a file
-          base = File.dirname @check.path
-        else # relative link, path is a directory
-          base = @check.path
-        end
-
-        file = File.join base, path
-
-        # implicit index support
-        if File.directory?(file) && !unslashed_directory?(file)
-          file = File.join file, @check.options[:directory_index_file]
-        end
-
-        file
-      end
-
-      # checks if a file exists relative to the current pwd
-      def exists?
-        return @checked_paths[absolute_path] if @checked_paths.key? absolute_path
-        @checked_paths[absolute_path] = File.exist? absolute_path
-      end
-
-      def absolute_path
-        path = file_path || @check.path
-        File.expand_path path, Dir.pwd
-      end
-
-      def ignores_pattern_check(links)
-        links.each do |ignore|
-          if ignore.is_a? String
-            return true if ignore == url
-          elsif ignore.is_a? Regexp
-            return true if ignore =~ url
-          end
-        end
-
-        false
-      end
-
-      def unslashed_directory?(file)
-        File.directory?(file) && !file.end_with?(File::SEPARATOR) && !follow_location?
-      end
-
-      def follow_location?
-        @check.typhoeus_opts && @check.typhoeus_opts[:followlocation]
-      end
-
-      private
-
-      def real_attr(attr)
-        attr.to_s unless attr.nil? || attr.empty?
-      end
-
-    end
-  end
-end

data/lib/html/proofer/checks/favicon.rb

@@ -1,21 +0,0 @@
-# encoding: utf-8
-
-class FaviconCheckable < ::HTML::Proofer::Checkable
-  def rel
-    @rel
-  end
-end
-
-class FaviconCheck < ::HTML::Proofer::CheckRunner
-
-  def run
-    @html.xpath('//link[not(ancestor::pre or ancestor::code)]').each do |favicon|
-      favicon = FaviconCheckable.new favicon, self
-      next if favicon.ignore?
-      return if favicon.rel.split(' ').last.eql? 'icon'
-    end
-
-    add_issue 'no favicon specified'
-  end
-
-end

data/lib/html/proofer/checks/html.rb

@@ -1,40 +0,0 @@
-# encoding: utf-8
-
-class HtmlCheck < ::HTML::Proofer::CheckRunner
-
-  # new html5 tags (source: http://www.w3schools.com/html/html5_new_elements.asp)
-  # and svg child tags (source: https://developer.mozilla.org/en-US/docs/Web/SVG/Element)
-  HTML5_TAGS = %w(article aside bdi details dialog figcaption
-                  figure footer header main mark menuitem meter
-                  nav progress rp rt ruby section summary
-                  time wbr datalist keygen output color date
-                  datetime datetime-local email month number
-                  range search tel time url week canvas
-                  svg audio embed source track video
-                  altGlyph altGlyphDef altGlyphItem animate
-                  animateColor animateMotion animateTransform
-                  circle clipPath color-profile cursor defs
-                  desc ellipse feBlend feColorMatrix
-                  feComponentTransfer feComposite feConvolveMatrix
-                  feDiffuseLighting feDisplacementMap feDistantLight
-                  feFlood feFuncA feFuncB feFuncG feFuncR feGaussianBlur
-                  feImage feMerge feMergeNode feMorphology feOffset
-                  fePointLight feSpecularLighting feSpotLight feTile
-                  feTurbulence filter font font-face font-face-format
-                  font-face-name font-face-src font-face-uri
-                  foreignObject g glyph glyphRef hkern image line
-                  linearGradient marker mask metadata missing-glyph
-                  mpath path pattern polygon polyline radialGradient
-                  rect set stop switch symbol text textPath tref tspan use
-                  view vkern)
-
-  def run
-    @html.errors.each do |e|
-      # Nokogiri (or rather libxml2 underhood) only recognizes html4 tags,
-      # so we need to skip errors caused by the new tags in html5
-      next if HTML5_TAGS.include? e.to_s[/Tag ([\w-]+) invalid/o, 1]
-
-      add_issue(e.to_s)
-    end
-  end
-end