middleman-spellcheck 0.9.0 → 0.9.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: fea98d00ab58220965c1e2b8118063d0fbf50b13
-  data.tar.gz: e35c46312d5c3a4afafea88635142e982d920368
+  metadata.gz: d50552923d8b8b4e3f9f0bcdc9a6f17f5229ecf9
+  data.tar.gz: fdd05ae0af2f6c4bcd0ca9674c28e226456c510c
 SHA512:
-  metadata.gz: ae6225286fc01dbcfd6abe17f91b3b3b99f0250a1ed3ea2bbc8d2eb9b499af404826262b57e14146931318904b21dc65b51a3836cad6c7f3405b12e68c1cbfe6
-  data.tar.gz: 694260deefae946b9b7c8d76aeb3eb17ae7ffd4edbda7fb81c361113daac96dcacd32ac66917dd291b41cd30b4e461a19d682351f143bf60b49bad4d790a3c55
+  metadata.gz: 14cb5229c5da423142e23b3efe3cd0d5d7f3e2e5b0c099211e20ad3e677a3031d778cb8cf3e83a0c225c32722ec6b2f3d758c6c9911e5c70fe7b9d4480f065d1
+  data.tar.gz: e04e1cf5302675eaac3d9d06161fb0b0a2307ff24a3c10d93a8e7b95f655a954f01f021e1009220e4fdd9e7a9f7dc81748a89ecf8b9fdc2e59f2378cc5d9ab1e

data/README.md

@@ -25,31 +25,50 @@ middleman spellcheck source/blog/
 
 ## Usage
 
-You can spellcheck only some resources using a regex with the URL:
+There are several ways to select what content will be checked.
 
-```ruby
-activate :spellcheck, page: "documentation/*" # you can use regexes, too, e.g. /post_[1-9]/
-```
+1. To spellcheck only some resources using a regex with the URL:
 
-You can limit which tags the spell checker will only run through:
+    ```ruby
+    activate :spellcheck, page: "documentation/*" # you can use regexes, too, e.g. /post_[1-9]/
+    ```
 
-```ruby
-activate :spellcheck, tags: :p  # pass an array of tags if you have more!
-```
+2. To limit which tags the spell checker will only run through:
 
-If there are some words that you would like to be allowed
+    ```ruby
+    activate :spellcheck, tags: :p  # pass an array of tags if you have more!
+    ```
 
-```ruby
-activate :spellcheck, allow: ["Gooby", "pls"]
-```
+3. To ignore sections by using css selectors
+    For example, to ignore all sections with a class of `CodeRay`:
+
+    ```ruby
+    activate :spellcheck, ignore_selector: '.CodeRay'
+    ```
+
+    Or to ignore all tables in a document:
+
+    ```ruby
+    activate :spellcheck, ignore_selector: 'table'
+    ```
 
-Middleman-spellcheck automatically ignores `.css`, `.js`, & `.coffee` file
+    Or to ignore all `<p class="technical-jargon">`:
+    ```ruby
+    activate :spellcheck, ignore_selector: 'p.technical-jargon'
+    ```
+
+    To ignore multiple selectors, seperate them with a comma
+    ```ruby
+    activate :spellcheck, ignore_selector: 'p.technical-jargon, .CodeRay'
+    ```
+
+4. Middleman-spellcheck automatically ignores `.css`, `.js`, & `.coffee` file
 extensions. If there are some additional file type extensions that you would
 like to skip:
 
-```ruby
-activate :spellcheck, ignored_exts: [".xml", ".png"]
-```
+    ```ruby
+    activate :spellcheck, ignored_exts: [".xml", ".png"]
+    ```
 
 To select a dictionary used by a spellchecker, use lang: option. For
 example, to use Polish dictionary, use:
@@ -60,12 +79,15 @@ activate :spellcheck, lang: "pl"
 
 If you define the ``lang`` metadata in your pages / articles, then spellcheck will use those language.
 
-Middleman-spellcheck can issue many warnings if you run it over a new
-content. If you want to give yourself a chance to fix mistakes gradually and
-not fail each time you build, use :dontfail flag:
+## Options
+
+
+For warnings only (allow build to pass), use `dontfail` option.
+This is helpful when you want to give yourself a chance to fix mistakes or false hits gradually and
+not fail each time you build.
 
 ```ruby
-activate :spellcheck, lang: "en", dontfail: 1
+activate :spellcheck, dontfail: 1
 ```
 
 You can also disable the automatic spellcheck after build (and only run manual checks from the command line):
@@ -89,9 +111,84 @@ who encountered issues, useful might be debug: option, which will turn on
 extensive amount of debugging.
 
 ```ruby
-activate :spellcheck, debug: 1
+activate :spellcheck, lang: "en", debug: 1
 ```
 
+If there are some words that you would like to be allowed you can pass them to the allow option as an array. **Depricated - Please see the "Fixing spelling mistakes" section for now prefered way to include allowed words
+
+```ruby
+activate :spellcheck, allow: ["Gooby", "pls"]
+```
+
+You can also pass a regex to the `ignore_regex` option. Any match will be ignored.
+For example to remove words in quotes
+```ruby
+activate :spellcheck, lang: "en", ignore_regex: /\s('|")\w*('|")(\s|\.|,)/
+```
+
+## Fixing spelling mistakes & false positives
+
+The `middleman-spellchecker` extension is likely to generate large number
+of false-positives, e.g.: words which the spellchecker will consider
+incorrect (not present in a dictionary), which yet may have a valid meaning
+in the article's context. Common problems are acronyms, technical terms and
+names. To solve this, `middleman-spellcheck` offers two solutions:
+
+1. The `spellcheck_allow_file` file, which points to the path with a file
+containing words considered correct. Author of the website may decide which
+words are allowed to be used site-wide. Example: if you write a lot about
+IBM products, this file would have names such as "IBM", "AIX" or "DB/2".
+Add the words one word per line without quotes.
+
+To set the global file, use the following clause in your `config.rb`:
+
+    ```set :spellcheck_allow_file, "./data/words_allowed.txt"```
+
+2. The `spellcheck-allow` keyword in a frontmatter, which will work in the
+context of this particular article, but not other articles. Example: your
+blog is about IBM, but 1 article is about AirBnB. You'd put `AirBnB` into
+your front-matter.
+
+
+To use 2nd solution, add the following to your frontmatter:
+
+    ```
+    title: "Blog about IBM"
+    ...
+    spellcheck-allow:
+    - "AirBnB"```
+    
+    Another example
+    
+    ```
+    title: "Some time ago"
+    ...
+    spellcheck-allowed:
+    - GitHub
+    - Linux
+    ```
+
+The `middleman-spellcheck` also comes with a simple CLI for fixing many
+problems in your articles. To invoke:
+
+    ```middleman spellcheck source/blog/2015-11-01-nginx-on-travis-ci.md --fix```
+
+This will pull up simple CLI menu and for each misspelled word, you'll have
+a following choice
+
+| Key to press | Effect |
+|--------------|--------|
+| g | Add the word to the `spellcheck_allow_file` |
+| f | Add the word to this article's front-matter |
+| i | Ignore the word for now and deal with it later |
+
+
+After the run is finished, `middleman-spellchecker` will write a fixed file
+to `source/blog/2015-11-01-nginx-on-travis-ci.md.fixed`. This is a safe
+choice for not creating damage. If you don't want to fiddle with it, the
+`--inplace` switch will make changes dynamically, and the input file will
+get overwritten.
+
 ## Contributing
 
 1. Fork it

data/lib/middleman-spellcheck/cli.rb

@@ -1,10 +1,75 @@
 module Middleman
   module Cli
-    # This class provides an "spellchecl" command for the middleman CLI.
+    # This class provides a "spellcheck" command for the middleman CLI.
     class Spellcheck < Thor
       include Thor::Actions
+
+      check_unknown_options!
+
+      namespace :spellcheck
+
+      no_commands {
+        def misspell_ask(misspell, allow_file)
+          if not options[:fix]
+            return false
+          end
+
+          print "-" * 70, "\n"
+          print "'#{misspell[:word]}' is misspelled. What to do?:\n"
+          print "g\tadd allowed word to '#{allow_file}'\n" if allow_file
+          print "f\tadd allowed word to frontmatter\n"
+          print "i\tignore word (deal with it later)\n"
+
+          print "Select option and press ENTER:\n"
+          STDIN.gets()[0]
+        end
+
+        def misspell_fixes_global(resource, words_allow_global, allow_file)
+          open(allow_file, 'a') { |f|
+            words_allow_global.each do |w|
+              f.puts "#{w}\t#\t#{resource.source_file}\n"
+            end
+          }
+        end
+
+        def misspell_fixes_frontmatter(resource, words_allow_frontmatter)
+          data = File.read(resource.source_file)
+          if options[:inplace] then fn_ext = "" else ".fixed" end
+          fn_fixed = "#{resource.source_file}#{fn_ext}"
+          fixed = File.open(fn_fixed, "w")
+          sep_tag_cnt = 0
+          had_spellcheck_tag = false
+          data.each_line do |line|
+            if line =~ /^\-\-\-$/ then
+              sep_tag_cnt += 1
+            end
+            if line =~ /spellcheck-allow:/ then
+              had_spellcheck_tag = true
+            end
+
+            if sep_tag_cnt == 2 then
+              if had_spellcheck_tag == false then
+                fixed.puts "spellcheck-allow:\n"
+              end
+              words_allow_frontmatter.each do |w|
+                fixed.puts "- \"#{w}\"\n"
+              end
+              fixed.puts "---\n"
+              sep_tag_cnt = nil
+            else
+              fixed.puts line
+            end
+          end
+          fixed.close()
+          print "Fixed spellchecked file written to #{fn_fixed}\n"
+        end
+      }
+
       namespace :spellcheck
       desc "spellcheck FILE", "Run spellcheck on given file or path"
+      method_option "fix", :type => :boolean, :aliases => "-f", :desc => "Fix spelling error interactively"
+      method_option "inplace", :type => :boolean, :aliases => "-i", :desc => "Modify files in place (dangerous)"
+
       def spellcheck(*paths)
         app = ::Middleman::Application.server.inst
 
@@ -14,18 +79,40 @@ module Middleman
           }
         }
         if resources.empty?
-          $stderr.puts "File / Directory #{path} not exist"
+          $stderr.puts "File / Directory #{paths} not exist"
           exit 1
         end
         ext = app.extensions[:spellcheck]
+
+        allow_file = app.config.defines_setting?(:spellcheck_allow_file) ?
+              app.config[:spellcheck_allow_file] : nil
+        if options[:fix] then
+          print "Spellchecker fix mode on! Will attempt to white-list some words\n"
+        end
         resources.each do |resource|
-          say_status :spellcheck, "Running spell checker for #{resource.url}", :blue
+          say_status :spellcheck, "Running spell checker for #{resource.source_file}", :blue
           current_misspelled = ext.spellcheck_resource(resource)
+
+          words_allow_frontmatter = []
+          words_allow_global = []
           current_misspelled.each do |misspell|
-            say_status :misspell, ext.error_message(misspell), :red
+            fix = misspell_ask(misspell, allow_file)
+            was_fixed = true
+            if    fix == 'g' then words_allow_global << misspell[:word]
+            elsif fix == 'f' then words_allow_frontmatter << misspell[:word]
+            else             was_fixed = false end
+
+            if was_fixed
+              say_status :spellcheck, "Fixed word #{misspell[:word]}"
+            else
+              say_status :misspell, ext.error_message(misspell), :red
+            end
           end
+          misspell_fixes_global(resource, words_allow_global, allow_file) unless words_allow_global.empty?
+          misspell_fixes_frontmatter(resource, words_allow_frontmatter) unless words_allow_frontmatter.empty?
         end
       end
+
     end
   end
 end

data/lib/middleman-spellcheck/extension.rb

@@ -6,14 +6,17 @@ module Middleman
   module Spellcheck
     class SpellcheckExtension < Extension
       REJECTED_EXTS = %w(.css .js .coffee)
+      ALLOWED_WORDS = []
       option :page, "/*", "Run only pages that match the regex through the spellchecker"
       option :tags, [], "Run spellcheck only on some tags from the output"
       option :allow, [], "Allow specific words to be misspelled"
       option :ignored_exts, [], "Ignore specific extensions (ex: '.xml')"
+      option :ignore_regex, false, "Ignore regex matches"
+      option :ignore_selector, false, "Ignore nodes with a css selector"
       option :lang, "en", "Language for spellchecking"
       option :cmdargs, "", "Pass alternative command line arguments"
       option :debug, 0, "Enable debugging (for developers only)"
-      option :dontfail, 0, "Don't fail when misspelled words are found"
+      option :dontfail, false, "Don't fail because misspelled words are found"
       option :run_after_build, true, "Run Spellcheck after build"
 
       def after_build(builder)
@@ -32,13 +35,14 @@ module Middleman
           total_misspelled += current_misspelled
         end
 
-        unless total_misspelled.empty?
-          estr = "Build failed. There are spelling errors."
+        builder.say_status :spellcheck, "Spellchecks done. #{total_misspelled.length} misspelling(s) found.", :blue
+
+        unless total_misspelled.empty? 
           if options.dontfail
-            print "== :dontfail set! Will issue warning only, but not fail.\n"
-            print estr, "\n"
+            builder.say_status :spellcheck, "dontfail is set! Builder will ignore misspellings.", :yellow
           else
-            raise Thor::Error, estr
+            desc = "Build failed. There are spelling errors."
+            raise Thor::Error, desc
           end
         end
       end
@@ -48,6 +52,10 @@ module Middleman
         doc = Nokogiri::HTML.fragment(rendered_resource)
         doc.search('code,style,script').each(&:remove)
 
+        if options.ignore_selector
+          doc.css(options.ignore_selector).each(&:remove)
+        end
+
         if options.tags.empty?
           doc.text
         else
@@ -55,6 +63,14 @@ module Middleman
         end
       end
 
+      def regex_filter_content(text)
+        if options.ignore_regex
+          text.to_s.gsub options.ignore_regex , ' '
+        else
+          text
+        end
+      end
+
       def option_tags
         if options.tags.is_a? Array
           options.tags
@@ -85,26 +101,58 @@ module Middleman
           else
             options.lang
           end
-        run_check(select_content(resource), lang)
+        run_check(resource, lang)
       end
 
-      def run_check(text, lang)
+      def run_check(resource, lang)
+        text = select_content(resource)
+        text = regex_filter_content(text)
         results = Spellchecker.check(text, lang)
-        results = exclude_allowed(results)
+        results = exclude_allowed(resource, results)
         results.reject { |entry| entry[:correct] }
       end
 
-      def exclude_allowed(results)
-        results.reject { |entry| option_allowed.include? entry[:word].downcase }
+      def exclude_allowed(resource, results)
+        results.reject { |entry| allowed_words(resource).include? entry[:word].downcase }
       end
 
-      def option_allowed
-        allowed = if options.allow.is_a? Array
+      def allowed_words(resource)
+        words_ok = if options.allow.is_a? Array
                     options.allow
                   else
                     [options.allow]
                   end
-        allowed.map(&:downcase)
+        words_ok += allowed_words_frontmatter(resource)
+        words_ok += allowed_words_file
+        words_ok.map(&:downcase)
+      end
+
+      def allowed_words_frontmatter(resource)
+        words_ok = []
+        if resource.data.include?("spellcheck-allow") then
+          allowed_tmp = resource.data["spellcheck-allow"]
+          words_ok += if allowed_tmp.is_a? Array
+                       allowed_tmp
+                     else
+                       [allowed_tmp]
+                     end
+        end
+        words_ok
+      end
+
+      def allowed_words_file
+        allow_file = nil
+        if app.config.defines_setting?(:spellcheck_allow_file)
+          allow_file = app.config[:spellcheck_allow_file]
+        end
+        if ALLOWED_WORDS.empty? && allow_file != nil
+          lines = File.read(allow_file)
+          lines.split("\n").each do |line|
+            next if line =~ /^#/ or line =~ /^$/
+            ALLOWED_WORDS << line.partition('#').first.strip
+          end
+        end
+        ALLOWED_WORDS
       end
 
       def option_ignored_exts

data/lib/middleman-spellcheck/spellchecker.rb

@@ -1,3 +1,5 @@
+# encoding: utf-8
+
 class Spellchecker
   @@aspell_path = "aspell"
   @@aspell_cmdargs = ""
@@ -76,7 +78,8 @@ class Spellchecker
     text.gsub! '’', '\''
     sdbg "self.check got raw text:\n#{text}\n"
 
-    words = text.split(/[^\p{L}']+/).select { |s|
+    #Split words and 
+    words = text.split(/'?[^\p{L}']+'?/).select { |s|
       s != "" and s != "'s" and s != "'"
     }.uniq
     sdbg "self.check word array:\n#{words}\n"

data/lib/middleman-spellcheck/version.rb

@@ -1,5 +1,5 @@
 module Middleman
   module Spellcheck
-    VERSION = "0.9.0"
+    VERSION = "0.9.2"
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: middleman-spellcheck
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.9.2
 platform: ruby
 authors:
 - Ivan Zarea
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-11-03 00:00:00.000000000 Z
+date: 2016-02-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: middleman-core
@@ -127,7 +127,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.2.2
+rubygems_version: 2.4.8
 signing_key: 
 specification_version: 4
 summary: Run spell checks as a build phase in middleman