middleman-spellcheck 0.9.0 → 0.9.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +118 -21
- data/lib/middleman-spellcheck/cli.rb +91 -4
- data/lib/middleman-spellcheck/extension.rb +62 -14
- data/lib/middleman-spellcheck/spellchecker.rb +4 -1
- data/lib/middleman-spellcheck/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d50552923d8b8b4e3f9f0bcdc9a6f17f5229ecf9
|
4
|
+
data.tar.gz: fdd05ae0af2f6c4bcd0ca9674c28e226456c510c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 14cb5229c5da423142e23b3efe3cd0d5d7f3e2e5b0c099211e20ad3e677a3031d778cb8cf3e83a0c225c32722ec6b2f3d758c6c9911e5c70fe7b9d4480f065d1
|
7
|
+
data.tar.gz: e04e1cf5302675eaac3d9d06161fb0b0a2307ff24a3c10d93a8e7b95f655a954f01f021e1009220e4fdd9e7a9f7dc81748a89ecf8b9fdc2e59f2378cc5d9ab1e
|
data/README.md
CHANGED
@@ -25,31 +25,50 @@ middleman spellcheck source/blog/
|
|
25
25
|
|
26
26
|
## Usage
|
27
27
|
|
28
|
-
|
28
|
+
There are several ways to select what content will be checked.
|
29
29
|
|
30
|
-
|
31
|
-
activate :spellcheck, page: "documentation/*" # you can use regexes, too, e.g. /post_[1-9]/
|
32
|
-
```
|
30
|
+
1. To spellcheck only some resources using a regex with the URL:
|
33
31
|
|
34
|
-
|
32
|
+
```ruby
|
33
|
+
activate :spellcheck, page: "documentation/*" # you can use regexes, too, e.g. /post_[1-9]/
|
34
|
+
```
|
35
35
|
|
36
|
-
|
37
|
-
activate :spellcheck, tags: :p # pass an array of tags if you have more!
|
38
|
-
```
|
36
|
+
2. To limit which tags the spell checker will only run through:
|
39
37
|
|
40
|
-
|
38
|
+
```ruby
|
39
|
+
activate :spellcheck, tags: :p # pass an array of tags if you have more!
|
40
|
+
```
|
41
41
|
|
42
|
-
|
43
|
-
|
44
|
-
|
42
|
+
3. To ignore sections by using css selectors
|
43
|
+
For example, to ignore all sections with a class of `CodeRay`:
|
44
|
+
|
45
|
+
```ruby
|
46
|
+
activate :spellcheck, ignore_selector: '.CodeRay'
|
47
|
+
```
|
48
|
+
|
49
|
+
Or to ignore all tables in a document:
|
50
|
+
|
51
|
+
```ruby
|
52
|
+
activate :spellcheck, ignore_selector: 'table'
|
53
|
+
```
|
45
54
|
|
46
|
-
|
55
|
+
Or to ignore all `<p class="technical-jargon">`:
|
56
|
+
```ruby
|
57
|
+
activate :spellcheck, ignore_selector: 'p.technical-jargon'
|
58
|
+
```
|
59
|
+
|
60
|
+
To ignore multiple selectors, seperate them with a comma
|
61
|
+
```ruby
|
62
|
+
activate :spellcheck, ignore_selector: 'p.technical-jargon, .CodeRay'
|
63
|
+
```
|
64
|
+
|
65
|
+
4. Middleman-spellcheck automatically ignores `.css`, `.js`, & `.coffee` file
|
47
66
|
extensions. If there are some additional file type extensions that you would
|
48
67
|
like to skip:
|
49
68
|
|
50
|
-
```ruby
|
51
|
-
activate :spellcheck, ignored_exts: [".xml", ".png"]
|
52
|
-
```
|
69
|
+
```ruby
|
70
|
+
activate :spellcheck, ignored_exts: [".xml", ".png"]
|
71
|
+
```
|
53
72
|
|
54
73
|
To select a dictionary used by a spellchecker, use lang: option. For
|
55
74
|
example, to use Polish dictionary, use:
|
@@ -60,12 +79,15 @@ activate :spellcheck, lang: "pl"
|
|
60
79
|
|
61
80
|
If you define the ``lang`` metadata in your pages / articles, then spellcheck will use those language.
|
62
81
|
|
63
|
-
|
64
|
-
|
65
|
-
|
82
|
+
## Options
|
83
|
+
|
84
|
+
|
85
|
+
For warnings only (allow build to pass), use `dontfail` option.
|
86
|
+
This is helpful when you want to give yourself a chance to fix mistakes or false hits gradually and
|
87
|
+
not fail each time you build.
|
66
88
|
|
67
89
|
```ruby
|
68
|
-
activate :spellcheck,
|
90
|
+
activate :spellcheck, dontfail: 1
|
69
91
|
```
|
70
92
|
|
71
93
|
You can also disable the automatic spellcheck after build (and only run manual checks from the command line):
|
@@ -89,9 +111,84 @@ who encountered issues, useful might be debug: option, which will turn on
|
|
89
111
|
extensive amount of debugging.
|
90
112
|
|
91
113
|
```ruby
|
92
|
-
activate :spellcheck, debug: 1
|
114
|
+
activate :spellcheck, lang: "en", debug: 1
|
93
115
|
```
|
94
116
|
|
117
|
+
If there are some words that you would like to be allowed you can pass them to the allow option as an array. **Depricated - Please see the "Fixing spelling mistakes" section for now prefered way to include allowed words
|
118
|
+
|
119
|
+
```ruby
|
120
|
+
activate :spellcheck, allow: ["Gooby", "pls"]
|
121
|
+
```
|
122
|
+
|
123
|
+
You can also pass a regex to the `ignore_regex` option. Any match will be ignored.
|
124
|
+
For example to remove words in quotes
|
125
|
+
```ruby
|
126
|
+
activate :spellcheck, lang: "en", ignore_regex: /\s('|")\w*('|")(\s|\.|,)/
|
127
|
+
```
|
128
|
+
|
129
|
+
## Fixing spelling mistakes & false positives
|
130
|
+
|
131
|
+
The `middleman-spellchecker` extension is likely to generate large number
|
132
|
+
of false-positives, e.g.: words which the spellchecker will consider
|
133
|
+
incorrect (not present in a dictionary), which yet may have a valid meaning
|
134
|
+
in the article's context. Common problems are acronyms, technical terms and
|
135
|
+
names. To solve this, `middleman-spellcheck` offers two solutions:
|
136
|
+
|
137
|
+
1. The `spellcheck_allow_file` file, which points to the path with a file
|
138
|
+
containing words considered correct. Author of the website may decide which
|
139
|
+
words are allowed to be used site-wide. Example: if you write a lot about
|
140
|
+
IBM products, this file would have names such as "IBM", "AIX" or "DB/2".
|
141
|
+
Add the words one word per line without quotes.
|
142
|
+
|
143
|
+
To set the global file, use the following clause in your `config.rb`:
|
144
|
+
|
145
|
+
```set :spellcheck_allow_file, "./data/words_allowed.txt"```
|
146
|
+
|
147
|
+
2. The `spellcheck-allow` keyword in a frontmatter, which will work in the
|
148
|
+
context of this particular article, but not other articles. Example: your
|
149
|
+
blog is about IBM, but 1 article is about AirBnB. You'd put `AirBnB` into
|
150
|
+
your front-matter.
|
151
|
+
|
152
|
+
|
153
|
+
To use 2nd solution, add the following to your frontmatter:
|
154
|
+
|
155
|
+
```
|
156
|
+
title: "Blog about IBM"
|
157
|
+
...
|
158
|
+
spellcheck-allow:
|
159
|
+
- "AirBnB"```
|
160
|
+
|
161
|
+
Another example
|
162
|
+
|
163
|
+
```
|
164
|
+
title: "Some time ago"
|
165
|
+
...
|
166
|
+
spellcheck-allowed:
|
167
|
+
- GitHub
|
168
|
+
- Linux
|
169
|
+
```
|
170
|
+
|
171
|
+
The `middleman-spellcheck` also comes with a simple CLI for fixing many
|
172
|
+
problems in your articles. To invoke:
|
173
|
+
|
174
|
+
```middleman spellcheck source/blog/2015-11-01-nginx-on-travis-ci.md --fix```
|
175
|
+
|
176
|
+
This will pull up simple CLI menu and for each misspelled word, you'll have
|
177
|
+
a following choice
|
178
|
+
|
179
|
+
| Key to press | Effect |
|
180
|
+
|--------------|--------|
|
181
|
+
| g | Add the word to the `spellcheck_allow_file` |
|
182
|
+
| f | Add the word to this article's front-matter |
|
183
|
+
| i | Ignore the word for now and deal with it later |
|
184
|
+
|
185
|
+
|
186
|
+
After the run is finished, `middleman-spellchecker` will write a fixed file
|
187
|
+
to `source/blog/2015-11-01-nginx-on-travis-ci.md.fixed`. This is a safe
|
188
|
+
choice for not creating damage. If you don't want to fiddle with it, the
|
189
|
+
`--inplace` switch will make changes dynamically, and the input file will
|
190
|
+
get overwritten.
|
191
|
+
|
95
192
|
## Contributing
|
96
193
|
|
97
194
|
1. Fork it
|
@@ -1,10 +1,75 @@
|
|
1
1
|
module Middleman
|
2
2
|
module Cli
|
3
|
-
# This class provides
|
3
|
+
# This class provides a "spellcheck" command for the middleman CLI.
|
4
4
|
class Spellcheck < Thor
|
5
5
|
include Thor::Actions
|
6
|
+
|
7
|
+
check_unknown_options!
|
8
|
+
|
9
|
+
namespace :spellcheck
|
10
|
+
|
11
|
+
no_commands {
|
12
|
+
def misspell_ask(misspell, allow_file)
|
13
|
+
if not options[:fix]
|
14
|
+
return false
|
15
|
+
end
|
16
|
+
|
17
|
+
print "-" * 70, "\n"
|
18
|
+
print "'#{misspell[:word]}' is misspelled. What to do?:\n"
|
19
|
+
print "g\tadd allowed word to '#{allow_file}'\n" if allow_file
|
20
|
+
print "f\tadd allowed word to frontmatter\n"
|
21
|
+
print "i\tignore word (deal with it later)\n"
|
22
|
+
|
23
|
+
print "Select option and press ENTER:\n"
|
24
|
+
STDIN.gets()[0]
|
25
|
+
end
|
26
|
+
|
27
|
+
def misspell_fixes_global(resource, words_allow_global, allow_file)
|
28
|
+
open(allow_file, 'a') { |f|
|
29
|
+
words_allow_global.each do |w|
|
30
|
+
f.puts "#{w}\t#\t#{resource.source_file}\n"
|
31
|
+
end
|
32
|
+
}
|
33
|
+
end
|
34
|
+
|
35
|
+
def misspell_fixes_frontmatter(resource, words_allow_frontmatter)
|
36
|
+
data = File.read(resource.source_file)
|
37
|
+
if options[:inplace] then fn_ext = "" else ".fixed" end
|
38
|
+
fn_fixed = "#{resource.source_file}#{fn_ext}"
|
39
|
+
fixed = File.open(fn_fixed, "w")
|
40
|
+
sep_tag_cnt = 0
|
41
|
+
had_spellcheck_tag = false
|
42
|
+
data.each_line do |line|
|
43
|
+
if line =~ /^\-\-\-$/ then
|
44
|
+
sep_tag_cnt += 1
|
45
|
+
end
|
46
|
+
if line =~ /spellcheck-allow:/ then
|
47
|
+
had_spellcheck_tag = true
|
48
|
+
end
|
49
|
+
|
50
|
+
if sep_tag_cnt == 2 then
|
51
|
+
if had_spellcheck_tag == false then
|
52
|
+
fixed.puts "spellcheck-allow:\n"
|
53
|
+
end
|
54
|
+
words_allow_frontmatter.each do |w|
|
55
|
+
fixed.puts "- \"#{w}\"\n"
|
56
|
+
end
|
57
|
+
fixed.puts "---\n"
|
58
|
+
sep_tag_cnt = nil
|
59
|
+
else
|
60
|
+
fixed.puts line
|
61
|
+
end
|
62
|
+
end
|
63
|
+
fixed.close()
|
64
|
+
print "Fixed spellchecked file written to #{fn_fixed}\n"
|
65
|
+
end
|
66
|
+
}
|
67
|
+
|
6
68
|
namespace :spellcheck
|
7
69
|
desc "spellcheck FILE", "Run spellcheck on given file or path"
|
70
|
+
method_option "fix", :type => :boolean, :aliases => "-f", :desc => "Fix spelling error interactively"
|
71
|
+
method_option "inplace", :type => :boolean, :aliases => "-i", :desc => "Modify files in place (dangerous)"
|
72
|
+
|
8
73
|
def spellcheck(*paths)
|
9
74
|
app = ::Middleman::Application.server.inst
|
10
75
|
|
@@ -14,18 +79,40 @@ module Middleman
|
|
14
79
|
}
|
15
80
|
}
|
16
81
|
if resources.empty?
|
17
|
-
$stderr.puts "File / Directory #{
|
82
|
+
$stderr.puts "File / Directory #{paths} not exist"
|
18
83
|
exit 1
|
19
84
|
end
|
20
85
|
ext = app.extensions[:spellcheck]
|
86
|
+
|
87
|
+
allow_file = app.config.defines_setting?(:spellcheck_allow_file) ?
|
88
|
+
app.config[:spellcheck_allow_file] : nil
|
89
|
+
if options[:fix] then
|
90
|
+
print "Spellchecker fix mode on! Will attempt to white-list some words\n"
|
91
|
+
end
|
21
92
|
resources.each do |resource|
|
22
|
-
say_status :spellcheck, "Running spell checker for #{resource.
|
93
|
+
say_status :spellcheck, "Running spell checker for #{resource.source_file}", :blue
|
23
94
|
current_misspelled = ext.spellcheck_resource(resource)
|
95
|
+
|
96
|
+
words_allow_frontmatter = []
|
97
|
+
words_allow_global = []
|
24
98
|
current_misspelled.each do |misspell|
|
25
|
-
|
99
|
+
fix = misspell_ask(misspell, allow_file)
|
100
|
+
was_fixed = true
|
101
|
+
if fix == 'g' then words_allow_global << misspell[:word]
|
102
|
+
elsif fix == 'f' then words_allow_frontmatter << misspell[:word]
|
103
|
+
else was_fixed = false end
|
104
|
+
|
105
|
+
if was_fixed
|
106
|
+
say_status :spellcheck, "Fixed word #{misspell[:word]}"
|
107
|
+
else
|
108
|
+
say_status :misspell, ext.error_message(misspell), :red
|
109
|
+
end
|
26
110
|
end
|
111
|
+
misspell_fixes_global(resource, words_allow_global, allow_file) unless words_allow_global.empty?
|
112
|
+
misspell_fixes_frontmatter(resource, words_allow_frontmatter) unless words_allow_frontmatter.empty?
|
27
113
|
end
|
28
114
|
end
|
115
|
+
|
29
116
|
end
|
30
117
|
end
|
31
118
|
end
|
@@ -6,14 +6,17 @@ module Middleman
|
|
6
6
|
module Spellcheck
|
7
7
|
class SpellcheckExtension < Extension
|
8
8
|
REJECTED_EXTS = %w(.css .js .coffee)
|
9
|
+
ALLOWED_WORDS = []
|
9
10
|
option :page, "/*", "Run only pages that match the regex through the spellchecker"
|
10
11
|
option :tags, [], "Run spellcheck only on some tags from the output"
|
11
12
|
option :allow, [], "Allow specific words to be misspelled"
|
12
13
|
option :ignored_exts, [], "Ignore specific extensions (ex: '.xml')"
|
14
|
+
option :ignore_regex, false, "Ignore regex matches"
|
15
|
+
option :ignore_selector, false, "Ignore nodes with a css selector"
|
13
16
|
option :lang, "en", "Language for spellchecking"
|
14
17
|
option :cmdargs, "", "Pass alternative command line arguments"
|
15
18
|
option :debug, 0, "Enable debugging (for developers only)"
|
16
|
-
option :dontfail,
|
19
|
+
option :dontfail, false, "Don't fail because misspelled words are found"
|
17
20
|
option :run_after_build, true, "Run Spellcheck after build"
|
18
21
|
|
19
22
|
def after_build(builder)
|
@@ -32,13 +35,14 @@ module Middleman
|
|
32
35
|
total_misspelled += current_misspelled
|
33
36
|
end
|
34
37
|
|
35
|
-
|
36
|
-
|
38
|
+
builder.say_status :spellcheck, "Spellchecks done. #{total_misspelled.length} misspelling(s) found.", :blue
|
39
|
+
|
40
|
+
unless total_misspelled.empty?
|
37
41
|
if options.dontfail
|
38
|
-
|
39
|
-
print estr, "\n"
|
42
|
+
builder.say_status :spellcheck, "dontfail is set! Builder will ignore misspellings.", :yellow
|
40
43
|
else
|
41
|
-
|
44
|
+
desc = "Build failed. There are spelling errors."
|
45
|
+
raise Thor::Error, desc
|
42
46
|
end
|
43
47
|
end
|
44
48
|
end
|
@@ -48,6 +52,10 @@ module Middleman
|
|
48
52
|
doc = Nokogiri::HTML.fragment(rendered_resource)
|
49
53
|
doc.search('code,style,script').each(&:remove)
|
50
54
|
|
55
|
+
if options.ignore_selector
|
56
|
+
doc.css(options.ignore_selector).each(&:remove)
|
57
|
+
end
|
58
|
+
|
51
59
|
if options.tags.empty?
|
52
60
|
doc.text
|
53
61
|
else
|
@@ -55,6 +63,14 @@ module Middleman
|
|
55
63
|
end
|
56
64
|
end
|
57
65
|
|
66
|
+
def regex_filter_content(text)
|
67
|
+
if options.ignore_regex
|
68
|
+
text.to_s.gsub options.ignore_regex , ' '
|
69
|
+
else
|
70
|
+
text
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
58
74
|
def option_tags
|
59
75
|
if options.tags.is_a? Array
|
60
76
|
options.tags
|
@@ -85,26 +101,58 @@ module Middleman
|
|
85
101
|
else
|
86
102
|
options.lang
|
87
103
|
end
|
88
|
-
run_check(
|
104
|
+
run_check(resource, lang)
|
89
105
|
end
|
90
106
|
|
91
|
-
def run_check(
|
107
|
+
def run_check(resource, lang)
|
108
|
+
text = select_content(resource)
|
109
|
+
text = regex_filter_content(text)
|
92
110
|
results = Spellchecker.check(text, lang)
|
93
|
-
results = exclude_allowed(results)
|
111
|
+
results = exclude_allowed(resource, results)
|
94
112
|
results.reject { |entry| entry[:correct] }
|
95
113
|
end
|
96
114
|
|
97
|
-
def exclude_allowed(results)
|
98
|
-
results.reject { |entry|
|
115
|
+
def exclude_allowed(resource, results)
|
116
|
+
results.reject { |entry| allowed_words(resource).include? entry[:word].downcase }
|
99
117
|
end
|
100
118
|
|
101
|
-
def
|
102
|
-
|
119
|
+
def allowed_words(resource)
|
120
|
+
words_ok = if options.allow.is_a? Array
|
103
121
|
options.allow
|
104
122
|
else
|
105
123
|
[options.allow]
|
106
124
|
end
|
107
|
-
|
125
|
+
words_ok += allowed_words_frontmatter(resource)
|
126
|
+
words_ok += allowed_words_file
|
127
|
+
words_ok.map(&:downcase)
|
128
|
+
end
|
129
|
+
|
130
|
+
def allowed_words_frontmatter(resource)
|
131
|
+
words_ok = []
|
132
|
+
if resource.data.include?("spellcheck-allow") then
|
133
|
+
allowed_tmp = resource.data["spellcheck-allow"]
|
134
|
+
words_ok += if allowed_tmp.is_a? Array
|
135
|
+
allowed_tmp
|
136
|
+
else
|
137
|
+
[allowed_tmp]
|
138
|
+
end
|
139
|
+
end
|
140
|
+
words_ok
|
141
|
+
end
|
142
|
+
|
143
|
+
def allowed_words_file
|
144
|
+
allow_file = nil
|
145
|
+
if app.config.defines_setting?(:spellcheck_allow_file)
|
146
|
+
allow_file = app.config[:spellcheck_allow_file]
|
147
|
+
end
|
148
|
+
if ALLOWED_WORDS.empty? && allow_file != nil
|
149
|
+
lines = File.read(allow_file)
|
150
|
+
lines.split("\n").each do |line|
|
151
|
+
next if line =~ /^#/ or line =~ /^$/
|
152
|
+
ALLOWED_WORDS << line.partition('#').first.strip
|
153
|
+
end
|
154
|
+
end
|
155
|
+
ALLOWED_WORDS
|
108
156
|
end
|
109
157
|
|
110
158
|
def option_ignored_exts
|
@@ -1,3 +1,5 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
|
1
3
|
class Spellchecker
|
2
4
|
@@aspell_path = "aspell"
|
3
5
|
@@aspell_cmdargs = ""
|
@@ -76,7 +78,8 @@ class Spellchecker
|
|
76
78
|
text.gsub! '’', '\''
|
77
79
|
sdbg "self.check got raw text:\n#{text}\n"
|
78
80
|
|
79
|
-
words
|
81
|
+
#Split words and
|
82
|
+
words = text.split(/'?[^\p{L}']+'?/).select { |s|
|
80
83
|
s != "" and s != "'s" and s != "'"
|
81
84
|
}.uniq
|
82
85
|
sdbg "self.check word array:\n#{words}\n"
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: middleman-spellcheck
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.9.
|
4
|
+
version: 0.9.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ivan Zarea
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2016-02-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: middleman-core
|
@@ -127,7 +127,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
127
127
|
version: '0'
|
128
128
|
requirements: []
|
129
129
|
rubyforge_project:
|
130
|
-
rubygems_version: 2.
|
130
|
+
rubygems_version: 2.4.8
|
131
131
|
signing_key:
|
132
132
|
specification_version: 4
|
133
133
|
summary: Run spell checks as a build phase in middleman
|