forspell 0.0.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 9ec99d8b67f987cb0cb0305b2af98796b971ea1a6fd61fe946ffe8685e5cacaf
+  data.tar.gz: 85cce6671e87533b82551377270fc4bf665b854e764a6c4358595d4cfd7622b7
+SHA512:
+  metadata.gz: 25c10c8d9218fd566ae1847f3d66aac56f183099459cd467239bb63f82fecfabce77e5da39b0edaf8c647b97a5db65f33e2021a9580d229fe9d3d42a4fffbfa7
+  data.tar.gz: e9034da3163e853dfcb3dad1a90be1ae3f6fcd8990b8d1a0e6aecddaf692c9f748ba3bca993de3333695181a1b8b730688bfefa1830ff73582884155c519f588

data/README.md

@@ -0,0 +1,88 @@
+# Forspell
+
+[![Gem Version](https://badge.fury.io/rb/forspell.svg)](http://badge.fury.io/rb/forspell)
+[![Build Status](https://travis-ci.org/kkuprikov/forspell.svg?branch=master)](https://travis-ci.org/kkuprikov/forspell)
+
+**Forspell** is spellchecker for code and documentation. It uses well-known [hunspell](https://en.wikipedia.org/wiki/Hunspell) tool and dictionary, provides customizable output, and could be easily integrated into CI pipeline.
+
+Example of output:
+
+![](https://user-images.githubusercontent.com/713419/55152630-d775a600-5161-11e9-9c56-d9fb45d8a3a4.png)
+
+## Features
+
+* Able to check Markdown files and comments inside Ruby, C and C++ source code;
+* Correctly skips over code examples and identifiers inside comments and Markdown most of the time;
+* Distributed with latest en-US hunspell dictionaries, but can be set to use any other dictionary;
+* Includes by default custom Ruby ecosystem-related dictionary (so it will not report as misspelled words like "Gemfile", "args", "middleware" and alike);
+* It is easy to create custom dictionary for your project, so your internal terms would not be reported as misspellings;
+* Option to auto-generate initial custom dictionary;
+* Several output formats, including pretty colorized human-readable, YAML and JSON.
+
+## Installation
+
+```
+gem install forspell
+```
+
+`forspell` requires `libhunspell` >= 1.2.0 to be installed, see [hunspell site](https://github.com/hunspell/hunspell) for installation instructions (it is probably as easy as `<yourpackagemanager> install hunspell` on most of Linuxes and MacOS).
+
+## Usage
+
+```
+$ forspell --help
+Usage: forspell [list of files and folders] [options]
+
+Options:
+    -e, --exclude pathes       List of pathes to exclude
+    -d, --dictionary path      Path to main hunspell dictionary to use (by default, forspell's en_US)
+    -c, --custom pathes        Pathes to custom dictionaries
+    -f, --format format_name   Output formats: readable (default), json, yaml
+    --gen-dictionary           Generate custom dictionary
+    -l, --logfile path         Log to specified path
+    -v, --verbose              Verbose mode
+    --help
+```
+
+forspell automatically loads command-line arguments from `.forspell` file in current folder, if it is present.
+
+### Dictionaries
+
+As already said, forspell uses hunspell dictionaries to check spelling. By default, it is `en_US` modern dictionary, distributed with forspell itself, but you can specify any other with `-d` option. If you have `/some/custom/path/fr_FR.aff` and `/some/custom/path/fr_FR.dic`, than you can use it instead of `en_US` with `-d /some/custom/path/fr_FR`.
+
+In addition to the main dictionary, forspell also includes (and always uses) small auxiliary dictionary with words specific for Ruby ecosystem, you can review it in [forspell repo](https://github.com/kkuprikov/forspell/blob/master/lib/forspell/ruby.dict).
+
+You can also add your project's **custom dictionary** with internal terms, for them not to obstruct spellchecker's output. Custom dictionary (or several) can be passed with `-c` option. Also, `forspell.dict` would be automatically read, if present in current directory. Format of custom dictionary:
+
+* Each word on its own line.
+* Line contains either just `word` (means only this word in this form).
+* ...or `word: example` -- in this case, "example" should be existing word from the main dictionary, and it means your custom term can have the same forms as example word. For example, "subclass: class" will mean that "subclass", "subclasses" and "subclass'" should be considered correct.
+* `#` marks comments.
+
+For larger projects, it is handy to generate custom dictionary with `forspell your,project,pathes --gen-dictionary`. This will create `forspell.dict` with all misspelling found in your project, you can proceed by deleting genuine errors and leave what should be considered correct internal terms.
+
+## Integration with CI
+
+Forspell return codes:
+
+* 0 - when no errors found
+* 1 - when there are any errors
+* 2 - when it could not process with provided options, i.e. no dictionary found, no directories for checking supplied.
+
+To integrate forspell with, say, TravisCI, you would add to `.travis.yml`:
+
+* `sudo apt-get install hunspell` into `before_install` section
+* Just `forspell` into `script` section (given you have proper `.forspell` file with all options what to check and how to output)
+
+## Authors
+
+* [Kirill Kuprikov](https://github.com/kkuprikov)
+* [Victor Shepelev](https://github.com/zverok)
+
+## License
+
+MIT
+
+## Contributing
+
+Feel free to create an issue or open a pull request!

data/exe/clone_repos.sh

@@ -0,0 +1,72 @@
+#!/bin/sh
+
+# Clone multiple repositories.
+
+echo "=== CLONENATOR ==="
+
+array=( "https://github.com/rspec/rspec-core.git"
+        "https://github.com/halostatue/diff-lcs/"  
+        "https://github.com/rspec/rspec-expectations.git"  
+        "https://github.com/rspec/rspec-mocks.git"  
+        "https://github.com/rspec/rspec.git"  
+        "https://github.com/rspec/rspec-support.git"  
+        "https://github.com/erikhuda/thor.git"  
+        "https://github.com/tzinfo/tzinfo.git"  
+        "https://github.com/rtomayko/tilt.git"  
+        "https://github.com/ruby-concurrency/thread_safe.git"  
+        "https://github.com/rack-test/rack-test.git"  
+        "https://github.com/seattlerb/minitest.git"  
+        "https://github.com/sparklemotion/nokogiri.git"  
+        "https://github.com/mikel/mail.git"  
+        "https://github.com/ffi/ffi.git"  
+        "https://github.com/lostisland/faraday.git" 
+        "https://github.com/lautis/uglifier.git"
+        "https://github.com/rest-client/rest-client.git"
+        "https://github.com/pry/pry.git"
+        "https://github.com/eventmachine/eventmachine.git"
+        "https://github.com/redis/redis-rb.git"
+        "https://github.com/sinatra/sinatra.git"
+        "https://github.com/rails/jbuilder.git"
+        "https://github.com/ruby/rdoc.git"
+        "https://github.com/sparklemotion/http-cookie.git"
+        "https://github.com/mileszs/wicked_pdf.git"
+        "https://github.com/kjvarga/sitemap_generator.git"
+        "https://github.com/oivoodoo/devise_masquerade.git"
+        "https://github.com/norman/friendly_id.git"
+        "https://github.com/alexreisner/geocoder.git"
+        "https://github.com/RubyMoney/money-rails.git"
+        "https://github.com/ryanb/letter_opener.git"
+        "https://github.com/varvet/pundit.git"
+        "https://github.com/plataformatec/devise.git"
+        "https://github.com/thoughtbot/factory_bot.git"
+        "https://github.com/teamcapybara/capybara.git"
+        "https://github.com/paper-trail-gem/paper_trail.git"
+        "https://github.com/aasm/aasm.git"
+        "https://github.com/rubysherpas/paranoia.git"
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        ""
+        )
+
+cd ~/gem_src
+
+for element in ${array[@]}
+do
+    echo "clonning $element"
+    git clone $element 
+done
+
+echo "=== DONE ==="

data/exe/create_dictionary

@@ -0,0 +1,47 @@
+#!/usr/bin/env ruby
+require 'yaml'
+require 'pry'
+require 'csv'
+
+class DictionaryCreator
+  attr_reader :result
+
+  def initialize(path: '../test/popular_gems/json/')
+    @path = path
+    @data = {}
+    @result = {}
+  end
+
+  def process
+    (Dir.entries(@path) - ['.', '..']).each do |filename|
+      # binding.pry
+      @data[filename] = YAML.load_file("#{@path}#{filename}")
+    end
+
+    @data.each_pair do |log_filename, data|
+      next if data == false
+
+      gem_name = log_filename.split('.').first
+      data.each do |data_entry|
+        data_entry[:errors].each do |error|
+          if @result[error]
+            @result[error] = @result[error].merge(count: @result.dig(error, :count) + 1, gems: (@result.dig(error, :gems) + [gem_name]).uniq)
+          else
+            @result[error] = { count: 1, gems: [gem_name] }
+          end
+        end
+      end
+    end
+
+    self
+  end
+end
+
+# DictionaryCreator.new.process.result
+
+CSV.open('../ruby_dict.csv', 'wb') do |csv|
+  csv << ['word', 'count', 'gems', 'should_include?']
+  DictionaryCreator.new.process.result.each_pair do |word, data|
+    csv << [word, data[:count], data[:gems].join(', '), nil]
+  end
+end

data/exe/forspell

@@ -0,0 +1,4 @@
+#!/usr/bin/env ruby
+require_relative '../lib/forspell/cli'
+
+Forspell::CLI.new(ARGV).call

data/exe/generate_logs

@@ -0,0 +1,8 @@
+#!/bin/bash
+cd ~/forspell
+rm ./test/popular_gems/json/*.log
+
+for var in $(ls ../gem_src/)
+do
+  ./bin/forspell ../gem_src/$var test/popular_gems/json/$var.log
+done

data/lib/forspell/cli.rb

@@ -0,0 +1,91 @@
+# frozen_string_literal: true
+
+require 'slop'
+require 'backports/2.5.0/hash/slice'
+require_relative 'runner'
+require_relative 'speller'
+require_relative 'reporter'
+require_relative 'file_list'
+
+module Forspell
+  class CLI
+    ERROR_CODE = 2
+    CONFIG_PATH = File.join(Dir.pwd, '.forspell')
+    DEFAULT_CUSTOM_DICT = File.join(Dir.pwd, 'forspell.dict')
+
+    FORMATS = %w[readable yaml json].freeze
+    FORMAT_ERR = "must be one of the following: #{FORMATS.join(', ')}"
+    DEFINITIONS = proc do |o|
+      o.array '-e', '--exclude-paths', 'List of paths to exclude'
+      o.string '-d', '--dictionary-path', 'Path to main hunspell dictionary to use (by default, forspell\'s en_US)', default: 'en_US'
+      o.array '-c', '--custom-paths', 'Paths to custom dictionaries', default: []
+      o.string '-f', '--format', 'Output formats: readable(default), JSON, YAML', default: 'readable'
+      o.boolean '--gen-dictionary', 'Generate custom dictionary'
+      o.string '-l', '--logfile', 'Log to specified path'
+      o.bool '-v', '--verbose', 'Verbose mode'
+      o.on '--help' do
+        puts o
+        exit
+      end
+    end
+
+    def initialize options
+      @options = options
+    end
+
+    def call
+      init_options
+      create_files_list
+      init_speller
+      init_reporter
+      run
+    end
+
+    private
+
+    def create_files_list
+      @files = FileList.new(paths: @opts.arguments, exclude_paths: @opts[:exclude_paths])
+    end
+
+    def init_options
+      @options += File.read(CONFIG_PATH).tr("\n", ' ').split(' ') if File.exist?(CONFIG_PATH)
+
+      @opts = Slop.parse(@options, &DEFINITIONS)
+
+      if @opts.arguments.empty?
+        puts 'Usage: forspell paths to check [options]'
+        puts 'Type --help for more info'
+        exit(ERROR_CODE)
+      end
+      
+      @opts[:format] = 'dictionary' if @opts[:gen_dictionary]
+      @opts[:format] = @opts[:format]&.downcase
+    end
+
+    def init_speller
+      @opts[:custom_paths].each do |path|
+        next if File.exist?(path)
+
+        puts "Custom dictionary not found: #{path}"
+        exit(ERROR_CODE)
+      end
+
+      @opts[:custom_paths] << DEFAULT_CUSTOM_DICT if File.exist?(DEFAULT_CUSTOM_DICT)
+
+      @speller = Speller.new(@opts[:dictionary_path], *@opts[:custom_paths])
+    end
+
+    def init_reporter
+      @reporter = Reporter.new(**@opts.to_hash.slice(:logfile, :format, :verbose))
+    end
+
+    def run
+      runner = Forspell::Runner.new(files: @files, speller: @speller, reporter: @reporter)
+      runner.call
+      exit @reporter.finalize
+    rescue Forspell::FileList::PathLoadError => path
+      @reporter.path_load_error path
+      exit ERROR_CODE
+    end
+  end
+end

data/lib/forspell/dictionaries/en_US.aff

@@ -0,0 +1,205 @@
+SET UTF-8
+TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ'
+ICONV 1
+ICONV ’ '
+NOSUGGEST !
+
+# ordinal numbers
+COMPOUNDMIN 1
+# only in compounds: 1th, 2th, 3th
+ONLYINCOMPOUND c
+# compound rules:
+# 1. [0-9]*1[0-9]th (10th, 11th, 12th, 56714th, etc.)
+# 2. [0-9]*[02-9](1st|2nd|3rd|[4-9]th) (21st, 22nd, 123rd, 1234th, etc.)
+COMPOUNDRULE 2
+COMPOUNDRULE n*1t
+COMPOUNDRULE n*mp
+WORDCHARS 0123456789
+
+PFX A Y 1
+PFX A   0     re         .
+
+PFX I Y 1
+PFX I   0     in         .
+
+PFX U Y 1
+PFX U   0     un         .
+
+PFX C Y 1
+PFX C   0     de          .
+
+PFX E Y 1
+PFX E   0     dis         .
+
+PFX F Y 1
+PFX F   0     con         .
+
+PFX K Y 1
+PFX K   0     pro         .
+
+SFX V N 2
+SFX V   e     ive        e
+SFX V   0     ive        [^e]
+
+SFX N Y 3
+SFX N   e     ion        e
+SFX N   y     ication    y 
+SFX N   0     en         [^ey] 
+
+SFX X Y 3
+SFX X   e     ions       e
+SFX X   y     ications   y
+SFX X   0     ens        [^ey]
+
+SFX H N 2
+SFX H   y     ieth       y
+SFX H   0     th         [^y] 
+
+SFX Y Y 1
+SFX Y   0     ly         .
+
+SFX G Y 2
+SFX G   e     ing        e
+SFX G   0     ing        [^e] 
+
+SFX J Y 2
+SFX J   e     ings       e
+SFX J   0     ings       [^e]
+
+SFX D Y 4
+SFX D   0     d          e
+SFX D   y     ied        [^aeiou]y
+SFX D   0     ed         [^ey]
+SFX D   0     ed         [aeiou]y
+
+SFX T N 4
+SFX T   0     st         e
+SFX T   y     iest       [^aeiou]y
+SFX T   0     est        [aeiou]y
+SFX T   0     est        [^ey]
+
+SFX R Y 4
+SFX R   0     r          e
+SFX R   y     ier        [^aeiou]y
+SFX R   0     er         [aeiou]y
+SFX R   0     er         [^ey]
+
+SFX Z Y 4
+SFX Z   0     rs         e
+SFX Z   y     iers       [^aeiou]y
+SFX Z   0     ers        [aeiou]y
+SFX Z   0     ers        [^ey]
+
+SFX S Y 4
+SFX S   y     ies        [^aeiou]y
+SFX S   0     s          [aeiou]y
+SFX S   0     es         [sxzh]
+SFX S   0     s          [^sxzhy]
+
+SFX P Y 3
+SFX P   y     iness      [^aeiou]y
+SFX P   0     ness       [aeiou]y
+SFX P   0     ness       [^y]
+
+SFX M Y 1
+SFX M   0     's         .
+
+SFX B Y 3
+SFX B   0     able       [^aeiou]
+SFX B   0     able       ee
+SFX B   e     able       [^aeiou]e
+
+SFX L Y 1
+SFX L   0     ment       .
+
+REP 90
+REP a ei
+REP ei a
+REP a ey
+REP ey a
+REP ai ie
+REP ie ai
+REP alot a_lot
+REP are air
+REP are ear
+REP are eir
+REP air are
+REP air ere
+REP ere air
+REP ere ear
+REP ere eir
+REP ear are
+REP ear air
+REP ear ere
+REP eir are
+REP eir ere
+REP ch te
+REP te ch
+REP ch ti
+REP ti ch
+REP ch tu
+REP tu ch
+REP ch s
+REP s ch
+REP ch k
+REP k ch
+REP f ph
+REP ph f
+REP gh f
+REP f gh
+REP i igh
+REP igh i
+REP i uy
+REP uy i
+REP i ee
+REP ee i
+REP j di
+REP di j
+REP j gg
+REP gg j
+REP j ge
+REP ge j
+REP s ti
+REP ti s
+REP s ci
+REP ci s
+REP k cc
+REP cc k
+REP k qu
+REP qu k
+REP kw qu
+REP o eau
+REP eau o
+REP o ew
+REP ew o
+REP oo ew
+REP ew oo
+REP ew ui
+REP ui ew
+REP oo ui
+REP ui oo
+REP ew u
+REP u ew
+REP oo u
+REP u oo
+REP u oe
+REP oe u
+REP u ieu
+REP ieu u
+REP ue ew
+REP ew ue
+REP uff ough
+REP oo ieu
+REP ieu oo
+REP ier ear
+REP ear ier
+REP ear air
+REP air ear
+REP w qu
+REP qu w
+REP z ss
+REP ss z
+REP shun tion
+REP shun sion
+REP shun cion
+REP size cise