lint_search 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: fcbcdbde064e95acad037b44eaec0eef298d2be4
+  data.tar.gz: 6461ff7b45244a76821bc2b12e7a2e11892b2d95
+SHA512:
+  metadata.gz: c88fe9748c5b59a3d7c88d6d3407a7fbcd6ccd2641188f33c396dbed8900501817d25978ad46585059f92205dab495ae9ffd113002aa34395e0dd613fa581b7f
+  data.tar.gz: e84030d65a984c9ca01e274664a9db08935706796e11330275e47ba53b7a6f11935d08b1d52f26a89c77ab324a439779dc5a6a94273142a658d0f824a294a8d9

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,4 @@
+language: ruby
+rvm:
+  - 2.1.2
+before_install: gem install bundler -v 1.10.6

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in lint_search.gemspec
+gemspec

data/README.md

@@ -0,0 +1,73 @@
+# LintSearch
+
+LintSearch is a ruby console application for searching for a keyword and linting the HTML of the first result.
+
+
+## Installation
+
+`gem install lint_search`
+
+## Usage
+
+For full information:
+
+`lint_search help`
+
+If no alternatives are provided, LintSearch will perform the search using Google and the linting using the W3C validator.
+
+To search for a keyword using only default settings run:
+
+`lint_search search [KEYWORD]`
+
+For example, `lint_search search banana` will output:
+
+```
+Searching for banana...
+Complete!
+Lint results of https://en.wikipedia.org/wiki/Banana via https://validator.w3.org
+(0)
+Info: The Content-Type was text/html. Using the HTML parser.
+---
+
+(1)
+Info: Using the schema for HTML with SVG 1.1, MathML 3.0, RDFa 1.1, and ITS 2.0 support.
+---
+
+(2)
+Warning: A table row was 2 columns wide, which is less than the column count established by the first row (3).From line 1128, column 22; to line 1129, column 5="2"></td>↩</tr>↩<tr>↩
+---
+
+(3)
+Warning: A table row was 2 columns wide, which is less than the column count established by the first row (3).From line 1187, column 22; to line 1188, column 5="2"></td>↩</tr>↩<tr>↩
+---
+
+(4)
+Warning: Potentially bad value zh-min-nan for attribute lang on element a: The grandfathered language tag zh-min-nan is deprecated. Use nan instead.From line 1680, column 1670; to line 1680, column 1797-min-nan"><a href="//zh-min-nan.wikipedia.org/wiki/Kin-chio" title="Kin-chio – Chinese (Min Nan)" lang="zh-min-nan" hreflang="zh-min-nan">Bân-lâSyntax of language tag:An RFC 5646 language tag consists of hyphen-separated ASCII-alphanumeric subtags. There is a primary tag identifying a natural language by its shortest ISO 639 language code (e.g. en for English) and zero or more additional subtags adding precision. The most common additional subtag type is a region subtag which most commonly is a two-letter ISO 3166 country code (e.g. GB for the United Kingdom). IANA maintains a registry of permissible subtags.
+---
+
+(5)
+Warning: Potentially bad value zh-min-nan for attribute hreflang on element a: The grandfathered language tag zh-min-nan is deprecated. Use nan instead.From line 1680, column 1670; to line 1680, column 1797-min-nan"><a href="//zh-min-nan.wikipedia.org/wiki/Kin-chio" title="Kin-chio – Chinese (Min Nan)" lang="zh-min-nan" hreflang="zh-min-nan">Bân-lâSyntax of language tag:An RFC 5646 language tag consists of hyphen-separated ASCII-alphanumeric subtags. There is a primary tag identifying a natural language by its shortest ISO 639 language code (e.g. en for English) and zero or more additional subtags adding precision. The most common additional subtag type is a region subtag which most commonly is a two-letter ISO 3166 country code (e.g. GB for the United Kingdom). IANA maintains a registry of permissible subtags.
+---
+
+(6)
+Warning: Potentially bad value zh-yue for attribute lang on element a: The language tag zh-yue is deprecated. Use yue instead.From line 1680, column 21690; to line 1680, column 21801i-zh-yue"><a href="//zh-yue.wikipedia.org/wiki/%E9%A6%99%E8%95%89" title="香蕉 – Cantonese" lang="zh-yue" hreflang="zh-yue">粵語</a>Syntax of language tag:An RFC 5646 language tag consists of hyphen-separated ASCII-alphanumeric subtags. There is a primary tag identifying a natural language by its shortest ISO 639 language code (e.g. en for English) and zero or more additional subtags adding precision. The most common additional subtag type is a region subtag which most commonly is a two-letter ISO 3166 country code (e.g. GB for the United Kingdom). IANA maintains a registry of permissible subtags.
+---
+
+(7)
+Warning: Potentially bad value zh-yue for attribute hreflang on element a: The language tag zh-yue is deprecated. Use yue instead.From line 1680, column 21690; to line 1680, column 21801i-zh-yue"><a href="//zh-yue.wikipedia.org/wiki/%E9%A6%99%E8%95%89" title="香蕉 – Cantonese" lang="zh-yue" hreflang="zh-yue">粵語</a>Syntax of language tag:An RFC 5646 language tag consists of hyphen-separated ASCII-alphanumeric subtags. There is a primary tag identifying a natural language by its shortest ISO 639 language code (e.g. en for English) and zero or more additional subtags adding precision. The most common additional subtag type is a region subtag which most commonly is a two-letter ISO 3166 country code (e.g. GB for the United Kingdom). IANA maintains a registry of permissible subtags.
+---
+```
+
+
+
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/OkayDave/lint_search.
+

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "lint_search"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/lint_search

@@ -0,0 +1,50 @@
+#!/usr/bin/env ruby
+
+require 'rubygems'
+require 'thor'
+require 'lint_search'
+
+module LintSearch
+  class CLI < Thor
+    include Thor::Actions
+    default_task :help
+
+
+    desc "search", "Search for a keyword and lint the first result"
+    method_option :query_url, aliases: "-q", desc: "Search query url", default: "http://www.google.co.uk/"
+    method_option :query_css_selector, aliases: "-c", desc: "Search result CSS selector", default: "h3.r a"
+    method_option :linter_url, aliases: "-l", desc: "Linter url", default: "https://validator.w3.org"
+    method_option :linter_css_selector, aliases: "-r", desc: "Linter CSS Selector", default: "li.info"
+    def search(keyword)
+      puts "Searching for #{keyword}..."
+      search_result = LintSearch::Search.new({
+        keyword:      keyword,
+        query_url:    options[:query_url],
+        css_selector: options[:query_css_selector]
+      }).search
+
+      puts "Linting #{search_result.url}..."
+      lint_result = LintSearch::Lint.new({
+        search_result: search_result,
+        linter_url: options[:linter_url],
+        css_selector: options[:linter_css_selector]
+      }).lint
+
+      puts "Complete!"
+      puts "Lint results of #{search_result.url} via #{options[:linter_url]}"
+      lint_result.each_with_index do |result, index|
+        puts "(#{index})"
+        puts result
+        puts "---"
+        puts ""
+      end
+
+
+    end
+
+  end
+
+end
+
+
+LintSearch::CLI.start

data/bin/setup

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/lint_search/lint.rb

@@ -0,0 +1,40 @@
+require 'rubygems'
+require 'mechanize'
+
+module LintSearch
+  class Lint
+
+    attr_reader :search_result, :linter_url, :css_selector
+
+    def initialize(options)
+      @search_result = options[:search_result]
+      @linter_url    = options[:linter_url]
+      @css_selector  = options[:css_selector]
+
+      raise Exception.new("Lint: search_result is required") if @search_result.nil?
+      raise Exception.new("Lint: search_result must be a LintSearch::SearchResult") unless @search_result.is_a?(LintSearch::SearchResult)
+      raise Exception.new("Lint: linter_url is required") if @linter_url.nil?
+      raise Exception.new("Lint: css_selector is required") if @css_selector.nil?
+    end
+
+    def lint
+      robot = Mechanize.new
+      robot.user_agent_alias = 'Linux Firefox'
+
+      robot.get(@linter_url) do |page|
+        form = page.forms.first
+        form.texts.first.value = @search_result.url
+        results_page = form.submit
+        
+        lint_results = []
+        results_page.search(@css_selector).each do |item|
+          lint_results << item.text
+        end
+        
+        return lint_results
+      end
+    end
+  end
+
+
+end

data/lib/lint_search/search.rb

@@ -0,0 +1,45 @@
+require 'rubygems'
+require 'mechanize'
+
+module LintSearch
+  class Search
+    attr_reader :query_url, :css_selector, :keyword
+                                                                         
+    def initialize(options={})
+      @query_url =    options[:query_url]     
+      @css_selector = options[:css_selector] 
+      @keyword =      options[:keyword]
+      
+      raise Exception.new("Keyword required") if @keyword.nil?
+      raise Exception.new("CSS Selector required") if @css_selector.nil?
+      raise Exception.new("Query url required") if @query_url.nil?
+    end
+  
+    def search
+      robot = Mechanize.new 
+      robot.user_agent_alias = 'Linux Firefox'
+
+      robot.get(@query_url) do |page|
+
+        # Fill out the search form
+        form = page.forms.first
+        form.texts.first.value = @keyword
+
+        # Nice opportunity to use the "I'm feeling lucky!" button for google,
+        # though as this app aims to be search agnostic it's not a good fit 
+        # for other providers
+        search_result = form.click_button(form.buttons.first)
+        first_result = robot.click(search_result.search(@css_selector).first)
+
+
+        # create a SearchResult object
+        return LintSearch::SearchResult.new(url: first_result.uri.to_s, title: first_result.title)    
+      end
+    end
+
+    def query
+      "#{@query_url}#{@keyword}"
+    end
+
+  end
+end

data/lib/lint_search/search_result.rb

@@ -0,0 +1,15 @@
+module LintSearch
+  class SearchResult
+    attr_reader :url, :title
+    attr_writer :url, :title
+
+    def initialize(url: nil, title: nil)
+      @url = url
+      @title = title
+
+      raise Exception.new("SearchResult: URL is required") if @url.nil?
+      raise Exception.new("SearchResult: title is required") if @title.nil?
+    end
+  end
+
+end

data/lib/lint_search/version.rb

@@ -0,0 +1,3 @@
+module LintSearch
+  VERSION = "0.1.0"
+end

data/lib/lint_search.rb

@@ -0,0 +1,7 @@
+require 'lint_search/version'
+require 'lint_search/search'
+require 'lint_search/search_result'
+require 'lint_search/lint'
+
+module LintSearch
+end

data/lint_search.gemspec

@@ -0,0 +1,28 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'lint_search/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "lint_search"
+  spec.version       = LintSearch::VERSION
+  spec.authors       = ["Dave Russell"]
+  spec.email         = ["dave.kerr@gmail.com"]
+
+  spec.summary       = %q{LintSearch is a an application which searchs for a keyword and then liunts the first results}
+  spec.homepage      = "http://github.com/OkayDave/lint_search"
+
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.10"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec"
+
+
+  spec.add_runtime_dependency "thor", "~> 0.19"
+  spec.add_runtime_dependency "mechanize", "~> 2.7"
+end

metadata

@@ -0,0 +1,130 @@
+--- !ruby/object:Gem::Specification
+name: lint_search
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Dave Russell
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2015-08-23 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.10'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: thor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.19'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.19'
+- !ruby/object:Gem::Dependency
+  name: mechanize
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.7'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.7'
+description: 
+email:
+- dave.kerr@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- Gemfile
+- README.md
+- Rakefile
+- bin/console
+- bin/lint_search
+- bin/setup
+- lib/lint_search.rb
+- lib/lint_search/lint.rb
+- lib/lint_search/search.rb
+- lib/lint_search/search_result.rb
+- lib/lint_search/version.rb
+- lint_search.gemspec
+homepage: http://github.com/OkayDave/lint_search
+licenses: []
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.2.2
+signing_key: 
+specification_version: 4
+summary: LintSearch is a an application which searchs for a keyword and then liunts
+  the first results
+test_files: []
+has_rdoc: