whatsa 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 36188353c2bbfb6f9d0c1f6a477a5fa9fe600543
+  data.tar.gz: 06d72756376c69c65a3a9e0c7f45cd0864428995
+SHA512:
+  metadata.gz: 2df002badfb325346250f2237aa990a151b82b7d0e36534dd2e1337b32542892e22924dcfd2f35e715721804c5eaebb0234303cf5260d6e4645c37a95df1c85e
+  data.tar.gz: cbca63bef785503e51ff4e350960eacdcd9582e1fdba8c6c0005d0f3f2ade18595037515ddab29e8482ee7586dd20dfbb79e62aba25f358568c6b95207539e7d

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in whatsa.gemspec
+gemspec

data/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2016 Keegan Leitz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,41 @@
+# Whatsa
+
+## The basics
+
+### What's a Whatsa?
+
+Whatsa is a CLI app that allows you to search a word or phrase and receive a quick summary about that subject.
+
+### How does it do that?
+
+It searches your query on Wikipedia and finds the page you're most likely looking for. If you've been somewhat vague, or your search term could refer to multiple things, it will ask you to select from a disambiguation of topics. Usually, however, your term will go straight to an article. Whatsa then gives you the first paragraph of that article (usually a surprisingly decent summary).
+
+If you're not super satisfied with that bit of information (and you need to know a little more) you can type `more` to get a better picture of that subject. If you're _still_ not satisfied, and you want to know something _specific_ about the thing you've searched, type `other`, and it will list the categories of information Wikipedia knows about the subject (its "History", "Early Life", "Uses", etc.). You can select one of those sections, if you'd like, and it will give you the first paragraph of that section, too (which you can extend similarly with another `more` command). You can make a new query with `new`, ask for help at any time with `help`, or exit the application with `exit`.
+
+Simple?
+
+Simple!
+
+## Using Whatsa
+
+Clone this repo, `cd` to the directory, run `bundle install` (assuming you've already installed bundler), then run the `bin/Whatsa` script in a terminal. It will ask you what you'd like to learn about, and you can enter a word or phrase.
+
+If your search term could mean multiple things, it will let you know, and you can select a choice by its name or number.
+
+If you would like a longer explanation, type `more`.
+
+If you would like to select a category of information about the topic, type `other`, then select a choice by its name or number.
+
+If you'd like to search for something else, type `new`.
+
+## Dependencies
+
+Requires Ruby 2.3.3 (doesn't seem to work with 2.0, at least, so see what you can get away with), the `nokogiri` gem, bundler (if you're planning on installing it the easy way), and `pry` (although it's really only in the gemspec for dev purposes).
+
+## Contributing
+
+Bug reports and pull requests for this project are welcome at its [GitHub page](https://github.com/kjleitz/Whatsa). If you choose to contribute, please adhere to the [Ruby Community Conduct Guideline](https://www.ruby-lang.org/en/conduct/) so I don't have to go around breaking necks, running out of bubblegum, etc.
+
+## License
+
+This project is open source, under the terms of the [MIT license](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,2 @@
+require "bundler/gem_tasks"
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "whatsa"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/bin/whatsa

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require_relative '../lib/whatsa'
+
+Whatsa::CLI.new.run

data/config/environment.rb

@@ -0,0 +1,10 @@
+require 'open-uri'
+require 'nokogiri'
+# require 'pry'
+
+require_relative '../lib/whatsa/version'
+require_relative '../lib/whatsa/cli'
+require_relative '../lib/whatsa/article'
+require_relative '../lib/whatsa/section'
+require_relative '../lib/whatsa/disambig'
+require_relative '../lib/whatsa/scraper'

data/lib/whatsa/article.rb

@@ -0,0 +1,66 @@
+class Whatsa::Article
+  attr_accessor :sections
+  attr_reader :contents, :title
+
+  def initialize(noko_doc)
+    @title = noko_doc.css('h1').text
+    @contents = noko_doc.css('#mw-content-text').children
+    @sections = make_sections
+  end
+
+  def summary
+    self.sections.first.summary
+  end
+
+  def full_text
+    # name might be a little confusing: it's not really the "full text" of the
+    # article, it's the full text of the article summary. I'm naming it #full_text
+    # for duck-typing reasons
+    self.sections.first.full_text
+  end
+
+  def section_titles
+    self.sections.map { |s| s.title }
+  end
+
+  def choose_section(choice)
+    if choice.to_i > 0
+      self.sections[choice.to_i - 1]
+    else
+      get_section_by_title(title)
+    end
+  end
+
+  private
+
+  def get_section_by_title(title)
+    self.sections.find { |s| s.title.downcase == title.downcase }
+  end
+
+  def intro_pars
+    breakpoint = self.contents.to_a.index { |element| element.name == 'h2' }
+    pars = self.contents[0...breakpoint].css('p').map { |par| par.text }
+    pars.reject { |par| par == "" }
+  end
+
+  def make_sections
+    indices = section_indices
+    indices << -1
+    secs = [Whatsa::Section.new("#{self.title} - Introduction", intro_pars)]
+    indices.each_cons(2) do |i, j|
+      title = self.contents[i].text.gsub('[edit]', '').strip
+      par_nodes = self.contents[i...j].select { |e| e.name == 'p' && e.text != "" }
+      pars = par_nodes.map { |par| par.text }
+      secs << Whatsa::Section.new(title, pars).tap { |s| s.article = self }
+    end
+    secs
+  end
+
+  def section_indices
+    indices = []
+    self.contents.each_with_index do |element, i|
+      indices << i if element.name == 'h2'
+    end
+    indices
+  end
+end

data/lib/whatsa/cli.rb

@@ -0,0 +1,152 @@
+class Whatsa::CLI
+
+  def welcome
+    system('clear')
+    puts "Whatsa is a quick-and-dirty lookup utility, powered by Wikipedia!"
+    puts "-----------------------------------------------------------------"
+  end
+
+  def instructions
+    puts "Enter a word (or phrase) to get a brief summary of that subject."
+    puts "If you aren't satisfied, type 'more' to get a slightly longer"
+    puts "description, or type 'other' to get a list of specific categories"
+    puts "regarding that subject, which you can choose by number or name."
+    puts "You can type 'exit' to close the program (or 'help' to receive"
+    puts "these instructions again) at any time!"
+  end
+
+  def ask
+    puts "What would you like to know about?"
+    gets_command
+  end
+
+  def gets_command
+    input = nil
+    loop do
+      print "> "
+      input = gets.strip
+      case input
+      when ""
+        puts "Please enter a valid input."
+      when "exit"
+        exit
+      when "help"
+        instructions
+      when "new"
+        run
+      else
+        break
+      end
+    end
+    input
+  end
+
+  def word_wrap(text)
+    count = 0
+    words = text.split(/ /)
+    words.each_with_index do |word, index|
+      count += word.length + 1
+      if count > 80
+        words.insert(index, "\n")
+        count = 0
+      elsif word.index("\n")
+        count = word.length
+      end
+    end
+    words.join(" ").gsub(/^ /, "")
+  end
+
+  def display_dmb(dmb)
+    raise TypeError unless dmb.is_a?(Whatsa::Disambig)
+    system("clear")
+    stripped_title = dmb.title.gsub("(disambiguation)", "").strip
+    puts "Hmmm... #{stripped_title} could mean a few different things:\n"
+    dmb.descriptions.each_with_index do |kvp, i|
+      puts "#{i + 1}. #{kvp[0].to_s} (#{kvp[1]})"
+    end
+    puts "\nPlease select a choice, either by name or number."
+  end
+
+  def display_sections(text)
+    text = text.article if text.is_a?(Whatsa::Section)
+    system("clear")
+    puts "Here are some specific subjects about '#{text.title}':\n"
+    text.section_titles.each_with_index {|title, i| puts "#{i + 1}. #{title}"}
+    puts "\nPlease select a choice, either by name or number."
+  end
+
+  def summary_helpline
+    puts "   _______________________________________________________________"
+    puts "   (type 'more' for a potentially longer summary, 'other' if you'd"
+    puts "   like to select a specific category of information on the topic,"
+    puts "             or 'new' to find out about something else)"
+  end
+
+  def full_text_helpline
+    puts "   _______________________________________________________________"
+    puts "    (type 'other' if you'd like to select a specific category of"
+    puts " information on the topic, or 'new' to find out about something else)"
+  end
+
+  def get_dmb_choice(disambig)
+    display_dmb(disambig)
+    choice = nil
+    loop do
+      choice = gets_command
+      in_choices = disambig.choices.detect { |c| c.downcase == choice.downcase }
+      break if in_choices || choice.to_i > 0
+    end
+    disambig.choose_article(choice)
+  end
+
+  def summarize(text)
+    system("clear")
+    puts word_wrap(text.summary)
+    summary_helpline
+    input = gets_command
+    input.downcase == "more" ? full(text) : input
+  end
+
+  def full(text)
+    system("clear")
+    puts word_wrap(text.full_text)
+    full_text_helpline
+    gets_command
+  end
+
+  def categories(article)
+    display_sections(article)
+    choice = gets_command
+    section = article.choose_section(choice)
+    summarize(section)
+  end
+
+  def run
+    welcome
+    instructions
+
+    loop do
+      # get a search term
+      input = ask
+      scraper = Whatsa::Scraper.new(input)
+
+      # get an article from the search, or restart the loop if it can't be found
+      if scraper.not_found?
+        puts "Hmmm... I don't know what '#{input}' means! Try something else."
+        next
+      elsif scraper.disambig?
+        article = get_dmb_choice(scraper.make_disambig)
+      else
+        article = scraper.make_article
+      end
+
+      # summarize that article
+      input = summarize(article)
+
+      # the only valid input here that would go uncaught is "other", so
+      # keep asking until you get a caught input (logic determined by
+      # #gets_command, e.g. "help", "exit", "new") or "other"
+      loop { input = input.downcase == "other" ? categories(article) : gets_command }
+    end
+  end
+end

data/lib/whatsa/disambig.rb

@@ -0,0 +1,42 @@
+class Whatsa::Disambig
+  attr_accessor :descriptions
+  attr_reader :title, :items
+
+  def initialize(noko_doc)
+    @title = noko_doc.css('h1').text
+    @items = noko_doc.css('#mw-content-text li')
+    @descriptions = make_descriptions
+  end
+
+  def choices
+    self.descriptions.keys
+  end
+
+  def choose_article(choice)
+    if choice.to_i > 0
+      Whatsa::Scraper.new(choices[choice.to_i - 1]).make_article
+    else
+      Whatsa::Scraper.new(choice).make_article
+    end
+  end
+
+  private
+
+  # make a hash with links and their descriptions
+  def make_descriptions
+    self.items.inject({}) do |memo, item|
+      unless item.css('a').empty?
+        key = item.css('a').first.text
+        toc_link = item["class"] && item["class"].match(/toc/)
+        dmb_link = key.match("(disambiguation)")
+        all_link = key.match("All pages with titles")
+        unless toc_link || dmb_link || all_link
+          desc = item.text.split("\n").first.strip
+          memo[key] = desc.gsub(key, "").gsub(/\A[^\w"]/, "").strip
+        end
+      end
+      memo
+    end
+  end
+
+end

data/lib/whatsa/scraper.rb

@@ -0,0 +1,50 @@
+class Whatsa::Scraper
+
+  WIKISEARCH = 'https://en.wikipedia.org/w/index.php?search='
+
+  attr_reader :query, :page
+
+  def initialize(term)
+    # only keep word chars and parens, turn everything between each 'word'
+    # to a single '+' and remove '+'s at the beginning and end if they're there
+    # @query = term.gsub(/\W+/, '+').gsub(/(\A\+|\+\z)/, '')
+    @query = term.gsub(/[^A-z0-9\(\)]+/, '+').gsub(/(\A\+|\+\z)/, '')
+
+    # store the page in an instance variable so we don't keep polling the site
+    @page = Nokogiri::HTML(open(WIKISEARCH + self.query))
+  end
+
+  def results_page?
+    !self.page.css('.searchresults').empty?
+  end
+
+  def not_found?
+    !self.page.css('.mw-search-nonefound').empty?
+  end
+
+  def article?
+    !self.page.css('#ca-nstab-main').empty? && !disambig?
+  end
+
+  def disambig?
+    !self.page.css('#disambigbox').empty?
+  end
+
+  def make_article
+    if article?
+      Whatsa::Article.new(self.page)
+    elsif results_page? && !not_found?
+      first_title = self.page.css('.mw-search-results li a').first.text
+      self.class.new(first_title).make_article
+    elsif disambig?
+      self.class.new(make_disambig.choices.first).make_article
+    else
+      nil
+    end
+  end
+
+  def make_disambig
+    disambig? ? Whatsa::Disambig.new(self.page) : nil
+  end
+
+end

data/lib/whatsa/section.rb

@@ -0,0 +1,24 @@
+class Whatsa::Section
+  attr_accessor :title, :paragraphs, :article
+
+  def initialize(title, paragraphs)
+    @title = title
+    @paragraphs = paragraphs
+    remove_citations
+  end
+
+  def summary
+    self.paragraphs.first
+  end
+
+  def full_text
+    self.paragraphs.join("\n\n")
+  end
+
+  private
+
+  def remove_citations
+    self.paragraphs.map! { |par| par.gsub(/\[\d+\]/, "") }
+  end
+
+end

data/lib/whatsa/version.rb

@@ -0,0 +1,3 @@
+module Whatsa
+  VERSION = "0.1.0"
+end

data/lib/whatsa.rb

@@ -0,0 +1,4 @@
+require_relative '../config/environment'
+
+module Whatsa
+end

data/whatsa.gemspec

@@ -0,0 +1,38 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'whatsa/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "whatsa"
+  spec.version       = Whatsa::VERSION
+  spec.authors       = ["Keegan Leitz"]
+  spec.email         = ["kjleitz@gmail.com"]
+
+  spec.summary       = %q{What is that? Okay, okay, I don't need a lecture... just gimme the short 'n sweet!}
+  spec.description   = %q{Whatsa is a CLI app that allows you to search a term or phrase and receive a quick summary about that subject. It searches your term on Wikipedia and finds the page you're most likely looking for. Whatsa then gives you the first paragraph of that article, then lists the sections on the page ("History", "Early Life", "Uses", etc.). You can select one of those sections, if you'd like, and it will give you the first paragraph of that section, too. If that's not enough for you, you can ask for more, and it will show you the full section.}
+  spec.homepage      = "https://github.com/kjleitz/whatsa"
+  spec.license       = "MIT"
+
+  # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
+  # to allow pushing to a single host or delete this section to allow pushing to any host.
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = "https://rubygems.org"
+  else
+    raise "RubyGems 2.0 or newer is required to protect against " \
+      "public gem pushes."
+  end
+
+  spec.files         = `git ls-files -z`.split("\x0").reject do |f|
+    f.match(%r{^(test|spec|features)/})
+  end
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.13"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "nokogiri"
+  # spec.add_development_dependency "pry"
+
+end

metadata

@@ -0,0 +1,111 @@
+--- !ruby/object:Gem::Specification
+name: whatsa
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Keegan Leitz
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2016-11-28 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.13'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: nokogiri
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Whatsa is a CLI app that allows you to search a term or phrase and receive
+  a quick summary about that subject. It searches your term on Wikipedia and finds
+  the page you're most likely looking for. Whatsa then gives you the first paragraph
+  of that article, then lists the sections on the page ("History", "Early Life", "Uses",
+  etc.). You can select one of those sections, if you'd like, and it will give you
+  the first paragraph of that section, too. If that's not enough for you, you can
+  ask for more, and it will show you the full section.
+email:
+- kjleitz@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- Gemfile
+- LICENSE.md
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- bin/whatsa
+- config/environment.rb
+- lib/whatsa.rb
+- lib/whatsa/article.rb
+- lib/whatsa/cli.rb
+- lib/whatsa/disambig.rb
+- lib/whatsa/scraper.rb
+- lib/whatsa/section.rb
+- lib/whatsa/version.rb
+- whatsa.gemspec
+homepage: https://github.com/kjleitz/whatsa
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.8
+signing_key: 
+specification_version: 4
+summary: What is that? Okay, okay, I don't need a lecture... just gimme the short
+  'n sweet!
+test_files: []