reddit_comments 0.1.0 → 0.1.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: a1d9128a40448839ed6e8546dd53f4d9a3132b99
-  data.tar.gz: df599032929e9253001f94734467d626d8336c4a
+  metadata.gz: af9696f7babb523a57b770e3ba76655616a50e5d
+  data.tar.gz: 35cd8331ae4c2e95efcb63f478d9b24ef1569d0a
 SHA512:
-  metadata.gz: 775b5fd31757a1b4a7f0cab3e2c68f44838da9ede7b6e3b4ae79b4ae920e0040972d1c408a2919c9cc1493763144e4f719fe85f1428afd7a7b7fcb09360c0274
-  data.tar.gz: a0ab838976b63e4866a6b9bdec988813be30eb4f9c555819bd9d0453c54160462614cf5c3a1e8affee2d78c3beaa178b79c6426ddb2d29fffbf413645552ea7d
+  metadata.gz: ef79c8adbeae9aeb0b988034550ae88bb94c9037171a33406d69da24e577ef9789b9b89967b54baec414a4c19d21d976033afa6ed85db7695cf326a92929da41
+  data.tar.gz: a86939a30cd29203f123e0d44832ac7a750d92b51e96180ca75f4426ee25b23be1a52d58b5e178ca2e1ccb4cf1e6c1ed337456fea506670a7005593291f1d379

data/.gitignore

@@ -0,0 +1,10 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+/*.gem/

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.0
+before_install: gem install bundler -v 1.12.5

data/Gemfile

@@ -0,0 +1,6 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in reddit_comments.gemspec
+gemspec
+
+gem 'pry'

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Esther Leytush
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,79 @@
+# RedditComments
+
+Use this gem to pull in Reddit comments from a given Reddit page.
+The comments will appear as an array of hashes, where each hash has the following:
+- comment id
+- comment author
+- comment text
+- parent id
+
+The parent id refers to the thing that the comment was made on: so if it was a comment on the original post, the id will refer
+to that, otherwise it will refer to the comment on which THIS comment was made.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'reddit_comments'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install reddit_comments
+
+## Usage
+
+Get the URL of the page from which you would like to retrieve comments. The URL should look something like this:
+
+```ruby
+  url = "https://www.reddit.com/r/worldnews/comments/4ozup2/cracks_emerge_in_the_european_consensus_on_russia/"
+```
+
+(The gem will append ```/.json``` to your URL unless you feed it a link with the ```/.json``` already appended.)
+
+```ruby
+RedditComments.retrieve(url)
+```
+
+### Example usage:
+
+```ruby
+require 'reddit_comments'
+
+url = "https://www.reddit.com/r/worldnews/comments/4ozup2/cracks_emerge_in_the_european_consensus_on_russia/"
+
+RedditComments.retrieve(url)
+```
+Result:
+
+```wrap:space
+[
+  {"id"=>"d4gvid3",
+ "parent_id"=>"t3_4ozup2"
+ "author"=>"extremelycynical",
+ "body"=>"As a European (and German, considering the content of the article) I can only say that I find it completely unacceptable that we impose sanctions on Russia. \n\nIt's nothing but the blind application of glaringly obvious and undeniable double standards as well as mind-boggling hypocrisy.\n\nRussia annexing Crimea is a move that by the best estimates can only be called *supported* by the Crimean people and a direct response to escalating US destabilization in the region akin to the destabilization it caused in Syria. Russia saw that the US took away Tartus and the Golan Heights and now wants to take away Sevastopol and put an end to it. And those sanctions come without regarding the wishes of the Crimean people or in any way negotiating with Russia.\n\nIf we sanction Russia why don't we sanction Israel? (A nation that is committing far worse crimes and is annexing more and more land constantly?) \nWhy don't we sanction the US? A warmongering nation that is committing war crimes and is violating the fundamental human rights of any internet user?  \nWhy don't we sanction Saudi Arabia? A country supporting radical Islam and terrorism?\n\nIt is quite undeniable that these sanctions are entirely based on US interests and the anti-Russian agenda pushed for by that country. These sanctions do not contribute to peace. These sanctions don't exist for humanitarian reasons and us caring about human rights. These sanctions don't even exist because anyone actually believes that Russia deserves them. They exist because the West hates Russia. And we need to stop pretending that Russia deserves our sanctions any more than countries like Israel, the US or Saudi Arabia. And we need to face the fact that before we sanction Russia we should first of all sanction those kind of countries."},
+
+ {"id"=>"d4gx7w3",
+  "parent_id"=>"t1_d4gvid3"
+  "author"=>"Esquina1",
+  "body"=>"Good points! I may add that in my view the Crimean case is the correction of a late Soviet-Union-internal mistake which should be solved bilaterally between the sucessors of the Soviet-Union. As for the East Ukrainian secesssion areas, Russia should find a solution to pull out and help with convincing evidence to solve the MH 17 case and indemnify the bereaved."},
+
+  {"id"=>"d4gxtf8",
+   "parent_id"=>"t1_d4gvid3"
+   "author"=>"princefalcon",
+   "body"=>">\nIt is quite undeniable that these sanctions are entirely based on US interests \n>\n\nNot entirely surely? Russia ruled what are now the Ukraine, the Baltic republics, Belarus, Poland right through the 19C as a result of being one of the European great powers and the tendency for the more powerful states to get what they wanted back then. Some of those ethnicities became independent countries after WW1, but got forced into membership of the Warsaw Pact after WW2, so were directly ruled by the USSR or were satellites. There is some doubt about how much Russia respects the principle of national sovereignty. Ideally Russia itself would join the EU so that we could be a happy family of nations, but that doesn't seem likely."}
+  ]
+```
+
+## Contributing
+
+Bug reports, pull requests, and feature requests are welcome. Please fork this repo, pull down to your local machine from your fork, push up your changes, and make a pull request from your fork to this repo.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "reddit_comments"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "pry"
+Pry.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/lib/reddit_comments.rb

@@ -0,0 +1,78 @@
+require "reddit_comments/version"
+require 'net/http'
+require 'open-uri'
+require 'json'
+
+module RedditComments
+  class IncorrectLinkFormat < StandardError; end
+
+  def self.retrieve(link)
+    comments = GetComments.new(link)
+    comments.retrieve
+  end
+
+  class GetComments
+    attr_accessor :url, :comments, :request, :post
+
+    def initialize(url)
+      @url = url
+      @comments = []
+    end
+
+    def recursive_comment_digging(child, comments=[])
+      post = {}
+      post["id"] = child["id"]
+      post["parent_id"] = child["parent_id"]
+      post["author"] = child["author"]
+      post["body"] = child["body"]
+
+      comments << post
+
+      if child["replies"] != nil && child["replies"] != ""
+        child["replies"]["data"]["children"].each do |comment|
+          comments << self.recursive_comment_digging(comment["data"])
+        end
+      end
+      comments
+    end
+
+    def get_comments
+      @post[1]["data"]["children"].each do |child|
+        @comments << recursive_comment_digging(child["data"])
+      end
+      @comments = comments.flatten
+    end
+
+    def parse_post
+      uri = URI.parse(url.dup)
+      http = Net::HTTP.new(uri.host)
+      request = Net::HTTP::Get.new(uri.request_uri)
+      request.initialize_http_header({"User-Agent" => "Reddit_Comments_Gem_0.1_mindplace"})
+      body = http.request(request)
+      @post = JSON.parse(body.body)
+    end
+
+    def append_json
+      unless url.match(/\/.json\z/)
+        if url[-1] == "/"
+          @url = url + ".json"
+        else
+          @url = url + "/.json"
+        end
+      end
+    end
+
+    def link_tested
+      unless url.match(/https:\/\/www.reddit.com\/r\/(.*)\/comments\/(.*)\/(.*)\//)
+        raise IncorrectLinkFormat, "Not an acceptable Reddit link, please see documentation."
+      end
+    end
+
+    def retrieve
+      link_tested
+      append_json
+      parse_post
+      get_comments
+    end
+  end
+end

data/lib/reddit_comments/version.rb

@@ -0,0 +1,3 @@
+module RedditComments
+  VERSION = "0.1.3"
+end

data/reddit_comments.gemspec

@@ -0,0 +1,32 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'reddit_comments/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "reddit_comments"
+  spec.version       = RedditComments::VERSION
+  spec.authors       = ["Esther Leytush"]
+  spec.email         = ["eleytush@gmail.com"]
+
+  spec.summary       = "Grabs all comments from a given Reddit page and delivers as an array of hashes."
+  spec.homepage      = "https://github.com/mindplace/reddit_comments_gem"
+  spec.license       = "MIT"
+
+  # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
+  # to allow pushing to a single host or delete this section to allow pushing to any host.
+  if spec.respond_to?(:metadata)
+    spec.metadata['allowed_push_host'] = 'https://rubygems.org'
+  else
+    raise "RubyGems 2.0 or newer is required to protect against public gem pushes."
+  end
+
+  spec.files         = `git ls-files`.split("\n")
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.12"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "rspec", "~> 3.0"
+end

data/spec/reddit_comments_spec.rb

@@ -0,0 +1,79 @@
+require 'spec_helper'
+require 'pry'
+
+describe RedditComments do
+  it 'has a version number' do
+    expect(RedditComments::VERSION).not_to be nil
+  end
+
+  describe 'link_tested' do
+    it 'raises error if given a non-Reddit link' do
+      url = 'http://www.example.com'
+      reddit_comments = RedditComments::GetComments.new(url)
+
+      expect{ reddit_comments.link_tested }.to raise_error(RedditComments::IncorrectLinkFormat)
+    end
+
+    it 'raises error for Reddit link if not correct format' do
+      url = 'https://www.reddit.com/r/worldnews/comments/were_living_worse_than_in_a_war_venezuelas/'
+      reddit_comments = RedditComments::GetComments.new(url)
+
+      expect{ reddit_comments.link_tested }.to raise_error(RedditComments::IncorrectLinkFormat)
+    end
+
+    it 'passes an acceptabe Reddit link' do
+      url = 'https://www.reddit.com/r/worldnews/comments/4p4orb/were_living_worse_than_in_a_war_venezuelas/'
+      reddit_comments = RedditComments::GetComments.new(url)
+
+      expect{ reddit_comments.link_tested }.not_to raise_error
+    end
+  end
+
+  describe 'append_json' do
+    it 'appends .json to the link if not present' do
+      url = 'https://www.reddit.com/r/worldnews/comments/4p4orb/were_living_worse_than_in_a_war_venezuelas/'
+      updated = url + '.json'
+      reddit_comments = RedditComments::GetComments.new(url)
+      reddit_comments.append_json
+
+      expect(reddit_comments.url).to eq(updated)
+    end
+
+    it 'does not append if present' do
+      url = 'https://www.reddit.com/r/worldnews/comments/4p4orb/were_living_worse_than_in_a_war_venezuelas/.json'
+      reddit_comments = RedditComments::GetComments.new(url)
+      reddit_comments.append_json
+
+      expect(reddit_comments.url).to eq(url)
+    end
+  end
+
+  describe 'parse_post' do
+    it 'takes a url and returns JSON object of url content' do
+      url = 'https://www.reddit.com/r/worldnews/comments/4p4orb/were_living_worse_than_in_a_war_venezuelas/.json'
+      reddit_comments = RedditComments::GetComments.new(url)
+      reddit_comments.parse_post
+
+      expect(reddit_comments.post).to be_an_instance_of(Array)
+      expect(reddit_comments.post[0]).to be_an_instance_of(Hash)
+    end
+  end
+
+  describe 'get_comments' do
+    let(:url) { "https://www.reddit.com/r/worldnews/comments/4p71mu/migrant_protests_calais_france_tried_to_force_way/.json" }
+    let(:comments) { RedditComments::GetComments.new(url).retrieve }
+
+    it 'returns an array of comment hashes' do
+      expect(comments).to be_an_instance_of(Array)
+
+      comments.each do |comment|
+        expect(comment).to be_an_instance_of(Hash)
+      end
+    end
+
+    it 'includes expected comment text' do
+      comment = comments.select{|c| c["body"] == "That moment you realize brexit does nothing to stop this."}
+      expect(comment).not_to be_empty
+    end
+  end
+end

data/spec/spec_helper.rb

@@ -0,0 +1,2 @@
+$LOAD_PATH.unshift File.expand_path('../../lib', __FILE__)
+require 'reddit_comments'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: reddit_comments
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.3
 platform: ruby
 authors:
 - Esther Leytush
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-06-20 00:00:00.000000000 Z
+date: 2016-09-29 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -58,7 +58,21 @@ email:
 executables: []
 extensions: []
 extra_rdoc_files: []
-files: []
+files:
+- ".gitignore"
+- ".rspec"
+- ".travis.yml"
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- lib/reddit_comments.rb
+- lib/reddit_comments/version.rb
+- reddit_comments.gemspec
+- spec/reddit_comments_spec.rb
+- spec/spec_helper.rb
 homepage: https://github.com/mindplace/reddit_comments_gem
 licenses:
 - MIT
@@ -80,7 +94,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.1
+rubygems_version: 2.6.7
 signing_key: 
 specification_version: 4
 summary: Grabs all comments from a given Reddit page and delivers as an array of hashes.