gh-trending 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 808ee3a7194a4284ee546f7ab167c69ac24439e7
4
+ data.tar.gz: 0422c8cea905cce431bffcf2ee90df020667725f
5
+ SHA512:
6
+ metadata.gz: d9b0d3524ea69290beeef5ec6125ae3b63887e1b4f6089af804f8013f023adf0829a9f2ab22b41f6c07a11e67dcc14cd2880e3a9a2c5bef36c542d54f91d65ca
7
+ data.tar.gz: 8f7e52cc4977c77222b65fed750cefb0344716c9c3105dedd66481a4ba210bcb3a7bd504e46630f543472ed8721d4392f3e6cc944bea53f0542b2c472c523d08
data/.gitignore ADDED
@@ -0,0 +1,10 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /Gemfile.lock
4
+ /_yardoc/
5
+ /coverage/
6
+ /doc/
7
+ /pkg/
8
+ /spec/reports/
9
+ /tmp/
10
+ *.gem
data/.rspec ADDED
@@ -0,0 +1,2 @@
1
+ --color
2
+ --require spec_helper
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at luispcosta18@gmail.com. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,6 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in gh-trending.gemspec
4
+ gemspec
5
+
6
+
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2017 Luís Costa
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2017 luispcosta
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,114 @@
1
+ # Gh::Trending
2
+
3
+ A simple gem to parse trending repositories and developers from the github's [trending page](https://github.com/trending).
4
+
5
+ ## Installation
6
+
7
+ Add this line to your application's Gemfile:
8
+
9
+ ```ruby
10
+ gem 'gh-trending'
11
+ ```
12
+
13
+ And then execute:
14
+
15
+ $ bundle
16
+
17
+ Or install it yourself as:
18
+
19
+ $ gem install gh-trending
20
+
21
+ ## Usage
22
+
23
+ There are two ways of using this gem:
24
+
25
+ * To fetch trending repositories:
26
+
27
+ `Gh::Trending::Repositories.new.get(language:, time_period:)`
28
+
29
+ * To fetch trending developers:
30
+
31
+ `Gh::Trending::Developers.new.get(language:, time_period:)`
32
+
33
+ The arguments `language` and `time_period` are provided in hash syntax, and they
34
+ are **not** required. Below there are simplified examples
35
+
36
+ Accepted values for the `time_period` argument (as of **Jun 2017**):
37
+
38
+ * `:daily` (default) => Will return the trending developers/repositories in the last day
39
+ * `:weekly` => Will return the trending developers/repositories in the last week
40
+ * `:monthly` => Will return the trending developers/repositories in the last month.
41
+
42
+ * If you don't provide any value for the `time_period` argument, the value `:daily` is used.
43
+ * If you don't provide any value for the `language` argument, it uses the default github's trending page option, which considers **all languages**.
44
+ * If you provide an unknown `time_period` argument, an error is raised
45
+ * If you don't provide any value for both arguments, this returns the trending developers/repositories for **all languages**, in the **last day**.
46
+
47
+ ### Examples
48
+
49
+ #### Trending developers
50
+
51
+ ```ruby
52
+ crawler = Gh::Trending::Developers.new
53
+ crawler.get
54
+ # => [
55
+ # <Gh::Trending::Developer:0x00000002f87760
56
+ # @full_name="Dawson Corwin",
57
+ # @hot_repository_url="http://denesiklehner.org/josianne",
58
+ # @user_profile_url="http://hane.co/camren"
59
+ # ,
60
+ # ...
61
+ # ]
62
+ ```
63
+
64
+ * `full_name` => The full name of the developer
65
+ * `hot_repository_url` => The name of the repository that is trending for this developer, in the provided `time_period` and `language`.
66
+ * In the example above, since no `language` and `time_period` were provided, the repository `"http://denesiklehner.org/josianne"` is the trending repository of this user, since the last day for any language.
67
+ * `user_profile_url` => The github user profile url.
68
+
69
+ #### Trending repositories
70
+
71
+ ```ruby
72
+ crawler = Gh::Trending::Repositories.new
73
+ crawler.get language: 'javascript'
74
+ # => [#<Gh::Trending::Repository:0x000000035231b0
75
+ # @description="Unicode's answer to Base64",
76
+ # @language="JavaScript",
77
+ # @name="/qntm/base65536",
78
+ # @period_stars=104,
79
+ # @url="https://github.com/qntm/base65536">,
80
+ # #<Gh::Trending::Repository:0x00000003511d70
81
+ # @description="Prettier is an opinionated JavaScript formatter.",
82
+ # @language="JavaScript",
83
+ # @name="/prettier/prettier",
84
+ # @period_stars=100,
85
+ # @url="https://github.com/prettier/prettier">,
86
+ # ]
87
+ ```
88
+ * `description` => The description of the repository
89
+ * `language` => Primary programming language of the repository (should match the one provided in the `language` argument, if provided)
90
+ * `name` => The name of the repository
91
+ * `url` => The github's URL of the repository
92
+ * `period_stars` => The number of stars of the repository in the requested `time_period`.
93
+
94
+ ## Development
95
+
96
+ After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
97
+
98
+ ## Contributing
99
+
100
+ You can run `bin/dev` to play around with the tool. You should run this command every time you make a change in the gem to test your modifications.
101
+
102
+ * If for some reason you can't run the `bin/dev` script, make sure the script has executions rights (`chmod +x bin/dev`).
103
+
104
+ Bug reports and pull requests are welcome on GitHub at https://github.com/luispcosta/gh-trending. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
105
+
106
+ ## Tests
107
+
108
+ To run the test suite:
109
+
110
+ `rake spec`
111
+
112
+ ## License
113
+
114
+ The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require 'rspec/core/rake_task'
3
+
4
+ task :default => :spec
5
+
6
+ RSpec::Core::RakeTask.new(:spec)
data/bin/console ADDED
@@ -0,0 +1,13 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "gh_trending"
5
+ require 'pry'
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ pry
data/bin/dev ADDED
@@ -0,0 +1,4 @@
1
+ #!/usr/bin/env bash
2
+
3
+ gem build gh-trending.gemspec
4
+ bin/console
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,44 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'gh_trending/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "gh-trending"
8
+ spec.version = Gh::Trending::VERSION
9
+ spec.authors = ["luispcosta"]
10
+ spec.email = ["luispcosta18@gmail.com"]
11
+
12
+ spec.summary = %q{Fetch trending repositories/developers from the github's trending page}
13
+ spec.description = %q{
14
+ Fetches the trending repositories and developers from the github's trending page, by time_period
15
+ and language
16
+ }
17
+ spec.homepage = "https://github.com/luispcosta/gh-trending"
18
+ spec.license = "MIT"
19
+
20
+ # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host'
21
+ # to allow pushing to a single host or delete this section to allow pushing to any host.
22
+ if spec.respond_to?(:metadata)
23
+ spec.metadata['allowed_push_host'] = "https://rubygems.org/"
24
+ else
25
+ raise "RubyGems 2.0 or newer is required to protect against " \
26
+ "public gem pushes."
27
+ end
28
+
29
+ spec.files = `git ls-files -z`.split("\x0").reject do |f|
30
+ f.match(%r{^(test|spec|features)/})
31
+ end
32
+ spec.bindir = "exe"
33
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
34
+ spec.require_paths = ["lib"]
35
+
36
+ spec.add_dependency "mechanize"
37
+
38
+ spec.add_development_dependency "bundler", "~> 1.14"
39
+ spec.add_development_dependency "rake", "~> 10.0"
40
+ spec.add_development_dependency "rspec"
41
+ spec.add_development_dependency "mechanize"
42
+ spec.add_development_dependency "pry-byebug"
43
+ spec.add_development_dependency "faker"
44
+ end
@@ -0,0 +1 @@
1
+ require 'gh_trending'
@@ -0,0 +1,23 @@
1
+ module Gh
2
+ module Trending
3
+ # This class represents a trending developer in the github's trending
4
+ # developers page.
5
+ class Developer
6
+ attr_reader :full_name, :user_profile_url, :hot_repository_url
7
+
8
+ def initialize(full_name: nil, user_profile_url: nil, hot_repository_url: nil)
9
+ @full_name = full_name || ''
10
+ @user_profile_url = user_profile_url || ''
11
+ @hot_repository_url = hot_repository_url || ''
12
+ end
13
+
14
+ def to_h
15
+ {
16
+ full_name: full_name,
17
+ user_profile_url: user_profile_url,
18
+ hot_repository_url: hot_repository_url
19
+ }
20
+ end
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,36 @@
1
+ require 'gh_trending/parsers/parser'
2
+ require 'gh_trending/parsers/developers_parser'
3
+ require 'gh_trending/parsers/parser_arguments'
4
+ require 'gh_trending/developer'
5
+
6
+ module Gh
7
+ module Trending
8
+ class Developers
9
+ attr_reader :developers_list
10
+
11
+ def initialize(parser: nil)
12
+ @parser = parser || Gh::Trending::Parsers::DevelopersParser.new
13
+ @developers_list = []
14
+ end
15
+
16
+ def get(*args)
17
+ parse_args = Gh::Trending::Parsers::ParserArguments.new(*args)
18
+ developers_list.clear
19
+ developers_list << parser.parse(parse_args).to_a.compact
20
+ developers_list.flatten!
21
+ end
22
+
23
+ def [](idx)
24
+ developers_list[idx]
25
+ end
26
+
27
+ def to_h
28
+ developers_list.map(&:to_h)
29
+ end
30
+
31
+ private
32
+
33
+ attr_reader :parser
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,69 @@
1
+ module Gh
2
+ module Trending
3
+ module Parsers
4
+ # Parses the trending developers page
5
+ class DevelopersParser < Parser
6
+ LIST_WITH_DEVS_NODES_HTML_CLASS = 'ol.leaderboard-list'.freeze
7
+ H2_WITH_DEVELOPER_NAME = 'h2.user-leaderboard-list-name'.freeze
8
+ ANCHOR_WITH_HOT_REPOSITORY = 'a.repo-snipit'.freeze
9
+ SPAN_WITH_FULL_NAME = 'span.full-name'.freeze
10
+ REGEXP_FOR_FULLNAME = /\((.+)\)/.freeze
11
+
12
+ def parse(parser_args)
13
+ url = build_request_url_with(parser_args)
14
+ page = agent.get(url)
15
+ parsed_page = page.at(LIST_WITH_DEVS_NODES_HTML_CLASS)
16
+
17
+ raise Gh::Trending::Parsers::HtmlChangedError.new(:front_page_with_devs) if parsed_page.nil?
18
+ devs = parsed_page.search('li')
19
+ build_developers_from_node devs
20
+ end
21
+
22
+ def resource_url
23
+ 'https://github.com/trending/developers'.freeze
24
+ end
25
+
26
+ private
27
+
28
+ def build_developers_from_node developers_node
29
+ developers_node.map { |dev_node| Developer.new extract_developer_info dev_node }
30
+ end
31
+
32
+ def extract_developer_info developer_node
33
+ {
34
+ full_name: get_full_name(developer_node),
35
+ hot_repository_url: get_hot_repository_url(developer_node),
36
+ user_profile_url: get_profile_url(developer_node)
37
+ }
38
+ end
39
+
40
+ def get_profile_url dev_node
41
+ parsed_url = dev_node.search(H2_WITH_DEVELOPER_NAME)
42
+ .children
43
+ .search('a')[0]
44
+ .attributes['href']
45
+ .value
46
+ build_github_url(parsed_url)
47
+ end
48
+
49
+ def get_hot_repository_url dev_node
50
+ parsed_url = dev_node.search(ANCHOR_WITH_HOT_REPOSITORY)[0].attributes['href'].value
51
+ build_github_url(parsed_url)
52
+ end
53
+
54
+ def get_full_name dev_node
55
+ span_with_full_name = dev_node.search(SPAN_WITH_FULL_NAME)
56
+ return nil if span_with_full_name.empty?
57
+
58
+ match = span_with_full_name
59
+ .text
60
+ .strip
61
+ .match REGEXP_FOR_FULLNAME
62
+
63
+ full_name_without_enclosing_parens = match[1]
64
+ full_name_without_enclosing_parens
65
+ end
66
+ end
67
+ end
68
+ end
69
+ end
@@ -0,0 +1,34 @@
1
+ module Gh
2
+ module Trending
3
+ module Parsers
4
+ # Class indicating that perhaps the HTML of the trending has changed
5
+ # for whatever reason.
6
+ #
7
+ #
8
+ class HtmlChangedError < StandardError
9
+ attr_reader :page_that_changed
10
+
11
+ def initialize(page_that_changed, msg = nil)
12
+ @page_that_changed = page_that_changed
13
+ page_changed_human_format = build_human_format_page_changed
14
+
15
+ msg ||= "The page can't be crawled anymore because #{page_changed_human_format} changed its HTML format. Submit a PR to fix it!"
16
+ super(msg)
17
+ end
18
+
19
+ private
20
+
21
+ def build_human_format_page_changed
22
+ case page_that_changed
23
+ when :front_page_with_repos
24
+ "the front page that lists all trending repositories"
25
+ when :front_page_with_devs
26
+ "the front page that lists all trending developers"
27
+ else
28
+ raise ArgumentError, "Unknown page_that_changed symbol: #{page_that_changed}"
29
+ end
30
+ end
31
+ end
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,49 @@
1
+ require 'mechanize'
2
+
3
+ module Gh
4
+ module Trending
5
+ module Parsers
6
+ class Parser
7
+ attr_reader :agent
8
+
9
+ def initialize(agent = nil)
10
+ @agent = agent || default_agent
11
+ end
12
+
13
+ def parse
14
+ raise NotImplementedError
15
+ end
16
+
17
+ protected
18
+
19
+ # Prepends the github's website URL for a given parsed url.
20
+ # url can be a repository url or a developer url.
21
+ #
22
+ # url has the format: /some/url, because that's the format returned to
23
+ # us by the crawler
24
+ def build_github_url(url)
25
+ "https://github.com#{url}"
26
+ end
27
+
28
+ def build_request_url_with(parser_args)
29
+ return resource_url if parser_args.no_args?
30
+
31
+ url = resource_url
32
+ url += "/#{parser_args.language}" if parser_args.language
33
+ url += "?since=#{parser_args.time_period}" if parser_args.time_period
34
+ url
35
+ end
36
+
37
+ def resource_url
38
+ raise NotImplementedError
39
+ end
40
+
41
+ private
42
+
43
+ def default_agent
44
+ Mechanize.new
45
+ end
46
+ end
47
+ end
48
+ end
49
+ end
@@ -0,0 +1,52 @@
1
+ module Gh
2
+ module Trending
3
+ module Parsers
4
+ # This class represents the arguments that both the trending
5
+ # repositories and languages repositories accept.
6
+ #
7
+ # If these pages accept new arguments, just add a new method here
8
+ # like:
9
+ # def new_arg
10
+ # args[:arg_name]
11
+ # end
12
+ #
13
+ # Also, complete the no_args? method.
14
+ class ParserArguments
15
+ attr_reader :args
16
+ UnknownTimePeriod = Class.new(StandardError)
17
+
18
+ TIME_PERIOD_ARG = :time_period
19
+ LANGUAGE_ARG = :language
20
+
21
+ KNOWN_TIME_PERIODS = %i(daily weekly monthly)
22
+ DEFAULT_TIME_PERIOD = KNOWN_TIME_PERIODS.first
23
+
24
+ def initialize(**args)
25
+ @args = args
26
+ @args = {} if @args.empty?
27
+ end
28
+
29
+ def time_period
30
+ sanitize_time_period(args[TIME_PERIOD_ARG])
31
+ end
32
+
33
+ def language
34
+ args[LANGUAGE_ARG]
35
+ end
36
+
37
+ def no_args?
38
+ time_period.nil? && language.nil?
39
+ end
40
+
41
+ private
42
+
43
+ def sanitize_time_period(time_period)
44
+ return nil if time_period.nil?
45
+ raise UnknownTimePeriod.new("The time period '#{time_period}' is invalid. Supported: #{KNOWN_TIME_PERIODS}") unless KNOWN_TIME_PERIODS.include? time_period
46
+
47
+ time_period
48
+ end
49
+ end
50
+ end
51
+ end
52
+ end
@@ -0,0 +1,74 @@
1
+ module Gh
2
+ module Trending
3
+ module Parsers
4
+ # Parses the trending repositories page
5
+ class ReposParser < Parser
6
+ LIST_WITH_REPO_NODES_HTML_CLASS = 'ol.repo-list'.freeze
7
+ DIV_WITH_DESCRIPTION = 'div.py-1'.freeze
8
+ SPAN_WITH_LANGUAGE = 'span.mr-3'.freeze
9
+ SPAN_WITH_STARS = 'span.float-right'.freeze
10
+
11
+ def parse(parser_args)
12
+ url = build_request_url_with(parser_args)
13
+ page = agent.get(url)
14
+ parsed_page = page.at(LIST_WITH_REPO_NODES_HTML_CLASS)
15
+
16
+ raise Gh::Trending::Parsers::HtmlChangedError.new(:front_page_with_repos) if parsed_page.nil?
17
+ repositories = parsed_page.search('li')
18
+ build_repositories_from_node(repositories)
19
+ end
20
+
21
+ def resource_url
22
+ 'https://github.com/trending'.freeze
23
+ end
24
+
25
+ private
26
+
27
+ def build_repositories_from_node repositories_node
28
+ repositories_node.map { |repo_node| Repository.new extract_repo_data repo_node }
29
+ end
30
+
31
+ def extract_repo_data repo_node
32
+ {
33
+ name: get_name(repo_node),
34
+ description: get_description(repo_node),
35
+ language: get_language(repo_node),
36
+ period_stars: get_stars(repo_node),
37
+ url: get_repo_url(repo_node)
38
+ }
39
+ end
40
+
41
+ def get_name repo_node
42
+ repo_node.search('h3')[0].children[1].attributes['href'].value
43
+ end
44
+
45
+ def get_description repo_node
46
+ repo_node.search(DIV_WITH_DESCRIPTION).text.strip
47
+ end
48
+
49
+ def get_language repo_node
50
+ repo_node.search(SPAN_WITH_LANGUAGE).text.strip
51
+ end
52
+
53
+ def get_stars repo_node
54
+ string_with_stars = repo_node.search(SPAN_WITH_STARS).text.strip
55
+ # Github shows the string "1,263 stars this week"
56
+ # Lets strip only the number.
57
+ string_with_stars.gsub!(',', '')
58
+
59
+ digits_array = string_with_stars.match(/\d+/)
60
+
61
+ # Since, after replacing the commans with '', there can only possibly
62
+ # be one number in the matched data, so we return the first one
63
+ # and convert it to integer.
64
+ digits_array.to_a[0].to_i
65
+ end
66
+
67
+ def get_repo_url(repo_node)
68
+ parsed_url = repo_node.search('h3').children.search('a')[0].attributes['href'].value
69
+ build_github_url(parsed_url)
70
+ end
71
+ end
72
+ end
73
+ end
74
+ end
@@ -0,0 +1,37 @@
1
+ require 'gh_trending/parsers/parser'
2
+ require 'gh_trending/parsers/repos_parser'
3
+ require 'gh_trending/parsers/parser_arguments'
4
+ require 'gh_trending/repository'
5
+ require 'gh_trending/parsers/html_changed_error'
6
+
7
+ module Gh
8
+ module Trending
9
+ class Repositories
10
+ attr_reader :repos_list
11
+
12
+ def initialize(parser: nil)
13
+ @parser = parser || Gh::Trending::Parsers::ReposParser.new
14
+ @repos_list = []
15
+ end
16
+
17
+ def get(*args)
18
+ parse_args = Gh::Trending::Parsers::ParserArguments.new(*args)
19
+ repos_list.clear
20
+ repos_list << parser.parse(parse_args).to_a.compact
21
+ repos_list.flatten!
22
+ end
23
+
24
+ def [](idx)
25
+ repos_list[idx]
26
+ end
27
+
28
+ def to_h
29
+ repos_list.map(&:to_h)
30
+ end
31
+
32
+ private
33
+
34
+ attr_reader :parser
35
+ end
36
+ end
37
+ end
@@ -0,0 +1,28 @@
1
+ module Gh
2
+ module Trending
3
+ # This class represents a trending repository in the github's trending
4
+ # repositories page.
5
+ class Repository
6
+ attr_reader :name, :description, :language, :period_stars, :url
7
+ # Period stars correspond to the stars that the repository has in the current
8
+ # period being considered, for example: 283 stars THIS WEEK.
9
+ def initialize(name: nil, description: nil, language: nil, period_stars: nil, url: nil)
10
+ @name = name || ''
11
+ @description = description || ''
12
+ @language = language || ''
13
+ @period_stars = period_stars || 0
14
+ @url = url || ''
15
+ end
16
+
17
+ def to_h
18
+ {
19
+ name: name,
20
+ description: description,
21
+ language: language,
22
+ period_stars: period_stars,
23
+ url: url
24
+ }
25
+ end
26
+ end
27
+ end
28
+ end
@@ -0,0 +1,5 @@
1
+ module Gh
2
+ module Trending
3
+ VERSION = "1.0.0"
4
+ end
5
+ end
@@ -0,0 +1,4 @@
1
+ require 'gh_trending/repository'
2
+ require 'gh_trending/repositories'
3
+ require 'gh_trending/developer'
4
+ require 'gh_trending/developers'
metadata ADDED
@@ -0,0 +1,168 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: gh-trending
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.0
5
+ platform: ruby
6
+ authors:
7
+ - luispcosta
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2017-06-04 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: mechanize
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: bundler
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '1.14'
34
+ type: :development
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '1.14'
41
+ - !ruby/object:Gem::Dependency
42
+ name: rake
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '10.0'
48
+ type: :development
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '10.0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: rspec
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: mechanize
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '0'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ - !ruby/object:Gem::Dependency
84
+ name: pry-byebug
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: faker
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - ">="
102
+ - !ruby/object:Gem::Version
103
+ version: '0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - ">="
109
+ - !ruby/object:Gem::Version
110
+ version: '0'
111
+ description: "\n Fetches the trending repositories and developers from the github's
112
+ trending page, by time_period\n and language\n "
113
+ email:
114
+ - luispcosta18@gmail.com
115
+ executables: []
116
+ extensions: []
117
+ extra_rdoc_files: []
118
+ files:
119
+ - ".gitignore"
120
+ - ".rspec"
121
+ - CODE_OF_CONDUCT.md
122
+ - Gemfile
123
+ - LICENSE
124
+ - LICENSE.txt
125
+ - README.md
126
+ - Rakefile
127
+ - bin/console
128
+ - bin/dev
129
+ - bin/setup
130
+ - gh-trending.gemspec
131
+ - lib/gh-trending.rb
132
+ - lib/gh_trending.rb
133
+ - lib/gh_trending/developer.rb
134
+ - lib/gh_trending/developers.rb
135
+ - lib/gh_trending/parsers/developers_parser.rb
136
+ - lib/gh_trending/parsers/html_changed_error.rb
137
+ - lib/gh_trending/parsers/parser.rb
138
+ - lib/gh_trending/parsers/parser_arguments.rb
139
+ - lib/gh_trending/parsers/repos_parser.rb
140
+ - lib/gh_trending/repositories.rb
141
+ - lib/gh_trending/repository.rb
142
+ - lib/gh_trending/version.rb
143
+ homepage: https://github.com/luispcosta/gh-trending
144
+ licenses:
145
+ - MIT
146
+ metadata:
147
+ allowed_push_host: https://rubygems.org/
148
+ post_install_message:
149
+ rdoc_options: []
150
+ require_paths:
151
+ - lib
152
+ required_ruby_version: !ruby/object:Gem::Requirement
153
+ requirements:
154
+ - - ">="
155
+ - !ruby/object:Gem::Version
156
+ version: '0'
157
+ required_rubygems_version: !ruby/object:Gem::Requirement
158
+ requirements:
159
+ - - ">="
160
+ - !ruby/object:Gem::Version
161
+ version: '0'
162
+ requirements: []
163
+ rubyforge_project:
164
+ rubygems_version: 2.6.12
165
+ signing_key:
166
+ specification_version: 4
167
+ summary: Fetch trending repositories/developers from the github's trending page
168
+ test_files: []