twingly-search 3.0.0 → 4.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.travis.yml +5 -4
- data/CHANGELOG.md +39 -0
- data/README.md +35 -15
- data/Rakefile +21 -3
- data/examples/Gemfile +1 -0
- data/examples/find_all_posts_mentioning_github.rb +20 -5
- data/examples/hello_world.rb +5 -3
- data/lib/twingly/search.rb +2 -1
- data/lib/twingly/search/client.rb +63 -6
- data/lib/twingly/search/error.rb +27 -0
- data/lib/twingly/search/parser.rb +32 -9
- data/lib/twingly/search/post.rb +19 -0
- data/lib/twingly/search/query.rb +33 -23
- data/lib/twingly/search/result.rb +12 -0
- data/lib/twingly/search/version.rb +1 -1
- data/spec/client_spec.rb +99 -12
- data/spec/error_spec.rb +35 -0
- data/spec/fixtures/non_xml_result.xml +3 -0
- data/spec/fixtures/{invalid_result.xml → nonexistent_api_key_result.xml} +0 -0
- data/spec/fixtures/service_unavailable_result.xml +3 -0
- data/spec/fixtures/unauthorized_api_key_result.xml +3 -0
- data/spec/fixtures/undefined_error_result.xml +3 -0
- data/spec/parser_spec.rb +46 -3
- data/spec/query_spec.rb +41 -26
- data/spec/spec_helper.rb +7 -0
- data/twingly-search-api-ruby.gemspec +1 -0
- metadata +31 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dd9953d1e03a6e3b8876f3a4b86580ddcfeeeccf
|
4
|
+
data.tar.gz: 36f8ccb71ca25e1016d00e366652e06dc71b6321
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 525c3009c34290b6a86a6a397c9e907d9defbf7089e82e0786274d0c46146d79d4bf9d69cb673120ebd50ea1f10626a13b94d5e113590095f1c557edbc6380ef
|
7
|
+
data.tar.gz: 3df9d3ddd711c7185005a81c93ce3fb4b362a7a526698a2203e475c2e6234054e054269ebb2d45aedb44bc4c31ea88d083598952debd78af2c63f38dcc680edd
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,44 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
+
## [v4.0.0](https://github.com/twingly/twingly-search-api-ruby/tree/v4.0.0)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/v3.0.0...v4.0.0)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Add JRuby 9000 to supported Ruby versions [\#27](https://github.com/twingly/twingly-search-api-ruby/issues/27)
|
10
|
+
- Rename analytics to search [\#21](https://github.com/twingly/twingly-search-api-ruby/issues/21)
|
11
|
+
|
12
|
+
**Fixed bugs:**
|
13
|
+
|
14
|
+
- Handle non-XML responses from server [\#31](https://github.com/twingly/twingly-search-api-ruby/issues/31)
|
15
|
+
- Changelog is not generated correctly [\#25](https://github.com/twingly/twingly-search-api-ruby/issues/25)
|
16
|
+
|
17
|
+
**Merged pull requests:**
|
18
|
+
|
19
|
+
- Example script: Retry on server error [\#35](https://github.com/twingly/twingly-search-api-ruby/pull/35) ([jage](https://github.com/jage))
|
20
|
+
- Yield self [\#34](https://github.com/twingly/twingly-search-api-ruby/pull/34) ([roback](https://github.com/roback))
|
21
|
+
- Handle non-XML responses from server [\#33](https://github.com/twingly/twingly-search-api-ruby/pull/33) ([roback](https://github.com/roback))
|
22
|
+
- Add YARD documentation comments [\#32](https://github.com/twingly/twingly-search-api-ruby/pull/32) ([roback](https://github.com/roback))
|
23
|
+
- Test with JRuby 9000 on travis [\#30](https://github.com/twingly/twingly-search-api-ruby/pull/30) ([roback](https://github.com/roback))
|
24
|
+
- Handle exceptions [\#29](https://github.com/twingly/twingly-search-api-ruby/pull/29) ([roback](https://github.com/roback))
|
25
|
+
- Let client handle api call [\#28](https://github.com/twingly/twingly-search-api-ruby/pull/28) ([roback](https://github.com/roback))
|
26
|
+
- Cleanup [\#26](https://github.com/twingly/twingly-search-api-ruby/pull/26) ([roback](https://github.com/roback))
|
27
|
+
- Show deprecation warnings for twingly-analytics [\#23](https://github.com/twingly/twingly-search-api-ruby/pull/23) ([roback](https://github.com/roback))
|
28
|
+
|
29
|
+
## [v3.0.0](https://github.com/twingly/twingly-search-api-ruby/tree/v3.0.0) (2015-11-20)
|
30
|
+
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/2.0.1...v3.0.0)
|
31
|
+
|
32
|
+
**Implemented enhancements:**
|
33
|
+
|
34
|
+
- Improve "Development and release" section in README [\#19](https://github.com/twingly/twingly-search-api-ruby/issues/19)
|
35
|
+
|
36
|
+
**Merged pull requests:**
|
37
|
+
|
38
|
+
- Rename analytics to search [\#24](https://github.com/twingly/twingly-search-api-ruby/pull/24) ([roback](https://github.com/roback))
|
39
|
+
- Improve "Development and release" section in README [\#22](https://github.com/twingly/twingly-search-api-ruby/pull/22) ([roback](https://github.com/roback))
|
40
|
+
- Rename analytics to search in readme [\#20](https://github.com/twingly/twingly-search-api-ruby/pull/20) ([roback](https://github.com/roback))
|
41
|
+
|
3
42
|
## [2.0.1](https://github.com/twingly/twingly-search-api-ruby/tree/2.0.1) (2015-09-24)
|
4
43
|
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/2.0.0...2.0.1)
|
5
44
|
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
# Twingly Search API Ruby
|
2
2
|
|
3
|
-
[![Build Status](https://travis-ci.org/twingly/twingly-
|
4
|
-
[![Code Climate](https://codeclimate.com/github/twingly/twingly-
|
3
|
+
[![Build Status](https://travis-ci.org/twingly/twingly-search-api-ruby.png?branch=master)](https://travis-ci.org/twingly/twingly-search-api-ruby)
|
4
|
+
[![Code Climate](https://codeclimate.com/github/twingly/twingly-search-api-ruby/badges/gpa.svg)](https://codeclimate.com/github/twingly/twingly-search-api-ruby)
|
5
5
|
|
6
6
|
A Ruby gem for Twingly's Search API (previously known as Analytics API). Twingly is a blog search service that provides a searchable API known as [Twingly Search API](https://developer.twingly.com/resources/search/).
|
7
7
|
|
@@ -9,25 +9,30 @@ A Ruby gem for Twingly's Search API (previously known as Analytics API). Twingly
|
|
9
9
|
|
10
10
|
Install via RubyGems
|
11
11
|
|
12
|
-
```
|
12
|
+
```shell
|
13
13
|
gem install twingly-search
|
14
14
|
```
|
15
15
|
|
16
16
|
Or add to your application's [Gemfile](http://bundler.io/gemfile.html) and then run `bundle`
|
17
17
|
|
18
|
-
```
|
19
|
-
gem
|
18
|
+
```ruby
|
19
|
+
gem "twingly-search"
|
20
20
|
```
|
21
21
|
|
22
22
|
## Usage
|
23
23
|
|
24
|
-
```
|
25
|
-
require
|
24
|
+
```ruby
|
25
|
+
require "twingly/search"
|
26
|
+
|
27
|
+
client = Twingly::Search::Client.new do |client|
|
28
|
+
client.user_agent = "MyCompany/1.0"
|
29
|
+
end
|
30
|
+
|
31
|
+
query = client.query do |query|
|
32
|
+
query.pattern = "github page-size:10"
|
33
|
+
query.language = "sv"
|
34
|
+
end
|
26
35
|
|
27
|
-
client = Twingly::Search::Client.new
|
28
|
-
query = client.query
|
29
|
-
query.pattern = 'github page-size:10'
|
30
|
-
query.language = 'sv'
|
31
36
|
result = query.execute
|
32
37
|
=> #<Twingly::Search::Result:0x3ff7adcbe3d4 @posts, @number_of_matches_returned=10, @number_of_matches_total=3035221>
|
33
38
|
result.posts # will include all returned posts
|
@@ -35,14 +40,19 @@ result.posts # will include all returned posts
|
|
35
40
|
|
36
41
|
The `twingly-search` gem talks to a commercial blog search API and requires an API key. Best practice is to set the `TWINGLY_SEARCH_KEY` environment variable to the obtained key. `Twingly::Search::Client` can be passed a key at initialization if your setup does not allow environment variables.
|
37
42
|
|
38
|
-
|
43
|
+
To learn more about the features of this gem, read the [gem documentation] or check out the example code that can be found in [examples/](examples/).
|
39
44
|
|
40
|
-
|
45
|
+
To learn more about the capabilities of the API, please read the [Twingly Search API documentation].
|
46
|
+
|
47
|
+
[gem documentation]: http://www.rubydoc.info/github/twingly/twingly-search-api-ruby
|
48
|
+
[Twingly Search API documentation]: https://developer.twingly.com/resources/search/
|
41
49
|
|
42
50
|
## Requirements
|
43
51
|
|
44
|
-
* API key, contact sales@twingly.com to get one
|
45
|
-
* Ruby
|
52
|
+
* API key, contact sales@twingly.com via [twingly.com](http://www.twingly.com/try-for-free/) to get one
|
53
|
+
* Ruby
|
54
|
+
* Ruby 1.9, 2.0, 2.1, 2.2
|
55
|
+
* JRuby 9000
|
46
56
|
|
47
57
|
## Development and release
|
48
58
|
|
@@ -53,6 +63,16 @@ Too learn more about the capabilities of this API you should read the [Twingly S
|
|
53
63
|
|
54
64
|
[releases page]: https://github.com/twingly/twingly-search-api-ruby/releases
|
55
65
|
|
66
|
+
### Documentation
|
67
|
+
|
68
|
+
This gem is documented using [YARD]. To start a local YARD server run:
|
69
|
+
|
70
|
+
bundle exec rake yard:server
|
71
|
+
|
72
|
+
The YARD server reloads the documentation automatically so there is no need to restart it when making changes.
|
73
|
+
|
74
|
+
[YARD]: http://yardoc.org/
|
75
|
+
|
56
76
|
## License
|
57
77
|
|
58
78
|
The MIT License (MIT)
|
data/Rakefile
CHANGED
@@ -16,8 +16,26 @@ GitHubChangelogGenerator::RakeTask.new(:changelog) do |config|
|
|
16
16
|
config.project = "twingly-search-api-ruby"
|
17
17
|
end
|
18
18
|
|
19
|
+
namespace :yard do
|
20
|
+
require "yard"
|
21
|
+
require "yard/rake/yardoc_task"
|
22
|
+
|
23
|
+
desc "Generate Yardoc documentation"
|
24
|
+
YARD::Rake::YardocTask.new(:generate)
|
25
|
+
|
26
|
+
desc "Start a Yard server"
|
27
|
+
task :server do
|
28
|
+
sh("yard", "server", "--reload")
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
desc "Synonym for yard:generate"
|
33
|
+
task yard: "yard:generate"
|
34
|
+
|
19
35
|
desc "Synonym for spec"
|
20
|
-
task :
|
36
|
+
task test: :spec
|
37
|
+
|
21
38
|
desc "Synonym for spec"
|
22
|
-
task :
|
23
|
-
|
39
|
+
task tests: :spec
|
40
|
+
|
41
|
+
task default: :spec
|
data/examples/Gemfile
CHANGED
@@ -3,15 +3,22 @@ Bundler.require
|
|
3
3
|
class SearchPostStream
|
4
4
|
def initialize(keyword, language: nil)
|
5
5
|
# Set environment variable TWINGLY_SEARCH_KEY
|
6
|
-
client = Twingly::Search::Client.new
|
7
|
-
|
8
|
-
|
9
|
-
|
6
|
+
client = Twingly::Search::Client.new do |client|
|
7
|
+
client.user_agent = "MyCompany/1.0" # Set optional user agent
|
8
|
+
end
|
9
|
+
|
10
|
+
@query = client.query do |query|
|
11
|
+
query.language = language
|
12
|
+
query.pattern = "sort-order:asc sort:published #{keyword}"
|
13
|
+
end
|
10
14
|
end
|
11
15
|
|
16
|
+
# Run block for each blog post returned from api.
|
17
|
+
# Uses a sliding time-based window to get all results.
|
18
|
+
# @see https://developer.twingly.com/resources/search/#pagination
|
12
19
|
def each
|
13
20
|
loop do
|
14
|
-
result =
|
21
|
+
result = execute_with_retry
|
15
22
|
result.posts.each do |post|
|
16
23
|
yield post
|
17
24
|
end
|
@@ -21,6 +28,14 @@ class SearchPostStream
|
|
21
28
|
@query.start_time = result.posts.last.published
|
22
29
|
end
|
23
30
|
end
|
31
|
+
|
32
|
+
private
|
33
|
+
|
34
|
+
def execute_with_retry
|
35
|
+
Retryable.retryable(on: Twingly::Search::ServerError) do
|
36
|
+
@query.execute
|
37
|
+
end
|
38
|
+
end
|
24
39
|
end
|
25
40
|
|
26
41
|
stream = SearchPostStream.new("(github) AND (hipchat OR slack)")
|
data/examples/hello_world.rb
CHANGED
@@ -2,9 +2,11 @@ Bundler.require
|
|
2
2
|
|
3
3
|
# Set environment variable TWINGLY_SEARCH_KEY
|
4
4
|
client = Twingly::Search::Client.new
|
5
|
-
query = client.query
|
6
|
-
query.pattern
|
7
|
-
query.start_time = Time.now - (24 * 3600) # search last day
|
5
|
+
query = client.query do |query|
|
6
|
+
query.pattern = '"hello world"'
|
7
|
+
query.start_time = Time.now - (24 * 3600) # search last day
|
8
|
+
end
|
9
|
+
|
8
10
|
result = query.execute
|
9
11
|
result.posts.each do |post|
|
10
12
|
puts post.url
|
data/lib/twingly/search.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
|
+
require 'twingly/search/version'
|
2
|
+
require 'twingly/search/error'
|
1
3
|
require 'twingly/search/client'
|
2
4
|
require 'twingly/search/query'
|
3
5
|
require 'twingly/search/result'
|
4
6
|
require 'twingly/search/parser'
|
5
7
|
require 'twingly/search/post'
|
6
|
-
require 'twingly/search/version'
|
@@ -1,19 +1,76 @@
|
|
1
|
+
require "faraday"
|
2
|
+
|
1
3
|
module Twingly
|
2
4
|
module Search
|
5
|
+
# Twingly Search API client
|
3
6
|
class Client
|
4
|
-
attr_accessor :api_key
|
7
|
+
attr_accessor :api_key, :user_agent
|
8
|
+
|
9
|
+
BASE_URL = "https://api.twingly.com"
|
10
|
+
SEARCH_PATH = "/analytics/Analytics.ashx"
|
11
|
+
|
12
|
+
DEFAULT_USER_AGENT = "Twingly Search Ruby Client/#{VERSION}"
|
13
|
+
|
14
|
+
# Creates a new Twingly Search API client
|
15
|
+
#
|
16
|
+
# @param api_key [optional, String] the API key provided by Twingly.
|
17
|
+
# If nil, reads api_key from environment (TWINGLY_SEARCH_KEY).
|
18
|
+
# @param options [Hash]
|
19
|
+
# @option options [String] :user_agent the user agent to be used
|
20
|
+
# for all requests
|
21
|
+
# @raise [AuthError] if an API key is not set.
|
22
|
+
def initialize(api_key = nil, options = {})
|
23
|
+
@api_key = api_key
|
24
|
+
@user_agent = options.fetch(:user_agent) { DEFAULT_USER_AGENT }
|
5
25
|
|
6
|
-
|
7
|
-
|
26
|
+
yield self if block_given?
|
27
|
+
|
28
|
+
@api_key ||= env_api_key || api_key_missing
|
29
|
+
end
|
30
|
+
|
31
|
+
# Returns a new Query object connected to this client
|
32
|
+
#
|
33
|
+
# @yield [Query]
|
34
|
+
# @return [Query]
|
35
|
+
def query(&block)
|
36
|
+
Query.new(self, &block)
|
8
37
|
end
|
9
38
|
|
10
|
-
|
11
|
-
|
39
|
+
# Executes the given Query and returns the result
|
40
|
+
#
|
41
|
+
# This method should not be called manually, as that is
|
42
|
+
# handled by {Query#execute}.
|
43
|
+
#
|
44
|
+
# @param query [Query] the query to be executed.
|
45
|
+
# @return [Result]
|
46
|
+
def execute_query(query)
|
47
|
+
response_body = get_response(query).body
|
48
|
+
Parser.new.parse(response_body)
|
12
49
|
end
|
13
|
-
|
50
|
+
|
51
|
+
# @return [String] the API endpoint URL
|
52
|
+
def endpoint_url
|
53
|
+
"#{BASE_URL}#{SEARCH_PATH}"
|
54
|
+
end
|
55
|
+
|
56
|
+
private
|
57
|
+
|
14
58
|
def env_api_key
|
15
59
|
ENV['TWINGLY_SEARCH_KEY']
|
16
60
|
end
|
61
|
+
|
62
|
+
def get_response(query)
|
63
|
+
connection = Faraday.new(url: BASE_URL) do |faraday|
|
64
|
+
faraday.request :url_encoded
|
65
|
+
faraday.adapter Faraday.default_adapter
|
66
|
+
end
|
67
|
+
connection.headers[:user_agent] = user_agent
|
68
|
+
connection.get(SEARCH_PATH, query.request_parameters)
|
69
|
+
end
|
70
|
+
|
71
|
+
def api_key_missing
|
72
|
+
fail AuthError, "No API key has been provided."
|
73
|
+
end
|
17
74
|
end
|
18
75
|
end
|
19
76
|
end
|
@@ -0,0 +1,27 @@
|
|
1
|
+
module Twingly
|
2
|
+
module Search
|
3
|
+
class Error < StandardError
|
4
|
+
# @param [String] message API response error message.
|
5
|
+
# @return [Error] an instance of {AuthError} or {ServerError}.
|
6
|
+
def self.from_api_response_message(message)
|
7
|
+
error =
|
8
|
+
if message =~ /API key/
|
9
|
+
AuthError
|
10
|
+
else
|
11
|
+
ServerError
|
12
|
+
end
|
13
|
+
|
14
|
+
error.new(message)
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
class AuthError < Error
|
19
|
+
end
|
20
|
+
|
21
|
+
class ServerError < Error
|
22
|
+
end
|
23
|
+
|
24
|
+
class QueryError < Error
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
@@ -3,29 +3,43 @@ require 'nokogiri'
|
|
3
3
|
module Twingly
|
4
4
|
module Search
|
5
5
|
class Parser
|
6
|
+
# Parse an API response body.
|
7
|
+
#
|
8
|
+
# @param [String] document containing an API response XML.
|
9
|
+
# @raise [Error] which error depends on the API response (see {Error.from_api_response_message}).
|
10
|
+
# @return [Result] containing the result.
|
6
11
|
def parse(document)
|
7
|
-
result = Result.new
|
8
12
|
nokogiri = Nokogiri::XML(document)
|
9
13
|
|
10
|
-
failure = nokogiri.at_xpath('//name:blogstream/name:operationResult[@resultType="failure"]', :
|
11
|
-
|
14
|
+
failure = nokogiri.at_xpath('//name:blogstream/name:operationResult[@resultType="failure"]', name: 'http://www.twingly.com')
|
15
|
+
handle_failure(failure) if failure
|
16
|
+
|
17
|
+
data_node = nokogiri.at_xpath('/twinglydata')
|
18
|
+
handle_non_xml_document(nokogiri) unless data_node
|
19
|
+
|
20
|
+
create_result(data_node)
|
21
|
+
end
|
22
|
+
|
23
|
+
private
|
12
24
|
|
13
|
-
|
14
|
-
result
|
15
|
-
result.
|
25
|
+
def create_result(data_node)
|
26
|
+
result = Result.new
|
27
|
+
result.number_of_matches_returned = data_node.attribute('numberOfMatchesReturned').value.to_i
|
28
|
+
result.number_of_matches_total = data_node.attribute('numberOfMatchesTotal').value.to_i
|
29
|
+
result.seconds_elapsed = data_node.attribute('secondsElapsed').value.to_f
|
16
30
|
|
17
|
-
|
31
|
+
data_node.xpath('//post').each do |post|
|
18
32
|
result.posts << parse_post(post)
|
19
33
|
end
|
20
34
|
|
21
35
|
result
|
22
36
|
end
|
23
|
-
|
37
|
+
|
24
38
|
def parse_post(element)
|
25
39
|
post_params = {}
|
26
40
|
element.element_children.each do |child|
|
27
41
|
if child.name == 'tags'
|
28
|
-
post_params[child.name] = parse_tags(child)
|
42
|
+
post_params[child.name] = parse_tags(child)
|
29
43
|
else
|
30
44
|
post_params[child.name] = child.text
|
31
45
|
end
|
@@ -40,6 +54,15 @@ module Twingly
|
|
40
54
|
child.text
|
41
55
|
end
|
42
56
|
end
|
57
|
+
|
58
|
+
def handle_failure(failure)
|
59
|
+
fail Error.from_api_response_message(failure.text)
|
60
|
+
end
|
61
|
+
|
62
|
+
def handle_non_xml_document(document)
|
63
|
+
response_text = document.search('//text()').map(&:text)
|
64
|
+
fail ServerError, response_text
|
65
|
+
end
|
43
66
|
end
|
44
67
|
end
|
45
68
|
end
|
data/lib/twingly/search/post.rb
CHANGED
@@ -4,10 +4,29 @@ require 'date'
|
|
4
4
|
|
5
5
|
module Twingly
|
6
6
|
module Search
|
7
|
+
# A blog post
|
8
|
+
#
|
9
|
+
# @attr_reader [String] url the post URL.
|
10
|
+
# @attr_reader [String] title the post title.
|
11
|
+
# @attr_reader [String] summary the blog post text.
|
12
|
+
# @attr_reader [String] language_code ISO two letter language code for the
|
13
|
+
# language that the post was written in.
|
14
|
+
# @attr_reader [DateTime] indexed the time, in UTC, when this post was indexed by Twingly.
|
15
|
+
# @attr_reader [DateTime] published the time, in UTC, when this post was published.
|
16
|
+
# @attr_reader [String] blog_url the blog URL.
|
17
|
+
# @attr_reader [String] blog_name name of the blog.
|
18
|
+
# @attr_reader [String] authority the blog's authority/influence.
|
19
|
+
# See https://developer.twingly.com/resources/search/#authority
|
20
|
+
# @attr_reader [Integer] blog_rank the rank of the blog, based on authority and language.
|
21
|
+
# See https://developer.twingly.com/resources/search/#authority
|
22
|
+
# @attr_reader [Array] tags
|
7
23
|
class Post
|
8
24
|
attr_reader :url, :title, :summary, :language_code, :indexed,
|
9
25
|
:published, :blog_url, :blog_name, :authority, :blog_rank, :tags
|
10
26
|
|
27
|
+
# Sets all instance variables for the {Post}, given a Hash.
|
28
|
+
#
|
29
|
+
# @param [Hash] params containing blog post data.
|
11
30
|
def set_values(params)
|
12
31
|
@url = params.fetch('url')
|
13
32
|
@title = params.fetch('title')
|
data/lib/twingly/search/query.rb
CHANGED
@@ -1,43 +1,62 @@
|
|
1
|
-
require
|
1
|
+
require "faraday"
|
2
2
|
|
3
3
|
module Twingly
|
4
4
|
module Search
|
5
|
+
# Twingly Search API query
|
6
|
+
#
|
7
|
+
# @attr [String] pattern the search query.
|
8
|
+
# @attr [String] language which language to restrict the query to.
|
9
|
+
# @attr [Client] client the client that this query is connected to.
|
10
|
+
# @attr [Time, #to_time] start_time search for posts published after
|
11
|
+
# this time (inclusive).
|
12
|
+
# @attr [Time, #to_time] end_time search for posts published before
|
13
|
+
# this time (inclusive).
|
5
14
|
class Query
|
6
15
|
attr_accessor :pattern, :language, :client, :start_time, :end_time
|
7
16
|
|
8
|
-
|
9
|
-
|
10
|
-
|
17
|
+
# No need to call this method manually, instead use {Client#query}.
|
18
|
+
#
|
19
|
+
# @param client [Client] the client that this query should be connected to.
|
11
20
|
def initialize(client)
|
12
21
|
@client = client
|
22
|
+
yield self if block_given?
|
13
23
|
end
|
14
24
|
|
25
|
+
# @return [String] the request url for the query.
|
15
26
|
def url
|
16
|
-
"#{
|
27
|
+
"#{client.endpoint_url}?#{url_parameters}"
|
17
28
|
end
|
18
29
|
|
30
|
+
# Executes the query and returns the result.
|
31
|
+
#
|
32
|
+
# @raise [QueryError] if {#pattern} is empty.
|
33
|
+
# @return [Result] the result for this query.
|
19
34
|
def execute
|
20
|
-
|
35
|
+
@client.execute_query(self)
|
21
36
|
end
|
22
37
|
|
38
|
+
# @see #url
|
39
|
+
# @return [String] the query part of the request url.
|
23
40
|
def url_parameters
|
24
41
|
Faraday::Utils.build_query(request_parameters)
|
25
42
|
end
|
26
43
|
|
44
|
+
# @raise [QueryError] if {#pattern} is empty.
|
45
|
+
# @return [Hash] the request parameters.
|
27
46
|
def request_parameters
|
28
|
-
fail
|
47
|
+
fail QueryError, "Missing pattern" if pattern.to_s.empty?
|
29
48
|
|
30
49
|
{
|
31
|
-
:
|
32
|
-
:
|
33
|
-
:
|
34
|
-
:
|
35
|
-
:
|
36
|
-
:
|
50
|
+
key: client.api_key,
|
51
|
+
searchpattern: pattern,
|
52
|
+
documentlang: language,
|
53
|
+
ts: ts,
|
54
|
+
tsTo: ts_to,
|
55
|
+
xmloutputversion: 2,
|
37
56
|
}
|
38
57
|
end
|
39
58
|
|
40
|
-
|
59
|
+
private
|
41
60
|
|
42
61
|
def ts
|
43
62
|
start_time.to_time.strftime("%F %T") if start_time
|
@@ -46,15 +65,6 @@ module Twingly
|
|
46
65
|
def ts_to
|
47
66
|
end_time.to_time.strftime("%F %T") if end_time
|
48
67
|
end
|
49
|
-
|
50
|
-
def get_response
|
51
|
-
connection = Faraday.new(:url => BASE_URL) do |faraday|
|
52
|
-
faraday.request :url_encoded
|
53
|
-
faraday.adapter Faraday.default_adapter
|
54
|
-
end
|
55
|
-
connection.headers[:user_agent] = "Twingly Search Ruby Client/#{VERSION}"
|
56
|
-
connection.get(SEARCH_PATH, request_parameters)
|
57
|
-
end
|
58
68
|
end
|
59
69
|
end
|
60
70
|
end
|
@@ -1,13 +1,25 @@
|
|
1
1
|
module Twingly
|
2
2
|
module Search
|
3
|
+
# Represents a result from a {Query} to the Search API
|
4
|
+
#
|
5
|
+
# @see Query#execute
|
6
|
+
# @attr [Integer] number_of_matches_returned number of {Post}s
|
7
|
+
# the {Query} returned.
|
8
|
+
# @attr [Integer] number_of_matches_total total number of {Post}s
|
9
|
+
# the {Query} matched.
|
10
|
+
# @attr [Integer] seconds_elapsed number of seconds it took to
|
11
|
+
# execute the {Query}.
|
3
12
|
class Result
|
4
13
|
attr_accessor :number_of_matches_returned, :number_of_matches_total,
|
5
14
|
:seconds_elapsed
|
6
15
|
|
16
|
+
# @return [Array<Post>] all posts that matched the {Query}.
|
7
17
|
def posts
|
8
18
|
@posts ||= []
|
9
19
|
end
|
10
20
|
|
21
|
+
# @return [true] if this result includes all {Post}s that matched the {Query}.
|
22
|
+
# @return [false] if there are more {Post}s to fetch from the API.
|
11
23
|
def all_results_returned?
|
12
24
|
number_of_matches_returned.to_i == number_of_matches_total.to_i
|
13
25
|
end
|
data/spec/client_spec.rb
CHANGED
@@ -4,24 +4,111 @@ include Twingly::Search
|
|
4
4
|
|
5
5
|
describe Client do
|
6
6
|
subject { Client.new('api_key') }
|
7
|
-
context 'with API key as arguments' do
|
8
|
-
it { should be_a Client }
|
9
|
-
end
|
10
7
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
8
|
+
describe ".new" do
|
9
|
+
context 'with API key as arguments' do
|
10
|
+
it { should be_a Client }
|
11
|
+
end
|
12
|
+
|
13
|
+
it "BASE_URL should be parsable" do
|
14
|
+
expect(URI(Client::BASE_URL).to_s).to eq(Client::BASE_URL)
|
15
|
+
end
|
16
|
+
|
17
|
+
context 'with API key from ENV variable' do
|
18
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return('api_key') }
|
19
|
+
subject { Client.new }
|
20
|
+
it { should be_a Client }
|
21
|
+
end
|
22
|
+
|
23
|
+
context 'without valid API key' do
|
24
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return(nil) }
|
25
|
+
subject { Client.new }
|
26
|
+
it { expect { subject }.to raise_error(AuthError, "No API key has been provided.") }
|
27
|
+
end
|
28
|
+
|
29
|
+
context "with optional :user_agent given" do
|
30
|
+
let(:user_agent) { "TwinglySearchTest/1.0" }
|
31
|
+
subject { Client.new('api_key', user_agent: user_agent) }
|
32
|
+
|
33
|
+
it "should use that user agent" do
|
34
|
+
expect(subject.user_agent).to eq(user_agent)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
context "with block" do
|
39
|
+
it "should yield self" do
|
40
|
+
yielded_client = nil
|
41
|
+
client = Client.new("api_key") do |c|
|
42
|
+
yielded_client = c
|
43
|
+
end
|
44
|
+
|
45
|
+
expect(yielded_client).to equal(client)
|
46
|
+
end
|
47
|
+
|
48
|
+
context "when api key gets set in block" do
|
49
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return(nil) }
|
50
|
+
let(:api_key) { "api_key_from_block" }
|
51
|
+
subject do
|
52
|
+
Client.new do |client|
|
53
|
+
client.api_key = api_key
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
it "should not raise an AuthError" do
|
58
|
+
expect { subject }.not_to raise_exception
|
59
|
+
end
|
16
60
|
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
61
|
+
it "should use that api key" do
|
62
|
+
expect(subject.api_key).to eq(api_key)
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
21
66
|
end
|
22
67
|
|
23
68
|
describe '#query' do
|
24
69
|
subject { Client.new('api_key').query }
|
25
70
|
it { should be_a Query }
|
71
|
+
|
72
|
+
context "with block" do
|
73
|
+
subject { Client.new("api_key") }
|
74
|
+
|
75
|
+
it "should yield the query" do
|
76
|
+
yielded_query = nil
|
77
|
+
query = subject.query do |q|
|
78
|
+
yielded_query = q
|
79
|
+
end
|
80
|
+
|
81
|
+
expect(yielded_query).to equal(query)
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
|
86
|
+
describe "#execute_query" do
|
87
|
+
context "with invalid API key" do
|
88
|
+
subject { Client.new("wrong") }
|
89
|
+
|
90
|
+
let(:query) do
|
91
|
+
query = subject.query
|
92
|
+
query.pattern = "something"
|
93
|
+
query
|
94
|
+
end
|
95
|
+
|
96
|
+
it "should raise error on invalid API key" do
|
97
|
+
VCR.use_cassette("search_without_valid_api_key") do
|
98
|
+
expect { subject.execute_query(query) }.to raise_error(AuthError, "The API key does not exist.")
|
99
|
+
end
|
100
|
+
end
|
101
|
+
end
|
102
|
+
end
|
103
|
+
|
104
|
+
describe "#endpoint_url" do
|
105
|
+
subject { Client.new("api_key").endpoint_url }
|
106
|
+
let(:expected) { "#{Client::BASE_URL}#{Client::SEARCH_PATH}" }
|
107
|
+
|
108
|
+
it { is_expected.to eq(expected) }
|
109
|
+
|
110
|
+
it "should be parsable" do
|
111
|
+
expect(URI(subject).to_s).to eq(expected)
|
112
|
+
end
|
26
113
|
end
|
27
114
|
end
|
data/spec/error_spec.rb
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
require "spec_helper"
|
2
|
+
|
3
|
+
describe Twingly::Search::Error do
|
4
|
+
it { is_expected.to be_a(StandardError) }
|
5
|
+
|
6
|
+
describe ".from_api_response_message" do
|
7
|
+
subject { described_class.from_api_response_message(server_response_message) }
|
8
|
+
|
9
|
+
context "when given message containing 'API key'" do
|
10
|
+
let(:server_response_message) { "... API key ..." }
|
11
|
+
|
12
|
+
it { is_expected.to be_an(AuthError) }
|
13
|
+
end
|
14
|
+
|
15
|
+
context "when given a server error message" do
|
16
|
+
let(:server_response_message) { "An error occured." }
|
17
|
+
|
18
|
+
it { is_expected.to be_an(ServerError) }
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
describe "all error classes" do
|
23
|
+
error_classes = [
|
24
|
+
AuthError,
|
25
|
+
ServerError,
|
26
|
+
QueryError,
|
27
|
+
]
|
28
|
+
|
29
|
+
error_classes.each do |error_class|
|
30
|
+
describe error_class do
|
31
|
+
it { is_expected.to be_kind_of(Twingly::Search::Error) }
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
File without changes
|
data/spec/parser_spec.rb
CHANGED
@@ -5,10 +5,53 @@ include Twingly::Search
|
|
5
5
|
describe Parser do
|
6
6
|
it { should respond_to(:parse) }
|
7
7
|
|
8
|
-
let(:document) { File.read('spec/fixtures/valid_result.xml') }
|
9
|
-
|
10
8
|
describe "#parse" do
|
11
9
|
subject { Parser.new.parse(document) }
|
12
|
-
|
10
|
+
|
11
|
+
context "with a valid result" do
|
12
|
+
let(:document) { Fixture.get(:valid) }
|
13
|
+
|
14
|
+
it { is_expected.to be_a Result }
|
15
|
+
end
|
16
|
+
|
17
|
+
context "with a nonexistent api key result" do
|
18
|
+
let(:document) { Fixture.get(:nonexistent_api_key) }
|
19
|
+
|
20
|
+
it "should raise AuthError" do
|
21
|
+
expect { subject }.to raise_error(AuthError)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
context "with an unauthorized api key result" do
|
26
|
+
let(:document) { Fixture.get(:unauthorized_api_key) }
|
27
|
+
|
28
|
+
it "should raise AuthError" do
|
29
|
+
expect { subject }.to raise_error(AuthError)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
context "with a service unavailable result" do
|
34
|
+
let(:document) { Fixture.get(:service_unavailable) }
|
35
|
+
|
36
|
+
it "should raise ServerError" do
|
37
|
+
expect { subject }.to raise_error(ServerError)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
context "with a undefined error result" do
|
42
|
+
let(:document) { Fixture.get(:undefined_error) }
|
43
|
+
|
44
|
+
it "should raise ServerError" do
|
45
|
+
expect { subject }.to raise_error(ServerError)
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
context "with a undefined error result" do
|
50
|
+
let(:document) { Fixture.get(:non_xml) }
|
51
|
+
|
52
|
+
it "should raise ServerError" do
|
53
|
+
expect { subject }.to raise_error(ServerError, /503 Service Unavailable/)
|
54
|
+
end
|
55
|
+
end
|
13
56
|
end
|
14
57
|
end
|
data/spec/query_spec.rb
CHANGED
@@ -4,25 +4,13 @@ require 'vcr_setup'
|
|
4
4
|
include Twingly::Search
|
5
5
|
|
6
6
|
describe Query do
|
7
|
-
|
8
|
-
it "BASE_URL should be parsable" do
|
9
|
-
expect(URI(Query::BASE_URL).to_s).to eq(Query::BASE_URL)
|
10
|
-
end
|
11
|
-
|
12
|
-
context "without client" do
|
13
|
-
subject { Query.new }
|
14
|
-
|
15
|
-
it "should not work" do
|
16
|
-
expect { subject }.to raise_error(ArgumentError)
|
17
|
-
end
|
18
|
-
end
|
7
|
+
let(:client_double) { double("Client") }
|
19
8
|
|
20
9
|
before(:each) do
|
21
|
-
|
22
|
-
allow(@client).to receive(:api_key).and_return('api_key')
|
10
|
+
allow(client_double).to receive(:api_key).and_return("api_key")
|
23
11
|
end
|
24
12
|
|
25
|
-
subject { Query.new(
|
13
|
+
subject { Query.new(client_double) }
|
26
14
|
|
27
15
|
it { should respond_to(:pattern) }
|
28
16
|
it { should respond_to(:language) }
|
@@ -31,8 +19,34 @@ describe Query do
|
|
31
19
|
it { should respond_to(:execute) }
|
32
20
|
it { should respond_to(:client) }
|
33
21
|
|
22
|
+
describe ".new" do
|
23
|
+
context "without client" do
|
24
|
+
subject { Query.new }
|
25
|
+
|
26
|
+
it "should not work" do
|
27
|
+
expect { subject }.to raise_error(ArgumentError)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
context "with block" do
|
32
|
+
it "should yield self" do
|
33
|
+
yielded_query = nil
|
34
|
+
query = Query.new(client_double) do |q|
|
35
|
+
yielded_query = q
|
36
|
+
end
|
37
|
+
|
38
|
+
expect(yielded_query).to equal(query)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
34
43
|
describe "#url" do
|
35
|
-
|
44
|
+
before do
|
45
|
+
endpoint_url = "https://api.twingly.com/analytics/Analytics.ashx"
|
46
|
+
allow(client_double).to receive(:endpoint_url).and_return(endpoint_url)
|
47
|
+
end
|
48
|
+
|
49
|
+
let(:query) { Query.new(client_double) }
|
36
50
|
|
37
51
|
context "with valid pattern" do
|
38
52
|
before { query.pattern = "christmas" }
|
@@ -43,7 +57,7 @@ describe Query do
|
|
43
57
|
|
44
58
|
context "without valid pattern" do
|
45
59
|
it "raises an error" do
|
46
|
-
expect { query.url }.to raise_error(
|
60
|
+
expect { query.url }.to raise_error(QueryError, "Missing pattern")
|
47
61
|
end
|
48
62
|
end
|
49
63
|
|
@@ -51,7 +65,7 @@ describe Query do
|
|
51
65
|
before { query.pattern = "" }
|
52
66
|
|
53
67
|
it "raises an error" do
|
54
|
-
expect { query.url }.to raise_error(
|
68
|
+
expect { query.url }.to raise_error(QueryError, "Missing pattern")
|
55
69
|
end
|
56
70
|
end
|
57
71
|
end
|
@@ -89,17 +103,18 @@ describe Query do
|
|
89
103
|
end
|
90
104
|
|
91
105
|
describe "#execute" do
|
92
|
-
context "
|
93
|
-
|
94
|
-
|
106
|
+
context "when called" do
|
107
|
+
let(:client) { instance_double("Client", "api_key") }
|
108
|
+
subject do
|
109
|
+
query = Query.new(client)
|
95
110
|
query.pattern = 'something'
|
96
111
|
query
|
97
|
-
|
112
|
+
end
|
98
113
|
|
99
|
-
it "should
|
100
|
-
|
101
|
-
|
102
|
-
|
114
|
+
it "should send the query to the client" do
|
115
|
+
expect(client).to receive(:execute_query).with(subject)
|
116
|
+
|
117
|
+
subject.execute
|
103
118
|
end
|
104
119
|
end
|
105
120
|
|
data/spec/spec_helper.rb
CHANGED
@@ -27,4 +27,5 @@ Gem::Specification.new do |spec|
|
|
27
27
|
spec.add_development_dependency "webmock", "~> 1.0"
|
28
28
|
spec.add_development_dependency "rake", "~> 0"
|
29
29
|
spec.add_development_dependency "github_changelog_generator", "~> 1.8"
|
30
|
+
spec.add_development_dependency "yard", "~> 0.8"
|
30
31
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: twingly-search
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 4.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Twingly AB
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-
|
11
|
+
date: 2015-12-02 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|
@@ -128,6 +128,20 @@ dependencies:
|
|
128
128
|
- - "~>"
|
129
129
|
- !ruby/object:Gem::Version
|
130
130
|
version: '1.8'
|
131
|
+
- !ruby/object:Gem::Dependency
|
132
|
+
name: yard
|
133
|
+
requirement: !ruby/object:Gem::Requirement
|
134
|
+
requirements:
|
135
|
+
- - "~>"
|
136
|
+
- !ruby/object:Gem::Version
|
137
|
+
version: '0.8'
|
138
|
+
type: :development
|
139
|
+
prerelease: false
|
140
|
+
version_requirements: !ruby/object:Gem::Requirement
|
141
|
+
requirements:
|
142
|
+
- - "~>"
|
143
|
+
- !ruby/object:Gem::Version
|
144
|
+
version: '0.8'
|
131
145
|
description: Twingly Search is a product from Twingly AB
|
132
146
|
email:
|
133
147
|
- support@twingly.com
|
@@ -147,13 +161,19 @@ files:
|
|
147
161
|
- examples/hello_world.rb
|
148
162
|
- lib/twingly/search.rb
|
149
163
|
- lib/twingly/search/client.rb
|
164
|
+
- lib/twingly/search/error.rb
|
150
165
|
- lib/twingly/search/parser.rb
|
151
166
|
- lib/twingly/search/post.rb
|
152
167
|
- lib/twingly/search/query.rb
|
153
168
|
- lib/twingly/search/result.rb
|
154
169
|
- lib/twingly/search/version.rb
|
155
170
|
- spec/client_spec.rb
|
156
|
-
- spec/
|
171
|
+
- spec/error_spec.rb
|
172
|
+
- spec/fixtures/non_xml_result.xml
|
173
|
+
- spec/fixtures/nonexistent_api_key_result.xml
|
174
|
+
- spec/fixtures/service_unavailable_result.xml
|
175
|
+
- spec/fixtures/unauthorized_api_key_result.xml
|
176
|
+
- spec/fixtures/undefined_error_result.xml
|
157
177
|
- spec/fixtures/valid_result.xml
|
158
178
|
- spec/fixtures/vcr_cassettes/search_for_spotify_on_sv_blogs.yml
|
159
179
|
- spec/fixtures/vcr_cassettes/search_without_valid_api_key.yml
|
@@ -184,13 +204,18 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
184
204
|
version: '0'
|
185
205
|
requirements: []
|
186
206
|
rubyforge_project:
|
187
|
-
rubygems_version: 2.5.
|
207
|
+
rubygems_version: 2.4.5.1
|
188
208
|
signing_key:
|
189
209
|
specification_version: 4
|
190
210
|
summary: Ruby API client for Twingly Search
|
191
211
|
test_files:
|
192
212
|
- spec/client_spec.rb
|
193
|
-
- spec/
|
213
|
+
- spec/error_spec.rb
|
214
|
+
- spec/fixtures/non_xml_result.xml
|
215
|
+
- spec/fixtures/nonexistent_api_key_result.xml
|
216
|
+
- spec/fixtures/service_unavailable_result.xml
|
217
|
+
- spec/fixtures/unauthorized_api_key_result.xml
|
218
|
+
- spec/fixtures/undefined_error_result.xml
|
194
219
|
- spec/fixtures/valid_result.xml
|
195
220
|
- spec/fixtures/vcr_cassettes/search_for_spotify_on_sv_blogs.yml
|
196
221
|
- spec/fixtures/vcr_cassettes/search_without_valid_api_key.yml
|
@@ -200,3 +225,4 @@ test_files:
|
|
200
225
|
- spec/result_spec.rb
|
201
226
|
- spec/spec_helper.rb
|
202
227
|
- spec/vcr_setup.rb
|
228
|
+
has_rdoc:
|