twingly-search 3.0.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +2 -0
- data/.travis.yml +5 -4
- data/CHANGELOG.md +39 -0
- data/README.md +35 -15
- data/Rakefile +21 -3
- data/examples/Gemfile +1 -0
- data/examples/find_all_posts_mentioning_github.rb +20 -5
- data/examples/hello_world.rb +5 -3
- data/lib/twingly/search.rb +2 -1
- data/lib/twingly/search/client.rb +63 -6
- data/lib/twingly/search/error.rb +27 -0
- data/lib/twingly/search/parser.rb +32 -9
- data/lib/twingly/search/post.rb +19 -0
- data/lib/twingly/search/query.rb +33 -23
- data/lib/twingly/search/result.rb +12 -0
- data/lib/twingly/search/version.rb +1 -1
- data/spec/client_spec.rb +99 -12
- data/spec/error_spec.rb +35 -0
- data/spec/fixtures/non_xml_result.xml +3 -0
- data/spec/fixtures/{invalid_result.xml → nonexistent_api_key_result.xml} +0 -0
- data/spec/fixtures/service_unavailable_result.xml +3 -0
- data/spec/fixtures/unauthorized_api_key_result.xml +3 -0
- data/spec/fixtures/undefined_error_result.xml +3 -0
- data/spec/parser_spec.rb +46 -3
- data/spec/query_spec.rb +41 -26
- data/spec/spec_helper.rb +7 -0
- data/twingly-search-api-ruby.gemspec +1 -0
- metadata +31 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: dd9953d1e03a6e3b8876f3a4b86580ddcfeeeccf
|
4
|
+
data.tar.gz: 36f8ccb71ca25e1016d00e366652e06dc71b6321
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 525c3009c34290b6a86a6a397c9e907d9defbf7089e82e0786274d0c46146d79d4bf9d69cb673120ebd50ea1f10626a13b94d5e113590095f1c557edbc6380ef
|
7
|
+
data.tar.gz: 3df9d3ddd711c7185005a81c93ce3fb4b362a7a526698a2203e475c2e6234054e054269ebb2d45aedb44bc4c31ea88d083598952debd78af2c63f38dcc680edd
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,44 @@
|
|
1
1
|
# Change Log
|
2
2
|
|
3
|
+
## [v4.0.0](https://github.com/twingly/twingly-search-api-ruby/tree/v4.0.0)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/v3.0.0...v4.0.0)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Add JRuby 9000 to supported Ruby versions [\#27](https://github.com/twingly/twingly-search-api-ruby/issues/27)
|
10
|
+
- Rename analytics to search [\#21](https://github.com/twingly/twingly-search-api-ruby/issues/21)
|
11
|
+
|
12
|
+
**Fixed bugs:**
|
13
|
+
|
14
|
+
- Handle non-XML responses from server [\#31](https://github.com/twingly/twingly-search-api-ruby/issues/31)
|
15
|
+
- Changelog is not generated correctly [\#25](https://github.com/twingly/twingly-search-api-ruby/issues/25)
|
16
|
+
|
17
|
+
**Merged pull requests:**
|
18
|
+
|
19
|
+
- Example script: Retry on server error [\#35](https://github.com/twingly/twingly-search-api-ruby/pull/35) ([jage](https://github.com/jage))
|
20
|
+
- Yield self [\#34](https://github.com/twingly/twingly-search-api-ruby/pull/34) ([roback](https://github.com/roback))
|
21
|
+
- Handle non-XML responses from server [\#33](https://github.com/twingly/twingly-search-api-ruby/pull/33) ([roback](https://github.com/roback))
|
22
|
+
- Add YARD documentation comments [\#32](https://github.com/twingly/twingly-search-api-ruby/pull/32) ([roback](https://github.com/roback))
|
23
|
+
- Test with JRuby 9000 on travis [\#30](https://github.com/twingly/twingly-search-api-ruby/pull/30) ([roback](https://github.com/roback))
|
24
|
+
- Handle exceptions [\#29](https://github.com/twingly/twingly-search-api-ruby/pull/29) ([roback](https://github.com/roback))
|
25
|
+
- Let client handle api call [\#28](https://github.com/twingly/twingly-search-api-ruby/pull/28) ([roback](https://github.com/roback))
|
26
|
+
- Cleanup [\#26](https://github.com/twingly/twingly-search-api-ruby/pull/26) ([roback](https://github.com/roback))
|
27
|
+
- Show deprecation warnings for twingly-analytics [\#23](https://github.com/twingly/twingly-search-api-ruby/pull/23) ([roback](https://github.com/roback))
|
28
|
+
|
29
|
+
## [v3.0.0](https://github.com/twingly/twingly-search-api-ruby/tree/v3.0.0) (2015-11-20)
|
30
|
+
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/2.0.1...v3.0.0)
|
31
|
+
|
32
|
+
**Implemented enhancements:**
|
33
|
+
|
34
|
+
- Improve "Development and release" section in README [\#19](https://github.com/twingly/twingly-search-api-ruby/issues/19)
|
35
|
+
|
36
|
+
**Merged pull requests:**
|
37
|
+
|
38
|
+
- Rename analytics to search [\#24](https://github.com/twingly/twingly-search-api-ruby/pull/24) ([roback](https://github.com/roback))
|
39
|
+
- Improve "Development and release" section in README [\#22](https://github.com/twingly/twingly-search-api-ruby/pull/22) ([roback](https://github.com/roback))
|
40
|
+
- Rename analytics to search in readme [\#20](https://github.com/twingly/twingly-search-api-ruby/pull/20) ([roback](https://github.com/roback))
|
41
|
+
|
3
42
|
## [2.0.1](https://github.com/twingly/twingly-search-api-ruby/tree/2.0.1) (2015-09-24)
|
4
43
|
[Full Changelog](https://github.com/twingly/twingly-search-api-ruby/compare/2.0.0...2.0.1)
|
5
44
|
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
# Twingly Search API Ruby
|
2
2
|
|
3
|
-
[](https://travis-ci.org/twingly/twingly-search-api-ruby)
|
4
|
+
[](https://codeclimate.com/github/twingly/twingly-search-api-ruby)
|
5
5
|
|
6
6
|
A Ruby gem for Twingly's Search API (previously known as Analytics API). Twingly is a blog search service that provides a searchable API known as [Twingly Search API](https://developer.twingly.com/resources/search/).
|
7
7
|
|
@@ -9,25 +9,30 @@ A Ruby gem for Twingly's Search API (previously known as Analytics API). Twingly
|
|
9
9
|
|
10
10
|
Install via RubyGems
|
11
11
|
|
12
|
-
```
|
12
|
+
```shell
|
13
13
|
gem install twingly-search
|
14
14
|
```
|
15
15
|
|
16
16
|
Or add to your application's [Gemfile](http://bundler.io/gemfile.html) and then run `bundle`
|
17
17
|
|
18
|
-
```
|
19
|
-
gem
|
18
|
+
```ruby
|
19
|
+
gem "twingly-search"
|
20
20
|
```
|
21
21
|
|
22
22
|
## Usage
|
23
23
|
|
24
|
-
```
|
25
|
-
require
|
24
|
+
```ruby
|
25
|
+
require "twingly/search"
|
26
|
+
|
27
|
+
client = Twingly::Search::Client.new do |client|
|
28
|
+
client.user_agent = "MyCompany/1.0"
|
29
|
+
end
|
30
|
+
|
31
|
+
query = client.query do |query|
|
32
|
+
query.pattern = "github page-size:10"
|
33
|
+
query.language = "sv"
|
34
|
+
end
|
26
35
|
|
27
|
-
client = Twingly::Search::Client.new
|
28
|
-
query = client.query
|
29
|
-
query.pattern = 'github page-size:10'
|
30
|
-
query.language = 'sv'
|
31
36
|
result = query.execute
|
32
37
|
=> #<Twingly::Search::Result:0x3ff7adcbe3d4 @posts, @number_of_matches_returned=10, @number_of_matches_total=3035221>
|
33
38
|
result.posts # will include all returned posts
|
@@ -35,14 +40,19 @@ result.posts # will include all returned posts
|
|
35
40
|
|
36
41
|
The `twingly-search` gem talks to a commercial blog search API and requires an API key. Best practice is to set the `TWINGLY_SEARCH_KEY` environment variable to the obtained key. `Twingly::Search::Client` can be passed a key at initialization if your setup does not allow environment variables.
|
37
42
|
|
38
|
-
|
43
|
+
To learn more about the features of this gem, read the [gem documentation] or check out the example code that can be found in [examples/](examples/).
|
39
44
|
|
40
|
-
|
45
|
+
To learn more about the capabilities of the API, please read the [Twingly Search API documentation].
|
46
|
+
|
47
|
+
[gem documentation]: http://www.rubydoc.info/github/twingly/twingly-search-api-ruby
|
48
|
+
[Twingly Search API documentation]: https://developer.twingly.com/resources/search/
|
41
49
|
|
42
50
|
## Requirements
|
43
51
|
|
44
|
-
* API key, contact sales@twingly.com to get one
|
45
|
-
* Ruby
|
52
|
+
* API key, contact sales@twingly.com via [twingly.com](http://www.twingly.com/try-for-free/) to get one
|
53
|
+
* Ruby
|
54
|
+
* Ruby 1.9, 2.0, 2.1, 2.2
|
55
|
+
* JRuby 9000
|
46
56
|
|
47
57
|
## Development and release
|
48
58
|
|
@@ -53,6 +63,16 @@ Too learn more about the capabilities of this API you should read the [Twingly S
|
|
53
63
|
|
54
64
|
[releases page]: https://github.com/twingly/twingly-search-api-ruby/releases
|
55
65
|
|
66
|
+
### Documentation
|
67
|
+
|
68
|
+
This gem is documented using [YARD]. To start a local YARD server run:
|
69
|
+
|
70
|
+
bundle exec rake yard:server
|
71
|
+
|
72
|
+
The YARD server reloads the documentation automatically so there is no need to restart it when making changes.
|
73
|
+
|
74
|
+
[YARD]: http://yardoc.org/
|
75
|
+
|
56
76
|
## License
|
57
77
|
|
58
78
|
The MIT License (MIT)
|
data/Rakefile
CHANGED
@@ -16,8 +16,26 @@ GitHubChangelogGenerator::RakeTask.new(:changelog) do |config|
|
|
16
16
|
config.project = "twingly-search-api-ruby"
|
17
17
|
end
|
18
18
|
|
19
|
+
namespace :yard do
|
20
|
+
require "yard"
|
21
|
+
require "yard/rake/yardoc_task"
|
22
|
+
|
23
|
+
desc "Generate Yardoc documentation"
|
24
|
+
YARD::Rake::YardocTask.new(:generate)
|
25
|
+
|
26
|
+
desc "Start a Yard server"
|
27
|
+
task :server do
|
28
|
+
sh("yard", "server", "--reload")
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
desc "Synonym for yard:generate"
|
33
|
+
task yard: "yard:generate"
|
34
|
+
|
19
35
|
desc "Synonym for spec"
|
20
|
-
task :
|
36
|
+
task test: :spec
|
37
|
+
|
21
38
|
desc "Synonym for spec"
|
22
|
-
task :
|
23
|
-
|
39
|
+
task tests: :spec
|
40
|
+
|
41
|
+
task default: :spec
|
data/examples/Gemfile
CHANGED
@@ -3,15 +3,22 @@ Bundler.require
|
|
3
3
|
class SearchPostStream
|
4
4
|
def initialize(keyword, language: nil)
|
5
5
|
# Set environment variable TWINGLY_SEARCH_KEY
|
6
|
-
client = Twingly::Search::Client.new
|
7
|
-
|
8
|
-
|
9
|
-
|
6
|
+
client = Twingly::Search::Client.new do |client|
|
7
|
+
client.user_agent = "MyCompany/1.0" # Set optional user agent
|
8
|
+
end
|
9
|
+
|
10
|
+
@query = client.query do |query|
|
11
|
+
query.language = language
|
12
|
+
query.pattern = "sort-order:asc sort:published #{keyword}"
|
13
|
+
end
|
10
14
|
end
|
11
15
|
|
16
|
+
# Run block for each blog post returned from api.
|
17
|
+
# Uses a sliding time-based window to get all results.
|
18
|
+
# @see https://developer.twingly.com/resources/search/#pagination
|
12
19
|
def each
|
13
20
|
loop do
|
14
|
-
result =
|
21
|
+
result = execute_with_retry
|
15
22
|
result.posts.each do |post|
|
16
23
|
yield post
|
17
24
|
end
|
@@ -21,6 +28,14 @@ class SearchPostStream
|
|
21
28
|
@query.start_time = result.posts.last.published
|
22
29
|
end
|
23
30
|
end
|
31
|
+
|
32
|
+
private
|
33
|
+
|
34
|
+
def execute_with_retry
|
35
|
+
Retryable.retryable(on: Twingly::Search::ServerError) do
|
36
|
+
@query.execute
|
37
|
+
end
|
38
|
+
end
|
24
39
|
end
|
25
40
|
|
26
41
|
stream = SearchPostStream.new("(github) AND (hipchat OR slack)")
|
data/examples/hello_world.rb
CHANGED
@@ -2,9 +2,11 @@ Bundler.require
|
|
2
2
|
|
3
3
|
# Set environment variable TWINGLY_SEARCH_KEY
|
4
4
|
client = Twingly::Search::Client.new
|
5
|
-
query = client.query
|
6
|
-
query.pattern
|
7
|
-
query.start_time = Time.now - (24 * 3600) # search last day
|
5
|
+
query = client.query do |query|
|
6
|
+
query.pattern = '"hello world"'
|
7
|
+
query.start_time = Time.now - (24 * 3600) # search last day
|
8
|
+
end
|
9
|
+
|
8
10
|
result = query.execute
|
9
11
|
result.posts.each do |post|
|
10
12
|
puts post.url
|
data/lib/twingly/search.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
|
+
require 'twingly/search/version'
|
2
|
+
require 'twingly/search/error'
|
1
3
|
require 'twingly/search/client'
|
2
4
|
require 'twingly/search/query'
|
3
5
|
require 'twingly/search/result'
|
4
6
|
require 'twingly/search/parser'
|
5
7
|
require 'twingly/search/post'
|
6
|
-
require 'twingly/search/version'
|
@@ -1,19 +1,76 @@
|
|
1
|
+
require "faraday"
|
2
|
+
|
1
3
|
module Twingly
|
2
4
|
module Search
|
5
|
+
# Twingly Search API client
|
3
6
|
class Client
|
4
|
-
attr_accessor :api_key
|
7
|
+
attr_accessor :api_key, :user_agent
|
8
|
+
|
9
|
+
BASE_URL = "https://api.twingly.com"
|
10
|
+
SEARCH_PATH = "/analytics/Analytics.ashx"
|
11
|
+
|
12
|
+
DEFAULT_USER_AGENT = "Twingly Search Ruby Client/#{VERSION}"
|
13
|
+
|
14
|
+
# Creates a new Twingly Search API client
|
15
|
+
#
|
16
|
+
# @param api_key [optional, String] the API key provided by Twingly.
|
17
|
+
# If nil, reads api_key from environment (TWINGLY_SEARCH_KEY).
|
18
|
+
# @param options [Hash]
|
19
|
+
# @option options [String] :user_agent the user agent to be used
|
20
|
+
# for all requests
|
21
|
+
# @raise [AuthError] if an API key is not set.
|
22
|
+
def initialize(api_key = nil, options = {})
|
23
|
+
@api_key = api_key
|
24
|
+
@user_agent = options.fetch(:user_agent) { DEFAULT_USER_AGENT }
|
5
25
|
|
6
|
-
|
7
|
-
|
26
|
+
yield self if block_given?
|
27
|
+
|
28
|
+
@api_key ||= env_api_key || api_key_missing
|
29
|
+
end
|
30
|
+
|
31
|
+
# Returns a new Query object connected to this client
|
32
|
+
#
|
33
|
+
# @yield [Query]
|
34
|
+
# @return [Query]
|
35
|
+
def query(&block)
|
36
|
+
Query.new(self, &block)
|
8
37
|
end
|
9
38
|
|
10
|
-
|
11
|
-
|
39
|
+
# Executes the given Query and returns the result
|
40
|
+
#
|
41
|
+
# This method should not be called manually, as that is
|
42
|
+
# handled by {Query#execute}.
|
43
|
+
#
|
44
|
+
# @param query [Query] the query to be executed.
|
45
|
+
# @return [Result]
|
46
|
+
def execute_query(query)
|
47
|
+
response_body = get_response(query).body
|
48
|
+
Parser.new.parse(response_body)
|
12
49
|
end
|
13
|
-
|
50
|
+
|
51
|
+
# @return [String] the API endpoint URL
|
52
|
+
def endpoint_url
|
53
|
+
"#{BASE_URL}#{SEARCH_PATH}"
|
54
|
+
end
|
55
|
+
|
56
|
+
private
|
57
|
+
|
14
58
|
def env_api_key
|
15
59
|
ENV['TWINGLY_SEARCH_KEY']
|
16
60
|
end
|
61
|
+
|
62
|
+
def get_response(query)
|
63
|
+
connection = Faraday.new(url: BASE_URL) do |faraday|
|
64
|
+
faraday.request :url_encoded
|
65
|
+
faraday.adapter Faraday.default_adapter
|
66
|
+
end
|
67
|
+
connection.headers[:user_agent] = user_agent
|
68
|
+
connection.get(SEARCH_PATH, query.request_parameters)
|
69
|
+
end
|
70
|
+
|
71
|
+
def api_key_missing
|
72
|
+
fail AuthError, "No API key has been provided."
|
73
|
+
end
|
17
74
|
end
|
18
75
|
end
|
19
76
|
end
|
@@ -0,0 +1,27 @@
|
|
1
|
+
module Twingly
|
2
|
+
module Search
|
3
|
+
class Error < StandardError
|
4
|
+
# @param [String] message API response error message.
|
5
|
+
# @return [Error] an instance of {AuthError} or {ServerError}.
|
6
|
+
def self.from_api_response_message(message)
|
7
|
+
error =
|
8
|
+
if message =~ /API key/
|
9
|
+
AuthError
|
10
|
+
else
|
11
|
+
ServerError
|
12
|
+
end
|
13
|
+
|
14
|
+
error.new(message)
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
class AuthError < Error
|
19
|
+
end
|
20
|
+
|
21
|
+
class ServerError < Error
|
22
|
+
end
|
23
|
+
|
24
|
+
class QueryError < Error
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
@@ -3,29 +3,43 @@ require 'nokogiri'
|
|
3
3
|
module Twingly
|
4
4
|
module Search
|
5
5
|
class Parser
|
6
|
+
# Parse an API response body.
|
7
|
+
#
|
8
|
+
# @param [String] document containing an API response XML.
|
9
|
+
# @raise [Error] which error depends on the API response (see {Error.from_api_response_message}).
|
10
|
+
# @return [Result] containing the result.
|
6
11
|
def parse(document)
|
7
|
-
result = Result.new
|
8
12
|
nokogiri = Nokogiri::XML(document)
|
9
13
|
|
10
|
-
failure = nokogiri.at_xpath('//name:blogstream/name:operationResult[@resultType="failure"]', :
|
11
|
-
|
14
|
+
failure = nokogiri.at_xpath('//name:blogstream/name:operationResult[@resultType="failure"]', name: 'http://www.twingly.com')
|
15
|
+
handle_failure(failure) if failure
|
16
|
+
|
17
|
+
data_node = nokogiri.at_xpath('/twinglydata')
|
18
|
+
handle_non_xml_document(nokogiri) unless data_node
|
19
|
+
|
20
|
+
create_result(data_node)
|
21
|
+
end
|
22
|
+
|
23
|
+
private
|
12
24
|
|
13
|
-
|
14
|
-
result
|
15
|
-
result.
|
25
|
+
def create_result(data_node)
|
26
|
+
result = Result.new
|
27
|
+
result.number_of_matches_returned = data_node.attribute('numberOfMatchesReturned').value.to_i
|
28
|
+
result.number_of_matches_total = data_node.attribute('numberOfMatchesTotal').value.to_i
|
29
|
+
result.seconds_elapsed = data_node.attribute('secondsElapsed').value.to_f
|
16
30
|
|
17
|
-
|
31
|
+
data_node.xpath('//post').each do |post|
|
18
32
|
result.posts << parse_post(post)
|
19
33
|
end
|
20
34
|
|
21
35
|
result
|
22
36
|
end
|
23
|
-
|
37
|
+
|
24
38
|
def parse_post(element)
|
25
39
|
post_params = {}
|
26
40
|
element.element_children.each do |child|
|
27
41
|
if child.name == 'tags'
|
28
|
-
post_params[child.name] = parse_tags(child)
|
42
|
+
post_params[child.name] = parse_tags(child)
|
29
43
|
else
|
30
44
|
post_params[child.name] = child.text
|
31
45
|
end
|
@@ -40,6 +54,15 @@ module Twingly
|
|
40
54
|
child.text
|
41
55
|
end
|
42
56
|
end
|
57
|
+
|
58
|
+
def handle_failure(failure)
|
59
|
+
fail Error.from_api_response_message(failure.text)
|
60
|
+
end
|
61
|
+
|
62
|
+
def handle_non_xml_document(document)
|
63
|
+
response_text = document.search('//text()').map(&:text)
|
64
|
+
fail ServerError, response_text
|
65
|
+
end
|
43
66
|
end
|
44
67
|
end
|
45
68
|
end
|
data/lib/twingly/search/post.rb
CHANGED
@@ -4,10 +4,29 @@ require 'date'
|
|
4
4
|
|
5
5
|
module Twingly
|
6
6
|
module Search
|
7
|
+
# A blog post
|
8
|
+
#
|
9
|
+
# @attr_reader [String] url the post URL.
|
10
|
+
# @attr_reader [String] title the post title.
|
11
|
+
# @attr_reader [String] summary the blog post text.
|
12
|
+
# @attr_reader [String] language_code ISO two letter language code for the
|
13
|
+
# language that the post was written in.
|
14
|
+
# @attr_reader [DateTime] indexed the time, in UTC, when this post was indexed by Twingly.
|
15
|
+
# @attr_reader [DateTime] published the time, in UTC, when this post was published.
|
16
|
+
# @attr_reader [String] blog_url the blog URL.
|
17
|
+
# @attr_reader [String] blog_name name of the blog.
|
18
|
+
# @attr_reader [String] authority the blog's authority/influence.
|
19
|
+
# See https://developer.twingly.com/resources/search/#authority
|
20
|
+
# @attr_reader [Integer] blog_rank the rank of the blog, based on authority and language.
|
21
|
+
# See https://developer.twingly.com/resources/search/#authority
|
22
|
+
# @attr_reader [Array] tags
|
7
23
|
class Post
|
8
24
|
attr_reader :url, :title, :summary, :language_code, :indexed,
|
9
25
|
:published, :blog_url, :blog_name, :authority, :blog_rank, :tags
|
10
26
|
|
27
|
+
# Sets all instance variables for the {Post}, given a Hash.
|
28
|
+
#
|
29
|
+
# @param [Hash] params containing blog post data.
|
11
30
|
def set_values(params)
|
12
31
|
@url = params.fetch('url')
|
13
32
|
@title = params.fetch('title')
|
data/lib/twingly/search/query.rb
CHANGED
@@ -1,43 +1,62 @@
|
|
1
|
-
require
|
1
|
+
require "faraday"
|
2
2
|
|
3
3
|
module Twingly
|
4
4
|
module Search
|
5
|
+
# Twingly Search API query
|
6
|
+
#
|
7
|
+
# @attr [String] pattern the search query.
|
8
|
+
# @attr [String] language which language to restrict the query to.
|
9
|
+
# @attr [Client] client the client that this query is connected to.
|
10
|
+
# @attr [Time, #to_time] start_time search for posts published after
|
11
|
+
# this time (inclusive).
|
12
|
+
# @attr [Time, #to_time] end_time search for posts published before
|
13
|
+
# this time (inclusive).
|
5
14
|
class Query
|
6
15
|
attr_accessor :pattern, :language, :client, :start_time, :end_time
|
7
16
|
|
8
|
-
|
9
|
-
|
10
|
-
|
17
|
+
# No need to call this method manually, instead use {Client#query}.
|
18
|
+
#
|
19
|
+
# @param client [Client] the client that this query should be connected to.
|
11
20
|
def initialize(client)
|
12
21
|
@client = client
|
22
|
+
yield self if block_given?
|
13
23
|
end
|
14
24
|
|
25
|
+
# @return [String] the request url for the query.
|
15
26
|
def url
|
16
|
-
"#{
|
27
|
+
"#{client.endpoint_url}?#{url_parameters}"
|
17
28
|
end
|
18
29
|
|
30
|
+
# Executes the query and returns the result.
|
31
|
+
#
|
32
|
+
# @raise [QueryError] if {#pattern} is empty.
|
33
|
+
# @return [Result] the result for this query.
|
19
34
|
def execute
|
20
|
-
|
35
|
+
@client.execute_query(self)
|
21
36
|
end
|
22
37
|
|
38
|
+
# @see #url
|
39
|
+
# @return [String] the query part of the request url.
|
23
40
|
def url_parameters
|
24
41
|
Faraday::Utils.build_query(request_parameters)
|
25
42
|
end
|
26
43
|
|
44
|
+
# @raise [QueryError] if {#pattern} is empty.
|
45
|
+
# @return [Hash] the request parameters.
|
27
46
|
def request_parameters
|
28
|
-
fail
|
47
|
+
fail QueryError, "Missing pattern" if pattern.to_s.empty?
|
29
48
|
|
30
49
|
{
|
31
|
-
:
|
32
|
-
:
|
33
|
-
:
|
34
|
-
:
|
35
|
-
:
|
36
|
-
:
|
50
|
+
key: client.api_key,
|
51
|
+
searchpattern: pattern,
|
52
|
+
documentlang: language,
|
53
|
+
ts: ts,
|
54
|
+
tsTo: ts_to,
|
55
|
+
xmloutputversion: 2,
|
37
56
|
}
|
38
57
|
end
|
39
58
|
|
40
|
-
|
59
|
+
private
|
41
60
|
|
42
61
|
def ts
|
43
62
|
start_time.to_time.strftime("%F %T") if start_time
|
@@ -46,15 +65,6 @@ module Twingly
|
|
46
65
|
def ts_to
|
47
66
|
end_time.to_time.strftime("%F %T") if end_time
|
48
67
|
end
|
49
|
-
|
50
|
-
def get_response
|
51
|
-
connection = Faraday.new(:url => BASE_URL) do |faraday|
|
52
|
-
faraday.request :url_encoded
|
53
|
-
faraday.adapter Faraday.default_adapter
|
54
|
-
end
|
55
|
-
connection.headers[:user_agent] = "Twingly Search Ruby Client/#{VERSION}"
|
56
|
-
connection.get(SEARCH_PATH, request_parameters)
|
57
|
-
end
|
58
68
|
end
|
59
69
|
end
|
60
70
|
end
|
@@ -1,13 +1,25 @@
|
|
1
1
|
module Twingly
|
2
2
|
module Search
|
3
|
+
# Represents a result from a {Query} to the Search API
|
4
|
+
#
|
5
|
+
# @see Query#execute
|
6
|
+
# @attr [Integer] number_of_matches_returned number of {Post}s
|
7
|
+
# the {Query} returned.
|
8
|
+
# @attr [Integer] number_of_matches_total total number of {Post}s
|
9
|
+
# the {Query} matched.
|
10
|
+
# @attr [Integer] seconds_elapsed number of seconds it took to
|
11
|
+
# execute the {Query}.
|
3
12
|
class Result
|
4
13
|
attr_accessor :number_of_matches_returned, :number_of_matches_total,
|
5
14
|
:seconds_elapsed
|
6
15
|
|
16
|
+
# @return [Array<Post>] all posts that matched the {Query}.
|
7
17
|
def posts
|
8
18
|
@posts ||= []
|
9
19
|
end
|
10
20
|
|
21
|
+
# @return [true] if this result includes all {Post}s that matched the {Query}.
|
22
|
+
# @return [false] if there are more {Post}s to fetch from the API.
|
11
23
|
def all_results_returned?
|
12
24
|
number_of_matches_returned.to_i == number_of_matches_total.to_i
|
13
25
|
end
|
data/spec/client_spec.rb
CHANGED
@@ -4,24 +4,111 @@ include Twingly::Search
|
|
4
4
|
|
5
5
|
describe Client do
|
6
6
|
subject { Client.new('api_key') }
|
7
|
-
context 'with API key as arguments' do
|
8
|
-
it { should be_a Client }
|
9
|
-
end
|
10
7
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
8
|
+
describe ".new" do
|
9
|
+
context 'with API key as arguments' do
|
10
|
+
it { should be_a Client }
|
11
|
+
end
|
12
|
+
|
13
|
+
it "BASE_URL should be parsable" do
|
14
|
+
expect(URI(Client::BASE_URL).to_s).to eq(Client::BASE_URL)
|
15
|
+
end
|
16
|
+
|
17
|
+
context 'with API key from ENV variable' do
|
18
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return('api_key') }
|
19
|
+
subject { Client.new }
|
20
|
+
it { should be_a Client }
|
21
|
+
end
|
22
|
+
|
23
|
+
context 'without valid API key' do
|
24
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return(nil) }
|
25
|
+
subject { Client.new }
|
26
|
+
it { expect { subject }.to raise_error(AuthError, "No API key has been provided.") }
|
27
|
+
end
|
28
|
+
|
29
|
+
context "with optional :user_agent given" do
|
30
|
+
let(:user_agent) { "TwinglySearchTest/1.0" }
|
31
|
+
subject { Client.new('api_key', user_agent: user_agent) }
|
32
|
+
|
33
|
+
it "should use that user agent" do
|
34
|
+
expect(subject.user_agent).to eq(user_agent)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
context "with block" do
|
39
|
+
it "should yield self" do
|
40
|
+
yielded_client = nil
|
41
|
+
client = Client.new("api_key") do |c|
|
42
|
+
yielded_client = c
|
43
|
+
end
|
44
|
+
|
45
|
+
expect(yielded_client).to equal(client)
|
46
|
+
end
|
47
|
+
|
48
|
+
context "when api key gets set in block" do
|
49
|
+
before { allow_any_instance_of(Client).to receive(:env_api_key).and_return(nil) }
|
50
|
+
let(:api_key) { "api_key_from_block" }
|
51
|
+
subject do
|
52
|
+
Client.new do |client|
|
53
|
+
client.api_key = api_key
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
it "should not raise an AuthError" do
|
58
|
+
expect { subject }.not_to raise_exception
|
59
|
+
end
|
16
60
|
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
61
|
+
it "should use that api key" do
|
62
|
+
expect(subject.api_key).to eq(api_key)
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
21
66
|
end
|
22
67
|
|
23
68
|
describe '#query' do
|
24
69
|
subject { Client.new('api_key').query }
|
25
70
|
it { should be_a Query }
|
71
|
+
|
72
|
+
context "with block" do
|
73
|
+
subject { Client.new("api_key") }
|
74
|
+
|
75
|
+
it "should yield the query" do
|
76
|
+
yielded_query = nil
|
77
|
+
query = subject.query do |q|
|
78
|
+
yielded_query = q
|
79
|
+
end
|
80
|
+
|
81
|
+
expect(yielded_query).to equal(query)
|
82
|
+
end
|
83
|
+
end
|
84
|
+
end
|
85
|
+
|
86
|
+
describe "#execute_query" do
|
87
|
+
context "with invalid API key" do
|
88
|
+
subject { Client.new("wrong") }
|
89
|
+
|
90
|
+
let(:query) do
|
91
|
+
query = subject.query
|
92
|
+
query.pattern = "something"
|
93
|
+
query
|
94
|
+
end
|
95
|
+
|
96
|
+
it "should raise error on invalid API key" do
|
97
|
+
VCR.use_cassette("search_without_valid_api_key") do
|
98
|
+
expect { subject.execute_query(query) }.to raise_error(AuthError, "The API key does not exist.")
|
99
|
+
end
|
100
|
+
end
|
101
|
+
end
|
102
|
+
end
|
103
|
+
|
104
|
+
describe "#endpoint_url" do
|
105
|
+
subject { Client.new("api_key").endpoint_url }
|
106
|
+
let(:expected) { "#{Client::BASE_URL}#{Client::SEARCH_PATH}" }
|
107
|
+
|
108
|
+
it { is_expected.to eq(expected) }
|
109
|
+
|
110
|
+
it "should be parsable" do
|
111
|
+
expect(URI(subject).to_s).to eq(expected)
|
112
|
+
end
|
26
113
|
end
|
27
114
|
end
|
data/spec/error_spec.rb
ADDED
@@ -0,0 +1,35 @@
|
|
1
|
+
require "spec_helper"
|
2
|
+
|
3
|
+
describe Twingly::Search::Error do
|
4
|
+
it { is_expected.to be_a(StandardError) }
|
5
|
+
|
6
|
+
describe ".from_api_response_message" do
|
7
|
+
subject { described_class.from_api_response_message(server_response_message) }
|
8
|
+
|
9
|
+
context "when given message containing 'API key'" do
|
10
|
+
let(:server_response_message) { "... API key ..." }
|
11
|
+
|
12
|
+
it { is_expected.to be_an(AuthError) }
|
13
|
+
end
|
14
|
+
|
15
|
+
context "when given a server error message" do
|
16
|
+
let(:server_response_message) { "An error occured." }
|
17
|
+
|
18
|
+
it { is_expected.to be_an(ServerError) }
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
describe "all error classes" do
|
23
|
+
error_classes = [
|
24
|
+
AuthError,
|
25
|
+
ServerError,
|
26
|
+
QueryError,
|
27
|
+
]
|
28
|
+
|
29
|
+
error_classes.each do |error_class|
|
30
|
+
describe error_class do
|
31
|
+
it { is_expected.to be_kind_of(Twingly::Search::Error) }
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
File without changes
|
data/spec/parser_spec.rb
CHANGED
@@ -5,10 +5,53 @@ include Twingly::Search
|
|
5
5
|
describe Parser do
|
6
6
|
it { should respond_to(:parse) }
|
7
7
|
|
8
|
-
let(:document) { File.read('spec/fixtures/valid_result.xml') }
|
9
|
-
|
10
8
|
describe "#parse" do
|
11
9
|
subject { Parser.new.parse(document) }
|
12
|
-
|
10
|
+
|
11
|
+
context "with a valid result" do
|
12
|
+
let(:document) { Fixture.get(:valid) }
|
13
|
+
|
14
|
+
it { is_expected.to be_a Result }
|
15
|
+
end
|
16
|
+
|
17
|
+
context "with a nonexistent api key result" do
|
18
|
+
let(:document) { Fixture.get(:nonexistent_api_key) }
|
19
|
+
|
20
|
+
it "should raise AuthError" do
|
21
|
+
expect { subject }.to raise_error(AuthError)
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
context "with an unauthorized api key result" do
|
26
|
+
let(:document) { Fixture.get(:unauthorized_api_key) }
|
27
|
+
|
28
|
+
it "should raise AuthError" do
|
29
|
+
expect { subject }.to raise_error(AuthError)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
context "with a service unavailable result" do
|
34
|
+
let(:document) { Fixture.get(:service_unavailable) }
|
35
|
+
|
36
|
+
it "should raise ServerError" do
|
37
|
+
expect { subject }.to raise_error(ServerError)
|
38
|
+
end
|
39
|
+
end
|
40
|
+
|
41
|
+
context "with a undefined error result" do
|
42
|
+
let(:document) { Fixture.get(:undefined_error) }
|
43
|
+
|
44
|
+
it "should raise ServerError" do
|
45
|
+
expect { subject }.to raise_error(ServerError)
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
context "with a undefined error result" do
|
50
|
+
let(:document) { Fixture.get(:non_xml) }
|
51
|
+
|
52
|
+
it "should raise ServerError" do
|
53
|
+
expect { subject }.to raise_error(ServerError, /503 Service Unavailable/)
|
54
|
+
end
|
55
|
+
end
|
13
56
|
end
|
14
57
|
end
|
data/spec/query_spec.rb
CHANGED
@@ -4,25 +4,13 @@ require 'vcr_setup'
|
|
4
4
|
include Twingly::Search
|
5
5
|
|
6
6
|
describe Query do
|
7
|
-
|
8
|
-
it "BASE_URL should be parsable" do
|
9
|
-
expect(URI(Query::BASE_URL).to_s).to eq(Query::BASE_URL)
|
10
|
-
end
|
11
|
-
|
12
|
-
context "without client" do
|
13
|
-
subject { Query.new }
|
14
|
-
|
15
|
-
it "should not work" do
|
16
|
-
expect { subject }.to raise_error(ArgumentError)
|
17
|
-
end
|
18
|
-
end
|
7
|
+
let(:client_double) { double("Client") }
|
19
8
|
|
20
9
|
before(:each) do
|
21
|
-
|
22
|
-
allow(@client).to receive(:api_key).and_return('api_key')
|
10
|
+
allow(client_double).to receive(:api_key).and_return("api_key")
|
23
11
|
end
|
24
12
|
|
25
|
-
subject { Query.new(
|
13
|
+
subject { Query.new(client_double) }
|
26
14
|
|
27
15
|
it { should respond_to(:pattern) }
|
28
16
|
it { should respond_to(:language) }
|
@@ -31,8 +19,34 @@ describe Query do
|
|
31
19
|
it { should respond_to(:execute) }
|
32
20
|
it { should respond_to(:client) }
|
33
21
|
|
22
|
+
describe ".new" do
|
23
|
+
context "without client" do
|
24
|
+
subject { Query.new }
|
25
|
+
|
26
|
+
it "should not work" do
|
27
|
+
expect { subject }.to raise_error(ArgumentError)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
context "with block" do
|
32
|
+
it "should yield self" do
|
33
|
+
yielded_query = nil
|
34
|
+
query = Query.new(client_double) do |q|
|
35
|
+
yielded_query = q
|
36
|
+
end
|
37
|
+
|
38
|
+
expect(yielded_query).to equal(query)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
34
43
|
describe "#url" do
|
35
|
-
|
44
|
+
before do
|
45
|
+
endpoint_url = "https://api.twingly.com/analytics/Analytics.ashx"
|
46
|
+
allow(client_double).to receive(:endpoint_url).and_return(endpoint_url)
|
47
|
+
end
|
48
|
+
|
49
|
+
let(:query) { Query.new(client_double) }
|
36
50
|
|
37
51
|
context "with valid pattern" do
|
38
52
|
before { query.pattern = "christmas" }
|
@@ -43,7 +57,7 @@ describe Query do
|
|
43
57
|
|
44
58
|
context "without valid pattern" do
|
45
59
|
it "raises an error" do
|
46
|
-
expect { query.url }.to raise_error(
|
60
|
+
expect { query.url }.to raise_error(QueryError, "Missing pattern")
|
47
61
|
end
|
48
62
|
end
|
49
63
|
|
@@ -51,7 +65,7 @@ describe Query do
|
|
51
65
|
before { query.pattern = "" }
|
52
66
|
|
53
67
|
it "raises an error" do
|
54
|
-
expect { query.url }.to raise_error(
|
68
|
+
expect { query.url }.to raise_error(QueryError, "Missing pattern")
|
55
69
|
end
|
56
70
|
end
|
57
71
|
end
|
@@ -89,17 +103,18 @@ describe Query do
|
|
89
103
|
end
|
90
104
|
|
91
105
|
describe "#execute" do
|
92
|
-
context "
|
93
|
-
|
94
|
-
|
106
|
+
context "when called" do
|
107
|
+
let(:client) { instance_double("Client", "api_key") }
|
108
|
+
subject do
|
109
|
+
query = Query.new(client)
|
95
110
|
query.pattern = 'something'
|
96
111
|
query
|
97
|
-
|
112
|
+
end
|
98
113
|
|
99
|
-
it "should
|
100
|
-
|
101
|
-
|
102
|
-
|
114
|
+
it "should send the query to the client" do
|
115
|
+
expect(client).to receive(:execute_query).with(subject)
|
116
|
+
|
117
|
+
subject.execute
|
103
118
|
end
|
104
119
|
end
|
105
120
|
|
data/spec/spec_helper.rb
CHANGED
@@ -27,4 +27,5 @@ Gem::Specification.new do |spec|
|
|
27
27
|
spec.add_development_dependency "webmock", "~> 1.0"
|
28
28
|
spec.add_development_dependency "rake", "~> 0"
|
29
29
|
spec.add_development_dependency "github_changelog_generator", "~> 1.8"
|
30
|
+
spec.add_development_dependency "yard", "~> 0.8"
|
30
31
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: twingly-search
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 4.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Twingly AB
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-
|
11
|
+
date: 2015-12-02 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|
@@ -128,6 +128,20 @@ dependencies:
|
|
128
128
|
- - "~>"
|
129
129
|
- !ruby/object:Gem::Version
|
130
130
|
version: '1.8'
|
131
|
+
- !ruby/object:Gem::Dependency
|
132
|
+
name: yard
|
133
|
+
requirement: !ruby/object:Gem::Requirement
|
134
|
+
requirements:
|
135
|
+
- - "~>"
|
136
|
+
- !ruby/object:Gem::Version
|
137
|
+
version: '0.8'
|
138
|
+
type: :development
|
139
|
+
prerelease: false
|
140
|
+
version_requirements: !ruby/object:Gem::Requirement
|
141
|
+
requirements:
|
142
|
+
- - "~>"
|
143
|
+
- !ruby/object:Gem::Version
|
144
|
+
version: '0.8'
|
131
145
|
description: Twingly Search is a product from Twingly AB
|
132
146
|
email:
|
133
147
|
- support@twingly.com
|
@@ -147,13 +161,19 @@ files:
|
|
147
161
|
- examples/hello_world.rb
|
148
162
|
- lib/twingly/search.rb
|
149
163
|
- lib/twingly/search/client.rb
|
164
|
+
- lib/twingly/search/error.rb
|
150
165
|
- lib/twingly/search/parser.rb
|
151
166
|
- lib/twingly/search/post.rb
|
152
167
|
- lib/twingly/search/query.rb
|
153
168
|
- lib/twingly/search/result.rb
|
154
169
|
- lib/twingly/search/version.rb
|
155
170
|
- spec/client_spec.rb
|
156
|
-
- spec/
|
171
|
+
- spec/error_spec.rb
|
172
|
+
- spec/fixtures/non_xml_result.xml
|
173
|
+
- spec/fixtures/nonexistent_api_key_result.xml
|
174
|
+
- spec/fixtures/service_unavailable_result.xml
|
175
|
+
- spec/fixtures/unauthorized_api_key_result.xml
|
176
|
+
- spec/fixtures/undefined_error_result.xml
|
157
177
|
- spec/fixtures/valid_result.xml
|
158
178
|
- spec/fixtures/vcr_cassettes/search_for_spotify_on_sv_blogs.yml
|
159
179
|
- spec/fixtures/vcr_cassettes/search_without_valid_api_key.yml
|
@@ -184,13 +204,18 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
184
204
|
version: '0'
|
185
205
|
requirements: []
|
186
206
|
rubyforge_project:
|
187
|
-
rubygems_version: 2.5.
|
207
|
+
rubygems_version: 2.4.5.1
|
188
208
|
signing_key:
|
189
209
|
specification_version: 4
|
190
210
|
summary: Ruby API client for Twingly Search
|
191
211
|
test_files:
|
192
212
|
- spec/client_spec.rb
|
193
|
-
- spec/
|
213
|
+
- spec/error_spec.rb
|
214
|
+
- spec/fixtures/non_xml_result.xml
|
215
|
+
- spec/fixtures/nonexistent_api_key_result.xml
|
216
|
+
- spec/fixtures/service_unavailable_result.xml
|
217
|
+
- spec/fixtures/unauthorized_api_key_result.xml
|
218
|
+
- spec/fixtures/undefined_error_result.xml
|
194
219
|
- spec/fixtures/valid_result.xml
|
195
220
|
- spec/fixtures/vcr_cassettes/search_for_spotify_on_sv_blogs.yml
|
196
221
|
- spec/fixtures/vcr_cassettes/search_without_valid_api_key.yml
|
@@ -200,3 +225,4 @@ test_files:
|
|
200
225
|
- spec/result_spec.rb
|
201
226
|
- spec/spec_helper.rb
|
202
227
|
- spec/vcr_setup.rb
|
228
|
+
has_rdoc:
|