pupa 0.1.11 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.rspec +2 -0
- data/.travis.yml +3 -0
- data/.yardopts +0 -1
- data/Gemfile +1 -3
- data/LICENSE +1 -1
- data/PERFORMANCE.md +1 -1
- data/README.md +19 -23
- data/Rakefile +2 -2
- data/lib/pupa.rb +0 -1
- data/lib/pupa/models/concerns/timestamps.rb +1 -1
- data/lib/pupa/processor/connection.rb +1 -1
- data/lib/pupa/processor/connection_adapters/mongodb_adapter.rb +5 -5
- data/lib/pupa/processor/connection_adapters/postgresql_adapter.rb +1 -1
- data/lib/pupa/processor/document_store/file_store.rb +2 -0
- data/lib/pupa/refinements/faraday_middleware.rb +3 -1
- data/lib/pupa/version.rb +1 -1
- data/pupa.gemspec +9 -9
- data/spec/models/area_spec.rb +1 -1
- data/spec/models/concerns/contactable_spec.rb +8 -8
- data/spec/models/concerns/identifiable_spec.rb +7 -7
- data/spec/models/concerns/linkable_spec.rb +3 -3
- data/spec/models/concerns/nameable_spec.rb +3 -3
- data/spec/models/concerns/sourceable_spec.rb +3 -3
- data/spec/models/concerns/timestamps_spec.rb +4 -4
- data/spec/models/contact_detail_list_spec.rb +6 -6
- data/spec/models/identifier_list_spec.rb +2 -2
- data/spec/models/membership_spec.rb +3 -3
- data/spec/models/model_spec.rb +32 -32
- data/spec/models/motion_spec.rb +1 -1
- data/spec/models/organization_spec.rb +3 -3
- data/spec/models/person_spec.rb +3 -3
- data/spec/models/post_spec.rb +2 -2
- data/spec/models/vote_event_spec.rb +8 -8
- data/spec/models/vote_spec.rb +1 -1
- data/spec/processor/client_spec.rb +4 -4
- data/spec/processor/connection_adapters/mongodb_adapter_spec.rb +8 -8
- data/spec/processor/connection_adapters/postgresql_adapter_spec.rb +1 -62
- data/spec/processor/connection_adapters/sqlite_adapter_spec.rb +5 -0
- data/spec/processor/connection_spec.rb +2 -2
- data/spec/processor/document_store/file_store_spec.rb +19 -19
- data/spec/processor/document_store/redis_store_spec.rb +18 -18
- data/spec/processor/document_store_spec.rb +2 -2
- data/spec/processor/middleware/logger_spec.rb +7 -7
- data/spec/processor/middleware/parse_json_spec.rb +1 -1
- data/spec/processor/yielder_spec.rb +4 -4
- data/spec/processor_spec.rb +41 -41
- data/spec/runner_spec.rb +9 -9
- data/spec/spec_helper.rb +9 -1
- data/spec/support/shared_examples_for_connection_adapters.rb +66 -0
- metadata +38 -35
- data/USAGE +0 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b658ac2d8e84f9eb39493e60888affea15547c1a
|
4
|
+
data.tar.gz: ef073506b9be5c9272422194209ab0bc64bc9af9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 33bff8ac8f6bec1512228a5f4c525dc2adc87aad8892a650e9742d18ea4f89931e2acde2fc306defeb4fcb2b9828ea6783c84c1d979ff64459259e2d3c0091fb
|
7
|
+
data.tar.gz: 3373eb0d882ab71c705cee00dbed2bd507867c14a0683cf8d57a7ccec5814133025abd49443281919257b4e985e64320393da22ac532639ef3c9ad71da2bcda8
|
data/.gitignore
CHANGED
data/.rspec
ADDED
data/.travis.yml
CHANGED
data/.yardopts
CHANGED
data/Gemfile
CHANGED
data/LICENSE
CHANGED
data/PERFORMANCE.md
CHANGED
@@ -4,7 +4,7 @@ Pupa.rb offers several ways to significantly improve performance.
|
|
4
4
|
|
5
5
|
In an example case, reducing disk I/O and skipping validation as described below reduced the time to scrape 10,000 documents from 100 cached HTTP responses from 100 seconds down to 5 seconds. Like fast tests, fast scrapers make development smoother.
|
6
6
|
|
7
|
-
The `import` action's performance is currently limited by the database when a dependency graph is used to determine the evaluation order. If a dependency graph cannot be used because you don't know a related object's ID, [several optimizations](https://github.com/
|
7
|
+
The `import` action's performance is currently limited by the database when a dependency graph is used to determine the evaluation order. If a dependency graph cannot be used because you don't know a related object's ID, [several optimizations](https://github.com/jpmckinney/pupa-ruby/issues/12) can be implemented to improve performance.
|
8
8
|
|
9
9
|
## Reducing HTTP requests
|
10
10
|
|
data/README.md
CHANGED
@@ -1,10 +1,10 @@
|
|
1
1
|
# Pupa.rb: A Data Scraping Framework
|
2
2
|
|
3
|
-
[![Gem Version](https://badge.fury.io/rb/pupa.svg)](
|
4
|
-
[![Build Status](https://secure.travis-ci.org/
|
5
|
-
[![Dependency Status](https://gemnasium.com/
|
6
|
-
[![Coverage Status](https://coveralls.io/repos/
|
7
|
-
[![Code Climate](https://codeclimate.com/github/
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/pupa.svg)](https://badge.fury.io/rb/pupa)
|
4
|
+
[![Build Status](https://secure.travis-ci.org/jpmckinney/pupa-ruby.png)](https://travis-ci.org/jpmckinney/pupa-ruby)
|
5
|
+
[![Dependency Status](https://gemnasium.com/jpmckinney/pupa-ruby.png)](https://gemnasium.com/jpmckinney/pupa-ruby)
|
6
|
+
[![Coverage Status](https://coveralls.io/repos/jpmckinney/pupa-ruby/badge.png?branch=master)](https://coveralls.io/r/jpmckinney/pupa-ruby)
|
7
|
+
[![Code Climate](https://codeclimate.com/github/jpmckinney/pupa-ruby.png)](https://codeclimate.com/github/jpmckinney/pupa-ruby)
|
8
8
|
|
9
9
|
Pupa.rb is a Ruby 2.x fork of Python [Pupa](https://github.com/opencivicdata/pupa). It implements an Extract, Transform and Load (ETL) process to scrape data from online sources, transform it, and write it to a database.
|
10
10
|
|
@@ -14,7 +14,7 @@ Pupa.rb is a Ruby 2.x fork of Python [Pupa](https://github.com/opencivicdata/pup
|
|
14
14
|
|
15
15
|
You can scrape any sort of data with Pupa.rb using your own models. You can also use Pupa.rb to scrape people, organizations, memberships and posts according to the [Popolo](http://www.popoloproject.com/) open government data specification. This gem is up-to-date with Popolo's 2014-10-28 version.
|
16
16
|
|
17
|
-
The [cat.rb](http://
|
17
|
+
The [cat.rb](http://jpmckinney.github.io/pupa-ruby/docs/cat.html) example shows you how to:
|
18
18
|
|
19
19
|
* write a simple Cat class that is compatible with Pupa.rb
|
20
20
|
* use mixins to add Popolo properties to your class
|
@@ -22,31 +22,31 @@ The [cat.rb](http://opennorth.github.io/pupa-ruby/docs/cat.html) example shows y
|
|
22
22
|
* register a scraping task with Pupa.rb
|
23
23
|
* run the processor to save the Cat objects to MongoDB
|
24
24
|
|
25
|
-
The [bill.rb](http://
|
25
|
+
The [bill.rb](http://jpmckinney.github.io/pupa-ruby/docs/bill.html) example shows you how to:
|
26
26
|
|
27
27
|
* create relations between objects
|
28
28
|
* relate two objects, even if you do not know the ID of one object
|
29
29
|
* write separate scraping tasks for different types of data
|
30
30
|
* run each scraping task separately
|
31
31
|
|
32
|
-
The [legislator.rb](http://
|
32
|
+
The [legislator.rb](http://jpmckinney.github.io/pupa-ruby/docs/legislator.html) example shows you how to:
|
33
33
|
|
34
34
|
* use a different HTTP client than the default [Faraday](https://github.com/lostisland/faraday)
|
35
35
|
* select a scraping method according to criteria like the legislative term
|
36
36
|
* pass selection criteria to the processor before running scraping tasks
|
37
37
|
|
38
|
-
The [organization.rb](http://
|
38
|
+
The [organization.rb](http://jpmckinney.github.io/pupa-ruby/docs/organization.html) example shows you how to:
|
39
39
|
|
40
40
|
* register a transformation task with Pupa.rb
|
41
41
|
* run the processor's transformation task
|
42
42
|
|
43
43
|
### Scraping method selection
|
44
44
|
|
45
|
-
1. For simple processing, your processor class need only define a single `scrape_objects` method, which will perform all scraping. See [cat.rb](http://
|
45
|
+
1. For simple processing, your processor class need only define a single `scrape_objects` method, which will perform all scraping. See [cat.rb](http://jpmckinney.github.io/pupa-ruby/docs/cat.html) for an example.
|
46
46
|
|
47
|
-
1. If you scrape many types of data from the same source, you may want to split the scraping into separate tasks according to the type of data being scraped. See [bill.rb](http://
|
47
|
+
1. If you scrape many types of data from the same source, you may want to split the scraping into separate tasks according to the type of data being scraped. See [bill.rb](http://jpmckinney.github.io/pupa-ruby/docs/bill.html) for an example.
|
48
48
|
|
49
|
-
1. You may want more control over the method used to perform a scraping task. For example, a legislature may publish legislators before 1997 in one format and legislators after 1997 in another format. In this case, you may want to select the method used to scrape legislators according to the year. See [legislator.rb](http://
|
49
|
+
1. You may want more control over the method used to perform a scraping task. For example, a legislature may publish legislators before 1997 in one format and legislators after 1997 in another format. In this case, you may want to select the method used to scrape legislators according to the year. See [legislator.rb](http://jpmckinney.github.io/pupa-ruby/docs/legislator.html).
|
50
50
|
|
51
51
|
### Automatic response parsing
|
52
52
|
|
@@ -54,7 +54,7 @@ JSON parsing is enabled by default. To enable automatic parsing of HTML and XML,
|
|
54
54
|
|
55
55
|
## Performance
|
56
56
|
|
57
|
-
Pupa.rb offers several ways to significantly improve performance. [Read the documentation.](https://github.com/
|
57
|
+
Pupa.rb offers several ways to significantly improve performance. [Read the documentation.](https://github.com/jpmckinney/pupa-ruby/blob/master/PERFORMANCE.md#readme)
|
58
58
|
|
59
59
|
## Integration with ODMs
|
60
60
|
|
@@ -73,9 +73,9 @@ Instead, have a simple scraping model that includes `Pupa::Model` and an app mod
|
|
73
73
|
|
74
74
|
Pupa.rb's goal is to make scraping less painful by solving common problems:
|
75
75
|
|
76
|
-
* If you are updating a database by scraping a website, you can either delete and recreate records, or you can merge the scraped records with the saved records. Pupa.rb offers a simple way to merge records, by [using an object's stable properties for identification](http://
|
77
|
-
* If you are scraping a source that references other sources – for example, a committee that references its members – you may want to link the source to its references with foreign keys. Pupa.rb will use whatever identifying information you scrape – for example, the members' names – to [fill in the foreign keys for you](http://
|
78
|
-
* Data sources may use different formats in different contexts. Pupa.rb makes it easy to [select scraping methods](#scraping-method-selection) according to criteria, like the year of publication [for example](http://
|
76
|
+
* If you are updating a database by scraping a website, you can either delete and recreate records, or you can merge the scraped records with the saved records. Pupa.rb offers a simple way to merge records, by [using an object's stable properties for identification](http://jpmckinney.github.io/pupa-ruby/docs/cat.html#section-7).
|
77
|
+
* If you are scraping a source that references other sources – for example, a committee that references its members – you may want to link the source to its references with foreign keys. Pupa.rb will use whatever identifying information you scrape – for example, the members' names – to [fill in the foreign keys for you](http://jpmckinney.github.io/pupa-ruby/docs/bill.html#section-4).
|
78
|
+
* Data sources may use different formats in different contexts. Pupa.rb makes it easy to [select scraping methods](#scraping-method-selection) according to criteria, like the year of publication [for example](http://jpmckinney.github.io/pupa-ruby/docs/legislator.html#section-3).
|
79
79
|
* By splitting the scrape (extract) and import (load) steps, it's easier for you and volunteers to start a scraper without any interaction with a database.
|
80
80
|
|
81
81
|
In short, Pupa.rb lets you spend more time on the tasks that are unique to your use case, and less time on common tasks like caching, merging and storing data. It also provides helpful features like:
|
@@ -83,8 +83,8 @@ In short, Pupa.rb lets you spend more time on the tasks that are unique to your
|
|
83
83
|
* Logging, to make debugging and monitoring a scraper easier
|
84
84
|
* [Automatic response parsing](#automatic-response-parsing) of JSON, XML and HTML
|
85
85
|
* Automatic response decompression
|
86
|
-
* [Option parsing](http://
|
87
|
-
* [Object validation](http://
|
86
|
+
* [Option parsing](http://jpmckinney.github.io/pupa-ruby/docs/legislator.html#section-9), to control your scraper from the command-line
|
87
|
+
* [Object validation](http://jpmckinney.github.io/pupa-ruby/docs/cat.html#section-4), using [JSON Schema](http://json-schema.org/)
|
88
88
|
|
89
89
|
Pupa.rb is extensible, so that you can add your own models, parsers, helpers, actions, etc. It also offers several ways to [improve your scraper's performance](#performance).
|
90
90
|
|
@@ -96,8 +96,4 @@ Both Pupa.rb and Python [Pupa](https://github.com/opencivicdata/pupa) implement
|
|
96
96
|
|
97
97
|
**DO NOT** run this gem's specs if you are using Redis database number 15 on `localhost`!
|
98
98
|
|
99
|
-
|
100
|
-
|
101
|
-
This project's main repository is on GitHub: [http://github.com/opennorth/pupa-ruby](http://github.com/opennorth/pupa-ruby), where your contributions, forks, bug reports, feature requests, and feedback are greatly welcomed.
|
102
|
-
|
103
|
-
Copyright (c) 2013 Open North Inc., released under the MIT license
|
99
|
+
Copyright (c) 2013 James McKinney, released under the MIT license
|
data/Rakefile
CHANGED
@@ -22,8 +22,8 @@ namespace :popolo do
|
|
22
22
|
|
23
23
|
require 'octokit'
|
24
24
|
|
25
|
-
Octokit.contents('
|
26
|
-
response = Octokit.contents('
|
25
|
+
Octokit.contents('popolo-project/popolo-spec', path: 'schemas', ref: 'gh-pages').each do |file|
|
26
|
+
response = Octokit.contents('popolo-project/popolo-spec', path: file.path, ref: 'gh-pages')
|
27
27
|
if response.encoding == 'base64'
|
28
28
|
content = Base64.decode64(response.content)
|
29
29
|
else
|
data/lib/pupa.rb
CHANGED
@@ -16,7 +16,7 @@ module Pupa
|
|
16
16
|
|
17
17
|
set_callback(:save, :before) do |object|
|
18
18
|
# The object may not set created_at.
|
19
|
-
# @see https://github.com/
|
19
|
+
# @see https://github.com/jpmckinney/pupa-ruby/issues/17
|
20
20
|
object.created_at = object.document['created_at'] if object.document
|
21
21
|
object.updated_at = Time.now.utc
|
22
22
|
end
|
@@ -13,7 +13,7 @@ module Pupa
|
|
13
13
|
# @return a configured connection to a database system
|
14
14
|
def self.new(database_url)
|
15
15
|
case URI.parse(database_url).scheme
|
16
|
-
when 'postgres'
|
16
|
+
when 'postgres', 'sqlite'
|
17
17
|
PostgreSQLAdapter.new(database_url)
|
18
18
|
when 'mongodb'
|
19
19
|
MongoDBAdapter.new(database_url)
|
@@ -1,4 +1,4 @@
|
|
1
|
-
require '
|
1
|
+
require 'mongo'
|
2
2
|
|
3
3
|
module Pupa
|
4
4
|
class Processor
|
@@ -10,7 +10,7 @@ module Pupa
|
|
10
10
|
# @param [String] database_url the database URL
|
11
11
|
def initialize(database_url)
|
12
12
|
uri = URI.parse(database_url)
|
13
|
-
@raw_connection =
|
13
|
+
@raw_connection = Mongo::Client.new(["#{uri.host}:#{uri.port}"], database: uri.path[1..-1])
|
14
14
|
@raw_connection.login(uri.user, uri.password) if uri.user && uri.password
|
15
15
|
end
|
16
16
|
|
@@ -60,16 +60,16 @@ module Pupa
|
|
60
60
|
when 0
|
61
61
|
object.run_callbacks(:save) do
|
62
62
|
object.run_callbacks(:create) do
|
63
|
-
collection.
|
63
|
+
collection.insert_one(object.to_h(persist: true))
|
64
64
|
[true, object._id.to_s]
|
65
65
|
end
|
66
66
|
end
|
67
67
|
when 1
|
68
68
|
# Make the document available to the callbacks.
|
69
|
-
# @see https://github.com/
|
69
|
+
# @see https://github.com/jpmckinney/pupa-ruby/issues/17
|
70
70
|
object.document = query.first
|
71
71
|
object.run_callbacks(:save) do
|
72
|
-
query.
|
72
|
+
query.update_one(object.to_h(persist: true).except(:_id))
|
73
73
|
[false, object.document['_id'].to_s]
|
74
74
|
end
|
75
75
|
else
|
@@ -71,7 +71,7 @@ module Pupa
|
|
71
71
|
end
|
72
72
|
when 1
|
73
73
|
# Make the document available to the callbacks.
|
74
|
-
# @see https://github.com/
|
74
|
+
# @see https://github.com/jpmckinney/pupa-ruby/issues/17
|
75
75
|
object.document = stringify_keys(query.first)
|
76
76
|
object.run_callbacks(:save) do
|
77
77
|
query.update(object.to_h(persist: true).except(:_id))
|
@@ -1,3 +1,5 @@
|
|
1
|
+
require 'openssl'
|
2
|
+
|
1
3
|
# Caches all requests, not only GET requests.
|
2
4
|
class FaradayMiddleware::Caching
|
3
5
|
def call(env)
|
@@ -20,6 +22,6 @@ class FaradayMiddleware::Caching
|
|
20
22
|
url.query = params.any? ? build_query(params) : nil
|
21
23
|
end
|
22
24
|
url.normalize!
|
23
|
-
url.scheme + '://' + url.host + url.request_uri + env[:body].to_s # XXX add for POST requests
|
25
|
+
url.scheme + '://' + url.host + url.request_uri + OpenSSL::Digest::MD5.hexdigest(env[:body].to_s) # XXX add for POST requests
|
24
26
|
end
|
25
27
|
end
|
data/lib/pupa/version.rb
CHANGED
data/pupa.gemspec
CHANGED
@@ -5,9 +5,8 @@ Gem::Specification.new do |s|
|
|
5
5
|
s.name = "pupa"
|
6
6
|
s.version = Pupa::VERSION
|
7
7
|
s.platform = Gem::Platform::RUBY
|
8
|
-
s.authors = ["
|
9
|
-
s.
|
10
|
-
s.homepage = "http://github.com/opennorth/pupa-ruby"
|
8
|
+
s.authors = ["James McKinney"]
|
9
|
+
s.homepage = "https://github.com/jpmckinney/pupa-ruby"
|
11
10
|
s.summary = %q{A data scraping framework}
|
12
11
|
s.license = 'MIT'
|
13
12
|
|
@@ -18,22 +17,23 @@ Gem::Specification.new do |s|
|
|
18
17
|
|
19
18
|
s.add_runtime_dependency('activesupport', '~> 4.0')
|
20
19
|
s.add_runtime_dependency('colored', '~> 1.2')
|
21
|
-
s.add_runtime_dependency('faraday_middleware', '~> 0.9.
|
22
|
-
s.add_runtime_dependency('json-schema', '~> 2.
|
20
|
+
s.add_runtime_dependency('faraday_middleware', '~> 0.9.2')
|
21
|
+
s.add_runtime_dependency('json-schema', '~> 2.3')
|
23
22
|
s.add_runtime_dependency('mail')
|
24
|
-
s.add_runtime_dependency('
|
23
|
+
s.add_runtime_dependency('mongo', '~> 2.0')
|
25
24
|
s.add_runtime_dependency('oj', '~> 2.1')
|
26
25
|
s.add_runtime_dependency('sequel', '~> 4.10.0')
|
27
26
|
s.add_runtime_dependency('pg', '~> 0.17.0')
|
28
27
|
|
29
28
|
s.add_development_dependency('coveralls')
|
29
|
+
s.add_development_dependency('rake')
|
30
|
+
s.add_development_dependency('rspec', '~> 2.10')
|
31
|
+
|
30
32
|
s.add_development_dependency('dalli')
|
31
|
-
s.add_development_dependency('json', '~> 1.7.7') # to silence coveralls warning
|
32
33
|
s.add_development_dependency('multi_xml')
|
33
34
|
s.add_development_dependency('nokogiri', '~> 1.6.0')
|
34
35
|
s.add_development_dependency('octokit') # to update Popolo schema
|
35
|
-
s.add_development_dependency('rake')
|
36
36
|
s.add_development_dependency('redis-store')
|
37
|
-
s.add_development_dependency('
|
37
|
+
s.add_development_dependency('sqlite3')
|
38
38
|
s.add_development_dependency('typhoeus')
|
39
39
|
end
|
data/spec/models/area_spec.rb
CHANGED
@@ -14,44 +14,44 @@ describe Pupa::Concerns::Contactable do
|
|
14
14
|
|
15
15
|
describe '#initialize' do
|
16
16
|
it 'should initialize an empty ContactDetailList' do
|
17
|
-
object.contact_details.
|
18
|
-
object.contact_details.
|
17
|
+
expect(object.contact_details).to be_a(Pupa::ContactDetailList)
|
18
|
+
expect(object.contact_details).to eq([])
|
19
19
|
end
|
20
20
|
|
21
21
|
it 'should initialize the given ContactDetailList' do
|
22
22
|
object = klass.new(contact_details: [{type: 'email', value: 'ceo@example.com', note: 'work'}])
|
23
|
-
object.contact_details.
|
23
|
+
expect(object.contact_details).to eq([{type: 'email', value: 'ceo@example.com', note: 'work'}])
|
24
24
|
end
|
25
25
|
end
|
26
26
|
|
27
27
|
describe '#contact_details=' do
|
28
28
|
it 'should use coerce to a ContactDetailList' do
|
29
29
|
object.contact_details = [{type: 'email', value: 'ceo@example.com', note: 'work'}]
|
30
|
-
object.contact_details.
|
30
|
+
expect(object.contact_details).to be_a(Pupa::ContactDetailList)
|
31
31
|
end
|
32
32
|
|
33
33
|
it 'should symbolize keys' do
|
34
34
|
object.contact_details = [{'type' => 'email', 'value' => 'ceo@example.com'}]
|
35
|
-
object.contact_details.
|
35
|
+
expect(object.contact_details).to eq([{type: 'email', value: 'ceo@example.com'}])
|
36
36
|
end
|
37
37
|
end
|
38
38
|
|
39
39
|
describe '#add_contact_detail' do
|
40
40
|
it 'should add a contact detail' do
|
41
41
|
object.add_contact_detail('email', 'ceo@example.com', label: 'Email', note: 'work', valid_from: '2010-01-01', valid_until: '2010-12-31')
|
42
|
-
object.contact_details.
|
42
|
+
expect(object.contact_details).to eq([{type: 'email', value: 'ceo@example.com', label: 'Email', note: 'work', valid_from: '2010-01-01', valid_until: '2010-12-31'}])
|
43
43
|
end
|
44
44
|
|
45
45
|
it 'should not add a contact detail without a type' do
|
46
46
|
object.add_contact_detail(nil, 'ceo@example.com')
|
47
47
|
object.add_contact_detail('', 'ceo@example.com')
|
48
|
-
object.contact_details.blank
|
48
|
+
expect(object.contact_details.blank?).to eq(true)
|
49
49
|
end
|
50
50
|
|
51
51
|
it 'should not add a contact detail without a value' do
|
52
52
|
object.add_contact_detail('email', nil)
|
53
53
|
object.add_contact_detail('email', '')
|
54
|
-
object.contact_details.blank
|
54
|
+
expect(object.contact_details.blank?).to eq(true)
|
55
55
|
end
|
56
56
|
end
|
57
57
|
end
|
@@ -14,38 +14,38 @@ describe Pupa::Concerns::Identifiable do
|
|
14
14
|
|
15
15
|
describe '#initialize' do
|
16
16
|
it 'should initialize an empty IdentifierList' do
|
17
|
-
object.identifiers.
|
18
|
-
object.identifiers.
|
17
|
+
expect(object.identifiers).to be_a(Pupa::IdentifierList)
|
18
|
+
expect(object.identifiers).to eq([])
|
19
19
|
end
|
20
20
|
|
21
21
|
it 'should initialize the given IdentifierList' do
|
22
22
|
object = klass.new(identifiers: [{identifier: '123456789', scheme: 'DUNS'}])
|
23
|
-
object.identifiers.
|
23
|
+
expect(object.identifiers).to eq([{identifier: '123456789', scheme: 'DUNS'}])
|
24
24
|
end
|
25
25
|
end
|
26
26
|
|
27
27
|
describe '#identifiers=' do
|
28
28
|
it 'should use coerce to a IdentifierList' do
|
29
29
|
object.identifiers = [{identifier: '123456789', scheme: 'DUNS'}]
|
30
|
-
object.identifiers.
|
30
|
+
expect(object.identifiers).to be_a(Pupa::IdentifierList)
|
31
31
|
end
|
32
32
|
|
33
33
|
it 'should symbolize keys' do
|
34
34
|
object.identifiers = [{'identifier' => '123456789', 'scheme' => 'DUNS'}]
|
35
|
-
object.identifiers.
|
35
|
+
expect(object.identifiers).to eq([{identifier: '123456789', scheme: 'DUNS'}])
|
36
36
|
end
|
37
37
|
end
|
38
38
|
|
39
39
|
describe '#add_identifier' do
|
40
40
|
it 'should add an identifier' do
|
41
41
|
object.add_identifier('123456789', scheme: 'duns')
|
42
|
-
object.identifiers.
|
42
|
+
expect(object.identifiers).to eq([{identifier: '123456789', scheme: 'duns'}])
|
43
43
|
end
|
44
44
|
|
45
45
|
it 'should not add an identifier without an identifier' do
|
46
46
|
object.add_identifier(nil)
|
47
47
|
object.add_identifier('')
|
48
|
-
object.identifiers.blank
|
48
|
+
expect(object.identifiers.blank?).to eq(true)
|
49
49
|
end
|
50
50
|
end
|
51
51
|
end
|
@@ -15,20 +15,20 @@ describe Pupa::Concerns::Linkable do
|
|
15
15
|
describe '#links' do
|
16
16
|
it 'should symbolize keys' do
|
17
17
|
object.links = [{'url' => 'http://example.com', 'note' => 'homepage'}]
|
18
|
-
object.links.
|
18
|
+
expect(object.links).to eq([{url: 'http://example.com', note: 'homepage'}])
|
19
19
|
end
|
20
20
|
end
|
21
21
|
|
22
22
|
describe '#add_link' do
|
23
23
|
it 'should add a link' do
|
24
24
|
object.add_link('http://example.com', note: 'homepage')
|
25
|
-
object.links.
|
25
|
+
expect(object.links).to eq([{url: 'http://example.com', note: 'homepage'}])
|
26
26
|
end
|
27
27
|
|
28
28
|
it 'should not add a link without a url' do
|
29
29
|
object.add_link(nil)
|
30
30
|
object.add_link('')
|
31
|
-
object.links.blank
|
31
|
+
expect(object.links.blank?).to eq(true)
|
32
32
|
end
|
33
33
|
end
|
34
34
|
end
|