darlingtonia 1.2.3 → 2.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +4 -1
- data/.rubocop.yml +6 -0
- data/CHANGELOG.md +7 -0
- data/README.md +44 -20
- data/darlingtonia.gemspec +2 -1
- data/docs/_config.yml +1 -0
- data/docs/index.md +98 -0
- data/lib/darlingtonia.rb +1 -0
- data/lib/darlingtonia/hyrax_basic_metadata_mapper.rb +23 -2
- data/lib/darlingtonia/hyrax_record_importer.rb +117 -0
- data/lib/darlingtonia/input_record.rb +2 -2
- data/lib/darlingtonia/version.rb +1 -1
- data/spec/darlingtonia/csv_parser_spec.rb +2 -2
- data/spec/darlingtonia/hyrax_basic_metadata_mapper_spec.rb +5 -1
- data/spec/darlingtonia/hyrax_record_importer_spec.rb +103 -0
- data/spec/darlingtonia/input_record_spec.rb +8 -4
- data/spec/darlingtonia/record_importer_spec.rb +8 -1
- data/spec/fixtures/hyrax/example.csv +1 -1
- data/spec/fixtures/images/animals/cat.png +0 -0
- data/spec/fixtures/images/darlingtonia.png +0 -0
- data/spec/spec_helper.rb +1 -0
- data/spec/support/shared_contexts/with_work_type.rb +42 -0
- metadata +24 -10
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ad3721dc07a3ded3cdd80bd95b0068f098e930a002b5f6e29eeb03a855d62731
|
4
|
+
data.tar.gz: c5b3895c77363da927d7952ff786fe6c63048054db96400cf809511ce5cba144
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0d02ffe72075c23e20aa4d9d222074aa50b669ef3cb7485739e5b9b5cb473b0ff94f9981d3896e2ce3c64b10218222fcd2b65083ec3fb6dafb823806f46492c0
|
7
|
+
data.tar.gz: 27451965e8f4d3493bb7083dc8e6f7f4ad0b189186812d50374b8a7627c4006551bf56c3586b70e62621d470a7b692bc824c2eacb7f6726acf4c3b7a951d3f49
|
data/.gitignore
CHANGED
data/.rubocop.yml
CHANGED
@@ -12,6 +12,7 @@ Lint/HandleExceptions:
|
|
12
12
|
Metrics/AbcSize:
|
13
13
|
Exclude:
|
14
14
|
- 'spec/support/hyrax/basic_metadata.rb'
|
15
|
+
- 'lib/darlingtonia/hyrax_record_importer.rb'
|
15
16
|
|
16
17
|
Metrics/BlockLength:
|
17
18
|
Exclude:
|
@@ -26,6 +27,11 @@ Metrics/MethodLength:
|
|
26
27
|
Exclude:
|
27
28
|
- 'spec/support/hyrax/basic_metadata.rb'
|
28
29
|
- 'lib/darlingtonia/hyrax_basic_metadata_mapper.rb'
|
30
|
+
- lib/darlingtonia/hyrax_record_importer.rb
|
31
|
+
|
32
|
+
Naming/AccessorMethodName:
|
33
|
+
Exclude:
|
34
|
+
- lib/darlingtonia/hyrax_record_importer.rb
|
29
35
|
|
30
36
|
RSpec/DescribeClass:
|
31
37
|
Exclude:
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -1,36 +1,60 @@
|
|
1
|
-
Darlingtonia
|
2
|
-
============
|
3
|
-
[![Build Status](https://travis-ci.org/curationexperts/darlingtonia.svg?branch=master)](https://travis-ci.org/curationexperts/darlingtonia)
|
4
|
-
[![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg)](http://www.rubydoc.info/gems/darlingtonia)
|
1
|
+
# Darlingtonia
|
5
2
|
|
6
|
-
|
7
|
-
|
3
|
+
<table width="100%">
|
4
|
+
<tr><td>
|
5
|
+
<img alt="Darlingtonia californica image" src="https://upload.wikimedia.org/wikipedia/commons/2/20/Darlingtonia_californica_ne1.JPG" width="500px">
|
6
|
+
</td><td>
|
7
|
+
Object import for Hyrax. See the <a href="https://www.rubydoc.info/gems/darlingtonia">API documentation</a> for more
|
8
|
+
information. See the <a href="https://curationexperts.github.io/darlingtonia/">Getting Started</a> guide for a gentle introduction.
|
9
|
+
<br/><br/>
|
10
|
+
<a href="https://en.wikipedia.org/wiki/Darlingtonia_californica"><em>Darlingtonia californica</em></a>,
|
11
|
+
also called the California pitcher plant, cobra lily, or cobra plant, is a species of carnivorous plant, the sole member of the genus <i>Darlingtonia</i> in the family <i>Sarraceniaceae</i>. It is native to Northern California and Oregon growing in bogs and seeps with cold running water.
|
12
|
+
<br/><br/>
|
8
13
|
|
9
|
-
|
10
|
-
|
14
|
+
[![Gem Version](https://badge.fury.io/rb/darlingtonia.svg)](https://badge.fury.io/rb/darlingtonia)
|
15
|
+
[![Build Status](https://travis-ci.org/curationexperts/darlingtonia.svg?branch=master)](https://travis-ci.org/curationexperts/darlingtonia)
|
16
|
+
[![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg)](http://www.rubydoc.info/gems/darlingtonia)
|
11
17
|
|
12
|
-
|
18
|
+
</td></tr>
|
19
|
+
</table>
|
13
20
|
|
21
|
+
## Usage
|
14
22
|
|
15
|
-
|
16
|
-
However, its dependency on `hyrax` is kept strictly optional so most of its code can be reused to
|
17
|
-
good effect elsewhere.
|
23
|
+
In your project's `Gemfile`, add: `gem 'darlingtonia'`, then run `bundle install`.
|
18
24
|
|
19
25
|
To do a basic Hyrax import, first ensure that a [work type is registered](http://www.rubydoc.info/github/samvera/hyrax/Hyrax/Configuration#register_curation_concern-instance_method)
|
20
|
-
with your `Hyrax` application.
|
21
|
-
import with `CsvParser`).
|
26
|
+
with your `Hyrax` application. Then write a class like this:
|
22
27
|
|
23
28
|
```ruby
|
24
|
-
|
25
|
-
parser = Darlingtonia::CsvParser.new(file: file)
|
29
|
+
require 'darlingtonia'
|
26
30
|
|
27
|
-
|
31
|
+
class MyImporter
|
32
|
+
def initialize(csv_file)
|
33
|
+
@csv_file = csv_file
|
34
|
+
raise "Cannot find expected input file #{csv_file}" unless File.exist?(csv_file)
|
35
|
+
end
|
28
36
|
|
29
|
-
|
37
|
+
def import
|
38
|
+
file = File.open(@csv_file)
|
39
|
+
Darlingtonia::Importer.new(parser: Darlingtonia::CsvParser.new(file: file), record_importer: Darlingtonia::HyraxRecordImporter.new).import
|
40
|
+
file.close # unless a block is passed to File.open, the file must be explicitly closed
|
41
|
+
end
|
42
|
+
end
|
30
43
|
```
|
31
44
|
|
32
|
-
|
33
|
-
|
45
|
+
You can find [an example csv file for import to Hyrax](https://github.com/curationexperts/darlingtonia/blob/master/spec/fixtures/hyrax/example.csv) in the fixtures directory. Files for attachment should have the filename in a column
|
46
|
+
with a heading of `files`, and the location of the files should be specified via an
|
47
|
+
environment variables called `IMPORT_PATH`. If `IMPORT_PATH` is not set, `HyraxRecordImporter` will look in `/opt/data` by default.
|
48
|
+
|
49
|
+
## Customizing
|
50
|
+
To input any kind of file other than CSV, you need to provide a `Parser` (out of the box, we support simple CSV import with `CsvParser`). We will be writing guides about
|
51
|
+
how to support custom mappers (for metadata outside of Hyrax's core and basic metadata fields).
|
52
|
+
|
53
|
+
This software is primarily intended for use in a [Hyrax](https://github.com/samvera/hyrax) project.
|
54
|
+
However, its dependency on `hyrax` is kept strictly optional so most of its code can be reused to
|
55
|
+
good effect elsewhere. Note: As of release 2.0, `HyraxBasicMetadataMapper` will be the default mapper.
|
56
|
+
|
57
|
+
## Development
|
34
58
|
|
35
59
|
```sh
|
36
60
|
git clone https://github.com/curationexperts/darlingtonia
|
data/darlingtonia.gemspec
CHANGED
@@ -17,7 +17,7 @@ Gem::Specification.new do |gem|
|
|
17
17
|
|
18
18
|
gem.required_ruby_version = '>= 2.3.4'
|
19
19
|
|
20
|
-
gem.add_dependency 'active-fedora', '>=
|
20
|
+
gem.add_dependency 'active-fedora', '>= 11.5.2'
|
21
21
|
|
22
22
|
gem.add_development_dependency 'yard', '~> 0.9'
|
23
23
|
gem.add_development_dependency 'bixby', '~> 0.3'
|
@@ -26,6 +26,7 @@ Gem::Specification.new do |gem|
|
|
26
26
|
gem.add_development_dependency 'coveralls', '~> 0.8'
|
27
27
|
gem.add_development_dependency 'solr_wrapper', '~> 2.1'
|
28
28
|
gem.add_development_dependency 'fcrepo_wrapper', '~> 0.9'
|
29
|
+
gem.add_development_dependency 'byebug'
|
29
30
|
|
30
31
|
gem.files = `git ls-files`.split("\n")
|
31
32
|
gem.test_files = `git ls-files -- {spec}/*`.split("\n")
|
data/docs/_config.yml
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
theme: jekyll-theme-minimal
|
data/docs/index.md
ADDED
@@ -0,0 +1,98 @@
|
|
1
|
+
# Importing to Hyrax with Darlingtonia
|
2
|
+
|
3
|
+
## Getting Started
|
4
|
+
The simplest use case is importing content that matches [Hyrax's core and basic metadata fields](http://samvera.github.io/metadata_application_profile.html), plus a few extra fields that let Hyrax know how to display the content properly. At a very high level we're going to:
|
5
|
+
1. Write a simple import test first
|
6
|
+
1. Get the test passing
|
7
|
+
1. Write a rake task to run your import with real data
|
8
|
+
|
9
|
+
**Note:** This guide assumes that you already have a working Hyrax instance with at least one work type. If you need help doing that, see [the Hyrax documentation](https://github.com/samvera/hyrax#creating-a-hyrax-based-app). This guide is going to assume we're using an `Image` work type and attaching image files, but should be applicable to any work type or kind of attachment.
|
10
|
+
|
11
|
+
### 1. Write a simple import test
|
12
|
+
1. Make a directory for your importer: `mkdir app/importers`
|
13
|
+
1. Make a directory for your importer tests: `mkdir spec/importers`
|
14
|
+
1. Make a directory for your fixture files: `mkdir spec/fixtures/images`
|
15
|
+
1. Put three small images in `spec/fixtures/images` In this guide, we're using copyright free images from https://www.pexels.com/, `birds.jpg`, `cat.jpg`, and `dog.jpg`.
|
16
|
+
1. Make a directory for your CSV fixture files: `mkdir spec/fixtures/csv_import`
|
17
|
+
1. Put a file like this in `spec/fixtures/csv_import`:
|
18
|
+
```
|
19
|
+
title,source,visibility
|
20
|
+
"A Cute Dog",https://www.pexels.com/photo/animal-blur-canine-close-up-551628/,open
|
21
|
+
"An Interesting Cat",https://www.pexels.com/photo/person-holding-white-cat-1383397/,open
|
22
|
+
"A Flock of Birds",https://www.pexels.com/photo/animal-avian-beak-birds-203088/,open
|
23
|
+
```
|
24
|
+
1. Make a file called `spec/importers/modular_importer_spec.rb` that contains this:
|
25
|
+
```ruby
|
26
|
+
# frozen_string_literal: true
|
27
|
+
|
28
|
+
require 'rails_helper'
|
29
|
+
require 'active_fedora/cleaner'
|
30
|
+
|
31
|
+
RSpec.describe ModularImporter do
|
32
|
+
let(:modular_csv) { 'spec/fixtures/csv_import/modular_input.csv' }
|
33
|
+
let(:user) { ::User.batch_user }
|
34
|
+
|
35
|
+
before do
|
36
|
+
DatabaseCleaner.clean
|
37
|
+
ActiveFedora::Cleaner.clean!
|
38
|
+
end
|
39
|
+
|
40
|
+
it "imports a csv" do
|
41
|
+
expect { ModularImporter.new(modular_csv).import }.to change { Work.count }.by 3
|
42
|
+
end
|
43
|
+
end
|
44
|
+
```
|
45
|
+
1. Make a file called `app/importers/modular_importer.rb` that contains just enough of an importer class that your test can run and give a meaningful error:
|
46
|
+
```ruby
|
47
|
+
class ModularImporter
|
48
|
+
def initialize(csv_file)
|
49
|
+
@csv_file = csv_file
|
50
|
+
raise "Cannot find expected input file #{csv_file}" unless File.exist?(csv_file)
|
51
|
+
end
|
52
|
+
|
53
|
+
def import
|
54
|
+
end
|
55
|
+
end
|
56
|
+
```
|
57
|
+
1. Run your test:
|
58
|
+
```
|
59
|
+
bundle exec rspec spec/importers/modular_importer_spec.rb
|
60
|
+
```
|
61
|
+
It should fail with a message like
|
62
|
+
```
|
63
|
+
expected `Work.count` to have changed by 3, but was changed by 0
|
64
|
+
```
|
65
|
+
|
66
|
+
So, at this point, your test is running, but the importer isn't yet creating any records.
|
67
|
+
|
68
|
+
### 2. Get the test passing
|
69
|
+
1. Add the darlingtonia gem to your `Gemfile` and run bundle update:
|
70
|
+
```
|
71
|
+
gem `darlingtonia`
|
72
|
+
```
|
73
|
+
2. Edit `app/importer/modular_importer.rb` so it looks like this:
|
74
|
+
```ruby
|
75
|
+
require 'darlingtonia'
|
76
|
+
|
77
|
+
class ModularImporter
|
78
|
+
def initialize(csv_file)
|
79
|
+
@csv_file = csv_file
|
80
|
+
raise "Cannot find expected input file #{csv_file}" unless File.exist?(csv_file)
|
81
|
+
end
|
82
|
+
|
83
|
+
def import
|
84
|
+
file = File.open(@csv_file)
|
85
|
+
Darlingtonia::Importer.new(parser: Darlingtonia::CsvParser.new(file: file), record_importer: Darlingtonia::HyraxRecordImporter.new).import
|
86
|
+
file.close # Note that we must close any files we open.
|
87
|
+
end
|
88
|
+
end
|
89
|
+
```
|
90
|
+
3. Now your test should pass with output something like this:
|
91
|
+
|
92
|
+
```
|
93
|
+
ModularImporter
|
94
|
+
Creating record: ["A Cute Dog"].Record created at: jw827b648Record created at: jw827b648Creating record: ["An Interesting Cat"].Record created at: 3n203z084Record created at: 3n203z084Creating record: ["A Flock of Birds"].Record created at: wm117n96bRecord created at: wm117n96b imports a csv
|
95
|
+
|
96
|
+
Finished in 7.56 seconds (files took 9.06 seconds to load)
|
97
|
+
1 example, 0 failures
|
98
|
+
```
|
data/lib/darlingtonia.rb
CHANGED
@@ -12,10 +12,25 @@ module Darlingtonia
|
|
12
12
|
#
|
13
13
|
# @see HashMapper Parent class for more info and examples.
|
14
14
|
class HyraxBasicMetadataMapper < HashMapper
|
15
|
+
# If your CSV headers don't exactly match the
|
16
|
+
# the method name for the property's setter
|
17
|
+
# method, add a mapping here.
|
18
|
+
# Example: the method name is work.resource_type,
|
19
|
+
# but in the CSV file, the header is
|
20
|
+
# 'resource type' (without the underscore).
|
21
|
+
CSV_HEADERS = {
|
22
|
+
resource_type: 'resource type',
|
23
|
+
description: 'abstract or summary',
|
24
|
+
rights_statement: 'rights statement',
|
25
|
+
date_created: 'date created',
|
26
|
+
based_near: 'location',
|
27
|
+
related_url: 'related url'
|
28
|
+
}.freeze
|
29
|
+
|
15
30
|
##
|
16
31
|
# @return [Enumerable<Symbol>] The fields the mapper can process.
|
17
32
|
def fields
|
18
|
-
core_fields + basic_fields
|
33
|
+
core_fields + basic_fields + [:visibility]
|
19
34
|
end
|
20
35
|
|
21
36
|
# Properties defined with `multiple: false` in
|
@@ -45,6 +60,10 @@ module Darlingtonia
|
|
45
60
|
metadata['import_url']
|
46
61
|
end
|
47
62
|
|
63
|
+
def visibility
|
64
|
+
metadata['visibility']
|
65
|
+
end
|
66
|
+
|
48
67
|
##
|
49
68
|
# @return [String] The delimiter that will be used to split a metadata field into separate values.
|
50
69
|
# @example
|
@@ -60,7 +79,9 @@ module Darlingtonia
|
|
60
79
|
##
|
61
80
|
# @see MetadataMapper#map_field
|
62
81
|
def map_field(name)
|
63
|
-
|
82
|
+
method_name = name
|
83
|
+
method_name = CSV_HEADERS[name] if CSV_HEADERS.keys.include?(name)
|
84
|
+
Array(metadata[method_name.to_s]&.split(delimiter))
|
64
85
|
end
|
65
86
|
|
66
87
|
protected
|
@@ -0,0 +1,117 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
module Darlingtonia
|
3
|
+
class HyraxRecordImporter < RecordImporter
|
4
|
+
# TODO: Get this from Hyrax config
|
5
|
+
DEFAULT_CREATOR_KEY = 'batchuser@example.com'
|
6
|
+
|
7
|
+
##
|
8
|
+
# @!attribute [rw] creator
|
9
|
+
# @return [User]
|
10
|
+
attr_accessor :creator
|
11
|
+
attr_accessor :collection_id
|
12
|
+
|
13
|
+
##
|
14
|
+
# @param creator [User]
|
15
|
+
def initialize(error_stream: Darlingtonia.config.default_error_stream,
|
16
|
+
info_stream: Darlingtonia.config.default_info_stream,
|
17
|
+
collection_id: nil)
|
18
|
+
self.creator = ::User.find_or_create_system_user(DEFAULT_CREATOR_KEY)
|
19
|
+
self.collection_id = collection_id
|
20
|
+
|
21
|
+
super(error_stream: error_stream, info_stream: info_stream)
|
22
|
+
end
|
23
|
+
|
24
|
+
##
|
25
|
+
# @param record [ImportRecord]
|
26
|
+
#
|
27
|
+
# @return [void]
|
28
|
+
def import(record:)
|
29
|
+
create_for(record: record)
|
30
|
+
rescue Faraday::ConnectionFailed, Ldp::HttpError => e
|
31
|
+
error_stream << e
|
32
|
+
rescue RuntimeError => e
|
33
|
+
error_stream << e
|
34
|
+
raise e
|
35
|
+
end
|
36
|
+
|
37
|
+
# TODO: You should be able to specify the import type in the import
|
38
|
+
def import_type
|
39
|
+
raise 'No curation_concern found for import' unless
|
40
|
+
defined?(Hyrax) && Hyrax&.config&.curation_concerns&.any?
|
41
|
+
|
42
|
+
Hyrax.config.curation_concerns.first
|
43
|
+
end
|
44
|
+
|
45
|
+
# Depositor is a required field for Hyrax
|
46
|
+
# If it hasn't been set in the metadata, set it to the Hyrax default batch user
|
47
|
+
# Even if it has been set, for now, override that with the batch user.
|
48
|
+
# @param Darlingtonia::InputRecord
|
49
|
+
def set_depositor(record)
|
50
|
+
record.mapper.metadata["depositor"] = @creator.user_key
|
51
|
+
end
|
52
|
+
|
53
|
+
# The path on disk where file attachments can be found
|
54
|
+
def file_attachments_path
|
55
|
+
ENV['IMPORT_PATH'] || '/opt/data'
|
56
|
+
end
|
57
|
+
|
58
|
+
# Create a Hyrax::UploadedFile for each file attachment
|
59
|
+
# TODO: What if we can't find the file?
|
60
|
+
# TODO: How do we specify where the files can be found?
|
61
|
+
# @param [Darlingtonia::InputRecord]
|
62
|
+
# @return [Array] an array of Hyrax::UploadedFile ids
|
63
|
+
def create_upload_files(record)
|
64
|
+
file_attachment_filenames = record.mapper.metadata["files"]
|
65
|
+
return [] if file_attachment_filenames.nil? || file_attachment_filenames.empty?
|
66
|
+
|
67
|
+
files_to_attach = file_attachment_filenames.split(record.mapper.delimiter)
|
68
|
+
|
69
|
+
uploaded_file_ids = []
|
70
|
+
files_to_attach.each do |filename|
|
71
|
+
file = File.open(find_file_path(filename))
|
72
|
+
uploaded_file = Hyrax::UploadedFile.create(user: @creator, file: file)
|
73
|
+
uploaded_file_ids << uploaded_file.id
|
74
|
+
file.close
|
75
|
+
end
|
76
|
+
uploaded_file_ids
|
77
|
+
end
|
78
|
+
|
79
|
+
##
|
80
|
+
# Within the directory specified by ENV['IMPORT_PATH'], find the first
|
81
|
+
# instance of a file matching the given filename.
|
82
|
+
# @param [String] filename
|
83
|
+
# @return [String] a full pathname to the found file
|
84
|
+
def find_file_path(filename)
|
85
|
+
Dir.glob("#{ENV['IMPORT_PATH']}/**/#{filename}").first
|
86
|
+
end
|
87
|
+
|
88
|
+
private
|
89
|
+
|
90
|
+
# Create an object using the Hyrax actor stack
|
91
|
+
# We assume the object was created as expected if the actor stack returns true.
|
92
|
+
def create_for(record:)
|
93
|
+
info_stream << 'Creating record: ' \
|
94
|
+
"#{record.respond_to?(:title) ? record.title : record}."
|
95
|
+
set_depositor(record)
|
96
|
+
uploaded_files = { uploaded_files: create_upload_files(record) }
|
97
|
+
created = import_type.new
|
98
|
+
|
99
|
+
attributes = record.attributes.merge(uploaded_files)
|
100
|
+
attributes = attributes.merge(member_of_collection_ids: [collection_id]) if collection_id
|
101
|
+
|
102
|
+
actor_env = Hyrax::Actors::Environment.new(created,
|
103
|
+
::Ability.new(@creator),
|
104
|
+
attributes)
|
105
|
+
|
106
|
+
if Hyrax::CurationConcern.actor.create(actor_env)
|
107
|
+
info_stream << "Record created at: #{created.id}"
|
108
|
+
else
|
109
|
+
created.errors.each do |attr, msg|
|
110
|
+
error_stream << "Validation failed: #{attr.capitalize}. #{msg}"
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
info_stream << "Record created at: #{created.id}"
|
115
|
+
end
|
116
|
+
end
|
117
|
+
end
|
@@ -14,7 +14,7 @@ module Darlingtonia
|
|
14
14
|
|
15
15
|
##
|
16
16
|
# @param mapper [#map_fields]
|
17
|
-
def initialize(mapper:
|
17
|
+
def initialize(mapper: HyraxBasicMetadataMapper.new)
|
18
18
|
self.mapper = mapper
|
19
19
|
end
|
20
20
|
|
@@ -25,7 +25,7 @@ module Darlingtonia
|
|
25
25
|
#
|
26
26
|
# @return [InputRecord] an input record mapping metadata with the given
|
27
27
|
# mapper
|
28
|
-
def from(metadata:, mapper:
|
28
|
+
def from(metadata:, mapper: HyraxBasicMetadataMapper.new)
|
29
29
|
mapper.metadata = metadata
|
30
30
|
new(mapper: mapper)
|
31
31
|
end
|
data/lib/darlingtonia/version.rb
CHANGED
@@ -11,7 +11,7 @@ describe Darlingtonia::CsvParser do
|
|
11
11
|
shared_context 'with content' do
|
12
12
|
let(:csv_content) do
|
13
13
|
<<-EOS
|
14
|
-
title,description,date
|
14
|
+
title,description,date created
|
15
15
|
The Moomins and the Great Flood,"The Moomins and the Great Flood (Swedish: Småtrollen och den stora översvämningen, literally The Little Trolls and the Great Flood) is a book written by Finnish author Tove Jansson in 1945, during the end of World War II. It was the first book to star the Moomins, but is often seen as a prelude to the main Moomin books, as most of the main characters are introduced in the next book.",1945
|
16
16
|
Comet in Moominland,"Comet in Moominland is the second in Tove Jansson's series of Moomin books. Published in 1946, it marks the first appearance of several main characters, like Snufkin and the Snork Maiden.",1946
|
17
17
|
EOS
|
@@ -44,7 +44,7 @@ EOS
|
|
44
44
|
end
|
45
45
|
|
46
46
|
it 'has correct other fields' do
|
47
|
-
expect(parser.records.map(&:
|
47
|
+
expect(parser.records.map(&:date_created)).to contain_exactly(['1945'], ['1946'])
|
48
48
|
end
|
49
49
|
end
|
50
50
|
|
@@ -19,12 +19,16 @@ describe Darlingtonia::HyraxBasicMetadataMapper do
|
|
19
19
|
:related_url, :bibliographic_citation, :source]
|
20
20
|
end
|
21
21
|
|
22
|
+
let(:tenejo_fields) do
|
23
|
+
[:visibility]
|
24
|
+
end
|
25
|
+
|
22
26
|
it_behaves_like 'a Darlingtonia::Mapper' do
|
23
27
|
let(:metadata) do
|
24
28
|
{ title: ['A Title for a Record'],
|
25
29
|
my_custom_field: ['This gets ignored'] }
|
26
30
|
end
|
27
|
-
let(:expected_fields) { core_fields + basic_fields }
|
31
|
+
let(:expected_fields) { core_fields + basic_fields + tenejo_fields }
|
28
32
|
end
|
29
33
|
|
30
34
|
context 'with metadata, but some missing fields' do
|
@@ -0,0 +1,103 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require 'spec_helper'
|
3
|
+
|
4
|
+
describe Darlingtonia::HyraxRecordImporter, :clean do
|
5
|
+
subject(:importer) do
|
6
|
+
described_class.new(error_stream: error_stream, info_stream: info_stream)
|
7
|
+
end
|
8
|
+
|
9
|
+
let(:error_stream) { [] }
|
10
|
+
let(:info_stream) { [] }
|
11
|
+
let(:record) { Darlingtonia::InputRecord.from(metadata: metadata) }
|
12
|
+
|
13
|
+
context 'collection id' do
|
14
|
+
subject(:importer) do
|
15
|
+
described_class.new(collection_id: collection_id)
|
16
|
+
end
|
17
|
+
let(:collection_id) { '123' }
|
18
|
+
|
19
|
+
load File.expand_path("../../support/shared_contexts/with_work_type.rb", __FILE__)
|
20
|
+
include_context 'with a work type'
|
21
|
+
|
22
|
+
it 'can have a collection id' do
|
23
|
+
expect(importer.collection_id).to eq collection_id
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
27
|
+
context 'with no attached files' do
|
28
|
+
let(:metadata) do
|
29
|
+
{
|
30
|
+
'title' => 'A Title',
|
31
|
+
'language' => 'English',
|
32
|
+
'visibility' => 'open'
|
33
|
+
}
|
34
|
+
end
|
35
|
+
load File.expand_path("../../support/shared_contexts/with_work_type.rb", __FILE__)
|
36
|
+
include_context 'with a work type'
|
37
|
+
|
38
|
+
it 'creates a work for record' do
|
39
|
+
expect { importer.import(record: record) }
|
40
|
+
.to change { Work.count }
|
41
|
+
.by 1
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
context 'with attached files' do
|
46
|
+
before do
|
47
|
+
ENV['IMPORT_PATH'] = File.expand_path('../fixtures/images', File.dirname(__FILE__))
|
48
|
+
end
|
49
|
+
let(:metadata) do
|
50
|
+
{
|
51
|
+
'title' => 'A Title',
|
52
|
+
'language' => 'English',
|
53
|
+
'visibility' => 'open',
|
54
|
+
'files' => 'darlingtonia.png|~|cat.png'
|
55
|
+
}
|
56
|
+
end
|
57
|
+
load File.expand_path("../../support/shared_contexts/with_work_type.rb", __FILE__)
|
58
|
+
include_context 'with a work type'
|
59
|
+
it 'finds a file even if it is in a subdirectory' do
|
60
|
+
expect(importer.find_file_path('cat.png')).to eq "#{ENV['IMPORT_PATH']}/animals/cat.png"
|
61
|
+
end
|
62
|
+
it 'creates a work for record' do
|
63
|
+
expect { importer.import(record: record) }
|
64
|
+
.to change { Work.count }
|
65
|
+
.by 1
|
66
|
+
end
|
67
|
+
it 'makes an uploaded file object for each file attachment' do
|
68
|
+
expect { importer.import(record: record) }
|
69
|
+
.to change { Hyrax::UploadedFile.count }
|
70
|
+
.by 2
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
context 'with and without a depositor value' do
|
75
|
+
context 'when there is no depositor set' do
|
76
|
+
let(:metadata) do
|
77
|
+
{
|
78
|
+
'title' => 'A Title',
|
79
|
+
'language' => 'English',
|
80
|
+
'visibility' => 'open'
|
81
|
+
}
|
82
|
+
end
|
83
|
+
it 'adds the batch user as the depositor' do
|
84
|
+
importer.set_depositor(record)
|
85
|
+
expect(record.mapper.metadata["depositor"]).to eq "batchuser@example.com"
|
86
|
+
end
|
87
|
+
end
|
88
|
+
context 'when there is a depositor set' do
|
89
|
+
let(:metadata) do
|
90
|
+
{
|
91
|
+
'title' => 'A Title',
|
92
|
+
'language' => 'English',
|
93
|
+
'visibility' => 'open',
|
94
|
+
'depositor' => 'my@custom.depositor'
|
95
|
+
}
|
96
|
+
end
|
97
|
+
it 'resets the batch user as the depositor' do
|
98
|
+
importer.set_depositor(record)
|
99
|
+
expect(record.mapper.metadata["depositor"]).to eq "batchuser@example.com"
|
100
|
+
end
|
101
|
+
end
|
102
|
+
end
|
103
|
+
end
|
@@ -6,13 +6,17 @@ describe Darlingtonia::InputRecord do
|
|
6
6
|
subject(:record) { described_class.from(metadata: metadata) }
|
7
7
|
|
8
8
|
let(:metadata) do
|
9
|
-
{ 'title'
|
10
|
-
'
|
9
|
+
{ 'title' => 'Comet in Moominland',
|
10
|
+
'abstract or summary' => 'A book about moomins.' }
|
11
|
+
end
|
12
|
+
|
13
|
+
it 'defaults to a Hyrax Mapper' do
|
14
|
+
expect(described_class.new).to have_attributes(mapper: an_instance_of(Darlingtonia::HyraxBasicMetadataMapper))
|
11
15
|
end
|
12
16
|
|
13
17
|
it 'has metadata and a mapper' do
|
14
18
|
is_expected
|
15
|
-
.to have_attributes(mapper: an_instance_of(Darlingtonia::
|
19
|
+
.to have_attributes(mapper: an_instance_of(Darlingtonia::HyraxBasicMetadataMapper))
|
16
20
|
end
|
17
21
|
|
18
22
|
describe '#attributes' do
|
@@ -51,7 +55,7 @@ describe Darlingtonia::InputRecord do
|
|
51
55
|
end
|
52
56
|
|
53
57
|
it 'has methods for additional mapped metadata fields' do
|
54
|
-
expect(record.description).to contain_exactly metadata['
|
58
|
+
expect(record.description).to contain_exactly metadata['abstract or summary']
|
55
59
|
end
|
56
60
|
|
57
61
|
it 'knows it responds to methods for metadata fields' do
|
@@ -9,7 +9,14 @@ describe Darlingtonia::RecordImporter, :clean do
|
|
9
9
|
|
10
10
|
let(:error_stream) { [] }
|
11
11
|
let(:info_stream) { [] }
|
12
|
-
let(:record) { Darlingtonia::InputRecord.
|
12
|
+
let(:record) { Darlingtonia::InputRecord.from(metadata: metadata) }
|
13
|
+
let(:metadata) do
|
14
|
+
{
|
15
|
+
'title' => 'A Title',
|
16
|
+
'language' => 'English',
|
17
|
+
'visibility' => 'open'
|
18
|
+
}
|
19
|
+
end
|
13
20
|
|
14
21
|
it 'raises an error when no work type exists' do
|
15
22
|
expect { importer.import(record: record) }
|
@@ -1,3 +1,3 @@
|
|
1
|
-
title,depositor,date_uploaded,date_modified,label,relative_path,import_url,
|
1
|
+
title,depositor,date_uploaded,date_modified,label,relative_path,import_url,resource type,creator,contributor,abstract or summary,keyword,license,rights statement,publisher,date created,subject,language,identifier,location,related url,bibliographic_citation,source
|
2
2
|
Work 1 Title,user@example.com,2018-12-21,2018-01-01,Work 1 Label,tmp/files,https://example.com,Work 1 Type,Work 1 creator,Work 1 contrib,Desc 1,Key 1,Lic 1,RS 1,Pub 1,2018-06-06,Subj 1,English|~|Japanese,Ident 1,Based 1,https://example.com/related,Bib 1,Source 1
|
3
3
|
Work 2 Title,,1970-12-21,,Work 2 Label,,,Work 2 Type,,,Desc 2,,,,Pub 2,,Subj 2
|
Binary file
|
Binary file
|
data/spec/spec_helper.rb
CHANGED
@@ -7,10 +7,25 @@ shared_context 'with a work type' do
|
|
7
7
|
load './spec/support/hyrax/basic_metadata.rb'
|
8
8
|
|
9
9
|
class Work < ActiveFedora::Base
|
10
|
+
attr_accessor :visibility
|
10
11
|
include ::Hyrax::CoreMetadata
|
11
12
|
include ::Hyrax::BasicMetadata
|
12
13
|
end
|
13
14
|
|
15
|
+
class User
|
16
|
+
def self.find_or_create_system_user(_email)
|
17
|
+
User.new
|
18
|
+
end
|
19
|
+
|
20
|
+
def user_key
|
21
|
+
'batchuser@example.com'
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
class Ability
|
26
|
+
def initialize(user); end
|
27
|
+
end
|
28
|
+
|
14
29
|
module Hyrax
|
15
30
|
def self.config
|
16
31
|
Config.new
|
@@ -21,6 +36,33 @@ shared_context 'with a work type' do
|
|
21
36
|
[Work]
|
22
37
|
end
|
23
38
|
end
|
39
|
+
|
40
|
+
class UploadedFile < ActiveFedora::Base
|
41
|
+
def self.create(*)
|
42
|
+
h = Hyrax::UploadedFile.new
|
43
|
+
h.save
|
44
|
+
h
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
module Actors
|
49
|
+
class Environment
|
50
|
+
def initialize(new_object, ability, attributes); end
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
class Actor
|
55
|
+
def create(_actor_env)
|
56
|
+
Work.create
|
57
|
+
true
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
class CurationConcern
|
62
|
+
def self.actor
|
63
|
+
Hyrax::Actor.new
|
64
|
+
end
|
65
|
+
end
|
24
66
|
end
|
25
67
|
end
|
26
68
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: darlingtonia
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 2.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Data Curation Experts
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-02-04 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: active-fedora
|
@@ -16,20 +16,14 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version:
|
20
|
-
- - "<="
|
21
|
-
- !ruby/object:Gem::Version
|
22
|
-
version: '12.99'
|
19
|
+
version: 11.5.2
|
23
20
|
type: :runtime
|
24
21
|
prerelease: false
|
25
22
|
version_requirements: !ruby/object:Gem::Requirement
|
26
23
|
requirements:
|
27
24
|
- - ">="
|
28
25
|
- !ruby/object:Gem::Version
|
29
|
-
version:
|
30
|
-
- - "<="
|
31
|
-
- !ruby/object:Gem::Version
|
32
|
-
version: '12.99'
|
26
|
+
version: 11.5.2
|
33
27
|
- !ruby/object:Gem::Dependency
|
34
28
|
name: yard
|
35
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -128,6 +122,20 @@ dependencies:
|
|
128
122
|
- - "~>"
|
129
123
|
- !ruby/object:Gem::Version
|
130
124
|
version: '0.9'
|
125
|
+
- !ruby/object:Gem::Dependency
|
126
|
+
name: byebug
|
127
|
+
requirement: !ruby/object:Gem::Requirement
|
128
|
+
requirements:
|
129
|
+
- - ">="
|
130
|
+
- !ruby/object:Gem::Version
|
131
|
+
version: '0'
|
132
|
+
type: :development
|
133
|
+
prerelease: false
|
134
|
+
version_requirements: !ruby/object:Gem::Requirement
|
135
|
+
requirements:
|
136
|
+
- - ">="
|
137
|
+
- !ruby/object:Gem::Version
|
138
|
+
version: '0'
|
131
139
|
description:
|
132
140
|
email:
|
133
141
|
- administrator@curationexperts.com
|
@@ -143,10 +151,13 @@ files:
|
|
143
151
|
- README.md
|
144
152
|
- Rakefile
|
145
153
|
- darlingtonia.gemspec
|
154
|
+
- docs/_config.yml
|
155
|
+
- docs/index.md
|
146
156
|
- lib/darlingtonia.rb
|
147
157
|
- lib/darlingtonia/always_invalid_validator.rb
|
148
158
|
- lib/darlingtonia/hash_mapper.rb
|
149
159
|
- lib/darlingtonia/hyrax_basic_metadata_mapper.rb
|
160
|
+
- lib/darlingtonia/hyrax_record_importer.rb
|
150
161
|
- lib/darlingtonia/importer.rb
|
151
162
|
- lib/darlingtonia/input_record.rb
|
152
163
|
- lib/darlingtonia/metadata_mapper.rb
|
@@ -169,6 +180,7 @@ files:
|
|
169
180
|
- spec/darlingtonia/formatted_message_stream_spec.rb
|
170
181
|
- spec/darlingtonia/hash_mapper_spec.rb
|
171
182
|
- spec/darlingtonia/hyrax_basic_metadata_mapper_spec.rb
|
183
|
+
- spec/darlingtonia/hyrax_record_importer_spec.rb
|
172
184
|
- spec/darlingtonia/importer_spec.rb
|
173
185
|
- spec/darlingtonia/input_record_spec.rb
|
174
186
|
- spec/darlingtonia/parser_spec.rb
|
@@ -180,6 +192,8 @@ files:
|
|
180
192
|
- spec/fixtures/bad_example.csv
|
181
193
|
- spec/fixtures/example.csv
|
182
194
|
- spec/fixtures/hyrax/example.csv
|
195
|
+
- spec/fixtures/images/animals/cat.png
|
196
|
+
- spec/fixtures/images/darlingtonia.png
|
183
197
|
- spec/integration/import_csv_spec.rb
|
184
198
|
- spec/integration/import_hyrax_csv.rb
|
185
199
|
- spec/spec_helper.rb
|