zizia 5.1.0 → 5.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0c996cb008f5c2688ad0268f351e7ece835fa9822c6b1d95cee5744df9efead9
4
- data.tar.gz: dd746445e693cc7a941f0670cfc3eb6e0ef0c674d0844975825a94673150e097
3
+ metadata.gz: a2ce3c9e0a30e76c3f33a6341945b902801c6ef1efbc131e0c8a19182f71eb71
4
+ data.tar.gz: 37c8b43375005b6e247b87341309e9d45146bf0aebb4297bcd8efb8ccc7128a8
5
5
  SHA512:
6
- metadata.gz: c9094c26f6385c215dc138d8a50495cc08f5962b732d3ba483de63d9c055a1ebebee7d7a01c5b33bb948c983777ebd2eee5b16abf0d7402e03cbe3940f7be16a
7
- data.tar.gz: b595f80d8f66d3fbd0e46a14a7b2baeab74f6feccfafd7f4aa14d249f4fb7c5a01da001848bb652b0b70f17b49d03c089845edca30485ab569456621b891deef
6
+ metadata.gz: 970072b31940c2bacfaaa46f5889d334dffc699a4c3bbfd539994ed035bf674717e612b32a798b514ea028e7523b4df79ae7f7618ecba656572141d2a9923325
7
+ data.tar.gz: 6d895774afd95f9ef4a1ef90944212be27627ff70ddde86562d0471dc00c62d705c162410530836587421e1ce4d0c18fb6ca413f1af2b9721259a49d2b87367d
data/README.md CHANGED
@@ -17,40 +17,58 @@ Object import for Hyrax.
17
17
  ## Usage
18
18
 
19
19
  In your project's `Gemfile`, add: `gem 'zizia'`, then run `bundle install`.
20
+ 1. Require 'zizia' in your `config/application.rb` file:
20
21
 
21
- 1. Add the engine to `routes.rb`:
22
+ ```
23
+ module MyApplication
24
+ class Application < Rails::Application
25
+ require 'zizia'
26
+ ```
27
+
28
+ 2. Add the engine to `routes.rb`:
22
29
  ```
23
30
  mount Zizia::Engine => '/'
24
31
  ```
25
32
 
26
- 2. Add the helpers to the `ApplicationController`
33
+ 3. Add the helpers to the `ApplicationController`
27
34
 
28
35
  ```
29
36
  helper Zizia::Engine.helpers
30
37
  ```
31
38
 
32
- 3. Give admin users permission to import in your `Ability.custom_permissions`:
39
+ 4. Give admin users permission to import in your `Ability.custom_permissions`:
33
40
 
34
41
  ```
35
42
  can :manage, Zizia::CsvImport if current_user.admin?
36
43
  can :manage, Zizia::CsvImportDetail if current_user.admin?
37
44
  ```
38
45
 
39
- 4. Add links to `/csv_imports/new` and `/importer_documentation/csv` in the Hyrax dashboard.
46
+ 5. Add links to `/csv_imports/new` and `/importer_documentation/csv` in the Hyrax dashboard.
40
47
 
41
- 5. In your Rails application's `application.css` and `application.js` include Zizia's assets:
48
+ 6. In your Rails application's `application.css` and `application.js` include Zizia's assets:
42
49
 
43
50
  ```
44
51
  *= require zizia/application
45
52
  ```
46
53
 
47
- 6. Run `rake db:migrate`
54
+ 7. Run `rake db:migrate`
48
55
 
49
56
  The `spec/dummy` folder in this application is a complete Hyrax application with Zizia installed.
50
57
  You can use that as an example for adding this to your current Hyrax application or copy that
51
58
  to create a new application with Zizia installed.
52
59
 
53
- To do a basic Hyrax import, first ensure that a [work type is registered](http://www.rubydoc.info/github/samvera/hyrax/Hyrax/Configuration#register_curation_concern-instance_method)
60
+ 8. Add a deduplication_key to your default work type's medata:
61
+
62
+ ```
63
+ property :deduplication_key, predicate: "http://curationexperts.com/vocab/predicates#deduplicationKey", multiple: false do |index|
64
+ index.as :stored_searchable
65
+ end
66
+ ```
67
+
68
+ 9. If you are using the default [Hyrax metadata profile](https://samvera.github.io/metadata_application_profile.html) aka `Hyrax::BasicMetadata`, you are ready to download a sample CSV and start importing.
69
+
70
+
71
+ If you aren't using `Hyrax::BasicMedata` you'll need to create a custom `importer` and `mapper` class. First ensure that a [work type is registered](http://www.rubydoc.info/github/samvera/hyrax/Hyrax/Configuration#register_curation_concern-instance_method)
54
72
  with your `Hyrax` application. Then write a class like this:
55
73
 
56
74
  ```ruby
@@ -84,9 +102,9 @@ environment variables called `IMPORT_PATH`. If `IMPORT_PATH` is not set, `HyraxR
84
102
 
85
103
  ## Testing
86
104
 
87
- To run Solr and Fedora for testing purposes:
105
+ To run Solr and Fedora for testing purposes, open a new terminal session for each and run the following commads:
88
106
 
89
- `solr_wrapper --config spec/dummy/config/solr_wrapper_test.yml`
107
+ `solr_wrapper --config spec/dummy/config/solr_wrapper_test.yml`
90
108
  `fcrepo_wrapper --config spec/dummy/config/fcrepo_wrapper_test.yml`
91
109
 
92
110
  After this you can run the whole suite, or individual specs. System specs are located
@@ -32,11 +32,12 @@ class ModularImporter
32
32
  }
33
33
 
34
34
  Rails.logger.info "[zizia] event: start_import, batch_id: #{@csv_import.id}, collection_id: #{@collection_id}, user: #{@user_email}"
35
+
35
36
  importer = Zizia::Importer.new(parser: Zizia::CsvParser.new(file: file), record_importer: Zizia::HyraxRecordImporter.new(attributes: attrs))
36
37
 
37
38
  importer.records.each_with_index do |record, index|
38
- pre_ingest_work = Zizia::PreIngestWork.new(csv_import_detail_id: csv_import_detail.id, deduplication_key: record.mapper.metadata["deduplication_key"])
39
-
39
+ pre_ingest_work = Zizia::PreIngestWork.find_or_create_by(deduplication_key: record.mapper.metadata['deduplication_key'])
40
+ pre_ingest_work.csv_import_detail_id = csv_import_detail.id
40
41
  record.mapper.files.each do |child_file|
41
42
  full_path = Dir.glob("#{ENV['IMPORT_PATH']}/**/#{child_file}").first
42
43
  pre_ingest_file = Zizia::PreIngestFile.new(row_number: index + 1,
@@ -2,5 +2,15 @@
2
2
  module Zizia
3
3
  class PreIngestWork < ::ApplicationRecord
4
4
  has_many :pre_ingest_files
5
+
6
+ # Returns the title based on the deduplication_key if the work has been indexed to solr
7
+ # @return [String] the work's title
8
+ def title
9
+ return 'This work does not have a deduplication key.' if deduplication_key.nil?
10
+ solr_title = ActiveFedora::SolrService.get("deduplication_key_tesim:#{deduplication_key}")
11
+ .dig('response', 'docs', 0, 'title_tesim', 0)
12
+ return solr_title unless solr_title.nil?
13
+ 'This work\'s metadata has not been indexed yet.'
14
+ end
5
15
  end
6
16
  end
@@ -0,0 +1,24 @@
1
+ <table id="files-table" class="dataTable display responsive nowrap table table-striped works-list" style="width:100%">
2
+ <tr>
3
+ <th>Filename</th>
4
+ <th>Size</th>
5
+ <th>Row Number</th>
6
+ <th>Date Created</th>
7
+ </tr>
8
+ <% pre_ingest_work.pre_ingest_files.each do |pre_ingest_file| %>
9
+ <tr>
10
+ <td>
11
+ <%= pre_ingest_file.basename %>
12
+ </td>
13
+ <td>
14
+ <%= number_to_human_size(pre_ingest_file.size) %>
15
+ </td>
16
+ <td>
17
+ <%= pre_ingest_file.row_number %>
18
+ </td>
19
+ <td>
20
+ <%= pre_ingest_file.created_at.strftime("%B %-d, %Y %H:%M") %>
21
+ </td>
22
+ </tr>
23
+ <% end %>
24
+ </table>
@@ -7,26 +7,30 @@
7
7
  <li>Total Size: <%= number_to_human_size(@csv_import_detail.total_size) %></li>
8
8
  </ul>
9
9
 
10
- <h3>Files</h3>
11
- <table id="files-table" class="dataTable display responsive nowrap table table-striped works-list" style="width:100%">
10
+ <h3>Works & Files</h3>
11
+
12
+ <table id="works-table" class="table works-list">
12
13
  <tr>
13
- <th>Filename</th>
14
- <th>Size</th>
15
- <th>Row Number</th>
14
+ <th>Deduplication Key</th>
15
+ <th>Title</th>
16
+ <th>Files</th>
17
+ <th>Date</th>
16
18
  </tr>
17
-
18
- <% @csv_import_detail.pre_ingest_files.each do |pre_ingest_file| %>
19
- <tr>
20
- <td>
21
- <%= pre_ingest_file.basename %>
22
- </td>
23
- <td>
24
- <%= number_to_human_size(pre_ingest_file.size) %>
25
- </td>
26
- <td>
27
- <%= pre_ingest_file.row_number %>
28
- </td>
29
- </tr>
30
- <% end %>
19
+ <% @csv_import_detail.pre_ingest_works.each do |pre_ingest_work| %>
20
+ <tr>
21
+ <td>
22
+ <%= pre_ingest_work.deduplication_key%>
23
+ </td>
24
+ <td>
25
+ <%= pre_ingest_work.title %>
26
+ </td>
27
+ <td>
28
+ <%= render 'pre_ingest_files_table', pre_ingest_work: pre_ingest_work %>
29
+ </td>
30
+ <td>
31
+ <%= pre_ingest_work.created_at.strftime("%B %-d, %Y %H:%M") %>
32
+ </td>
33
+ </tr>
34
+ <% end %>
31
35
  </table>
32
36
  </div>
@@ -0,0 +1,7 @@
1
+ class AddUniquenessConstraintToPreIngestWorkDeduplicationKey < ActiveRecord::Migration[5.1]
2
+ def change
3
+ remove_index :zizia_pre_ingest_works, :deduplication_key if index_exists?(:zizia_pre_ingest_works, :deduplication_key)
4
+
5
+ add_index :zizia_pre_ingest_works, :deduplication_key, unique: true
6
+ end
7
+ end
@@ -0,0 +1,7 @@
1
+ class RemoveUniquenessConstraintToPreIngestWorkDeduplicationKey < ActiveRecord::Migration[5.1]
2
+ def change
3
+ remove_index :zizia_pre_ingest_works, :deduplication_key if index_exists?(:zizia_pre_ingest_works, :deduplication_key)
4
+
5
+ add_index :zizia_pre_ingest_works, :deduplication_key, unique: false
6
+ end
7
+ end
data/lib/zizia/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Zizia
4
- VERSION = '5.1.0'
4
+ VERSION = '5.2.0'
5
5
  end
@@ -1,4 +1,5 @@
1
1
  # Place any default configuration for solr_wrapper here
2
+ version: 4.7.5
2
3
  port: 8984
3
4
  enable_jms: false
4
5
  fcrepo_home_dir: tmp/fcrepo4-development-data
@@ -1,5 +1,5 @@
1
1
  # Place any default configuration for solr_wrapper here
2
- # version: 6.0.0
2
+ version: 7.7.2
3
3
  # port: 8983
4
4
  instance_dir: tmp/solr-development
5
5
  collection:
@@ -8,6 +8,10 @@ class Work < ActiveFedora::Base
8
8
  # self.valid_child_concerns = []
9
9
  validates :title, presence: { message: 'Your work must have a title.' }
10
10
 
11
+
12
+ property :deduplication_key, predicate: 'http://metadata.example.com/vocab/predicates#deduplicationKey', multiple: false do |index|
13
+ index.as :stored_searchable
14
+ end
11
15
  # This must be included at the end, because it finalizes the metadata
12
16
  # schema (by adding accepts_nested_attributes)
13
17
  include ::Hyrax::BasicMetadata
@@ -1,4 +1,5 @@
1
1
  #config/fcrepo_wrapper_test.yml.sample
2
2
  port: 8986
3
3
  enable_jms: false
4
+ version: 4.7.5
4
5
  fcrepo_home_dir: tmp/fcrepo4-test-data
@@ -1,5 +1,5 @@
1
1
  #config/solr_wrapper_test.yml
2
- version: 6.1.0
2
+ version: 7.7.2
3
3
  port: 8985
4
4
  instance_dir: tmp/solr-test
5
5
  collection:
@@ -33,6 +33,7 @@ RSpec.describe 'viewing the csv import detail page' do
33
33
  expect(page).to have_content('undetermined')
34
34
  click_on '1'
35
35
  expect(page).to have_content('Total Size')
36
+ expect(page).to have_content('Deduplication Key')
36
37
  end
37
38
 
38
39
  it 'has links to sort' do
@@ -246,9 +246,13 @@ RSpec.describe 'Importing records from a CSV file', :perform_jobs, :clean, type:
246
246
  # Viewing additional details after an import
247
247
  visit "/csv_import_details/index"
248
248
  expect(page).to have_content('Total Size')
249
- find(:xpath, '//*[@id="content-wrapper"]/div[2]/table/tbody/tr[3]/td[1]/a').click
249
+ click_on '4'
250
250
  expect(page).to have_content('dog.jpg')
251
+ expect(page).to have_content('cat.jpg')
251
252
  expect(page).to have_content('5.74 MB')
253
+ expect(page).to have_content('abc/123')
254
+ expect(page).to have_content('This work\'s metadata has not been indexed yet.')
255
+ expect(page).to have_content('Date Created')
252
256
  end
253
257
  end
254
258
  end
@@ -2,8 +2,23 @@
2
2
  require 'rails_helper'
3
3
 
4
4
  RSpec.describe Zizia::PreIngestWork do
5
- let(:pre_ingest_work) { FactoryBot.create(:pre_ingest_work, deduplication_key: "42") }
5
+ let(:pre_ingest_work) { FactoryBot.create(:pre_ingest_work, deduplication_key: '42') }
6
+ let(:pre_ingest_work_indexed) { FactoryBot.create(:pre_ingest_work, deduplication_key: '43') }
7
+ let(:work) { Work.new(title: ['A Nice Title'], deduplication_key: '43') }
8
+
9
+ before do
10
+ work.save
11
+ end
12
+
6
13
  it 'has a deduplication_key' do
7
- expect(pre_ingest_work.deduplication_key).to eq "42"
14
+ expect(pre_ingest_work.deduplication_key).to eq '42'
15
+ end
16
+
17
+ it 'can return that metadata has not been indexed yet' do
18
+ expect(pre_ingest_work.title).to eq('This work\'s metadata has not been indexed yet.')
19
+ end
20
+
21
+ it 'can return a title for a work based on the deduplication key' do
22
+ expect(pre_ingest_work_indexed.title).to eq('A Nice Title')
8
23
  end
9
24
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: zizia
3
3
  version: !ruby/object:Gem::Version
4
- version: 5.1.0
4
+ version: 5.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Data Curation Experts
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-11-04 00:00:00.000000000 Z
11
+ date: 2019-11-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: active-fedora
@@ -682,6 +682,7 @@ files:
682
682
  - app/uploaders/zizia/csv_manifest_uploader.rb
683
683
  - app/uploaders/zizia/csv_manifest_validator.rb
684
684
  - app/views/layouts/zizia/application.html.erb
685
+ - app/views/zizia/csv_import_details/_pre_ingest_files_table.html.erb
685
686
  - app/views/zizia/csv_import_details/_toggle_my_imports.html.erb
686
687
  - app/views/zizia/csv_import_details/index.html.erb
687
688
  - app/views/zizia/csv_import_details/show.html.erb
@@ -720,6 +721,8 @@ files:
720
721
  - db/migrate/20190923182946_add_update_actor_stack_to_csv_import_detail.rb
721
722
  - db/migrate/20191029162902_add_status_to_csv_import_detail.rb
722
723
  - db/migrate/20191031131041_add_deduplication_key_to_pre_ingest_work.rb
724
+ - db/migrate/20191105165518_add_uniqueness_constraint_to_pre_ingest_work_deduplication_key.rb
725
+ - db/migrate/20191105211127_remove_uniqueness_constraint_to_pre_ingest_work_deduplication_key.rb
723
726
  - docs/_config.yml
724
727
  - docs/customizing_metadata.md
725
728
  - docs/index.md
@@ -775,7 +778,6 @@ files:
775
778
  - spec/dummy/.rspec
776
779
  - spec/dummy/.ruby-version
777
780
  - spec/dummy/.solr_wrapper
778
- - spec/dummy/.solr_wrapper.yml
779
781
  - spec/dummy/Gemfile
780
782
  - spec/dummy/README.md
781
783
  - spec/dummy/Rakefile
@@ -1563,7 +1565,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
1563
1565
  - !ruby/object:Gem::Version
1564
1566
  version: '0'
1565
1567
  requirements: []
1566
- rubygems_version: 3.0.6
1568
+ rubygems_version: 3.0.4
1567
1569
  signing_key:
1568
1570
  specification_version: 4
1569
1571
  summary: Hyrax importers.
@@ -1,5 +0,0 @@
1
- # Place any default configuration for solr_wrapper here
2
- # port: 8983
3
- collection:
4
- dir: solr/conf/
5
- name: blacklight-core