hydra 7.0.0 → 7.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 57ba23f57ce7053f1ba3ba3b46c306ffb2cc12ad
-  data.tar.gz: 2e133722302297e19f5fbebd4a2780721395e7c6
+  metadata.gz: af11fbafc2cb92fb77c3cf7c82f4c16e6b70b349
+  data.tar.gz: 6af18c8e04cca1800ad03d2461f27adffd574ea7
 SHA512:
-  metadata.gz: c2c93c5b5b2e8ac5a8328aceace75d5fc112bbde2cc1b532d3b36bc8eaf94f7c312c40c1f2095769684ec9ada0f4b882c83c82e1596bc147478f745fd6971cc4
-  data.tar.gz: 3ca2915c65fd0f0cab91ad9fe5534f6ec618ce5c926c5912dcb5fcccb7760d7c567e63e3cdf0bffff5d5c76357ca1057344381f10fd0fb08c845745605f8b035
+  metadata.gz: 3760f9be5683badf70a974e841dd3555a200cbeb5d79049b22b9d7a280e46e4e76c26263626388cb13854790c9f7a8d473fb1680d10d0fce6f62bfbd46653062
+  data.tar.gz: 72d8d542411398662cb402a3e567530749322a51649bfeb26475a4150ed06d488ead9213b6aa785b3739cdd86a9a42e953b106c34f2237a5620b5d7fc9fa11c9

data/doc/Hydra-Recipes.md

@@ -1 +1,9 @@
-[[Use HTTP POST for Solr requests]] - This may be necessary if Hydra's Solr requests, due to access control conditions, become too long for the default HTTP GET method.
+**Caveat emptor:** These "recipes" are community contributions, not officially supported Hydra "solutions". YMMV.
+
+***
+
+[[Use HTTP POST for Solr requests]] - This may be necessary if Hydra's Solr requests, due to access control conditions, become too long for the default HTTP GET method.
+
+[[Indexing non English content]] - if you are indexing non English content into Hydra, you may want to update your Solr config to take advantage of Solr's language specific stemming capabilities.
+
+[Uniqueness validator for ActiveFedora](https://gist.github.com/dchandekstark/f969ad21bf518c7cd3c5) - Inspired by the ActiveRecord uniqueness validator, with some limitations.

data/doc/Indexing-non-English-content.md

@@ -0,0 +1,45 @@
+_Note - the information in this page only refers to our (The Royal Library of Denmark) experience with this problem. Please feel free to edit or update this page with corrections or additional information._
+
+## Background
+
+By default, Hydra will index all text content as Solr dynamic fields of type `*_tesim`. 
+```xml
+<dynamicField name="*_tesim" type="text_en" stored="true" indexed="true" multiValued="true"/>
+```
+This means that all text stored like this will be indexed according to the rules specified in the ```text_en``` field type. This is defined to use stemming rules appropriate for the English language. For example, the text `appointment` will also be stored as `appoint` and will be retrievable by searches for both values.
+
+Obviously, this is inappropriate if your Hydra head will store content in a language other than English as users will need to specify the exact text string they are searching for in order to retrieve content. To give an example from our case, the search `Minister` will not retrieve documents with titles such as `Ministeren` (Danish, the minister).
+
+## A quick and dirty solution
+
+The dynamic field name `*_tesim` is generated by Solrizer. The optimal solution would be to pass Solrizer extra arguments when calling it in order to generate a different type of dynamic field which would in turn refer to a different Solr field type. I couldn't find any obvious way to do this, so instead I ended up customising the `text_en` field type as follows:
+
+```xml
+    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.ICUTokenizerFactory"/>
+        <filter class="solr.ICUFoldingFilterFactory"/> <!-- NFKC, case folding, diacritics removed -->
+        <filter class="solr.SnowballPorterFilterFactory" language="Danish"/>
+        <filter class="solr.TrimFilterFactory"/>
+      </analyzer>
+    </fieldType>
+```
+Here, I have removed the English specific stemming filters and added a filter with a Danish configuration. A huge number of different languages are supported by Solr without any extra configuration needed. See the [Language Analysis](https://wiki.apache.org/solr/LanguageAnalysis) page in the Solr Wiki and look under your language to see if it is supported.
+
+If your content is already indexed in Solr, you can re-index without needing to re-import. Simply restart Solr with the new configuration, log into a rails console for the appropriate environment and enter:
+```ruby
+ActiveFedora::Base.all.each{ |e| e.update_index }
+```
+This will run through all objects in your repository and update the index according to the new configuration. It may take a bit of time if you have a lot of content in your repository.
+
+## A better solution?
+
+The above solution is problematic in that it modifies the `text_en` field type to store non-English content. This is a bit confusing. A better solution would be to define a new field type e.g. `text_da` containing the same values which can be referenced from the `*_tesim` dynamic field definition e.g.
+```xml
+<dynamicField name="*_tesim" type="text_da" stored="true" indexed="true" multiValued="true"/>
+```
+Alternatively, if Solrizer can be called to generate custom fields type, it should be utilised to generate a custom dynamic field such as `*_tdsim` which in turn references the `text_da` field type. I don't know how to do this, but anyone who does is more than welcome to update this guide with that information.
+
+## Finally...
+
+In writing this documentation, I discovered that [Solr's example schema](http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_3_6/solr/example/solr/conf/schema.xml) contains example field configurations for a wide range of different languages which are more detailed than the example I have provided above. Try and apply these configurations for your language and see if they work as expected. Please note however that I have not tried these examples myself, so I cannot promise that they will work with the Solr shipped with Jetty.

data/doc/Lesson:-Define-Relationships-Between-Objects.md

@@ -121,9 +121,9 @@ The answer is that active-fedora has a config file that it uses to look up RDF p
 
 Now that we've added page relationships, it's a great time to commit to git:
 
-```bash
-$> git add .
-$> git commit -m "Created a book page model with relationship to the book model"
+```text
+git add .
+git commit -m "Created a book page model with relationship to the book model"
 ```
 
 # Next Step

data/doc/Lesson:-Generate-Rails-Scaffolding-for-Creating-and-Editing-Books.md

@@ -61,7 +61,7 @@ Explore the pages for creating, editing and showing Books.
 
 ```text
 git add .
-git commit -m "Ran scaffold generator"
+git commit -m "Ran Book scaffold generator"
 ```
 
 ### Step 4: Make the Display view show Authors as a multi-valued field

data/doc/Lesson:-adding-content-datastreams.md

@@ -34,8 +34,8 @@ To add the file to one of our page objects, open up the console again:
 Now you're ready to add the file.  Choose a file on your computer that you want to add as the "pageContent".  In the lines below we're pretending that the path to the file is "/Users/adamw/Desktop/page1.pdf".  Replace that with the correct local path for the file you want to use.
 
 ```ruby
- > p.pageContent.content = File.open("/Users/adamw/Desktop/page1.pdf")
- => #<File:/Users/adamw/Desktop/page1.pdf> 
+ > p.pageContent.content = File.open("../AK Page 4.pdf")
+ => #<File:../AK Page 4.pdf>  
  > p.save
  => true
 ```
@@ -49,8 +49,8 @@ Now if you go to [[http://localhost:8983/fedora/objects/changeme:2/datastreams]]
 Now that we've added a content datastream, it's a great time to commit to git:
 
 ```bash
-$> git add .
-$> git commit -m "Created a content datastream"
+git add .
+git commit -m "Created a content datastream"
 ```
 
 # Next Step

data/doc/Lesson:-build-a-book-model.md

@@ -254,9 +254,9 @@ Now your object is indexed properly, but it **won't show up in Blacklight's sear
 
 Now that we've got our model working, it's a great time to commit to git:
 
-```bash
-$> git add .
-$> git commit -m "Created a book model and a datastream"
+```text
+git add .
+git commit -m "Created a book model and a datastream"
 ```
 
 # Next Step

data/doc/Lesson:-install-hydra-jetty.md

@@ -25,6 +25,12 @@ Use the hydra:jetty generator to install the hydra-jetty package by running:
 rails g hydra:jetty
 ```
 
+Note: this requires that your system have curl installed. If it does not, you may see an unhelpful error:
+
+```text
+Unable to download jetty from https://github.com/projecthydra/hydra-jetty/archive/v7.0.0.zip
+```
+
 This generator is provided by the jettywrapper gem.
 
 

data/doc/Lesson:-make-blacklight-return-search-results.md

@@ -59,9 +59,9 @@ Save the file, and refresh your web browser. You should now see a result for "An
 
 Now that we've updated our search functionality, it's a great time to commit to git:
 
-```bash
-$> git add .
-$> git commit -m "Disabled access controls and set default search fields"
+```text
+git add .
+git commit -m "Disabled access controls and set default search fields"
 ```
 
 # Next Step

data/hydra.gemspec

@@ -23,16 +23,16 @@ Gem::Specification.new do |gem|
   gem.require_paths = ["lib"]
   gem.license = 'APACHE2'
 
-  gem.add_dependency 'hydra-head', '~> 7.0.1'
-  gem.add_dependency 'jettywrapper', '~> 1.7.0'
-  gem.add_dependency 'active-fedora', '~> 7.0.2'
+  gem.add_dependency 'hydra-head', '~> 7.2.0'
+  gem.add_dependency 'jettywrapper', '~> 1.8.2'
+  gem.add_dependency 'active-fedora', '~> 7.1.0'
   gem.add_dependency 'rails', '>= 3.2.15', '< 5.0'
-  gem.add_dependency 'om', '~> 3.0.4'
-  gem.add_dependency 'solrizer', '~> 3.1.1'
+  gem.add_dependency 'om', '~> 3.1.0'
+  gem.add_dependency 'solrizer', '~> 3.3.0'
   gem.add_dependency 'rsolr', '~> 1.0.10'
-  gem.add_dependency 'blacklight', '~> 5.4.0'
+  gem.add_dependency 'blacklight', '~> 5.5.1'
   gem.add_dependency 'nokogiri', '~> 1.6.0'
-  gem.add_dependency 'rubydora', '~> 1.7.4'
+  gem.add_dependency 'rubydora', '~> 1.8.0'
   gem.add_dependency 'nom-xml', '~> 0.5.1'
   gem.add_development_dependency 'github_api', '~> 0.10.1'
 end

data/lib/hydra/version.rb

@@ -1,3 +1,3 @@
 module Hydra
-  VERSION = "7.0.0"
+  VERSION = "7.1.0"
 end

data/script/changelog.sh

@@ -2,8 +2,7 @@
 
 function show_help() {
   echo "Usage: changelog.sh [options]"
-  echo "Generates a changelog from git history, ommitting commit messages that"
-  echo "  are merges or contain the following: \"$skip_tag\""
+  echo "Generates a changelog from git history"
   echo
   echo "Format:"
   echo "YYYY-MM-DD: commit subject [committer name]"
@@ -19,7 +18,6 @@ function show_help() {
 verbose=0
 range_parameter=0
 banner=0
-skip_tag="\[log skip\]"
 repository_path="./"
 
 function default_range() {
@@ -75,12 +73,8 @@ function get_format() {
 pretty_format=`get_format`
 
 function changelog() {
-  # Get a list of all SHA1 commits
-  #   Filter the list to exclude all SHA1 commits with $skip_tag
-  #   Then requery the log and output format
-  cd $repository_path && git log $range --no-merges --format=%H $@ |
-    grep -v -f <(cd $repository_path && git log $range --no-merges --format=%H --grep="$skip_tag" $@) |
-    git log $range --no-merges --pretty="$pretty_format" --date=short --stdin --no-walk
+  # Get a list of all commits for the ranger that were not merges
+  cd $repository_path && git log $range --no-merges --pretty="$pretty_format" --date=short
 }
 
 function main() {

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: hydra
 version: !ruby/object:Gem::Version
-  version: 7.0.0
+  version: 7.1.0
 platform: ruby
 authors:
 - Jeremy Friesen
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-06 00:00:00.000000000 Z
+date: 2014-08-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: hydra-head
@@ -17,42 +17,42 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 7.0.1
+        version: 7.2.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 7.0.1
+        version: 7.2.0
 - !ruby/object:Gem::Dependency
   name: jettywrapper
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.7.0
+        version: 1.8.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.7.0
+        version: 1.8.2
 - !ruby/object:Gem::Dependency
   name: active-fedora
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 7.0.2
+        version: 7.1.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 7.0.2
+        version: 7.1.0
 - !ruby/object:Gem::Dependency
   name: rails
   requirement: !ruby/object:Gem::Requirement
@@ -79,28 +79,28 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.0.4
+        version: 3.1.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.0.4
+        version: 3.1.0
 - !ruby/object:Gem::Dependency
   name: solrizer
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.1.1
+        version: 3.3.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 3.1.1
+        version: 3.3.0
 - !ruby/object:Gem::Dependency
   name: rsolr
   requirement: !ruby/object:Gem::Requirement
@@ -121,14 +121,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.4.0
+        version: 5.5.1
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 5.4.0
+        version: 5.5.1
 - !ruby/object:Gem::Dependency
   name: nokogiri
   requirement: !ruby/object:Gem::Requirement
@@ -149,14 +149,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.7.4
+        version: 1.8.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.7.4
+        version: 1.8.0
 - !ruby/object:Gem::Dependency
   name: nom-xml
   requirement: !ruby/object:Gem::Requirement
@@ -209,6 +209,7 @@ files:
 - doc/For-Developers.md
 - doc/Home.md
 - doc/Hydra-Recipes.md
+- doc/Indexing-non-English-content.md
 - doc/Lesson:-Define-Relationships-Between-Objects.md
 - doc/Lesson:-Generate-Rails-Scaffolding-for-Creating-and-Editing-Books.md
 - doc/Lesson:-Reading-Hydra-rightsMetadata-XML.md