rika-stevedore 1.1.4-java

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d5fd2d1d229514de3b11e87a02c06248fed2965a
+  data.tar.gz: 27a040889f25d447535a65af1240c9dc73b219b0
+SHA512:
+  metadata.gz: 3972d118579b7444160479976ad9eaa10a4f8ec07621486c568c856966a694191e4f2812431a8501e3f99a0aea53b52219cc144a721408c8785fbe342c65bc52
+  data.tar.gz: f302bccd1252ca66fdb7343e6783900a2a7b3c94731d34c521dc0a660d9988071324615a221209770366d19900e3cfa1f9409dea77784a31c95e0453f7380dfc

data/.gitignore

@@ -0,0 +1,21 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+
+.DS_Store
+projectFilesBackup/
+.idea/

data/.rspec

@@ -0,0 +1,2 @@
+--color
+--format progress

data/.travis.yml

@@ -0,0 +1,7 @@
+language: ruby
+rvm:
+  - jruby-19mode
+  - jruby-head
+notifications:
+  recipients:
+    - ricny046@gmail.com

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in rika.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 Richard Nyström
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,92 @@
+# Rika
+
+A JRuby wrapper for Apache Tika to extract text and metadata from various file formats.
+
+More information about Apache Tika can be found here: http://tika.apache.org/
+
+[![Code Climate](https://codeclimate.com/github/ricn/rika.png)](https://codeclimate.com/github/ricn/rika)
+[![Build Status](https://travis-ci.org/ricn/rika.png?branch=master)](https://travis-ci.org/ricn/rika)
+
+## Jeremy's modifications
+basically, just using my own version of Tika with special email parsing fixes, adds X-Attachments metadata key (listing attachment filenames from emails) and removes bouncycastle from Tika-parsers's requirements because everything is awful.
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'rika'
+
+Remember that this gem only works on JRuby.
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install rika
+
+## Usage
+
+For a quick start with the simplest use cases, the following functions
+are provided to get what you need in a single function call, for your convenience:
+
+```ruby
+require 'rika'
+
+content           = Rika.parse_content('document.pdf')    # string containing all content text
+metadata          = Rika.parse_metadata('document.pdf')   # hash containing the document metadata
+content, metadata = Rika.parse_content_and_metadata('document.pdf')   # both of the above
+```
+
+For other use cases and finer control, you can work directly with the Rika::Parser object:
+
+```ruby
+require 'rika'
+
+parser = Rika::Parser.new('document.pdf')
+
+# Return the content of the document:
+parser.content 
+
+# Return the media type for the document:
+parser.media_type 
+=> "application/pdf"
+
+# Return the metadata field title if it exists:
+parser.metadata["title"] if parser.metadata_exists?("title") 
+
+# Return all the available metadata keys that can be read from the document
+parser.available_metadata
+
+# Return only the first 10000 chars of the content:
+parser = Rika::Parser.new('document.pdf', 10000)
+parser.content # 10000 first chars returned
+
+# Return content from URL
+parser = Rika::Parser.new('http://riakhandbook.com/sample.pdf', 200)
+parser.content
+
+# Return the language for the content
+parser = parser = Rika::Parser.new('german document.pdf')
+parser.language
+=> "de"
+
+# Check whether the langugage identification is certain enough to be trusted
+parser.language_is_reasonably_certain?
+	
+```
+
+## Credits
+The following people have contributed ideas, documentation, or code to Rika:
+* Keith Bennett
+* Richard Nyström
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,11 @@
+require "bundler/gem_tasks"
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec
+
+desc 'Download jars'
+task :download_jars do
+	system "mvn dependency:copy-dependencies"
+end

data/lib/rika/version.rb

@@ -0,0 +1,3 @@
+module Rika
+  VERSION = "1.1.4"
+end

data/lib/rika.rb

@@ -0,0 +1,129 @@
+# encoding: utf-8
+
+raise "You need to run JRuby to use Rika" unless RUBY_PLATFORM =~ /java/
+
+require "rika/version"
+require 'uri'
+require 'net/http'
+require 'java'
+
+Dir[File.join(File.dirname(__FILE__), "../target/dependency/*.jar")].each do |jar|
+  require jar
+end
+
+$rika_tika ||= nil
+
+# Heavily based on the Apache Tika API: http://tika.apache.org/1.5/api/org/apache/tika/Tika.html
+module Rika
+  import org.apache.tika.metadata.Metadata
+  import org.apache.tika.Tika
+  import org.apache.tika.language.LanguageIdentifier
+  import org.apache.tika.detect.DefaultDetector
+  import java.io.FileInputStream
+  import java.net.URL
+
+  def self.parse_content_and_metadata(file_location, max_content_length = -1)
+    parser = Parser.new(file_location, max_content_length)
+    [parser.content, parser.metadata]
+  end
+
+  def self.parse_content(file_location, max_content_length = -1)
+    parser = Parser.new(file_location, max_content_length)
+    parser.content
+  end
+
+  def self.parse_metadata(file_location)
+    parser = Parser.new(file_location, 0)
+    parser.metadata
+  end
+
+  class Parser
+
+    def initialize(file_location, max_content_length = -1, detector = DefaultDetector.new)
+      @uri = file_location
+      $rika_tika = @tika = if $rika_tika.nil?
+                              puts "creating a new Tika"
+                              Tika.new(detector)
+                           else
+                              $rika_tika
+                           end
+      @tika.set_max_string_length(max_content_length)
+      @metadata_java = Metadata.new
+      @metadata_ruby = nil
+      @input_type = get_input_type
+    end
+
+    def content
+      self.parse
+      @content
+    end
+
+    def metadata
+      unless @metadata_ruby
+        self.parse
+        @metadata_ruby = {}
+
+        @metadata_java.names.each do |name|
+          @metadata_ruby[name] = @metadata_java.get(name)
+        end
+      end
+      @metadata_ruby
+    end
+
+    def media_type
+      if file?
+        @media_type ||= @tika.detect(java.io.File.new(@uri))
+      else
+        @media_type ||= @tika.detect(input_stream)
+      end
+    end
+
+    def available_metadata
+      metadata.keys
+    end
+
+    def metadata_exists?(name)
+      metadata[name] != nil
+    end
+
+    def file?
+      @input_type == :file
+    end
+
+    def language
+      @lang ||= LanguageIdentifier.new(content)
+
+      @lang.language
+    end
+
+    def language_is_reasonably_certain?
+      @lang ||= LanguageIdentifier.new(content)
+
+      @lang.is_reasonably_certain
+    end
+
+    protected
+
+    def parse
+      @content ||= @tika.parse_to_string(input_stream, @metadata_java).to_s.strip
+    end
+
+    def get_input_type
+      if File.exists?(@uri) && File.directory?(@uri) == false
+        :file
+      elsif URI(@uri).scheme.to_s.match(%r{https?})
+        :http
+      else
+        raise IOError, "Input (#{@uri}) is neither file nor http."
+      end
+    end
+
+    def input_stream
+      if file?
+        FileInputStream.new(java.io.File.new(@uri))
+      else # :http
+        URL.new(@uri).open_stream
+      end
+    end
+  end
+end

data/pom.xml

@@ -0,0 +1,20 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <name>Rika</name>
+
+  <groupId>org.rika</groupId>
+  <artifactId>Rika</artifactId>
+  <version>1.0-SNAPSHOT</version>
+  <packaging>jar</packaging>
+
+  <dependencies>
+    <dependency>
+      <groupId>org.apache.tika</groupId>
+      <artifactId>tika-parsers</artifactId>
+      <version>1.9</version>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+</project>

data/rika-stevedore.gemspec

@@ -0,0 +1,21 @@
+# -*- encoding: utf-8 -*-
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'rika/version'
+
+Gem::Specification.new do |gem|
+  gem.name          = "rika-stevedore"
+  gem.version       = Rika::VERSION
+  gem.authors       = ["Richard Nyström", "Jeremy B. Merrill"]
+  gem.email         = ["jeremybmerrill@gmail.com"]
+  gem.description   = %q{ A JRuby wrapper for Apache Tika to extract text and metadata from various file formats, slightly modified. }
+  gem.summary       = %q{ A JRuby wrapper for Apache Tika to extract text and metadata from various file formats, slightly modified. }
+  gem.homepage      = "https://github.com/jeremybmerrill/rika"
+  gem.files         = `git ls-files`.split($/)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.require_paths = ["lib"]
+  gem.add_development_dependency "rspec", "2.14.1"
+  gem.add_development_dependency "rake", "10.3.1"
+  gem.platform = "java"
+end

data/spec/fixtures/de.txt

@@ -0,0 +1 @@
+Er hörte leise Schritte hinter sich. Das bedeutete nichts Gutes. Wer würde ihm schon folgen, spät in der Nacht und dazu noch in dieser engen Gasse mitten im übel beleumundeten Hafenviertel? Gerade jetzt, wo er das Ding seines Lebens gedreht hatte und mit der Beute verschwinden wollte! Hatte einer seiner zahllosen Kollegen dieselbe Idee gehabt, ihn beobachtet und abgewartet, um ihn nun um die Früchte seiner Arbeit zu erleichtern? Oder gehörten die Schritte hinter ihm zu einem der unzähligen Gesetzeshüter dieser Stadt, und die stählerne Acht um seine Handgelenke würde gleich zuschnappen? Er konnte die Aufforderung stehen zu bleiben schon hören. Gehetzt sah er sich um. Plötzlich erblickte er den schmalen Durchgang. Blitzartig drehte er sich nach rechts und verschwand zwischen den beiden Gebäuden. Beinahe wäre er dabei über den umgestürzten Mülleimer gefallen, der mitten im Weg lag. Er versuchte, sich in der Dunkelheit seinen Weg zu ertasten und erstarrte: Anscheinend gab es keinen anderen Ausweg aus diesem kleinen Hof als den Durchgang, durch den er gekommen war. Die Schritte wurden lauter und lauter, er sah eine dunkle Gestalt um die Ecke biegen. Fieberhaft irrten seine Augen durch die nächtliche Dunkelheit und suchten einen Ausweg. War jetzt wirklich alles vorbei,

data/spec/fixtures/en.txt

@@ -0,0 +1 @@
+Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts. Separated they live in Bookmarksgrove right at the coast of the Semantics, a large language ocean. A small river named Duden flows by their place and supplies it with the necessary regelialia. It is a paradisematic country, in which roasted parts of sentences fly into your mouth. Even the all-powerful Pointing has no control about the blind texts it is an almost unorthographic life One day however a small line of blind text by the name of Lorem Ipsum decided to leave for the far World of Grammar. The Big Oxmox advised her not to do so, because there were thousands of bad Commas, wild Question Marks and devious Semikoli, but the Little Blind Text didn’t listen. She packed her seven versalia, put her initial into the belt and made herself on the way. When she reached the first hills of the Italic Mountains, she had a last view back on the skyline of her hometown Bookmarksgrove, the headline of Alphabet Village and the subline of her own road, the Line Lane. Pityful a rethoric question ran over her cheek, then

data/spec/fixtures/es.txt

@@ -0,0 +1 @@
+Una mañana, tras un sueño intranquilo, Gregorio Samsa se despertó convertido en un monstruoso insecto. Estaba echado de espaldas sobre un duro caparazón y, al alzar la cabeza, vio su vientre convexo y oscuro, surcado por curvadas callosidades, sobre el que casi no se aguantaba la colcha, que estaba a punto de escurrirse hasta el suelo. Numerosas patas, penosamente delgadas en comparación con el grosor normal de sus piernas, se agitaban sin concierto. - ¿Qué me ha ocurrido? No estaba soñando. Su habitación, una habitación normal, aunque muy pequeña, tenía el aspecto habitual. Sobre la mesa había desparramado un muestrario de paños - Samsa era viajante de comercio-, y de la pared colgaba una estampa recientemente recortada de una revista ilustrada y puesta en un marco dorado. La estampa mostraba a una mujer tocada con un gorro de pieles, envuelta en una estola también de pieles, y que, muy erguida, esgrimía un amplio manguito, asimismo de piel, que ocultaba todo su antebrazo. Gregorio miró hacia la ventana; estaba nublado, y sobre el cinc del alféizar repiqueteaban las gotas de lluvia, lo que le hizo sentir una gran melancolía. «Bueno -pensó-; ¿y si siguiese durmiendo un rato y me olvidase de

data/spec/fixtures/fr.txt

@@ -0,0 +1 @@
+En se réveillant un matin après des rêves agités, Gregor Samsa se retrouva, dans son lit, métamorphosé en un monstrueux insecte. Il était sur le dos, un dos aussi dur qu’une carapace, et, en relevant un peu la tête, il vit, bombé, brun, cloisonné par des arceaux plus rigides, son abdomen sur le haut duquel la couverture, prête à glisser tout à fait, ne tenait plus qu’à peine. Ses nombreuses pattes, lamentablement grêles par comparaison avec la corpulence qu’il avait par ailleurs, grouillaient désespérément sous ses yeux.« Qu’est-ce qui m’est arrivé ? » pensa-t-il. Ce n’était pas un rêve. Sa chambre, une vraie chambre humaine, juste un peu trop petite, était là tranquille entre les quatre murs qu’il connaissait bien. Au-dessus de la table où était déballée une collection d’échantillons de tissus - Samsa était représentant de commerce - on voyait accrochée l’image qu’il avait récemment découpée dans un magazine et mise dans un joli cadre doré. Elle représentait une dame munie d’une toque et d’un boa tous les deux en fourrure et qui, assise bien droite, tendait vers le spectateur un lourd manchon de fourrure où tout son avant-bras avait disparu. Le regard de Gregor se tourna ensuite vers

data/spec/fixtures/lang_cant_be_determined.txt

@@ -0,0 +1 @@
+hej