rika 1.6.0-java → 2.0.0-java

data/spec/fixtures/de.txt

@@ -1 +1,21 @@
-Er hörte leise Schritte hinter sich. Das bedeutete nichts Gutes. Wer würde ihm schon folgen, spät in der Nacht und dazu noch in dieser engen Gasse mitten im übel beleumundeten Hafenviertel? Gerade jetzt, wo er das Ding seines Lebens gedreht hatte und mit der Beute verschwinden wollte! Hatte einer seiner zahllosen Kollegen dieselbe Idee gehabt, ihn beobachtet und abgewartet, um ihn nun um die Früchte seiner Arbeit zu erleichtern? Oder gehörten die Schritte hinter ihm zu einem der unzähligen Gesetzeshüter dieser Stadt, und die stählerne Acht um seine Handgelenke würde gleich zuschnappen? Er konnte die Aufforderung stehen zu bleiben schon hören. Gehetzt sah er sich um. Plötzlich erblickte er den schmalen Durchgang. Blitzartig drehte er sich nach rechts und verschwand zwischen den beiden Gebäuden. Beinahe wäre er dabei über den umgestürzten Mülleimer gefallen, der mitten im Weg lag. Er versuchte, sich in der Dunkelheit seinen Weg zu ertasten und erstarrte: Anscheinend gab es keinen anderen Ausweg aus diesem kleinen Hof als den Durchgang, durch den er gekommen war. Die Schritte wurden lauter und lauter, er sah eine dunkle Gestalt um die Ecke biegen. Fieberhaft irrten seine Augen durch die nächtliche Dunkelheit und suchten einen Ausweg. War jetzt wirklich alles vorbei,
+Eines verschneiten Abends im Wald verhaltend (German)
+
+Wohl weiß ich, wer den Wald besitzt,
+Sein Haus jedoch im Dorf dort ist;
+Er merkt nicht, wie ich hier verharr'
+Zu schaun, wie Wald den Schnee begrüßt.
+
+Mein Pferdchen denkt, ich wär ein Narr,
+Zu halten, wo nicht Haus noch Farm,
+Im Wald, nah beim gefrornen Teich,
+Am finstersten Abend im Jahr.
+
+Des Zaumzeugs Glöckchen klingen leis:
+Es fragt: was ist? geht's weiter gleich?
+Ansonsten hört man nur ganz sacht
+Den Schneefall und den Wind, der streicht.
+
+Der Wald ist nett, tief wie die Nacht,
+Doch halte ich, was ich versprach,
+Und Meilen sind's noch bis zum Schlaf,
+Und Meilen sind's noch bis zum Schlaf.

data/spec/fixtures/document.txt

@@ -0,0 +1,23 @@
+Stopping by Woods on a Snowy Evening
+
+By Robert Frost
+
+Whose woods these are I think I know.   
+His house is in the village though;   
+He will not see me stopping here   
+To watch his woods fill up with snow.   
+
+My little horse must think it queer   
+To stop without a farmhouse near   
+Between the woods and frozen lake   
+The darkest evening of the year.   
+
+He gives his harness bells a shake   
+To ask if there is some mistake.   
+The only other sound’s the sweep   
+Of easy wind and downy flake.   
+
+The woods are lovely, dark and deep,   
+But I have promises to keep,   
+And miles to go before I sleep,   
+And miles to go before I sleep.

data/spec/fixtures/en.txt

@@ -1 +1,23 @@
-Far far away, behind the word mountains, far from the countries Vokalia and Consonantia, there live the blind texts. Separated they live in Bookmarksgrove right at the coast of the Semantics, a large language ocean. A small river named Duden flows by their place and supplies it with the necessary regelialia. It is a paradisematic country, in which roasted parts of sentences fly into your mouth. Even the all-powerful Pointing has no control about the blind texts it is an almost unorthographic life One day however a small line of blind text by the name of Lorem Ipsum decided to leave for the far World of Grammar. The Big Oxmox advised her not to do so, because there were thousands of bad Commas, wild Question Marks and devious Semikoli, but the Little Blind Text didn’t listen. She packed her seven versalia, put her initial into the belt and made herself on the way. When she reached the first hills of the Italic Mountains, she had a last view back on the skyline of her hometown Bookmarksgrove, the headline of Alphabet Village and the subline of her own road, the Line Lane. Pityful a rethoric question ran over her cheek, then
+Stopping by Woods on a Snowy Evening
+
+By Robert Frost
+
+Whose woods these are I think I know.   
+His house is in the village though;   
+He will not see me stopping here   
+To watch his woods fill up with snow.   
+
+My little horse must think it queer   
+To stop without a farmhouse near   
+Between the woods and frozen lake   
+The darkest evening of the year.   
+
+He gives his harness bells a shake   
+To ask if there is some mistake.   
+The only other sound’s the sweep   
+Of easy wind and downy flake.   
+
+The woods are lovely, dark and deep,   
+But I have promises to keep,   
+And miles to go before I sleep,   
+And miles to go before I sleep.

data/spec/fixtures/es.txt

@@ -1 +1,21 @@
-Una mañana, tras un sueño intranquilo, Gregorio Samsa se despertó convertido en un monstruoso insecto. Estaba echado de espaldas sobre un duro caparazón y, al alzar la cabeza, vio su vientre convexo y oscuro, surcado por curvadas callosidades, sobre el que casi no se aguantaba la colcha, que estaba a punto de escurrirse hasta el suelo. Numerosas patas, penosamente delgadas en comparación con el grosor normal de sus piernas, se agitaban sin concierto. - ¿Qué me ha ocurrido? No estaba soñando. Su habitación, una habitación normal, aunque muy pequeña, tenía el aspecto habitual. Sobre la mesa había desparramado un muestrario de paños - Samsa era viajante de comercio-, y de la pared colgaba una estampa recientemente recortada de una revista ilustrada y puesta en un marco dorado. La estampa mostraba a una mujer tocada con un gorro de pieles, envuelta en una estola también de pieles, y que, muy erguida, esgrimía un amplio manguito, asimismo de piel, que ocultaba todo su antebrazo. Gregorio miró hacia la ventana; estaba nublado, y sobre el cinc del alféizar repiqueteaban las gotas de lluvia, lo que le hizo sentir una gran melancolía. «Bueno -pensó-; ¿y si siguiese durmiendo un rato y me olvidase de
+Un alto en el bosque mientras nieva
+
+De quién es este bosque, saber creo
+-en el poblado su morada veo-
+no habrá de sorprenderme contemplando
+cubrir su bosque el invernal blanqueo.
+
+Mi caballito se dirá extrañado
+que, sin granja cercana, hemos parado
+de este año en la tarde más oscura,
+entre el bosque y el lago congelado.
+
+Sacudiéndose, agita su cencerro
+preguntando quizá: -¿será algún yerro?
+Sólo el cierzo y los copos rumorean
+blandamente del bosque en el encierro.
+
+Yo, el bosque hondo y fusco veo risueño...
+Mas, en cumplir promesas tengo empeño,
+y millas debo andar antes del sueño,
+un largo andar para llegar al sueño.

data/spec/fixtures/fr.txt

@@ -1 +1,23 @@
-En se réveillant un matin après des rêves agités, Gregor Samsa se retrouva, dans son lit, métamorphosé en un monstrueux insecte. Il était sur le dos, un dos aussi dur qu’une carapace, et, en relevant un peu la tête, il vit, bombé, brun, cloisonné par des arceaux plus rigides, son abdomen sur le haut duquel la couverture, prête à glisser tout à fait, ne tenait plus qu’à peine. Ses nombreuses pattes, lamentablement grêles par comparaison avec la corpulence qu’il avait par ailleurs, grouillaient désespérément sous ses yeux.« Qu’est-ce qui m’est arrivé ? » pensa-t-il. Ce n’était pas un rêve. Sa chambre, une vraie chambre humaine, juste un peu trop petite, était là tranquille entre les quatre murs qu’il connaissait bien. Au-dessus de la table où était déballée une collection d’échantillons de tissus - Samsa était représentant de commerce - on voyait accrochée l’image qu’il avait récemment découpée dans un magazine et mise dans un joli cadre doré. Elle représentait une dame munie d’une toque et d’un boa tous les deux en fourrure et qui, assise bien droite, tendait vers le spectateur un lourd manchon de fourrure où tout son avant-bras avait disparu. Le regard de Gregor se tourna ensuite vers
+En s'arrêtant par les bois un soir de neige
+
+Auteur: Robert Frost
+
+À qui sont ces bois, je crois que je sais.
+Sa maison est au village pourtant ;
+Il ne me verra pas m'arrêter ici
+À regarder ses bois recouverts de neige.
+
+Mon petit cheval doit trouver cela étrange
+De s'arrêter loin de toute ferme aux alentours
+Entre ces bois et ce lac gelé
+Au soir le plus sombre de l'année.
+
+Il fait tinter les clochettes de son harnais en tirant dessus
+Comme pour demander s'il n'y a pas là quelque erreur.
+Les seuls autres bruits le souffle
+D'une brise légère et le son duveteux des flocons.
+
+Les bois sont beaux, sombres et profonds,
+Mais j'ai des promesses à tenir
+Et un long chemin à parcourir avant de dormir,
+Et un long chemin à parcourir avant de dormir.

data/spec/fixtures/ru.txt

@@ -1 +1,21 @@
-Любя, съешь щипцы, — вздохнёт мэр, — кайф жгуч. Шеф взъярён тчк щипцы с эхом гудбай Жюль. Эй, жлоб! Где туз? Прячь юных съёмщиц в шкаф. Экс-граф? Плюш изъят. Бьём чуждый цен хвощ! Эх, чужак! Общий съём цен шляп (юфть) — вдрызг! Любя, съешь щипцы, — вздохнёт мэр, — кайф жгуч. Шеф взъярён тчк щипцы с эхом гудбай Жюль. Эй, жлоб! Где туз? Прячь юных съёмщиц в шкаф. Экс-граф? Плюш изъят. Бьём чуждый цен хвощ! Эх, чужак! Общий съём цен шляп (юфть) — вдрызг! Любя, съешь щипцы, — вздохнёт мэр, — кайф жгуч. Шеф взъярён тчк щипцы с эхом гудбай Жюль. Эй, жлоб! Где туз? Прячь юных съёмщиц в шкаф. Экс-граф? Плюш изъят. Бьём чуждый цен хвощ! Эх, чужак! Общий съём цен шляп (юфть) — вдрызг! Любя, съешь щипцы, — вздохнёт мэр, — кайф жгуч. Шеф взъярён тчк щипцы с эхом гудбай Жюль. Эй, жлоб! Где туз? Прячь юных съёмщиц в шкаф. Экс-граф? Плюш изъят. Бьём чуждый цен хвощ! Эх, чужак! Общий съём цен шляп (юфть) — вдрызг!Любя, съешь щипцы, — вздохнёт мэр, — кайф жгуч. Шеф взъярён тчк щипцы с эхом гудбай Жюль. Эй, жлоб! Где туз? Прячь юных съёмщиц в шкаф. Экс-граф? Плюш изъят. Бьём чуждый цен
+Остановившись у леса снежным вечером
+
+Чей лес, мне кажется, я знаю:
+в селе живет его хозяин.
+Он не увидит, как на снежный
+я лес его стою взираю.
+
+В недоуменье конь, конечно,
+зачем в ночи за год темнейшей
+мы стали там, где нет жилья,
+у леса с озером замерзшим.
+
+Он, бубенцом слегка звеня,
+как будто бы корит меня,
+да веет слабый ветерок,
+пушистым снегом шелестя.
+
+Лес сладок, темен и глубок,
+но в путь пора мне — долг есть долг.
+И ехать долго — сон далек,
+и ехать долго — сон далек.

data/spec/fixtures/tiny.txt

@@ -0,0 +1 @@
+hello

data/spec/rika/cli/args_parser_spec.rb

@@ -0,0 +1,117 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+require 'rika/cli/args_parser'
+
+describe ArgsParser do
+  let(:versions_regex) { /Versions:.*Rika: (\d+\.\d+\.\d+(-\w+)?).*Tika: (\d+\.\d+\.\d+(-\w+)?)/ }
+
+  specify 'returns a hash of options, a target array, and help text' do
+    options, targets, help_text = described_class.call([])
+    expect(options).to be_a(Hash)
+    expect(targets).to be_an(Array)
+    expect(help_text).to be_a(String)
+  end
+
+  context 'when parsing options' do
+    RSpec.shared_examples 'sets_options_correctly' do |args, option_key, expected_value|
+      specify "correctly sets #{option_key} to #{expected_value} when args are #{args}" do
+        options, _, _ = described_class.call(args)
+        expect(options[option_key]).to eq(expected_value)
+      end
+    end
+
+    # Test default option values:
+    include_examples('sets_options_correctly', [], :as_array, false)
+    include_examples('sets_options_correctly', [], :text, true)
+    include_examples('sets_options_correctly', [], :metadata, true)
+    include_examples('sets_options_correctly', [], :format, 'at')
+    include_examples('sets_options_correctly', [], :key_sort, true)
+    include_examples('sets_options_correctly', [], :source, true)
+
+    # Test -a as_array option:
+    include_examples('sets_options_correctly', %w[-a], :as_array, true)
+    include_examples('sets_options_correctly', %w[--as_array], :as_array, true)
+    include_examples('sets_options_correctly', %w[-a -a-], :as_array, false)
+    include_examples('sets_options_correctly', %w[--no-as_array], :as_array, false)
+
+    # Test -f format option:
+    include_examples('sets_options_correctly', %w[-fyy], :format, 'yy')
+    include_examples('sets_options_correctly', %w[--format yy], :format, 'yy')
+    include_examples('sets_options_correctly', %w[-f yy], :format, 'yy')
+    include_examples('sets_options_correctly', %w[-f y], :format, 'yy')
+    include_examples('sets_options_correctly', %w[-f yj], :format, 'yj')
+    include_examples('sets_options_correctly', %w[-f yjJ], :format, 'yj') # Test extra characters after valid format
+
+    # Test -m metadata option:
+    include_examples('sets_options_correctly', %w[-m- -m], :metadata, true)
+    include_examples('sets_options_correctly', %w[-m- -m+], :metadata, true)
+    include_examples('sets_options_correctly', %w[--metadata false --metadata], :metadata, true)
+    include_examples('sets_options_correctly', %w[-m -m-], :metadata, false)
+    include_examples('sets_options_correctly', %w[-m yes], :metadata, true)
+    include_examples('sets_options_correctly', %w[-m no], :metadata, false)
+    include_examples('sets_options_correctly', %w[-m true], :metadata, true)
+    include_examples('sets_options_correctly', %w[-m false], :metadata, false)
+    include_examples('sets_options_correctly', %w[--metadata false], :metadata, false)
+    include_examples('sets_options_correctly', %w[--no-metadata], :metadata, false)
+
+    # Test -t text option:
+    include_examples('sets_options_correctly', %w[-t], :text, true)
+    include_examples('sets_options_correctly', %w[-t -t-], :text, false)
+    include_examples('sets_options_correctly', %w[-t yes], :text, true)
+    include_examples('sets_options_correctly', %w[-t no], :text, false)
+    include_examples('sets_options_correctly', %w[-t true], :text, true)
+    include_examples('sets_options_correctly', %w[-t false], :text, false)
+    include_examples('sets_options_correctly', %w[--text false], :text, false)
+    include_examples('sets_options_correctly', %w[--text false --text], :text, true)
+
+    # Test -k key sort option:
+    include_examples('sets_options_correctly', %w[-k-], :key_sort, false)
+
+    # Test -s source option:
+    include_examples('sets_options_correctly', %w[-s-], :source, false)
+  end
+
+  describe '#versions_string' do
+    specify 'returns a Rika version and a Tika version' do
+      expect(described_class.new.send(:versions_string)).to match(versions_regex)
+    end
+  end
+
+  context 'when processing environment variables' do
+    it 'adds arguments from the environment to the args list' do
+      args_parser = described_class.new
+      allow(args_parser).to receive(:environment_options).and_return('-t-')
+      options, _, _ = args_parser.call([])
+      expect(options[:text]).to be(false)
+    end
+
+    it 'overrides environment variable options with command line options' do
+      env_format_arg = '-fyy'
+      cmd_line_format = 'JJ'
+      cmd_line_args = ["-f#{cmd_line_format}"]
+      args_parser = described_class.new
+      allow(args_parser).to receive(:environment_options).and_return(env_format_arg)
+      options, _, _ = args_parser.call(cmd_line_args)
+      expect(options[:format]).to eq(cmd_line_format)
+    end
+  end
+
+  describe 'DEFAULT_OPTIONS hash' do
+    specify 'has the correct default values' do
+      expect(described_class::DEFAULT_OPTIONS).to eq(
+        as_array: false,
+        text: true,
+        metadata: true,
+        format: 'at',
+        key_sort: true,
+        source: true
+      )
+    end
+
+    specify 'is frozen' do
+      expect(described_class::DEFAULT_OPTIONS).to be_frozen
+    end
+  end
+end

data/spec/rika/cli/rika_command_spec.rb

@@ -0,0 +1,120 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+require 'rika/cli/rika_command'
+
+RF = Rika::Formatters
+
+describe RikaCommand do
+  let(:versions_regex) { /Versions:.*Rika: (\d+\.\d+\.\d+(-\w+)?).*Tika: (\d+\.\d+\.\d+(-\w+)?)/ }
+
+  before do
+    @original_stdout = $stdout
+    @original_stderr = $stderr
+    $stdout = StringIO.new
+    $stderr = StringIO.new
+  end
+
+  after do
+    $stdout = @original_stdout
+    $stderr = @original_stderr
+  end
+
+  describe '#call' do
+    specify 'call should run the command without error' do
+      expect { described_class.new([fixture_path('tiny.txt')]).call }.not_to raise_error
+    end
+
+    specify 'prints version and exits when -v or --version is specified' do
+      expect { described_class.new(%w[-v]).call }.to output(versions_regex).to_stdout.and raise_error(SystemExit)
+    end
+
+    specify 'prints help and exits when -h or --help is specified' do
+      regex = /Usage: rika \[options\] <file or url> /m
+      expect { described_class.new(%w[-h]).call }.to output(regex).to_stdout.and raise_error(SystemExit)
+    end
+
+    specify 'when run in array mode, outputs the string representation of an array of parse results' do
+      original_stdout = $stdout
+      $stdout = StringIO.new
+      begin
+        tiny_filespec = fixture_path('tiny.txt')
+        args = ['-a', '-fJ', tiny_filespec, tiny_filespec]
+        described_class.new(args).call
+        output = $stdout.string
+        object = JSON.parse(output)
+        expect(object).to be_an(Array)
+        expect(object.size).to eq(2)
+        expect(object.map(&:class)).to eq([Hash, Hash])
+      ensure
+        $stdout = original_stdout
+      end
+    end
+  end
+
+  describe '#single_document_output' do
+    RSpec.shared_examples 'verify_result_is_hash' do |format_chars, parser|
+      specify "correctly uses result hash for JSON and YAML when options are #{format_chars}" do
+        original_stdout = $stdout
+        $stdout = StringIO.new
+        begin
+          rika_command = described_class.new(["-f#{format_chars}", fixture_path('tiny.txt')])
+          rika_command.call
+          output = $stdout.string
+          warn output
+          result_hash = parser.call(output)
+          expect(result_hash).to be_a(Hash)
+          expect(result_hash['metadata']).to be_a(Hash)
+          expect(result_hash['text']).to be_a(String)
+        ensure
+          $stdout = original_stdout
+        end
+      end
+    end
+
+    include_examples('verify_result_is_hash', 'JJ', ->(s) { JSON.parse(s) })
+    include_examples('verify_result_is_hash', 'jj', ->(s) { JSON.parse(s) })
+    include_examples('verify_result_is_hash', 'yy', ->(s) { YAML.safe_load(s) })
+  end
+
+  describe '#set_output_formats' do
+    RSpec.shared_examples 'verify_correct_output_formats_selected' \
+      do |format_chars, expected_m_formatter, expected_t_formatter|
+      specify "correctly sets output formats when options are #{format_chars}" do
+        rika_command = described_class.new(["-f#{format_chars}"])
+        rika_command.send(:prepare)
+        expect(rika_command.send(:metadata_formatter)).to eq(expected_m_formatter)
+        expect(rika_command.send(:text_formatter)).to eq(expected_t_formatter)
+      end
+    end
+
+    include_examples('verify_correct_output_formats_selected', 'aj', RF::AWESOME_PRINT_FORMATTER, RF::JSON_FORMATTER)
+    include_examples('verify_correct_output_formats_selected', 'Jy', RF::PRETTY_JSON_FORMATTER,   RF::YAML_FORMATTER)
+    include_examples('verify_correct_output_formats_selected', 'ti', RF::TO_S_FORMATTER,          RF::INSPECT_FORMATTER)
+
+    RSpec.shared_examples 'verify_bad_output_format_exits' do |format_chars|
+      specify "exits when a bad output format is specified with #{format_chars}" do
+        expect { described_class.new(["-f#{format_chars}"]).call }.to raise_error(SystemExit)
+      end
+    end
+
+    include_examples 'verify_bad_output_format_exits', 'ax'
+    include_examples 'verify_bad_output_format_exits', 'xa'
+    include_examples 'verify_bad_output_format_exits', 'x'
+  end
+
+  describe '#warn_if_no_targets_specified' do
+    it 'prints a warning if no targets are specified' do
+      rika_command = described_class.new([])
+      allow(rika_command).to receive_messages(
+        targets: [],
+        help_text: 'sample help text'
+      )
+      expect { rika_command.send(:report_and_exit_if_no_targets_specified) }.to raise_error(SystemExit)
+      expect(rika_command).to have_received(:help_text).once
+      output = $stderr.string
+      expect(output).to match(/No targets specified/)
+      expect(output).to include('sample help text')
+    end
+  end
+end

data/spec/rika/formatters_spec.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+require 'rika/formatters'
+
+describe Rika::Formatters do
+  describe '.get' do
+    let(:rf) { described_class }
+
+    it 'returns the correct formatter for each option character' do
+      expect(rf.get('a')).to eq(rf::AWESOME_PRINT_FORMATTER)
+      expect(rf.get('i')).to eq(rf::INSPECT_FORMATTER)
+      expect(rf.get('j')).to eq(rf::JSON_FORMATTER)
+      expect(rf.get('J')).to eq(rf::PRETTY_JSON_FORMATTER)
+      expect(rf.get('t')).to eq(rf::TO_S_FORMATTER)
+      expect(rf.get('y')).to eq(rf::YAML_FORMATTER)
+    end
+
+    it 'raises an error if the option character is invalid' do
+      expect { rf.get('x') }.to raise_error(KeyError)
+    end
+  end
+end

data/spec/rika/parse_result_spec.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+require 'rika/parse_result'
+
+describe Rika::ParseResult do
+  context 'when initialized' do
+    specify 'contains the necessary fields' do
+      expect(described_class.new).to respond_to(
+        :content,
+        :text, # alias for content
+        :metadata,
+        :metadata_java,
+        :content_type,
+        :language,
+        :input_type,
+        :data_source,
+        :max_content_length
+      )
+    end
+  end
+
+  describe '#file?' do
+    specify 'returns true if input_type is :file' do
+      expect(described_class.new(input_type: :file).file?).to be true
+    end
+
+    specify 'returns false if input_type is not :file' do
+      expect(described_class.new.file?).to be false
+    end
+  end
+
+  describe '#http?' do
+    specify 'returns true if input_type is :http' do
+      expect(described_class.new(input_type: :http).http?).to be true
+    end
+
+    specify 'returns false if input_type is not :http' do
+      expect(described_class.new.http?).to be false
+    end
+  end
+end