opener-tokenizer 1.0.1 → 1.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 07977da1fd3a94d9a88697f43235d2c7202bbe33
-  data.tar.gz: e86a4ca7440d2e2b1ed8bfa8851e90d842c5eaab
+  metadata.gz: 68417ce5d0cd433b5d46037849fe28f2d9672352
+  data.tar.gz: 481f30d8f1a16a929665b6895a7fe6d4689035b9
 SHA512:
-  metadata.gz: dd20250047b206a9df53fc462fd27fc9e0d826baa07584edbf87d2afbc63a68e398b4a729755ffab04c467231645153ac8f0dee2c6b2719902eeee23b420a79b
-  data.tar.gz: 286ceb3f33edf03e2db5ea0fd123aaca4a46a39e5c3afd6d0157af73cbf912e4cc680c1bfb73c53598159fad6fae5dd512e1855d02c0d27b1a547d690f6ed0c9
+  metadata.gz: f6ade2200023fe2a04cb89f490abdcd87254df45b53124a6965336e3a11309c01b01593f497c783ead7a213d87d07c0adc315e6236944bbdc632c5186f2149b0
+  data.tar.gz: 72f1d3c07c0fece79fb507cc7ad73d6cb0adab54982f1acceae57ddbabeaf6704d3a70975df184c414ef8ce00823486849f84f458a0064e0827e54edc5a24a64

data/README.md

@@ -7,7 +7,7 @@ The tokenizer tokenizes a text into sentences and words.
 
 This software is part of a larger collection of natural language processing
 tools known as "the OpeNER project". You can find more information about the
-project at (the OpeNER portal)[http://opener-project.github.io]. There you can
+project at [the OpeNER portal](http://opener-project.github.io). There you can
 also find references to terms like KAF (an XML standard to represent linguistic
 annotations in texts), component, cores, scenario's and pipelines.
 
@@ -25,7 +25,7 @@ output KAF by default.
 ### Command line interface
 
 You should now be able to call the tokenizer as a regular shell
-command: by its name. Once installed the gem normalyl sits in your path so you can call it directly from anywhere.
+command: by its name. Once installed the gem normally sits in your path so you can call it directly from anywhere.
 
 Tokenizing some text:
 
@@ -112,14 +112,11 @@ Description of dependencies
 ---------------------------
 
 This component runs best if you run it in an environment suited for OpeNER
-components. You can find an installation guide and helper tools in the (OpeNER
-installer)[https://github.com/opener-project/opener-installer] and (an
-installation guide on the Opener
-Website)[http://opener-project.github.io/getting-started/how-to/local-installation.html]
+components. You can find an installation guide and helper tools in the [OpeNER installer](https://github.com/opener-project/opener-installer) and [an installation guide on the Opener Website](http://opener-project.github.io/getting-started/how-to/local-installation.html)
 
 At least you need the following system setup:
 
-### Depenencies for normal use:
+### Dependencies for normal use:
 
 * Perl 5
 * MRI 1.9.3
@@ -132,27 +129,26 @@ At least you need the following system setup:
 Language Extension
 ------------------
 
-  TODO
+The tokenizer module is a wrapping around a Perl script, which performs the actual tokenization based on rules (when to break a character sequence). The tokenizer already supports a lot of languages. Have a look to the core script to figure out how to extend to new languages.
 
 The Core
 --------
 
-The component is a fat wrapper around the actual language technology core. You
-can find the core technolies in the following repositories:
+The component is a fat wrapper around the actual language technology core. The core is a rule based tokenizer implemented in Perl. You can find the core technologies in the following repositories:
 
-* (tokenizer-base)[http://github.com/opener-project/tokenizer-base]
+* [tokenizer-base](http://github.com/opener-project/tokenizer-base)
 
 Where to go from here
 ---------------------
 
-* Check (the project websitere)[http://opener-project.github.io]
-* (Checkout the webservice)[http://opener.olery.com/tokenizer]
+* [Check the project website](http://opener-project.github.io)
+* [Checkout the webservice](http://opener.olery.com/tokenizer)
 
 Report problem/Get help
 -----------------------
 
-If you encounter problems, please email support@opener-project.eu or leave an
-issue in the (issue tracker)[https://github.com/opener-project/tokenizer/issues].
+If you encounter problems, please email <support@opener-project.eu> or leave an
+issue in the [issue tracker](https://github.com/opener-project/tokenizer/issues).
 
 
 Contributing

data/lib/opener/tokenizer/version.rb

@@ -1,5 +1,5 @@
 module Opener
   class Tokenizer
-    VERSION = "1.0.1"
+    VERSION = "1.0.2"
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: opener-tokenizer
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.0.2
 platform: ruby
 authors:
 - development@olery.com
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-05-20 00:00:00.000000000 Z
+date: 2014-05-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: opener-tokenizer-base