natural 0.2.1 → 0.4.0

package/.npmignore

@@ -1,11 +1,4 @@
-*~
-\#*
-*.kpf
-*classifier.json
-node_modules
-io_spec/test_data/wordnet/download/*
-*.noun
-*.verb
-*.adj
-*.adv
-npm-debug.log
+benchmarks
+spec
+examples
+io_spec

package/README.md

@@ -1,6 +1,7 @@
 natural
 =======
 
+[![NPM version](https://img.shields.io/npm/v/natural.svg)](https://www.npmjs.com/package/natural)
 [![Build Status](https://travis-ci.org/NaturalNode/natural.png?branch=master)](https://travis-ci.org/NaturalNode/natural)
 
 "Natural" is a general natural language facility for nodejs. Tokenizing,
@@ -18,8 +19,30 @@ will be in order. Thanks to Polyakov Vladimir, Russian stemming has been added!,
 
 Aside from this README, the only documentation is [this DZone article](http://www.dzone.com/links/r/using_natural_a_nlp_module_for_nodejs.html) and [here on my blog](http://www.chrisumbel.com/article/node_js_natural_language_porter_stemmer_lancaster_bayes_naive_metaphone_soundex), which is a bit older.
 
-Installation
-------------
+### TABLE OF CONTENTS
+
+* [Installation](#installation)
+* [Tokenizers](#tokenizers)
+* [String Distance](#string-distance)
+* [Stemmers](#stemmers)
+* [Classifiers](#classifiers)
+* [Phonetics](#phonetics)
+* [Inflectors](#inflectors)
+* [N-Grams](#n-grams)
+* [tf-idf](#tf-idf)
+* [Tries](#tries)
+* [EdgeWeightedDigraph](#edgeweighteddigraph)
+* [ShortestPathTree](#shortestpathtree)
+* [LongestPathTree](#longestpathtree)
+* [WordNet](#wordnet)
+* [Spellcheck](#spellcheck)
+* [POS Tagger](#pos-tagger)
+* [Acknowledgements/references](#acknowledgements-and-references)
+* [Development](#development)
+* [License](#license)
+
+
+## Installation
 
 If you're just looking to use natural without your own node application,
 you can install via NPM like so:
@@ -29,8 +52,7 @@ you can install via NPM like so:
 If you're interested in contributing to natural, or just hacking on it, then by all
 means fork away!
 
-Tokenizers
-----------
+## Tokenizers
 
 Word, Regexp, and [Treebank tokenizers](http://www.cis.upenn.edu/~treebank/tokenization.html) are provided for breaking text up into
 arrays of tokens:
@@ -58,8 +80,8 @@ console.log(tokenizer.tokenize("my dog hasn't any fleas."));
 // [ 'my',  'dog',  'hasn',  '\'',  't',  'any',  'fleas',  '.' ]
 ```
 
-String Distance
-----------------------
+## String Distance
+
 Natural provides an implementation of the [Jaro–Winkler](http://en.wikipedia.org/wiki/Jaro%E2%80%93Winkler_distance) string distance measuring algorithm.
 This will return a number between 0 and 1 which tells how closely the strings match (0 = not at all, 1 = exact match):
 
@@ -122,8 +144,7 @@ Output:
 0
 ```
 
-Stemmers
---------
+## Stemmers
 
 Currently stemming is supported via the [Porter](http://tartarus.org/martin/PorterStemmer/index.html) and [Lancaster](http://www.comp.lancs.ac.uk/computing/research/stemming/) (Paice/Husk) algorithms.
 
@@ -167,8 +188,7 @@ console.log("i am waking up to the sounds of chainsaws".tokenizeAndStem());
 console.log("chainsaws".stem());
 ```
 
-Classifiers
-----------------------
+## Classifiers
 
 Two classifiers are currently supported, [Naive Bayes](http://en.wikipedia.org/wiki/Naive_Bayes_classifier) and [logistic regression](http://en.wikipedia.org/wiki/Logistic_regression).
 The following examples use the BayesClassifier class, but the 
@@ -234,7 +254,8 @@ The training process can be monitored by subscribing to the event `trainedWithDo
        *   total: 23 // There are 23 total documents being trained against
        *   index: 12 // The index/number of the document that's just been trained against
        *   doc: {...} // The document that has just been indexed
-       */ }
+       *  }
+       */ 
     });
 
 A classifier can also be persisted and recalled so you can reuse a training
@@ -268,8 +289,7 @@ var restoredClassifier = natural.BayesClassifier.restore(JSON.parse(raw));
 console.log(restoredClassifier.classify('i should sell that'));
 ```
 
-Phonetics
----------
+## Phonetics
 
 Phonetic matching (sounds-like) matching can be done with the [SoundEx](http://en.wikipedia.org/wiki/Soundex),
 [Metaphone](http://en.wikipedia.org/wiki/Metaphone) or [DoubleMetaphone](http://en.wikipedia.org/wiki/Metaphone#Double_Metaphone) algorithms
@@ -356,8 +376,7 @@ if(wordA.soundsLike(wordB))
 console.log('phonetics'.phonetics());
 ```
 
-Inflectors
-----------
+## Inflectors
 
 ### Nouns
 
@@ -442,8 +461,7 @@ console.log('walks'.pluralizePresentVerb());
 ```
 
 
-N-Grams
--------
+## N-Grams
 
 n-grams can be obtained for either arrays or strings (which will be tokenized
 for you):
@@ -534,8 +552,7 @@ console.log(NGramsZH.ngrams(['一', '个', '中', '文', '测',
     '试'], 4));
 ```
 
-tf-idf
------
+## tf-idf
 
 [Term Frequency–Inverse Document Frequency (tf-idf)](http://en.wikipedia.org/wiki/Tf%E2%80%93idf) is implemented to determine how important a word (or words) is to a 
 document relative to a corpus. The following example will add four documents to 
@@ -663,8 +680,7 @@ var s = JSON.stringify(tfidf);
 var tfidf = new TfIdf(JSON.parse(s));
 ```
 
-Tries
------
+## Tries
 
 Tries are a very efficient data structure used for prefix-based searches. 
 Natural comes packaged with a basic Trie implementation which can support match collection along a path,
@@ -741,8 +757,8 @@ ciTrie.contains("TEsT"); // true
 ```
 In the case of the searches which return strings, all strings returned will be in lower case if you are in case-_in_sensitive mode.
 
-EdgeWeightedDigraph
--------------------
+## EdgeWeightedDigraph
+
 EdgeWeightedDigraph represents a digraph, you can add an edge, get the number vertexes, edges, get all edges and use toString to print the Digraph.
 
 initialize a digraph:
@@ -775,8 +791,7 @@ you will get 5.
 
 
 
-ShortestPathTree
-----------------
+## ShortestPathTree
 
 ShortestPathTree represents a data type for solving the single-source shortest paths problem in
 edge-weighted directed acyclic graphs (DAGs).
@@ -827,8 +842,7 @@ output will be:
 [5, 4]
 ```
 
-LongestPathTree
-----------------
+## LongestPathTree
 
 LongestPathTree represents a data type for solving the single-source shortest paths problem in
 edge-weighted directed acyclic graphs (DAGs).
@@ -879,21 +893,18 @@ output will be:
 [5, 1, 3, 6, 4]
 ```
 
-WordNet
--------
+## WordNet
 
 One of the newest and most experimental features in natural is WordNet integration. Here's an
 example of using natural to look up definitions of the word node. To use the WordNet module,
-first install the WordNet database files using the [WNdb module](https://github.com/moos/WNdb):
-
-    npm install WNdb
+first install the WordNet database files using [wordnet-db](https://github.com/moos/wordnet-db):
 
-(For node < v0.6, please use 'npm install WNdb@3.0.0')
+    npm install wordnet-db
 
 Keep in mind that the WordNet integration is to be considered experimental at this point,
-and not production-ready. The API is also subject to change.
+and not production-ready. The API is also subject to change.  For an implementation with vastly increased performance, as well as a command-line interface, see [wordpos](https://github.com/moos/wordpos). 
 
-Here's an example of looking up definitions for the word, "node".
+Here's an example of looking up definitions for the word "node".
 
 ```javascript
 var wordnet = new natural.WordNet();
@@ -935,8 +946,8 @@ As of v0.1.11, WordNet data files are no longer automatically downloaded.
 
 Princeton University "About WordNet." WordNet. Princeton University. 2010. <http://wordnet.princeton.edu>
 
-Spellcheck
-------------
+## Spellcheck
+
 A probabilistic spellchecker based on http://norvig.com/spell-correct.html
 
 This is best constructed with an array of tokens from a corpus, but a simple list of words from a dictionary will work. 
@@ -959,9 +970,114 @@ spellcheck.getCorrections('soemthing', 1); // ['something']
 spellcheck.getCorrections('soemthing', 2); // ['something', 'soothing']
 ```
 
+## POS Tagger
+
+This is a part-of-speech tagger based on Eric Brill's transformational 
+algorithm. Transformation rules are specified in external files.
+
+### Usage
+```javascript
+var Tagger = require("./lib/natural").BrillPOSTagger;
+
+var base_folder = "some_path/lib/natural/brill_pos_tagger";
+var rules_file = base_folder + "/data/tr_from_posjs.txt";
+var lexicon_file = base_folder + "/data/lexicon_from_posjs.json";
+var default_category = 'N';
+
+var tagger = new Tagger(lexicon_file, rules_file, default_category, function(error) {
+  if (error) {
+    console.log(error);
+  }
+  else {
+    var sentence = ["I", "see", "the", "man", "with", "the", "telescope"];
+    console.log(JSON.stringify(tagger.tag(sentence)));
+  }
+});
+```
+
+### Lexicon
+The lexicon is either a JSON file that has the following structure:
+```javascript
+{
+  "word1": ["cat1"],
+  "word2": ["cat2", "cat3"],
+  ...
+}
+```
+or a text file:
+```
+word1 cat1 cat2
+word2 cat3
+...
+```
+Words may have multiple categories in the lexicon file. The tagger uses only 
+the first category specified.
+
+### Specifying transformation rules
+Transformation rules are specified as follows:
+```
+OLD_CAT NEW_CAT PREDICATE PARAMETER
+```
+This means that if the category of the current position is OLD_CAT and the predicate is true, the category is replaced by NEW_CAT. The predicate 
+may use the parameter in different ways: sometimes the parameter is used for 
+specifying the outcome of the predicate:
+```
+NN CD CURRENT-WORD-IS-NUMBER YES
+```
+This means that if the outcome of predicate CURRENT-WORD-IS-NUMBER is YES, the 
+category is replaced by <code>CD</code>.
+The parameter can also be used to check the category of a word in the sentence:
+```
+VBD NN PREV-TAG DT
+```
+Here the category of the previous word must be <code>DT</code> for the rule to be applied.
+
+### Algorithm
+The tagger applies transformation rules that may change the category of words. The input sentence must be split into words which are assigned with categories. The tagged sentence is then processed from left to right. At each step all rules are applied once; rules are applied in the order in which they are specified. Algorithm:
+```javascript
+function(sentence) {
+  var tagged_sentence = new Array(sentence.length);
+
+  // snip
+
+  // Apply transformation rules
+  for (var i = 0, size = sentence.length; i < size; i++) {
+    this.transformation_rules.forEach(function(rule) {
+      rule.apply(tagged_sentence, i);
+    });
+  }
+  return(tagged_sentence);
+}
+```
+
+### Adding a predicate
+Predicates are defined in module <code>lib/Predicate.js</code>. In that file 
+a function must be created that serves as predicate. A predicate accepts a 
+tagged sentence, the current position in the sentence that should be tagged, and
+ the
+ outcome(s) of the predicate. An example of a predicate that checks the category of the current word:
+```javascript
+function current_word_is_tag(tagged_sentence, i, parameter) {
+  return(tagged_sentence[i][0] === parameter);
+}
+```
+Some predicates accept two parameters. Next step is to map a keyword to this predicate so that it can be used in the transformation rules. The mapping is also defined in <code>lib/Predicate.js</code>:
+```javascript
+var predicates = {
+  "CURRENT-WORD-IS-TAG": current_word_is_tag,
+  "PREV-WORD-IS-CAP": prev_word_is_cap
+}
+```
+
+### Acknowledgements and References
+* Part of speech tagger by Percy Wegmann, https://code.google.com/p/jspos/
+* Node.js version of jspos: https://github.com/neopunisher/pos-js
+* A simple rule-based part of speech tagger, Eric Brill, Published in: Proceeding ANLC '92 Proceedings of the third conference on Applied natural language processing, Pages 152-155. http://dl.acm.org/citation.cfm?id=974526
+
+
+
+## Development
 
-Development
------------
 When developing, please:
 
 + Write unit tests
@@ -972,8 +1088,7 @@ The current configuration of the unit tests requires the following environment v
     export NODE_PATH=.
 
 
-License
--------
+## License
 
 Copyright (c) 2011, 2012 Chris Umbel, Rob Ellis, Russell Mull
 

package/bayes_classifier.json

@@ -0,0 +1 @@
+{"classifier":{"classFeatures":{"computing":{"0":2,"1":2,"2":2,"3":3,"4":2,"5":2},"literature":{"2":2,"6":4,"7":2,"8":2}},"classTotals":{"computing":4,"literature":4},"totalExamples":7,"smoothing":1},"docs":[{"label":"computing","text":["fix","box"]},{"label":"computing","text":["write","code"]},{"label":"computing","text":["nasti","script","code"]},{"label":"literature","text":["write","book"]},{"label":"literature","text":["read","book"]},{"label":"literature","text":["studi","book"]}],"features":{"fix":1,"box":1,"write":2,"code":2,"nasti":1,"script":1,"book":3,"read":1,"studi":1},"stemmer":{},"lastAdded":6,"events":{"domain":null,"_events":{},"_maxListeners":10}}

package/lib/natural/brill_pos_tagger/data/Dutch/README.txt

@@ -0,0 +1,2 @@
+Transformation rules and lexicon are from
+https://mlnl.net/jg/software/bnl/

package/lib/natural/brill_pos_tagger/data/Dutch/brill_CONTEXTRULES.jg

@@ -0,0 +1,285 @@
+Pron(betr,neut,zelfst) Pron(aanw,neut,attr) PREVTAG Prep(voor)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXT1OR2TAG Adv(gew,geen_func,stell,onverv)
+Pron(onbep,neut,zelfst) Art(bep,onzijd,neut) NEXT1OR2TAG N(soort,ev,neut)
+Prep(voor) Adv(deel_v) NEXT1OR2TAG STAART
+Prep(voor) Adv(deel_v) NEXTTAG Prep(voor_inf)
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) PREV1OR2TAG Art(onbep,zijd_of_onzijd,neut)
+Pron(betr,neut,zelfst) Pron(aanw,neut,attr) NEXTTAG N(soort,ev,neut)
+Prep(voor) Adv(deel_v) NEXTTAG Prep(voor)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXTTAG Adv(gew,geen_func,stell,onverv)
+Conj(onder,met_fin) Pron(aanw,neut,attr) NEXTTAG N(soort,ev,neut)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) PREVTAG N(soort,mv,neut)
+Adv(gew,er) Adv(pron,er) WDAND2TAGAFT er Prep(voor)
+Prep(voor) Adv(deel_v) NEXTTAG Punc(komma)
+Conj(neven) Adv(gew,geen_func,stell,onverv) PREVTAG Adv(gew,geen_func,stell,onverv)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXTTAG V(hulp_of_kopp,ott,3,ev)
+Pron(onbep,neut,zelfst) Pron(bez,2,ev,neut,attr) NEXTTAG N(soort,ev,neut)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXT1OR2TAG Art(bep,zijd_of_mv,neut)
+V(hulp,ott,1_of_2_of_3,mv) V(hulp,inf) NEXTTAG V(trans,inf)
+Pron(vrag,neut,attr) Pron(betr,neut,zelfst) PREV1OR2TAG Prep(voor)
+V(hulp,ott,3,ev) V(hulp,ott,1,ev) NEXT1OR2OR3TAG Pron(per,1,ev,nom)
+Pron(betr,neut,zelfst) Pron(aanw,neut,zelfst) PREVTAG Conj(neven)
+Pron(onbep,gen,attr) Adv(gew,geen_func,stell,onverv) CURWD anders
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) PREV1OR2OR3TAG N(soort,mv,neut)
+Adv(gew,aanw) Conj(onder,met_fin) PREV1OR2OR3TAG Adv(gew,geen_func,vergr,onverv)
+Adv(gew,geen_func,vergr,onverv) Num(hoofd,onbep,zelfst,vergr,onverv) NEXT1OR2OR3TAG Conj(onder,met_fin)
+Adv(gew,er) Adv(pron,er) NEXT1OR2OR3TAG Adv(deel_v)
+Adv(deel_v) Adv(deel_adv) PREV1OR2OR3TAG Adv(pron,er)
+Prep(voor) Adv(deel_adv) NEXTTAG Punc(punt)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXT1OR2OR3TAG Adv(gew,aanw)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,inf) NEXTTAG Punc(punt)
+Prep(voor_inf) Adv(gew,geen_func,stell,onverv) NEXTTAG Adj(attr,stell,onverv)
+Pron(onbep,neut,zelfst) Art(bep,onzijd,neut) PREV1OR2OR3TAG Prep(voor)
+V(hulp,ott,3,ev) V(hulp,ott,1,ev) PREV1OR2OR3TAG Pron(per,1,ev,nom)
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) CURWD werkelijk
+Adv(gew,aanw) Conj(onder,met_fin) PREV1OR2TAG Adj(adv,vergr,onverv)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,inf) NEXTTAG Punc(komma)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) PREVTAG Conj(neven)
+N(soort,ev,neut) Adj(attr,stell,onverv) SURROUNDTAG Art(onbep,zijd_of_onzijd,neut) N(soort,ev,neut)
+V(trans,inf) V(trans,ott,1_of_2_of_3,mv) PREV1OR2OR3TAG Punc(komma)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXTTAG Prep(voor)
+V(intrans,inf) V(intrans,ott,1_of_2_of_3,mv) PREV1OR2TAG STAART
+Adj(attr,stell,verv_neut) N(soort,ev,neut) NEXTTAG Prep(voor)
+N(soort,ev,neut) V(trans,ott,1,ev) PREVTAG Pron(per,1,ev,nom)
+Pron(per,1,ev,dat_of_acc) Pron(ref,1,ev) PREV1OR2TAG Pron(per,1,ev,nom)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) PREVTAG V(hulp_of_kopp,ott,3,ev)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXT1OR2TAG Adv(gew,aanw)
+Adv(gew,aanw) Conj(onder,met_fin) NEXTTAG Pron(per,3,ev,nom)
+Conj(neven) Adv(gew,geen_func,stell,onverv) WDNEXTTAG maar Art(onbep,zijd_of_onzijd,neut)
+Prep(voor_inf) Adv(gew,geen_func,stell,onverv) NEXT1OR2TAG Adj(adv,stell,onverv)
+V(intrans,ovt,1_of_2_of_3,ev) V(trans,ovt,1_of_2_of_3,ev) CURWD dacht
+V(trans,inf) V(trans,ott,1_of_2_of_3,mv) PREV1OR2TAG STAART
+N(soort,mv,neut) V(trans,inf) PREVTAG V(hulp,ovt,1_of_2_of_3,ev)
+Num(hoofd,bep,attr,onverv) Num(hoofd,bep,zelfst,onverv) NEXTTAG Punc(haak_sluit)
+V(trans,inf) V(trans,ott,1_of_2_of_3,mv) PREV1OR2WD en
+Pron(onbep,neut,zelfst) Pron(per,2,ev,nom) NEXT1OR2TAG Adj(adv,stell,onverv)
+V(hulp_of_kopp,inf) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXT2TAG N(soort,mv,neut)
+V(hulp,ott,1_of_2_of_3,mv) V(trans,ott,1_of_2_of_3,mv) NEXT1OR2TAG Adv(gew,geen_func,stell,onverv)
+Pron(onbep,neut,zelfst) Art(bep,onzijd,neut) NEXTTAG Adj(attr,stell,verv_neut)
+Adv(gew,er) Adv(pron,er) PREV1OR2WD ik
+Adv(gew,aanw) Conj(onder,met_fin) PREV1OR2OR3TAG Adj(attr,vergr,onverv)
+Pron(onbep,neut,zelfst) Art(bep,onzijd,neut) NEXTTAG N(soort,ev,neut)
+Adv(gew,geen_func,stell,onverv) N(soort,ev,neut) PREVTAG Art(bep,zijd_of_mv,neut)
+V(intrans,inf) V(hulp,inf) NEXTTAG N(soort,mv,neut)
+Pron(onbep,neut,zelfst) Pron(per,2,ev,dat_of_acc) PREVTAG Prep(voor)
+V(intrans,teg_dw,onverv) Adv(gew,geen_func,stell,onverv) CURWD voortdurend
+Pron(onbep,neut,zelfst) Pron(onbep,neut,attr) WDNEXTTAG ander N(soort,ev,neut)
+N(eigen,ev,neut) N(soort,ev,neut) NEXTWD aan
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) NEXTTAG Adj(attr,stell,onverv)
+Adj(attr,stell,onverv) N(soort,ev,neut) PREV1OR2TAG Art(bep,onzijd,neut)
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) CURWD hard
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) CURWD ruim
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXT1OR2TAG Pron(aanw,neut,attr)
+N(soort,mv,neut) V(intrans,inf) PREVTAG V(hulp,inf)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXT1OR2OR3TAG Punc(punt)
+V(hulp,ott,3,ev) V(trans,ott,3,ev) RBIGRAM heeft een
+Prep(voor) N(eigen,ev,neut) WDNEXTTAG Van N(eigen,ev,neut)
+V(hulp,ott,1_of_2_of_3,mv) V(hulp,inf) PREV1OR2TAG V(hulp,ovt,1_of_2_of_3,ev)
+Conj(neven) Adv(gew,geen_func,stell,onverv) PREV1OR2OR3TAG Pron(per,2,ev,nom)
+V(hulp,ovt,1_of_2_of_3,ev) V(trans,ovt,1_of_2_of_3,ev) NEXT1OR2OR3TAG Conj(onder,met_fin)
+Pron(vrag,neut,attr) Pron(betr,neut,zelfst) PREVTAG Pron(onbep,neut,zelfst)
+Adv(gew,geen_func,vergr,onverv) Num(hoofd,onbep,zelfst,vergr,onverv) PREV1OR2OR3TAG Conj(neven)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,inf) PREV1OR2TAG Prep(voor_inf)
+Pron(bez,1,mv,neut,attr) Pron(per,1,mv,dat_of_acc) PREV1OR2OR3TAG Punc(komma)
+Pron(betr,neut,zelfst) Pron(aanw,neut,attr) NEXTTAG Adj(attr,stell,verv_neut)
+Pron(vrag,neut,attr) Pron(onbep,neut,attr) PREVTAG Adv(gew,geen_func,stell,onverv)
+Pron(onbep,neut,attr) Pron(onbep,neut,zelfst) NEXTWD van
+V(intrans,verl_dw,onverv) V(trans,verl_dw,onverv) PREV1OR2OR3TAG N(soort,ev,neut)
+Pron(aanw,neut,zelfst) Pron(betr,neut,zelfst) PREVTAG N(soort,ev,neut)
+V(hulp,imp) V(hulp,ott,3,ev) CURWD laat
+Adj(attr,stell,onverv) Adv(gew,geen_func,stell,onverv) CURWD opeens
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) NEXTTAG V(hulp_of_kopp,ott,3,ev)
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) CURWD haastig
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) SURROUNDTAG Adv(gew,geen_func,stell,onverv) Punc(punt)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXTTAG V(hulp_of_kopp,ott,3,ev)
+N(soort,ev,neut) Adj(adv,stell,onverv) PREVTAG Adv(gew,aanw)
+N(soort,mv,neut) V(trans,inf) PREVWD moeten
+Num(hoofd,bep,attr,onverv) Num(hoofd,bep,zelfst,onverv) NEXTTAG V(hulp_of_kopp,ovt,1_of_2_of_3,ev)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXT2TAG Pron(per,1,ev,nom)
+N(soort,ev,neut) N(eigen,ev,neut) CURWD februari
+V(hulp,ott,3,ev) V(trans,ott,3,ev) NEXTTAG Punc(komma)
+V(hulp_of_kopp,inf) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXTTAG Adv(gew,geen_func,stell,onverv)
+V(hulp,ott,3,ev) V(trans,ott,3,ev) WDNEXTTAG heeft Prep(voor)
+Pron(onbep,neut,zelfst) Pron(per,2,ev,nom) PREV1OR2OR3TAG Punc(aanhaal_dubb)
+V(trans,ott,3,ev) V(trans,ott,2,ev) PREV1OR2OR3TAG Pron(per,2,ev,nom)
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) PREV1OR2OR3TAG Pron(per,2,ev,nom)
+Prep(voor) Adv(deel_v) NEXTTAG Conj(onder,met_fin)
+N(soort,mv,neut) V(trans,verl_dw,onverv) CURWD verloren
+Adj(attr,stell,onverv) N(soort,ev,neut) WDAND2TAGBFR Prep(voor) dood
+Pron(aanw,neut,attr) Pron(aanw,neut,zelfst) NEXTTAG Prep(voor)
+Prep(voor) N(eigen,ev,neut) SURROUNDTAG N(eigen,ev,neut) N(eigen,ev,neut)
+Adv(gew,aanw) Adv(pron,aanw) NEXT1OR2TAG Pron(per,1,ev,nom)
+Pron(bez,3,ev,neut,attr) Pron(per,3,ev,dat_of_acc) NEXTTAG Adv(gew,geen_func,stell,onverv)
+N(soort,ev,neut) V(trans,ovt,1_of_2_of_3,ev) PREVTAG Pron(per,3,ev,nom)
+V(hulp,ott,1_of_2_of_3,mv) V(hulp,inf) PREV1OR2TAG Prep(voor)
+Adv(gew,er) Adv(pron,er) PREV2TAG Conj(onder,met_fin)
+Prep(voor) Adv(deel_adv) PREV1OR2OR3TAG Adv(pron,er)
+Pron(bez,3,ev,neut,attr) Pron(per,3,ev,dat_of_acc) NEXT1OR2TAG Art(onbep,zijd_of_onzijd,neut)
+Num(hoofd,bep,zelfst,onverv) Num(hoofd,bep,attr,onverv) NEXTTAG N(soort,ev,neut)
+Adv(gew,aanw) Conj(onder,met_fin) PREV1OR2TAG Adj(attr,stell,onverv)
+V(hulp,ott,1_of_2_of_3,mv) V(trans,ott,1_of_2_of_3,mv) PREV1OR2TAG Pron(bez,3,mv,neut,attr)
+Conj(neven) Adv(gew,geen_func,stell,onverv) PREV1OR2WD ik
+Conj(neven) Conj(onder,met_fin) PREV1OR2WD net
+Pron(vrag,neut,attr) Pron(betr,neut,zelfst) NEXTTAG N(eigen,ev,neut)
+Num(hoofd,bep,zelfst,onverv) Num(hoofd,bep,attr,onverv) NEXT1OR2OR3TAG Num(hoofd,bep,zelfst,onverv)
+Art(bep,onzijd,neut) Pron(per,3,ev,nom) NEXTTAG Art(onbep,zijd_of_onzijd,neut)
+V(intrans,ovt,1_of_2_of_3,ev) V(hulp,ovt,1_of_2_of_3,ev) CURWD begon
+Adv(gew,geen_func,stell,onverv) N(soort,ev,neut) PREV1OR2WD op
+N(soort,ev,neut) Num(hoofd,bep,attr,onverv) CURWD 7
+Adv(gew,geen_func,vergr,onverv) Num(hoofd,onbep,zelfst,vergr,onverv) PREV1OR2TAG Prep(voor)
+V(trans,inf) V(intrans,ott,1_of_2_of_3,mv) PREV1OR2OR3TAG STAART
+V(intrans,ott,1_of_2_of_3,mv) V(intrans,inf) NEXT1OR2OR3TAG Conj(neven)
+V(intrans,inf) V(hulp,inf) NEXTTAG V(trans,inf)
+Adv(gew,geen_func,stell,onverv) Conj(onder,met_fin) PREVTAG Punc(komma)
+Adv(gew,geen_func,stell,onverv) Adj(attr,stell,onverv) CURWD tevreden
+V(trans,ott,1_of_2_of_3,mv) V(intrans,inf) CURWD beginnen
+V(trans,ott,1,ev) V(trans,ott,2,ev) NEXTTAG Pron(per,2,ev,nom)
+V(intrans,teg_dw,verv_neut) N(soort,ev,neut) CURWD schande
+V(intrans,ott,3,ev) N(soort,ev,neut) NEXT1OR2OR3TAG Prep(voor_inf)
+N(soort,ev,neut) Adj(attr,vergr,onverv) PREVWD steeds
+N(soort,ev,neut) V(intrans,ott,3,ev) WDNEXTTAG sterft Prep(voor)
+N(eigen,ev,neut) Adj(attr,stell,verv_neut) SURROUNDTAG STAART N(soort,mv,neut)
+V(trans,ovt,1_of_2_of_3,ev) Adj(attr,stell,verv_neut) PREV1OR2TAG Prep(voor)
+V(intrans,inf) V(hulp,inf) NEXTTAG V(intrans,inf)
+V(hulp,inf) V(trans,inf) NEXTTAG Punc(punt)
+V(refl,ott,3,ev) V(trans,ott,3,ev) CURWD voelt
+Pron(bez,3,ev,neut,attr) V(intrans,ott,1_of_2_of_3,mv) PREVTAG Adv(gew,er)
+V(hulp,ott,2,ev) V(hulp,ott,3,ev) CURWD hoeft
+Pron(onbep,neut,zelfst) Pron(ref,2,ev) PREV1OR2WD je
+V(intrans,teg_dw,verv_neut) Pron(onbep,neut,attr) WDNEXTTAG verschillende N(soort,mv,neut)
+Adv(gew,vrag) Adj(attr,stell,onverv) PREV1OR2OR3TAG V(hulp_of_kopp,ott,3,ev)
+Adj(attr,vergr,verv_neut) Adj(attr,stell,verv_neut) CURWD nare
+V(trans,ott,1,ev) Int CURWD hoor
+V(intrans,ovt,1_of_2_of_3,mv) V(intrans,verl_dw,onverv) CURWD verdwenen
+Adv(pron,vrag) Adv(pron,betr) CURWD waardoor
+N(eigen,ev,neut) N(soort,ev,neut) PREVTAG Art(onbep,zijd_of_onzijd,neut)
+N(eigen,ev,neut) N(soort,ev,neut) NEXT1OR2WD worden
+N(eigen,ev,neut) N(soort,ev,neut) NEXTBIGRAM Conj(neven) N(soort,ev,neut)
+N(eigen,ev,neut) N(soort,ev,neut) SURROUNDTAG STAART Prep(voor)
+N(eigen,ev,neut) N(soort,ev,neut) CURWD Koningin
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) PREV1OR2TAG V(hulp_of_kopp,ott,1_of_2_of_3,mv)
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) PREVTAG STAART
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) WDPREVTAG N(soort,mv,neut) nodig
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) NEXTTAG Prep(voor_inf)
+Adj(adv,stell,onverv) Adj(attr,stell,onverv) CURWD eenzaam
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) CURWD regelmatig
+Adj(attr,stell,onverv) Adj(adv,stell,onverv) CURWD geestelijk
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) PREVTAG Pron(per,2,ev,nom)
+V(trans,inf) V(intrans,inf) NEXT1OR2TAG N(eigen,ev,neut)
+V(intrans,ovt,1_of_2_of_3,ev) V(hulp,ovt,1_of_2_of_3,ev) NEXT1OR2TAG V(intrans,inf)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXTTAG Punc(komma)
+V(trans,inf) V(intrans,inf) PREV1OR2OR3TAG Adj(adv,vergr,onverv)
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXT1OR2TAG Pron(onbep,neut,zelfst)
+V(trans,inf) V(intrans,inf) CURWD wennen
+Art(bep,onzijd,neut) Pron(onbep,neut,zelfst) NEXT1OR2TAG Conj(onder,met_inf)
+N(soort,ev,neut) V(trans,verl_dw,onverv) PREV1OR2TAG V(hulp_of_kopp,ovt,1_of_2_of_3,mv)
+N(soort,ev,neut) Adj(adv,stell,onverv) NEXTTAG V(trans,verl_dw,verv_neut)
+Prep(voor) Adv(deel_adv) NEXTTAG V(trans,inf)
+N(soort,ev,neut) Adj(adv,stell,onverv) NEXTBIGRAM Adj(attr,stell,onverv) N(soort,ev,neut)
+N(soort,mv,neut) V(trans,inf) PREV1OR2TAG Pron(per,3,ev,dat_of_acc)
+Prep(voor) Adv(deel_adv) NEXTBIGRAM V(intrans,ott,3,ev) Punc(punt)
+V(intrans,inf) V(intrans,ott,1_of_2_of_3,mv) PREV1OR2TAG Conj(neven)
+N(soort,mv,neut) V(trans,inf) PREV1OR2OR3TAG V(hulp,ott,2,ev)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXTTAG V(intrans,ott,3,ev)
+V(intrans,inf) V(intrans,ott,1_of_2_of_3,mv) PREV1OR2OR3TAG Punc(komma)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) PREVTAG Pron(per,1,mv,nom)
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXT1OR2TAG V(trans,ott,3,ev)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXTTAG Prep(voor)
+Conj(onder,met_inf) Prep(voor) NEXTTAG N(soort,mv,neut)
+V(hulp_of_kopp,inf) V(hulp_of_kopp,ott,1_of_2_of_3,mv) PREVTAG V(trans,verl_dw,onverv)
+V(hulp,ott,3,ev) V(trans,ott,3,ev) WDPREVTAG Pron(per,3,ev,nom) heeft
+Conj(onder,met_fin) Pron(aanw,neut,zelfst) NEXT1OR2WD doen
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXTTAG V(trans,verl_dw,onverv)
+V(hulp_of_kopp,ott,1_of_2_of_3,mv) V(hulp_of_kopp,inf) PREV1OR2TAG V(hulp,ott,3,ev)
+N(soort,ev,neut) Adj(attr,stell,verv_neut) PREVBIGRAM Adj(attr,stell,verv_neut) Punc(komma)
+N(soort,mv,neut) V(trans,verl_dw,onverv) CURWD ingenomen
+Conj(onder,met_inf) Prep(voor) NEXTTAG N(soort,ev,neut)
+V(trans,ovt,1_of_2_of_3,ev) V(intrans,ovt,1_of_2_of_3,ev) PREVTAG N(eigen,ev,neut)
+Pron(bez,3,ev,neut,attr) V(hulp_of_kopp,ott,1_of_2_of_3,mv) NEXTWD het
+Pron(aanw,neut,attr) Pron(aanw,neut,zelfst) NEXTTAG V(hulp_of_kopp,ott,3,ev)
+Adv(gew,aanw) Adv(pron,aanw) PREVTAG Punc(punt_komma)
+Adv(deel_v) Adv(deel_adv) RBIGRAM bij ,
+V(intrans,inf) V(trans,inf) CURWD schrijven
+Pron(onbep,neut,zelfst) Pron(per,2,ev,nom) PREV1OR2OR3TAG N(eigen,ev,neut)
+Prep(voor) Adv(deel_v) NEXTTAG V(hulp,inf)
+N(soort,mv,neut) V(trans,verl_dw,onverv) CURWD betrokken
+V(hulp_of_kopp,inf) V(hulp_of_kopp,ott,1_of_2_of_3,mv) PREV1OR2WD en
+Pron(aanw,neut,zelfst) Conj(onder,met_fin) WDAND2TAGAFT Dat N(soort,ev,neut)
+Adv(deel_v) Adv(deel_adv) PREV1OR2OR3TAG Adv(gew,er)
+Adv(gew,er) Adv(pron,er) NEXT1OR2OR3TAG Adv(deel_adv)
+N(soort,ev,neut) N(soort,mv,neut) WDNEXTTAG dll Num(hoofd,bep,attr,onverv)
+N(eigen,ev,neut) N(eigen,mv,neut) NEXTWD Staten
+Adj(attr,stell,verv_neut) N(soort,ev,neut) NEXTTAG Punc(punt)
+V(hulp_of_kopp,inf) V(hulp_of_kopp,ott,1_of_2_of_3,mv) PREV1OR2OR3TAG Adj(attr,stell,verv_neut)
+V(hulp,ovt,1_of_2_of_3,ev) V(trans,ovt,1_of_2_of_3,ev) SURROUNDTAG N(soort,ev,neut) Punc(komma)
+Pron(bez,3,ev,neut,attr) Pron(per,3,ev,dat_of_acc) NEXTTAG Prep(voor)
+Prep(voor) N(eigen,ev,neut) NEXTTAG Art(bep,zijd_of_mv,gen)
+Art(bep,zijd_of_mv,neut) N(eigen,ev,neut) WDNEXTTAG De N(eigen,ev,neut)
+Adj(attr,stell,verv_neut) Adj(zelfst,stell,verv_neut) PREVWD Het
+N(eigen,ev,neut) V(trans,imp) SURROUNDTAG STAART Adv(gew,aanw)
+N(eigen,ev,neut) N(eigen,mv,neut) PREVTAG N(eigen,mv,neut)
+Adv(gew,aanw) Conj(onder,met_fin) LBIGRAM , toen
+V(hulp,ott,1_of_2_of_3,mv) V(trans,ott,1_of_2_of_3,mv) RBIGRAM hebben ,
+V(trans,ott,1_of_2_of_3,mv) V(hulp,ott,1_of_2_of_3,mv) PREV1OR2TAG STAART
+V(hulp,inf) V(hulp,ott,1_of_2_of_3,mv) NEXTTAG Pron(per,1,mv,nom)
+Pron(bez,3,ev,neut,attr) Pron(per,3,ev,dat_of_acc) NEXTTAG V(trans,verl_dw,onverv)
+Pron(aanw,neut,attr) Pron(betr,neut,zelfst) WDPREVTAG N(soort,mv,neut) die
+V(hulp,ott,3,ev) V(hulp,ott,2,ev) PREVTAG Pron(per,2,ev,nom)
+Pron(bez,1,mv,neut,attr) Pron(per,1,mv,dat_of_acc) NEXTTAG Punc(punt)
+N(eigen,ev,neut) Prep(voor) SURROUNDTAG STAART Art(bep,zijd_of_mv,neut)
+Conj(neven) Adv(gew,geen_func,stell,onverv) PREVTAG Adv(gew,aanw)
+Pron(betr,neut,zelfst) Pron(aanw,neut,zelfst) PREV1OR2TAG Conj(onder,met_fin)
+Pron(aanw,neut,zelfst) Pron(aanw,neut,attr) NEXTBIGRAM Adj(attr,stell,verv_neut) N(soort,ev,neut)
+Prep(voor_inf) Adv(gew,geen_func,stell,onverv) NEXTTAG Adj(attr,stell,verv_neut)
+Num(hoofd,bep,zelfst,onverv) Num(hoofd,bep,attr,onverv) PREV1OR2OR3TAG Num(hoofd,bep,attr,onverv)
+N(soort,ev,neut) V(intrans,inf) PREVTAG Prep(voor_inf)
+Prep(voor_inf) Prep(voor) NEXTTAG N(soort,ev,neut)
+Adj(adv,stell,onverv) N(soort,ev,neut) WDAND2AFT geheel .
+V(trans,ott,3,ev) N(soort,ev,neut) PREVTAG Art(onbep,zijd_of_onzijd,neut)
+V(trans,inf) V(trans,inf,subst) PREV1OR2TAG Art(bep,onzijd,neut)
+V(hulp,ott,3,ev) V(hulp,ott,2,ev) NEXTWD u
+Adv(gew,geen_func,stell,onverv) N(soort,ev,neut) PREVTAG Adj(attr,stell,verv_neut)
+Adv(deel_v) Prep(voor) WDNEXTTAG tot Prep(voor)
+V(hulp,ovt,1_of_2_of_3,mv) V(trans,ovt,1_of_2_of_3,mv) CURWD hadden
+Adv(gew,geen_func,stell,onverv) Pron(onbep,neut,attr) NEXTWD die
+V(trans,inf) V(refl,inf) PREV1OR2OR3TAG Pron(ref,3,ev_of_mv)
+V(hulp,inf) V(hulp,ott,1_of_2_of_3,mv) WDNEXTTAG moeten V(trans,inf)
+V(hulp,ott,1_of_2_of_3,mv) V(hulp,inf) NEXTBIGRAM V(trans,inf) Punc(komma)
+Pron(aanw,neut,zelfst) Pron(betr,neut,zelfst) SURROUNDTAG Punc(komma) Prep(voor)
+N(eigen,ev,neut) Int SURROUNDTAG Punc(aanhaal_dubb) Punc(komma)
+N(eigen,ev,neut) Art(bep,zijd_of_mv,neut) LBIGRAM STAART De
+Conj(onder,met_fin) Conj(neven) PREV1OR2WD zowel
+Adv(gew,geen_func,stell,onverv) Adj(attr,stell,onverv) NEXTTAG Conj(neven)
+Adj(attr,stell,verv_neut) Pron(onbep,neut,attr) WDNEXTTAG Elke N(soort,ev,neut)
+Adj(adv,vergr,onverv) Adj(attr,vergr,onverv) PREV1OR2TAG Adv(gew,geen_func,stell,onverv)
+V(trans,ovt,1_of_2_of_3,ev) N(soort,ev,neut) PREVTAG Adj(attr,stell,verv_neut)
+V(intrans,teg_dw,verv_neut) N(soort,ev,neut) CURWD bende
+V(hulp,inf) V(hulp,ott,1_of_2_of_3,mv) WDPREVTAG N(soort,ev,neut) kunnen
+Pron(vrag,neut,attr) Pron(onbep,neut,attr) NEXTTAG N(soort,mv,neut)
+Pron(betr,neut,zelfst) Pron(vrag,neut,attr) NEXT1OR2OR3TAG Adv(gew,aanw)
+Adv(gew,vrag) Adv(pron,vrag) PREVTAG Pron(onbep,neut,zelfst)
+Adv(gew,geen_func,vergr,onverv) Num(hoofd,onbep,attr,vergr,onverv) NEXT1OR2TAG N(soort,mv,neut)
+V(trans,ott,1,ev) V(trans,ott,3,ev) PREVTAG Pron(onbep,neut,zelfst)
+V(hulp_of_kopp,ovt,1_of_2_of_3,mv) V(intrans,ovt,1_of_2_of_3,mv) NEXTTAG Adv(gew,er)
+Pron(onbep,neut,zelfst) Pron(per,2,ev,dat_of_acc) PREV1OR2TAG Pron(per,1,ev,nom)
+Num(hoofd,bep,attr,onverv) Num(rang,bep,attr,onverv) PREVTAG Art(bep,onzijd,neut)
+N(soort,ev,neut) Int CURWD oh
+V(intrans,ott,1,ev) N(soort,ev,neut) NEXT1OR2OR3TAG Prep(voor)
+Pron(aanw,neut,zelfst) Pron(betr,neut,zelfst) SURROUNDTAG Punc(komma) Adv(gew,geen_func,stell,onverv)
+Adj(zelfst,stell,verv_mv) N(soort,mv,neut) CURWD armen
+V(trans,ott,1,ev) V(intrans,ovt,1_of_2_of_3,ev) NEXT1OR2WD in
+V(intrans,inf) N(soort,mv,neut) WDNEXTTAG verschillen Prep(voor)
+Prep(voor) V(trans,imp) NEXTTAG Pron(aanw,neut,attr)
+N(eigen,ev,neut) Art(bep,zijd_of_mv,neut) WDNEXTTAG DE N(eigen,ev,neut)
+Adv(gew,aanw) Conj(onder,met_fin) WDAND2TAGBFR Art(bep,zijd_of_mv,neut) toen
+V(trans,ovt,1_of_2_of_3,mv) V(trans,ott,1_of_2_of_3,mv) NEXTTAG Pron(per,3,ev_of_mv,nom)
+V(trans,ovt,1_of_2_of_3,ev) Conj(onder,met_fin) CURWD zodra
+V(intrans,verl_dw,verv_neut) V(intrans,ovt,1_of_2_of_3,ev) CURWD gebeurde
+V(intrans,teg_dw,onverv) Adv(gew,geen_func,stell,onverv) CURWD dringend
+V(hulp,ovt,1_of_2_of_3,ev) Adj(attr,stell,verv_neut) PREV1OR2TAG Prep(voor)
+V(intrans,ott,3,ev) N(soort,ev,neut) PREVTAG Adj(attr,stell,verv_neut)
+V(hulp,inf) Num(hoofd,bep,attr,onverv) CURWD veertien
+Prep(voor) N(soort,mv,neut) NEXTWD zijn
+Num(hoofd,onbep,zelfst,stell,onverv) Num(hoofd,onbep,attr,stell,onverv) NEXTTAG N(soort,mv,neut)
+Art(bep,zijd_of_mv,gen) N(eigen,ev,neut) PREVTAG N(eigen,ev,neut)
+N(eigen,ev,neut) N(soort,ev,neut) PREVTAG Adj(attr,overtr,verv_neut)