biblicit 1.0 → 2.0.3

data/.gitmodules

@@ -0,0 +1,3 @@
+[submodule "parscit"]
+	path = parscit
+	url = git@github.com:academia-edu/ParsCit.git

data/Gemfile

@@ -1,6 +1,6 @@
 # encoding: UTF-8
 
-source :rubygems
+source 'https://rubygems.org'
 
 # Specify your gem's dependencies in the gemspec
 gemspec

data/README.md

@@ -3,33 +3,130 @@ biblicit
 
 Extract citations from PDFs.
 
-## Usage
+Note: The version is 2.x, but really should be 0.2.x.
+
+
+# Usage
 
 ```ruby
-  # Extract metadata from a file using the code from CiteSeerX
-  Biblicit.extract(file: "myfile.pdf", tool: :citeseer)
+  # Extract metadata from a file using default tools and settings
+  result = Biblicit::Extractor.extract(content: "a string containing the content of a PDF file")
+
+  # Extract metadata from a file using all available tools
+  result = Biblicit::Extractor.extract(file: "myfile.pdf", tools: [:citeseer, :parshed, :cb2bib], remote: true, token: false)
 
-  # Extract metadata from the contents of a PDF using cb2bib
-  Biblicit.extract(contents: IO.read("myfile.pdf"), tool: :cb2bib, remote: true)
+  # See reference information for "myfile.pdf"
+  result[:citeseer][:title]
+  result[:parshed][:title]
+  result[:citeseer][:authors]
+  # etc
 ```
 
-## Algorithms
+
+# Algorithms
 
 ### CiteSeer (default)
 
 Wrapper around Perl code extracted from [CiteSeerX](http://citeseer.ist.psu.edu/). 
 
-Uses [Apache PDFBox](http://pdfbox.apache.org/) to extract text from the PDF, uses a model trained with the [svm-light](http://svmlight.joachims.org/) Support Vector Machine library to extract citation data for the PDF itself, and then uses [ParsCit](http://aye.comp.nus.edu.sg/parsCit/)'s model trained with the [CRF++](http://code.google.com/p/crfpp/) Conditional Random Fields library to parse citations from the PDF's bibliography, if any.
+Uses a model trained with the [svm-light](http://svmlight.joachims.org/) Support Vector Machine library.
+
+### ParsCit (default) 
+
+Wrapper around Perl & Ruby code from [ParsCit](http://aye.comp.nus.edu.sg/parsCit/), which is included as a Git submodule.
+
+Uses a model trained with the [CRF++](http://code.google.com/p/crfpp/) Conditional Random Fields library.
 
 ### cb2Bib
 
 Wrapper around [cb2Bib](http://www.molspaces.com/cb2bib/) in command-line mode.
 
-Uses pdf2text from [Xpdf](http://www.foolabs.com/xpdf/download.html) to extract text from the PDF, uses an apparently less-sophisticated parsing algorithm than the CiteSeerX code to parse metadata, but then, if :remote=true, scrapes one of a large number of journal or public repository websites for a structured version of the citation data.
+Uses an apparently less-sophisticated parsing algorithm than the others to parse metadata, but then, if :remote=true, scrapes one of a large number of journal or public repository websites for a structured version of the citation data. Warning: sometimes it finds the wrong work!
+
+
+# Requirements
+
+There are a lot, but you may not need all of them, depending on your use case.
+
+
+## Required to support various input file formats
+
+Different tools are used for different input file formats.
+
+#### PDF - [Poppler](http://poppler.freedesktop.org/)
+
+This provides `pdftotext`. You could install `xpdf` instead.
+
+##### From source
+
+Requires fontconfig.
+
+    wget http://poppler.freedesktop.org/poppler-0.22.1.tar.gz
+    tar -xzf poppler-0.22.1.tar.gz
+    cd poppler-0.22.1
+    ./configure
+    make
+    sudo make install
+
+##### On Debian/Ubuntu
+
+    sudo apt-get install poppler-utils
+
+##### On OS X with Homebrew
+
+    brew install poppler
+
+#### Postscript - [Ghostscript](http://www.ghostscript.com/)
+
+This provides `ps2ascii`.
+
+##### From source
+
+    wget http://downloads.ghostscript.com/public/ghostscript-9.06.tar.gz
+    tar -xzf ghostscript-9.06.tar.gz
+    cd ghostscript-9.06
+    make
+    sudo make install
+
+##### On Debian/Ubuntu
+
+    sudo apt-get install ghostscript
+
+##### On OS X with Homebrew
+
+    brew install ghostscript
+
+#### Other (e.g. docx) - [AbiWord](http://www.abisource.com/)
+
+This provides `abiword`.
+
+##### On Debian/Ubuntu
+
+    sudo apt-get install abiword
+
+##### On OS X
+
+As of writing, you're out of luck, because AbiWord doesn't compile on recent versions of OS X. According to their website, however, this is being actively worked on.
+
+
+## Required to use either the ParsCit or CiteSeer algorithms
+
+#### Perl modules
+
+More than these might be required; this is what I had to add to my default installation.
 
-## Requirements
+##### From CPAN
+
+    sudo cpan install Digest::SHA1
+    sudo cpan install String::Approx
+    sudo cpan install XML::Writer::String
+    sudo cpan install XML::Twig
+
+## Required to use the ParsCit algorithm
 
-### CRF++
+#### CRF++
+
+You can specify where you have installed CRF++ by setting the CRFPP_HOME environment variable.
  
 ##### From source
 
@@ -44,15 +141,19 @@ Uses pdf2text from [Xpdf](http://www.foolabs.com/xpdf/download.html) to extract
 
     sudo apt-add-repository 'deb http://cl.naist.jp/~eric-n/ubuntu-nlp oneiric all'
     sudo apt-get update
-    sudo apt-get install libcrf++
+    sudo apt-get install libcrf++ crf++
 
 ##### On OS X with Homebrew
 
     brew install crf++
 
-### svm-light
+## Required to use the CiteSeer algorithm
+
+#### svm-light
 
-The included model requires version 5, not the current version.
+Required for header extraction (reference information for the input work itself).
+
+The included model requires version 5, not the current version. You can specify where you have installed svm-light by setting the SVM_LIGHT_HOME environment variable.
 
 ##### From source
 
@@ -61,22 +162,12 @@ The included model requires version 5, not the current version.
     wget http://download.joachims.org/svm_light/v5.00/svm_light.tar.gz
     tar -xzf svm_light.tar.gz
     make
-    sudo ln -s $(readlink -f "$(dirname svm_classify)/$(basename svm_classify)") /usr/bin/svm_classify5
-    sudo ln -s $(readlink -f "$(dirname svm_learn)/$(basename svm_learn)") /usr/bin/svm_learn5
+    echo "export SVM_LIGHT_HOME=`pwd`" >> ~/.profile # or .bashrc or whatever
+    source ~/.profile
 
-Note: On OS X you'll need to use greadlink instead of readlink if you have coreutils installed, or another workaround for the absence of `-f`.
+## Required to use the cb2bib algorithm
 
-### Perl modules
-
-##### From CPAN
-
-    sudo cpan install DBI
-    sudo cpan install Digest::SHA1
-    sudo cpan install Log::Log4perl
-    sudo cpan install Log::Dispatch
-    sudo cpan install String::Approx
-
-### cb2bib
+#### cb2Bib
 
 ##### From source (Linux)
 
@@ -105,15 +196,19 @@ Requires Qt & X11, unfortunately, and still requires a hack to work on recent ve
 
     sudo apt-get install cb2bib
 
-### Other
+
+## Other
+
+(I'm not currently sure what this was required for; TODO figure it out!)
 
 ##### On Debian/Ubuntu
 
     sudo apt-get install libicu-dev
 
-## Copying
 
-Copyright Academia.edu or the original author(s).
+# Copying
+
+Copyright Academia.edu or the original author(s) - see documentation in the included parscit and svm-header-parse directories.
 
 Apache licensed (see LICENSE.TXT).
 

data/Rakefile

@@ -6,3 +6,25 @@ require 'rspec/core/rake_task'
 require 'biblicit'
 
 RSpec::Core::RakeTask.new :spec
+
+desc "Tag #{Bundler::GemHelper.new.send(:version_tag)}, build and push to gemfury"
+task :release_internal do |t|
+  require 'gemfury'
+
+  class ReleaseInternalGem < Bundler::GemHelper
+    def release_gem
+      guard_clean
+      built_gem_path = build_gem
+      if Bundler::VERSION =~ /1\.3\.\d/
+        tag_version { git_push } unless already_tagged?
+      else
+        guard_already_tagged
+        tag_version { git_push }
+      end
+      `fury push #{built_gem_path}`
+      Bundler.ui.confirm "Pushed #{name} #{version} to gemfury"
+    end
+  end
+
+  ReleaseInternalGem.new.release_gem
+end

data/biblicit.gemspec

@@ -5,14 +5,15 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |gem|
   gem.name          = "biblicit"
-  gem.version       = "1.0"
+  gem.version       = "2.0.3"
   gem.authors       = ["David Judd"]
   gem.email         = ["david@academia.edu"]
-  gem.description   = %q{Extract citations from PDFs.}
   gem.summary       = %q{Extract citations from PDFs.}
   gem.homepage      = "http://github.com/academia-edu/biblicit"
 
-  gem.files         = `git ls-files`.split("\n")
+  gem.files         =
+    `git ls-files`.split("\n") +
+    `cd parscit && git ls-files`.split("\n").map{ |f| "parscit/#{f}" }
   gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
   gem.test_files    = gem.files.grep(%r{^spec/})
   gem.require_paths = ["lib"]
@@ -25,9 +26,10 @@ Gem::Specification.new do |gem|
   gem.add_development_dependency 'pry'
   gem.add_development_dependency 'pry-debugger'
 
-  gem.requirements << 'java'
-  gem.requirements << 'perl'
-  gem.requirements << 'for the :citeseer algorithm, the CRF++ Conditional Random Fields library (try "which crf_test")'
-  gem.requirements << 'for the :cb2bib algorithm, the cb2bib citation extraction tool'
+  gem.requirements << 'For PDFs, Poppler or XPDF (try "which pdftotext")'
+  gem.requirements << 'For Postscript files, Ghostscript (try "which ps2ascii")'
+  gem.requirements << 'For word processor files, AbiWord (try "which abiword")'
+  gem.requirements << 'For the :citeseer algorithm, Perl, CPAN, CRF++ (try "which crf_test"), and svm-light 5.0, aliased to svm_classify5 (try "svm_classify -h")'
+  gem.requirements << 'For the :cb2bib algorithm, cb2Bib (try "which cb2bib")'
 
 end

data/lib/biblicit/cb2bib.rb

@@ -4,30 +4,28 @@ require 'tempfile'
 
 module Cb2Bib
 
-  def self.extract(file, opts)
-    ParseOperation.new(file, opts)
+  def self.extract(file, opts={})
+    ParseOperation.new(file, opts).result
   end
 
   class ParseOperation
 
+    attr_reader :result
+
     def initialize(file, opts)
       extract_from_file(file, opts[:remote] || false, opts[:sloppy] || true)
     end
 
-    def header
-      @result
-    end
-
   private
 
-    def extract_from_file(pdf, remote=false, sloppy=true)
+    def extract_from_file(doc, remote=false, sloppy=true)
       bib = Tempfile.new(['out','.bib'])
       conf = Tempfile.new(['cb2bib','.conf']) # we'll put our custom configuration here, and then cb2bib will fill in the rest with its defaults
 
       begin
         conf.write(cb2bib_config(remote))
         conf.open # not clear why we have to do this, but otherwise cb2bib doesn't read it
-        `cb2bib #{sloppy ? '--sloppy' : ''} --doc2bib #{pdf.shellescape} #{bib.path} --conf #{conf.path}`
+        `cb2bib #{sloppy ? '--sloppy' : ''} --txt2bib #{doc.path} #{bib.path} --conf #{conf.path}`
         bibtext = bib.read
       ensure
         conf.close!
@@ -45,12 +43,13 @@ module Cb2Bib
           end
         end
       end
-
-      @result[:valid] = !@result[:title].blank?
     end
 
     def cb2bib_config(remote)
-      "[cb2Bib]\nAutomaticQuery=#{!!remote}"
+      """
+      [cb2Bib]
+      AutomaticQuery=#{!!remote}
+      """
     end
 
     def cleaned_field(field)

data/lib/biblicit/citeseer.rb

@@ -6,48 +6,36 @@ require 'nokogiri'
 
 module CiteSeer
 
-  PERL_DIR = "#{File.dirname(__FILE__)}/../../perl"
+  PERL_DIR = "#{File.dirname(__FILE__)}/../../svm-header-parse"
 
-  def self.extract(in_file, opts)
-    ParseOperation.new(in_file)
+  def self.extract(in_file, opts={})
+    ParseOperation.new(in_file).result
   end
 
   class ParseOperation
 
+    attr_reader :result
+
     def initialize(in_file)
       Dir.mktmpdir do |out_dir|
-        `#{PERL_DIR}/extract.pl #{in_file.shellescape} #{out_dir}`
-        @header_xml = IO.read("#{out_dir}/out.header")
-        @citations_xml = IO.read("#{out_dir}/out.parscit")
+        `#{PERL_DIR}/extract.pl #{in_file.path} #{out_dir}`
+        output = IO.read("#{out_dir}/out.header")
+        xml = Nokogiri::XML output
+        @result = parse(xml)
       end
     end
 
-    def header
-      @header ||= get_header
-    end
-
-    def citations
-      @citations ||= get_citations
-    end
-
   private
 
-    def get_header
-      parsed = Nokogiri::XML @header_xml
-
+    def parse(xml)
       {
-        title: parsed.css('title').text,
-        authors: parsed.css('author > name').map { |n| n.text },
-        abstract: parsed.css('abstract').text,
-        valid: parsed.css('validHeader').first.text == '1',
+        title: xml.css('title').text,
+        authors: xml.css('author > name').map { |n| n.text },
+        abstract: xml.css('abstract').text,
+        valid: xml.css('validHeader').first.text == '1',
       }
     end
 
-    def get_citations
-      # TODO
-      []
-    end
-
   end
 
 end

data/lib/biblicit/extractor.rb

@@ -2,36 +2,57 @@
 
 require 'biblicit/cb2bib'
 require 'biblicit/citeseer'
+require 'biblicit/parscit'
 
 require 'tempfile'
 
 module Biblicit
 
-  def self.extract(opts)
-    if (content = opts.delete(:content))
-      Tempfile.open(['in','.pdf']) do |pdf|
-        pdf.write(content)
-        extract_from_file pdf.path, opts
+  SH_DIR = "#{File.dirname(__FILE__)}/../../sh"
+
+  module Extractor
+
+    def self.extract(opts)
+      if (content = opts.delete(:content))
+        Tempfile.open('in') do |in_file|
+          in_file.binmode
+          in_file.write(content)
+          extract_from_file in_file.path, opts
+        end
+      elsif (file = opts.delete(:file))
+        extract_from_file file, opts
+      else
+        raise 'Either file or content is required'
       end
-    elsif (file = opts.delete(:file))
-      extract_from_file file, opts
-    else
-      raise 'Either file or content is required'
     end
-  end
 
-private
+  private
+
+    def self.extract_from_file(file, opts)
+      file = File.realpath(file)
+      tools = opts.delete(:tools) || [:parshed, :citeseer]
+
+      result = {}
+
+      Tempfile.open(['in','.txt']) do |in_txt|
+        `#{SH_DIR}/convert_to_text.sh #{file.shellescape} #{in_txt.path}`
 
-  def self.extract_from_file(file, opts)
-    tool = opts.delete(:tool) || :citeseer
+        if tools.include?(:parshed)
+          result.merge!( parshed: ParsCit.extract(in_txt, opts) )
+        end
 
-    if tool == :citeseer
-      CiteSeer.extract(file, opts)
-    elsif tool == :cb2bib
-      Cb2Bib.extract(file, opts)
-    else
-      raise "Unknown tool #{tool}"
+        if tools.include?(:citeseer)
+          result.merge!( citeseer: CiteSeer.extract(in_txt, opts) )
+        end
+
+        if tools.include?(:cb2bib)
+          result.merge!( cb2bib: Cb2Bib.extract(in_txt, opts) )
+        end
+      end
+
+      result
     end
+
   end
 
 end

data/lib/biblicit/parscit.rb

@@ -0,0 +1,38 @@
+# encoding: UTF-8
+
+require 'tmpdir'
+require 'shellwords'
+require 'nokogiri'
+
+module ParsCit
+
+  PERL_DIR = "#{File.dirname(__FILE__)}/../../parscit"
+
+  def self.extract(in_file, opts={})
+    ParseOperation.new(in_file, opts).result
+  end
+
+  class ParseOperation
+
+    attr_reader :result
+
+    def initialize(in_txt, opts={})
+      ENV['CRFPP_HOME'] ||= "#{File.dirname(`which crf_test`)}/../"
+      output = `#{PERL_DIR}/bin/citeExtract.pl -q -m extract_all #{in_txt.path}`
+      @result = parse(Nokogiri::XML output)
+    end
+
+  private
+
+    def parse(xml)
+      parsed = xml.css("algorithm[name=ParsHed]")
+      {
+        title: parsed.css('title').text.gsub(/\s+/,' ').strip,
+        authors: parsed.css('author').map { |a| a.text.gsub(/\s+/,' ').strip },
+        abstract: parsed.css('abstract').text
+      }
+    end
+
+  end
+
+end

data/parscit/.gitignore

@@ -0,0 +1,8 @@
+doc/*.zip
+doc/*.tgz
+lib/cgiLog.txt
+*~
+thang/*
+tmp/*
+archive/*
+doc/.htaccess

data/parscit/CHANGELOG

@@ -0,0 +1,125 @@
+110505 (done by Huy)
+- New features: Sectlabel
+- New sectlabel xml model with more training data (resources/sectLabel/sectLabel.modelXml.v2)
+
+- Major changes: Sectlabel
+- Addded Omni (lib/Omni/Omnitable.pm lib/Omni/Omnicell.pm lib/Omni/Omnidd.pm lib/Omni/Omniframe.pm lib/Omni/Traversal.pm)
+- ParsCit now uses the reference output from Sectlabel as its input
+
+- Minor changes: Sectlabel and ParsCit
+- Cleanup all temporary files in /tmp properly
+
+- Minor changes: bug fixes
+- Biblioscript: patch from Tim Brody
+
+110121 (done by Huy)
+- Major changes: Omni classes for Omnipage XML process using XML::Twig and XML::Writer instead of regular expression
+- Added Omni (lib/Omni/Config.pm lib/Omni/Omnicol.pm lib/Omni/Omnidoc.pm lib/Omni/Omniline.pm lib/Omni/Omnipage.pm lib/Omni/Omnipara.pm lib/Omni/Omnirun.pm lib/Omni/Omniword.pm)
+- Use the new Omni classes in Parscit module
+
+- Major changes: improve Parscit performance when there's no reference marker
+- Added new crf++ XML model, template and train data (resources/parsCit.split.model crfpp/traindata/parsCit/parsCit.split.template crfpp/traindata/parsCit/parsCit.split.train.data)
+- Use the new Parscit model when there's no reference marker
+
+- Minor changes: bug fixes
+- Parscit post-process: stripPunctuation function removes the semi-colon in XML special characters, e.g. & Thanks to Qasemizadeh, Behrang for reporting these issues.
+- Parscit controller: terminate properly when no reference is found (normBodyText size 1 != posArray size 0)
+- Parscit post-process: fix the volume number truncation, e.g. vol 5(1) becomes vol 5; Thanks to Lennart Borgman for reporting these issues
+
+- Minor changes: Parscit
+- Add bin/xml2train.pl: extract reference text and XML information from Omnipage and save it into Parscit train file's format
+
+100901 (done by Thang)
+- Incorporate BiblioScrip (http://github.com/mromanello/BiblioScript) and BibUtils (http://www.scripps.edu/~cdputnam/software/bibutils/)
+
+100401e (done by Min on 100725)
+- Minor changes to paths and to make it work again from wing.nus directory
+  (moved from forecite, due to restructuring of WING server)
+
+100401d
+- Minor changes to documentation and ParsHed library updating.
+
+100401c
+- Minor changes for correcting errors with punctuation and XML
+  entities in reference string parsing. Reported by Cheong Chi Hong
+  and Mario Lipinski. Fixed by Minh-Thang Luong.
+
+100401b
+- Minor changes (bug fixes) to section labeler model.
+
+100401 (done by Thang)
+- Major Change: Added SectLabel module (due to Minh-Thang Luong and Thuy Dung Nguyen)
+- Added Iconip training data from Cheong Chi Hong
+- Updated default model to include Iconip data
+- Updated CGI demo to call new SectLabel module as well
+- Updated documentation
+- Corrected small regexp error in lib/ParsCit/PreProcess.pm
+- Corrected small problem with training data in mixed-humanities
+
+- Added SectLabel (bin/sectExtract.pl, bin/sectExtract/, resource/sectExtract, lib/SectExtract)
+- Modified bin/citeExtract.pl, lib/ParsCit/PostProcess.pm, lib/ParsHed/PostProcess.pm to combine ParsCit, ParsHed, SectLabel, and standardize XML output
+- Added test/ for testing purpose with 12 samples documents and standard-output of citeExtract.pl in 5 modes (citations, header, section, meta, and all) using both txt and XML inputs
+- Added SectLabel annotated data doc/sectLabel.tagged.txt and doc/sectLabelXml.tagged.txt (40 documents fully annotated)
+- Added in crfpp/traindata CRF++ feature files (sectLabel.train.dataXml, sectLabel.train.data) and templates (sectLabel.templateXml, sectLabel.template)
+
+- Incorporated works by Emma 
+- Added GenericSect code (bin/sectLabel/genericSectExtract.rb, crfpp/traindata/genericSect.train.data, bin/sectLabel/genericSect/) into SectLabel
+- Added GenericSect annotated data doc/genericSect.tagged.txt (211 documents with headers annotated)
+- Added in crfpp/traindata CRF++ feature file (genericSect.train.data) and template (genericSect.template)
+
+090625b 
+- Updated documentation only. no change to executables
+- Released on 30 September 2009
+
+090625 (due to Minh-Thang Luong)
+- Standardized and improved ParsHed model with line-level classification instead of token-level as previously.
+- Add a post-processing module for ParsHed to normalize field data, e.g. authors, email, etc.
+- Detailed changes are reflected as follows:
+  * Added resources/parsHed - all parsHed-related including models (old models in resources/parsHed/archive), template file, and top frequent keyword files.
+  * Added lib/ParsHed - similar architect as lib/ParsCit to modularizes and faciliates line-level training in ParsHed.
+	lib/ParsHed/Tr2crf.pm: line-level CRF feature extractor
+	lib/ParsHed/PostProcess.pm: post-processing of field data
+  * Added bin/parsHed - all parsHed-related scripts (redo.parsHed.pl, and tr2crffpp_parsHed.pl).
+    Includes parseXmlHeader.pl and convert2TokenLevel.pl used by redo.parsHed.pl to convert output from line to token-level.
+  * Updated bin/headExtract.pl - to use the new model lib/ParsHed, as well as old model (with -tokenLevel flag).
+  * Bug fixes in Citation.pm and Preprocess.pm (see doc/v090625-Artemy-issues.txt). Thanks to Artemy Kolchinsky for reporting these issues.
+  * Reordered the CHANGELOG into descending chronological order. 
+090625 (due to Min-Yen KAN)
+- Deprecated and unified ParsHedClient.rb into ParsCitClient.rb
+- Deprecated and unified ParsHedServer.rb into ParsCitServer.rb
+- Added wsdl/forecite.wsdl which describes the ParsCit portion of the services
+- Added bin/ParsCitClientWSDL.rb which demonstrates the use of forecite.wsdl
+
+090316:
+- Adds ParsHed module, updates to:
+  * resources/ -  parsHed.*.model model files for binaries
+  * doc/ - svm_headerparse.tagged.txt (from CiteSeer; 935 headers)
+  * bin/ - headExtract.pl, phOutput2xml.pl, redo.parsCit.pl, redo.parsHed.pl
+  * crfpp/traindata/ - parsHed.train.data (converted from svm_headerparse.tagged.txt)
+  Changes doc/index.html, doc/parsCit.cgi to handle parsHed call (not
+  yet entirely integrated, still separate module).
+
+081201:
+- Bug fixes from Scienstein.org team
+  * Added context positions
+  * Handle reference patterns such as [1-5]
+  * Handles context references within same window
+  * See doc/v081201-sciensteinEmail.txt for detailed notes
+- Updated ParsCit.cgi to update for context position output
+
+080917:
+- Added new training data from Matteo Romanello
+- Fixed Preprocess.pm bug (thanks to Dain Kaplan)
+- Upgraded CRF++ model to 0.51 (now bundled with CRF++ source just in case it is no longer available on sourceforge)
+- Added bin/redo.pl script to retrain model
+
+080402:
+- Added re-tagged data from FLUX-CiM
+- Added conlleval.pl evaluation script
+- Added output2xml.pl transformation script
+- Corrected warning in parseRefString.pl (thanks to Ayeh Bandeh-Ahmadi)
+
+080310:
+- First released version to Peter Weiland
+- Web services working for wing machines at NUS
+