docbook_status 0.1.1 → 0.3.0

data/Gemfile

@@ -1,7 +1,9 @@
 source "http://rubygems.org"
 gem "libxml-ruby", "~> 2.2.2", :require => 'xml'
 gem "directory_watcher"
+gem "json"
+gem "term-ansicolor"
 
 group :development do
 	gem "bones"
-end
+end

data/Gemfile.lock

@@ -1,16 +1,24 @@
 GEM
   remote: http://rubygems.org/
   specs:
+    bones (3.7.1)
+      little-plugger (>= 1.1.2)
+      loquacious (>= 1.8.1)
+      rake (>= 0.8.7)
     directory_watcher (1.4.1)
-    hoe (2.12.3)
-      rake (~> 0.8)
+    json (1.6.1)
     libxml-ruby (2.2.2)
+    little-plugger (1.1.2)
+    loquacious (1.8.1)
     rake (0.9.2)
+    term-ansicolor (1.0.6)
 
 PLATFORMS
   ruby
 
 DEPENDENCIES
+  bones
   directory_watcher
-  hoe
+  json
   libxml-ruby (~> 2.2.2)
+  term-ansicolor

data/History.txt

@@ -1,3 +1,22 @@
+== 0.3.0 / 2011-09-28
+
+* Writing goals (total and daily) can be defined and tracked
+* Added YAML and JSON output formats, available with option --outputformat
+* Minor fixes
+** Removed daemon state persistence file dw_state.yml
+** Colorized screen output
+** Reorganized library
+
+== 0.2.1 / 2011-09-18
+
+* Minor fixes for Gem packaging
+
+== 0.2.0 / 2011-09-18
+
+* Added remarks listing.
+* Minor fixes
+** removed _formalpara_ from the list of countable text elements because they contain _para_ elements
+
 == 0.1.1 / 2011-09-07
 
 * Documentation corrections
@@ -5,4 +24,3 @@
 == 0.1.0 / 2011-09-07
 
 * Initial version
-

data/README.rdoc

@@ -0,0 +1,142 @@
+= docbook_status
+
+A utility for DocBook authors/publishers showing the document structure (sections) and word count of a DocBook project. It is intended to provide an overview of a DocBook project's structure and volume while you are writing or editing it.
+
+== Features
+
+* lists the section hierarchy (set, book, ... section, simplesect) of a DocBook file
+* calculates a word count for each section (words in paras, simparas and formalparas)
+* works with included sections (XInclude)
+* finds and lists remarks (remarks are not included in the word count)
+* output in YAML and JSON format (optional)
+* tracks writing progress
+
+== Synopsis
+
+docbook_status is mainly a comandline application, bin/docbook_status, that helps with writing and editing DocBook 5 documents. The application provides information about the content of a DocBook project. That project can consist of a single file or of several files that are included in the master file via XInclude.
+
+To run docbook_status manually:
+
+   docbook_status myproject.xml
+
+This will show the section hierarchy and the word count of the individual sections of myproject.xml.
+
+If the file contains editing remarks, you could list them, too:
+
+   docbook_status --remarks myproject.xml
+
+Here docbook_status looks for all _remark_ elements. The application uses the same convention as many source code tools: if the content of the remark element starts with a all-uppercase word, like FIXME or TODO, this will be listed as the remark type, otherwise just REMARK will be used.
+
+If there are many remarks and you just want to concentrate on the most important ones, let's say the FIXMEs, you could restrict the listing with:
+
+   docbook_status --remarks=FIXME myproject.xml
+
+If you need to preprocess the XML before feeding it to docbook_status, there is an option for that:
+
+   docbook_status --pre "asciddoc -bdocbook50 myproject.txt" myproject.xml
+
+--pre takes shell commands as arguments and executes them before starting the analysis on the XML file.
+
+Finally, if you are tired to run docbook_status manually each time you changed your file, you could run the application in demon mode, continually:
+
+   docbook_status --demon --glob "*.xml" --dir "." myproject.xml
+
+In demon-mode the application checks the files matched by the _glob_ pattern in the directory specified by _dir_ for changes, and redisplays the document analysis whenever a change occures. The demon can be terminated by simply pressing RETURN.
+
+== Tracking writing progress
+
+As an experiment docbook_status provides features to define and track writing goals/schedules. Currently there are the following options:
+
+* an end date, or deadline, with option --end=[YYYY-MM-DD]
+* a total word count goal, with option --total=[number]
+* a daily word count goal, with option --daily=[number]
+
+These features are currently not related, you can use any combination of them. When an end date is defined the application will simply remind you on every run of how many days are left.
+
+If one of this options is used, a file called 'dbs_work.yml' is created in the working directory. This file is used to store the goals and the tracking information. If you want to get rid of all tracking, simply delete this file. To disable a specific kind of tracking, just call the options mentioned above with no arguments. That would delete the defined value. An example:
+
+  docbook_status --end=2099-01-01 --total=1000000  endofworld.xml
+
+This call would define the goals: scheduled delivery date of 2099-01-01, and a total document size of one million words. To disable the time tracking call it again with no argument:
+
+  docbook_status --end endofworld.xml
+
+This would delete the defined delivery date but not the total. Once defined the goal options must not be repeated, since they are stored in the 'dbs_work.yml' file.
+
+
+== Integration, Customization
+
+In the unlikely case that you don't like the standard screen output and would prefer to replace it, there are other output formats available, JSON and YAML, which make that possible. Normally all output is formatted for output on a terminal screen, but the option _--outputformat_ alllows to specify a different output format, that is printed to STDOUT. Using that you could create your own frontend or integrate the application with other tools (like editors).
+
+Use
+
+   docbook_status --outputformat=yaml
+or
+   docbook_status --outputformat=json
+
+to get YAML or JSON structures back. The structure returned is equivalent to the normal terminal output:
+
+ * file - path to XML file
+ * modified - the modification time of the XML file
+
+ * sections - an array of section entries, each with
+  * title - section name
+  * words - word count
+  * level - section level in the document hierarchy
+  * tag - the section's DocBook tag
+
+ * remarks - (optional) an array of remark entries, each with
+  * keyword - uppercase keyword, e.g. REMARK, FIXME
+  * text - remark text
+  * file - file name, location of the remark
+  * line - line number, location of the remark
+
+ * goals - (optional) information about the defined writing goals
+  * start - start date of the tracking
+  * end - scheduled end date or nil
+  * goal_total - planned total word count or 0
+  * goal_daily - planned daily word count or 0
+
+ * today - (optional) document size information
+  * min - minimum no. of words
+  * max - maximum no. of words
+  * start - first word count
+  * end - last (current) word count
+  * ctr - number of runs
+
+== Download
+
+https://rubygems.org/gems/docbook_status
+
+== Requirements
+
+* libxml2
+
+== Install
+
+* gem install docbook_status
+
+== License
+
+The MIT License
+
+Copyright (c) 2011 Rainer Volz
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -1,4 +1,4 @@
-
+#-*- encoding: utf-8 ; mode:ruby -*-
 begin
   require 'bones'
 rescue LoadError
@@ -13,5 +13,8 @@ Bones {
   authors  'Rainer Volz'
   email    'dev@textmulch.de'
   url      'http://projekte.textmulch.de/docbook_status/'
+  depend_on     'directory_watcher'
+  depend_on     'libxml-ruby'
+  depend_on     'json'
+  depend_on     'term-ansicolor'
 }
-

data/bin/docbook_status

@@ -1,68 +1,233 @@
 #!/usr/bin/env ruby
 # -*-encoding:utf-8 ; mode:ruby-*-
 ##
-# docbook_status is the commandline application for the library.
-# Its main purpose is to display the structure and word counts for DocBook 5 documents.
-# docbook_status can be used in single-run mode or demon-mode. In demon_mode it stays active
-# and looks for changes in the filesystem.
+# docbook_status is the commandline application for the library.  Its
+# main purpose is to display the structure and word counts for DocBook
+# 5 documents. docbook_status can be used in single-run mode or
+# demon-mode. In demon-mode it stays active and looks for changes in
+# the filesystem.
 #
 
 require 'rubygems'
 require "bundler/setup"
 require 'optparse'
 require 'directory_watcher'
+require 'yaml'
+require 'json'
+require 'term/ansicolor'
+
+class String
+  include Term::ANSIColor
+end
 
 require File.expand_path(File.join(File.dirname(__FILE__), %w[.. lib docbook_status]))
 
+NOVAL = -1
+
 @demon = false
 @glob = '*.xml'
 @dir = '.'
 @pre = []
+@remarks = false
+@remarks_filter = []
+@output_format = :screen
+@total_words = NOVAL
+@daily_words = NOVAL
+@end_date = NOVAL
+
 banner = <<EOB
 docbook_status, Version #{DocbookStatus.version}
 
 Display DocBook 5 document structure and word counts.
 
-Usage: docbook_status [--demon] [--glob <PATTERN>] [--dir <DIR>] [--pre COMMAND] <DOCBOOK-FILE>
+Usage: docbook_status [options] <DOCBOOK-FILE>
 EOB
 
 opts = OptionParser.new
 opts.banner = banner
 opts.on('--demon', 'Keep running, act when files change') {|val| @demon = true}
 opts.on('--glob PATTERN', String, 'File mask for demon mode, default = "*.xml"') {|val| @glob = val}
-opts.on('--dir DIR', String, 'Source directory for demon mode, default = "." ') {|val| @dir = val}
+opts.on('--dir DIR', String, 'Source directory for demon mode, default = "." ') {|val| @dir = File.expand_path(val)}
 opts.on('--pre COMMAND', String, 'A shell command that should be executed before') {|val| @pre << val}
+opts.on('--remarks[=FILTER]', String, 'Show the remarks, comments. Use a keyword FILTER to restrict the listing') {|val|
+  @remarks = true
+  unless (val.nil? || val.empty?)
+    @remarks_filter << val
+  end
+}
+opts.on('--end[=DATE]', String, 'Goal: planned delivery date, YYYY-MM-DD') {|val|
+  if val.nil?
+    @end_date = nil
+  else
+    @end_date = Date.parse(val)
+  end
+}
+opts.on('--total[=WORDS]',Integer, 'Goal: total number of words') {|val|
+  if val.nil?
+    @total_words = 0
+  else
+    @total_words = val.abs
+  end}
+opts.on('--daily[=WORDS]',Integer, 'Goal: daily number of words') {|val|
+  if val.nil?
+    @daily_words = 0
+  else
+    @daily_words = val.abs
+  end}
+opts.on('--outputformat=yaml|json',['json','yaml'],'Return the result in YAML or JSON format instead of printing it') {|format|
+  case
+  when format == 'yaml'
+    @output_format = :yaml
+  when format == 'json'
+    @output_format = :json
+  else
+    STDERR.puts "Unknown output format #{format}. Using screen output."
+    @output = :screen
+  end
+}
 rest = opts.parse(ARGV)
 
-def run(file)
-  @pre.each {|cmd| system(cmd)}
-  doc = XML::Document.file(file)
-  dbs = DocbookStatus.new
-  if !dbs.is_docbook?(doc)
-    puts "Error: #{file} is apparently not DocBook 5."
-    return
+# See, if there is need to track the writing progress.
+def history_on?()
+  (DocbookStatus::History.exists? || @end_date != NOVAL || @total_words != NOVAL || @daily_words != NOVAL)
+end
+
+
+# Color the actuals according to the schema
+#
+# * bold red if <= 30%
+# * red if > 30% and <= 70%
+# * magenta if >70 and < 100%
+# * green if >= 100%
+#
+def meter(actual,goal)
+  ad = actual.to_f / goal.to_f
+  percs = ""
+  case
+  when ad <= 0.3
+    fct = lambda {|s| s.red.bold}
+  when ad > 0.3 && ad <= 0.7
+    fct = lambda {|s| s.red}
+  when ad > 0.7 && ad < 1.0
+    fct = lambda {|s| s.magenta}
+  else
+    fct = lambda {|s| s.green}
+  end
+  [actual.to_s,"%3.0f\%" % (ad*100)].map {|s| fct.call(s)}
+end
+
+# Format the output for the screen and print it
+#
+def output_terminal(doc_info)
+
+  # Header
+  puts
+  puts "File:               #{doc_info[:file]}"
+  puts "Modified:           #{doc_info[:modified]}"
+
+  # Goal section
+  unless doc_info[:goals].nil?
+    gdend = doc_info[:goals][:end]
+    gwtotal = doc_info[:goals][:goal_total]
+    gwdaily = doc_info[:goals][:goal_daily]
+    unless gwtotal == 0
+      tactual = doc_info[:sections][0][:words]
+      (tactuals,tactperc) = meter(tactual,gwtotal)
+    end
+    unless gwdaily == 0
+      # Coloring the actuals
+      dactual = doc_info[:today][:end] - doc_info[:today][:start]
+      (dactuals,dactperc) = meter(dactual,gwdaily)
+    end
+    unless gdend.nil?
+      dstart = doc_info[:goals][:start]
+      ddur = (gdend - dstart).round
+      ddiff = (gdend - Date.today).round
+      dsleft = case
+               when ddiff == 0 then 'today'
+               when ddiff > 0 then "#{ddiff} day(s) left"
+               else "#{ddiff} day(s) behind schedule"
+               end
+    end
+    unless (gdend.nil? && gwtotal == 0 && gwdaily == 0)
+      puts "Goals".bold
+      puts "Delivery:           #{dsleft}, #{gdend}" unless gdend.nil?
+      puts "Words, total:       #{tactperc} #{tactuals}/#{gwtotal}" unless gwtotal == 0
+      puts "Words, daily:       #{dactperc} #{dactuals}/#{gwdaily}" unless gwdaily == 0
+    end
   end
-  doc.xinclude if dbs.has_xinclude?(doc)
-  sections = dbs.analyze_document(doc)
+  # Structure
   puts
-  puts "File:               #{file}"
-  puts "Modified:           #{File.ctime(file)}"
-  puts "Document structure:"
-  puts "%-50.50s %-16s %5s" % ['Title','Tag','Words']
-  puts "-"*73
-  sections.each do |s|
-    puts "%-50.50s %-16s %5d" % [(' ' * s[2])+s[0], s[3], s[1]]
+  puts "Document structure".bold
+  puts "%-57.57s %-16s %5s" % ['Title','Tag','Words']
+  puts "-"*80
+  doc_info[:sections].each do |s|
+    puts "%-57.57s %-16s %5d" % [(' ' * s[:level])+s[:title], s[:tag], s[:words]]
+  end
+  #Remarks
+  if doc_info[:remarks]
+    puts
+    puts "Remarks".bold
+    puts "%-10s %10s %5s %-52s" % %w[Type File Line Content]
+    puts "-"*80
+    doc_info[:remarks].each do |r|
+      puts "%-10.10s %10s %5d %-52.52s" % ["#{r[:keyword]}",r[:file],r[:line],r[:text]]
+    end
   end
 end
 
+# Processes the DocBook document once. Runs first all defined
+# preprocessing (--pre) commands. Checks then for a DocBook namespace
+# declaration, which would imply DocBook 5. If the namespaces include
+# the XInclude-NS, XInclude-processing is started. At last the
+# resulting document is analyzed.
+#
+# If history processing is on the writing progress is recorded.
+#
+def run(file)
+  # OPTIMIZE Detailed output for --pre commands with popen4?
+  @pre.each { |cmd|
+    ret = system(cmd)
+    unless ret
+        STDERR.puts "Error: This preprocessing command signalled failure (#{$?}), please check --> #{cmd}"
+        return
+    end
+  }
+  dbs = DocbookStatus::Status.new(file)
+  doc_info = dbs.analyze_file
+  doc_info[:remarks] = dbs.find_remarks(@remarks_filter) if @remarks
+  if history_on?()
+    dbs_h = DocbookStatus::History.new(file)
+    dbs_h.planned_end(@end_date) if @end_date != NOVAL
+    dbs_h.total_words(@total_words) if @total_words != NOVAL
+    dbs_h.daily_words(@daily_words) if @daily_words != NOVAL
+    dbs_h.add(DateTime.now,doc_info[:sections][0][:words])
+    dbs_h.save
+    doc_info[:goals] = dbs_h.goals
+    doc_info[:today] = dbs_h.today
+  end
+  case
+  when @output_format == :yaml
+    YAML.dump(doc_info,STDOUT)
+  when @output_format == :json
+    STDOUT.puts doc_info.to_json
+  else
+    output_terminal(doc_info)
+  end
+end
+
+
+# Demon routine for continuous processing of a DocBook document.
+# This routine calls _run_ whenever the filesystem signals changes. To
+# reduce load it waits for a defined time (interval * stable) before
+# starting the actual processing.
+#
 def demon(file)
   dw = DirectoryWatcher.new @dir, :glob => @glob, :pre_load => true
   dw.interval = 5.0
   dw.stable = 2
-  dw.persist = "dw_state.yml"
   dw.add_observer {|*args|
     args.each {|event|
-      #puts event
       if event.type == :stable
         run(file)
       end
@@ -79,7 +244,13 @@ if rest.length < 1
   exit 1
 end
 
-puts("docbook_status, Version #{DocbookStatus.version}")
+unless File.exists?(rest[0])
+  STDERR.puts "Error: File #{rest[0]} not found."
+  exit 1
+end
+
+# The main routine
+puts("docbook_status, Version #{DocbookStatus.version}") if @output_format == :screen
 run(rest[0])
 if (@demon)
   demon(rest[0])