webgen 0.5.3 → 0.5.4

data/doc/reference_configuration.page

@@ -151,7 +151,7 @@ This reference describes all available configurations that can be set via the co
     Specifies whether path names should be considered case-sensitive (set to `false`) or
     case-insensitive (set to `true`).
 
-    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`
+    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`.
 
     * Examples:
 
@@ -162,7 +162,7 @@ This reference describes all available configurations that can be set via the co
 
     Specifies whether hidden files (those starting with a dot) should be used.
 
-    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`
+    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`.
 
     * Examples:
 
@@ -186,7 +186,7 @@ This reference describes all available configurations that can be set via the co
     Specifies whether output paths in the default language should have the language code in their
     name.
 
-    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`
+    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`.
 
     * Examples:
 
@@ -240,6 +240,31 @@ This reference describes all available configurations that can be set via the co
           config['contentprocessor.map']['newcp'] = 'Extension::MyNewContentProcessor'
 
 
+*   ### contentprocessor.erubis.use\_pi
+
+    Specifies whether Erubis should look for XML processing instructions or the standard ERB tags
+    when processing content.
+
+    * Syntax: `BOOLEAN` where `BOOLEAN` is either `true` or `false`.
+
+    * Examples:
+
+          contentprocessor.erubis.use_pi: true
+
+
+*   ### contentprocessor.erubis.options
+
+    This configuration option can be used to specify additional options that the Erubis processor
+    should use.
+
+    * Syntax: `\{KEY: VALUE, ...}` where `KEY` and `VALUE` are key-value pairs of options where
+      `KEY` needs to be a Symbol and not a String.
+
+    * Examples:
+
+          contentprocessor.erubis.options: {:trim: true}
+
+
 *   ### contentprocessor.tags.prefix
 
     Specifies the optional prefix that is used for tag names to avoid name clashes when another

data/doc/reference_metainfo.page

@@ -157,7 +157,8 @@ Controls whether the index path should appear in a breadcrumb trail despite the
 Sets a custom output path style for the specified path. The basename is substituted for the value
 `:cnbase` and the language for the value `:lang`. Strings are used verbatim. If `:lang` is specified
 in a sub-array, the whole sub-array is omitted, if the configuration option
-`sourcehandler.default_lang_in_output_path` is false.
+`sourcehandler.default_lang_in_output_path` is false. For more and detailed information, have look
+at the [output path creation section]({relocatable: manual.html#source-output}) of the manual!
 
 > This meta information has to be set BEFORE a node gets created. Setting this value is therefore
 > only useful, for example, in the `paths` block of a meta information backing file.

data/doc/sourcehandler/page.page

@@ -12,3 +12,19 @@ meta information.
 set, the default language specified using the configuration option `website.lang` is used. This
 means that for the default language set to English and a source path named `index.page`, the
 language meta information is automatically set to English.
+
+## Fragment nodes
+
+The page handler automatically generates fragment nodes for all found header tags in the block named
+`content` (i.e. `h1`, `h2`, ...) that have an `id` attribute set. The default markup language Maruku
+automatically generates an `id` attribute for all headers. If you use another markup language or
+plain old HTML, you might need to set the `id` attributes by hand.
+
+> The reason why only header tags with an `id` attribute are used is that only those can be
+> referenced and linked to later.
+{.information}
+
+The generated fragment nodes can be used like any other node. So you can link to them and use them
+in a menu. Concerning the menu, there is a setting for the `tag.menu.used_nodes` option called
+`fragments` which only uses the fragment node of the current page to generate a menu. This allows to
+generate a nice overview of the page.

data/doc/upgrading.page

@@ -3,31 +3,44 @@ title: Upgrading from 0.4.x
 ---
 # Upgrading
 
-Here are some helpers to make upgrading a webgen website from 0.4.x to 0.5.x more easy:
+Here are step-by-step instructions on how to update your webgen website from 0.4.x to 0.5.x:
 
-* **Files in [Webgen Page Format]({relocatable: webgen_page_format.html})**: Since the format of
-  these files changed a little bit you may need to adapt all your files that use it, that is
-  primarily page and template files. The main change in the format was a different use of the block
-  separator line. Whereas before you would write
+* **Update the configuration file `config.yaml`**
 
-      --- content, textile
+  The configuration file syntax as well as the names of the configuration options and some defaults
+  changed. For example, the default processing pipeline now uses Maruku (a Markdown converter) as
+  markup language processor instead of Textile. You can find an overview over all available
+  configuration options in the [configuration option reference]({relocatable:
+  reference_configuration.html}). Also have a look at the [configuration file
+  documentation]({relocatable: manual.html#website-configfile}) for more information on the syntax
+  of this file and the available helpers.
 
-  for specifying the name of the block and its processor, you now can specify any number of
-  options. Two options are currently used by webgen: `name` and `pipeline`. So you could change the
-  name and the processing pipeline of a block by using a block start line like:
+  * Name changes: All configuration options now use underscores to separate word parts instead of
+    camelCase.
+  * Syntax changes: The configuration options are not specific to a certain extension anymore. You
+    now need the full configuration option name to specify it. So instead of
 
-      --- name:other pipeline:tags,maruku,blocks
+        Tag/Menu:
+          maxLevels: 4
+
+    you now use
+
+        tag.menu.max_levels: 4
+
+* **Convert your `metainfo.yaml`**
 
-* **Block inclusion in template/page files**: The way how named blocks are included has
-  changed. This feature is now provided by the content processor [blocks]({relocatable:
-  contentprocessor/blocks.html}) instead of the tag `block`. This allows you to specify the point in
-  the processing pipeline when a block should be included. So you definitely need to update your
-  `default.template` file as well as any other page/template file where you used the `block` tag.
+  This file is not supported anymore since webgen 0.5.x uses a more flexible way for specifying meta
+  information and virtual paths. You need to migrate its data to `metainfo` and `virtual` files in
+  the source directory. Have a look at the documentation of the [metainfo source
+  handler]({relocatable: sourcehandler/metainfo.html}) and the [virtual source
+  handler]({relocatable: sourcehandler/virtual.html}).
+
+* **Update meta information names and values**
 
-* **Meta information names**: The names of some meta information keys have been changed. Meta
-  information names are not specified in camelCase anymore but with under\_scores. You can find a
-  complete list of supported meta information names in the [meta information
-  reference]({relocatable: reference_metainfo.html}). The most notable changes are:
+  The names of some meta information keys have been changed. Meta information names are not
+  specified in camelCase anymore but with under\_scores. You can find a complete list of supported
+  meta information names in the [meta information reference]({relocatable:
+  reference_metainfo.html}). The most notable changes are:
 
   * directoryName → routed\_title
   * inMenu → in\_menu
@@ -36,32 +49,91 @@ Here are some helpers to make upgrading a webgen website from 0.4.x to 0.5.x mor
   * outputNameStyle → output\_path\_style
   * orderInfo → sort\_info
 
-* **website\_dir/metainfo.yaml**: This file is not supported anymore since webgen 0.5.x uses a more
-  flexible way for specifying meta information and virtual paths. You need to migrate its data to
-  `metainfo` and `virtual` files in the source directory. Have a look at the documentation of the
-  [metainfo source handler]({relocatable: sourcehandler/metainfo.html}) and the [virtual source
-  handler]({relocatable: sourcehandler/virtual.html}).
+  Also be aware that the syntax of some meta information keys has changed. For example, all meta
+  information keys that took a source path name, e.g. `index_path`, now take an localized canonical
+  name.
 
-* **Configuration file syntax**: The configuration file syntax as well as the names of the
-  configuration options and some defaults changed. For example, the default processing pipeline now
-  uses Maruku (a Markdown converter) as markup language processor. You can find an overview over all
-  available configuration options in the [configuration option reference]({relocatable:
-  reference_configuration.html}). Also have a look at the [configuration file
-  documentation]({relocatable: manual.html#website-configfile}) for more information on the syntax
-  of this file and the available helpers.
+  You need to change the names/value in all places where meta information can be specified:
+
+  * `metainfo` files
+  * `virtual` files
+  * page and template files
+
+* **Files in [Webgen Page Format]({relocatable: webgen_page_format.html})**
+
+  Since the format of these files changed a little bit you may need to adapt all your files that use
+  it, that are primarily page and template files. The main change in the format was a different use
+  of the block start line. Whereas before you would write
+
+      --- content, textile
+
+  for specifying the name of the block and its processor, you now can specify any number of
+  options. Two options are currently used by webgen: `name` and `pipeline`. So you could change the
+  name and the processing pipeline of a block by using a block start line like:
+
+      --- name:other pipeline:tags,maruku,blocks
+
+* **Block inclusion in template/page files**
+
+  The way how named blocks are included has changed. This feature is now provided by the content
+  processor [blocks]({relocatable: contentprocessor/blocks.html}) instead of the tag `block`. This
+  allows you to specify the point in the processing pipeline when a block should be included. So you
+  definitely need to update your `default.template` file as well as any other page/template file
+  where you used the `block` tag.
+
+  So you need to look for `\{block: content}` tags (where `content` is just a place holder for the
+  name of the block that should be included) and replace them with `<webgen:block name='content'
+  />`.
+
+* **Update tag names and parameters**
+
+  Since the names of the configuration options changed (from using camelCase to using under\_scores)
+  and some tags have different options, you need to change all tag parameters. You may also need to
+  convert old tag names to new ones (same reason: camelCase to under\_score), for example,
+  `includeFile` is now `include_file`.
+
+* **Update your ERB code**
+
+  If you have any ERB code in your template or page files you will most certainly have to adapt them
+  to the new [API]({relocatable: api.html}). One thing that has been used often is the check if a
+  page file has a certain block:
+
+      <%% if node.node_info[:page_data].blocks.has_key?('NAME') %>
+      ...
+      <%% end %>
+
+  This needs to be changed into the following:
+
+      <%% if context.content_node.node_info[:page].blocks.has_key?('NAME') %>
+      ...
+      <%% end %>
+
+* **Extensions development**
+
+  Since the complete core of webgen has changed you need to rewrite all your plugins for the 0.5.x
+  series. Howver, webgen has complete [API documentation]({relocatable: api.html}) now which
+  provides you with all needed information as well as examples on how to implement source handlers,
+  tags, content processors, ... If you still have any questions, don't hesitate to contact me or
+  write a mail to the mailing list!
+
+* **Running webgen on the converted website**
+
+  You now can run webgen 0.5.x on the converted website. This helps in ironing out the remaining
+  errors, for example:
+
+  * If you have forgotten to change a block start line, you will get an application error and the
+    name of the file where the error occured.
 
-* **Extensions development**: Since the complete core of webgen has changed you need to rewrite all
-  your plugins for the 0.5.x series. Howver, webgen has complete [API documentation]({relocatable:
-  api.html}) now which provides you with all needed information as well as examples on how to
-  implement source handlers, tags, content processors, ... If you still have any questions, don't
-  hesitate to contact me or write a mail to the mailing list!
+  * If you have overlooked changing a tag parameter, you will find `ERROR` and `WARN` lines in the
+    log output showing you what still needs to be changed.
 
 * **Not Implement Yet**: There are several features of the 0.4.x series which are currently not
   implemented in the 0.5.x series:
 
   * source handlers: gallery, sipttra
   * tags: customvar (won't be ported), download, htmlmetainfo, news (won't be ported, superceded
-    by blogging support), wikilink
+    by blogging support), resource, wikilink
   * misc: smiley replacer, html validators
+  * CLI commands: check, show, use
 
   If you need any of those you have to wait till they are implemented or port them on your on.

data/doc/webgen_page_format.page

@@ -96,7 +96,9 @@ The first block has no identifieres set (there is no line with three dashes and
 Therefore the default value for the name is used: `content`. The second block is named `sidebar` and
 uses the processing pipeline `maruku,tags`. As you can see, the name of a block as well as
 additional options are specified by stating the key (e.g. `name` or `pipeline`) followed by a colon
-and the value. Multiple options are separated via one or more spaces.
+and the value. Multiple options are separated via one or more spaces. The value of a block option is
+parsed with YAML. For example, when specifying `use_something:true` the value `true` is
+automatically converted from the string `true` to the boolean `true`.
 
 > Only the first block gets the default name of `content`. The second and following blocks have
 > numbered names like `block2`, `block3` and so on.

data/lib/webgen/configuration.rb

@@ -35,7 +35,7 @@ module Webgen
     end
 
     # This module provides methods for setting more complex configuration options. It is mixed into
-    # Webgen::Configuration so that it methods can be used. Detailed information on the use of the
+    # Webgen::Configuration so that its methods can be used. Detailed information on the use of the
     # methods can be found in the "User Manual" in the "Configuration File" section.
     #
     # All public methods defined in this module are available for direct use in the
@@ -85,14 +85,27 @@ module Webgen
         end
       end
 
+
+      # Set the default processing pipeline for a source handler.
+      def default_processing_pipeline(args)
+        args.each do |sh_name, pipeline|
+          raise ArgumentError, 'Invalid argument for configuration helper pipeline' unless pipeline.kind_of?(String)
+          mi_hash = (self['sourcehandler.default_meta_info'][complete_source_handler_name(sh_name)] ||= {})
+          ((mi_hash['blocks'] ||= {})['default'] ||= {})['pipeline'] = pipeline
+        end
+      end
+
+
       # Complete +sh_name+ by checking if a source handler called
       # <tt>Webgen::SourceHandler::SH_NAME</tt> exists.
       def complete_source_handler_name(sh_name)
         (Webgen::SourceHandler.constants.include?(sh_name) ? 'Webgen::SourceHandler::' + sh_name : sh_name)
       end
       private :complete_source_handler_name
+
     end
 
+
     include Helpers
 
     # The hash which stores the meta info for the configuration options.

data/lib/webgen/contentprocessor.rb

@@ -63,6 +63,8 @@ module Webgen
     autoload :Sass, 'webgen/contentprocessor/sass'
     autoload :RDoc, 'webgen/contentprocessor/rdoc'
     autoload :Builder, 'webgen/contentprocessor/builder'
+    autoload :Erubis, 'webgen/contentprocessor/erubis'
+    autoload :RDiscount, 'webgen/contentprocessor/rdiscount'
 
     # Return the list of all available content processors.
     def self.list

data/lib/webgen/contentprocessor/erubis.rb

@@ -0,0 +1,40 @@
+require 'webgen/websiteaccess'
+
+module Webgen::ContentProcessor
+
+  # Processes embedded Ruby statements with the +erubis+ library.
+  class Erubis
+
+    include Webgen::WebsiteAccess
+
+    # Process the Ruby statements embedded in the content of +context+.
+    def call(context)
+      require 'erubis'
+      # including Erubis because of problem with resolving Erubis::XmlHelper et al
+      self.class.class_eval "include ::Erubis"
+
+      node = context.content_node
+      ref_node = context.ref_node
+      dest_node = context.dest_node
+
+      options = website.config['contentprocessor.erubis.options']
+      if context[:block]
+        use_pi = context[:block].options['erubis_use_pi']
+        context[:block].options.select {|k,v| k =~ /^erubis_/}.
+          each {|k,v| options[k.sub(/^erubis_/, '').to_sym] = v }
+      end
+      erubis = if (!use_pi.nil? && use_pi) || (use_pi.nil? && website.config['contentprocessor.erubis.use_pi'])
+                    ::Erubis::PI::Eruby.new(context.content, options)
+                  else
+                    ::Erubis::Eruby.new(context.content, options)
+                  end
+      erubis.filename = context.ref_node.absolute_lcn
+      context.content = erubis.result(binding)
+      context
+    rescue Exception => e
+      raise RuntimeError, "Erubis processing failed in <#{context.ref_node.absolute_lcn}>: #{e.message}", e.backtrace
+    end
+
+  end
+
+end

data/lib/webgen/contentprocessor/rdiscount.rb

@@ -0,0 +1,15 @@
+module Webgen::ContentProcessor
+
+  # Processes content in Markdown markup with the fast +rdiscount+ library.
+  class RDiscount
+
+    # Convert the content in +context+ to HTML.
+    def call(context)
+      require 'rdiscount'
+      context.content = ::RDiscount.new(context.content).to_html
+      context
+    end
+
+  end
+
+end

data/lib/webgen/default_config.rb

@@ -67,6 +67,7 @@ config.sourcehandler.default_lang_in_output_path(false, :doc => 'Specifies wheth
 
 config.sourcehandler.default_meta_info({
                                          :all => {
+                                           'output_path' => 'standard',
                                            'output_path_style' => [:parent, :cnbase, ['.', :lang], :ext]
                                          },
                                          'Webgen::SourceHandler::Copy' => {
@@ -117,7 +118,9 @@ Webgen::WebsiteAccess.website.blackboard.add_service(:output_instance, Webgen::O
 
 # All things regarding content processors
 config.contentprocessor.map({
+                              'markdown' => 'Webgen::ContentProcessor::Maruku',
                               'maruku' => 'Webgen::ContentProcessor::Maruku',
+                              'textile' => 'Webgen::ContentProcessor::RedCloth',
                               'redcloth' => 'Webgen::ContentProcessor::RedCloth',
                               'tags' => 'Webgen::ContentProcessor::Tags',
                               'blocks' => 'Webgen::ContentProcessor::Blocks',
@@ -126,6 +129,8 @@ config.contentprocessor.map({
                               'sass' => 'Webgen::ContentProcessor::Sass',
                               'rdoc' => 'Webgen::ContentProcessor::RDoc',
                               'builder' => 'Webgen::ContentProcessor::Builder',
+                              'erubis' => 'Webgen::ContentProcessor::Erubis',
+                              'rdiscount' => 'Webgen::ContentProcessor::RDiscount',
                             }, :doc => 'Content processor name to class map')
 
 Webgen::WebsiteAccess.website.blackboard.add_service(:content_processor_names, Webgen::ContentProcessor.method(:list))
@@ -146,6 +151,9 @@ config.contentprocessor.tags.map({
                                    :default => 'Webgen::Tag::Metainfo'
                                  }, :doc => 'Tag processor name to class map')
 
+config.contentprocessor.erubis.use_pi(false, :doc => 'Specify whether processing instructions should be used')
+config.contentprocessor.erubis.options({}, :doc => 'A hash of additional options')
+
 config.tag.relocatable.path(nil, :doc => 'The path which should be made relocatable', :mandatory => 'default')
 
 config.tag.menu.start_level(1, :doc => 'The level at which the menu starts.')

data/lib/webgen/node.rb

@@ -119,18 +119,22 @@ module Webgen
     # Return +true+ if the node has changed since the last webgen run. If it has changed, +dirty+ is
     # set to +true+.
     def changed?
-      @dirty = @dirty || meta_info_changed?
-      @dirty = node_info[:used_nodes].any? {|n| n != @absolute_lcn && (!tree[n] || tree[n].changed?)} unless @dirty
-      website.blackboard.dispatch_msg(:node_changed?, self) unless @dirty
+      if_not_checked(:node) do
+        @dirty = @dirty || meta_info_changed?
+        @dirty = node_info[:used_nodes].any? {|n| n != @absolute_lcn && (!tree[n] || tree[n].changed?)} unless @dirty
+        website.blackboard.dispatch_msg(:node_changed?, self) unless @dirty
+      end
       @dirty
     end
 
     # Return +true+ if the meta information of the node has changed.
     def meta_info_changed?
-      @dirty_meta_info = node_info[:used_meta_info_nodes].any? do |n|
-         n != @absolute_lcn && (!tree[n] || tree[n].meta_info_changed?)
-      end unless @dirty_meta_info
-      website.blackboard.dispatch_msg(:node_meta_info_changed?, self) unless @dirty_meta_info
+      if_not_checked(:meta_info) do
+        @dirty_meta_info = node_info[:used_meta_info_nodes].any? do |n|
+          n != @absolute_lcn && (!tree[n] || tree[n].meta_info_changed?)
+        end unless @dirty_meta_info
+        website.blackboard.dispatch_msg(:node_meta_info_changed?, self) unless @dirty_meta_info
+      end
       @dirty_meta_info
     end
 
@@ -317,6 +321,17 @@ module Webgen
       self.node_info[:used_meta_info_nodes] = Set.new
     end
 
+    # Only run the code in the block if this node has not already been checked. Different checks are
+    # supported by setting a different +type+ value.
+    def if_not_checked(type)
+      array = (website.cache.volatile[:node_change_checking] ||= {})[type] ||= []
+      if !array.include?(self)
+        array << self
+        yield
+        array.delete(self)
+      end
+    end
+
     # Delegate missing methods to a processor. The current node is placed into the argument array as
     # the first argument before the method +name+ is invoked on the processor.
     def method_missing(name, *args, &block)