nanoc3 3.1.9 → 3.2.0a1

data/LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2007-2011 Denis Defreyne and contributors
+Copyright (c) 2007-2010 Denis Defreyne and contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/NEWS.md

@@ -1,55 +1,5 @@
 # nanoc news
 
-## 3.1.9 (2011-06-30)
-
-* Really fixed dependency generation between Sass partials this time
-* Updated Less filter to 2.0
-* Made colorize_syntax filter throw an error if pygmentize is not available
-
-## 3.1.8 (2011-06-25)
-
-* Made link validator accept https: URLs
-* Fixed erronous handling of layouts with names ending in index
-* Fixed dependency generation between Sass partials
-* Fixed errors related to thread requires
-* Fixed crash while handling load errors
-* Improved encoding handling while reading files
-
-## 3.1.7 (2011-05-03)
-
-* Restored compatibility with Sass 3.1
-
-## 3.1.6 (2010-11-21)
-
-* Fixed issues with incompatible encodings
-
-## 3.1.5 (2010-08-24)
-
-* Improved `#render` documentation
-* Improved metadata section check so that e.g. raw diffs are handled properly
-* Deprecated using `Nanoc3::Site#initialize` with a non-`"."` argument
-* Added Ruby engine to version string
-* Allowed the `created_at` and `updated_at` attributes used in the `Blogging`
-  helper to be `Date` instances
-
-## 3.1.4 (2010-07-25)
-
-* Made INT and TERM signals always quit the CLI
-* Allowed relative imports in LESS
-* Made sure modification times are unchanged for identical recompiled items
-* Improved link validator error handling
-* Made pygmentize not output extra divs and pres
-* Allowed colorizers to be specified using symbols instead of strings
-* Added scss to the default list of text extensions
-
-## 3.1.3 (2010-04-25)
-
-* Removed annoying win32console warning [Eric Sunshine]
-* Removed color codes when not writing to a terminal, or when writing to
-  Windows’ console when win32console is not installed [Eric Sunshine]
-* Added .xhtml and .xml to list of text extensions
-* Improved support for relative Sass @imports [Chris Eppstein]
-
 ## 3.1.2 (2010-04-07)
 
 * Fixed bug which could cause incorrect output when compilation of an item is

data/README.md

@@ -52,38 +52,26 @@ you want to explore nanoc from a technical perspective.
 
 ## Dependencies
 
-nanoc has few dependencies. It is possible to use nanoc programmatically
-without any dependencies at all, but if you want to use nanoc in a proper way,
-you’ll likely need some dependencies:
+nanoc itself can be used without installing any dependencies. Some
+components, however, do have dependencies:
 
-* The **commandline frontend** depends on `cli`.
 * The **autocompiler** depends on `mime-types` and `rack`.
-* Filters and helpers likely have dependencies on their own too.
-
-If you’re developing for nanoc, such as writing custom filters or helpers, you
-may be interested in the development dependencies:
-
 * For **documentation generation** you’ll need `yard`.
 * For **packaging** you’ll need `rubygems` (1.3 or newer).
-* For **testing** you’ll need `mocha` and `minitest`.
+* For **testing** you’ll need `mocha`.
 
 ## Contributors
 
 (In alphabetical order)
 
-* Ben Armston
 * Colin Barrett
 * Dmitry Bilunov
 * Brian Candler
 * Chris Eppstein
-* Felix Hanley
 * Starr Horne
-* Tuomas Kareinen
 * Nicky Peeters
 * Christian Plessl
 * Šime Ramov
-* Xavier Shay
-* Arnau Siches
 * “Soryu”
 * Eric Sunshine
 * Dennis Sutch

data/bin/nanoc3

@@ -1,6 +1,8 @@
 #!/usr/bin/env ruby
 # encoding: utf-8
 
+puts "Starting..."
+
 # Add lib to load path
 $LOAD_PATH.unshift(File.expand_path(File.dirname(__FILE__) + '/../lib'))
 

data/lib/nanoc3/base/checksummer.rb

@@ -0,0 +1,40 @@
+# encoding: utf-8
+
+module Nanoc3
+
+  # Responsible for creating checksums of files. Such checksums are used to
+  # determine whether files have changed between compilations.
+  class Checksummer
+
+    # Returns a checksum for the specified file. Multiple invocations of this
+    # method with the same filename argument will yield the same result.
+    #
+    # The returned checksum has the property that for two given files with
+    # different content, the returned checksum will be different with a very
+    # high probability. In practice, this boils down to using a secure
+    # cryptographic hash function, such as SHA-1.
+    #
+    # The format of the resulting checksum should not be relied upon. In
+    # future versions of nanoc, this function may use a different method for
+    # generating checksums.
+    #
+    # @param [String] filename The name of the file for which the checksum
+    #   should be calculated
+    #
+    # @return [String] The checksum for the given files
+    def self.checksum_for(filename)
+      require 'digest'
+
+      digest = Digest::SHA1.new
+      File.open(filename, 'r') do |io|
+        until io.eof
+          data = io.readpartial(2**10)
+          digest.update(data)
+        end
+      end
+      digest.hexdigest
+    end
+
+  end
+
+end

data/lib/nanoc3/base/code_snippet.rb

@@ -2,9 +2,7 @@
 
 module Nanoc3
 
-  # Nanoc3::CodeSnippet represent a piece of custom code of a nanoc site. It
-  # contains the textual source code as well as a mtime, which is used to
-  # speed up site compilation.
+  # Nanoc3::CodeSnippet represent a piece of custom code of a nanoc site.
   class CodeSnippet
 
     # The {Nanoc3::Site} this code snippet belongs to.
@@ -22,24 +20,38 @@ module Nanoc3
     # @return [String]
     attr_reader :filename
 
-    # The time where this code snippet was last modified.
-    #
-    # @return [Time]
-    attr_reader :mtime
+    # @return [String] The checksum of this code snippet that was in effect
+    #   during the previous site compilation
+    attr_accessor :old_checksum
+
+    # @return [String] The current, up-to-date checksum of this code snippet
+    attr_reader   :new_checksum
 
     # Creates a new code snippet.
     #
     # @param [String] data The raw source code which will be executed before
-    # compilation
+    #   compilation
     #
     # @param [String] filename The filename corresponding to this code snippet
     #
-    # @param [Time] mtime The time when the code was last modified (can be
-    # nil)
-    def initialize(data, filename, mtime=nil)
+    # @param [Time, Hash] params Extra parameters. For backwards
+    #   compatibility, this can be a Time instance indicating the time when
+    #   this code snippet was last modified (mtime).
+    #
+    # @option params [Time, nil] :mtime (nil) The time when this code snippet
+    #   was last modified
+    #
+    # @option params [String, nil] :checksum (nil) The current, up-to-date
+    #   checksum of this code snippet
+    def initialize(data, filename, params=nil)
+      # Get mtime and checksum
+      params ||= {}
+      params = { :mtime => params } if params.is_a?(Time)
+      @new_checksum = params[:checksum]
+      @mtime        = params[:mtime]
+
       @data     = data
       @filename = filename
-      @mtime    = mtime
     end
 
     # Loads the code by executing it.
@@ -49,6 +61,12 @@ module Nanoc3
       eval(@data, TOPLEVEL_BINDING, @filename)
     end
 
+    # @return [Boolean] true if the code snippet was modified since it was
+    #   last compiled, false otherwise
+    def outdated?
+      !self.old_checksum || !self.new_checksum || self.new_checksum != self.old_checksum
+    end
+
   end
 
 end

data/lib/nanoc3/base/compiled_content_cache.rb

@@ -0,0 +1,86 @@
+# encoding: utf-8
+
+module Nanoc3
+
+  # Represents a cache than can be used to store already compiled content,
+  # to prevent it from being needlessly recompiled.
+  #
+  # This class is intended for internal use only. Do not rely on its
+  # presence; future versions of nanoc, even in the 3.x branch, may no
+  # longer contain this class.
+  class CompiledContentCache
+
+    # @return [String] The filename where the cache will be loaded from
+    #   and stored to
+    attr_reader :filename
+
+    # Creates a new cache for the given filename.
+    #
+    # @param [String] filename The filename where the cache will be loaded
+    #   from and stored to
+    def initialize(filename)
+      require 'pstore'
+
+      @filename = filename
+    end
+
+    # Loads the cache from the filesystem into memory.
+    #
+    # @return [void]
+    def load
+      cache = nil
+      return if !File.file?(filename)
+      pstore.transaction { cache = pstore[:compiled_content] }
+    end
+
+    # Stores the content of the (probably modified) in-memory cache to the
+    #   filesystem.
+    #
+    # @return [void]
+    def store
+      FileUtils.mkdir_p(File.dirname(filename))
+      pstore.transaction { pstore[:compiled_content] = cache }
+    end
+
+    # Returns the cached compiled content for the given item
+    # representation. This cached compiled content is a hash where the keys
+    # are the snapshot names and the values the compiled content at the
+    # given snapshot.
+    #
+    # @param [Nanoc3::ItemRep] rep The item rep to fetch the content for
+    #
+    # @return [Hash<Symbol,String>] A hash containing the cached compiled
+    #   content for the given item representation
+    def [](rep)
+      item_cache = cache[rep.item.identifier] || {}
+      item_cache[rep.name]
+    end
+
+    # Sets the compiled content for the given representation.
+    #
+    # @param [Nanoc3::ItemRep] rep The item representation for which to set
+    #   the compiled content
+    #
+    # @param [Hash<Symbol,String>] content A hash containing the compiled
+    #   content of the given representation
+    #
+    # @return [void]
+    def []=(rep, content)
+      cache[rep.item.identifier] ||= {}
+      cache[rep.item.identifier][rep.name] = content
+    end
+
+  private
+
+    def cache
+      @cache ||= {}
+    end
+
+    def pstore
+      require 'pstore'
+      @store ||= PStore.new(@filename)
+    end
+
+  end
+
+end

data/lib/nanoc3/base/compiler.rb

@@ -3,8 +3,38 @@
 module Nanoc3
 
   # Responsible for compiling a site’s item representations.
+  #
+  # The compilation process makes use of notifications (see
+  # {Nanoc3::NotificationCenter}) to track dependencies between items,
+  # layouts, etc. The following notifications are used:
+  #
+  # * `compilation_started` — indicates that the compiler has started
+  #   compiling this item representation. Has one argument: the item
+  #   representation itself. Only one item can be compiled at a given moment;
+  #   therefore, it is not possible to get two consecutive
+  #   `compilation_started` notifications without also getting a
+  #   `compilation_ended` notification in between them.
+  #
+  # * `compilation_ended` — indicates that the compiler has finished compiling
+  #   this item representation (either successfully or with failure). Has one
+  #   argument: the item representation itself.
+  #
+  # * `visit_started` — indicates that the compiler requires content or
+  #   attributes from the item representation that will be visited. Has one
+  #   argument: the visited item identifier. This notification is used to
+  #   track dependencies of items on other items; a `visit_started` event
+  #   followed by another `visit_started` event indicates that the item
+  #   corresponding to the former event will depend on the item from the
+  #   latter event.
+  #
+  # * `visit_ended` — indicates that the compiler has finished visiting the
+  #   item representation and that the requested attributes or content have
+  #   been fetched (either successfully or with failure)
   class Compiler
 
+    # The name of the file where cached compiled content will be stored
+    COMPILED_CONTENT_CACHE_FILENAME = 'tmp/compiled_content'
+
     # The compilation stack. When the compiler begins compiling a rep or a
     # layout, it will be placed on the stack; when it is done compiling the
     # rep or layout, it will be removed from the stack.
@@ -47,34 +77,31 @@ module Nanoc3
     # representations.
     #
     # @param [Nanoc3::Item] item The item that should be compiled, along with
-    # its dependencies. Pass `nil` if the entire site should be compiled.
+    #   its dependencies. Pass `nil` if the entire site should be compiled.
     #
     # @option params [Boolean] :force (false) true if the rep should be
-    # compiled even if it is not outdated, false if not
+    #   compiled even if it is not outdated, false if not
     #
     # @return [void]
     def run(item=nil, params={})
+      # Parse params
+      params[:force] = false if !params.has_key?(:force)
+
       # Create output directory if necessary
       FileUtils.mkdir_p(@site.config[:output_dir])
 
-      # Load dependencies
+      # Load necessary data
+      compiled_content_cache = CompiledContentCache.new(COMPILED_CONTENT_CACHE_FILENAME)
+      compiled_content_cache.load
       dependency_tracker.load_graph
 
       # Get items and reps to compile
-      if item
-        items = [ item ] + dependency_tracker.successors_of(item)
-        items.uniq!
-      else
-        items = @site.items
-      end
+      items = item ? ([ item ] + dependency_tracker.successors_of(item)).uniq : @site.items
       reps = items.map { |i| i.reps }.flatten
 
-      # Prepare dependencies
-      if params.has_key?(:force) && params[:force]
-        reps.each { |r| r.force_outdated = true }
-      else
-        dependency_tracker.propagate_outdatedness
-      end
+      # Determine which reps need to be recompiled
+      reps.each { |r| r.force_outdated = true } if params[:force]
+      dependency_tracker.propagate_outdatedness
       forget_dependencies_if_outdated(items)
 
       # Compile reps
@@ -82,11 +109,21 @@ module Nanoc3
       compile_reps(reps)
       dependency_tracker.stop
 
+      # Store necessary data
+      compiled_content_cache.store
+      @site.store_checksums
+      dependency_tracker.store_graph
+    ensure
       # Cleanup
       FileUtils.rm_rf(Nanoc3::Filter::TMP_BINARY_ITEMS_DIR)
+    end
 
-      # Store dependencies
-      dependency_tracker.store_graph
+    # Returns the dependency tracker for this site, creating it first if it
+    # does not yet exist.
+    # 
+    # @return [Nanoc3::DependencyTracker] The dependency tracker for this site
+    def dependency_tracker
+      @dependency_tracker ||= Nanoc3::DependencyTracker.new(@site.items + @site.layouts)
     end
 
     # Finds the first matching compilation rule for the given item
@@ -95,7 +132,7 @@ module Nanoc3
     # @param [Nanoc3::ItemRep] rep The item rep for which to fetch the rule
     #
     # @return [Nanoc3::Rule, nil] The compilation rule for the given item rep,
-    # or nil if no rules have been found
+    #   or nil if no rules have been found
     def compilation_rule_for(rep)
       @item_compilation_rules.find do |rule|
         rule.applicable_to?(rep.item) && rule.rep_name == rep.name
@@ -107,19 +144,37 @@ module Nanoc3
     # @param [Nanoc3::ItemRep] rep The item rep for which to fetch the rule
     #
     # @return [Nanoc3::Rule, nil] The routing rule for the given item rep, or
-    # nil if no rules have been found
+    #   nil if no rules have been found
     def routing_rule_for(rep)
       @item_routing_rules.find do |rule|
         rule.applicable_to?(rep.item) && rule.rep_name == rep.name
       end
     end
 
+    # Returns the list of routing rules that can be applied to the given item
+    # representation. For each snapshot, the first matching rule will be
+    # returned. The result is a hash containing the corresponding rule for
+    # each snapshot.
+    #
+    # @return [Hash<Symbol, Nanoc3::Rule>] The routing rules for the given rep
+    def routing_rules_for(rep)
+      rules = {}
+      @item_routing_rules.each do |rule|
+        next if !rule.applicable_to?(rep.item)
+        next if rule.rep_name != rep.name
+        next if rules.has_key?(rule.snapshot_name)
+
+        rules[rule.snapshot_name] = rule
+      end
+      rules
+    end
+
     # Finds the filter name and arguments to use for the given layout.
     #
     # @param [Nanoc3::Layout] layout The layout for which to fetch the filter.
     #
     # @return [Array, nil] A tuple containing the filter name and the filter 
-    # arguments for the given layout.
+    #   arguments for the given layout.
     def filter_for_layout(layout)
       @layout_filter_mapping.each_pair do |layout_identifier, filter_name_and_args|
         return filter_name_and_args if layout.identifier =~ layout_identifier
@@ -135,69 +190,41 @@ module Nanoc3
     #
     # @return [void]
     def compile_reps(reps)
-      active_reps, skipped_reps = reps.partition { |rep| rep.outdated? || rep.item.outdated_due_to_dependencies? }
-      inactive_reps = []
-      compiled_reps = []
-
-      # Repeat as long as something is successfully compiled...
-      changed = true
-      until !changed
-        changed = false
-
-        # Attempt to compile all active reps
-        until active_reps.empty?
-          @stack.clear
-          begin
-            rep = active_reps.shift
-            puts "*** Attempting to compile #{rep.inspect}" if $DEBUG
-
-            @stack.push(rep)
-            compile_rep(rep)
-          rescue Nanoc3::Errors::UnmetDependency => e
-            puts "*** Attempt failed due to unmet dependency on #{e.rep.inspect}" if $DEBUG
-
-            # Reinitialize rep
-            rep.forget_progress
-
-            # Save rep to compile it later
-            inactive_reps << rep
-
-            # Add dependency to list of items to compile
-            unless active_reps.include?(e.rep) || inactive_reps.include?(e.rep)
-              changed = true
-              skipped_reps.delete(e.rep)
-              inactive_reps.unshift(e.rep)
-            end
-          else
-            puts "*** Attempt succeeded" if $DEBUG
-
-            changed = true
-            compiled_reps << rep
-          end
-          puts if $DEBUG
-        end
-
-        # Retry
-        if inactive_reps.empty?
-          puts "*** Nothing left to compile!" if $DEBUG
-          break
-        else
-          puts "*** No active reps left; activating all (#{inactive_reps.size}) inactive reps" if $DEBUG
-          puts if $DEBUG
-          active_reps   = inactive_reps
-          inactive_reps = []
-        end
+      require 'set'
+
+      # Partition in outdated and non-outdated
+      outdated_reps = Set.new
+      skipped_reps  = Set.new
+      reps.each do |rep|
+        target = (rep.outdated? || rep.item.outdated_due_to_dependencies?) ? outdated_reps : skipped_reps
+        target.add(rep)
       end
 
-      # Notify skipped reps
-      skipped_reps.each do |rep|
-        Nanoc3::NotificationCenter.post(:compilation_started, rep)
-        Nanoc3::NotificationCenter.post(:compilation_ended,   rep)
+      # Build graph for outdated reps
+      content_dependency_graph = Nanoc3::DirectedGraph.new(outdated_reps)
+
+      # Attempt to compile all active reps
+      loop do
+        # Find rep to compile
+        break if content_dependency_graph.roots.empty?
+        rep = content_dependency_graph.roots.each { |e| break e }
+        @stack = [ rep ]
+
+        begin
+          compile_rep(rep)
+          content_dependency_graph.delete_vertex(rep)
+        rescue Nanoc3::Errors::UnmetDependency => e
+          content_dependency_graph.add_edge(e.rep, rep)
+          unless content_dependency_graph.vertices.include?(e.rep)
+            skipped_reps.delete(e.rep)
+            content_dependency_graph.add_vertex(e.rep)
+          end
+        end
       end
 
-      # Raise error if some active but non-compileable reps are left
-      if !active_reps.empty?
-        raise Nanoc3::Errors::RecursiveCompilation.new(active_reps)
+      # Check whether everything was compiled
+      if !content_dependency_graph.vertices.empty?
+        raise Nanoc3::Errors::RecursiveCompilation.new(content_dependency_graph.vertices)
       end
     end
 
@@ -211,34 +238,35 @@ module Nanoc3
     #
     # @return [void]
     def compile_rep(rep)
-      # Start
       Nanoc3::NotificationCenter.post(:compilation_started, rep)
       Nanoc3::NotificationCenter.post(:visit_started,       rep.item)
 
-      # Apply matching rule
-      compilation_rule_for(rep).apply_to(rep)
-      rep.compiled = true
+      if !rep.outdated? && !rep.item.outdated_due_to_dependencies && compiled_content_cache[rep]
+        Nanoc3::NotificationCenter.post(:cached_content_used, rep)
+        rep.content = compiled_content_cache[rep]
+      else
+        rep.snapshot(:raw)
+        rep.snapshot(:pre, :final => false)
+        compilation_rule_for(rep).apply_to(rep)
+        rep.snapshot(:post) if rep.has_snapshot?(:post)
+        rep.snapshot(:last)
+      end
 
-      # Write if rep is routed
-      rep.write unless rep.raw_path.nil?
+      rep.compiled = true
+      compiled_content_cache[rep] = rep.content
+    rescue => e
+      rep.forget_progress
+      Nanoc3::NotificationCenter.post(:compilation_failed, rep)
+      raise e
     ensure
-      # Stop
       Nanoc3::NotificationCenter.post(:visit_ended,       rep.item)
       Nanoc3::NotificationCenter.post(:compilation_ended, rep)
     end
 
-    # Returns the dependency tracker for this site, creating it first if it
-    # does not yet exist.
-    # 
-    # @return [Nanoc3::DependencyTracker] The dependency tracker for this site
-    def dependency_tracker
-      @dependency_tracker ||= Nanoc3::DependencyTracker.new(@site.items)
-    end
-
     # Clears the list of dependencies for items that will be recompiled.
     #
     # @param [Array<Nanoc3::Item>] items The list of items for which to forget
-    # the dependencies
+    #   the dependencies
     #
     # @return [void]
     def forget_dependencies_if_outdated(items)
@@ -249,6 +277,17 @@ module Nanoc3
       end
     end
 
+    # Returns the cache used for storing compiled content.
+    #
+    # @return [CompiledContentCache] The compiled content cache
+    def compiled_content_cache
+      @compiled_content_cache ||= begin
+        cache = Nanoc3::CompiledContentCache.new(COMPILED_CONTENT_CACHE_FILENAME)
+        cache.load
+        cache
+      end
+    end
+
   end
 
 end