jekyll 0.2.1 → 0.3.0

data/History.txt

@@ -1,3 +1,17 @@
+== 0.3.0 / 2008-12-24
+  * Major Enhancements
+    * Added --server option to start a simple WEBrick server on destination directory [github.com/johnreilly and github.com/mchung]
+  * Minor Enhancements
+    * Added post categories based on directories containing _posts [github.com/mreid]
+    * Added post topics based on directories underneath _posts
+    * Added new date filter that shows the full month name [github.com/mreid]
+    * Merge Post's YAML front matter into its to_liquid payload [github.com/remi]
+    * Restrict includes to regular files underneath _includes
+  * Bug Fixes
+    * Change YAML delimiter matcher so as to not chew up 2nd level markdown headers [github.com/mreid]
+    * Fix bug that meant page data (such as the date) was not available in templates [github.com/mreid]
+    * Properly reject directories in _layouts
+
 == 0.2.1 / 2008-12-15
   * Major Changes
     * Use Maruku (pure Ruby) for Markdown by default [github.com/mreid]

data/Manifest.txt

@@ -8,7 +8,10 @@ lib/jekyll.rb
 lib/jekyll/albino.rb
 lib/jekyll/converters/csv.rb
 lib/jekyll/converters/mephisto.rb
+lib/jekyll/converters/mt.rb
+lib/jekyll/converters/wordpress.rb
 lib/jekyll/convertible.rb
+lib/jekyll/core_ext.rb
 lib/jekyll/filters.rb
 lib/jekyll/layout.rb
 lib/jekyll/page.rb
@@ -22,11 +25,12 @@ test/source/_layouts/default.html
 test/source/_layouts/simple.html
 test/source/_posts/2008-10-18-foo-bar.textile
 test/source/_posts/2008-11-21-complex.textile
+test/source/_posts/2008-12-03-permalinked-post.textile
 test/source/_posts/2008-12-13-include.markdown
 test/source/css/screen.css
 test/source/index.html
-test/source/posts/2008-12-03-permalinked-post.textile
 test/suite.rb
+test/test_generated_site.rb
 test/test_jekyll.rb
 test/test_post.rb
 test/test_site.rb

data/README.textile

@@ -31,9 +31,10 @@ various data about my site. A reverse chronological list of all my blog posts
 can be found in <code>site.posts</code>. Each post, in turn, contains various
 fields such as <code>title</code> and <code>date</code>.
 
-Jekyll gets the list of blog posts by parsing the files in the
-"_posts":http://github.com/mojombo/tpw/tree/master/_posts directory. Each
-post's filename contains the publishing date and slug (what shows up in the
+Jekyll gets the list of blog posts by parsing the files in any
+"_posts":http://github.com/mojombo/tpw/tree/master/_posts directory found in
+subdirectories below the root. 
+Each post's filename contains the publishing date and slug (what shows up in the
 URL) that the final HTML file should have. Open up the file corresponding to a
 blog post:
 "2008-11-17-blogging-like-a-hacker.textile":http://github.com/mojombo/tpw/tree/master/_posts/2008-11-17-blogging-like-a-hacker.textile.
@@ -52,6 +53,12 @@ filename is used to construct the URL in the generated site. The example post,
 for instance, ends up at
 <code>http://tom.preston-werner.com/2008/11/17/blogging-like-a-hacker.html</code>.
 
+Categories for posts are derived from the directory structure the posts were
+found within. A post that appears in the directory foo/bar/_posts is placed in
+the categories 'foo' and 'bar'. By selecting posts from particular categories
+in your Liquid templates, you will be able to host multiple blogs within a
+site.
+
 Files that do not reside in directories prefixed with an underscore are
 mirrored into a corresponding directory structure in the generated site. If a
 file does not have a YAML preface, it is not run through the Liquid
@@ -129,6 +136,14 @@ compilation), you must install it (gem install rdiscount) and then you can
 have it used instead:
 
   $ jekyll --rdiscount
+  
+When previewing complex sites locally, simply opening the site in a web 
+browser (using file://) can cause problems with links that are relative to
+the site root (e.g., "/stylesheets/style.css"). To get around this, Jekyll
+can launch a simple WEBrick server (works well in conjunction with --auto).
+Default port is 4000:
+
+  $ jekyll --server [PORT]
 
 h2. Data
 
@@ -149,7 +164,7 @@ h3. Global
 
   content
     In layout files, this contains the content of the subview(s). In Posts or
-    pages, this is undefined.
+    Pages, this is undefined.
 
 h3. Site
 
@@ -165,6 +180,9 @@ h3. Site
     high quality but slow to compute results, run the jekyll command with the 
     --lsi (latent semantic indexing) option.
 
+  site.categories.CATEGORY
+    The list of all Posts in category CATEGORY.
+
 h3. Post
 
   post.title
@@ -181,6 +199,18 @@ h3. Post
     An identifier unique to the Post (useful in RSS feeds).
     e.g. /2008/12/14/my-post
 
+  post.categories
+    The list of categories to which this post belongs. Categories are
+    derived from the directory structure above the _posts directory. For
+    example, a post at /work/code/_posts/2008-12-24-closures.textile
+    would have this field set to ['work', 'code'].
+
+  post.topics
+    The list of topics for this Post. Topics are derived from the directory
+    structure beneath the _posts directory. For example, a post at
+    /_posts/music/metal/2008-12-24-metalocalypse.textile would have this field
+    set to ['music', 'metal'].
+
   post.content
     The content of the Post.
 
@@ -258,7 +288,8 @@ becomes
 
 h3. Include (Tag)
 
-If you have small page fragments that you wish to include in multiple places on your site, you can use the <code>include</code> tag.
+If you have small page fragments that you wish to include in multiple places
+on your site, you can use the <code>include</code> tag.
 
 <pre>{% include sig.textile %}</pre>
 
@@ -296,6 +327,22 @@ highlighting stylesheet. For an example stylesheet you can look at
 are the same styles as used by GitHub and you are free to use them for your
 own site.
 
+h2. Categories
+
+Posts are placed into categories based on the directory structure they are found
+within (see above for an example). The categories can be accessed from within
+a Liquid template as follows:
+
+<pre>
+{% for post in site.categories.foo %}
+	<li><span>{{ post.date | date_to_string }}</span> - {{ post.title }}</li>
+{% endfor %}
+</pre>
+
+This would list all the posts in the category 'foo' by date and title.
+
+The posts within each category are sorted in reverse chronological order.
+
 h2. Contribute
 
 If you'd like to hack on Jekyll, grab the source from GitHub. To get
@@ -314,6 +361,24 @@ The best way to get your changes merged back into core is as follows:
 # Push the branch up to GitHub
 # Send me (mojombo) a pull request for your branch
 
+h2. Blog migrations
+
+h3. Movable Type
+
+To migrate your MT blog into Jekyll, you'll need read access to the database.
+The lib/jekyll/converters/mt.rb module provides a simple convert to create
+.markdown files in a _posts directory based on the entries contained therein.
+
+  $ export DB=my_mtdb
+  $ export USER=dbuser
+  $ export PASS=dbpass
+  $ ruby -r './lib/jekyll/converters/mt' -e 'Jekyll::MT.process( \
+    "#{ENV["DB"]}", "#{ENV["USER"]}", "#{ENV["PASS"]}")'
+
+You may need to adjust the SQL query used to retrieve MT entries. Left alone,
+it will attempt to pull all entries across all blogs regardless of status.
+Please check the results and verify the posts before publishing.
+
 h2. License
 
 (The MIT License)

data/Rakefile

@@ -18,4 +18,7 @@ namespace :convert do
   task :mephisto do
     sh %q(ruby -r './lib/jekyll/converters/mephisto' -e 'Jekyll::Mephisto.postgres(:database => "#{ENV["DB"]}")')
   end
+  task :mt do
+    sh %q(ruby -r './lib/jekyll/converters/mt' -e 'Jekyll::MT.process("#{ENV["DB"]}", "#{ENV["USER"]}", "#{ENV["PASS"]}")')
+  end
 end

data/bin/jekyll

@@ -25,6 +25,11 @@ opts = OptionParser.new do |opts|
     options[:auto] = true
   end
   
+  opts.on("--server [PORT]", "Start web server (default port 4000)") do |port|
+    options[:server] = true
+    options[:server_port] = port || 4000
+  end
+  
   opts.on("--lsi", "Use LSI for better related posts") do
     Jekyll.lsi = true
   end
@@ -95,7 +100,28 @@ if options[:auto]
   
   dw.start
   
-  loop { sleep 1000 }
+  unless options[:server]
+    loop { sleep 1000 }
+  end
 else
   Jekyll.process(source, destination)
+  puts "Successfully generated site in #{destination}"
+end
+
+if options[:server]
+  require 'webrick'
+  include WEBrick
+  
+  FileUtils.mkdir_p(destination)
+
+  s = HTTPServer.new(
+    :Port            => options[:server_port],
+    :DocumentRoot    => destination
+  )
+  t = Thread.new {
+    s.start
+  }
+  
+  trap("INT") { s.shutdown }
+  t.join()
 end

data/jekyll.gemspec

@@ -1,22 +1,22 @@
 Gem::Specification.new do |s|
   s.name = %q{jekyll}
-  s.version = "0.2.1"
+  s.version = "0.3.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Tom Preston-Werner"]
-  s.date = %q{2008-12-15}
+  s.date = %q{2008-12-24}
   s.default_executable = %q{jekyll}
   s.email = ["tom@mojombo.com"]
   s.executables = ["jekyll"]
   s.extra_rdoc_files = ["History.txt", "Manifest.txt"]
-  s.files = ["History.txt", "Manifest.txt", "README.textile", "Rakefile", "bin/jekyll", "jekyll.gemspec", "lib/jekyll.rb", "lib/jekyll/albino.rb", "lib/jekyll/converters/csv.rb", "lib/jekyll/converters/mephisto.rb", "lib/jekyll/convertible.rb", "lib/jekyll/filters.rb", "lib/jekyll/layout.rb", "lib/jekyll/page.rb", "lib/jekyll/post.rb", "lib/jekyll/site.rb", "lib/jekyll/tags/highlight.rb", "lib/jekyll/tags/include.rb", "test/helper.rb", "test/source/_includes/sig.markdown", "test/source/_layouts/default.html", "test/source/_layouts/simple.html", "test/source/_posts/2008-10-18-foo-bar.textile", "test/source/_posts/2008-11-21-complex.textile", "test/source/_posts/2008-12-13-include.markdown", "test/source/css/screen.css", "test/source/index.html", "test/source/posts/2008-12-03-permalinked-post.textile", "test/suite.rb", "test/test_jekyll.rb", "test/test_post.rb", "test/test_site.rb"]
+  s.files = ["History.txt", "Manifest.txt", "README.textile", "Rakefile", "bin/jekyll", "jekyll.gemspec", "lib/jekyll.rb", "lib/jekyll/albino.rb", "lib/jekyll/converters/csv.rb", "lib/jekyll/converters/mephisto.rb", "lib/jekyll/converters/mt.rb", "lib/jekyll/converters/wordpress.rb", "lib/jekyll/convertible.rb", "lib/jekyll/core_ext.rb", "lib/jekyll/filters.rb", "lib/jekyll/layout.rb", "lib/jekyll/page.rb", "lib/jekyll/post.rb", "lib/jekyll/site.rb", "lib/jekyll/tags/highlight.rb", "lib/jekyll/tags/include.rb", "test/helper.rb", "test/source/_includes/sig.markdown", "test/source/_layouts/default.html", "test/source/_layouts/simple.html", "test/source/_posts/2008-10-18-foo-bar.textile", "test/source/_posts/2008-11-21-complex.textile", "test/source/_posts/2008-12-03-permalinked-post.textile", "test/source/_posts/2008-12-13-include.markdown", "test/source/css/screen.css", "test/source/index.html", "test/suite.rb", "test/test_generated_site.rb", "test/test_jekyll.rb", "test/test_post.rb", "test/test_site.rb"]
   s.has_rdoc = true
   s.rdoc_options = ["--main", "README.txt"]
   s.require_paths = ["lib"]
   s.rubyforge_project = %q{jekyll}
   s.rubygems_version = %q{1.3.0}
   s.summary = %q{Jekyll is a simple, blog aware, static site generator.}
-  s.test_files = ["test/test_jekyll.rb", "test/test_post.rb", "test/test_site.rb"]
+  s.test_files = ["test/test_generated_site.rb", "test/test_jekyll.rb", "test/test_post.rb", "test/test_site.rb"]
 
   if s.respond_to? :specification_version then
     current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION

data/lib/jekyll.rb

@@ -6,6 +6,7 @@ require 'rubygems'
 # core
 require 'fileutils'
 require 'time'
+require 'yaml'
 
 # stdlib
 
@@ -30,6 +31,7 @@ rescue LoadError
 end
 
 # internal requires
+require 'jekyll/core_ext'
 require 'jekyll/site'
 require 'jekyll/convertible'
 require 'jekyll/layout'
@@ -41,7 +43,7 @@ require 'jekyll/tags/include'
 require 'jekyll/albino'
 
 module Jekyll
-  VERSION = '0.2.1'
+  VERSION = '0.3.0'
   
   class << self
     attr_accessor :source, :dest, :lsi, :pygments, :markdown_proc

data/lib/jekyll/converters/mt.rb

@@ -0,0 +1,59 @@
+# Created by Nick Gerakines, open source and publically available under the
+# MIT license. Use this module at your own risk.
+# I'm an Erlang/Perl/C++ guy so please forgive my dirty ruby.
+
+require 'rubygems'
+require 'sequel'
+require 'fileutils'
+
+# NOTE: This converter requires Sequel and the MySQL gems.
+# The MySQL gem can be difficult to install on OS X. Once you have MySQL
+# installed, running the following commands should work:
+# $ sudo gem install sequel
+# $ sudo gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config
+ 
+module Jekyll
+  module MT
+    # This query will pull blog posts from all entries across all blogs. If
+    # you've got unpublished, deleted or otherwise hidden posts please sift
+    # through the created posts to make sure nothing is accidently published.
+    QUERY = "SELECT entry_id, entry_basename, entry_text, entry_text_more, entry_created_on, entry_title FROM mt_entry"
+ 
+    def self.process(dbname, user, pass, host = 'localhost')
+      db = Sequel.mysql(dbname, :user => user, :password => pass, :host => host)
+      
+      FileUtils.mkdir_p "_posts"
+            
+      db[QUERY].each do |post|
+        title = post[:entry_title]
+        slug = post[:entry_basename]
+        date = post[:entry_created_on]
+        content = post[:entry_text]
+        more_content = post[:entry_text_more]
+        
+        # Be sure to include the body and extended body.
+        if more_content != nil
+          conent = content + " \n" + more_content
+        end
+        
+        # Ideally, this script would determine the post format (markdown, html
+        # , etc) and create files with proper extensions. At this point it
+        # just assumes that markdown will be acceptable.
+        name = [date.year, date.month, date.day, slug].join('-') + ".markdown"
+
+        data = {
+           'layout' => 'post',
+           'title' => title.to_s,
+           'mt_id' => post[:entry_id],
+         }.delete_if { |k,v| v.nil? || v == ''}.to_yaml
+ 
+        File.open("_posts/#{name}", "w") do |f|
+          f.puts data
+          f.puts "---"
+          f.puts content
+        end
+      end
+ 
+    end
+  end
+end

data/lib/jekyll/converters/wordpress.rb

@@ -0,0 +1,54 @@
+require 'rubygems'
+require 'sequel'
+require 'fileutils'
+
+# NOTE: This converter requires Sequel and the MySQL gems.
+# The MySQL gem can be difficult to install on OS X. Once you have MySQL
+# installed, running the following commands should work:
+# $ sudo gem install sequel
+# $ sudo gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config
+
+module Jekyll
+  module WordPress
+    
+    # Reads a MySQL database via Sequel and creates a post file for each 
+    # post in wp_posts that has post_status = 'publish'.
+    # This restriction is made because 'draft' posts are not guaranteed to
+    # have valid dates.
+    QUERY = "select * from wp_posts where post_status = 'publish' and post_type = 'post'"
+
+    def self.process(dbname, user, pass, host = 'localhost')
+      db = Sequel.mysql(dbname, :user => user, :password => pass, :host => host)
+      
+      FileUtils.mkdir_p "_posts"
+            
+      db[QUERY].each do |post|
+        # Get required fields and construct Jekyll compatible name
+        title = post[:post_title]
+        slug = post[:post_name]
+        date = post[:post_date]
+        content = post[:post_content]
+        
+        name = [date.year, date.month, date.day, slug].join('-') + ".markdown"
+
+        # Get the relevant fields as a hash, delete empty fields and convert
+        # to YAML for the header        
+        data = {
+           'layout' => 'post',
+           'title' => title.to_s,
+           'excerpt' => post[:post_excerpt].to_s,
+           'wordpress_id' => post[:ID],
+           'wordpress_url' => post[:guid]
+         }.delete_if { |k,v| v.nil? || v == ''}.to_yaml
+
+        # Write out the data and content to file
+        File.open("_posts/#{name}", "w") do |f|
+          f.puts data
+          f.puts "---"
+          f.puts content
+        end
+      end
+
+    end
+  end
+end

data/lib/jekyll/convertible.rb

@@ -13,13 +13,13 @@ module Jekyll
     def read_yaml(base, name)
       self.content = File.read(File.join(base, name))
       
-      if self.content =~ /^(---.*\n.*?)\n---.*\n/m
+      if self.content =~ /^(---\s*\n.*?)\n---\s*\n/m
         self.content = self.content[($1.size + 5)..-1]
         
         self.data = YAML.load($1)
       end
     end
-  
+    
     # Transform the contents based on the file extension.
     #
     # Returns nothing
@@ -39,10 +39,8 @@ module Jekyll
     #   +site_payload+ is the site payload hash
     #
     # Returns nothing
-    def do_layout(payload, layouts, site_payload)
-      # construct payload
-      payload = payload.merge(site_payload)
-      # render content
+    def do_layout(payload, layouts)
+      # render and transform content (this becomes the final content of the object)
       self.content = Liquid::Template.parse(self.content).render(payload, [Jekyll::Filters])
       self.transform
       
@@ -52,7 +50,7 @@ module Jekyll
       # recursively render layouts
       layout = layouts[self.data["layout"]]
       while layout
-        payload = payload.merge({"content" => self.output, "page" => self.data})
+        payload = payload.deep_merge({"content" => self.output, "page" => layout.data})
         self.output = Liquid::Template.parse(layout.content).render(payload, [Jekyll::Filters])
         
         layout = layouts[layout.data["layout"]]

data/lib/jekyll/core_ext.rb

@@ -0,0 +1,22 @@
+class Hash
+  # Merges self with another hash, recursively.
+  # 
+  # This code was lovingly stolen from some random gem:
+  # http://gemjack.com/gems/tartan-0.1.1/classes/Hash.html
+  # 
+  # Thanks to whoever made it.
+  def deep_merge(hash)
+    target = dup
+    
+    hash.keys.each do |key|
+      if hash[key].is_a? Hash and self[key].is_a? Hash
+        target[key] = target[key].deep_merge(hash[key])
+        next
+      end
+      
+      target[key] = hash[key]
+    end
+    
+    target
+  end
+end

data/lib/jekyll/filters.rb

@@ -4,6 +4,10 @@ module Jekyll
     def date_to_string(date)
       date.strftime("%d %b %Y")
     end
+
+    def date_to_long_string(date)
+      date.strftime("%d %B %Y")
+    end
     
     def date_to_xmlschema(date)
       date.xmlschema
@@ -15,7 +19,6 @@ module Jekyll
     
     def number_of_words(input)
       input.split.length
-    end
-  end
-  
+    end    
+  end  
 end

data/lib/jekyll/layout.rb

@@ -28,21 +28,6 @@ module Jekyll
     def process(name)
       self.ext = File.extname(name)
     end
-    
-    # Add any necessary layouts to this post
-    #   +layouts+ is a Hash of {"name" => "layout"}
-    #   +site_payload+ is the site payload hash
-    #
-    # Returns nothing
-    def add_layout(layouts, site_payload)
-      payload = {"page" => self.data}.merge(site_payload)
-      self.content = Liquid::Template.parse(self.content).render(payload, [Jekyll::Filters])
-      
-      layout = layouts[self.data["layout"]] || self.content
-      payload = {"content" => self.content, "page" => self.data}
-      
-      self.content = Liquid::Template.parse(layout).render(payload, [Jekyll::Filters])
-    end
   end
 
 end

data/lib/jekyll/page.rb

@@ -37,9 +37,9 @@ module Jekyll
     #   +site_payload+ is the site payload hash
     #
     # Returns nothing
-    def add_layout(layouts, site_payload)
-      payload = {"page" => self.data}
-      do_layout(payload, layouts, site_payload)
+    def render(layouts, site_payload)
+      payload = {"page" => self.data}.deep_merge(site_payload)
+      do_layout(payload, layouts)
     end
     
     # Write the generated page file to the destination directory.

data/lib/jekyll/post.rb

@@ -3,12 +3,12 @@ module Jekyll
   class Post
     include Comparable
     include Convertible
-    
+        
     class << self
       attr_accessor :lsi
     end
     
-    MATCHER = /^(\d+-\d+-\d+)-(.*)(\.[^.]+)$/
+    MATCHER = /^(.+\/)*(\d+-\d+-\d+)-(.*)(\.[^.]+)$/
     
     # Post name validator. Post filenames must be like:
     #   2008-11-05-my-awesome-post.textile
@@ -18,22 +18,26 @@ module Jekyll
       name =~ MATCHER
     end
     
-    attr_accessor :date, :slug, :ext
+    attr_accessor :date, :slug, :ext, :categories, :topics
     attr_accessor :data, :content, :output
     
     # Initialize this Post instance.
     #   +base+ is the String path to the dir containing the post file
     #   +name+ is the String filename of the post file
+    #   +categories+ is an Array of Strings for the categories for this post
     #
     # Returns <Post>
-    def initialize(base, name)
-      @base = base
+    def initialize(source, dir, name)
+      @base = File.join(source, dir, '_posts')
       @name = name
       
+      self.categories = dir.split('/').reject { |x| x.empty? }
+      
+      parts = name.split('/')
+      self.topics = parts.size > 1 ? parts[0..-2] : []
+      
       self.process(name)
-      self.read_yaml(base, name)
-      #Removed to avoid munging of liquid tags, replaced in convertible.rb#48
-      #self.transform
+      self.read_yaml(@base, name)
     end
     
     # Spaceship is based on Post#date
@@ -48,7 +52,7 @@ module Jekyll
     #
     # Returns nothing
     def process(name)
-      m, date, slug, ext = *name.match(MATCHER)
+      m, cats, date, slug, ext = *name.match(MATCHER)
       self.date = Time.parse(date)
       self.slug = slug
       self.ext = ext
@@ -61,9 +65,12 @@ module Jekyll
     #
     # Returns <String>
     def dir
-      permalink ?
-        permalink.to_s.split("/")[0..-2].join("/") :
-        date.strftime("/%Y/%m/%d/")
+      if permalink
+        permalink.to_s.split("/")[0..-2].join("/")
+      else
+        prefix = self.categories.empty? ? '' : '/' + self.categories.join('/')
+        prefix + date.strftime("/%Y/%m/%d/")
+      end
     end
     
     # The full path and filename of the post.
@@ -90,7 +97,7 @@ module Jekyll
     def id
       self.dir + self.slug
     end
-    
+        
     # Calculate related posts.
     #
     # Returns [<Post>]
@@ -118,11 +125,16 @@ module Jekyll
     #   +site_payload+ is the site payload hash
     #
     # Returns nothing
-    def add_layout(layouts, site_payload)
-      # construct post payload
-      related = related_posts(site_payload["site"]["posts"])
-      payload = {"page" => self.to_liquid.merge(self.data)}
-      do_layout(payload, layouts, site_payload.merge({"site" => {"related_posts" => related}}))
+    def render(layouts, site_payload)
+      # construct payload
+      payload =
+      {
+        "site" => { "related_posts" => related_posts(site_payload["site"]["posts"]) },
+        "page" => self.to_liquid
+      }
+      payload = payload.deep_merge(site_payload)
+      
+      do_layout(payload, layouts)
     end
     
     # Write the generated post file to the destination directory.
@@ -146,7 +158,12 @@ module Jekyll
         "url" => self.url,
         "date" => self.date,
         "id" => self.id,
-        "content" => self.content }
+        "topics" => self.topics,
+        "content" => self.content }.deep_merge(self.data)
+    end
+    
+    def inspect
+      "<Post: #{self.id}>"
     end
   end
 

data/lib/jekyll/site.rb

@@ -24,19 +24,19 @@ module Jekyll
     # Returns nothing
     def process
       self.read_layouts
-      self.read_posts
-      self.write_posts
       self.transform_pages
+      self.write_posts
     end
     
-    # Read all the files in <source>/_layouts into memory for
-    # later use.
+    # Read all the files in <source>/_layouts except backup files
+    # (end with "~") into memory for later use.
     #
     # Returns nothing
     def read_layouts
       base = File.join(self.source, "_layouts")
       entries = Dir.entries(base)
-      entries = entries.reject { |e| File.directory?(e) }
+      entries = entries.reject { |e| e[-1..-1] == '~' }
+      entries = entries.reject { |e| File.directory?(File.join(base, e)) }
       
       entries.each do |f|
         name = f.split(".")[0..-2].join(".")
@@ -46,17 +46,29 @@ module Jekyll
       # ignore missing layout dir
     end
     
-    # Read all the files in <source>/posts and create a new Post
-    # object with each one.
+    # Read all the files in <base>/_posts except backup files (end with "~")
+    # and create a new Post object with each one.
     #
     # Returns nothing
-    def read_posts
-      base = File.join(self.source, "_posts")
-      entries = Dir.entries(base)
-      entries = entries.reject { |e| File.directory?(e) }
+    def read_posts(dir)
+      base = File.join(self.source, dir, '_posts')
+      
+      entries = []
+      Dir.chdir(base) { entries = Dir['**/*'] }
+      entries = entries.reject { |e| e[-1..-1] == '~' }
+      entries = entries.reject { |e| File.directory?(File.join(base, e)) }
 
+      # first pass processes, but does not yet render post content
       entries.each do |f|
-        self.posts << Post.new(base, f) if Post.valid?(f)
+        if Post.valid?(f)
+          post = Post.new(self.source, dir, f)
+          self.posts << post
+        end
+      end
+      
+      # second pass renders each post now that full site payload is available
+      self.posts.each do |post|
+        post.render(self.layouts, site_payload)
       end
       
       self.posts.sort!
@@ -69,14 +81,14 @@ module Jekyll
     # Returns nothing
     def write_posts
       self.posts.each do |post|
-        post.add_layout(self.layouts, site_payload)
         post.write(self.dest)
       end
     end
     
     # Copy all regular files from <source> to <dest>/ ignoring
-    # any files/directories that are hidden (start with ".") or contain
-    # site content (start with "_")
+    # any files/directories that are hidden or backup files (start
+    # with "." or end with "~") or contain site content (start with "_")
+    # unless they are "_posts" directories
     #   The +dir+ String is a relative path used to call this method
     #            recursively as it descends through directories
     #
@@ -84,7 +96,17 @@ module Jekyll
     def transform_pages(dir = '')
       base = File.join(self.source, dir)
       entries = Dir.entries(base)
-      entries = entries.reject { |e| ['.', '_'].include?(e[0..0]) }
+      entries = entries.reject { |e| e[-1..-1] == '~' }
+      entries = entries.reject do |e|
+        (e != '_posts') and ['.', '_'].include?(e[0..0])
+      end
+
+      # we need to make sure to process _posts *first* otherwise they 
+      # might not be available yet to other templates as {{ site.posts }}
+      if entries.include?('_posts')
+        entries.delete('_posts')
+        read_posts(dir)
+      end
 
       entries.each do |f|
         if File.directory?(File.join(base, f))
@@ -93,13 +115,13 @@ module Jekyll
         else
           first3 = File.open(File.join(self.source, dir, f)) { |fd| fd.read(3) }
           
-          # if the file appears to have a YAML header then process it as a page
           if first3 == "---"
+            # file appears to have a YAML header so process it as a page
             page = Page.new(self.source, dir, f)
-            page.add_layout(self.layouts, site_payload)
+            page.render(self.layouts, site_payload)
             page.write(self.dest)
-          # otherwise copy the file without transforming it
           else
+            # otherwise copy the file without transforming it
             FileUtils.mkdir_p(File.join(self.dest, dir))
             FileUtils.cp(File.join(self.source, dir, f), File.join(self.dest, dir, f))
           end
@@ -111,7 +133,17 @@ module Jekyll
     #
     # Returns {"site" => {"time" => <Time>, "posts" => [<Post>]}}
     def site_payload
-      {"site" => {"time" => Time.now, "posts" => self.posts.sort.reverse}}
+      # Build the category hash map of category ( names => arrays of posts )
+      # then sort each array in reverse order
+      categories = Hash.new { |hash, key| hash[key] = Array.new }
+      self.posts.each { |p| p.categories.each { |c| categories[c] << p } }
+      categories.values.map { |cats| cats.sort! { |a, b| b <=> a} }
+      
+      {"site" => {
+        "time" => Time.now, 
+        "posts" => self.posts.sort { |a,b| b <=> a },
+        "categories" => categories
+      }}
     end
   end
 

data/lib/jekyll/tags/include.rb

@@ -7,7 +7,18 @@ module Jekyll
     end
     
     def render(context)
-      File.read(File.join(Jekyll.source, '_includes', @file))
+      if @file !~ /^[a-zA-Z0-9_\/\.-]+$/ || @file =~ /\.\// || @file =~ /\/\./
+        return "Include file '#{@file}' contains invalid characters or sequences"
+      end
+      
+      Dir.chdir(File.join(Jekyll.source, '_includes')) do
+        choices = Dir['**/*'].reject { |x| File.symlink?(x) }
+        if choices.include?(@file)
+          File.read(@file)
+        else
+          "Included file '#{@file}' not found in _includes directory"
+        end
+      end
     end
   end
   

data/test/source/index.html

@@ -5,8 +5,18 @@ title: Tom Preston-Werner
 
 h1. Welcome to my site
 
+h2. Please read our {{ site.posts | size }} Posts
+
 <ul>
   {% for post in site.posts %}
     <li>{{ post.date }} <a href="{{ post.url }}">{{ post.title }}</a></li>
   {% endfor %}
-</ul>
+</ul>
+
+{% assign first_post = site.posts.first %}
+<div id="first_post">
+  <h1>{{ first_post.title }}</h1>
+  <div>
+    {{ first_post.content }}
+  </div>
+</div>

data/test/test_generated_site.rb

@@ -0,0 +1,21 @@
+require File.dirname(__FILE__) + '/helper'
+
+class TestGeneratedSite < Test::Unit::TestCase
+  def setup
+    clear_dest
+    source = File.join(File.dirname(__FILE__), *%w[source])
+    @s = Site.new(source, dest_dir)
+    @s.process
+    @index = File.read(File.join(dest_dir, 'index.html'))
+  end
+  
+  def test_site_posts_in_index
+    # confirm that {{ site.posts }} is working
+    assert @index.include?("#{@s.posts.size} Posts")
+  end
+
+  def test_post_content_in_index
+    # confirm that the {{ post.content }} is rendered OK
+    assert @index.include?('<p>This <em>is</em> cool</p>')
+  end
+end

data/test/test_post.rb

@@ -7,6 +7,9 @@ class TestPost < Test::Unit::TestCase
   
   def test_valid
     assert Post.valid?("2008-10-19-foo-bar.textile")
+    assert Post.valid?("foo/bar/2008-10-19-foo-bar.textile")
+    
+    assert !Post.valid?("lol2008-10-19-foo-bar.textile")
     assert !Post.valid?("blah")
   end
   
@@ -21,6 +24,7 @@ class TestPost < Test::Unit::TestCase
   
   def test_url
     p = Post.allocate
+    p.categories = []
     p.process("2008-10-19-foo-bar.textile")
     
     assert_equal "/2008/10/19/foo-bar.html", p.url
@@ -29,7 +33,7 @@ class TestPost < Test::Unit::TestCase
   def test_permalink
     p = Post.allocate
     p.process("2008-12-03-permalinked-post.textile")
-    p.read_yaml(File.join(File.dirname(__FILE__), *%w[source posts]), "2008-12-03-permalinked-post.textile")
+    p.read_yaml(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-12-03-permalinked-post.textile")
 
     assert_equal "my_category/permalinked-post", p.permalink
   end
@@ -37,7 +41,7 @@ class TestPost < Test::Unit::TestCase
   def test_dir_respects_permalink
     p = Post.allocate
     p.process("2008-12-03-permalinked-post.textile")
-    p.read_yaml(File.join(File.dirname(__FILE__), *%w[source posts]), "2008-12-03-permalinked-post.textile")
+    p.read_yaml(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-12-03-permalinked-post.textile")
 
     assert_equal "my_category", p.dir
   end
@@ -59,10 +63,10 @@ class TestPost < Test::Unit::TestCase
     assert_equal "<h1>{{ page.title }}</h1>\n<p>Best <strong>post</strong> ever</p>", p.content
   end
   
-  def test_add_layout
-    p = Post.new(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-10-18-foo-bar.textile")
+  def test_render
+    p = Post.new(File.join(File.dirname(__FILE__), *%w[source]), '',  "2008-10-18-foo-bar.textile")
     layouts = {"default" => Layout.new(File.join(File.dirname(__FILE__), *%w[source _layouts]), "simple.html")}
-    p.add_layout(layouts, {"site" => {"posts" => []}})
+    p.render(layouts, {"site" => {"posts" => []}})
     
     assert_equal "<<< <h1>Foo Bar</h1>\n<p>Best <strong>post</strong> ever</p> >>>", p.output
   end
@@ -70,25 +74,25 @@ class TestPost < Test::Unit::TestCase
   def test_write
     clear_dest
     
-    p = Post.new(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-10-18-foo-bar.textile")
+    p = Post.new(File.join(File.dirname(__FILE__), *%w[source]), '', "2008-10-18-foo-bar.textile")
     layouts = {"default" => Layout.new(File.join(File.dirname(__FILE__), *%w[source _layouts]), "simple.html")}
-    p.add_layout(layouts, {"site" => {"posts" => []}})
+    p.render(layouts, {"site" => {"posts" => []}})
     p.write(dest_dir)
   end
   
   def test_data
-    p = Post.new(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-11-21-complex.textile")
+    p = Post.new(File.join(File.dirname(__FILE__), *%w[source]), '', "2008-11-21-complex.textile")
     layouts = {"default" => Layout.new(File.join(File.dirname(__FILE__), *%w[source _layouts]), "simple.html")}
-    p.add_layout(layouts, {"site" => {"posts" => []}})
+    p.render(layouts, {"site" => {"posts" => []}})
     
     assert_equal "<<< <p>url: /2008/11/21/complex.html<br />\ndate: #{Time.parse("2008-11-21")}<br />\nid: /2008/11/21/complex</p> >>>", p.output
   end
   
   def test_include
     Jekyll.source = File.join(File.dirname(__FILE__), *%w[source])
-    p = Post.new(File.join(File.dirname(__FILE__), *%w[source _posts]), "2008-12-13-include.markdown")
+    p = Post.new(File.join(File.dirname(__FILE__), *%w[source]), '', "2008-12-13-include.markdown")
     layouts = {"default" => Layout.new(File.join(File.dirname(__FILE__), *%w[source _layouts]), "simple.html")}
-    p.add_layout(layouts, {"site" => {"posts" => []}})
+    p.render(layouts, {"site" => {"posts" => []}})
     
     assert_equal "<<< <hr />\n<p>Tom Preston-Werner github.com/mojombo</p>\n\n<p>This <em>is</em> cool</p> >>>", p.output
   end

data/test/test_site.rb

@@ -15,11 +15,11 @@ class TestSite < Test::Unit::TestCase
     
     assert_equal ["default", "simple"].sort, @s.layouts.keys.sort
   end
-  
+ 
   def test_read_posts
-    @s.read_posts
+    @s.read_posts('')
     
-    assert_equal 3, @s.posts.size
+    assert_equal 4, @s.posts.size
   end
   
   def test_write_posts
@@ -27,4 +27,4 @@ class TestSite < Test::Unit::TestCase
     
     @s.process
   end
-end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: jekyll
 version: !ruby/object:Gem::Version 
-  version: 0.2.1
+  version: 0.3.0
 platform: ruby
 authors: 
 - Tom Preston-Werner
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2008-12-15 00:00:00 -08:00
+date: 2008-12-24 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -103,7 +103,10 @@ files:
 - lib/jekyll/albino.rb
 - lib/jekyll/converters/csv.rb
 - lib/jekyll/converters/mephisto.rb
+- lib/jekyll/converters/mt.rb
+- lib/jekyll/converters/wordpress.rb
 - lib/jekyll/convertible.rb
+- lib/jekyll/core_ext.rb
 - lib/jekyll/filters.rb
 - lib/jekyll/layout.rb
 - lib/jekyll/page.rb
@@ -117,11 +120,12 @@ files:
 - test/source/_layouts/simple.html
 - test/source/_posts/2008-10-18-foo-bar.textile
 - test/source/_posts/2008-11-21-complex.textile
+- test/source/_posts/2008-12-03-permalinked-post.textile
 - test/source/_posts/2008-12-13-include.markdown
 - test/source/css/screen.css
 - test/source/index.html
-- test/source/posts/2008-12-03-permalinked-post.textile
 - test/suite.rb
+- test/test_generated_site.rb
 - test/test_jekyll.rb
 - test/test_post.rb
 - test/test_site.rb
@@ -153,6 +157,7 @@ signing_key:
 specification_version: 2
 summary: Jekyll is a simple, blog aware, static site generator.
 test_files: 
+- test/test_generated_site.rb
 - test/test_jekyll.rb
 - test/test_post.rb
 - test/test_site.rb