caramelize 0.1.0 → 0.1.1

data/README.md

@@ -1,41 +1,26 @@
 # caramelize
 
-Caramelize is a compact and flexible wiki converter. It is intended for easily creating export of otherwise rare supported legacy wikis. With caramelize you can create your own export configurations and transfer your data into a git-based gollum-wiki retaing all your history.
+Caramelize is a compact and flexible wiki content migration tool. It is intended for easily transfering content from otherwise rare supported legacy wikis. With caramelize you can create your own export configurations and migrate your data into a git-based [gollum](gollum)-wiki retaining all your history and gaining the most flexible access to your wiki content.
 
-In the future more target wiki system may be added. For the moment export is supported for WikkaWiki and Redmine-Wiki.
+In the future more target wikis may be added. For the moment migration is supported for [WikkaWiki](wikka) and [Redmine](redmine)-Wiki.
 
 ## Usage
 
-### Building
-
-To try it you require bundler to build it.
-
-    $ gem bundler install
-
-Clone this repository and start building.
+### Installation
 
-    $ git clone git@github.com:Dahie/caramelize.git
-    $ gem build caramelize.gemspec
+    $ gem install caramelize
     
-Now to build the gem do 
-
-    $ rake build
-
-or
-
-    $ rake install
-
-to install the new gem right to the system.
+Install the latest release of caramelize using RubyGems.
 
 ### Use
 
     $ caramelize create 
 
-Creates a template configuration file "caramel.rb". This includes documentation on how to use the preset Wiki-connectors and how to write addition customized connectors.
+Creates a template configuration file "caramel.rb". This includes documentation on how to use the preset Wiki-connectors and how to write addition customized connectors. More about this below.
 
     $ caramelize run 
     
-Will execute a wiki migration based on a found configuration file. These are found in predefined paths.
+Will start the wiki migration based on the configuration file. These are either found in predefined paths (./caramel.rb, ./config.rb, …), or passed as argument, as below.
 
     $ caramelize help
     
@@ -47,14 +32,112 @@ Returns version and release information.
 
 ### Options
 
-$ caramelize create --config my_caramel_configuration.rb
+    $ caramelize create --config my_caramel_configuration.rb
 
-Creates an example config with the given name.
+Creates an example configuration by the given name.
 
-$ caramelize run --config my_caramel_configuration.rb
+    $ caramelize run --config my_caramel_configuration.rb
 
 Executes the given configuration.
 
+    $ caramelize --verbose [command]
+    $ caramelize -v [command]
+    
+Displays more verbose output to the command line.
+
+## Content migration
+
+### Wiki support
+
+Caramelize comes with direct support for [WikkaWiki](wikka) and [Redmine](redmine)-Wiki.
+More custom wikis can be supported by creating a suitable configuration file. 
+
+Any imported wiki exports into a [gollum](gollum) git-repository. This is a wiki based around a git-repository. This gives you the flexibility of having all wiki pages exported as physical files, while keeping the history and having an easy and wide-supported way of access by using the wiki server gollum features.
+
+Since wiki software may have special features, that are not common among other wikis, content migration may always have a loss of style or information. Caramelize tries to support the most common features.
+
+* Page meta data
+  * title 
+  * content body
+  * author name
+  * author email address
+  * date
+  * revisions
+* Markup conversion to markdown
+  * limited to "simple" formatting, excluding complex formats such as tables
+  * conversion using regular expressions -> somewhat easy to learn and extend  
+
+### Configuration recipes
+
+The `caramel.rb` configuration contains the settings on how to import the data of the existing wiki and how to convert it into the format required by caramelize to export to gollum.
+You also find the predefined definitions for importing from WikkaWiki and Redmine and and example for a custom import.
+
+Custom import allows you to import data from wikis that are not natively supported by caramelize. Defining your own wiki import requires a bit of knowledge on Ruby and MySQL as you setup the access to your wiki database and need to define how the data is to be transformed. Depending on the database model of the wiki this can be one simple call for all revisions in the database, or it can get more complicated with multiple mysql-calls as the database becomes more complex.
+
+For a custom wiki you need to create a `wiki` instance object, that receives the necessary database creditials.
+
+    wiki = Caramelize::Wiki.new({:host => "localhost", 
+                    :username => "user", 
+                    :database => "database_name", 
+                    :password => 'monkey', 
+                    :markup => :wikka})
+
+This example ignores custom markup conversion and assumes WikkaWiki-markup. 
+
+Once the object is established we need to hook in a method that defines how revisions are read from the database and how they are processed.
+
+    wiki.instance_eval do
+    	def read_pages
+      		sql = "SELECT id, tag, body, time, latest, user, note FROM wikka_pages ORDER BY time;"
+      		@revisions, @titles = [], []
+      		results = database.query(sql)
+      		results.each do |row|
+        		@titles << row["tag"]
+        		author = @authors[row["user"]]
+		        page = Page.new({:id => row["id"],
+                            :title =>   row["tag"],
+                            :body =>    row["body"],
+                            :markup =>  'wikka',
+                            :latest =>  row["latest"] == "Y",
+                            :time =>    row["time"],
+                            :message => row["note"],
+                            :author =>  author,
+                            :author_name => row["user"]})
+       		 @revisions << page
+      	end
+      # titles is the list of all unique page titles contained in the wiki
+      @titles.uniq!
+      # revisions is the list of all revisions ordered by date
+      @revisions
+    end
+
+In the end the `wiki` instance needs the `@titles` and `@revisions` filled.
+
+Some wikis don't have all necessary metadata saved in the revision. In this case additional database queries are necessary. **The configuration recipe is pure ruby code, that is included on execution. This gives you alot of freedom in writing your configuration, but also a lot of power to break things for yourself. Be advised.**
+
+I'm happy to give support on your recipes and I'd also like to extend caramelize with more wiki modules, if you send in your configurations (minus database credentials of course).
+
+### Building
+
+This is how you can build caramelize, in case you'd like to develop it further. To get startered you'll need Bundler.
+
+    $ gem install bundler
+
+Clone or fork this repository and start building.
+
+    $ git clone git@github.com:Dahie/caramelize.git
+    $ gem build caramelize.gemspec
+    
+Now to build and package the gem do 
+
+    $ rake build
+
+or
+
+    $ rake install
+
+to install the new gem right to your system.
+
 ## Contributing to caramelize
  
 * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
@@ -62,10 +145,13 @@ Executes the given configuration.
 * Fork the project
 * Start a feature/bugfix branch
 * Commit and push until you are happy with your contribution
-* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
 * Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
 
+
 ## Copyright
 
-Copyright (c) 2011 Daniel Senff. See LICENSE.md for further details.
+Copyright (c) 2011-2013 Daniel Senff. See LICENSE.md for further details.
 
+[wikka]: http://wikkawiki.org/
+[gollum]: https://github.com/github/gollum
+[redmine]: http://www.redmine.org/

data/lib/caramelize/content_transferer.rb

@@ -50,11 +50,6 @@ module Caramelize
           else
             body_new = original_wiki.to_textile rev.body
           end
-          
-          if body_new.start_with?('Einige interessante')
-            puts rev.body
-            puts body_new
-          end
 
           unless body_new.eql? rev.body
             rev.body = body_new

data/lib/caramelize/version.rb

@@ -1,3 +1,3 @@
 module Caramelize
-  VERSION = "0.1.0"
+  VERSION = "0.1.1"
 end

data/lib/caramelize/wiki/wikka_converter.rb

@@ -13,10 +13,16 @@ module Caramelize
       body.gsub!(/(\/\/)(.*?)(\/\/)/) {|s| '_' + $2 + '_' }   #italic
       #str.gsub!(/(===)(.*?)(===)/) {|s| '`' + $2 + '`'}   #code
       body.gsub!(/(__)(.*?)(__)/) {|s| '<u>' + $2 + '</u>'}   #underline
+      body.gsub!(/(---)/, '  ')   #forced linebreak
       
-      body.gsub!(/(.*?)(\n\t-)(.*?)/) {|s| $1 + '\n' + $3 }   #list
+      #body.gsub!(/(.*?)(\n\t-)(.*?)/) {|s| $1 + $3 }   #list
       
-      body.gsub!(/(\t-)(.*?)/) {|s| '*' + $2 }   #list
+      body.gsub!(/(\t-)(.*)/, '*\2')    # unordered list
+      body.gsub!(/(~-)(.*)/, '*\2')     # unordered list
+      # TODO ordered lists
+
+      # TODO images: ({{image)(url\=?)?(.*)(}})
+
       #str.gsub!(/(----)/) {|s| '~~~~'}   #seperator
 
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: caramelize
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-01-02 00:00:00.000000000 Z
+date: 2013-01-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: mysql2