caramelize 0.4.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f7fe435304327d466829f0dcbe0ab9ed23978da90c0c992f1464d123cffa3965
-  data.tar.gz: f89cdef42b840eff1869081e813cb37f93172fea6a5c7446ffba2685b34aae88
+  metadata.gz: 794ae829880cb349361e52ec4ca6a313e08c228aee66816e1be297b55dcc21f5
+  data.tar.gz: a755af756c4a6b450166ecbb8ebd23b4dd24a9e840728ddb40d2b8e6e214f04b
 SHA512:
-  metadata.gz: 2b1c74be41c73bd7e0f35e4f9441f9bc96a5a6f04e994502b70fec73fda7444e8d7e8e4fb8868d5822d4333e4f3a37b2348f954d8d37064674ee68046e8d6c58
-  data.tar.gz: 3b7a59b339ef5779ab86be0ee4a054f0772f58483db5edee3479d736f355ad4f04bfac06e46fda39319fc431c2003885687666abc0da854b080631484f3bcd10
+  metadata.gz: 9a3b6071991c74dbc86218e7646a8cd09c90aba0e51708a7f698a0b38e3338f3ea95580822e316ddd3c7ad8ba28ee3ae8874bc3c6e3c65ca2838fd7943d44e43
+  data.tar.gz: 8f6de4f36f2aca5a6b08c230a02f0a56e51bfb3f0e3cf1b57e04d052ee3ecd963d09845c0416b27b89b99a3f8ffafbfe4b43e1c04ceb5b596575be7d01f9eecd

data/.travis.yml

@@ -0,0 +1,5 @@
+language: ruby
+rvm:
+ - 2.6
+before_install:
+  - gem install bundler

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at public@danielsenff.de. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

data/README.md

@@ -1,4 +1,6 @@
-# caramelize
+# Caramelize
+
+[![Build Status](https://travis-ci.org/dahie/caramelize.svg?branch=master)](https://travis-ci.org/dahie/caramelize) [![Maintainability](https://api.codeclimate.com/v1/badges/7fe3ef34e09ba8133424/maintainability)](https://codeclimate.com/github/Dahie/caramelize/maintainability)
 
 Caramelize is a compact and flexible wiki content migration tool. It is intended for easily transfering content from otherwise rare supported legacy wikis. With caramelize you can create your own export configurations and migrate your data into a git-based [gollum](https://github.com/github/gollum)-wiki retaining all your history and gaining the most flexible access to your wiki content.
 

data/Rakefile

@@ -4,7 +4,7 @@ require 'bundler/gem_tasks'
 # Default directory to look in is `/specs`
 # Run with `rake spec`
 RSpec::Core::RakeTask.new(:spec) do |task|
-  task.rspec_opts = ['--color', '--format', 'nested']
+  task.rspec_opts = ['--color', '--format', 'documentation']
 end
 
-task :default => :spec
+task :default => :spec

data/bin/caramelize

@@ -6,7 +6,7 @@ require 'commander/import'
 # :name is optional, otherwise uses the basename of this executable
 program :name, 'caramelize'
 program :version, Caramelize::VERSION
-program :description, 'TODO'
+program :description, 'With Caramelize you can migrate any wiki to git-based Gollum wiki repositories.'
 program :help, 'Author', 'Daniel Senff <public@danielsenff.de>'
 
 default_command :run
@@ -25,7 +25,7 @@ command :new do |c|
     options.default({ config: 'caramel.rb' })
     FileUtils.cp(File.dirname(__FILE__) + '/../lib/caramelize/caramel.rb',
                  options.config)
-    say "Created new configuration file: #{optinos.config}"
+    say "Created new configuration file: #{options.config}"
   end
 end
 alias_command :create, :new
@@ -39,12 +39,19 @@ command :run do |c|
   c.example 'Run transfer for "config.rb"', 'caramelize run --config config.rb'
   c.action do |args, options|
     time_start = Time.now
-    options.default({ config: 'caramel.rb' })
-    puts options.inspect
+    options.default({ config: 'caramel.rb', target: 'wiki-export' })
 
     instance_eval(File.read(options.config))
 
-    wiki_options = input_wiki.options.merge(verbose: options.verbose)
+    target = options.target
+
+    if File.exists?(target)
+      answer = agree("#{target} already exists. Overwrite with fresh repository?")
+      FileUtils.rm_rf(target) if answer
+    end
+
+    wiki_options = input_wiki.options.merge(verbose: options.verbose,
+                                            target: options.target)
     Caramelize::ContentTransferer.new(input_wiki, wiki_options).execute
     say "Time required: #{Time.now - time_start} s"
   end

data/caramelize.gemspec

@@ -11,7 +11,7 @@ Gem::Specification.new do |spec|
   spec.email       = ["mail@danielsenff.de"]
   spec.homepage    = "http://github.com/Dahie/caramelize"
   spec.summary     = %q{Flexible and modular wiki conversion tool}
-  spec.description = %q{By defining the connectors from the input wiki you can migrate any wiki to git-based Gollum wiki repositories.}
+  spec.description = %q{With Caramelize you can migrate any wiki to git-based Gollum wiki repositories.}
 
   spec.files         = `git ls-files -z`.split("\x0")
   spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
@@ -20,9 +20,8 @@ Gem::Specification.new do |spec|
 
   spec.add_dependency('mysql2')
   spec.add_dependency('commander')
-  spec.add_dependency('docile')
   spec.add_dependency('ruby-progressbar')
-  spec.add_dependency('gollum-lib', '>= 1.0.0') # grit dependency implicit through gollum
+  spec.add_dependency('gollum-lib')
 
   spec.add_development_dependency "bundler", "~> 1.6"
   spec.add_development_dependency "rake"

data/lib/caramelize.rb

@@ -1,6 +1,5 @@
 require 'caramelize/version'
 require 'caramelize/page'
-require 'caramelize/ext'
 require 'caramelize/content_transferer'
 require 'caramelize/filter_processor'
 require 'caramelize/database_connector'
@@ -10,3 +9,5 @@ require 'caramelize/input_wiki/wiki'
 require 'caramelize/input_wiki/redmine_wiki'
 require 'caramelize/input_wiki/wikkawiki'
 
+Encoding.default_external = Encoding::UTF_8
+Encoding.default_internal = Encoding::UTF_8

data/lib/caramelize/content_transferer.rb

@@ -1,11 +1,8 @@
-require 'gollum-lib'
-require 'grit'
 require 'ruby-progressbar'
 
 module Caramelize
 
   require 'caramelize/page'
-  require 'caramelize/ext'
   require 'caramelize/content_transferer'
   require 'caramelize/database_connector'
   require 'caramelize/output_wiki/gollum'
@@ -15,12 +12,13 @@ module Caramelize
   class ContentTransferer
     attr_reader :input_wiki, :options, :filter_processor
 
+    DEFAULT_GOLLUM_HOME_TITLE = 'Home'.freeze
+
     def initialize(input_wiki, options)
       @input_wiki = input_wiki
       @options = options
 
       options[:default_author] = options.fetch(:default_author, "Caramelize")
-      options[:target_directory] = './wiki-export'
       options[:markup] = target_markup
     end
 
@@ -39,6 +37,8 @@ module Caramelize
 
       puts 'Create Namespace Overview' if verbose?
       create_overview_page_of_namespaces if options[:create_namespace_overview]
+
+      rename_home_page
     end
 
     private
@@ -57,7 +57,7 @@ module Caramelize
     end
 
     def output_wiki
-      @output_wiki ||= OutputWiki::Gollum.new(options[:target_directory])
+      @output_wiki ||= OutputWiki::Gollum.new(options[:target])
     end
 
     def filter_processor
@@ -72,51 +72,65 @@ module Caramelize
       revisions.count
     end
 
+    def latest_revisions_count
+      input_wiki.latest_revisions.count
+    end
+
     def create_overview_page_of_namespaces
       output_wiki.commit_namespace_overview(input_wiki.namespaces)
     end
 
-    def migrate_markup_of_latest_revisions
-      progress_bar = ProgressBar.create(title: "Markup filters",
-                                        total: revisions_count)
+    def migrate_markup_progress_bar
+      @migrate_markup_progress_bar ||=
+        ProgressBar.create(title: "Markup filters",
+                           total: latest_revisions_count)
+    end
 
+    def commit_history_progress_bar
+      @commit_history_progress_bar ||=
+        ProgressBar.create(title: "Revisions",
+                           total: revisions_count)
+    end
+
+    def migrate_markup_of_latest_revisions
       input_wiki.latest_revisions.each do |revision|
-        migrate_markup_of_revision(revision, progress_bar)
+        migrate_markup_of_revision(revision)
       end
     end
 
     def commit_history
-      puts revisions_count
-      progress_bar = ProgressBar.create(title: "Revisions",
-                                        total: revisions_count)
-
       output_wiki.commit_history(revisions, options) do |page, index|
         if verbose?
           puts "(#{index + 1}/#{revisions_count}) #{page.time} #{page.title}"
         else
-          progress_bar.increment
+          commit_history_progress_bar.increment
         end
       end
     end
 
-    def migrate_markup_of_revision(revision, progress_bar)
+    def migrate_markup_of_revision(revision)
       if verbose?
         puts "Filter source: #{revision.title} #{revision.time}"
       else
-        progress_bar.increment
+        migrate_markup_progress_bar.increment
       end
 
       body_new = filter_processor.run(revision.body)
 
       unless body_new == revision.body
         revision.body = body_new
-        revision.author_name = target_markup
+        revision.author_name = 'Caramelize'
         revision.time = Time.now
         revision.author = nil
+        revision.message = "Markup of '#{revision.title}' converted to #{target_markup}"
 
         # commit as latest page revision
         output_wiki.commit_revision(revision, options[:markup])
       end
     end
+
+    def rename_home_page
+      output_wiki.rename_page(options[:home_page_title], DEFAULT_GOLLUM_HOME_TITLE)
+    end
   end
 end

data/lib/caramelize/filter_processor.rb

@@ -1,5 +1,4 @@
 require 'gollum-lib'
-require 'grit'
 require 'ruby-progressbar'
 
 module Caramelize
@@ -16,7 +15,7 @@ module Caramelize
     def run(body)
       body_new = body
       filters.each do |filter|
-        body_new = filter.run(body_new)
+        body_new = filter.new(body_new).run
       end
       body_new
     end

data/lib/caramelize/filters/remove_table_tab_line_endings.rb

@@ -1,9 +1,13 @@
 module Caramelize
   class RemoveTableTabLineEndings
 
+    def initialize(body)
+      @body = body
+    end
+
     # take an input stream and convert all wikka syntax to markdown syntax
-    def run body
-      migrated_body = body.dup
+    def run
+      migrated_body = @body.dup
       migrated_body.gsub!(/\|[\t ]*\r?[\n]/, "|\n")
       migrated_body
     end

data/lib/caramelize/filters/swap_wiki_links.rb

@@ -1,9 +1,13 @@
 module Caramelize
   class SwapWikiLinks
 
+    def initialize(body)
+      @body = body
+    end
+
     # take an input stream and convert all wikka syntax to markdown syntax
-    def run body
-      migrated_body = body.dup
+    def run
+      migrated_body = @body.dup
 
       migrated_body.gsub!(/\[\[(\S+)\|(.+?)\]\]/, '[[\2|\1]]')
       migrated_body.gsub!(/\[\[([\w\s\.]*)\]\]/) do |s|

data/lib/caramelize/filters/wikka_to_markdown.rb

@@ -1,39 +1,67 @@
 module Caramelize
   class Wikka2Markdown
+    attr_reader :source_body
 
-    # take an input stream and convert all wikka syntax to markdown syntax
-    def run body
-      body = body.dup
-      body.gsub!(/(======)(.*?)(======)/ ) {|s| '# ' + $2 } #h1
-      body.gsub!(/(=====)(.*?)(=====)/) {|s| '## ' + $2 }   #h2
-      body.gsub!(/(====)(.*?)(====)/) {|s| '### ' + $2 }   #h3
-      body.gsub!(/(===)(.*?)(===)/) {|s| '#### ' + $2 }   #h4
-
-      body.gsub!(/(\*\*)(.*?)(\*\*)/) {|s| '**' + $2 + '**' }   #bold
-      body.gsub!(/(\/\/)(.*?)(\/\/)/) {|s| '_' + $2 + '_' }   #italic
-      #str.gsub!(/(===)(.*?)(===)/) {|s| '`' + $2 + '`'}   #code
-      body.gsub!(/(__)(.*?)(__)/) {|s| '<u>' + $2 + '</u>'}   #underline
-      body.gsub!(/(---)/, '  ')   #forced linebreak
-
-      #body.gsub!(/(.*?)(\n\t-)(.*?)/) {|s| $1 + $3 }   #list
-
-      body.gsub!(/(\t-)(.*)/, '*\2')    # unordered list
-      body.gsub!(/(~-)(.*)/, '*\2')     # unordered list
-      body.gsub!(/(    -)(.*)/, '*\2')     # unordered list
-      # TODO ordered lists
+    def initialize(source_body)
+      @source_body = source_body
+    end
 
+    def run
       # TODO images: ({{image)(url\=?)?(.*)(}})
+      # markdown: ![Tux, the Linux mascot](/assets/images/tux.png)
+
+      replace_headlines
+      replace_formatting
+      replace_lists
+      replace_wiki_links
+      replace_links
+      replace_code_block
+
+      target_body
+    end
+
+    def replace_headlines
+      target_body.gsub!(/(======)(.*?)(======)/ ) {|s| '# ' + $2 } #h1
+      target_body.gsub!(/(=====)(.*?)(=====)/) {|s| '## ' + $2 }   #h2
+      target_body.gsub!(/(====)(.*?)(====)/) {|s| '### ' + $2 }   #h3
+      target_body.gsub!(/(===)(.*?)(===)/) {|s| '#### ' + $2 }   #h4
+      target_body.gsub!(/(==)(.*?)(==)/) {|s| '##### ' + $2 }   #h5
+    end
 
-      #str.gsub!(/(----)/) {|s| '~~~~'}   #seperator
+    def replace_formatting
+      target_body.gsub!(/(\*\*)(.*?)(\*\*)/) {|s| '**' + $2 + '**' }   #bold
+      target_body.gsub!(/(\/\/)(.*?)(\/\/)/, '*\2*') #italic
+      target_body.gsub!(/(__)(.*?)(__)/) {|s| '<u>' + $2 + '</u>'}   #underline
+      target_body.gsub!(/(---)/, '  ')   #forced linebreak
+    end
 
+    def replace_lists
+      target_body.gsub!(/(\t-\s?)(.*)/, '* \2')    # unordered list
+      target_body.gsub!(/(~-\s?)(.*)/, '* \2')     # unordered list
+      target_body.gsub!(/(    -\s?)(.*)/, '* \2')     # unordered list
 
-      body.gsub!(/(\[\[)(\w+)\s(.+?)(\]\])/, '[[\3|\2]]')
-      #body.gsub!(/\[\[(\w+)\s(.+)\]\]/, ' [[\1 | \2]] ')
+      target_body.gsub!(/(~1\)\s?)(.*)/, '1. \2')     # unordered list
+      # TODO ordered lists
+    end
 
+    def replace_wiki_links
+      target_body.gsub!(/[\[]{2}(\w+)[\s|](.+?)[\]]{2}/, '[[\2|\1]]')
+    end
 
-      # TODO more syntax conversion for links and images
+    def replace_links
+      target_body.gsub!(/[\[]{2}((\w+):[\S][^\| ]*)[\| ](.*)[\]]{2}/,
+                 '[\3](\1)')
+      target_body.gsub!(/[\[]{2}((\w+):.*)[\]]{2}/, '<\1>')
+    end
+
+    def replace_code_block
+      target_body.gsub!(/^%%\s(.*?)%%\s?/m) do
+        $1.gsub(/^/, '    ')
+      end
+    end
 
-      body
+    def target_body
+      @target_body ||= source_body.dup
     end
   end
 end