perforce2svn 0.7.0

data/Gemfile

@@ -0,0 +1,2 @@
+source :rubygems
+gemspec

data/Gemfile.lock

@@ -0,0 +1,38 @@
+PATH
+  remote: .
+  specs:
+    perforce2svn (0.1.0)
+      choosy (>= 0.4.8)
+      log4r (>= 1.1.7)
+
+GEM
+  remote: http://rubygems.org/
+  specs:
+    ZenTest (4.5.0)
+    autotest (4.4.6)
+      ZenTest (>= 4.4.1)
+    autotest-notification (2.3.1)
+      autotest (~> 4.3)
+    choosy (0.4.8)
+    diff-lcs (1.1.2)
+    log4r (1.1.9)
+    mocha (0.9.12)
+    rspec (2.5.0)
+      rspec-core (~> 2.5.0)
+      rspec-expectations (~> 2.5.0)
+      rspec-mocks (~> 2.5.0)
+    rspec-core (2.5.1)
+    rspec-expectations (2.5.0)
+      diff-lcs (~> 1.1.2)
+    rspec-mocks (2.5.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  ZenTest
+  autotest
+  autotest-notification
+  mocha
+  perforce2svn!
+  rspec

data/LICENSE

@@ -0,0 +1,23 @@
+MIT LICENSE
+
+Copyright (c) 2011 Gabe McArthur
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+This Software shall be used for Good, not Evil.

data/README.markdown

@@ -0,0 +1,66 @@
+# Perforce2Svn Migration Tool
+
+This tool is designed to help you with relatively sophisticated migrations from a Perforce server into Subversion.  The Subversion repository must be on the file system for this command to function correctly.  If a repository does not exist, one will be created.
+
+## Prerequisites && Installation
+
+This gem has only been tested on Ubuntu, though it should work on any Linux/Unix derivative with the correct binary libraries. It has only been tested on 1.8.7, though it should work on 1.8.6; it most certainly does not work at all under 1.9, particularly since the binary libraries are build against 1.8.
+
+Sidenote: if you do run Ubuntu (or Debian), please do yourself a favor and install the latest from rubygems.org. Gem packaging and maintenance on Debian is complete crap and should not be trusted.
+
+For this gem to function correctly, you will need to install certain tools on your own.  You will need at least:
+
+- <code>svnadmin</code>: Required for cleaning up stale transactions if you Ctrl-C during a migration.
+- <code>p4</code>: The Perforce command line utility should be installed. You will need to log in with it (or p4v) before you can do any migration.
+
+In addition to these console tools, you will need to install some gems on your own. First, the <code>p4ruby (>= 1.0.9)</code> gem is required. I would list it as a dependency in the gemspec, but I have yet to have it succesfully install itself on its own. You should try the following:
+
+    gem install p4ruby
+    # ... Watch it fail, record the FTP url so that you can go to the 10.2 sources
+    cd ${GEM_HOME}/gems/p4ruby-1.0.9/
+    ruby install.rb --version 10.2 --platform ${YOUR_PLATFORM_HERE}
+
+Additionally, you will need to install the Subversion bindings for Ruby. These should install their code under RUBY_HOME. On Ubuntu, you can run:
+
+   apt-get install libsvn-ruby
+
+I'm sure there are other binary packages out there for other platforms.
+
+At this point, you can now install the main gem:
+
+    gem install perforce2svn
+
+This installs the <code>perforce2svn</code> command line client onto your path.
+
+## Usage
+
+The <code>perforce2svn</code> tool requires a mapping file that describes the migration activities that should occur. The general outline of a mapping file should look something like this:
+
+    # This is a comment. 
+    migrate //depot/from/perforce/path   /to/svn/trunk
+    migrate //depot/another/path         /to/svn/trunk/subdir
+
+    # Post-migration actions can also occur:
+    copy    /svn/path/in/tree   /to/another/path
+    move    /same/as/copy       /but/gets/rid/of/old/path
+    delete  /deletes/svn/file
+
+    # You can even add files at the end of a migration
+    update  /live/file/in/tree.txt
+
+The mapping file is fairly utilitarian, so you should feel free take what you need and leave the rest. There is a much more detailed discussion of the format of the mapping file and how to use it if you run <code>perforce2svn --mapping-file</code>.
+
+The only quirky bit is the use of the <code>--live-path</code> flag, which points at the base directory of some directory structure that you want to use in your mapping file. In particularly large and complicated migrations and refactorings, this can be useful to update post-migration files so that any fixes that need to be immediately applied can be. This can be a bit of a time-saver if you have to manually test and re-test the migration operation several times to be absolutely sure that the code is in a usable state after the migration.
+
+Running <code>perforce2svn --help</code> should give you most of the details that may be missing here.
+
+## Hacking
+
+This tool is not the most refined. It doesn't handle branches and merges, really at all. It's probably better than the Perl tool (since you can see what's actually happenning), but it can still be a bit difficult to maintain full backward history.
+
+If you want to add that functionality and take this project from me, go for it! I created an older version of this tool some time ago, and it should really be extended by somebody with more insight into Perforce's internals than myself. There are at least a few tests to see what you can do, and several of the tricky parts of the SVN API have been carefully masked for most operations.
+
+In fact, you could take some of the work here and create other Perforce migration clients, since you can pull out file contents fairly easily.
+
+May the Source be With You.
+

data/Rakefile

@@ -0,0 +1,24 @@
+$LOAD_PATH.unshift File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift File.expand_path("../spec", __FILE__)
+
+require 'fileutils'
+require 'rake'
+require 'rubygems'
+require 'rspec/core/rake_task'
+require 'choosy/rake'
+
+task :default => :spec
+task :test => [:integration, :spec]
+
+desc "Run the RSpec tests for the main perforce2svn tree"
+RSpec::Core::RakeTask.new(:spec) do |t|
+  t.pattern = 'spec/perforce2svn/**/*_spec.rb'
+end
+
+desc "Runs the integration tests"
+RSpec::Core::RakeTask.new(:integration) do |t|
+  t.pattern = 'spec/integration/**/*_spec.rb'
+end  
+
+desc "Clean up"
+task :clean => ['gem:clean']

data/bin/perforce2svn

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+begin
+  require 'rubygems'
+  require 'perforce2svn/cli'
+rescue LoadError
+  $LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', 'lib')
+  require 'perforce2svn/cli'
+end
+
+Perforce2Svn::CLI.new.execute!(ARGV)

data/lib/VERSION.yml

@@ -0,0 +1,6 @@
+--- 
+date: 01/04/2011
+version: 
+  tiny: 0
+  major: 0
+  minor: 7

data/lib/perforce2svn/cli.rb

@@ -0,0 +1,117 @@
+require 'perforce2svn/errors'
+require 'perforce2svn/version_range'
+require 'perforce2svn/mapping/mapping_file'
+require 'perforce2svn/migrator'
+require 'choosy'
+
+module Perforce2Svn
+  class CLI
+    include Choosy::Terminal
+
+    def execute!(args)
+      command.execute!(args)
+    end
+
+    def parse!(args, propagate=false)
+      command.parse!(args, propagate)
+    end
+
+    def command
+      ctx = self
+      version_info = Choosy::Version.load_from_parent
+      Choosy::Command.new :perforce2svn do
+        printer :manpage, 
+                :version => version_info.to_s,
+                :date => version_info.date,
+                :manual => 'Perforce2Svn'
+
+        executor do |args, options|
+          migrator = Migrator.new(args[0], options)
+          migrator.run!
+        end
+
+        summary 'Migrates Perforce repository files into Subversion'
+
+        section 'DESCRIPTION' do
+          para 'This is a migration tool for migrating specific branches in Perforce into Subversion.  It uses a mapping file to define the branch mappings at the directory level.'
+          para 'Because these migrations can be quite complex, and involve more sophisticated translations, this mapping file also allows for much more sophisticated operations on the Subversion repository after the migration, at least somewhat mitigating the difficulties in doing complex transformations in a unified way.'
+          para 'This utility assumes that you have already logged into Perforce and that the P4USER and P4PORT environment variables are set correctly.'
+        end
+
+        section 'OPTIONS' do
+          string :repository, "The path to the SVN repository. Required." do
+            required
+            depends_on :mapping_file # So that the mapping file will get printed and exit before this option gets validated
+          end
+          string :live_path, "The path to the files you want to add or update." do
+            validate do |args, options|
+              if !File.directory?(options[:live_path])
+                die "The --live-path must be a directory: #{options[:live_path]}"
+              end
+            end
+          end
+          string :changes, "The revision range to import from. This has the format START:END where START >= 1 and END can be any number or 'HEAD'." do
+            validate do |args, options|
+              options[:changes] = VersionRange.build(options[:changes])
+            end
+          end
+          boolean :skip_commands, "Skip the embedded commands in the configuration that are run after the perforce migration.", :short => '-u'
+          boolean :skip_perforce, "Skip the perforce migration, and run only the embedded commands.", :short => '-p'
+          boolean :analyze_only, "Only analyzes your mapping files for possible errors, but does not attempt to run the migration."
+
+          # Informative
+          para 
+          boolean :debug, "Prints extra debug information"
+          version version_info.to_s
+          boolean :mapping_file, "Shows a detailed mapping file example" do
+            validate do |args, options|
+              ctx.page(Mapping::MappingFile.help_file)
+              exit 0
+            end
+          end
+          help
+        end
+
+        section 'FILES' do
+          para "Rather than define all of the file path mappings between the old Perforce repository and the new Subversion repository, this tool requires you to define a mapping file."
+          para "This file contains all of the relevant commands for migrating a complicated set of paths and branches from Perforce into Subversion."
+          para "The mapping file has an explicit syntax.  Each line contains a directive and a set of arguments.  Each argument is separated by a space, though that space can be escaped with a '\\' character.  Lines can have comments starting with the '#' character and they continue to the end of the line."
+          para "Please use the '--mapping-help' command for more detailed information."
+        end
+
+        section 'ENVIRONMENT' do
+          para "P4USER   - The username used to connect to the Perforce server."
+          para "P4PORT   - The Perforce server name."
+          para "svnadmin - This tool must be installed."
+          para "p4       - The Perforce command line utility must be installed."
+        end
+
+        section 'BUGS AND LIMITATIONS' do
+          para "While this tool works better than the other, Perl-based tool, it has the same kind of limitations. Namely, it cannot track certain file changes well (like copying or moving). That requires information that isn't readily accessible."
+          para "Also, you may notice that some files are not exactly the same after the migration. The p4 utility occasionally adds newline characters at the end of the file stream, for inexplicable reasons, so sometimes there is an extra newline at the end of some text files. There's really no way around it."
+        end
+
+        section "LICENSE" do
+          para "This software is licensed under the MIT license. No warranty is expressed or implied by this software's use."
+          para "This software shall be used for Good, not Evil."
+        end
+
+        section "AUTHOR" do
+          para "Gabe McArthur <madeonamac@gmail.com>"
+          para "Submit fixes to: https://github.com/gabemc/perforce2svn"
+        end
+
+        arguments do
+          count 1
+          metaname 'MAPPING_FILE'
+
+          validate do |args, options|
+            if !File.file?(args[0])
+              die "the mapping file doesn't exist: #{args[0]}"
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/perforce2svn/environment.rb

@@ -0,0 +1,66 @@
+require 'choosy/terminal'
+require 'perforce2svn/perforce/p4_depot'
+
+module Perforce2Svn
+  class Environment
+    include Choosy::Terminal
+
+    def check!
+      check_svnadmin
+      check_svnlib
+      check_perforce
+      check_p4lib
+      check_p4_liveness
+    end
+
+    private
+    def check_svnadmin
+      if !command_exists?('svnadmin')
+        die "Unable to locate svnadmin"
+      end
+    end
+
+    def check_svnlib
+      begin
+        require 'svn/core'
+      rescue LoadError
+        die "Unable to locate the native subversion bindings. Please install."
+      end
+    end
+
+    def check_perforce
+      user = check_env('P4USER')
+      server = check_env('P4PORT')
+
+      if !system('p4 help > /dev/null 2>&1')
+        die "Unable to locate or execute the 'p4' command. Is it on the PATH? Are you logged in?"
+      end
+    end
+
+    def check_env(name)
+      value = ENV[name]
+      if value.nil? || value.empty?
+        die "Unable to locate the '#{name}' environment variable"
+      end
+      value
+    end
+
+    def check_p4lib
+      begin
+        require 'P4'
+        if P4.identify =~ /\((\d+.\d+) API\)/
+          maj, min = $1.split(/\./)
+          if maj.to_i < 2009
+            die "Requires a P4 library version >= 2009.2"
+          end
+        end
+      rescue LoadError
+        die 'Unable to locate the P4 library, please install p4ruby'
+      end
+    end
+
+    def check_p4_liveness
+      Perforce::P4Depot.instance.connect!
+    end
+  end
+end

data/lib/perforce2svn/errors.rb

@@ -0,0 +1,16 @@
+
+module Perforce2Svn
+  Error = Class.new(RuntimeError)
+
+  # Configuration
+  ConfigurationError = Class.new(Perforce2Svn::Error)
+
+  # Subversion errors
+  SvnTransactionError = Class.new(Perforce2Svn::Error)
+
+  # Perforce
+  P4Error = Class.new(Perforce2Svn::Error)
+
+  # Mappings
+  MappingParserError = Class.new(Perforce2Svn::Error)
+end

data/lib/perforce2svn/logging.rb

@@ -0,0 +1,35 @@
+require 'log4r'
+
+module Perforce2Svn
+  # A convenient module for mixins that allow to
+  # share the logging configuration everywhere
+  # easily
+  module Logging
+    @@log = nil
+
+    def self.configure(debug)
+      if @@log.nil?
+        @@log = Log4r::Logger.new 'perforce2svn'
+        @@log.outputters = Log4r::Outputter.stdout
+        @@log.level = if ENV['RSPEC_RUNNING']
+                        Log4r::FATAL
+                      elsif debug
+                        Log4r::DEBUG
+                      else
+                        Log4r::INFO
+                      end
+        Log4r::Outputter.stdout.formatter = Log4r::PatternFormatter.new(:pattern => "[%l]\t%M")
+      end
+      @@log
+    end
+    
+    def self.log
+      @@log ||= configure(true)
+    end
+    
+    # Meant for mixing into other classes for simplified logging
+    def log
+      @@log ||= Logging.log
+    end
+  end
+end

data/lib/perforce2svn/mapping/analyzer.rb

@@ -0,0 +1,30 @@
+require 'perforce2svn/logging'
+
+module Perforce2Svn::Mapping
+  class Analyzer
+    include Perforce2Svn::Logging
+    
+    def initialize(base_path)
+      @base_path = base_path
+    end
+
+    def check(commands)
+      # TODO: May want to make this more robust, like checking perforce paths and overlaps
+      succeeded = true
+      commands.each do |command|
+        if command.respond_to? :live_path
+          path = command.live_path
+          if path !~ /^\//
+            path = File.join(@base_path, path)
+          end
+          
+          unless File.file? path
+            log.error("(line #{command.line_number}) The live path doesn't exist: #{command.live_path}")
+            succeeded = false
+          end
+        end
+      end
+      succeeded
+    end
+  end
+end