gjp 0.22.0 → 0.23.0

data/README.md

@@ -38,7 +38,7 @@ The basic process is:
 * a `gjp` project is created;
 * package sources are added in `src/<package name>`;
 * any other file that is needed for the build, except the JDK, is added in binary form (jars, the Maven executable, plugins,  etc.) in `kit/`. In `gjp` a **kit** is a set of binary files that satisfies all build dependencies in a `gjp` project;
-* a build is attempted and during the build, `gjp` keeps track of file changes. When it finishes, `gjp` restores `src/` in its original state, making it a "repeatable dry-run build". `gjp` will retain any files that were automatically downloaded by Maven or other similar tools in `kit/`, along with other binary build dependencies, and create a list of files produced by the build for later spec generation;
+* a build is attempted and during the build, `gjp` keeps track of file changes. When it finishes, `gjp` restores `src/` in its original state, making it a "repeatable dry-run build". `gjp` will retain any files that were automatically downloaded by Maven or other similar tools in `kit/`, along with other binary build dependencies, and note any produced file for later spec generation;
 * `gjp` produces spec files for two packages: one for the project itself and one for the kit needed to build it;
 * kit and project packages can be submitted to [OBS](http://en.opensuse.org/openSUSE:Build_Service). Project package will rebuild cleanly because it needs no Internet access - all files were already downloaded during the dry-run phase above and are included in the kit.
 
@@ -91,7 +91,7 @@ Success! Now we have to tell gjp to return in normal mode:
 
     gjp finish
 
-At this point `gjp` restored `src/` as it was before the build and listed outputs in `output/commons-collections/produced_file_list`. It will be used later to compile the `%install` and `%files` sections of the project spec.
+At this point `gjp` restored `src/` as it was before the build while taking note of any produced file, so that later it will be able to output `%install` and `%files` sections of the project spec file automatically.
 
 Note that, if the build was unsusccesful, the following command can be used to cancel it and return to pre-dry running state:
 
@@ -99,9 +99,9 @@ Note that, if the build was unsusccesful, the following command can be used to c
 
 #### Generating a build script
 
-`gjp` expects that all commands needed to build a package are put in a `build.sh` script in `src/<package name>`. If you are a Bash user you are lucky - `gjp` can do this for you by looking at your command history! Just type:
+`gjp` expects that all commands needed to build a package are in a `build.sh` script in `src/<package name>`. If you are a Bash user you are lucky - `gjp` can create one for you by looking at your command history! Just type:
 
-    gjp generate-build-script commons-collections
+    gjp generate-package-script commons-collections
 
 Note that `gjp` will substitute the `gjp mvn` calls with equivalent lines that are actually runnable on a build host without `gjp` itself.
 
@@ -132,6 +132,14 @@ You can also edit the specs file manually if you want. When you later regenerate
 
 Packages are ready to be submitted to an OBS project. As OBS integration is not yet implemented, refer to OBS documentation to do that.
 
+#### Ant packages
+
+Building Ant packages is not really different from Maven ones, as `gjp ant` will operate exactly like `gjp mvn`.
+
+Sometimes you will have jar files distributed along with the source archive that will end up in `src/`: you don't want that, just run `gjp purge-jars` to have them moved to the kit. The command will generate a symlink back to the original, so builds will not fail.
+
+Once built, you are advised to grab an "informational" pom.xml (via `gjp get-pom` or Maven Central) in order to generate its spec.
+
 #### Optional: running networkless dry-run builds
 
 `gjp` has a subcommand to setup a `nonet` user without Internet access, courtesy of `iptables`. You can simply retry the build using that user to see if it works. Note that the following commands will alter group permissions to allow both your current user and `nonet` to work on the same files.

data/integration-tests/commons.sh

@@ -15,9 +15,9 @@ unzip commons-collections-3.2.1-src.zip
 rm commons-collections-3.2.1-src.zip
 
 cd ../../kit
-wget http://apache.fastbull.org/maven/maven-3/3.1.0/binaries/apache-maven-3.1.0-bin.zip
-unzip apache-maven-3.1.0-bin.zip
-rm apache-maven-3.1.0-bin.zip
+wget http://apache.fastbull.org/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.zip
+unzip apache-maven-*.zip
+rm apache-maven-*.zip
 cd ..
 
 gjp dry-run
@@ -55,4 +55,4 @@ gjp generate-package-spec commons-fileupload src/commons-fileupload/commons-file
 
 echo "**************** All Done ****************"
 
-ls -lah output/*
+ls -lah output/*

data/lib/gjp.rb

@@ -2,7 +2,7 @@ require "gjp/version"
 require "gjp/logger"
 require "gjp/template_manager"
 require "gjp/git"
-require "gjp/build_script_generator"
+require "gjp/script_generator"
 require "gjp/project"
 require "gjp/pom"
 require "gjp/version_matcher"

data/lib/gjp/cli.rb

@@ -109,21 +109,11 @@ module Gjp
       end
     end
 
-    subcommand "purge-jars", "Locates jars in src/ and moves them to kit/" do
-      def execute
-        Gjp::Project.new(".").purge_jars.each do |original, final|
-          puts "Replaced #{original} with symlink pointing to to #{final}"
-        end
-      end
-    end
-
-    subcommand "generate-build-script", "Create or refresh a build.sh file" do
-      parameter "NAME", "name of a package, that is, an src/ subdirectory name"
+    subcommand "generate-kit-spec", "Create or refresh a spec file for the kit" do
       def execute
         checking_exceptions do
           project = Gjp::Project.new(".")
-          history_file = File.join(Dir.home, ".bash_history")
-          result_path, conflict_count = Gjp::BuildScriptGenerator.new(project, history_file).generate_build_script(name)
+          result_path, conflict_count = Gjp::SpecGenerator.new(project).generate_kit_spec
           puts "#{format_path(result_path, project)} generated"
           if conflict_count > 0
             puts "Warning: #{conflict_count} unresolved conflicts"
@@ -132,26 +122,28 @@ module Gjp
       end
     end
 
-    subcommand "generate-kit-spec", "Create or refresh a spec file for the kit" do
+    subcommand "generate-kit-archive", "Create or refresh the kit tarball" do
+      option ["-i", "--incremental"], :flag, "create an archive with only the difference from the previous one"
       def execute
         checking_exceptions do
           project = Gjp::Project.new(".")
-          result_path, conflict_count = Gjp::SpecGenerator.new(project).generate_kit_spec
+          result_path = Gjp::Archiver.new(project).archive_kit(incremental?)
           puts "#{format_path(result_path, project)} generated"
-          if conflict_count > 0
-            puts "Warning: #{conflict_count} unresolved conflicts"
-          end
         end
       end
     end
 
-    subcommand "generate-kit-archive", "Create or refresh the kit tarball" do
-      option ["-i", "--incremental"], :flag, "create an archive with only the difference from the previous one"
+    subcommand "generate-package-script", "Create or refresh a build.sh file for a package" do
+      parameter "NAME", "name of a package, that is, an src/ subdirectory name"
       def execute
         checking_exceptions do
           project = Gjp::Project.new(".")
-          result_path = Gjp::Archiver.new(project).archive_kit(incremental?)
+          history_file = File.join(Dir.home, ".bash_history")
+          result_path, conflict_count = Gjp::ScriptGenerator.new(project, history_file).generate_build_script(name)
           puts "#{format_path(result_path, project)} generated"
+          if conflict_count > 0
+            puts "Warning: #{conflict_count} unresolved conflicts"
+          end
         end
       end
     end
@@ -183,6 +175,14 @@ module Gjp
       end
     end
 
+    subcommand "purge-jars", "Locates jars in src/ and moves them to kit/" do
+      def execute
+        Gjp::Project.new(".").purge_jars.each do |original, final|
+          puts "Replaced #{original} with symlink pointing to to #{final}"
+        end
+      end
+    end
+
     subcommand "set-up-nonet-user", "Sets up a \"nonet\" user that cannot access the network" do
       def execute
         checking_exceptions do

data/lib/gjp/git.rb

@@ -26,8 +26,20 @@ module Gjp
     # returns a list of filenames that changed in the repo
     # since the specified tag
     def changed_files_since(tag)
+      changed_files_between(tag, nil, ".")
+    end
+
+    # returns a list of filenames that changed in the repo
+    # between specified tags, in a certain directory
+    def changed_files_between(start_tag, end_tag, directory)
       Dir.chdir(@directory) do
-        `git diff-tree --no-commit-id --name-only -r gjp_#{tag} HEAD`.split("\n")
+        prefixed_start_tag = "gjp_#{start_tag}"
+        prefixed_end_tag = if end_tag
+          "gjp_#{end_tag}"
+        else
+          "HEAD"
+        end
+        `git diff-tree --no-commit-id --name-only -r #{prefixed_start_tag} #{prefixed_end_tag} -- #{directory}`.split("\n")
       end
     end
 

data/lib/gjp/package_spec_adapter.rb

@@ -36,15 +36,8 @@ module Gjp
       @runtime_dependency_ids = pom.runtime_dependency_ids
       @description = cleanup_description(pom.description, 1500)
 
-      produced_file_list = File.join(project.full_path, "output", @name, "produced_file_list")
-      @outputs = if File.exist?(produced_file_list)
-        File.open(produced_file_list).readlines.map do |line|
-          line.strip
-        end.select do |line|
-          File.fnmatch? filter, File.basename(line.strip)
-        end
-      else
-        []
+      @outputs = project.get_produced_files(package_name).select do |file|
+        File.fnmatch? filter, File.basename(file)
       end
     end
 

data/lib/gjp/project.rb

@@ -90,13 +90,9 @@ module Gjp
           @git.revert_whole_directory(".", latest_tag(:dry_run_started))
           @git.delete_tag(latest_tag(:dry_run_started))
         else
-          take_snapshot "Changes during dry-run"
-
-          update_produced_file_lists
-          take_snapshot "File list updates"
+          take_snapshot "Changes during dry-run", :dry_run_changed
 
           @git.revert_whole_directory("src", latest_tag(:dry_run_started))
-          take_snapshot "Sources reverted as before dry-run"
 
           take_snapshot "Dry run finished", :dry_run_finished
         end
@@ -105,38 +101,6 @@ module Gjp
       false
     end
 
-    # updates files that contain lists of the output files produced by
-    # the build of each package
-    def update_produced_file_lists
-      each_package_directory do |name, path|
-        FileUtils.mkdir_p(File.join("output", name))
-
-        list_file = File.join("output", name, "produced_file_list")
-        tracked_files = if File.exists?(list_file)
-          File.readlines(list_file).map { |line| line.strip }
-        else
-          []
-        end
-
-        files = (
-          @git.changed_files_since(latest_tag(:dry_run_started))
-            .select { |file| file.start_with?(path) }
-            .map { |file|file[path.length + 1, file.length] }
-            .concat(tracked_files)
-            .uniq
-            .sort
-        )
-
-        log.debug("writing file list for #{path}: #{files.to_s}")
-
-        File.open(list_file, "w+") do |file_list|
-          files.each do |file|
-            file_list.puts file
-          end
-        end
-      end
-    end
-
     # takes a revertable snapshot of this project
     def take_snapshot(message, tag_prefix = nil, tag_message = nil)
       tag = if tag_prefix
@@ -200,23 +164,28 @@ module Gjp
       end
     end
 
-    # runs a block for each package directory in src/
-    def each_package_directory
-      from_directory do
-        Dir.foreach("src") do |entry|
-          if File.directory?(File.join(Dir.getwd, "src", entry)) and entry != "." and entry != ".."
-            directory = File.join("src", entry)
-            yield entry, directory
-          end
-        end
-      end
-    end
-
     # returns the latest dry run start directory
     def latest_dry_run_directory
       @git.get_message(latest_tag(:dry_run_started))
     end
 
+    # returns a list of files produced during dry-runs in a certain package
+    def get_produced_files(package)
+      dry_run_count = latest_tag_count(:dry_run_changed)
+      log.debug "Getting produced files from #{dry_run_count} dry runs"
+      if dry_run_count >= 1
+        package_dir = File.join("src", package)
+        (1..dry_run_count).map do |i|
+          @git.changed_files_between("dry_run_started_#{i}", "dry_run_changed_#{i}", package_dir)
+        end
+          .flatten
+          .uniq
+          .sort
+          .map { |file| Pathname.new(file).relative_path_from(Pathname.new(package_dir)).to_s }
+      else
+        []
+      end
+    end
 
     # moves any .jar from src/ to kit/ and links it back
     def purge_jars      

data/lib/gjp/{build_script_generator.rb → script_generator.rb}

@@ -2,7 +2,7 @@
 
 module Gjp
   # generates build scripts from bash_history
-  class BuildScriptGenerator
+  class ScriptGenerator
     include Logger
 
     def initialize(project, history_path)

data/lib/gjp/version.rb

@@ -1,5 +1,5 @@
 # encoding: UTF-8
 
 module Gjp
-  VERSION = "0.22.0"
+  VERSION = "0.23.0"
 end

data/spec/lib/git_spec.rb

@@ -0,0 +1,83 @@
+# encoding: UTF-8
+
+require 'spec_helper'
+
+describe Gjp::Git do
+  before(:each) do
+    @git_path = File.expand_path(File.join("spec", "data", "test-repo"))
+    Dir.mkdir(@git_path)
+
+    @git = Gjp::Git.new(@git_path)
+    @git.init
+  end
+
+  after(:each) do
+    FileUtils.rm_rf(@git_path)
+  end
+
+  describe "#init"  do
+    it "complains if a double initialization is attempted" do
+      expect {
+        @git.init
+      }.to raise_error(Gjp::GitAlreadyInitedException)
+    end
+  end
+
+  describe "#changed_files_since"  do
+    it "lists files changed since a gjp tag" do
+      Dir.chdir(@git_path) do
+        File.open("file1", "w") do |file|
+          file.write "test"
+        end
+
+        @git.commit_whole_directory("test", "test")
+
+        File.open("file2", "w") do |file|
+          file.write "test"
+        end
+
+        @git.commit_whole_directory("test end")
+
+        files = @git.changed_files_since("test")
+
+        files.should_not include("file1")
+        files.should include("file2")
+      end
+    end
+  end
+
+  describe "#changed_files_between"  do
+    it "lists files changed between gjp tags" do
+      Dir.chdir(@git_path) do
+        File.open("file1", "w") do |file|
+          file.write "test"
+        end
+
+        @git.commit_whole_directory("test", "test_start")
+
+        File.open("file2", "w") do |file|
+          file.write "test"
+        end
+        Dir.mkdir("subdir")
+        File.open(File.join("subdir","file3"), "w") do |file|
+          file.write "test"
+        end
+
+        @git.commit_whole_directory("test", "test_end")
+
+        File.open("file4", "w") do |file|
+          file.write "test"
+        end
+
+        @git.commit_whole_directory("test")
+
+        files = @git.changed_files_between("test_start", "test_end", "subdir")
+
+        files.should_not include("file1")
+        files.should_not include("file2")
+        files.should include("subdir/file3")
+        files.should_not include("file4")
+      end
+    end
+  end
+end

data/spec/lib/project_spec.rb

@@ -97,11 +97,10 @@ describe Gjp::Project do
       @project.is_dry_running.should be_false
 
       @project.from_directory do
-        `git rev-list --all`.split("\n").length.should eq 5
+        `git rev-list --all`.split("\n").length.should eq 4
         File.read("src/abc/test").should eq "A\n"
-        File.readlines(File.join("output", "abc", "produced_file_list")).should include("test2\n")
 
-        `git diff-tree --no-commit-id --name-only -r HEAD~`.split("\n").should_not include("src/abc/test2")
+        `git diff-tree --no-commit-id --name-only -r HEAD~`.split("\n").should include("src/abc/test2")
         File.exists?("src/abc/test2").should be_false
       end
     end
@@ -134,8 +133,6 @@ describe Gjp::Project do
 
         File.read("kit/test").should eq "A\n"
         File.exists?("kit/test2").should be_false
-
-        File.exists?(File.join("output", "abc", "produced_file_list")).should be_false
       end
     end
   end
@@ -161,6 +158,35 @@ describe Gjp::Project do
     end    
   end
 
+  describe "#get_produced_files" do
+    it "gets a list of produced files" do
+      @project.from_directory do
+        Dir.mkdir("src/abc")
+        `echo A > src/abc/added_outside_dry_run`
+      end
+
+      @project.dry_run.should be_true
+      @project.from_directory do
+        `echo A > src/abc/added_in_first_dry_run`
+        `echo A > src/added_outside_directory`
+      end
+      @project.finish(false).should be_true
+
+      @project.dry_run.should be_true
+      @project.from_directory do
+        `echo A > src/abc/added_in_second_dry_run`
+      end
+      @project.finish(false).should be_true
+
+      list = @project.get_produced_files("abc")
+      list.should include("added_in_first_dry_run")
+      list.should include("added_in_second_dry_run")
+
+      list.should_not include("added_outside_dry_run")
+      list.should_not include("added_outside_directory")
+    end
+  end
+
   describe "#purge_jars" do
     it "moves jars in kit/jars" do
       @project.from_directory do

data/spec/lib/{build_script_generator_spec.rb → script_generator_spec.rb}

@@ -2,7 +2,7 @@
 
 require 'spec_helper'
 
-describe Gjp::BuildScriptGenerator do
+describe Gjp::ScriptGenerator do
   before(:each) do
     @project_path = File.join("spec", "data", "test-project")
     Dir.mkdir(@project_path)
@@ -22,7 +22,7 @@ describe Gjp::BuildScriptGenerator do
 
       FileUtils.mkdir_p(File.join("src", "test-package"))
 
-      @generator = Gjp::BuildScriptGenerator.new(@project, "history")
+      @generator = Gjp::ScriptGenerator.new(@project, "history")
     end
 
     mock_maven_executable

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: gjp
 version: !ruby/object:Gem::Version
-  version: 0.22.0
+  version: 0.23.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-10-11 00:00:00.000000000 Z
+date: 2013-11-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -158,7 +158,6 @@ files:
 - integration-tests/commons.sh
 - lib/gjp.rb
 - lib/gjp/archiver.rb
-- lib/gjp/build_script_generator.rb
 - lib/gjp/cli.rb
 - lib/gjp/git.rb
 - lib/gjp/kit_runner.rb
@@ -171,6 +170,7 @@ files:
 - lib/gjp/pom.rb
 - lib/gjp/pom_getter.rb
 - lib/gjp/project.rb
+- lib/gjp/script_generator.rb
 - lib/gjp/source_address_getter.rb
 - lib/gjp/source_getter.rb
 - lib/gjp/spec_generator.rb
@@ -202,7 +202,7 @@ files:
 - spec/data/struts-apps/pom.xml
 - spec/data/tomcat/pom.xml
 - spec/lib/archiver_spec.rb
-- spec/lib/build_script_generator_spec.rb
+- spec/lib/git_spec.rb
 - spec/lib/kit_runner_spec.rb
 - spec/lib/limited_network_user_spec_interactive.rb
 - spec/lib/maven_website_spec.rb
@@ -210,6 +210,7 @@ files:
 - spec/lib/pom_getter_spec.rb
 - spec/lib/pom_spec.rb
 - spec/lib/project_spec.rb
+- spec/lib/script_generator_spec.rb
 - spec/lib/source_address_getter_spec.rb
 - spec/lib/source_getter_spec.rb
 - spec/lib/spec_generator_spec.rb
@@ -259,7 +260,7 @@ test_files:
 - spec/data/struts-apps/pom.xml
 - spec/data/tomcat/pom.xml
 - spec/lib/archiver_spec.rb
-- spec/lib/build_script_generator_spec.rb
+- spec/lib/git_spec.rb
 - spec/lib/kit_runner_spec.rb
 - spec/lib/limited_network_user_spec_interactive.rb
 - spec/lib/maven_website_spec.rb
@@ -267,6 +268,7 @@ test_files:
 - spec/lib/pom_getter_spec.rb
 - spec/lib/pom_spec.rb
 - spec/lib/project_spec.rb
+- spec/lib/script_generator_spec.rb
 - spec/lib/source_address_getter_spec.rb
 - spec/lib/source_getter_spec.rb
 - spec/lib/spec_generator_spec.rb