tetra 1.2.2 → 2.0.0

data/CONTRIBUTING.md

@@ -0,0 +1,11 @@
+# Development environment
+
+To develop tetra, you will need:
+ * to install Bundler. On SUSE distros, use `sudo zypper in rubygem-bundler`;
+ * to get development dependencies: `bundle install`;
+
+To install a development version of tetra use:
+
+```
+sudo rake install
+```

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    tetra (1.1.0)
+    tetra (1.2.2)
       clamp
       json_pure
       open4
@@ -11,9 +11,30 @@ PATH
 GEM
   remote: http://rubygems.org/
   specs:
-    clamp (0.6.5)
+    aruba (0.6.2)
+      childprocess (>= 0.3.6)
+      cucumber (>= 1.1.1)
+      rspec-expectations (>= 2.7.0)
+    builder (3.2.2)
+    childprocess (0.5.6)
+      ffi (~> 1.0, >= 1.0.11)
+    clamp (1.0.0)
+    cucumber (2.0.0)
+      builder (>= 2.1.2)
+      cucumber-core (~> 1.1.3)
+      diff-lcs (>= 1.1.3)
+      gherkin (~> 2.12)
+      multi_json (>= 1.7.5, < 2.0)
+      multi_test (>= 0.1.2)
+    cucumber-core (1.1.3)
+      gherkin (~> 2.12.0)
     diff-lcs (1.2.5)
+    ffi (1.9.8)
+    gherkin (2.12.2)
+      multi_json (~> 1.3)
     json_pure (1.8.2)
+    multi_json (1.11.0)
+    multi_test (0.1.2)
     open4 (1.3.4)
     rake (10.4.2)
     rspec (3.2.0)
@@ -36,6 +57,7 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
+  aruba
   rake
   rspec
   tetra!

data/README.md

@@ -12,7 +12,7 @@ You need:
 
 * [Ruby 1.9.2](https://www.ruby-lang.org/en/) or later;
 * [git](http://git-scm.com/);
-* bash;
+* some basic Unix commands: bash, unzip, tar;
 * a JDK that can compile whatever software you need to package;
 * Fedora only: packages to compile native gems, use `yum install ruby-devel gcc-c++ zlib-devel`;
 
@@ -24,9 +24,9 @@ Install `tetra` via RubyGems:
 
 Building a package with `tetra` is quite unusual — this is a deliberate choice, so don't worry. Basic steps are:
 
-* `tetra init` a new project;
-* add sources to `src/<package name>` and anything else needed for the build in `kit/` in binary form (Ant and Maven are already pre-bundled);
-* execute `tetra dry-run`, which will open a bash subshell. In there, build your project, and when you are done conclude quitting it with `Ctrl+D`;
+* `tetra init <package name> package_sources.tar.gz` to initialize the project and unpack original sources;
+* if anything other than `ant` and `mvn` is needed in order to build the project, add it to the `kit/` directory in binary form;
+* execute `tetra dry-run`, which will open a bash subshell. Build your project, and when you are done conclude by exiting the subshell with `Ctrl+D`;
 * execute `tetra generate-all`: tetra will scaffold spec files and tarballs.
 
 Done!
@@ -38,25 +38,20 @@ During the dry-run `tetra`:
  - keeps track of changed files, in particular produced jars, which are included in the spec's `%files` section;
  - saves files downloaded from the Internet (eg. by Maven) and packs them to later allow networkless builds.
 
-Note that with `tetra` you are not building all dependencies from source - build dependencies are aggregated in a binary-only "blob" package. While this is not ideal it is sufficient to fulfill most open source licenses and to have a repeatable, networkless build, while being a lot easier to automate.
+Note that with `tetra` you are not building all dependencies from source - build dependencies are aggregated in a binary-only "kit" package. This is typically sufficient to fulfill open source licenses and to have a repeatable, networkless build.
 
 ## A commons-collections walkthrough
 
-First, ceate a new `tetra` project named after the package that we want to build:
+First, grab the sources:
 
-    mkdir commons-collections
-    cd commons-collections
-    tetra init
+    wget http://archive.apache.org/dist/commons/collections/source/commons-collections4-4.0-src.tar.gz
 
-Second, place source files in the `src/` folder:
+Second, ceate a new `tetra` project named `commons-collections` based on those sources:
 
-    cd src
-    wget http://archive.apache.org/dist/commons/collections/source/commons-collections4-4.0-src.tar.gz
-    tar xf commons-collections4-4.0-src.tar.gz
-    rm commons-collections4-4.0-src.tar.gz
-    cd commons-collections4-4.0-src/
+    tetra init commons-collections commons-collections4-4.0-src.tar.gz
+    cd commons-collections/src/commons-collections4-4.0-src
 
-Third, you need to show `tetra` how to build your package. Run `tetra dry-run` a new subshell will open, in there do anything you would normally do to build the package:
+Third, you need to show `tetra` how to build your package. Run `tetra dry-run` and a new subshell will open, in there do anything you would normally do to build the package (in this case, run Maven):
 
     tetra dry-run
 

data/SPECIAL_CASES.md

@@ -13,7 +13,6 @@ You can edit any file generated by tetra - regenerating it later will not overwr
 You can generate single files with the following commands:
 
 * `tetra generate-script`: (re)generates the `build.sh` file from commands used in the latest successful dry-run;
-* `tetra generate-archive`: (re)generates the package tarball;
 * `tetra generate-spec`: (re)generates the package spec;
 * `tetra generate-kit`: (re)generates the kit tarball and spec;
 
@@ -23,7 +22,7 @@ If you need to modify your project's sources for whatever reason, feel free to d
 
 After that you can retry the dry-run and spec generation: tetra will take care of creating patch files and adding `%patch` instructions for you.
 
-In case you want to swap sources completely and throw away all previous patches use `tetra patch --new-tarball`.
+In case you want to swap sources with a completely new archive, discarding all previous patches use `tetra change-sources <archive> "my message"`.
 
 ## Ant builds
 
@@ -92,9 +91,23 @@ Packages will rebuild cleanly in OBS because no Internet access is needed - all
 
 Note that the kit packages is only needed at build time by OBS, no end user should ever install it, so you can place it in a non-public project/repository if you so wish.
 
+## Packaging software without a source archive
+
+If your package sources do not ship in a source archive you can use `tetra init --no-sources <package_name>` to start a tetra project without sources. This is in general not recommended but it might be needed if your package sources are shipped in a format other than tar or zip, or it is distributed in multiple files, etc. Note that most version control systems allow you to export a checkout of a certain version in tarball form - this is recommended in tetra (and RPM in general) over using raw source files.
+
+Then you can add sources manually by:
+ * placing the archives in `packages/<project name>`
+ * unpacking them in `src`
+ * running `tetra change-sources --no-archive`.
+
+`tetra` assumes that sources in `packages/<project name>` and `src` match, except for patches as described above - it is your responsibility to keep them in sync.
+
+Note that most version control systems are able to produce an archive from a specific revision, using that is recommended over checking out raw files for `tetra` use.
+
 ## Gotchas
 
 * `tetra` internally uses `git` to keep track of files, any tetra project is actually also a `git` repo. Feel free to use it as any ordinary git repo, including pushing to a remote repo, rebasing, merging or using GitHub's pull requests. Just make sure any `tetra: ` comments are preserved;
+* `tetra` will commit in its `git` repo the `src` and `kit` directories during dry-run and any generated `spec` file. You are free to commit other files but it's not strictly necessary (eg. generated tarballs);
 * some Maven plugins like the Eclipse Project ones ([Tycho](https://www.eclipse.org/tycho/)) will save data in `/tmp` downloaded from the Internet and will produce errors if this data is not there during offline builds. One way to work around that is to force Java to use a kit subdirectory as `/tmp`. Add the following option to `mvn` during your build:
 
     -Djava.io.tmpdir=<full path to project>/kit/tmp

data/lib/template/kit.spec

@@ -23,6 +23,7 @@ Source0:        %{name}.tar.xz
 BuildRoot:      %{_tmppath}/%{name}-%{version}-build
 BuildArch:      noarch
 BuildRequires:  xz
+BuildRequires:  fdupes
 Provides:       tetra-kit
 Conflicts:      otherproviders(tetra-kit)
 
@@ -42,6 +43,7 @@ thus it should never be installed on end users' systems.
 export NO_BRP_CHECK_BYTECODE_VERSION=true
 install -d -m 0755 %{buildroot}%{_datadir}/tetra/
 cp -a * %{buildroot}%{_datadir}/tetra/
+%fdupes -s %{buildroot}%{_datadir}/tetra/
 
 %files
 %defattr(-,root,root)

data/lib/template/package.spec

@@ -22,7 +22,7 @@ License:        <%= license %>
 Summary:        <%= summary %>
 Url:            <%= url %>
 Group:          Development/Libraries/Java
-Source0:        %{name}.tar.xz
+Source0:        <%= src_archive %>
 Source1:        build.sh
 <% patches.to_enum.with_index.each do |patch, i| %>
 Patch<%= i %>:         <%= patch %>
@@ -46,7 +46,7 @@ Requires:       mvn(<%= dependency_id[0] %>:<%= dependency_id[1] %>) <% if depen
 %>
 
 %prep
-%setup -q -n src
+%setup -q -c -n src
 <% patches.to_enum.with_index.each do |patch, i| %>
 %patch<%= i %> -p2
 <% end %>

data/lib/tetra.rb

@@ -30,9 +30,12 @@ require "tetra/facades/git"
 require "tetra/facades/ant"
 require "tetra/facades/mvn"
 require "tetra/facades/bash"
+require "tetra/facades/unzip"
+require "tetra/facades/tar"
 
 # main internal classes
 require "tetra/version"
+require "tetra/project_initer"
 require "tetra/project"
 require "tetra/pom"
 require "tetra/kit"
@@ -51,11 +54,11 @@ require "tetra/ui/subcommand"
 require "tetra/ui/dry_run_subcommand"
 require "tetra/ui/generate_all_subcommand"
 require "tetra/ui/generate_kit_subcommand"
-require "tetra/ui/generate_archive_subcommand"
 require "tetra/ui/generate_script_subcommand"
 require "tetra/ui/generate_spec_subcommand"
 require "tetra/ui/get_pom_subcommand"
 require "tetra/ui/init_subcommand"
 require "tetra/ui/move_jars_to_kit_subcommand"
 require "tetra/ui/patch_subcommand"
+require "tetra/ui/change_sources_subcommand"
 require "tetra/ui/main"

data/lib/tetra/facades/bash.rb

@@ -29,8 +29,12 @@ module Tetra
           bashrc_file.write(bashrc_content)
           bashrc_file.flush
 
-          run_interactive("bash --rcfile #{bashrc_file.path}")
-          File.read(history_file).split("\n").map(&:strip)
+          run_interactive("bash --rcfile #{bashrc_file.path} -i")
+          history = File.read(history_file)
+          log.debug "history contents:"
+          log.debug history
+
+          history.split("\n").map(&:strip)
         end
       end
     end

data/lib/tetra/facades/git.rb

@@ -43,15 +43,17 @@ module Tetra
       end
     end
 
-    # adds all files in the current directory,
-    # removes all files not in the current directory,
+    # adds all files in the specified directories,
+    # removes all files not in the specified directories,
     # commits with message
-    def commit_whole_directory(directory, message)
-      Dir.chdir(@directory) do
-        log.debug "committing with message: #{message}"
+    def commit_directories(directories, message)
+      log.debug "committing with message: #{message}"
 
-        run("git rm -r --cached --ignore-unmatch #{directory}")
-        run("git add #{directory}")
+      Dir.chdir(@directory) do
+        directories.each do |directory|
+          run("git rm -r --cached --ignore-unmatch #{directory}")
+          run("git add #{directory}")
+        end
         run("git commit --allow-empty -F -", false, message)
       end
     end
@@ -65,20 +67,22 @@ module Tetra
       end
     end
 
-    # reverts path contents as per specified id
-    def revert_whole_directory(path, id)
+    # reverts multiple directories' contents as per specified id
+    def revert_directories(directories, id)
       Dir.chdir(@directory) do
-        # reverts added and modified files, both in index and working tree
-        run("git checkout -f #{id} -- #{path}")
-
-        # compute the list of deleted files
-        files_in_commit = run("git ls-tree --name-only -r #{id} -- #{path}").split("\n")
-        files_in_head = run("git ls-tree --name-only -r HEAD -- #{path}").split("\n")
-        files_added_after_head = run("git ls-files -o -- #{path}").split("\n")
-        files_to_delete = files_in_head - files_in_commit + files_added_after_head
-
-        files_to_delete.each do |file|
-          FileUtils.rm_rf(file)
+        directories.each do |directory|
+          # reverts added and modified files, both in index and working tree
+          run("git checkout -f #{id} -- #{directory}")
+
+          # compute the list of deleted files
+          files_in_commit = run("git ls-tree --name-only -r #{id} -- #{directory}").split("\n")
+          files_in_head = run("git ls-tree --name-only -r HEAD -- #{directory}").split("\n")
+          files_added_after_head = run("git ls-files -o -- #{directory}").split("\n")
+          files_to_delete = files_in_head - files_in_commit + files_added_after_head
+
+          files_to_delete.each do |file|
+            FileUtils.rm_rf(file)
+          end
         end
       end
     end
@@ -91,6 +95,17 @@ module Tetra
       end
     end
 
+    # renames git special files to 'disable' them
+    def disable_special_files(path)
+      Dir.chdir(File.join(@directory, path)) do
+        Find.find(".") do |file|
+          next unless file =~ /\.git(ignore)?$/
+
+          FileUtils.mv(file, "#{file}_disabled_by_tetra")
+        end
+      end
+    end
+
     # 3-way merges the git file at path with the one in new_path
     # assuming they have a common ancestor at the specified id
     # returns the conflict count

data/lib/tetra/facades/process_runner.rb

@@ -36,7 +36,9 @@ module Tetra
 
     # runs an interactive executable in a subshell
     def run_interactive(command)
+      log.debug "running `#{command}`"
       success = system({}, command)
+      log.debug "`#{command}` exited with success #{success}"
       fail ExecutionFailed.new(command, $CHILD_STATUS, nil, nil) unless success
     end
 

data/lib/tetra/facades/tar.rb

@@ -0,0 +1,14 @@
+# encoding: UTF-8
+
+module Tetra
+  # encapsulates unzip
+  class Tar
+    include ProcessRunner
+
+    # decompresses a file in a target directory
+    def decompress(tarfile, directory)
+      result = run("tar xvf #{tarfile} --directory #{directory}")
+      result.strip if result != ""
+    end
+  end
+end

data/lib/tetra/facades/unzip.rb

@@ -0,0 +1,14 @@
+# encoding: UTF-8
+
+module Tetra
+  # encapsulates unzip
+  class Unzip
+    include ProcessRunner
+
+    # decompresses a file in a target directory
+    def decompress(zipfile, directory)
+      result = run("unzip #{zipfile} -d #{directory}")
+      result.strip if result != ""
+    end
+  end
+end

data/lib/tetra/packages/package.rb

@@ -9,7 +9,8 @@ module Tetra
 
     attr_reader :patches
 
-    def_delegator :@project, :name, :name
+    def_delegator :@project, :name
+    def_delegator :@project, :src_archive
     def_delegator :@kit, :name, :kit_name
     def_delegator :@kit, :version, :kit_version
     def_delegator :@pom, :license_name, :license

data/lib/tetra/project.rb

@@ -3,6 +3,7 @@
 module Tetra
   # encapsulates a Tetra project directory
   class Project
+    include ProjectIniter
     include Logging
 
     # path of the project template files
@@ -39,53 +40,6 @@ module Tetra
       result
     end
 
-    # returns true if the specified directory is a valid tetra project
-    def self.project?(dir)
-      File.directory?(File.join(dir, "src")) &&
-        File.directory?(File.join(dir, "kit")) &&
-        File.directory?(File.join(dir, ".git"))
-    end
-
-    # inits a new project directory structure
-    def self.init(dir, include_bundled_software = true)
-      Dir.chdir(dir) do
-        Tetra::Git.new(".").init
-
-        FileUtils.mkdir_p("src")
-        FileUtils.mkdir_p("kit")
-
-        # populate the project with templates and commit it
-        project = Project.new(".")
-
-        project.template_files(include_bundled_software).each do |source, destination|
-          FileUtils.cp_r(File.join(TEMPLATE_PATH, source), destination)
-        end
-
-        project.commit_whole_directory(".", "Template files added")
-      end
-    end
-
-    # returns a hash that maps filenames that should be copied from TEMPLATE_PATH
-    # to the value directory
-    def template_files(include_bundled_software)
-      result = {
-        "kit" => ".",
-        "packages" => ".",
-        "src" => ".",
-        "gitignore" => ".gitignore"
-      }
-
-      if include_bundled_software
-        Dir.chdir(TEMPLATE_PATH) do
-          Dir.glob(File.join("bundled", "*")).each do |file|
-            result[file] = "kit"
-          end
-        end
-      end
-
-      result
-    end
-
     # checks whether there were edits to src/
     # since last mark
     def src_patched?
@@ -100,7 +54,7 @@ module Tetra
     def dry_run
       current_directory = Pathname.new(Dir.pwd).relative_path_from(Pathname.new(@full_path))
 
-      commit_whole_directory(".", "Dry-run started\n", "tetra: dry-run-started: #{current_directory}")
+      @git.commit_directories(%w(src kit), "Dry-run started\n\ntetra: dry-run-started: #{current_directory}")
     end
 
     # returns true iff we are currently dry-running
@@ -119,52 +73,33 @@ module Tetra
       changed_files = @git.changed_files("src", start_id)
 
       # revert to pre-dry-run status
-      @git.revert_whole_directory("src", start_id)
+      @git.revert_directories(["src"], start_id)
 
       # prepare commit comments
       comments = ["Dry run finished\n", "tetra: dry-run-finished"]
       comments += changed_files.map { |f| "tetra: file-changed: #{f}" }
       comments += build_script_lines.map { |l| "tetra: build-script-line: #{l}" }
 
-      # if this is the first dry-run, mark sources as tarball
-      comments << "tetra: sources-tarball" if first_dry_run
-
       # commit end of dry run
-      commit_whole_directory(".", *comments)
-    end
-
-    # returns true if this is the first dry-run
-    def first_dry_run
-      @git.latest_id("tetra: dry-run-finished").nil?
+      @git.commit_directories(["kit"], comments.join("\n"))
     end
 
     # ends a dry-run assuming the build went wrong
-    # reverts the whole project directory
+    # reverts src/ and kit/ directories
     def abort
-      @git.revert_whole_directory(".", @git.latest_id("tetra: dry-run-started"))
+      @git.revert_directories(%w(src kit), @git.latest_id("tetra: dry-run-started"))
       @git.undo_last_commit
     end
 
-    # commits all files in the directory
-    def commit_whole_directory(directory, *comments)
-      # rename all .gitignore files that might have slipped in
-      from_directory("src") do
-        Find.find(".") do |file|
-          next unless file =~ /\.gitignore$/
-
-          FileUtils.mv(file, "#{file}_disabled_by_tetra")
-        end
-      end
-
-      @git.commit_whole_directory(directory, comments.join("\n"))
-    end
-
     # commits files in the src/ dir as a patch or tarball update
-    def commit_sources(message, new_tarball = false)
+    def commit_sources(message, new_tarball)
       from_directory do
-        comments = ["#{message}\n"]
-        comments << "tetra: sources-tarball" if new_tarball
-        commit_whole_directory("src", comments)
+        comments = "#{message}\n"
+        if new_tarball
+          comments << "\ntetra: sources-tarball"
+          @git.disable_special_files("src")
+        end
+        @git.commit_directories(["src"], comments)
       end
     end
 
@@ -237,16 +172,6 @@ module Tetra
         .sort
     end
 
-    # archives a tarball of src/ in packages/
-    # the latest commit marked as tarball is taken as the version
-    def archive_sources
-      from_directory do
-        id = @git.latest_id("tetra: sources-tarball")
-        destination_path = File.join(full_path, packages_dir, name, "#{name}.tar.xz")
-        @git.archive("src", id, destination_path)
-      end
-    end
-
     # archives a tarball of kit/ in packages/
     # the latest commit marked as dry-run-finished is taken as the version
     def archive_kit
@@ -257,6 +182,18 @@ module Tetra
       end
     end
 
+    # returns the name of the source archive file, if any
+    def src_archive
+      from_directory do
+        Find.find(File.join("packages", name)) do |file|
+          if File.file?(file) && file.match(/\.(spec)|(sh)|(patch)$/).nil?
+            return File.basename(file)
+          end
+        end
+        nil
+      end
+    end
+
     # generates patches of src/ in packages/
     # the latest commit marked as tarball is taken as the base version,
     # other commits are assumed to be patches on top