polisher 0.4 → 0.5.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (72) hide show
  1. checksums.yaml +7 -0
  2. data/LICENSE +20 -661
  3. data/README.md +39 -0
  4. data/Rakefile +11 -88
  5. data/bin/binary_gem_resolver.rb +95 -0
  6. data/bin/gem_dependency_checker.rb +165 -0
  7. data/bin/git_gem_updater.rb +86 -0
  8. data/bin/ruby_rpm_spec_updater.rb +29 -0
  9. data/lib/polisher.rb +19 -0
  10. data/lib/polisher/apt.rb +12 -0
  11. data/lib/polisher/bodhi.rb +21 -0
  12. data/lib/polisher/bugzilla.rb +9 -0
  13. data/lib/polisher/core.rb +33 -0
  14. data/lib/polisher/errata.rb +43 -0
  15. data/lib/polisher/fedora.rb +43 -0
  16. data/lib/polisher/gem.rb +155 -0
  17. data/lib/polisher/gemfile.rb +72 -0
  18. data/lib/polisher/gemspec.rb +32 -0
  19. data/lib/polisher/git.rb +135 -0
  20. data/lib/polisher/koji.rb +54 -0
  21. data/lib/polisher/rhn.rb +14 -0
  22. data/lib/polisher/rpmspec.rb +254 -0
  23. data/lib/polisher/upstream.rb +29 -0
  24. data/lib/polisher/vendor.rb +9 -0
  25. data/lib/polisher/version_checker.rb +100 -0
  26. data/lib/polisher/yum.rb +28 -0
  27. data/spec/core_spec.rb +64 -0
  28. data/spec/fedora_spec.rb +14 -0
  29. data/spec/gem_spec.rb +82 -0
  30. data/spec/gemfile_spec.rb +45 -0
  31. data/spec/git_spec.rb +74 -0
  32. data/spec/rpmspec_spec.rb +105 -0
  33. data/spec/spec_helper.rb +50 -37
  34. data/spec/upstream_spec.rb +39 -0
  35. metadata +173 -179
  36. data/COPYING +0 -8
  37. data/README.rdoc +0 -105
  38. data/TODO +0 -7
  39. data/bin/server +0 -4
  40. data/config.ru +0 -25
  41. data/config/database.yml +0 -13
  42. data/config/polisher.yml +0 -5
  43. data/db/connection.rb +0 -52
  44. data/db/migrations/001_create_projects.rb +0 -23
  45. data/db/migrations/002_create_sources.rb +0 -25
  46. data/db/migrations/003_create_project_source_versions.rb +0 -28
  47. data/db/migrations/004_create_events.rb +0 -27
  48. data/db/migrations/005_create_project_dependencies.rb +0 -28
  49. data/db/models/event.rb +0 -87
  50. data/db/models/project.rb +0 -110
  51. data/db/models/project_dependency.rb +0 -27
  52. data/db/models/project_source_version.rb +0 -31
  53. data/db/models/source.rb +0 -101
  54. data/lib/common.rb +0 -71
  55. data/lib/dsl.rb +0 -292
  56. data/lib/event_handlers.rb +0 -166
  57. data/lib/gem_adapter.rb +0 -94
  58. data/lib/sinatra/url_for.rb +0 -40
  59. data/polisher.rb +0 -372
  60. data/public/stylesheets/style.css +0 -67
  61. data/spec/common_spec.rb +0 -28
  62. data/spec/dsl_spec.rb +0 -357
  63. data/spec/event_handlers_spec.rb +0 -300
  64. data/spec/gem_adapter_spec.rb +0 -89
  65. data/spec/models_spec.rb +0 -721
  66. data/spec/polisher_spec.rb +0 -573
  67. data/views/layout.haml +0 -22
  68. data/views/projects/index.haml +0 -42
  69. data/views/projects/index.html.haml +0 -38
  70. data/views/result.haml +0 -9
  71. data/views/sources/index.haml +0 -24
  72. data/views/sources/index.html.haml +0 -26
data/COPYING DELETED
@@ -1,8 +0,0 @@
1
- Polisher is free software, you can redistribute it and/or modify
2
- it under the terms of the GNU Affero General Public License
3
- as published by the Free Software Foundation, either version 3
4
- of the License, or (at your option) any later version.
5
-
6
- You should have received a copy of the the GNU Affero
7
- General Public License, along with Polisher. If not, see
8
- <http://www.gnu.org/licenses/>
@@ -1,105 +0,0 @@
1
- == Polisher
2
- Copyright (C) 2010 Red Hat, Inc.
3
-
4
- Written by Mohammed Morsi <mmorsi@redhat.com>
5
-
6
- === Intro
7
- Polisher is a project release management tool which can be used to register various event
8
- handlers to be invoked on specific versions of project / source releases. Polisher provides
9
- a simple {REST}[http://en.wikipedia.org/wiki/Representational_State_Transfer] interface which
10
- to create projects w/ sources and dependencies, and to download and transform them into any
11
- target output format.
12
-
13
- Polisher also provides a simple {DSL}[http://en.wikipedia.org/wiki/Domain-specific_language]
14
- frontend to the REST interface, able to be used to create and trigger elegant release
15
- workflows for any number of projects simultaneously. Workflows for many major Ruby based projects
16
- have been setup, transforming various versions into {Fedora}[http://fedoraproject.org/] repos and
17
- can be found at [http://github.com/movitto/polisher-scripts].
18
-
19
-
20
- == Running
21
- To install simply run:
22
-
23
- 'gem install polisher'
24
-
25
-
26
- Alternatively checkout the source from github via
27
-
28
- 'git clone git://github.com/movitto/polisher.git'
29
-
30
-
31
- Polisher requires various rubygems to work. gem install should pull in all dependencies
32
- but if running polisher from source, be sure to gem install:
33
-
34
- * rubygems
35
- * sinatra
36
- * libxml
37
- * curb
38
- * activerecord
39
- * libxml
40
-
41
- For the complete list of dependencies, see the
42
- {Rakefile}[http://github.com/movitto/polisher/blob/master/Rakefile].
43
-
44
-
45
- Polisher currently pulls config from the config/ subdir, the database
46
- params are defined in database.yml (currently defaulting to a sqlite flat file
47
- db residing in db/data), and the general application config resides in
48
- polisher.yml. The only general app config currently used is 'gem_api_key',
49
- which will be read from ~/.gem/credentials if missing from polisher.yml.
50
-
51
- To run the server, run './bin/server' in the project root.
52
-
53
- To run the spec suite simply run 'rake spec'
54
-
55
- Config is environment specific, the server runs in the 'development' environment
56
- by default and the spec suite runs in 'test' by default. To change the environment
57
- simply run
58
-
59
- 'export RACK_ENV="production"'
60
-
61
- before './bin/server' or 'rake spec', etc
62
-
63
- === Using
64
- Run 'rake spec' in the project root to generate the API docs in the doc/ subdir.
65
- Also see the spec suite and polisher scripts for more detailed usage.
66
-
67
- When running, the server can be accessed by navigating to 'http://localhost:3000'.
68
- REST requests may be issued against that URI and the actions defined in polisher.rb.
69
- The DSL sits on top of this REST interface and provides a nice / simple means which
70
- to setup projects, sources, release workflows, etc, in an easily reproducable manner.
71
-
72
- At typical DSL use case might look like:
73
-
74
- project :name => "ruby" do |proj|
75
- proj.add_archive :name => 'ruby_source', :uri => "ftp://ftp.ruby-lang.org/pub/ruby/%{rubyxver}/ruby-%{arcver}.tar.bz2" do |archive|
76
- archive.version "1.8.6", :rubyxver => "1.8", :arcver => "1.8.7-p311", :corresponds_to => proj.version("1.8.6")
77
- proj.version "1.9.1", :corresponds_to => archive.version("1.9.1", :rubyxver => "1.9", :arcver => "1.8.7-p300")
78
- end
79
- proj.add_patch "http://cvs.fedoraproject.org/viewvc/rpms/ruby/F-13/ruby-deadcode.patch?view=markup"
80
- # etc...
81
-
82
- proj.on_version "*", "create package"
83
- proj.on_version "=", "1.8.6", "update repo", "stable"
84
- proj.on_version ">=", "1.9.1", "update repo", "devel"
85
- end
86
-
87
- Polisher is a work in progress, and features are being added / removed as neccessary.
88
- Comments, bug reports, and patches are all very welcome.
89
-
90
- == Extending
91
- Out of the box Polisher is able to handle {Rubygem}[http://rubygems.org/] based projects and
92
- generate rpm packages and yum repos on project releases. The
93
- {Gemcutter API}[http://rubygems.org/pages/api_docs],
94
- {gem2rpm}[http://rubyforge.org/projects/gem2rpm/], and rpm / yum tools are used to
95
- accomplish this, though Polisher can be easily extended to support any input / output format.
96
-
97
- The code itself currently has to be modified (most likely will be changed in the future),
98
- but to add support for downloading another input source format, extend
99
- {source.rb::download_to}[http://github.com/movitto/polisher/blob/master/db/models/source.rb]
100
- and optionally add another add_source_type method to the
101
- {Polisher::Project DSL}[http://github.com/movitto/polisher/blob/master/lib/dsl.rb].
102
-
103
- To add another event handler, simple add a new method to the
104
- {EventHandlers}[http://github.com/movitto/polisher/blob/master/lib/event_handlers.rb] module.
105
- Polisher will take care of the rest.
data/TODO DELETED
@@ -1,7 +0,0 @@
1
- - Remaining TODO's, FIXME's, etc
2
- - Parallelism w/ syncronization locks needed! (on all shared artifacts, db objects)
3
- - More event handlers, generic handler interface,
4
- workflow integration to automatically chain handlers together
5
- - Generic source download interface, easily be able to download sources via and
6
- number of mechanisms, and perform pre-release transformations (taring a source
7
- checkout for example)
data/bin/server DELETED
@@ -1,4 +0,0 @@
1
- #!/bin/sh
2
- mkdir -p db/data/
3
- rake db:migrate
4
- thin start --debug -R config.ru
data/config.ru DELETED
@@ -1,25 +0,0 @@
1
- require 'rubygems'
2
- require 'sinatra'
3
-
4
- dir = File.dirname(__FILE__)
5
-
6
- set :artifacts_dir => dir + '/artifacts'
7
- set :polisher_config => dir + '/config/polisher.yml'
8
- set :db_config => dir + '/config/database.yml'
9
- set :db_data_dir => dir + '/db/data'
10
- set :log_dir => dir + '/log/'
11
- set :environment => :development
12
- set :run => false
13
- set :logging, true
14
-
15
- #log = File.new(dir + "/log/sinatra.log", "a+")
16
- #$stdout.reopen(log)
17
- #$stderr.reopen(log)
18
-
19
- set :raise_errors => true
20
- use Rack::ShowExceptions
21
-
22
- $: << "lib/"
23
-
24
- require 'polisher.rb'
25
- run Sinatra::Application
@@ -1,13 +0,0 @@
1
- # polisher database configuration
2
-
3
- production:
4
- adapter: sqlite3
5
- database: db/data/polisher.sqlite
6
-
7
- development:
8
- adapter: sqlite3
9
- database: db/data/polisher-development.sqlite
10
-
11
- test:
12
- adapter: sqlite3
13
- database: db/data/polisher-test.sqlite
@@ -1,5 +0,0 @@
1
- development:
2
-
3
- test:
4
-
5
- production:
@@ -1,52 +0,0 @@
1
- # ruby gem polisher common routines
2
- #
3
- # Copyright (C) 2010 Red Hat, Inc.
4
- # Written by Mohammed Morsi <mmorsi@redhat.com>
5
- #
6
- # This program is free software, you can redistribute it and/or modify
7
- # it under the terms of the GNU Affero General Public License
8
- # as published by the Free Software Foundation, either version 3
9
- # of the License, or (at your option) any later version.
10
- #
11
- # You should have received a copy of the the GNU Affero
12
- # General Public License, along with Polisher. If not, see
13
- # <http://www.gnu.org/licenses/>
14
-
15
- require 'active_record'
16
-
17
- Dir[File.dirname(__FILE__) + '/models/*.rb'].each { |model| require model }
18
-
19
- module Polisher
20
- module DB
21
-
22
- # Load db config from specified file / environment
23
- def self.load_config(file_path, env = 'development')
24
- YAML::load(File.open(file_path))[env.to_s]
25
- end
26
-
27
- # Connect to database specified in config
28
- def self.connect(config, logger)
29
- ActiveRecord::Base.logger = logger
30
- ActiveRecord::Base.establish_connection config
31
- end
32
-
33
- # Perform any outstanding db migrations
34
- def self.migrate(migrations_dir)
35
- ActiveRecord::Migration.verbose = true
36
- ActiveRecord::Migrator.migrate(migrations_dir)
37
- end
38
-
39
- # Perform a single polisher db rollback
40
- def self.rollback(migrations_dir)
41
- ActiveRecord::Migration.verbose = true
42
- ActiveRecord::Migrator.rollback(migrations_dir)
43
- end
44
-
45
- # Drop all tables in the db
46
- def self.drop_tables(migrations_dir)
47
- ActiveRecord::Migration.verbose = true
48
- ActiveRecord::Migrator.down(migrations_dir, 0)
49
- end
50
-
51
- end
52
- end
@@ -1,23 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- class CreateProjects < ActiveRecord::Migration
14
- def self.up
15
- create_table :projects do |t|
16
- t.string :name
17
- end
18
- end
19
-
20
- def self.down
21
- drop_table :projects
22
- end
23
- end
@@ -1,25 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- class CreateSources < ActiveRecord::Migration
14
- def self.up
15
- create_table :sources do |t|
16
- t.string :name
17
- t.string :source_type
18
- t.string :uri
19
- end
20
- end
21
-
22
- def self.down
23
- drop_table :sources
24
- end
25
- end
@@ -1,28 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- class CreateProjectSourceVersions < ActiveRecord::Migration
14
- def self.up
15
- create_table :project_source_versions do |t|
16
- t.references :project
17
- t.references :source
18
- t.boolean :primary_source, :default => false
19
- t.string :project_version
20
- t.string :source_version
21
- t.string :source_uri_params
22
- end
23
- end
24
-
25
- def self.down
26
- drop_table :project_source_versions
27
- end
28
- end
@@ -1,27 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- class CreateEvents < ActiveRecord::Migration
14
- def self.up
15
- create_table :events do |t|
16
- t.references :project
17
- t.string :process
18
- t.string :version
19
- t.string :version_qualifier
20
- t.string :process_options
21
- end
22
- end
23
-
24
- def self.down
25
- drop_table :events
26
- end
27
- end
@@ -1,28 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- class CreateProjectDependencies < ActiveRecord::Migration
14
- def self.up
15
- create_table :project_dependencies do |t|
16
- t.references :project
17
- t.string :project_version, :default => nil
18
-
19
- t.integer :depends_on_project_id
20
- t.string :depends_on_project_version, :default => nil
21
- t.string :depends_on_project_params
22
- end
23
- end
24
-
25
- def self.down
26
- drop_table :project_dependencies
27
- end
28
- end
@@ -1,87 +0,0 @@
1
- # Copyright (C) 2010 Red Hat, Inc.
2
- # Written by Mohammed Morsi <mmorsi@redhat.com>
3
- #
4
- # This program is free software, you can redistribute it and/or modify
5
- # it under the terms of the GNU Affero General Public License
6
- # as published by the Free Software Foundation, either version 3
7
- # of the License, or (at your option) any later version.
8
- #
9
- # You should have received a copy of the the GNU Affero
10
- # General Public License, along with Polisher. If not, see
11
- # <http://www.gnu.org/licenses/>
12
-
13
- require 'lib/event_handlers'
14
-
15
- class Event < ActiveRecord::Base
16
- include EventHandlers
17
-
18
- belongs_to :project
19
-
20
- validates_presence_of :project_id
21
- validates_presence_of :process
22
-
23
- # Dynamically generate from methods we define in the lib/event_handlers module.
24
- # Add more methods to that module for them to appear here
25
- def self.processes
26
- EventHandlers.public_instance_methods.collect { |m| m.to_s }
27
- end
28
-
29
- # XXX FIXME we need this for security
30
- #validates_inclusion_of :process, :in => Event.processes
31
-
32
- # Version qualifiers
33
- VERSION_QUALIFIERS = ['', '=', '>', '<', '>=', '<=']
34
-
35
- validates_inclusion_of :version_qualifier, :in => VERSION_QUALIFIERS,
36
- :if => Proc.new { |e| !e.version_qualifier.nil? }
37
-
38
- # nil version and version_qualifier indicates event will be run for _all_ versons
39
-
40
- validates_presence_of :version,
41
- :if => Proc.new { |e| !e.version_qualifier.nil? }
42
-
43
- validates_presence_of :version_qualifier,
44
- :if => Proc.new { |e| !e.version.nil? }
45
-
46
- # Determine if event applies to specified version
47
- def applies_to_version?(cversion)
48
- raise ArgumentError, "valid event version #{version} and version #{cversion} required" unless (version.nil? || version.class == String) && cversion.class == String
49
-
50
- # XXX remove any non-numeric chars from the version number (eg if a version has a '-beta' or whatnot, not pretty but works for now
51
- cversion.gsub(/[a-zA-Z]*/, '')
52
-
53
- # TODO this will evaluate to false "1.1" = "1.1.0" here, is this correct? (implement this in a more robust way, eg split version into array around delims, compare each array elements w/ special case handling)
54
- gv, ev = cversion, version
55
- return (["", nil].include? version_qualifier ) ||
56
- (version_qualifier == "=" && gv == ev) ||
57
- (version_qualifier == ">" && gv > ev) ||
58
- (version_qualifier == "<" && gv < ev) ||
59
- (version_qualifier == ">=" && gv >= ev) ||
60
- (version_qualifier == "<=" && gv <= ev)
61
- end
62
-
63
- # Run the event, :version should be passed in via args; any other keys/values are optional
64
- # and will be forwarded onto the event handler
65
- def run(args = {})
66
- version = args[:version]
67
- raise ArgumentError, "must specify version when running event" if version.nil?
68
-
69
- handler = nil
70
- begin
71
- # covert process to method name
72
- handler = method(process.intern)
73
- rescue NameError => e
74
- raise ArgumentError, "could not find event handler #{process}"
75
- end
76
-
77
- # generate array of event params from project, the event itself, the version being run, and any optional params passed in
78
- event_params = [self, version, args]
79
-
80
- begin
81
- # invoke
82
- handler.call *event_params
83
- rescue Exception => e
84
- raise RuntimeError, "error when running event handler #{process}: #{e}"
85
- end
86
- end
87
- end