polisher 0.5.1 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +106 -3
- data/Rakefile +17 -6
- data/lib/polisher/bodhi.rb +3 -0
- data/lib/polisher/errata.rb +1 -0
- data/lib/polisher/fedora.rb +13 -10
- data/lib/polisher/gem.rb +56 -14
- data/lib/polisher/git.rb +15 -13
- data/lib/polisher/koji.rb +16 -0
- data/lib/polisher/rpmspec.rb +25 -5
- data/lib/polisher/version.rb +3 -0
- data/lib/polisher/yum.rb +4 -2
- data/spec/bodhi_spec.rb +32 -0
- data/spec/data/Gemfile +5 -0
- data/spec/data/mysql-2.9.1.gemspec +40 -0
- data/spec/data/rails.json +1 -0
- data/spec/data/rspec-2.12.0.gem +0 -0
- data/spec/data/rspec.json +1 -0
- data/spec/data/rubygem-activesupport.spec +238 -0
- data/spec/errata_spec.rb +43 -0
- data/spec/fedora_spec.rb +15 -0
- data/spec/git_spec.rb +117 -13
- data/spec/koji_spec.rb +71 -0
- data/spec/rpmspec_spec.rb +9 -3
- data/spec/spec_helper.rb +13 -3
- data/spec/yum_spec.rb +37 -0
- metadata +50 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 376315592c88c3de00fca0125ae64583fb660764
|
4
|
+
data.tar.gz: 4d11de1d33a201b7195c090c4831d8e8ee61783c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6217a46af865d179c17d64e8c7cc030f624470b35b480e04f5a4cf2907161acd1afff8b57f283a8c394e0fd401baba77bef07e9318c57b9ac39e7360a461bc66
|
7
|
+
data.tar.gz: fa067ed1e2161d7b185108476c588c24f58f2bc26ce894f499b6b212563366fe68df5e431528d1ef6ce252263e4b930e2a4939585ba90eb908b5b1ca4b3794fa
|
data/README.md
CHANGED
@@ -29,11 +29,114 @@ Provided are a series of tools geared towards querying rubygems.org and other up
|
|
29
29
|
ruby sources for metadata as well as downstream sources such as the Fedora and Debian
|
30
30
|
distributions to cross reference various supported stacks vendored by each.
|
31
31
|
|
32
|
-
|
32
|
+
Polisher also includes utilities to integrate and hook into various build and install workflows
|
33
33
|
used by both upstream and downstream developers to synergize operations and streamline
|
34
34
|
the packaging and support process.
|
35
35
|
|
36
|
-
|
37
|
-
|
36
|
+
## Dependencies
|
37
|
+
|
38
|
+
Polisher is made available through [rubygems](http://rubygems.org/gems/polisher).
|
39
|
+
|
40
|
+
Polisher dependends on the 'curb' rubygem which in return depends on the curl-devel
|
41
|
+
headers. The user should install those and the development tools for their platform
|
42
|
+
or install the curb gem packaged by their distribution like so:
|
43
|
+
|
44
|
+
# install the ruby & curl headers & development tools...
|
45
|
+
yum install ruby-devel libcurl-devel
|
46
|
+
yum group install "Development Tools"
|
47
|
+
|
48
|
+
# ... or install the gem manually
|
49
|
+
yum install rubygem-curb
|
50
|
+
|
51
|
+
Replace the 'yum install' commands with the corresponding ones on alternate platforms.
|
52
|
+
|
53
|
+
Various polisher subcomponents depend on various command line utilities, these include:
|
54
|
+
|
55
|
+
* /usr/bin/git - to checkout git repos
|
56
|
+
* /usr/bin/koji - to query and build against koji
|
57
|
+
* /usr/bin/yum - to query yum
|
58
|
+
* /usr/bin/sed - to manipulated metadata
|
59
|
+
* /usr/bin/md5sum - to generate required metadata
|
60
|
+
* /usr/bin/fedpkg - to query fedora
|
61
|
+
|
62
|
+
Some of these are pre-installed on many platforms and some are available via a quick
|
63
|
+
'yum install' / 'apt-get' or other. Not all are required for all utilities, see the
|
64
|
+
specific executables and modules for details.
|
65
|
+
|
66
|
+
## Installation
|
67
|
+
|
68
|
+
To install the latest release and all ruby dependencies simply run:
|
69
|
+
|
70
|
+
gem install polisher
|
71
|
+
|
72
|
+
See the bin/ directory for all executables available, pass '-h' to any to
|
73
|
+
see specific command line options. To run any command from a local git checkout
|
74
|
+
of polisher, run the following beforehand:
|
75
|
+
|
76
|
+
export RUBYLIB='lib'
|
77
|
+
|
78
|
+
A few select utilities are highlighted below.
|
79
|
+
|
80
|
+
### gem_dependency_checker
|
81
|
+
|
82
|
+
A utility to cross reference rubygems and bundler gemfiles against various downstream
|
83
|
+
resources including the koji build system, distgit (as used by Fedora), and more.
|
84
|
+
|
85
|
+
For example to check a specified ruby app for compatability in koji and yum:
|
86
|
+
|
87
|
+
gem_dependency_checker.rb --gemfile ~/myapp/Gemfile -k -y
|
88
|
+
|
89
|
+
### git_gem_updater
|
90
|
+
|
91
|
+
A script that clones a rubygem maintained by distgit and update it to the latest
|
92
|
+
version of the package available on rubygems (or the specified version if given).
|
93
|
+
A scratch built will be run and if all goes will a commit staged so that the end
|
94
|
+
user just has to 'git push' the updated package to the distro.
|
95
|
+
|
96
|
+
Simply specify the name of the gem to update like so:
|
97
|
+
|
98
|
+
git_gem_updater.rb -n rails
|
99
|
+
|
100
|
+
Alternatively if "-u" is specified with a Fedora username, all the packages the
|
101
|
+
user owns will be checked out and updated.
|
102
|
+
|
103
|
+
### ruby_rpm_spec_updater
|
104
|
+
|
105
|
+
A tool to update a given ruby gem or application based rpm spec to the specified
|
106
|
+
new source (as specified by a gem, gemspec, or gemfile).
|
107
|
+
|
108
|
+
Simply pass it the location of the spec to update, and optionally a upstream
|
109
|
+
source which to update from and the utility will print out the updated spec
|
110
|
+
to STDOUT.
|
111
|
+
|
112
|
+
ruby_rpm_spec_updater.rb ~/rpmbuild/SPECS/rubygem-rails.spec rails-5.0.0.gem
|
113
|
+
|
114
|
+
### Other
|
115
|
+
|
116
|
+
There are various other tools and utilities provided and in conceptual phases.
|
117
|
+
See the bin dir and TODO document for more detailed listing.
|
118
|
+
|
119
|
+
### polisher
|
120
|
+
|
121
|
+
The core library behind these utilities, polisher provides reusable modules to
|
122
|
+
query and interface with upstream ruby sources and downstream vendors and systems.
|
123
|
+
|
124
|
+
## Documentation and Spec Suite
|
125
|
+
|
126
|
+
Polisher comes with complete documentation and a full test suite.
|
127
|
+
|
128
|
+
Documentation relies on the 'yard' gem / documentation system, to
|
129
|
+
generate run:
|
130
|
+
|
131
|
+
rake yard
|
132
|
+
|
133
|
+
The test suite is based on rspec, to run:
|
134
|
+
|
135
|
+
rake spec
|
136
|
+
|
137
|
+
## Legal & Other
|
38
138
|
|
39
139
|
Polisher is Licensed under the MIT License, Copyright (C) 2013 Red Hat, Inc.
|
140
|
+
|
141
|
+
See the commit log for authors of the project. All feedback and contributions
|
142
|
+
are more than welcome.
|
data/Rakefile
CHANGED
@@ -3,15 +3,26 @@
|
|
3
3
|
# Licensed under the MIT license
|
4
4
|
# Copyright (C) 2013 Red Hat, Inc.
|
5
5
|
|
6
|
-
|
7
|
-
|
8
|
-
desc "Run all specs"
|
9
|
-
RSpec::Core::RakeTask.new(:spec) do |spec|
|
10
|
-
|
11
|
-
|
6
|
+
begin
|
7
|
+
require "rspec/core/rake_task"
|
8
|
+
desc "Run all specs"
|
9
|
+
RSpec::Core::RakeTask.new(:spec) do |spec|
|
10
|
+
spec.pattern = 'specs/**/*_spec.rb'
|
11
|
+
spec.rspec_opts = ['--backtrace', '-fd', '-c']
|
12
|
+
end
|
13
|
+
rescue LoadError
|
12
14
|
end
|
13
15
|
|
14
16
|
desc "build the polisher gem"
|
15
17
|
task :build do
|
16
18
|
system "gem build polisher.gemspec"
|
17
19
|
end
|
20
|
+
|
21
|
+
begin
|
22
|
+
require "yard"
|
23
|
+
YARD::Rake::YardocTask.new do |t|
|
24
|
+
#t.files = ['lib/**/*.rb', OTHER_PATHS] # optional
|
25
|
+
#t.options = ['--any', '--extra', '--opts'] # optional
|
26
|
+
end
|
27
|
+
rescue LoadError
|
28
|
+
end
|
data/lib/polisher/bodhi.rb
CHANGED
@@ -8,6 +8,9 @@ require 'pkgwat'
|
|
8
8
|
module Polisher
|
9
9
|
class Bodhi
|
10
10
|
def self.versions_for(name, &bl)
|
11
|
+
# XXX issue w/ retreiving packages from pkgwat causing issues:
|
12
|
+
# https://github.com/fedora-infra/fedora-packages/issues/55
|
13
|
+
|
11
14
|
# fedora pkgwat provides a frontend to bodhi
|
12
15
|
updates = Pkgwat.get_updates("rubygem-#{name}", 'all', 'all') # TODO set timeout
|
13
16
|
updates.reject! { |u|
|
data/lib/polisher/errata.rb
CHANGED
data/lib/polisher/fedora.rb
CHANGED
@@ -6,18 +6,24 @@
|
|
6
6
|
require 'curb'
|
7
7
|
require 'pkgwat'
|
8
8
|
|
9
|
+
require 'polisher/bodhi'
|
10
|
+
|
9
11
|
module Polisher
|
10
12
|
class Fedora
|
11
13
|
PACKAGE_LIST = 'https://admin.fedoraproject.org/pkgdb/users/packages/'
|
12
14
|
|
15
|
+
def self.client
|
16
|
+
@client ||= Curl::Easy.new
|
17
|
+
end
|
18
|
+
|
13
19
|
# Retrieve list of gems owned by the specified user
|
14
20
|
#
|
15
21
|
# @param [String] user Fedora username to lookup
|
16
22
|
# @return [Array<String>] list of gems which the user owns/has access to
|
17
23
|
def self.gems_owned_by(user)
|
18
|
-
|
19
|
-
|
20
|
-
packages =
|
24
|
+
client.url = "#{PACKAGE_LIST}#{user}"
|
25
|
+
client.http_get
|
26
|
+
packages = client.body_str
|
21
27
|
# TODO instantiate Polisher::Gem instances & return
|
22
28
|
Nokogiri::HTML(packages).xpath("//a[@class='PackageName']").
|
23
29
|
select { |i| i.text =~ /rubygem-.*/ }.
|
@@ -31,13 +37,10 @@ module Polisher
|
|
31
37
|
# @param [Callable] bl optional callback to invoke with versions retrieved
|
32
38
|
# @return [Array<String>] list of versions in Fedora
|
33
39
|
def self.versions_for(name, &bl)
|
34
|
-
#
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
versions = versions.collect { |pkg| pkg['stable_version'] }
|
39
|
-
bl.call(:fedora, name, versions) unless(bl.nil?)
|
40
|
-
versions
|
40
|
+
# simply dispatch to bodhi to get latest updates
|
41
|
+
Polisher::Bodhi.versions_for name do |target,name,versions|
|
42
|
+
bl.call(:fedora, name, versions) unless(bl.nil?)
|
43
|
+
end
|
41
44
|
end
|
42
45
|
end # class Fedora
|
43
46
|
end # module Polisher
|
data/lib/polisher/gem.rb
CHANGED
@@ -5,6 +5,7 @@
|
|
5
5
|
|
6
6
|
require 'curb'
|
7
7
|
require 'json'
|
8
|
+
require 'yaml'
|
8
9
|
require 'tempfile'
|
9
10
|
require 'pathname'
|
10
11
|
require 'rubygems/installer'
|
@@ -14,6 +15,9 @@ require 'polisher/version_checker'
|
|
14
15
|
|
15
16
|
module Polisher
|
16
17
|
class Gem
|
18
|
+
GEM_CMD = '/usr/bin/gem'
|
19
|
+
|
20
|
+
attr_accessor :spec
|
17
21
|
attr_accessor :name
|
18
22
|
attr_accessor :version
|
19
23
|
attr_accessor :deps
|
@@ -21,6 +25,7 @@ module Polisher
|
|
21
25
|
attr_accessor :files
|
22
26
|
|
23
27
|
def initialize(args={})
|
28
|
+
@spec = args[:spec]
|
24
29
|
@name = args[:name]
|
25
30
|
@version = args[:version]
|
26
31
|
@deps = args[:deps] || []
|
@@ -51,19 +56,35 @@ module Polisher
|
|
51
56
|
|
52
57
|
if args.is_a?(String)
|
53
58
|
specj = JSON.parse(args)
|
54
|
-
metadata[:
|
55
|
-
metadata[:
|
56
|
-
metadata[:
|
57
|
-
|
59
|
+
metadata[:spec] = specj
|
60
|
+
metadata[:name] = specj['name']
|
61
|
+
metadata[:version] = specj['version']
|
62
|
+
|
63
|
+
metadata[:deps] =
|
64
|
+
specj['dependencies']['runtime'].collect { |d|
|
65
|
+
::Gem::Dependency.new d['name'], *d['requirements'].split(',')
|
66
|
+
}
|
67
|
+
|
68
|
+
metadata[:dev_deps] =
|
69
|
+
specj['dependencies']['development'].collect { |d|
|
70
|
+
::Gem::Dependency.new d['name'], d['requirements']
|
71
|
+
}
|
58
72
|
|
59
73
|
elsif args.has_key?(:gemspec)
|
60
74
|
gemspec = ::Gem::Specification.load(args[:gemspec])
|
61
|
-
metadata[:
|
62
|
-
metadata[:
|
63
|
-
metadata[:
|
64
|
-
|
75
|
+
metadata[:spec] = gemspec # TODO to json
|
76
|
+
metadata[:name] = gemspec.name
|
77
|
+
metadata[:version] = gemspec.version.to_s
|
78
|
+
|
79
|
+
metadata[:deps] =
|
80
|
+
gemspec.dependencies.select { |dep|
|
81
|
+
dep.type == :runtime
|
82
|
+
}.collect { |dep| dep }
|
83
|
+
|
65
84
|
metadata[:dev_deps] =
|
66
|
-
gemspec.dependencies.select { |dep|
|
85
|
+
gemspec.dependencies.select { |dep|
|
86
|
+
dep.type == :development
|
87
|
+
}.collect { |dep| dep }
|
67
88
|
|
68
89
|
elsif args.has_key?(:gem)
|
69
90
|
# TODO
|
@@ -82,6 +103,27 @@ module Polisher
|
|
82
103
|
gemf = curl.body_str
|
83
104
|
end
|
84
105
|
|
106
|
+
# Refresh the gem spec
|
107
|
+
def refresh_spec
|
108
|
+
gemf = download_gem
|
109
|
+
tgem = Tempfile.new(@name)
|
110
|
+
tgem.write gemf
|
111
|
+
tgem.close
|
112
|
+
|
113
|
+
@spec = YAML.load `#{GEM_CMD} specification #{tgem.path}`
|
114
|
+
|
115
|
+
# update deps, dev_deps
|
116
|
+
@deps =
|
117
|
+
@spec.dependencies.select { |dep|
|
118
|
+
dep.type == :runtime
|
119
|
+
}.collect { |dep| dep }
|
120
|
+
|
121
|
+
@dev_deps =
|
122
|
+
@spec.dependencies.select { |dep|
|
123
|
+
dep.type == :development
|
124
|
+
}.collect { |dep| dep }
|
125
|
+
end
|
126
|
+
|
85
127
|
# Retrieve the list of files in the gem
|
86
128
|
#
|
87
129
|
# @return [Array<String>] list of files in the gem
|
@@ -115,7 +157,7 @@ module Polisher
|
|
115
157
|
gem
|
116
158
|
end
|
117
159
|
|
118
|
-
# Retreive versions of gem available
|
160
|
+
# Retreive versions of gem available in all configured targets (optionally recursively)
|
119
161
|
#
|
120
162
|
# @param [Hash] args hash of options to configure retrieval
|
121
163
|
# @option args [Boolean] :recursive indicates if versions of dependencies
|
@@ -134,16 +176,16 @@ module Polisher
|
|
134
176
|
|
135
177
|
if recursive
|
136
178
|
self.deps.each { |dep|
|
137
|
-
unless versions.has_key?(dep)
|
138
|
-
gem = Polisher::Gem.retrieve(dep)
|
179
|
+
unless versions.has_key?(dep.name)
|
180
|
+
gem = Polisher::Gem.retrieve(dep.name)
|
139
181
|
versions.merge! gem.versions(args, &bl)
|
140
182
|
end
|
141
183
|
}
|
142
184
|
|
143
185
|
if dev_deps
|
144
186
|
self.dev_deps.each { |dep|
|
145
|
-
unless versions.has_key?(dep)
|
146
|
-
gem = Polisher::Gem.retrieve(dep)
|
187
|
+
unless versions.has_key?(dep.name)
|
188
|
+
gem = Polisher::Gem.retrieve(dep.name)
|
147
189
|
versions.merge! gem.versions(args, &bl)
|
148
190
|
end
|
149
191
|
}
|
data/lib/polisher/git.rb
CHANGED
@@ -5,6 +5,8 @@
|
|
5
5
|
|
6
6
|
# TODO use ruby git api and others
|
7
7
|
|
8
|
+
require 'tmpdir'
|
9
|
+
require 'awesome_spawn'
|
8
10
|
require 'polisher/rpmspec'
|
9
11
|
|
10
12
|
module Polisher
|
@@ -50,7 +52,7 @@ module Polisher
|
|
50
52
|
rpm_name = "#{RPM_PREFIX}#{name}"
|
51
53
|
|
52
54
|
unless File.directory? rpm_name
|
53
|
-
|
55
|
+
AwesomeSpawn.run "#{PKG_CMD} clone #{rpm_name}"
|
54
56
|
end
|
55
57
|
|
56
58
|
# cd into working directory
|
@@ -62,9 +64,9 @@ module Polisher
|
|
62
64
|
|
63
65
|
# checkout the latest rawhide
|
64
66
|
# TODO allow other branches to be specified
|
65
|
-
|
66
|
-
|
67
|
-
|
67
|
+
AwesomeSpawn.run "#{GIT_CMD} checkout master"
|
68
|
+
AwesomeSpawn.run "#{GIT_CMD} reset HEAD~ --hard"
|
69
|
+
AwesomeSpawn.run "#{GIT_CMD} pull"
|
68
70
|
|
69
71
|
self.new :name => name
|
70
72
|
end
|
@@ -74,19 +76,19 @@ module Polisher
|
|
74
76
|
# @param [Polisher::Gem] gem instance of gem containing metadata to update to
|
75
77
|
def update_to(gem)
|
76
78
|
# TODO use Polisher::RPMSpec to update spec
|
77
|
-
|
78
|
-
|
79
|
-
|
79
|
+
AwesomeSpawn.run "#{SED_CMD} -i 's/Version.*/Version: #{gem.version}/' #{spec}"
|
80
|
+
AwesomeSpawn.run "#{SED_CMD} -i 's/Release:.*/Release: 1%{?dist}/' #{spec}"
|
81
|
+
AwesomeSpawn.run "#{MD5SUM_CMD} #{gem.name}-#{gem.version}.gem > sources"
|
80
82
|
File.open(".gitignore", "w") { |f| f.write "#{gem.name}-#{gem.version}.gem" }
|
81
83
|
end
|
82
84
|
|
83
85
|
# Build the locally cloned package using the configured build command
|
84
86
|
def build
|
85
87
|
# build srpm
|
86
|
-
|
88
|
+
AwesomeSpawn.run "#{PKG_CMD} srpm"
|
87
89
|
|
88
90
|
# attempt to build packages
|
89
|
-
|
91
|
+
AwesomeSpawn.run "#{BUILD_CMD} build --scratch #{BUILD_TGT} #{srpm}"
|
90
92
|
# TODO if build fails, spit out error, exit
|
91
93
|
end
|
92
94
|
|
@@ -94,7 +96,7 @@ module Polisher
|
|
94
96
|
#
|
95
97
|
# @return [Boolean] true/false depending on whether or not spec has %check
|
96
98
|
def has_check?
|
97
|
-
open(spec, "r") do |spec|
|
99
|
+
File.open(spec, "r") do |spec|
|
98
100
|
spec.lines.any? { |line| line.include?("%check") }
|
99
101
|
end
|
100
102
|
end
|
@@ -102,9 +104,9 @@ module Polisher
|
|
102
104
|
# Command the local package to the local git repo
|
103
105
|
def commit
|
104
106
|
# git add spec, git commit w/ message
|
105
|
-
|
107
|
+
AwesomeSpawn.run "#{GIT_CMD} add #{spec} sources .gitignore"
|
106
108
|
#`git add #{gem_name}-#{version}.gem`
|
107
|
-
|
109
|
+
AwesomeSpawn.run "#{GIT_CMD} commit -m 'updated to #{self.version}'"
|
108
110
|
end
|
109
111
|
|
110
112
|
# Retrieve list of the version of the specified package in git
|
@@ -119,7 +121,7 @@ module Polisher
|
|
119
121
|
Dir.mktmpdir do |dir|
|
120
122
|
Dir.chdir(dir) do |path|
|
121
123
|
version = nil
|
122
|
-
|
124
|
+
AwesomeSpawn.run "#{GIT_CMD} clone #{DIST_GIT_URL}#{rpm_name}.git ."
|
123
125
|
begin
|
124
126
|
spec = Polisher::RPMSpec.parse(File.read("#{rpm_name}.spec"))
|
125
127
|
version = spec.version
|
data/lib/polisher/koji.rb
CHANGED
@@ -35,6 +35,22 @@ module Polisher
|
|
35
35
|
end
|
36
36
|
end
|
37
37
|
|
38
|
+
# Return bool indiciating if koji has a build exactly
|
39
|
+
# matching the specified version
|
40
|
+
def self.has_build?(name, version)
|
41
|
+
versions = self.versions_for name
|
42
|
+
versions.include?(version)
|
43
|
+
end
|
44
|
+
|
45
|
+
# Return bool indicating if koji has a build which
|
46
|
+
# satisfies the specified ruby dependency
|
47
|
+
def self.has_build_satisfying?(name, version)
|
48
|
+
dep = ::Gem::Dependency.new name, version
|
49
|
+
self.versions_for(name).any? { |v|
|
50
|
+
dep.match?(name, v)
|
51
|
+
}
|
52
|
+
end
|
53
|
+
|
38
54
|
# Retrieve list of the version of the specified package in koji
|
39
55
|
#
|
40
56
|
# @param [String] name name of package to lookup
|